How to monitor batch latency statistics

The Aerospike Knowledge Base has moved to https://support.aerospike.com. Content on https://discuss.aerospike.com is being migrated to either https://support.aerospike.com or https://docs.aerospike.com. Maintenance on articles stored in this repository ceased on December 31st 2022 and this article may be stale. If you have any questions, please do not hesitate to raise a case via https://support.aerospike.com.

FAQ – How to monitor batch latency statistics

Context

How to monitor batch-index latency statistics.

Currently, the asinfo -v "latencies:" command (tracked histograms) supports the following histogram names ({ns} being the namespace name enclosed in literal curly braces, which will require the histogram name to be enclosed in single or double quotes when used on the command line, as in the asloglatency examples below):

{ns}-read
{ns}-write
{ns}-udf
{ns}-query

The batch-index histogram is currently not a tracked histogram, meaning it cannot be directly fetched through an info command.

Methods

As of version 4.9.0.3, the batch-index histogram has been added to the list of tracked histograms. For earlier versions, the batch-index histogram details can only be checked through the Aerospike logs and it is one of the auto-enabled histograms as documented in the Monitoring Latencies documentation page.

A simple shell script be used to process all the log files, using the asloglatency tool. For example:

for f in `find . -name "aerospike.log"`
do
        asloglatency -l $f -h batch-index
done

For detailed batch transaction analysis, enable the batch sub transactions benchmarks by running the following command:

asinfo -v 'set-config:context=namespace;id=<namespaceName>;enable-benchmarks-batch-sub=true'

The following commands can then be used to analyze the batch-sub details:

asloglatency -l $f -h "{ns}-batch-sub-start"
asloglatency -l $f -h "{ns}-batch-sub-restart"
asloglatency -l $f -h "{ns}-batch-sub-dup-res"
asloglatency -l $f -h "{ns}-batch-sub-read-local"
asloglatency -l $f -h "{ns}-batch-sub-response"
asloglatency -l $f -h "{ns}-batch-sub-repl-ping"

Notes

  • The histogram command of the asadm tool can also be used to go through the latency log details.

  • Prior to 3.9, read latency included batch sub read transactions. Those were separated on release 3.9 in order to facilitate specific latency related analysis.

Keywords

BATCH INDEX LATENCY MONITORING

Timestamp

May 2021