How to monitor batch latency statistics

FAQ – How to monitor batch latency statistics

Context

How to monitor batch-index latency statistics.

Currently, the asinfo -v "latency:" command (tracked histograms) supports the following histogram names (ns being the namespace name):

{ns}-read
{ns}-write
{ns}-udf
{ns}-query

The batch-index histogram is currently not a tracked histogram, meaning it cannot be directly fetched through an info command.

Methods

Internal jira AER-6116 tracks the feature request to add the batch-index histograms to the list of tracked histograms. Until then, the batch-index histogram details can only be checked through the Aerospike logs and is one of the auto-enabled histograms as documented in the Monitoring Latencies documentation page.

A simple shell script be used to process all the log files, using the asloglatency tool. For example:

for f in `find . -name "aerospike.log"`
do
        asloglatency -l $f -h batch-index
done

For detailed batch transaction analysis, enable the batch sub transactions benchmarks by running the following command:

asinfo -v 'set-config:context=namespace;id=<namespaceName>;enable-benchmarks-batch-sub=true'

The following commands can then be used to analyze the batch-sub details:

asloglatency -l $f -h "{ns}-batch-sub-start"
asloglatency -l $f -h "{ns}-batch-sub-restart"
asloglatency -l $f -h "{ns}-batch-sub-dup-res"
asloglatency -l $f -h "{ns}-batch-sub-read-local"
asloglatency -l $f -h "{ns}-batch-sub-response"
asloglatency -l $f -h "{ns}-batch-sub-repl-ping"

Notes

  • The histogram command of the asadm tool can also be used to go through the latency log details.

  • Prior to 3.9, read latency included batch sub read transactions. Those were separated on release 3.9 in order to facilitate specific latency related analysis.

Keywords

BATCH INDEX LATENCY MONITORING

Timestamp

November 2019

© 2015 Copyright Aerospike, Inc. | All rights reserved. Creators of the Aerospike Database.