Batch Read Latency Monitoring

Operations Monitoring
1

Hello,

We are using the newer batch read api with aerospike 3.14.1.1 (previously using 3.7.4) and I am confused on this command:

asinfo -v "latency:"

On the 3.7.4 version we would see results for the reads when we used the batch api, now we don’t see it, instead we see:

error-no-data-yet-or-back-too-small

Also, the newer documentation shows 5 histograms available:

  1. {ns}-read
  2. {ns}-write
  3. {ns}-udf
  4. {ns}-query
  5. batch-index

I only see the first 4 every returned, my guess is the metrics I am looking for now come in under batch-index but that is not returned by default that I can tell. Also when I do:

asinfo -v 'latency:hist=batch-index;slice=30;back=300;duration:120'

I get bad hist name, its possible that batch-index is not what I am looking for but in general, I want the latency statistics from this as this is what the collectd plugin does to retrieve it. When I use asadm -l and then histogram -h batch-index -d 200 -f -4400 then I do see metrics for the batch-index. Any thoughts here and any explanation about the change of stats for read on batches with this newer version of aerospike?

Thanks in advance

2

I want to clarify my questions in hopes of getting a response:

  1. In 3.7.4 it appeared that batch read statistics were coming in under individual reads, this does not appear to be the case in 3.14.1.1, can someone confirm that?

  2. Assuming 1 is true, I do see in the log stats for batch-index histogram, which appears to have the information I want, did the stats move to this metric? If so does this mean that it is measuring the time for total batch completion instead of individual per read completion?

  3. In 3.14.1.1 with tools 3.13.0.1 the docs show that with asinfo -v "latency:" I should be able to see the batch-index stats but when I am running it I don’t. I can see them if I do asloglatency -h batch-index -d 30 -f -30, is this a bug?

Thanks in advance

4

Correct, there was a stats overhaul in Aerospike 3.9.0 which included separating batch reads from regular reads. Batch reads were incorporated into the reads histogram in Aerospike 3.6.0 (IIRC) and this was problematic for a few customers.

The batch-index histograms tracks the full batch request. For the individual reads see the histograms under Batch transaction analysis.

There are a lot of histograms that are only available from the logs. I agree that it would be very convenient to be able to access these via the info API. I would consider this a missing histogram feature.

5

Hi Kevin,

Really appreciate the response and everything you say makes sense. Just want to add to that third question:

If you go to the docs Info Command Reference | Aerospike Documentation, the latency command actually shows batch-index as an option to see the histogram for, however I could not get the command to return that histogram. So either the docs are wrong and need to be updated or this feature is broken somehow, just letting you know for the next person in case they are wondering the same thing.

Thanks again!

6

The documentation is incorrect.

7

Thank you for clarification and quick responses, greatly appreciated.

8

This topic was automatically closed 6 days after the last reply. New replies are no longer allowed.