Reads fail with 'too many requests' error

Hi,

Once in a while a read from AS multi node cluster fails with “Too many requests” error. Unfortunately there is not much explanation what that really means and if it possible to do anything about it. I’d appreciate any help with that issue.

Tank you.

  • What kind of read is it? Is it a batch read?

  • Where is this error experienced, on the client or the server?

  • Do the server logs have any non-info messages that correlate ‘grep -v INFO /var/log/aerospike/aerospike.log’?

  • How are your histograms looking at this time for read/write latency?

  • What server/client version are you using?

What kind of read is it? Is it a batch read?

Just a read. Not a batch.

Where is this error experienced, on the client or the server?

Client

Do the server logs have any non-info messages that correlate ‘grep -v INFO /var/log/aerospike/aerospike.log’?

Nothing related to the error itself except HWM breaches and connectivity issues to other nodes

How are your histograms looking at this time for read/write latency?

Relatively good, actually.

Reads: up to 3 ms in avg and 8 ms at max Writes: are UDF writes with massive updates up to 100 ms.

What server/client version are you using?

Server 3.12.1, GO Client 1.27.0

The problem found. It appeared that the underlying library interprets AS request timeout that way…

What is the underlying library you are using?