My AS version is 3.16.0.6, and I am using it for in-memory cache, where application does batch gets and puts on cache misses.
When the stop-write state is reached, which I am setting to around 50% using stop-writes-pct config, the writes on the server stop, but batch get latencies start shooting up at the same time.
What’s the exact behaviour of stop-write-pct? Should it affect the batch get latencies?
Maybe check if there is a connection churn that would impact the performance if clients end up more aggressively writing as those would be failing quickly? The client_connections statistic would give you the number of open connections but there is also stats and logs (search for proto in the logs) that gives you cumulative so you can see if there is a lot of connections created and/or closed.
The other thing I can think of is that somehow the reads are tapping less into the post write queue? Does the cache_read_pct change?