AEROSPIKE_ERR_RECORD_NOT_FOUND during rebalance


#1

Configuration: 5 servers, CentOS 7, aerospike server 3.8.3 1 namespace, 2 ssd on each server for data Clients use citrusleaf_client_libevent2 library (2.1.43), batch reads

How to reproduce: Shutdown one server, replace/zeroize one disk and then turn the server on.

Problem: After server is back online number of NOT_FOUND answers increases drastically. Then during rebalance number of NOT_FOUND answers decreases proportionally. See attachment.

https://dl.dropboxusercontent.com/u/17481057/aero_rebalance.jpg

Server config template https://dl.dropboxusercontent.com/u/17481057/aerospike.conf

If we use synchronous c client with read policy AS_POLICY_REPLICA_ANY we get a slowdown but no NOT_FOUND.