Hi Anish,
The current implementation of batch reads implement scatter gather approach. The batch request is sent (scattered) to all nodes in the cluster, the individual nodes return the records that they have and client stitches (gather) responses from individual nodes and pass on the combined results to the application.
These aren’t atomic operations. The individual node gets the batch request, reads the record digests for requested records from the primary index tree and then start reading the individual records from the storage (memory or disk). Any changes to records (delete, expiry, eviction) or any changes to cluster state (migration) would affect the records returned by the batch call.
Consider 4 node cluster, 100 records in batch request and ~25 records being returned by each node. 5th node is added, and now with migrations kicking in you might only have ~20 records on each node and the batch request that was initiated with 4 node cluster, might get you ~80 records and wouldn’t find ~20 records which moved to other nodes.
Please see the article Batch Operations for more information.
thanks, samir