Partition-level scan: is it possible to split up a scan into finer-grained things than a node? (AER-5474)

Feature Requests Delivered Requests
1

When scanning using mapreduce, it is common to divide the data source logically into a large number of smaller sections, and then read each section individually. However when using Aerospike as the data source, I cannot see a way to split up a scan into anything smaller than a node.

I think that being able to scan a single partition would make logical sense, but I don’t see any way to do that. It it possible to do this in a way that has decent performance?

Thanks,
Marc

2

Currently partition level scan is not possible. We understand the desire, and had some internal discussions on the merit and danger of providing granularity at the partition level. For now, we are not moving forward with it, but will revisit in the future.

3

Partition based scans were released in 4.9.0.3.

  • [AER-5474] - (SCAN) Basic scans now based on lists of partitions and/or digests sent by client.