We have a problem with Scan operations with Aerospike Server EE 3.5.9 and Java Client 3.1.1.
-
When running more than one scan with the same priority (low or high), only one runs. All subsequent ones apparently are queued and do not start until the first one finishes.
-
When starting a high priority scan while a low priority scan is running, the high priority scan will run, but will always halt at 79% progress, until the low priority scan has finished.
It does not matter what kind of Namespaces (in memory, ssd; same, different ones) or Sets the scans are running on. It also does not matter if the scans are issued from the same client instance of different ones. Tests performed on single or multiple nodes active. Nothing related is being logged in the server log.
The scan policies:
ScanPolicy sP = new ScanPolicy();
sP.includeBinData = true;
sP.scanPercent = percent;
if (isHighPriority) {
sP.concurrentNodes = true;
sP.priority = Priority.HIGH;
} else {
sP.consistencyLevel = ConsistencyLevel.CONSISTENCY_ONE;
sP.concurrentNodes = false;
sP.maxConcurrentNodes = 1;
sP.priority = Priority.LOW;
sP.failOnClusterChange = true;
}
asyncClient.scanAll(sP, new RecordSequenceListener<>(), namespace, set, "");
The AsyncClient policy:
cPolicy = new AsyncClientPolicy();
cPolicy.timeout = 500;
cPolicy.asyncMaxCommandAction = MaxCommandAction.BLOCK;
cPolicy.writePolicyDefault = wPolicy;
cPolicy.asyncTaskThreadPool = ThreadPools.aeroTaskExecutor;
cPolicy.asyncSelectorThreads = 2;
Also, while a Backup is running: Low priority scans do no progress, high priority halt at 79%. If another scan is running when a backup is initiated, the backup scan does not progress (regardless the priority of the other scan).
What can be the cause?