FAQ - Scans in versions 4.7 and above

FAQ - Scans since 4.7

This FAQ covers the Aerospike Scan sub-system changes introduced in version 4.7. For versions prior to 4.7, refer to this older FAQ - Scans in Aerospike

In addition to the below mentioned changes, server version 4.9 and above Aerospike introduced partition-based basic scans, enabling accurate scans during migrations and added a max records option for sampling with basic scans.

Configuration Changes and Compatability

What are the changes?

  • Scans do not have a priority anymore.
  • Concurrent scans each use their own dedicated threads, so they will not directly interfere with each other by having one high priority scan starve all others of scan threads.
  • Scans can be independently throttled, via a specified requested records per second (rps) value in the client policy.
  • Scans begin with a single thread and add threads as needed (up to the configured limits) to achieve the specified rps. If a single thread would exceed the specified rps, throttling will occur within the thread to achieve the specified rps.
  • A scan’s requested rps cannot be changed dynamically.

What are the server configuration parameters for the new scan?

  • scan-threads-limit
    • specifies the maximum number of threads allowed for all concurrent scans. If this limit is reached any new scan requests will fail with an AEROSPIKE_ERR_FAIL_FORBIDDEN error. See knowledge-base article on Error code 22 for more details.
  • background-scan-max-rps
    • specifies the maximum rps value allowed for a background scan.
  • single-scan-threads
    • specifies the maximum number of threads allowed for a single scan.
  • scan-max-done
    • specifies the maximum number of scan jobs history to keep.

Will my old applications be compatible with the new scan?

Yes, the new Aerospike server will accept old applications that have priority being set. As a matter of fact, when priority is low, the rps will be set to 5000. For example, in Java code prior to 4.4.8:

import com.aerospike.client.policy.Priority;
...
...
                policy.priority = Priority.LOW;

The Aerospike serverlog for the new scan will have this:

Feb 04 2020 17:57:50 GMT: INFO (scan): (scan.c:672) starting basic scan job 6028148656843906502 {nsSSD:set4} rps 5000 sample-pct 100 socket-timeout 30000 from 127.0.0.1:43924

Priorities of HIGH and MEDIUM will default to rps 0.

Throttling Scans

Read more about how to throttle scans in the Throttling Options knowledge base article.

Monitoring Scans

Read more about how to monitor scans in the Monitoring Scans knowledge base article.

Partial Scans in Aerospike

Read more about issuing partial scans in the Partial Scans knowledge base article.

Compression in Scans

Does the compression option work for scans?

Yes, the command sent by the client wouldn’t be compressed, but the returned data would be compressed if that feature is enabled (requires Aerospike Server version 4.8 or above).

For Java client, it can be set like this:

               policy.compress = true;

The related metrics for compression are:

Keywords

SCAN RPS SCANPERCENT COMPRESS

Timestamp

July 2020

© 2015 Copyright Aerospike, Inc. | All rights reserved. Creators of the Aerospike Database.