What are the options for reducing XDR's network utilization?

Knowledge Base XDR
1

The Aerospike Knowledge Base has moved to https://support.aerospike.com. Content on https://discuss.aerospike.com is being migrated to either https://support.aerospike.com or https://docs.aerospike.com. Maintenance on articles stored in this repository ceased on December 31st 2022 and this article may be stale. If you have any questions, please do not hesitate to raise a case via https://support.aerospike.com.

XDR throttling configuration

Question

Is there any parameter that would let us control or limit the network being used over XDR.

Explanation

XDR ships data across data-centers and can heavily utilize the available network bandwidth. XDR will throttle shipping data under some circumstances (refer to the knowledge base on XDR Throttling for details). However, if you intend on imposing limits on XDR’s bandwidth utilization, here are some configuration options to consider.

Solution

  • Configure the maximum throughput - Tuning the maximum shipping throughput (number of records written per second) which will control the number of records in flight (number of records on the wire between the source and destination data-centers) per destination data-center. By default, this is set to 0 which means it will be unlimited (well, technically capped at a maximum of 50,000 records in flight per destination).

Refer to this link for more details: xdr-max-ship-throughput

  • Enable the bin-level shipping - By default, XDR will ship the complete record and will replace the record at the destination. Turning xdr-ship-bins to true will only ship the modified or new bins in most cases thus reducing the network bandwidth depending on your usage of the bins.

Refer to this link for more details: xdr-ship-bins

  • Enable record level compression - By default, compression is disabled. Setting the compression threshold to 1000, all the records with a size greater than 1000 bytes will be compressed using the lz algorithm. Compressed records will be uncompressed at the destination. Note: if data is already compressed, this option will not be effective.

Refer to this link for more details: xdr-compression-threshold

  • Configure hotkeys skipping - Avoid shipping too many updates for the same key in a short period of time. The xdr-hotkey-time-ms configures how long to wait for shipping a record after it has already been shipped once. The effectiveness of this setting can be checked through the xdr_hotkey_skip and the xdr_hotkey_fetch statistics. The larger the difference, the bigger the saving in term of shipping updates of the same records, albeit by delaying some of those.

Refer to this link for more details: xdr-hotkey-time-ms

Applies To

Server prior to v. 5.0

Keywords

XDR throttling bandwidth throughput

Timestamp

08/18/2017