What are the options for reducing XDR's network utilization?


#1

XDR throttling configuration

Question

Is there any parameter that would let us control or limit the network being used over XDR.

Explanation

XDR ships data across data-centers and can heavily utilize the available network bandwidth. XDR will throttle shipping data under some circumstances (refer to the knowledge base on XDR Throttling for details). However, if you intend on imposing limits on XDR’s bandwidth utilization, here are some configuration options to consider.

Solution

  • Configure the maximum throughput - Tuning the maximum shipping throughput (number of records written per second) which will control the number of records in flight (number of records on the wire between the source and destination data-centers) per destination data-center. By default, this is set to 0 which means it will be unlimited (well, technically capped at a maximum of 50,000 records in flight per destination).

Refer to this link for more details: xdr-max-ship-throughput

  • Enable the bin-level shipping - By default, XDR will ship the complete record and will replace the record at the destination. Turning xdr-ship-bins to true will only ship the modified or new bins in most cases thus reducing the network bandwidth depending on your usage of the bins.

Refer to this link for more details: xdr-ship-bins

  • Enable record level compression - By default, compression is disabled. If you set its value to 1000, all the records beyond 1000 bytes will be compressed using lz algorithm. And these records will be uncompressed on the destination.

Refer to this link for more details: xdr-compression-threshold

Keywords

XDR throttling bandwidth throughput

Timestamp

08/18/2017


What is the unit for XDR max ship throughput?