XDR Overhead on a Cluster

Operations XDR (Cross Data Center Replication)
1

Hello,

Q1 I wanted to understand the XDR overhead a cluster would incur. Is there a way to estimate this overhead to see if would have to increase count/RAM/SDD/NW?

Q2 I have a situation that I have to XDR to 8 locations from my 3 DCs.

So roughly 3 XDR configurations per cluster. How does the number of XDR locations impact the parent cluster (liner, exponential, logarithmic).

Q3 How does it impact if the destinations do no have equal latencies?

Any suggestions?

Thanks!

2

Hi Amod, thank you for posting your question on Aerospike forum. Let me try and answer this for you.

Q1. The only overhead XDR will have on the source cluster is the extra read which it does while shipping the data to a remote cluster. There is no overhead on RAM/SSD. But since it is cross data center replication there will be overhead on the network, based on the amount of data you are replicating.

Q2. The number of XDR locations does not affect the parent cluster in any way, having said that this is true when links to all the destination clusters are up and running. But when there is a remote DC down situation, you will have that extra overhead of read on the source cluster. This is because Aerospike does not stop shipping to all DCs just because one DC is down.

Q3. It is not recommended and not a good idea to have destinations with large latency differences. XDR throughput will only be as good as your slowest destination.

If you have 8 DCs to ship to, then it’s better to have the source DC ship to the lower latency ones first, and then forward it to other DCs from there, you can configure XDR to do this automatically using “forward-xdr-writes” feature.

Ref: http://www.aerospike.com/docs/reference/configuration/#forward-xdr-writes

As a rule of thumb, having one source and too many destinations is not a good idea. Your network can be a bottleneck in this case. Hope this helps.

Ref: http://www.aerospike.com/docs/operations/configure/cross-datacenter/#cross-datacenter-replication-xdr

Thanks

Rakesh

2 Likes
3

Thank you for elaborate answer.

4

Few follow up questions

  1. Does XDR read the complete data for a key and replaces it with the remote DC or works with changes.
  2. So the amount of data transfer would be (data size of batch * number of xdr locations).
  3. Is there any optimization in terms of compression of data?
  4. Can I bind XDR to a N/W interface and make use of other N/W interfaces for read. So they do not impact each other.
1 Like
5

My answers corresponding to your questions.

  1. Yes
  2. correct. because of (1)
  3. We have the config option ‘xdr-compression-threshold’. If you set its value to 1000, all the records beyond 1000 bytes will be compressed using lz algorithm. We have seen good returns of this above 1000 bytes. You can go lower too.
  4. Not available as a setting using XDR config. XDR on the source is just a client to the destination. XDR does not bind to a socket (except for info commands which are rare). So, you can define your network traffic rules using external tools like ‘ip rule’,‘ip route’. But make sure the NIC is really the bottleneck before doing this. Otherwise, its lot of customization without a real need.