XDR Topology - Star vs. Chain

We are deploying Aerospike in multiple AWS locations (4 regions - 8 AZ). At the moment our data comes from one location only. We are planning on XDR setup. We have two options

  1. Follow star topology - the only location does XDR to all other location (ten other installations) - will this not put lot of pressure on the source cluster.

  2. Chain topology - Source → Location A → Location B → …

Please suggest.

Good question. You have to choose the topology based on the roundtrip times between the DCs. Its not a good idea to mix DCs with very varied roundtrip times in the same star topology (for e.g US DC → US & Europe & Asia). The reason is that the replication will be as slow as the slowest link. To avoid this, you can use a mixture of Chain and star topologies. Make sure that each of the hop is as efficient as it can be.

You can do something like

A->B,C,D
B->E
C->F
E->G,H

thanks a lot!

Do we save anything by doing A → B, C, D than A → B - >C- > D

  • Ease of configuration and management. XDR runs at only one place.
  • XDR at A reads the record once and ships to all DCs (B,C,D).

Thank you! Probably the last follow up question :smile:

Suppose all these locations are AWS regions. What is the strategy we should follow provided

  1. We want to keep Aerospike in private subnet.

  2. We want to lower latency and lower cost - may be chain the sync process. us-west → us-east → eu-central → ap-north

  3. VPC peering is not possible across regions (to allow private IPs).

Your questions 1 & 3 are similar.

1&3 : It is little cumbersome to handle this case but nevertheless it is supported. When configuring DCs in XDR section, you can use the public-ips of the remote DCs. However, you need to add a mapping of internal-ip to external-ip using the ‘dc-int-ext-ipmap’ config option. Make sure you give the mapping for all the nodes of remote DCs. For details of why and how to do this, refer to http://www.aerospike.com/docs/operations/configure/cross-datacenter/network/.

2 : I dont think you can save the cost as the amount of data transfer is the same in star topology or chain topology (AWS charges by the amount of cross-region data transfer). You can save on the latencies by seeing the latencies across amazon regions and choosing your topology. One thing to note in amazon setup is that the latencies are not highly consistent. In AWS setup, we have seen scenarios where the XDR transfers are stalled for a bit and then resume. XDR is designed to handle this scenarios, so it will eventually catchup.

Thanks Sunil.

Its nice to see the support that is provided by Aerospike. Which public IP are you referring to?

The public IPs that AWS would assign to those nodes.

Do you mean that for XDR to work we would need public IP to be associated with each node in the cluster? In my case I have the the Aerospike in private subnet.

If it has public IP then it would be a normal scenario where specifying one IP is good enough and we would not need dc-int-ext-ipmap. Please let me know if that is not the case.

You would need in-ext mapping if you want the local clients to talk on the private IP but the XDR on public IP. You can create security group rules accordingly.

However, My colleague Kevin suggests that you can also use VPC peering across regions using the “Direct connection” Feature of AWS. You should be able to talk across regions using the private IPs if you setup a VPN tunnel between the regions. For more details refer to : AWS Direct Connect virtual interfaces - AWS Direct Connect