We are deploying Aerospike in multiple AWS locations (4 regions - 8 AZ). At the moment our data comes from one location only. We are planning on XDR setup. We have two options
Follow star topology - the only location does XDR to all other location (ten other installations) - will this not put lot of pressure on the source cluster.
Chain topology - Source → Location A → Location B → …
Good question. You have to choose the topology based on the roundtrip times between the DCs. Its not a good idea to mix DCs with very varied roundtrip times in the same star topology (for e.g US DC → US & Europe & Asia). The reason is that the replication will be as slow as the slowest link. To avoid this, you can use a mixture of Chain and star topologies. Make sure that each of the hop is as efficient as it can be.
1&3 : It is little cumbersome to handle this case but nevertheless it is supported. When configuring DCs in XDR section, you can use the public-ips of the remote DCs. However, you need to add a mapping of internal-ip to external-ip using the ‘dc-int-ext-ipmap’ config option. Make sure you give the mapping for all the nodes of remote DCs. For details of why and how to do this, refer to http://www.aerospike.com/docs/operations/configure/cross-datacenter/network/.
2 : I dont think you can save the cost as the amount of data transfer is the same in star topology or chain topology (AWS charges by the amount of cross-region data transfer). You can save on the latencies by seeing the latencies across amazon regions and choosing your topology. One thing to note in amazon setup is that the latencies are not highly consistent. In AWS setup, we have seen scenarios where the XDR transfers are stalled for a bit and then resume. XDR is designed to handle this scenarios, so it will eventually catchup.
Do you mean that for XDR to work we would need public IP to be associated with each node in the cluster? In my case I have the the Aerospike in private subnet.
If it has public IP then it would be a normal scenario where specifying one IP is good enough and we would not need dc-int-ext-ipmap. Please let me know if that is not the case.
You would need in-ext mapping if you want the local clients to talk on the private IP but the XDR on public IP. You can create security group rules accordingly.
However, My colleague Kevin suggests that you can also use VPC peering across regions using the “Direct connection” Feature of AWS. You should be able to talk across regions using the private IPs if you setup a VPN tunnel between the regions. For more details refer to : AWS Direct Connect virtual interfaces - AWS Direct Connect