How to set up XDR remote Data Centers on AWS

Knowledge Base XDR
1

The Aerospike Knowledge Base has moved to https://support.aerospike.com. Content on https://discuss.aerospike.com is being migrated to either https://support.aerospike.com or https://docs.aerospike.com. Maintenance on articles stored in this repository ceased on December 31st 2022 and this article may be stale. If you have any questions, please do not hesitate to raise a case via https://support.aerospike.com.

How to set up XDR remote Data Centers on AWS

Context

When running Aerospike 3.8.1 and higher on AWS and using XDR to replicate between datacenters, extra configuration is necessary. AWS instances expose a public IP for external connections but use an internal IP for intra-cluster communication. For XDR to discover the remote datacenter properly it must be able to map between the two. There are two methods to carry out this mapping.

Method 1 - Alternate Address

The simplest method to map between internal and external IP is to use the alternate-address parameter within the network stanza. When using any node where multiple network interfaces are available, we specify the access-address parameter and give the IP address to be used here. On AWS and Google Compute Engine we add the keyword virtual. To configure the mapping between the internal and external IP addresses we simply add the alternate-address parameter to the network stanza and instruct XDR to use this by setting dc-use-alternate-service to true in the DC specification of the XDR stanza. An example is shown below:

network {
  service {
    address any                  # IP of the NIC on which the service is
                              	 # listening.
    port 3000                    # port on which the service is listening.
    access-address 172.31.20.188
    alternate-address 52.53.54.55
                            
  }

...

  xdr {
...
   datacenter DC1 {
       dc-node-address-port 52.53.54.55 3000
       dc-node-address-port 52.53.54.56 3000
       dc-use-alternate-address true
   }
}

XDR will then use the alternate address specified on the remote node and discover the remote cluster properly. It is much more flexible to use the alternate-address method of specifying remote nodes as it does not require the source cluster to be updated when a node is added at the destination. This is advantageous when the XDR topology is complex. To add flexibility, it is entirely possible to use a Round Robin DNS name in the dc-node-address-port when using alternate-address, the stanza would therefore look as follows:

  xdr {
...
   datacenter DC1 {
       dc-node-address-port remotehost.mydatacenter.com 3000
       dc-use-alternate-address true
   }
}

Where remotehost.mydatacenter.com is the Round Robin DNS that would resolve to a host in the remote cluster.

Method 2 - Int Ext Ip Mapping

The other method to map between internal and external addresses on the remote datacenter is to include the mappings directly within the XDR stanza using the dc-int-ext-ipmap parameter which can be added at the bottom of the datacenter sub-stanza in the XDR stanza. An example is shown below with the addresses starting with 172 being internal.

xdr {
...
   datacenter DC1 {
	dc-node-address-port 52.53.54.55 3000
        dc-node-address-port 52.53.54.56 3000
        dc-int-ext-ipmap 172.31.41.38 52.53.54.55
        dc-int-ext-ipmap 172.31.41.39 52.53.54.56
}

Notes

  • From Aerospike 3.8.1 onwards it is recommended to use method 1 as it is more flexible if internal IP addresses change.

Applies To

Server prior to v. 5.0

Keywords

AWS XDR REMOTE DATACENTER INTERNAL EXTERNAL DC-INT-EXT-MAPPING ALTENATE-ADDRESS DC-USE-ALTERNATE SERVICES

Timestamp

4/29/16