How to move Aerospike XDR traffic onto a different network interface

How to move Aerospike XDR traffic onto a different network interface from your client traffic

Context

If your network interface doesn’t have enough bandwidth to simultaneously serve client traffic and the XDR traffic to your remote data centers, you can move your XDR traffic to a different network interface.

Background

When an Aerospike cluster connects to another Aerospike cluster via XDR, every node in the source cluster connects to the destination cluster as follows:

  1. Connect to the destination cluster on one of the IP address configured in node-address-port. This is known as seeding.
  2. Request the IP addresses of all nodes in the destination cluster.
  3. Using this list of IP addresses, connect to all nodes in the destination cluster in order to send the XDR traffic from this node.

Method

To switch XDR to a different interface, configure the destination cluster nodes to return the IP address of the interface that you will use for XDR. Do not use the normal IP address returned in Step 2. To do this, configure Aerospike with an alternate-access-address, and then tell the XDR source cluster nodes to request this address when requesting a list of IP addresses for all nodes in the destination cluster.

On all nodes in the XDR destination cluster

  1. If you have address configured in your network stanza, and it is not set to any, you must add an additional address entry for your new XDR interface.
  2. Configure an alternate-access-addresss which will be used for the incoming XDR traffic.
  3. Restart Aerospike on the node to make the new config take effect
  4. Once the node has rejoined the cluster, move on to the next node

On all nodes in the XDR source cluster

  1. Disconnect XDR from the destination cluster:
asinfo -v "set-config:context=xdr;dc=DC1;namespace=myNamespace;action=remove"
  1. Configure the dc configuration for the XDR destination to use the new addresses we configured by setting use-alternate-access-address. This should be added to the config file and also dynamically:
asinfo -v "set-config:context=xdr;dc=DC1;use-alternate-access-address=true"
  1. Reconnect XDR to the destination dc, rewinding by 600 seconds to capture any updates that happened since we disconnected from the destination dc in step 1:
asinfo -v "set-config:context=xdr;dc=DC1;namespace=myNamespace;action=add;rewind=600"
  1. Check that XDR traffic is now using the new network interface

  2. Repeat on all nodes in the source cluster

Notes

  • The two interfaces should be on different subnets, as this process relies on the OS routing to choose the correct route to the destination cluster. If the interfaces are on the same subnet, it cannot be guaranteed that the source node OS will route the packets as expected.
  • This process assumes that you are moving your XDR traffic to a new interface on both source and destination clusters. If you only need to move the traffic to a different network interface on the source cluster, you can add a second IP address in a different subnet to the destination nodes to achieve the same result.

Applies To

Aerospike version 5.0 or later

Keywords

XDR NETWORK INTERFACE ALTERNATE

Timestamp

December 2021