Enable XDR Dynamically


#1

FAQ - Can XDR be enabled dynamically?

Detail

XDR is a feature that some customers may wish to try once they have already got their first Aerospike cluster up and running. In this instance it may be advantageous to have XDR configured but not running so that it can be started later.

Answer

Aerospike 3.8 and higher

Configuring XDR so that it can be started dynamically is a simple process. The two parameters requred for the namespace stanza, enable-xdr and xdr_remote_datacenter are both dynamic and can be set using asinfo without a node restart. The following commands would do this:

Enable XDR shipping for the namespace:

 asinfo -v 'set-config:context=namespace;id=<NAMESPACE>;enable-xdr=true'

Specify the remote datacenter the namespace should ship to

asinfo -v "set-config:context=namespace;id=<NAMESPACE>;xdr-remote-datacenter=DC1;action=add"

It is necessary to have a skeleton definition for the remote datacenter already present in the aerospike.conf file. This only needs to have the datacenter name, the dc-use-alternate_services parameter if that is used, and the dc-security-config-file if the remote cluster uses the security feature. An example XDR skeleton stanza is shown below:

xdr {
     enable-xdr false 

     xdr-digestlog-path /opt/aerospike/xdr/digestlog 100G 
     # make sure the /opt/aerospike/xdr folder exists

     datacenter DC1 {

           dc-use-alternate-services true
           dc-security-config-file /private/aerospike/security_credentials_DC1.txt
     }
}

In the example above the xdr directory must be created and writeable by Aerospike under /opt/aerospike/ and the security configuration file must be available and readable by Aerospike under /private/aerospike/security_credentials_DC1.txt.

Remote nodes can be added dynamically using the following command:

$ asinfo -v "set-config:context=xdr;dc=DC1;dc-node-address-port=xx.xx.xx.xx:3000;action=add"

Finally enable-xdr in the xdr stanza is switched as follows:

asinfo -v 'set-config:context=xdr;enable-xdr=true'

Now XDR if fully enabled and shipping after having had only a very basic skeleton configuration in the aerospike.conf file.

Note: In versions prior to 3.14.1.1 dynamically setting enable-xdr to true at the xdr context does not properly work. In those versions, enable-xdr should always be set to true at the xdr stanza level in the configuration file and controlled dynamically (or in the configuration file) at the namespace stanza level in order to enable or disable writing digest log entries into the digest log.

Aerospike releases pre-3.8.x

Prior to Aerospike 3.8 the XDR process was separate to the main Aerospike process. The aerospike.conf file has to be fully populated with XDR information xdr-remote-datacenter in the namespace stanza and a full xdr stanza including all seed nodes within the datacenter definition. If this is done, XDR can be enabled as follows:

$    asinfo -v 'set-config:context=namespace;id=<NAMESPACE>;enable-xdr=true'
$    asinfo -v 'set-config:context=xdr;enable-xdr=true'

The XDR service is then started as follows:

$ sudo /etc/init.d/aerospike_xdr resume

Notes

  • Because a skeleton remote datacenter definition is required to enable XDR dynamically this should be added to the aerospike.conf file before the node is started, ideally after the upgrade or install of Aerospike 3.8.x. This is necessary as with the 3.8 release XDR shipping became part of the central Aerospike process.
  • Namespaces can be disassociated with a remote datacenter as follows:
asinfo -v "set-config:context=namespace;id=test;xdr-remote-datacenter=DC1;action=remove"
  • Namespace Sets replication can also be whitelisted or blacklisted for replication dynamically:
asinfo -v 'set-config:context=namespace;id=<NAMESPACE>;sets-enable-xdr=<true|false>'

asinfo -v 'set-config:context=namespace;id=<NAMESPACE>;set=<SETNAME>;set-enable-xdr=<false|true>'

addtional info : XDR Sets replication whitelisting

  • Once XDR shipping has been enabled it is good practise to amend the aerospike.conf to add the XDR details to the namespace stanza and xdr stanza so that these persist in the event of a node restart.
  • asadm can be used to issue the asinfo commands shown here to all nodes within the cluster
  • A pending feature request (AER-4993) will enable dynamically setting the location of the security credentials file.

Keywords

XDR DYNAMIC START CONFIGURE RESTART 3.8

Timestamp

10/03/16


FAQ - How to disable XDR
How to add a new Datacenter (DC) in XDR
#2

I believe, an entry like below in aerospike.conf

dataceneter DC1 {
 }

is good enough to make “DC1” a skeletal DC. The alternate-services and security-config file are dynamic and can be added later.