The Aerospike Knowledge Base has moved to https://support.aerospike.com. Content on https://discuss.aerospike.com is being migrated to either https://support.aerospike.com or https://docs.aerospike.com. Maintenance on articles stored in this repository ceased on December 31st 2022 and this article may be stale. If you have any questions, please do not hesitate to raise a case via https://support.aerospike.com.
XDR 5.0 behaviour when adding a new DC.
Description
XDR 5.0 has a different behaviour when adding a new dc dynamically and when adding via rolling restart.
Behaviour when adding a new dc via rolling restart.
XDR maintains a last ship time for each DC within the system metadata directory (SMD) of each node. This is necessary as from Aerospike 5.0 there is no file based logging and incoming digests are held in in-memory queues prior to shipment. At start up, the node checks the last ship time for each configured DC and ships anything later than that last ship time. When a new destination is added while the node is down, XDR just ships from the beginning. The reason that recoveries metric increases is that the shipping described here is done in recovery mode whereby XDR reduces a partition rather than shipping from entries in an in-memory queue. This partition reduction process may occur several times within a given recovery cycle and so it is not unexpected to see an increment of greater than 4096 listed for the recoveries statistic. XDR will only ship for the master paritions that it owns after the restart.
The most efficient and the recommended way to add a new DC is to add it dynamically. As a standard practice, when ever a dynamic change is done, we should also change the config file to reflect that change, so that if there is a restart in the future the same config takes effect.
Behaviour when adding a new DC dynamically.
When adding a new DC dynamically, please refer to the Enable XDR Dynamically article for detailed steps.
We only ship records from that point and not the whole namespace. If there is a need to ship from a certain point or the whole namespace, the rewind feature can be leveraged.
NOTE: the latest versions from each lineage starting from 5.0 have the same behaviour. Only some intermediate builds from 5.1.0.3 had a different behaviour which was reverted as part of AER-6365.
For 5.1, it was fixed in 5.1.0.23 For 5.2, it was fixed in 5.2.0.15 For 5.3, it was fixed in 5.3.0.6.
Everything beyond 5.3.0.6 has a similar behaviour.
- Full documentation on how to configure XDR in both static and dynamic modes can be found here.
Keywords
XDR 5.0 STATIC NAMESPACE SHIPPING EVERYTHING ADDITION REWIND
Timestamp
March 2021