FAQ Does XDR Ship deletes?

Knowledge Base XDR
#1

FAQ Does XDR ship deletes?

Detail

When XDR is shipping to a remote data center, does it ship deletes from the local cluster?

Answer

In default configuration XDR will ship all deletes that have been issued explicitly from a client. This behaviour can be controlled by using the ignore-expungeswhich defaults to false (or, for XDR versions prior to 5.0, xdr-delete-shipping-enabled which defaults to true).

By default XDR would not ship deletes that have been issued by the namespace supervisor sub-system, NSUP. NSUP issues deletes as part of the following processes:

  • Expirations
  • Evictions
  • Set deletions from the asinfo command (deprecated in version 3.12)

If the use case requires these deletes to be sent to the remote DC this can be controlled with the ship-nsup-deletes (of, for XDR versions prior to 5.0, xdr-nsup-deletes-enabled).

Notes

  • Sets deleted using any scan based utility or UDF are sent to remote DCs by default as these are not controlled by NSUP.
  • Sets truncated using truncate command will not be propagated via XDR.
  • Data deleted as durable deletes will always be propogated via XDR.
  • Deletes are propagated from the node which is the master for the partition in question. If that node leaves the cluster, the new master will ship updates or subsequent deletes.
  • For XDR versions prior to 5.0 only: If a delete arrives while the master node is going down, the delete will occur on the new master which would be the previous replica and will propagate from there. If the original master then rejoins the cluster it may then ship a copy of the deleted record (assuming that the record was in the digest log when the node went down but the delete was not) and therefore it would be feasible for deleted data to reappear on the target DC. Of course, in such cases, the record does also re-appear at the source cluster. In version 5.0, the removal of the digestlog eliminates this and other similar issues. Use of truncate (at the source and destination) or durable delete would avoid such issues in the older XDR versions.

Keywords

DELETE EVICTION NSUP XDR MISSING REMOTE

Timestamp

July 2020

#2

What is the update on this topic vis-a-vis set-truncate, durable deletes and tombstones? Per Meher: truncate will not propagate. set-delete (deprecated 3.12.1+) would need xdr-nsup-deletes-enabled (off by default) for regular client deletes, they should go through and should be durable and need xdr-delete-shipping-enabled (on by default)

#3

KB has been updated so you should have your answers now. :slight_smile:

© 2015 Copyright Aerospike, Inc. | All rights reserved. Creators of the Aerospike Database.