FAQ - Is entire record shipped to remote destination in XDR


#1

FAQ - Is the entire record shipped to remote destination in XDR

Detail

In the active-passive XDR configuration, the source cluster will ship records to the destination cluster. When updating a few bins in a record at the source, does the entire record get send to the destination or just the modified bins? This article clarifies one of the important concept related to XDR.

Answer

  • In Aerospike 3.6 or earlier, the entire record is sent as an update. This prevents bin deletions from being replicated.:

    XDR bin deletion

  • In Aerospike 3.7, the entire record is also sent but as a replace by default:

    http://www.aerospike.com/docs/reference/configuration#xdr-replace-record

  • In Aerospike 3.8, the xdr-replace-record configuration got deprecated and replaced by xdr-ship-bins which allows XDR to ship just the modified bins. The drawback of shipping modified bins only is the additional ‘read’ on the destination cluster before updating the record.

    http://www.aerospike.com/docs/reference/configuration#xdr-ship-bins

  • By default, xdr-ship-bins is false so the entire record will be sent to the remote destination. If the xdr-ship-bins configuration is set to true, then only the modified bins or new bins will be shipped except in the following cases:

    • Bins were deleted during the write. In this case we need to replace the complete record at the destination
    • If a local write updates more than 80% of the data in the record (or, for builds prior to 3.10.0.3, more than 80% of the bins in the record), then the complete record is shipped to save the record read at the destination.
    • The underlying implementation makes use of a bloom filter which may lead to false positives. In most common cases, though, the probability of a false positive (shipping an extra bin that was not modified) is extremely low (< 0.001 %). Do not hesitate to contact Aerospike for further details regarding specific use cases.

This can be summarized by this table:

| *version*    | *configuration*     | *default* | *ship partial record* |   *operation*   |
| ------------ | ------------------- | --------- | --------------------- | --------------- |
| pre-3.7      |                     |           |  No                   | update          |
| 3.7.x        |  xdr-replace-record | true      |  No                   | replace if true |
| 3.8+         |  xdr-ship-bins      | false     |  Yes if true          | replace if false|  
|              |                     |           |  (with exceptions )   | depends if true |

Keywords

XDR DC THROTTLE LATENCY REMOTE SHIP BINS PARTIAL RECORD

Timestamp

9/21/16