Bin convergence and Support of multiple counters in XDR

[Atul]

Hi All,

I am posting to this thread to validate our approach. And also, How can bin convergence help us.

We are using 5.7+ enterprise aerospike version. We are supporting a high TPS system, say 100 thousand request per second. We have introduced a system to track number of hits by our each customer. That is why, we are using counters in Aerospike for every customer. Also we have two data centers in Aerospike with XDR support more like active-active HA.

Current approach: we have 7 counters for each and every day of week in both data center (DC).

PROD DC: P_SUN, P_MON, P_TUE, P_WED, P_THU, P_FRI and P_SAT

HA DC: H_SUN, H_MON, H_TUE, H_WED, H_THU, H_FRI and H_SAT

Lets say today is Monday then,

Customer request 1 → PROD cluster → Prod aerospike opr.add(“P_MON”, 1)

Customer request 2 → HA cluster → HA aerospike opr.add(“H_MON”, 1)

Since, we have the two bins for single Day of week. Thus respective DC request for same customer updates the counter in DC’s bin only and data is sync through XDR to other DC.

For a Aerospike key (i.e. customer_id) request can land on any of DCs in active-active HA mode. We have separate counter on both DC. So to calculate the total usage, we have to sum all 14 bins in the runtime call.

How can bin convergence help us? Can we have single bin for a day of week in XDR mode of cluster, so that we don’t loose modification request in XDR sync?

[kporter]

Bin convergence makes sure that bins converge to the same values on all DCs using the last update metadata which is added per bin for this feature. If you have one or more bins that are updated by multiple DCs then this feature will ensure that all DCs eventually arrive at the same values for these bins.

Consider a record containing the bin “favorite-color” that is replicated to 3 DCs. Currently, all DCs agree that the value is “Green”. Suddenly this bin is updated on 2 of the 3 DCs - one is updated from “Green” to “Red” and the other is updated from “Green” to “Purple”. Without the bin-level LUT metadata, we cannot make a deterministic decision for the final state for the 3rd DC - so it may converge to either “Red” or “Purple” (also the two DCs that received these updates will likely swap values). With the additional metadata in bin convergence, we’ve defined rules to deterministically converge to the same value on all DCs. One of the states will be dropped based on the convergence rules, either “Red” or “Purple”.

This wouldn’t help if you are trying to maintain an accurate count of something across DCs. The algorithm will ensure all DCs have the same value for this count but some of the updates will be dropped as seen in the color example. Splitting a counter into a counter per DC is still the best practice for an accurate cross DC counter.