Default write consistency not working as expected


#1

The Aerospike consistency docs state that:

By default, write transactions (including deletes and UDF applications) commit all replicas synchronously before successfully returning from the transaction.

Also, the default commitLevel for WritePolicy in Java client is COMIT_ALL (consistent with docs).

We are running Aerospike server 3.9.1.1 community edition and using the 3.3.0 release of Java client. The server consists of 10 nodes with a replication factor of 3.

What I have observed (through integration testing) is that writes are not synchronous across the cluster. I have tested this with UDFs and with atomic map operations and in both cases there appears to be a race condition between the write and subsequent read, where the updated data is sometimes not returned. In both cases the default commit level is used for write consistency. I am using the async java client to perform an update, and then fetching the record when the update (i.e. operate/execute) is completed (i.e. the Java client calls onSuccess on the listener). If the write was truly synchronous across all replicas then I should be guaranteed that after onSuccess is called that a subsequent read will fetch the updated record.

Furthermore, our aerospike.conf does not override the default write consistency (i.e. write-commit-level-override).

I can provide more details about the cluster configuration, and even test code for reproducing if it’s helpful.

I am not sure if this is a bug with the aerospike server, the java client or I am not understanding the docs correctly.


#2

Could you share the .conf and test code… via github?