Set-delete flow


#1

Details on the set-delete flow

Aerospike Server version 3.12.1 or above

As of version 3.12.1, released in April 2017, set-delete is deprecated. The new feature of deleting all the data in a set or namespace is now supported in the database.

See the following truncate documentation: http://www.aerospike.com/docs/reference/info#truncate

Truncate can also be executed from the client APIs. Here is the java documentation: http://www.aerospike.com/apidocs/java/com/aerospike/client/AerospikeClient.html

Aerospike Server version 3.12.0 or below

When a set-delete command is issued on a node, it flips a flag on the set. On the next nsup run, partitions are reduced one by one (primary index’s rb tree) and, for each, deleted records (belonging to the set marked as such) are put into the dq (delete queue).

The dq is only used for nsup, to put records that are: evicted, expired and set-deleted. nsup runs every 120 seconds by default, for each namespace, sequentially. (see nsup-period - http://www.aerospike.com/docs/reference/configuration/#nsup-period - to tune this).

So, all deleted records for the first partition are dumped on this dq queue. Then, another thread, part of the nsup process looks at this queue and transfers them over into the transaction queue (called q in the logs), for a transaction thread to pick them up from there, do the job (delete), and park them in wr (writes in progress) for the prole part to be taken care of.

On the prole side, those delete transactions go straight into the transaction queue.

The next partition will not be reduced until the delete queue (dq) gets below a specific count (10,000). So if evictions or expirations are also hapening, this would delay the next partition reduction.

In build 3.5.8 and above, the default nsup-delete-sleep value changed from 0 to 100 micro seconds (time to sleep between generation of each delete transaction generated). This throttles exactly how fast the dq queue is consumed and how fast the transaction queue is built up. (This is more important for the transaction queue in order to help avoid filling up the wr queue too fast which could cause spike in memory usage, and contention with other writes.)


Guidelines for Deleting Data
Namespace stays in stop-writes even though memory is less than stop-writes-pct