Consider me curious. I am sure Aerospike would be using this if it was viable, so I’d be interested to hear why something like this isn’t viable.
Currently, Aerospike removes the reference from the index in memory, however this leads to data potentially reappearing if the sector (for want of better word) is not written over before a restart by new data.
Would it not be effective to do the current implementation and have a background thread over write it when IO is not being used as intensively so that it doesn’t impact performance, which is what the current design is aiming to do?
A technical explanation would be great, I’d love to learn more about the inner workings of Aerospike.
That is sort of what happens in the background by the nsup and defrag threads. Its the time lag in completing it and the way erase works on SSDs that exposes you to the problem. Durable delete feature in the enterprise edition writes a placeholder record, the tombstone, to compare against and resolve this problem.