Thanks. All clear now.
For what it’s worth, in my opinion, I don’t think there’s a use case where unpredictable/inconsistent behavior is preferable to consistency. Whether it’s that the result is the deletes are always ignored on scan, or the ttls are ignored, or enforced no matter if it was re-inserted, or something.
In any situation even when you might have re-inserted the record this behavior wouldn’t be great since it may result in the record being deleted on restart or the generation before the delete, or after the reinsert. So the inconsistency makes this edge case you reference not really ok either.
I’d personally prefer after any static rule so that I can say that the server will be in a predictable state after a restart, even if that predictable state is that deletes are always ignored so I have the latest generation of records regardless of any ttl or delete. Then at least it’s easier to come up with a consistent solution to consistent behavior.
I believe this is an aerospike bug but I understand you guys don’t have the same perspective. I think non-deterministic/unpredictable behavior in the way data is loaded from disk in any data store should be considered a bug.
Anyway we can work around this now that we are aware of the exact behavior and we will just never remove a record except when we’re prepared to zero disks to avoid inconsistent behavior.