I apologize if this isn’t the correct place to discuss the Aerospike Server source code, but I couldn’t find another other place for discussion, Github or otherwise.
I have a use case where the KV pairs have a pretty small value field (~100 bytes), so 64 bytes of overhead per key means a 1:2 ratio between RAM and SSD, which is less than idea.
With this in mind, I started going through the source code to figure out if the index memory could be reduced. I looked through the as_index_s structure to see where memory could be saved. Here is my assessment:
- Dropping the flex_bits_1, flex_bits_2 and dim fields should be pretty harmless if there is no need for in-memory storage and only SSDs are being used. This saves 10 bytes.
- generation could perhaps be made smaller without significant increase in chances of accessing stale data.
- In my use-case, the keys are unique 64-bit numbers, so the key could be smaller as well, though this would be a more involved change because many parts of code assume a 20-byte key. The LDT code, in particular, munges data in the key digest. It should be easier to have a smaller key if LDT support is not needed.
- I don’t understand the comment around the color and migrate_mark fields. Would it be incorrect to pack these bits in one byte?
I’d like to solicit your (read: the developers) thoughts on this.