The secondary index storage as of today mandates complete presence in main memory at all times. While dealing with indexes accommodating millions of records, I did see a significant increase in the usage of secondary index memory. Is there any work planned/(in progress) for providing support for paged index structure(sort of multi level index) so as to avoid indices to be loaded completely in memory.
Kratos,
Right now there is no such plan. Thanks for the requests, will pass it on the product engineering.
Thanks – Raj
Raj, Can you recommend any approach for storage in scenarios wherein the individual objects stored are small enough to make the difference between their size and the size of their corresponding index entry not big enough so that the memory benefits of not keeping the actual data in memory aren’t significant. Ideally if a namespace is configured for in-memory storage, the index memory requirements(the one present in memory at all times) should be small enough to make the whole “fast ssd access” compelling enough.