Bytes as Outline Data


I have a table with records like: PK, Int1, Int2, Bytes1, Bytes2, Bytes3 Those BytesX Bins are about 1~3KB each, and never changed once put in DB.

When add a new Bin Bytes4 into the same PK: PK, Int1, Int2, Bytes1, Bytes2, Bytes3, Bytes4 What Aerospike does is to remove previous record (with 3 Bins of Bytes as Inline data) for PK, and add a new record (with 4 Bins of Bytes as Inline data) with the same PK. That results in a large disk I/O Writing (DB In-Memory + disk Persistence).

Could those Bins of Bytes store as Outline data? Thus only a Pointer (int) is stored as Inline data in the record which linked to the Outline Bytes data, and reduce disk I/O Writing significantly.

Aerospike LDT lmap sub-records is an idea way for my use case, but with bad performance (8X slower):

An alternative way is to separate those Bins of Bytes into another table: Table1: PK, Int1, Int2, 1, 2, 3, …

Table2: PK:1, Bytes1 PK:2, Bytes2 PK:3, Bytes3 …

But Aerospike lack of Batch Remove support (need to remove multiple records from Table2 when remove one record from Table1):


Thanks for the feature request. Our first priority is to improve the performance of LDTs so they can be reasonable for this use case, when the data sizes are reasonable. That work should see strong results shortly.