Large ordered list: explicit append()-operation (AER-2989)

When working with LDTs above a certain size it would be cool to have an append()-operation that allows inserts to the end of list (ascending values). As far as I can see, AS currently traverses the whole B±Tree down even when an entry will be inserted to the very end. Should be easy to implement and save some Read OPs especially with very large lists. The operation should simply throw an exception if max key in tree is bigger than what you are trying to append.

Useful e.g. for time series data or where u don’t have any key (in that case, it would be cool for it to automatically assign an auto-incremented key, if possible. For now we have used 64-bit timestamps not necessarily related to the data to get around the “unique key’s only”-constraint.

Manuel,

Your suggestion is already considered. Where the operation if possible performed only on list in ROOT. And at certain threshold incur cost of traversing through tree and make bulk insert. This should save lot of IO cost for the cases where people need queue or stack or have append only case data. Thanks for bringing up again.

Generally the system generating its own ID has its drawback when there are retries and timeouts. This can cause system to insert duplicate values. But still generating such ID at the client (application need not) similar to other’s :slight_smile: could be considered. But that as well has its own pitfalls like clock sync etc.

– R

1 Like

FYI,

JIRA AER-2898 for this.

– R

1 Like

That’s cool to hear! Sounds like this would potentially increase write-throughput by a factor x and also help with lock contension (if that is a bottleneck for LLists).

About the ID: would see that in relation to the min/max feature mentioned in docs. If you have a max, it should be easy to just increment on int64 max key (server side, while locked), or am I missing something? (Dunno how easy that would be with replication logic). Not really important, client-app-sided 64-bit timestamps work too for now, it’s just a ‘dirty hack’ without any logical guarantues. Would rather move timestamps into the value part for all the reasons you mentioned.

The LDT feature has been removed in Aerospike Server 3.15.1. For more details see Aerospike Will No Longer Support the Large Data Type (LDT) Feature | Aerospike.

This topic was automatically closed 6 days after the last reply. New replies are no longer allowed.