Ordered List Scan / Range Limit


#1

I want to use Large Ordered List for paging. But when scan / execute range i don’t want all item on the list / range to be returned, i want just the top/lower n, is there a way to limit how many item that should be returned?


#2

We do not have a “first N” or “last N” limit on the scan() or range() functions. However, this is one of the benefits of the UDF mechanism. It would be relatively easy to create a UDF wrapper around the LDT functions to perform these extra features (such as get the first N, last N elements). So, although some extra elements might have been extracted from the LDT, at least the UDF could select just the needed elements out of the result list before the full result set got shipped to the client.

And, if it turns out that this becomes a desirable feature for LLIST, we will consider putting that feature into the mainline LDT function.


#3

Can you provide a simple example to get the first N or last N of an ordered list? I find that it can’t be done with the api here http://www.aerospike.com/docs/udf/ldt/lolist.html

I think this will be a really desirable feature for LLIST, as paging is a very common use. And it will make Aerospike more appealing for those who has used Cassandra before.


#4

Sure thing. I’ll see if I can get something posted next week.

By the way – this is exactly the type of feedback we’re looking for. Collective interest in a particular feature definitely raises its priority.


#5

I am going to give this an extreme +1 here.

Better support for querying with limit not only LDT but querying / filtering items in an entire set.

Use Case: INSERT INTO blog.posts (PK, title, body) VALUES (‘1’, ‘Foo’, ‘Foo Body’) … INSERT INTO blog.posts (PK, title, body) VALUES (‘100000000’, ‘Bar’, ‘Bar Body’)

SELECT * FROM blog.posts LIMIT 10 SKIP 100

I’m sure the internals of Aerospike make selecting data in a set and doing limits/skips on “default” criteria is probably impossible/difficult? I haven’t looked into it THAT much. Just wish I could do that easily.

I know there are workarounds such as creating perhaps a LDT and storing pre-made paginated lists then issuing a batch GET on those keys. But having it built into it would be so buttery. :stuck_out_tongue:


#6

We often have similar discussions regarding the function vs performance tradeoffs.
More feedback is going to help us navigate this landscape.

Regarding the LDT llist.find_first(n) and llist.find_last(n). It is implemented, but I don’t know how long it will take to be externalized.
I’ll update this post when I have an approximate data or server release number.


#7

how about creating another type of ldt that can use limit query? So everyone can choose wether to use the one with more function or the one that has better performance.


#8

ok – I think I know what you mean by that, but can you give me a more detailed example? I don’t want to misinterpret your question.


#9

Sorry, Can anyone share simple UDF wrapper which supports limit of results? I’m new in Aerospike and UDF. I will be nice to get the example.

To be more precise I wounder is it possible somehow limit size of result set (Java client)?

ResultSet rs = client.queryAggregate(null, stmt);

Is it possible to same as here https://github.com/helipilot50/aerospike-batch-processing , but using “query” or “queryAggregate” methods?


#10

In the Java client, if you want to stop processing a RecordSet (query) or a ResultSet (aggregation) after a certain number of results, you need to call its close() method.


#11

Thank you for the answer. But can I continue fetch next results after calling close()? I need functionality like pagination.


#12

@arypurnomoz, @jrdn and @sergsh:

Thank you for posting about LDTs in our forum. Please see the LDT Feature Guide for current LDT recommendations and best practices.


#13

@arypurnomoz, @jrdn and @sergsh:

Effective immediately, we will no longer actively support the LDT feature and will eventually remove the API. The exact deprecation and removal timeline will depend on customer and community requirements. Instead of LDTs, we advise that you use our newer List and SortedMap APIs, which are now available in all Aerospike-supported clients at the General Availability level. Read our blog post for details.