Official AS approach to pagination?

How Developers Are Using Aerospike Use Cases
1

Hi,

I’ve started building an AS-based POC as a replacement for Cassandra, but stuck with pagination (user posts with an infinite scroll). To be fair - there are a lot of places already discussing pagination (people asking either for offset/limit or cursor implementation), however all answers are actually workarounds with their disadvantages, rather than pointers to “official” function/implementation, for example - using LLIST.

Please advise, is it just my misunderstanding of AS’s goal and its just not meant to be used in pagination scenarios working more like an advanced version of redis, or there is actually a correct/native/official way to provide pagination like offset,limit of MySQL or pagestate in Cassandra (maybe using UDFs)? If latter, could you please provide a link to documentation (ideally, of the Go driver) describing the approach?

Thank you! D.

2

In the limited scope of discussion of secondary index query, no, Aerospike does not support pagination/cursor.

However, the discussion here is more about how to handle the specific use-case here of time-based events.

Options exist via -

  • secondary index
  • application bucketing
  • llist

Please take look at http://www.aerospike.com/docs/guide/ldt_guide.html for comparison of the various modeling options.

One note-worthy advantage with application bucketing is that it will expire naturally with Aerospike’s TTL feature on records.

3

@wchu, Hi,

the discussion here is more about how to handle the specific use-case here of time-based events.

Correction: the use-case in discussion is not for time-based events, rather - about a fixed amount of items to be returned.

Regarding LDT/LLIST: I’m new to AS so please correct me if I’m wrong, but I cannot get rid of impression that LLIST approach comes down to:

  1. Creating a dedicated set
  2. That set will have 1 row
  3. That row will have an LLIST-type bin, for example, post_index
  4. post_index will be a list of actual post IDs, sorted by creation desc (if supported), that I will be able to use doing classic pagination (cursor, preferred, or an offset/limit).

If all above is correct, then questions:

  1. This is definitely a workaround, i.e. something not inherently native to AS. What disadvantages I should expect?

  2. Could you please explain: “…For Enterprise with XDR, LDT data is not shipped…” in the end of http://www.aerospike.com/docs/guide/ldt.html

  3. In case we are using lists for this, should this be done via UDFs?

Thank you! D.

4

I assumed that your posts would be sorted by time, thus categorized to the same.

Your understanding of LLIST is correct. It absolutely has the caveat of lower performance of going through UDF, and not native to Aerospike.

The best solution IMO for your use case would be bucketing at the application level, where you can group the posts/events at the application level.

Hope this helps.

5

@wchu, may I ask you to elaborate on this approach, please?

Thank you, D.

6

Hi I am having the same problem , I have a “products” set which contains many products , I have a timestamp bin in the “products” set , what I need to do is to implement a query something like “Get the next 10 products from the current timestamp” , so can anyone suggest me any technique or method using which I can achieve my goal, Any help would be appreciated…

7

I would like to understand that approach better as well.

Thanks.

8

Check the latest Map APIs - make timestamp the key, “product” the value, and look at the GetByKeyRelativeIndexRange() API.

image
image877×141 19.7 KB
…

9

Currently there’s the approach of keeping the IDs in a List or Map and paginating on them with the native List and Map API methods, as Piyush suggested. It’s not ideal, but it’s functional enough as a workaround.

In upcoming versions of Aerospike there will be a way to scan and query with the ability to get several records at a time, optionally starting at the specific digest. You’ll implement pagination using that feature, asking for N records and then for another batch of N starting with the digest of the last record you got.

Stay tuned for the release announcements.

10

This topic was automatically closed 6 days after the last reply. New replies are no longer allowed.