What is the preferred approach to implement pagination in the below scenario?
Use Case - A growing record set - on which paging has to be supported in the API request on this data set.
Scenario-1.
Challenge - A separate set has to be created for this scenario, since the primary key will serve as the lookup for paging.
The paging will be done for the records whose lookup will be based on Primary Key.
Scenario-2
Challenge - The size for the data in the bin could exceed 1 MB. As a result the value has to be marshalled/serialized before storing in bin.
Cannot use secondary keys
Data has to be read in memory unmarshalled and then be paged based on the required field
Scenario-1 seems to be the way to go. In this case, are there any operations/api’s that limit the records than be read in a single I/O operation, kind of like paging on the server side?
Suggest use of List or Map for pagination to get the control over count from a specific point in a specific order. This would entail keeping these data structures in sync with the underlying primary-key data. Hope this helps.
From what I understand, Map or List types are at a bin for a given record. This would mean, I create some sort of my own index algorithm myself to support pagination
In my case, I will have “x” number of total records and each request will have to serve “x/pagezise” records, where pagesize can vary.
The records will be indexed in a List or Map bin. You can keep the list or map in the desired sorted order by including the appropriate field in each element along with the record key. This “index” will need to be kept in sync with the underlying data, and can be queried for any start point and pagesize to get the pageful keys of records for display.