What is the preferred approach to implement pagination in the below scenario?
Use Case - A growing record set - on which paging has to be supported in the API request on this data set.
Scenario-1. Challenge - A separate set has to be created for this scenario, since the primary key will serve as the lookup for paging.
- The paging will be done for the records whose lookup will be based on Primary Key.
Scenario-2 Challenge - The size for the data in the bin could exceed 1 MB. As a result the value has to be marshalled/serialized before storing in bin.
- Cannot use secondary keys
- Data has to be read in memory unmarshalled and then be paged based on the required field
Scenario-1 seems to be the way to go. In this case, are there any operations/api’s that limit the records than be read in a single I/O operation, kind of like paging on the server side?