Is there an IO operation if performing a query with includeBinData=false against TTL?


#1

Is there an I/O operation if performing a query with includeBinData=false against TTL?

Question

My application uses predicate filtering and has includeBinData set to false. In what scenario would it incur an IO operation?

qp.includeBinData=false;
RecordSet rs = client.query(qp, genStatement());
....
Statement stmt = new Statement();
stmt.setNamespace("test");
stmt.setPredExp(
PredExp.recVoidTime(),
PredExp.integerValue(min),
PredExp.integerGreaterEq(),
PredExp.recVoidTime(),
PredExp.integerValue(max),
PredExp.integerLess(),
PredExp.and(2)
);

Solution

The order of the operations in such scenario is as follows:

  1. Fetch list of candidate digests from index.
  2. Apply predicate filter based on metadata Generation / TTL etc.
  3. Filter out expired records.
  4. Read records from the disk and cross check secondary index reference.
  5. Apply predicate filter based on data binvalue etc.

Thus, if a record does not qualify after TTL check - then there will be no IO.

This is also specified in the code base for the server on GitHub repository.

But if record qualifies after TTL check, note that even though user has specified includeBinData false - it will incur IO primarily due to the need to check if reference in sindex is valid (sindex does lazy garbage collection in case of data not in memory while performing read).

Notes

Keywords

includeBinData

Timestamp

Sept 11 2018