Concurrent read with multiple services

How Developers Are Using Aerospike Data Modeling
1

Hello, I want to read records from aerospike with multiple instances without double-reading. My plan is:

Add time field.

Read records which are older that 1 sec.

Update time field (prevent reading for next 1 sec)

I can read this records but time doesn’t react on my operation, bin-value still with old time

But how can i update time-field and get UPDATED records in one query?

Aerospike
Aerospike775×466 35 KB

2

As of Aerospike Database version 5.1, you can’t update while doing a scan or query. Those are read operations.

You can combine a read+update in a single key operation, using operate(), which initiates a multi-operation transaction on the record. So the first operations can be reading bins, and the other operation can be modifying the ‘Time’ bin.

However, records older than time X are easy to find via metadata, without use of that secondary index on the “Time” bin. The record metadata includes last-update-time and you can use a predicate expression to filter out records whose last update time >= time-X (too new). This predicate filter can be applied to a scan, query, batch-read or single key operation.

The great thing about using metadata operations is that they are much faster, and if the predicate fails the record isn’t read off the disk at all. The metadata about the record is stored in the primary index, a 64 byte entry per record. So, in your case you shouldn’t bother with the “Time” bin at all, nor the secondary index over it.

But I’m not sure about your reluctance with regards to reading + updating. You could do the following:

  • Scan for all the records that have changed in the last x time, using the predicate expression I described above, with a scan policy of ‘no bins’. This is a metadata only operation that walks the primary index and doesn’t read anything from storage. What you get back is just the record metadata, and from that you can assemble the digests of all the records you want to manipulate.
  • Next you can run through this list of digests, either synchronous operations on multiple threads, or async, and do an operate transaction with two operations: read the data you want + touch the record. This causes the metadata to update, resetting the last-update-time.

I don’t really understand your use case, so this is one generic way to do it, mostly I want to clarify how operations work in Aerospike.

I might have misunderstood you, and you don’t want to read all the records older than 1 sec. You just want to avoid reading records that are too new (a pause). In that case, if you know the keys of the records you’re interested in, you would do a batch-read on all of them, with a predicate expression to filter out any of those records whose last-update-time is too new. Or alternatively you could read those records asynchronously with the transaction (operate()) I mentioned.

There are different ways to model things in Aerospike. Take a look at the Aerospike developer blogs on Medium and DEV.