The Aerospike Knowledge Base has moved to https://support.aerospike.com. Content on https://discuss.aerospike.com is being migrated to either https://support.aerospike.com or https://docs.aerospike.com. Maintenance on articles stored in this repository ceased on December 31st 2022 and this article may be stale. If you have any questions, please do not hesitate to raise a case via https://support.aerospike.com.
Impact of lowering or resetting a TTL to never expire on the TTL histograms
Abstract
Aerospike provides the ability to expire a record by setting its TTL (Time To Live) from a client. The TTL value is the number of seconds that a record will live (from the current time) before being removed by the server. What gets stored, though, is the void time (which is an absolute time computed in citrusleaf epoch – number of seconds since January 1st 2010 at 00:00:00 UTC). The expiration and eviction algorithms use the record’s TTL to determine eligibility for removal. The TTL value can be changed by a client with a simple record update. Changing the value can either accelerate, delay or prevent expiration or eviction of a record.
Record Expiration values (TTL)
| TTL (Client) | TTL (Server) | Description |
|---|---|---|
| -2 | same | Do not change ttl when record is updated. Supported by Aerospike server versions >= 3.10.1. |
| -1 | 0 | Never expire. Supported by Aerospike server versions >= 3.1.4. |
| 0 | def. | Default to namespace configuration variable “default-ttl” on the server. |
| >0 | >0 | Actual ttl in seconds (some client support the following time units S, M, H, D) |
Impact of changing the TTL on the TTL histogram
The TTL histogram is used in the server expiration algorithm to expire records at the specified TTL value. The histogram is based on the tracked max void time. This max value is used to determine the bucket width of the TTL and eviction histograms.
The max void time value can be influenced by one of these actions:
- The TTL value of a record is increased.
- The namespace storage is persistent to disk and the server is cold restarted.
- The Server is fast restarted.
Resetting the TTL of an existing record to “never expire” by updating the TTL to -1, or setting it to a lower TTL value from a client does not trigger the bucket width of the TTL histogram to get re-calculated. The Max Void time will remain the same until one of the following has occurred:
-
Server is fast restarted. The max void time will be adjusted to the next highest TTL.
-
A record with a Higher TTL is inserted or updated. The max void time will adjust to the higher TTL.
Conclusion
In certain cases where bucket widths and evictions precision is important and using persisteny storage, one should be aware of the behavior of changing a record’s TTL (to never expire or to a lower value then originally set).
The record will not be eligible for evictions or expirations after its TTL has been set to “Never Expire” (-1 from an Aerospike Client, or 0 from default-ttl)
The TTL histogram width will not change since the max void time would remain the same. This particular scenario can impact the eviction precision as more records may fall on same buckets.
Notes
Here is why a cold restart of the service may not help as a workaround:
If multiple versions of the record are read from persistent storage, the effect on the tracked max void time would depend on the order in which those versions are read.
- If the older version with the large void time is read first, it would set a large max void time and the histogram bucket width would still be too big.
- If the newer version with the zero void time (i.e. never expire) is read first, the histogram would be sized accurately.
Example
Create multiple records in aql with different TTLs, 3600, 7200, 5 days, 20 days Lower nsup-period to 10 seconds
aql inserts:
aql> set OUTPUT json
aql> set RECORD_PRINT_METADATA true
aql> set RECORD_TTL -1
aql> INSERT INTO bar.demo (PK, foo, bar) VALUES ('key1', 123, 'abc')
OK, 1 record affected.
aql> set RECORD_TTL 3600
aql> INSERT INTO bar.demo (PK, foo, bar) VALUES ('key1', 123, 'abc')
OK, 1 record affected.
aql> set RECORD_TTL 7200
aql> INSERT INTO bar.demo (PK, foo, bar) VALUES ('key2', 123, 'abc')
OK, 1 record affected.
aql> set RECORD_TTL 17280000
aql> INSERT INTO bar.demo (PK, foo, bar) VALUES ('key3', 123, 'abc')
OK, 1 record affected.
aql> INSERT INTO bar.demo (PK, foo, bar) VALUES ('key4', 123, 'abc')
OK, 1 record affected.
aql> set RECORD_TTL -1
aql> INSERT INTO bar.demo (PK, foo, bar) VALUES ('key4', 123, 'abc')
OK, 1 record affected.
aql> INSERT INTO bar.demo (PK, foo, bar) VALUES ('key3', 123, 'abc')
OK, 1 record affected.
aql> set RECORD_TTL 432000
aql> INSERT INTO bar.demo (PK, foo, bar) VALUES ('key5', 123, 'abc')
OK, 1 record affected.
The output of ttl histogram shows that the never expire record (0-vt) is correctly removed from the histogram, but the bucket width remained the same:
$ asinfo -v 'hist-dump:ns=bar;hist=ttl'
bar:ttl=100,172799,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2;
$ asinfo -v 'hist-dump:ns=bar;hist=ttl'
bar:ttl=100,172799,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2;
$ asinfo -v 'hist-dump:ns=bar;hist=ttl'
bar:ttl=100,172799,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1;
$ asinfo -v 'hist-dump:ns=bar;hist=ttl'
bar:ttl=100,172798,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0;
Keywords
max-vod-time,ttl,0-vt,evictions,histogram
Timestamp
08/08/17