FAQ - Evictions Depth

Knowledge Base Operating
1

The Aerospike Knowledge Base has moved to https://support.aerospike.com. Content on https://discuss.aerospike.com is being migrated to either https://support.aerospike.com or https://docs.aerospike.com. Maintenance on articles stored in this repository ceased on December 31st 2022 and this article may be stale. If you have any questions, please do not hesitate to raise a case via https://support.aerospike.com.

FAQ - Evictions Depth

Evictions in Aerospike is the process of accelerating the expirations of records when one of the High Water Mark (HWM) is breached (high-water-disk-pct, high-water-memory-pct or mounts-high-water-pct).

Evictions can also occur at startup if records fall within the stored eviction depth from a previous HWM breach.

More details on evictions mechanism and bucket widths can be found on the Eviction Mechanisms in Aerospike article.

During any eviction cycle on a particular namespace, the server will store the eviction depth. This eviction depth (which can be checked through the evict_ttl metric), is the highest TTL of the records that have been evicted.

It is important to be aware of the eviction depth value and its impact during startup.

What happens during evictions at startup?

At startup the eviction process will check the stored eviction depth for a namespace and evict all records that have a ttl within that expiration range. In other words, all records that would expire in less than evict_ttl time from now would get deleted. This is to avoid breaching a HWM during a cold start which could considerably slow down the cold restart.

Note: When upgrading from version 4.5.0 or below to a version between 4.5.1 and 4.5.8, the eviction would occur after start up, during the first nsup cycle, potentially leading to very slow start up. This issue (internally tracked under AER-6179) was addressed in the following hotfix releases: 4.8.0.5, 4.7.0.10, 4.6.0.12, 4.5.3.14, 4.5.2.14 and 4.5.1.19.

How do I reset the eviction depth?

The following command can be used to reset the evictions depth on version 4.5.1 and above:

asinfo -v 'eviction-reset:namespace=<NAMESPACE>;ttl=<Eviction depth TTL in SECONDS>'

Example:

Resetting the namespace eviction depth to 1 second in the future. This targets any records that would expire in 1 sec from now and set that as the eviction depth in the evict.smd file for the namespace bar.

asinfo -v 'eviction-reset:namespace=bar;ttl=1'

A null value can be set by omitting the ttl setting in the command. This would set the eviction depth for that namespace to null in the evict.smd file but would not prevent evictions using a potential pre-existing eviction depth stored on a namespace disk header (pre-4.5.1):

asinfo -v 'eviction-reset:namespace=bar'

What is the format of evict.smd?

Server version 4.5.1 introduced a new system metadata protocol (smd) which included, for the first time, a .smd file to keep track of evictions depth per namespace. In previous versions, the eviction depth was written to the device header.

The file will have the following key values pairs:

key: The namespace that had the evictions event.

value: The current eviction depth, expressed as a void time in seconds since 1 January 2010 UTC (which is 1262304000 in Linux Epoch).

generation: number of updates that occured for that entry.

timestamp: time that entry was created.

Example:

cat /opt/aerospike/smd/evict.smd 
[
   [
      201131949969731,
      20
   ],
   {
      "key": "bar",
      "value": "322691594",
      "generation": 20,
      "timestamp": 317508342151
   }

How do I check the current eviction depth?

The current eviction depth can be found through asadm, through the following metrics:

Example:

Admin> show stat like evict_
~~~~~~~~~~~~~~~~~~~bar Namespace Statistics (2020-01-23 22:04:00 UTC)~~~~~~~~~~~~~~~~~~~
NODE               :   NODE1:3000             NODE2:3000   
evict_ttl          :   0                      5182017                                      
evict_void_time    :   322691594              322691594                                    
smd_evict_void_time:   322691594              322691594

How do I exclude a set from being impacted by the eviction depth?

The set-disable-evictions configuration can be used to exclude a set from being impacted by the existing eviction depth (or any evictions for that matter).

asinfo -v "set-config:context=namespace;id=namespaceName;set=setname;set-disable-eviction=true"

How do I convert the eviction void time to a date?

Let’s assume an eviction void time of 322691719.

The forecasted expiration is: 322691719 + 1262304000 = Monday, 23-Mar-20 20:35:19 UTC

This can be obtained through the following bash command line:

date -d @$(($AEROSPIKE_EVICT_VOID_TIME + 1262304000))

Example:

Linux:

$ date -d @$((322691719 + 1262304000))
Mon Mar 23 13:35:19 PDT 2020

MAC OS X:

$ date -r $((322691719 + 1262304000))
Mon Mar 23 13:35:19 PDT 2020

Notes

  • The following log line can be used as well, when evictions are occurring:

{ns-name} nsup-start: evict-threads 1 evict-ttl 2745 evict-void-time (287665530,287665930)

Keywords

EVICTIONS EVICT-TTL EVICT-VOID-TIME

Timestamp

January 2020