Per-namespace cold-start behaviour on erasing storage for one namespace


#1

Per-namespace cold-start behaviour on erasing storage for one namespace

Summary

When you remove, or empty, storage for a namespace, upon aerospike restart, it will find the storage does not contain any Aerospike header. As such, that namespace will be cold-started (as it is empty). Any data, which may have still been in the system’s shared memory for that namespace’s will be freed.

All other namespaces will still fast restart, as would normally be expected, as the mechanism for deciding cold/fast (or warm) start behaviour is per-namespace.

Behavioural details and logs

First start of node with namespaces ‘test’ and ‘bar’ configured to use file-backed storage.

$ service aerospike start
...
Oct 29 09:17:23 aero-mydc_1 asd[6127]: Oct 29 2018 09:17:23 GMT: INFO (drv_ssd): (drv_ssd_ee.c:768) {test} peek couldn't open file /opt/aerospike/data/test.dat: No such file or directory
Oct 29 09:17:23 aero-mydc_1 asd[6127]: Oct 29 2018 09:17:23 GMT: INFO (namespace): (namespace_ee.c:268) {test} found no stored data, will cold start
Oct 29 09:17:23 aero-mydc_1 asd[6127]: Oct 29 2018 09:17:23 GMT: INFO (drv_ssd): (drv_ssd_ee.c:768) {bar} peek couldn't open file /opt/aerospike/data/bar.dat: No such file or directory
Oct 29 09:17:23 aero-mydc_1 asd[6127]: Oct 29 2018 09:17:23 GMT: INFO (namespace): (namespace_ee.c:268) {bar} found no stored data, will cold start
Oct 29 09:17:23 aero-mydc_1 asd[6127]: Oct 29 2018 09:17:23 GMT: INFO (namespace): (namespace_ee.c:376) {test} beginning cold start
Oct 29 09:17:23 aero-mydc_1 asd[6127]: Oct 29 2018 09:17:23 GMT: INFO (namespace): (namespace_ee.c:376) {bar} beginning cold start
...

Behaviour on normal restart

Clearly marked in logs as fast restart for both namespaces (referred to as warm restart):

$ service aerospike stop
$ service aerospike start
...
Oct 29 09:18:56 aero-mydc_1 asd[12035]: Oct 29 2018 09:18:56 GMT: INFO (namespace): (namespace_ee.c:360) {test} beginning warm restart
Oct 29 09:18:56 aero-mydc_1 asd[12035]: Oct 29 2018 09:18:56 GMT: INFO (namespace): (namespace_ee.c:360) {bar} beginning warm restart
...

Behaviour on emptying/removing storage for a single ‘test’ namespace

As seen in the logs below, the ‘test’ namespace will cold restart, as there is no storage found, while the namespace we didn’t touch, ‘bar’, will still fast restart. On such a situation, the data in shared memory which used to belong to the ‘test’ namespace, will be freed.

$ service aerospike stop
$ rm -rf /opt/aerospike/data/test.dat
$ service aerospike start
...
Oct 29 09:19:54 aero-mydc_1 asd[12269]: Oct 29 2018 09:19:54 GMT: INFO (drv_ssd): (drv_ssd_ee.c:768) {test} peek couldn't open file /opt/aerospike/data/test.dat: No such file or directory
Oct 29 09:19:54 aero-mydc_1 asd[12269]: Oct 29 2018 09:19:54 GMT: INFO (namespace): (namespace_ee.c:268) {test} found no stored data, will cold start
Oct 29 09:19:54 aero-mydc_1 asd[12269]: Oct 29 2018 09:19:54 GMT: INFO (namespace): (namespace_ee.c:376) {test} beginning cold start
Oct 29 09:19:54 aero-mydc_1 asd[12269]: Oct 29 2018 09:19:54 GMT: INFO (namespace): (namespace_ee.c:360) {bar} beginning warm restart
...

Keywords

COLD RESTART EMPTY NAMESPACE

Timestamp

10/29/2018