The Aerospike Knowledge Base has moved to https://support.aerospike.com. Content on https://discuss.aerospike.com is being migrated to either https://support.aerospike.com or https://docs.aerospike.com. Maintenance on articles stored in this repository ceased on December 31st 2022 and this article may be stale. If you have any questions, please do not hesitate to raise a case via https://support.aerospike.com.
How to move, recreate or retain the System Metadata Directory
Context
The System Metadata (SMD) directory holds information on Secondary Indexes, UDF definitions, Truncation, Strong Consistency, User/Roles and more. It may be necessary to move, migrate or retain this information, for example, when upgrading to or over Aerospike version 4.5.1 (especially when replacing instances as part of the upgrade as commonly done in cloud environments). This article discusses methods by which this information can be moved, migrated or retained.
Method 1 - Using persistent shared storage to hold the file
Normally SMD information is held in /opt/aerospike/smd. It is possible to use the
work-directory configuration parameter to specify where files and directories
that would normally be under /opt/aerospike are held. This parameter can be set to a persistent shared storage location and so the information would
not be lost should a new machine or cloud instance be required.
Method 2 - Copy to a safe location
If the Aerospike daemon on the node in question is shut down the /opt/aerospike/smd directory can be copied to a remote location and moved back once
any changes have been made. This means that the work-directory does not need
to change.
Method 3 - Recreate from other nodes in the cluster
If a node in the cluster starts without any information in the SMD directory, such as when a brand new node joins the cluster, the information will be received from other nodes within the cluster.
Notes
- Aerospike 4.5.1 involves a non backward compatible protocol change for the SMD sub-system. Therefore, in a mixed clusterm SMD operations such as truncations or changes to UDFs will not propagate across all the nodes in the cluster. Method 3, given above, is not suitable for an upgrade to or over Aerospike 4.5.1
- Be careful with the SMD directory when bringing a node in from another cluster or even a past state of the same cluster, as the propagating SMD information could lead to unwanted truncates or other problems.
- The /opt/aerospike/smd/security.smd contains timestamp/generation for a user:
{
"key": "user3|P",
"value": null,
"generation": 6,
"timestamp": 361731948039
}
Even when a user is “dropped”, the entry is still there with a value of “null” which allows resolving any conflict.
Keywords
SMD MIGRATE RECREATE
Timestamp
03/18/2019