We need to complete migrate an aerospike cluster from one network to the other in Google Cloud. We are currently using local SSD for data storage with indexes in memory.
For the migration, we need to delete the old nodes and create new nodes in the new network. We are planning to add shadow disks in the existing clusters and once they are synced, and start the new cluster in the new network using the shadow disks.
During the tests, when we started the nodes in new cluster one-by-one and partition migrations started happening as soon as new nodes joining the cluster. Since we already know that new nodes are going to come up with data in a few minutes, is there any way to delay/avoid the partitions migrations between the already started nodes? Ideally, there should not be any migrations as we know that in a matter of a few minutes all nodes will join the cluster with properly distributed partitions.
This task has been made very simple for Aerospike Enterprise users. You would just start the new nodes and then quiese the old nodes and issue a recluster. Once migrations have completed, stop the old nodes.
While the new cluster is starting with the shadow disks, is there any write load on the old cluster? If so, how do you plan to keep these clusters consistent?
Yes, migrate-fill-delay delays ‘fill’ migrations by a specified number of seconds. Fill migrations are basically migrations targeting partitions that have never been filled.
The distribution is based on the node-ids that are members of the cluster. The distribution will change when you move to a new set of machines unless you configure the node-id on each node to match the node-id the shadow disk originated from.
Sorry, I forgot to mention that we are using Aerospike Community Edition build 3.9.0.3. The migrate-fill-delay & node-id configurations are not available for this version. Is there any way to achieve the same thing for this version?
Just before starting the new cluster, we will take the old cluster offline. So, there will be a downtime until the new cluster is up and stable. That’s the reason we want to minimize the migrations in the new cluster.
.