FAQ Why are migration messages shown even when migrate-fill-delay

FAQ Why are migration messages shown even when migrate-fill-delay is set

Detail

A cluster has a non-zero value set for migrate-fill-delay however in the event of a node removal the aerospike.log shows partitions to be migrated as follows:

Jun 04 2019 13:56:01 GMT: INFO (partition): (partition_balance.c:957) {migrate-test} rebalanced: expected-migrations (1340,1389,1340) fresh-partitions 0

Why are these messages shown?

Answer

The answer lies in the two part mechanism by which an Aerospike cluster will rebalance after a cluster event.

  • The first part of that is the cluster determines which partitions should go to which nodes in the new cluster state.

  • The second part of the process is the movement of partitions between nodes to reflect the partition mapping decision.

When migrate-fill-delay is set to a non-zero value it inserts the specified delay between these two steps. The cluster decides where the partitions will go and then waits for migrate-fill-delay until it actually moves the data.

This will be reflected in a subsequent log line as follows:

Jun 04 2019 13:56:04 GMT: INFO (info): (ticker.c:454) {test} migrations: remaining (1340,1389,1340) active (0,0,0) complete-pct 0.00

The line is stating that there are pending migrations but that there are none active as the migrations have been delayed.

The product is working as designed.

Keywords

MIGRATE FILL DELAY MIGRATIONS ACTIVE

Timestamp

June 17 2019