The current cluster is using the heartbeat in mode multicast, however the multicast support is unstable on my network recently. So we want to change the cluster heartbeat to mode mesh.
Any suggestions for this operation to avoid any data loss during the migration? Can I change the heartbeat mode without losing mine data? Or I can only do this by setting up a new cluster?
Thank you.
You should stop your application traffic, shut down all the nodes in the cluster, make the heartbeat configuration change in the cluster and then bring up the nodes one at a time. If your data is persisted (storage-engine device) you will not lose any data.
This way will cause lots of migration and bring back some deleted data, is that right? It may take very long time to balance the cluster also. As far as I know, maybe it is the only way to get things done.
Thank you.
Yes, this is correct when using the Community Edition. The Enterprise Edition provides the Fast Restart option (for namespaces not in memory) which would not have this issue of deleted data resurrected upon restart. The latest 3.8.3 Enterprise Edition release also provides Rapid Rebalance which is order of magnitude faster for migrations when there is small amount of writes on the cluster during migrations.