The Aerospike Knowledge Base has moved to https://support.aerospike.com. Content on https://discuss.aerospike.com is being migrated to either https://support.aerospike.com or https://docs.aerospike.com. Maintenance on articles stored in this repository ceased on December 31st 2022 and this article may be stale. If you have any questions, please do not hesitate to raise a case via https://support.aerospike.com.
Synopsis:
After removing X nodes from the cluster, I still see them when running Asmonitor or Asadm.
# asmonitor
Enter help for commands
request to 172.23.17.13 : 3000 returned error
request to 173.27.17.11 : 3000 returned error
request to 172.23.17.10 : 3000 returned error
request to 172.23.17.12 : 3000 returned error
Solution:
Reason why the phantom nodes are still showing up in Asmonitor or Asadm is due to the services-alumni list. This list keeps all the neighbor addresses (services) this server has ever know about since its last restart. Diagnostic tools, eg. asmonitor and asadm, uses the alumni list to enable full cluster view even during cluster disruptions.
For versions 3.6.1 and earlier, the only way to remove these phantom nodes from the services-alumni list is to restart the existing node(s), or wait for the next planned restart of the nodes.
In version 3.6.2 and above, we have a reset option that can be triggerd in such events to refresh the services-alumni list and make it equal to the services list (which keeps the list of addresses of all the current neighbour nodes which is advertised to the clients).
For more information, see below page: