The Aerospike Knowledge Base has moved to https://support.aerospike.com. Content on https://discuss.aerospike.com is being migrated to either https://support.aerospike.com or https://docs.aerospike.com. Maintenance on articles stored in this repository ceased on December 31st 2022 and this article may be stale. If you have any questions, please do not hesitate to raise a case via https://support.aerospike.com.
XDR not shipping from one or multiple nodes after upgrading to new 3.8.1 version.
Problem Description
After upgrading to 3.8.1 there could be a possibilty that XDR will show zero throughput on one or many nodes. And you see this message in the log file.
Apr 19 2016 09:47:40 GMT-0400: WARNING (cf:socket): (socket.c:249) bind: socket in use, waiting (port:3004)
Explanation
This is due to the fact that while upgrading, the old XDR process wasn’t stopped and both old and new xdr are trying to use the same digest log file. The above warning about “socket in use” is because of the old XDR port still defined in the configuration.
Even if the configuration is updated, digestlog will still be corrupted if the old XDR process is not stopped before the new ASD is installed.
Solution
To recover from either situations:
Stop the old XDR . Delete the digest log file and restart ASD.
sudo service aerospike_xdr stop
sudo service aerospike restart
For versions 3.8.3 and above, you can issue a XDR restart by ignoring the pending records in the digestlog to mitigate the issue:
asinfo -v 'xdr-command:skip-outstanding'
Notes
Please follow the rolling upgrade to XDR in ASD doc diligently while upgrading to the new XDR. http://www.aerospike.com/docs/operations/upgrade/xdr_to_3_8 http://www.aerospike.com/docs/operations/upgrade/xdr_to_3_8/release_details_3_8
Applies To
Server prior to v. 5.0
Keywords
XDR, Upgrade, 3.8.1, xdr_in_asd.
Timestamp
4/21/16