Out of local ports - cannot connect


#1

I previously posted an issue where we could not connect via asadm as it was using the loopback address 127.0.0.1. I determined, with the help of the awesome people on this forum, that it was running out of ports to connect to. I increased this on the server side but now I need to know why one of my servers is using over 40k ports when the other servers are around 300.

Details: 14 nodes in the cluster each node has roughly ~300 ports open at any given time whereas the server in question has over 43k.

Server 1 that has the issue:

[root@server1 aerospike]$ netstat |grep hbci |wc -l
43848

Another server that is running normal:

[root@server3 ~]# netstat |grep hbci |wc -l
312

Any help would be greatly appreciated!!


#2

Could be caused by a hot key on that server, though this disparity is surprising.

Could provide the following outputs:

asadm -e "watch 11 3 show latency"
asadm -e "watch 11 3 show stat like connections"