During some peak times, my client failed to connect to aerospike with Error Code 11 as below
ERROR [resin-port-127.0.0.1:6800-7625] (AirPriceUapiOperationImpl.java:856) - com.aerospike.client.AerospikeException$Connection: Error Code 11: Failed to connect to host(s):
127.0.0.1:3000 Error Code 11: java.net.ConnectException: Cannot assign requested address
at com.aerospike.client.cluster.Cluster.seedNodes(Cluster.java:400)
at com.aerospike.client.cluster.Cluster.tend(Cluster.java:269)
at com.aerospike.client.cluster.Cluster.waitTillStabilized(Cluster.java:234)
at com.aerospike.client.cluster.Cluster.initTendThread(Cluster.java:144)
at com.aerospike.client.AerospikeClient.<init>(AerospikeClient.java:208)
at com.aerospike.client.AerospikeClient.<init>(AerospikeClient.java:149)
at com.mycompany.webservices.websvc.AirPriceUapiOperationImpl.getPricedItinerary(AirPriceUapiOperationImpl.java:206)
I have tried with “telnet localhost 3000”, it works
I have also tried to look up the transactions
[root@localhost ~]# grep trans_in_progress /var/log/aerospike/aerospike.log | tail
Dec 13 2015 05:52:17 GMT: INFO (info): (thr_info.c::4843) trans_in_progress: wr 0 prox 0 wait 0 ::: q 0 ::: iq 0 ::: dq 0 : fds - proto (11464, 44757055, 44745591) : hb (0, 0, 0) : fab (16, 16, 0)
[root@localhost ~]# grep ERROR /var/log/aerospike/aerospike.log | tail
nothing
As I check out the local ports…
[root@localhost ~]# sysctl net.ipv4.ip_local_port_range
net.ipv4.ip_local_port_range = 32768 61000
Max connections of the server is set to 2048
[root@wefly ~]# sysctl net.core.somaxconn=2048
net.core.somaxconn = 2048
What else could I try to tune?