pturrr
December 27, 2016, 3:27pm
1
Hi,
We’ve faced an issue today:
Dec 27 2016 13:59:04 GMT: WARNING (demarshal): (thr_demarshal.c:446) dropping incoming client connection: hit limit 150001 connections
while netstat showed not more than 1k connections to aerospike
we are using aerospike-server-community 3.10.1-1
How is it possible?
kporter
December 28, 2016, 1:00am
2
Which client/version are you using (Java, Python, etc)?
pturrr
December 28, 2016, 8:33am
4
We again noticed same issue:
Dec 28 2016 08:28:49 GMT: WARNING (demarshal): (thr_demarshal.c:446) dropping incoming client connection: hit limit 190001 connections
while asadm shows 1500 connections (same as netstat)
the host is extremely slow during that time and whole cluster experience issues with latency till aerospike restarted on this node.
we have 4 nodes cluster with replication-factor 2
kporter
December 28, 2016, 8:55pm
5
Was anything recently updated (client or server)?
Is this a new usecase? A common mistake is closing and reopening the aerospike client for each request. The aerospike clients are thread safe and should only need one for an application.
pturrr
December 30, 2016, 8:42am
6
We think we found a bug at our client side. Thanks for help