Event loop max connections are exceeding upon new connections being created


#1

HI, I am using the Aerospike Java Client with version 4.0.1. I am calling Async APIs of AerospikeClient. In the below situation i see an error : “Caused by: com.aerospike.client.AerospikeException$Connection: Error Code -7: Node BB9F73E0B270008 172.28.128.3 3000 event loop 2 max connections 75 would be exceeded”

Situation Created 4 event loops and the ‘maxConnsPerNode’ default value is not changes which is 300. Connection time outs are the default values used which is 1 minute.

Initially when i submit around 200 Async operations, 200 connections are getting created which executes successfully. After a while, say 5 minutes during which the previous connections would have expired (1 minute is the default expire time which is not changed), if 200 more new Async commands are issued then the Aerospike client throws the error which is reported initially.

Ideally, since the old connections are expired 200 more connections should be created and executed. But instead results in the above error.

I had a look at the class - com.aerospike.client.cluster.Node,

public final AsyncConnection getAsyncConnection(int index, ByteBuffer byteBuffer) { AsyncPool pool = asyncConnectionPools[index]; ArrayDeque queue = pool.queue; AsyncConnection conn;

	while ((conn = queue.pollFirst()) != null) {
		if (conn.isValid(byteBuffer)) {
			return conn;
		}
		conn.close();
	}
	
	if (pool.total >= pool.capacity) {
		throw new AerospikeException.Connection(ResultCode.NO_MORE_CONNECTIONS, 
				"Node " + this + " event loop " + index + " max connections " + pool.capacity + " would be exceeded.");
	}
	pool.total++;
	return null;
}

Here i see that before returning a connection, it checks if any connections in the pool are valid then return them, else create a new connection and return. But the counter is not decremented correctly when closing the connections. Hence the connections are not tracked correctly and results in the situation which i have described.

Can you please fix this (If already fixed let me know the fixed version)? Let me know if any further information is needed.

Thanks, Chandrashekar


#2

We will provide a fix soon.


#3

Fixed in 4.0.3

http://www.aerospike.com/download/client/java/4.0.3/