Aerospike Kafka connector throwing error max connections 25 would be exceeded

Connectors Kafka
1

Hi,

Aerospike connect to Kafka from time to time throwing the following error when running in standalone mode as below

$KAFKA_HOME/bin/connect-standalone.sh etc/connect-standalone.properties etc/aerospike-sink.properties

This is the error:

Caused by: com.aerospike.client.AerospikeException$Connection: Error Code -7: Node BB96D5801CA8B7C 50.140.197.220 3000 event loop 2 max connections 25 would be exceeded.
        at com.aerospike.client.cluster.Node.getAsyncConnection(Node.java:573)*
        at com.aerospike.client.async.NettyCommand.executeCommand(NettyCommand.java:150)
        at com.aerospike.client.async.NettyCommand.run(NettyCommand.java:144)*
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:446)*
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)

I have seen similar case for Java client. Is there a default hard limit or this is bug please?

Thanks

2

That is the underlying java client’s hard connection limit. Assuming you are using 12 event loops and the default ClientPolicy.maxConnsPerNode (300), there will be 25 connections available per node/event loop (300/12 = 25).

If you were using the java client directly, you could either increase maxConnsPerNode or just disable the limit by setting EventPolicy.maxCommandsInProcess to 25 for each event loop. That way, only 25 concurrent commands are allowed per event loop and excess commands are queued for later execution when slots become available.

Unfortunately, there does not seem to be a way to configure these java client variables from the current Aerospike Kafka connector.

3

Like Brian says, the current Kafka connector does not allow setting these values aside from the default. The next major release has a reworking of the configs that give better options that effect policy in the client which includes these variables.

4

Thanks Joe and Brian.