FAQ Tuning async Java Client connections


FAQ Tuning the Async Java Client connections

Why use the Async Client:

  • If the application is deployed within an asynchronous framework.

  • The Async API provides tighter controls through event loop and single thread.

In general, it is preferred to use the Synchronous API as it is much easier to tune and debug and the potential performance gains are often not noticeable.

Async client tuning:

The following knobs can be used to tune the async Java Client connections: asyncMaxCommandAction, asyncMaxCommands, asyncSelectorThreads, asyncSelectorTimeout, timeoutDelay

In more details:

asyncMaxCommandAction (DEFAULT: BLOCK)

asyncMaxCommandAction uses 3 states to handle cases when the asynchronous maximum number of concurrent connections have been reached:

  • ACCEPT: Accept and process command. This implies the user is responsible for throttling asynchronous load. You are not limited when choosing Accept.
  • BLOCK: Block until a previous command completes. (Default value)
  • REJECT: Reject database command.

asyncMaxCommands (DEFAULT: 200)

asyncMaxCommands has a default value of 200 commands (tested on enterprise class machine with 8 cores). This value is the maximum number of concurrent asynchronous commands that are active at any point in time. Concurrent commands can be used to estimate concurrent connections. The number of concurrent open connections is limited by:

max open connections = asyncMaxCommands * Number of Cluster Nodes

The actual number of open connections consumed depends on how balanced the commands are between nodes and if asyncMaxCommandAction is set to ACCEPT.

The maximum open connections should not exceed the total socket file descriptors available on the client machine. The socket file descriptors available can be determined by the following command: ulimit -n

asyncSelectorThreads (DEFAULT: 1)

asyncSelectorThreads is the number of event loops configured in the client.(Java Library calls Selector what C client calls an events loop) Number of selector threads used to process asynchronous network events. The default is a single threaded network handler. Some applications may benefit from increasing this value to the number of unused cores on a machine running other Core dependent services… There is no benefit to having more threads than you have cores.

ayncSelectorTimeout (DEFAULT: 0)

ayncSelectorTimeout has a default of zero (milliseconds) with no timeouts of selector. Default may have to change. Every time you put a new command on the queue, the selector wakes up. If using async client sporadically, then you should modify the asyncSelectorTimeout to limit Wait state (if no commands sent).

For example: if transactions of 5 ms, the Selector timeout should also be at 5ms.

How timeouts are implemented in java async.
  1. run Selector thread
  2. Add new command to timeout queue
  3. Check for timeout
  4. Enter Selector wait state until it fires some results
  5. Process Socket reads and Writes.

In Wait state we do not check for timeout. Its not an independent thread.

ayncSelectorTimeout should be average timeout for your transactions. The exception would be if you are always pumping data through client all the time. Than no risk to go into a Wait State for too long. Low Selector Timeout may increase CPU usage as you are checking for timeouts more often.

timeoutDelay (DEFAULT: 0 (no delay, connection closed on timeout))

It may be worth also tuning timeoutDelay if you have aggressive timeout and you are closing a lot of connections. This may result in an open and close loop with lots of Socket in Close Wait state.(There is a cost to opening Socket connections). The timeoutDelay is a delay in milliseconds after a transaction timeout before closing a socket in async mode only. Async transactions only retry on invalid connections in the connection pool which is not bounded. When a transaction is stopped prematurely, the socket must be closed and not placed back on the pool. This is done to prevent unread socket data from corrupting the next transaction that would use that socket. This field delays the closing of the socket to give the transaction more time to complete in the hope that the socket can be reused. This is helpful when timeouts are aggressive and a certain percentage of timeouts is expected. The user is still notified of the timeout in async mode at the original timeout value. The transaction’s async timer is then reset to this delay and the transaction is allowed to continue. If the transactions succeeds within the delay, then the socket is placed back on the pool and the transaction response is discarded. Otherwise, the socket must be closed. This field is ignored in sync mode because control must be returned back to user on timeout and there is no currently available thread pool to process the delay.

Recommendation - Iterative method

Tuning async client requires running a benchmark on your system. It’s usually best to iterate starting with default values and incrementing while monitoring your service. asyncSelectorThreads should not exceed number of free CPU cores on your client. One should take into consideration any other applications running on the client machine that may also be using dedicated CPU Cores when adjusting this number. Trial and error to minimize resource consumption and maximize Throughput.



asyncMaxCommandAction, asyncMaxCommands , asyncSelectorThreads , ayncSelectorTimeout