Multiple NICs in the Aerospike Java Client


#1

Introduction

To get improved network throughput, one may go down the path of adding multiple NICs on nodes in the cluster. The problem is how to get clients distribute their requests on the different NICs of the server node.

A Client instances cannot use multiple IPs for a node

There does not seem to be a way to make sure that a single client communicates with all network interfaces of a node - say in a round-robin fashion.

Clarification: Seed hosts passed in client init cannot be used for this

The Host(s) passed in the AerospikeClient constructor is only used to request the server host list and populate the cluster map. The multiple Hosts passed here are used only in case of network failure on the first one.

Lets say there are 2 nodes A and B, each with 2 network interfaces (A1, A2 and B1, B2). Also the server config does not mention any access-address. This means it will advertise all its IPs.

The Java client will get all 4 IPs - but it determines that two IPs belong to the same node and uses the first one.

  • A1 -> A
  • A2 -> A
  • B1 -> B
  • B2 -> B Here the client will use A1 and B1.

So, if there are multiple clients, the internal cluster map on each client depends on the list returned by the INFO command response from the server. If this is the same list in the same order (which is likely), all clients will use only one IP for communicating with a node.

Workaround (not recommended): Using ipMap in ClientPolicy

Lets say we have multiple clients. And we want different clients use different IPs to communicate to the same node. Say P and Q are two clients. We would like P use A1 and Q use A2 to connect to the same node A. This can be achieved using the “ipMap” parameter in com.aerospike.client.policy.ClientPolicy where we specify different "ipMap"s for the two clients.

P can be initialized with the following ipMap:

  • A1 -> A1
  • A2 -> A1
  • B1 -> B1
  • B2 -> B1

and Q can initialized with the following ipMap:

  • A1 -> A2
  • A2 -> A2
  • B1 -> B2
  • B2 -> B2

Better solution: Link Aggregation/Bonding

Link Aggregation or NIC bonding are methods of combining multiple network connections in parallel in order to increase throughput beyond what a single connection could sustain (and also provide fault tolerance).