Facing TLS write failed: -2 error during put operation

Operations Configuration
1

I am trying to use aerospike 6.0 as cache with the following config for my namespace:

namespace application_cache {
        memory-size 5G
        replication-factor 2
        default-ttl 1h
        nsup-period 1h
        high-water-memory-pct 80
        stop-writes-pct 90
        storage-engine memory 
        }

There is TLS configured for the database and I am using Node.js client version 5.2.0 to connect to the database. This is the config I am using:

"config": {
      "captureStackTraces":true,
      "connTimeoutMs": 10000,
      "maxConnsPerNode": 1000,
      "maxCommandsInQueue" : 300,
      "maxCommandsInProcess" : 100,
      "user": "test",
      "password": "test",
      "policies": {
        "read": {
          "maxRetries": 3,
          "totalTimeout": 0,
          "socketTimeout": 0
        },
        "write": {
          "maxRetries": 3,
          "totalTimeout": 0,
          "socketTimeout": 0
        },
        "info": {
          "timeout": 10000
        }
      },
      "tls": {
        "enable": true,
        "cafile": "./certs/ca.crt",
        "keyfile": "./certs/server.key",
        "certfile": "./certs/server.crt"
      },
      "hosts": [
        {
          "addr": "------.com",
          "port": 4333,
          "tlsname": "------"
        },
         {
          "addr": "------.com",
          "port": 4333,
          "tlsname": "------"
        },
         {
          "addr": "------.com",
          "port": 4333,
          "tlsname": "------"
        }
      ]
    }

I am then writing to a set in this namespace from my application using

function writeToCache( bins, meta) {
const key = new Aerospike.Key(namespace, set, packageName);       
let meta = {
             ttl: 3600
         }
       const writePolicy = const writePolicy = new Aerospike.WritePolicy({
    maxRetries: 3,
    socketTimeout: 0,
    totalTimeout: 0,
    exists: Aerospike.policy.exists.CREATE_OR_UPDATE,
    key: Aerospike.policy.key.SEND
});
       await asc.put(key, bins, meta, writePolicy);
}

I have anywhere between 12-15 bins in a record and the size of one record is anywhere between 300 Kb to 1.5 Mb.

Once in a while I am getting this error while doing this:

Aerospike Error: {"params":{"namespace":"application_cache","set":"apps","packageName":"com.futureplay.battleground"},"err":{"code":-6,"message":"TLS write failed: -2 34D53CA5C4E0734F 10.57.49.180:4333"}}

The bigger the size of the record, the higher the chance of this occuring. According to this Any record in memory should be able to have 128MiB of memory. Any idea what this issue could be cased by?

2

Crossposted here also: node.js - Facing TLS write failed: -2 issue while writing to aerospike - Stack Overflow

3

Hi @Ansh1207 looking at the server error codes a code: 6 is AS_ERR_BIN_EXISTS. Your write policy is setup with exists: Aerospike.policy.exists.CREATE_OR_UPDATE but the options for exists are:

Name Description
IGNORE Write the record, regardless of existence. (I.e. create or update.)
CREATE Create a record, ONLY if it doesn’t exist.
UPDATE Update a record, ONLY if it exists.
REPLACE Completely replace a record, ONLY if it exists.
CREATE_OR_REPLACE Completely replace a record if it exists, otherwise create it.

My guess is that it’s somehow defaulting to a CREATE only causing it to throw that error. Could you update your write policy to use exists: Aerospike.policy.exists.IGNORE and let us know if you’re still seeing that error?

4

Hi @aanderson. I have tried using both the IGNORE and CREATE_OR_REPLACE policy but it is giving the same issue under bigger payloads.

5

Hi @Ansh1207, the server logs from before and after one of these events may be able to paint a clearer picture of what is happening here. Feel free to post those and we could dig a little deeper.

Since you are using Aerospike EE, it may be worth opening up a case with support.