Benchmarks errors on node.js OS X (vagrant)

Client Libraries Node.js Client
1

Hey, I’m running the latest nodejs benchmarks on macbook pro (vagrant - storage-engine memory) and getting lots of errors (see below) - especially at the beginning of test. Is this expected? Should I change the configs for this environment?

Aerospike server version: Aerospike Community Edition build 3.10.0.3 Aerospike client version: aerospike@2.4.4 Node version: v7.2.1

➜  benchmarks git:(master) ✗ node main.js
info: namespace: test, set: demo, worker processes: 4, keys: 100000, read: 50%, write: 50%
info: worker connected:  localhost:3000
info: worker connected:  localhost:3000
info: worker connected:  localhost:3000
info: worker connected:  localhost:3000
info: Tue Dec 20 2016 14:46:35 GMT+0100 (CET) read(tps=3890 timeouts=0 errors=87) write(tps=3888 timeouts=0 errors=75)
info: Tue Dec 20 2016 14:46:36 GMT+0100 (CET) read(tps=6123 timeouts=0 errors=4) write(tps=6124 timeouts=0 errors=0)
info: Tue Dec 20 2016 14:46:37 GMT+0100 (CET) read(tps=5736 timeouts=0 errors=92) write(tps=5744 timeouts=0 errors=93)
...
SUMMARY

  Environment
    Node.js Version       7.2.1
    UV_THREADPOOL_SIZE    -

  Configuration
    operations    100
    iterations    undefined
    processes     4
    time          24.00 hours

  Transactions
               total      read       write      query    scan
    Total      221,092    110,548    110,544    0        0
    TPS        12,850     6,425      6,425      0        0
    Min TPS    7,778      3,890      3,888      0        0
    Max TPS    14,955     7,477      7,478      0        0

  Durations
    <= 1    > 1      > 2      > 4      > 8      > 16     > 32
    4.1%    13.7%    26.5%    20.5%    21.3%    13.2%    0.6%

  Status Codes
    0        2       -6
    99.6%    0.0%    0.3%

Similarly in the nodejs examples range_put, range_remove, etc,…:

➜  examples git:(master) ✗ node range_put --start 1 --end 10000
ERR -  { AerospikeError: Failed to connect: BB9A9DD73270008 127.0.0.1:3000
    at Function.AerospikeError.fromASError (/Users/sasa/development/workspace/testing/aerospike-client-nodejs/examples/node_modules/aerospike/lib/aerospike_error.js:113:10)
    at Client.DefaultCallbackHandler [as callbackHandler] (/Users/sasa/development/workspace/testing/aerospike-client-nodejs/examples/node_modules/aerospike/lib/client.js:71:72)
    at putCb (/Users/sasa/development/workspace/testing/aerospike-client-nodejs/examples/node_modules/aerospike/lib/client.js:1328:10)
    at Function.module.exports.loopWhile (/Users/sasa/development/workspace/testing/aerospike-client-nodejs/node_modules/deasync/index.js:72:22)
    at putStart (/Users/sasa/development/workspace/testing/aerospike-client-nodejs/examples/range_put.js:209:15)
    at /Users/sasa/development/workspace/testing/aerospike-client-nodejs/examples/range_put.js:214:3
    at Client.DefaultCallbackHandler [as callbackHandler] (/Users/sasa/development/workspace/testing/aerospike-client-nodejs/examples/node_modules/aerospike/lib/client.js:74:12)
    at connectCb (/Users/sasa/development/workspace/testing/aerospike-client-nodejs/examples/node_modules/aerospike/lib/client.js:427:10)
    at Client.connect (/Users/sasa/development/workspace/testing/aerospike-client-nodejs/examples/node_modules/aerospike/lib/client.js:420:18)
    at Aerospike.connect (/Users/sasa/development/workspace/testing/aerospike-client-nodejs/examples/node_modules/aerospike/lib/aerospike.js:154:10)
  name: 'AerospikeError',
  message: 'Failed to connect: BB9A9DD73270008 127.0.0.1:3000',
  code: -6,
  func: 'as_uv_connected',
  file: 'src/main/aerospike/as_event_uv.c',
  line: 572 } undefined

...

RANGE: start=1 end=10000 skip=0
RESULTS: (10000 completed, 9989 success, 11 failed, 0 skipped)

I’ve got better results (no errors) when I reduced the constant maxConcurrent significantly (from 200 to 20).

Thanks, Sasa

2

AerospikeError: Failed to connect: BB9A9DD73270008 127.0.0.1:3000 – looks like you did not connect to the Aerospike server.???

3

In general the connection to the server looks fine. Those errors occur only at the beginning of test and later somehow self-stabilizes. See the summary at the end - 11 failed out of 10000:

RANGE: start=1 end=10000 skip=0
RESULTS: (10000 completed, 9989 success, 11 failed, 0 skipped)
4

The failed transactions should be mostly (all?) record-not-found errors. The benchmark writes records and then immediately reads them back without waiting for the write to succeed. These errors only occur at the beginning of the benchmark because after a short while all the possible keys have been created and therefore the record-not-found errors no longer occur.

5

Hey Jan, thanks for reply! I understand why record-not-found errors occur. But there are errors (status code: -6) too, see stats below. They disappear after some iterations.

I’m a bit concerned about these errors… My use case is to implement removing of multiple records (>1000) at once (in a loop). Similar to what is done in range_remove.js in node.js examples, which also produces some errors (~1…3%): ‘Failed to connect: BB9A9DD73270008 127.0.0.1:3000’, code: -6 Do you suggest other approach?

➜  benchmarks git:(master) ✗ node main.js
info: namespace: test, set: demo, worker processes: 4, keys: 100000, read: 50%, write: 50%
info: worker connected:  localhost:3000
info: worker connected:  localhost:3000
info: worker connected:  localhost:3000
info: worker connected:  localhost:3000
info: Thu Dec 22 2016 10:15:29 GMT+0100 (CET) read(tps=180 timeouts=0 errors=115) write(tps=180 timeouts=0 errors=117)
info: Thu Dec 22 2016 10:15:30 GMT+0100 (CET) read(tps=0 timeouts=0 errors=0) write(tps=0 timeouts=0 errors=0)
info: Thu Dec 22 2016 10:15:31 GMT+0100 (CET) read(tps=210 timeouts=0 errors=90) write(tps=202 timeouts=0 errors=83)
info: Thu Dec 22 2016 10:15:32 GMT+0100 (CET) read(tps=53 timeouts=0 errors=10) write(tps=59 timeouts=0 errors=15)
info: Thu Dec 22 2016 10:15:33 GMT+0100 (CET) read(tps=4080 timeouts=0 errors=55) write(tps=4086 timeouts=0 errors=58)
info: Thu Dec 22 2016 10:15:34 GMT+0100 (CET) read(tps=5693 timeouts=0 errors=0) write(tps=5691 timeouts=0 errors=0)
info: Thu Dec 22 2016 10:15:35 GMT+0100 (CET) read(tps=5174 timeouts=0 errors=0) write(tps=5173 timeouts=0 errors=0)
info: Thu Dec 22 2016 10:15:36 GMT+0100 (CET) read(tps=4876 timeouts=0 errors=0) write(tps=4876 timeouts=0 errors=0)
info: Thu Dec 22 2016 10:15:37 GMT+0100 (CET) read(tps=4875 timeouts=0 errors=0) write(tps=4877 timeouts=0 errors=0)
...
...
...

  Status Codes
    0        2       -6
    99.6%    0.0%    0.3%

Btw: I also ran Java client benchmarks (asynch) and got better results: only few errors on 1st iteration and no errors after.