Hi, Since i has working on project where i have to evaluate Aerospike vs Redis performance.
Redis result:-
shikhil.g$ redis-benchmark -c 50 -t set -d 8 -n 1000000
====== SET ======
1000000 requests completed in 12.69 seconds
50 parallel clients
8 bytes payload
keep alive: 1
99.98% <= 1 milliseconds
100.00% <= 2 milliseconds
100.00% <= 3 milliseconds
78796.00 requests per second
Aeropsike Result:-
IBLMDVSHIKHIL:benchmarks shikhil.g$ ./target/benchmarks -w I -c 50
hosts: 127.0.0.1
port: 3000
user: (null)
namespace: test
set: testset
startKey: 1
keys/records: 1000000
bins: 1
object spec: int
random values: false
workload: initialize 100% of records
threads: 16
max throughput: unlimited
read timeout: 0 ms
write timeout: 0 ms
max retries: 1
debug: false
latency: false
shared memory: false
read replica: sequence
read mode AP: one
read mode SC: session
write commit level: all
conn pools per node: 1
asynchronous mode: off
auth mode: INTERNAL
2019-05-22 18:10:08 INFO Add node BB9020011AC4202 127.0.0.1:3000
2019-05-22 18:10:08 INFO Initialize 1000000 records
2019-05-22 18:10:08 INFO Start 16 generator threads
2019-05-22 18:10:09 INFO write(tps=19721 timeouts=0 errors=0 total=19800)
2019-05-22 18:10:10 INFO write(tps=19014 timeouts=0 errors=0 total=38890)
2019-05-22 18:10:11 INFO write(tps=19064 timeouts=0 errors=0 total=57954)
2019-05-22 18:10:12 INFO write(tps=17921 timeouts=0 errors=0 total=75875)
2019-05-22 18:10:13 INFO write(tps=19735 timeouts=0 errors=0 total=95649)
2019-05-22 18:10:14 INFO write(tps=19753 timeouts=0 errors=0 total=115442)
2019-05-22 18:10:15 INFO write(tps=18138 timeouts=0 errors=0 total=133634)
2019-05-22 18:10:16 INFO write(tps=18051 timeouts=0 errors=0 total=151685)
2019-05-22 18:10:17 INFO write(tps=17540 timeouts=0 errors=0 total=169225)
2019-05-22 18:10:18 INFO write(tps=18484 timeouts=0 errors=0 total=187764)
2019-05-22 18:10:19 INFO write(tps=17856 timeouts=0 errors=0 total=205656)
2019-05-22 18:10:20 INFO write(tps=16875 timeouts=0 errors=0 total=222582)
2019-05-22 18:10:21 INFO write(tps=16739 timeouts=0 errors=0 total=239354)
2019-05-22 18:10:22 INFO write(tps=17353 timeouts=0 errors=0 total=256724)
2019-05-22 18:10:23 INFO write(tps=17458 timeouts=0 errors=0 total=274252)
2019-05-22 18:10:24 INFO write(tps=17211 timeouts=0 errors=0 total=291515)
2019-05-22 18:10:25 INFO write(tps=15501 timeouts=0 errors=0 total=307016)
2019-05-22 18:10:26 INFO write(tps=16161 timeouts=0 errors=0 total=323177)
2019-05-22 18:10:27 INFO write(tps=15492 timeouts=0 errors=0 total=338700)
2019-05-22 18:10:28 INFO write(tps=16870 timeouts=0 errors=0 total=355621)
2019-05-22 18:10:29 INFO write(tps=15577 timeouts=0 errors=0 total=371229)
2019-05-22 18:10:30 INFO write(tps=15805 timeouts=0 errors=0 total=387050)
2019-05-22 18:10:31 INFO write(tps=16153 timeouts=0 errors=0 total=403203)
2019-05-22 18:10:32 INFO write(tps=16376 timeouts=0 errors=0 total=419628)
2019-05-22 18:10:33 INFO write(tps=16248 timeouts=0 errors=0 total=435892)
2019-05-22 18:10:34 INFO write(tps=13572 timeouts=0 errors=0 total=449518)
2019-05-22 18:10:35 INFO write(tps=11857 timeouts=0 errors=0 total=461399)
2019-05-22 18:10:36 INFO write(tps=15463 timeouts=0 errors=0 total=476862)
2019-05-22 18:10:37 INFO write(tps=16165 timeouts=0 errors=0 total=493027)
2019-05-22 18:10:38 INFO write(tps=17239 timeouts=0 errors=0 total=510266)
2019-05-22 18:10:39 INFO write(tps=16203 timeouts=0 errors=0 total=526469)
2019-05-22 18:10:40 INFO write(tps=15357 timeouts=0 errors=0 total=541887)
2019-05-22 18:10:41 INFO write(tps=15249 timeouts=0 errors=0 total=557166)
2019-05-22 18:10:42 INFO write(tps=15269 timeouts=0 errors=0 total=572466)
2019-05-22 18:10:43 INFO write(tps=14109 timeouts=0 errors=0 total=586589)
2019-05-22 18:10:44 INFO write(tps=14365 timeouts=0 errors=0 total=600954)
2019-05-22 18:10:45 INFO write(tps=14464 timeouts=0 errors=0 total=615418)
2019-05-22 18:10:46 INFO write(tps=14228 timeouts=0 errors=0 total=629703)
2019-05-22 18:10:47 INFO write(tps=13870 timeouts=0 errors=0 total=643573)
2019-05-22 18:10:48 INFO write(tps=13718 timeouts=0 errors=0 total=657291)
2019-05-22 18:10:49 INFO write(tps=15088 timeouts=0 errors=0 total=672379)
2019-05-22 18:10:50 INFO write(tps=14810 timeouts=0 errors=0 total=687189)
2019-05-22 18:10:51 INFO write(tps=14679 timeouts=0 errors=0 total=701897)
2019-05-22 18:10:52 INFO write(tps=15200 timeouts=0 errors=0 total=717097)
2019-05-22 18:10:53 INFO write(tps=14190 timeouts=0 errors=0 total=731287)
2019-05-22 18:10:54 INFO write(tps=15042 timeouts=0 errors=0 total=746359)
2019-05-22 18:10:55 INFO write(tps=14680 timeouts=0 errors=0 total=761083)
2019-05-22 18:10:56 INFO write(tps=12240 timeouts=0 errors=0 total=773360)
2019-05-22 18:10:57 INFO write(tps=11572 timeouts=0 errors=0 total=784944)
2019-05-22 18:10:58 INFO write(tps=13169 timeouts=0 errors=0 total=798166)
2019-05-22 18:10:59 INFO write(tps=13376 timeouts=0 errors=0 total=811596)
2019-05-22 18:11:00 INFO write(tps=14045 timeouts=0 errors=0 total=825669)
2019-05-22 18:11:01 INFO write(tps=13945 timeouts=0 errors=0 total=839614)
2019-05-22 18:11:02 INFO write(tps=14053 timeouts=0 errors=0 total=853667)
2019-05-22 18:11:03 INFO write(tps=13914 timeouts=0 errors=0 total=867609)
2019-05-22 18:11:04 INFO write(tps=14177 timeouts=0 errors=0 total=881829)
2019-05-22 18:11:05 INFO write(tps=13930 timeouts=0 errors=0 total=895787)
2019-05-22 18:11:06 INFO write(tps=14543 timeouts=0 errors=0 total=910388)
2019-05-22 18:11:07 INFO write(tps=14825 timeouts=0 errors=0 total=925257)
2019-05-22 18:11:08 INFO write(tps=15422 timeouts=0 errors=0 total=940710)
2019-05-22 18:11:09 INFO write(tps=15812 timeouts=0 errors=0 total=956538)
2019-05-22 18:11:10 INFO write(tps=14581 timeouts=0 errors=0 total=971148)
2019-05-22 18:11:11 INFO write(tps=15164 timeouts=0 errors=0 total=986373)
2019-05-22 18:11:12 INFO write(tps=12157 timeouts=0 errors=0 total=998530)
2019-05-22 18:11:13 INFO write(tps=1467 timeouts=0 errors=0 total=1000000)
I can see aerospike is giving me average 12000 to 15000 thousand transaction per second and redis average value is 78796.00 per second.
I believe aerospike gives better performance compare to redis. So do i am missing here any thing while running benchmark tool?
You have configured 50 client threads on Redis and 16 for Aerospike. Threads are configured with the ā-zā option but are irrelevant for async tests and you would use:-Y,--connPoolsPerNode <arg> Number of synchronous connection pools per node. Default 1.
You are setting maxAsyncCommands to 50 but this is irrelevant when not running the client in async mode.-a,--async Benchmark asynchronous methods instead of synchronous methods.
You are using a set-name which adds a few bytes per transaction which arenāt present in the Redis benchmark. I donāt see a way to totally disable the set-name parameter, but you can minimize its impact by setting it to a 1 byte set name.-s,--set <arg> Set the Aerospike set name. Default: testset
You are running the benchmark and the server from the same machine. You will get better results if the client and server arenāt completing for the same resources.
Benchmarks settings and examples can be found at https://www.aerospike.com/docs/client/java/benchmarks.html .
Additionally, if you want to monitor the latency of these transactions use the -latency option.
Since I donāt know how many cores your client has access to I will assume 16 cores:
So the command would be:
./target/benchmarks -latency 7,1 -s 's' -w I -a -z 1 -y 16 -c 640
Thanks for quick reply. I am having Mac book pro having 12 virtual core. I have tried as you have suggessted
IBLMDVSHIKHIL:benchmarks shikhil.g$ ./target/benchmarks -s 's' -w I -a -z 1 -Y 12 -c 480
hosts: 127.0.0.1
port: 3000
user: (null)
namespace: test
set: s
startKey: 1
keys/records: 1000000
bins: 1
object spec: int
random values: false
workload: initialize 100% of records
threads: 1
max throughput: unlimited
read timeout: 0 ms
write timeout: 0 ms
max retries: 1
debug: false
latency: false
shared memory: false
read replica: sequence
read mode AP: one
read mode SC: session
write commit level: all
conn pools per node: 12
asynchronous mode: on
async max commands: 480
event loops: 1
auth mode: INTERNAL
2019-05-22 23:33:43 INFO Add node BB9020011AC4202 127.0.0.1:3000
2019-05-22 23:33:43 INFO Initialize 1000000 records
2019-05-22 23:33:44 INFO write(tps=20336 timeouts=0 errors=46937 total=20417)
2019-05-22 23:33:45 INFO write(tps=21875 timeouts=0 errors=66121 total=42379)
2019-05-22 23:33:46 INFO write(tps=20455 timeouts=0 errors=54610 total=62895)
2019-05-22 23:33:47 INFO write(tps=21644 timeouts=0 errors=60553 total=84561)
2019-05-22 23:33:48 INFO write(tps=20614 timeouts=0 errors=66060 total=105175)
2019-05-22 23:33:49 INFO write(tps=20066 timeouts=0 errors=55010 total=125241)
2019-05-22 23:33:50 INFO write(tps=21215 timeouts=0 errors=60276 total=146498)
2019-05-22 23:33:51 INFO write(tps=19679 timeouts=0 errors=60135 total=166177)
2019-05-22 23:33:52 INFO write(tps=20680 timeouts=0 errors=67503 total=186878)
2019-05-22 23:33:53 INFO write(tps=20619 timeouts=0 errors=69225 total=207518)
2019-05-22 23:33:54 INFO write(tps=20774 timeouts=0 errors=67167 total=228375)
2019-05-22 23:33:55 INFO write(tps=20620 timeouts=0 errors=56913 total=249016)
2019-05-22 23:33:56 INFO write(tps=18984 timeouts=0 errors=61763 total=268076)
2019-05-22 23:33:57 INFO write(tps=20483 timeouts=0 errors=61154 total=288600)
2019-05-22 23:33:58 INFO write(tps=19583 timeouts=0 errors=64457 total=308242)
2019-05-22 23:33:59 INFO write(tps=19760 timeouts=0 errors=54320 total=328042)
2019-05-22 23:34:00 INFO write(tps=19412 timeouts=0 errors=66386 total=347512)
2019-05-22 23:34:01 INFO write(tps=17964 timeouts=0 errors=59704 total=365512)
2019-05-22 23:34:02 INFO write(tps=18589 timeouts=0 errors=59404 total=384194)
2019-05-22 23:34:03 INFO write(tps=19642 timeouts=0 errors=62825 total=403875)
2019-05-22 23:34:04 INFO write(tps=18979 timeouts=0 errors=61687 total=422873)
2019-05-22 23:34:05 INFO write(tps=17798 timeouts=0 errors=45688 total=440742)
2019-05-22 23:34:06 INFO write(tps=17584 timeouts=0 errors=56908 total=458361)
2019-05-22 23:34:07 INFO write(tps=18228 timeouts=0 errors=57084 total=476625)
2019-05-22 23:34:08 INFO write(tps=20446 timeouts=0 errors=59908 total=497173)
2019-05-22 23:34:09 INFO write(tps=17967 timeouts=0 errors=50736 total=515158)
2019-05-22 23:34:10 INFO write(tps=17503 timeouts=0 errors=43294 total=532696)
2019-05-22 23:34:11 INFO write(tps=17296 timeouts=0 errors=45275 total=550044)
2019-05-22 23:34:12 INFO write(tps=16970 timeouts=0 errors=53679 total=567048)
2019-05-22 23:34:13 INFO write(tps=17629 timeouts=0 errors=41122 total=584748)
2019-05-22 23:34:14 INFO write(tps=17330 timeouts=0 errors=40035 total=602095)
2019-05-22 23:34:15 INFO write(tps=18278 timeouts=0 errors=53278 total=620410)
2019-05-22 23:34:16 INFO write(tps=15054 timeouts=0 errors=41487 total=635509)
2019-05-22 23:34:17 INFO write(tps=17625 timeouts=0 errors=48308 total=653204)
2019-05-22 23:34:18 INFO write(tps=18029 timeouts=0 errors=63360 total=671233)
2019-05-22 23:34:19 INFO write(tps=17993 timeouts=0 errors=49610 total=689226)
2019-05-22 23:34:20 INFO write(tps=18788 timeouts=0 errors=53257 total=708052)
2019-05-22 23:34:21 INFO write(tps=17898 timeouts=0 errors=41481 total=725986)
2019-05-22 23:34:22 INFO write(tps=18543 timeouts=0 errors=45182 total=744622)
2019-05-22 23:34:23 INFO write(tps=19060 timeouts=0 errors=59906 total=763739)
2019-05-22 23:34:24 INFO write(tps=18148 timeouts=0 errors=59209 total=781941)
2019-05-22 23:34:25 INFO write(tps=19161 timeouts=0 errors=59881 total=801102)
2019-05-22 23:34:26 INFO write(tps=17725 timeouts=0 errors=51831 total=818862)
2019-05-22 23:34:27 INFO write(tps=18254 timeouts=0 errors=57431 total=837116)
2019-05-22 23:34:28 INFO write(tps=17619 timeouts=0 errors=56827 total=854735)
2019-05-22 23:34:29 INFO write(tps=17758 timeouts=0 errors=54997 total=872546)
2019-05-22 23:34:30 INFO write(tps=17357 timeouts=0 errors=51449 total=889903)
2019-05-22 23:34:31 INFO write(tps=14816 timeouts=0 errors=35529 total=904763)
2019-05-22 23:34:32 INFO write(tps=16834 timeouts=0 errors=41983 total=921681)
2019-05-22 23:34:33 INFO write(tps=15644 timeouts=0 errors=52528 total=937341)
2019-05-22 23:34:34 INFO write(tps=16789 timeouts=0 errors=44436 total=954130)
2019-05-22 23:34:35 INFO write(tps=16788 timeouts=0 errors=46082 total=970985)
2019-05-22 23:34:36 INFO write(tps=16810 timeouts=0 errors=46608 total=987795)
2019-05-22 23:34:37 INFO write(tps=12193 timeouts=0 errors=27690 total=1000000)
Could you share your /etc/aerospike/aerospike.conf.
Here is my /etc/aerospike/aerospike.conf .
root@89127dcae9fb:/# cat /etc/aerospike/aerospike.conf
# Aerospike database configuration file.
# This stanza must come first.
service {
user root
group root
paxos-single-replica-limit 1 # Number of nodes where the replica count is automatically reduced to 1.
pidfile /var/run/aerospike/asd.pid
service-threads 6
transaction-queues 6
transaction-threads-per-queue 4
proto-fd-max 15000
}
logging {
# Log file must be an absolute path.
file /dev/null {
context any info
}
# Send log messages to stdout
console {
context any info
}
}
network {
service {
address any
port 3000
# Uncomment the following to set the `access-address` parameter to the
# IP address of the Docker host. This will the allow the server to correctly
# publish the address which applications and other nodes in the cluster to
# use when addressing this node.
# access-address <IPADDR>
}
heartbeat {
address any
# mesh is used for environments that do not support multicast
mode mesh
port 3002
# use asinfo -v 'tip:host=<ADDR>;port=3002' to inform cluster of
# other mesh nodes
interval 150
timeout 10
}
fabric {
address any
port 3001
}
info {
address any
port 3003
}
}
namespace test {
replication-factor 2
memory-size 1G
default-ttl 30d # 5 days, use 0 to never expire/evict.
# storage-engine memory
# To use file storage backing, comment out the line above and use the
# following lines instead.
storage-engine device {
file /opt/aerospike/data/test.dat
filesize 4G
data-in-memory true # Store data in memory in addition to file.
}
}
Testing from a VM on a laptop isnāt representative and introduces more variables - such as how the VM was configured. Also file system access from VMs is more resource intensive than non-VM.
Answer:- You mean testing should be performed on VM Right?
storage-engine memory
Answer:- Can you tell me what to set here?
service-threads and transaction-queues default to the number of cores, why have you set them to 6?
Answer:- Ok, I will test with 12 Core.
Why are you logging to `/dev/null? Should just remove that context.
Answer:- I did not get this. Shall I remove it or give path to file.
Iām curious what āerrorsā the Aerospike benchmark was seeing, could you run the command with the --debug option so we may see the errors.
Answer:- OK, i will share the result with debug option enabled.
Also, I would like to thank you for answering quickly on the post
No, unless you plan to go production from your laptop, you should benchmark a realistic environment. Anecdotally, Iāve have done algorithm benchmarks during development from a laptop/VM environment just to later find that a realistic environment (actual server hardware) favored a different algorithm than what my VM favored.
If you plan to continue in Laptop/VM environment, could you provide the output of:
grep processor /proc/cpuinfo
Replace your ānamespace testā stanza with:
namespace test {
replication-factor 2
memory-size 1G
default-ttl 30d # 5 days, use 0 to never expire/evict.
storage-engine memory
# To use file storage backing, comment out the line above and use the
# following lines instead.
#storage-engine device {
# file /opt/aerospike/data/test.dat
# filesize 4G
# data-in-memory true # Store data in memory in addition to file.
#}
}
Where did it come from? Its odd, assuming you are on a VM with SystemD, the console logging stanza should be enough.
Hi Please find my aerospike file:-
IBLMDVSHIKHIL:aerospike-conf shikhil.g$ cat aerospike.conf
# Aerospike database configuration file.
service {
user root
group root
paxos-single-replica-limit 1 # Number of nodes where the replica count is automatically reduced to 1.
pidfile /var/run/aerospike/asd.pid
service-threads 12
transaction-queues 12
transaction-threads-per-queue 4
proto-fd-max 15000
}
logging {
console {
context any info
}
# Log file must be an absolute path.
#file /var/log/aerospike/aerospike.log {
# context any info
#}
}
network {
service {
address any
port 3000
}
heartbeat {
mode multicast
multicast-group 239.1.99.222
port 9918
# To use unicast-mesh heartbeats, remove the 3 lines above, and see
# aerospike_mesh.conf for alternative.
interval 150
timeout 10
}
fabric {
port 3001
}
info {
port 3003
}
}
namespace test {
replication-factor 2
memory-size 4G
default-ttl 30d # 30 days, use 0 to never expire/evict.
storage-engine memory
}
namespace bar {
replication-factor 2
memory-size 4G
default-ttl 30d # 30 days, use 0 to never expire/evict.
storage-engine memory
# To use file storage backing, comment out the line above and use the
# following lines instead.
# storage-engine device {
# file /opt/aerospike/data/bar.dat
# filesize 16G
# data-in-memory true # Store data in memory in addition to file.
# }
}
IBLMDVSHIKHIL:benchmarks shikhil.g$ sysctl -a | grep machdep.cpu | grep core_count
machdep.cpu.core_count: 6
IBLMDVSHIKHIL:benchmarks shikhil.g$ sysctl -a | grep machdep.cpu | grep thread_count
'machdep.cpu.thread_count: 12
I ran this command to start aerospike server using docker:-
docker run -tid -v /Users/shikhil.g/dev/githome/release/aerospike-conf:/opt/aerospike/etc --name aerospike -p 3000:3000 -p 3001:3001 -p 3002:3002 -p 3003:3003 aerospike/aerospike-server /usr/bin/asd --foreground --config-file /opt/aerospike/etc/aerospike.conf
Test performed result:-
IBLMDVSHIKHIL:benchmarks shikhil.g$ ./target/benchmarks -s 's' --debug -w I -a -z 1 -Y 12 -c 480
hosts: 127.0.0.1
port: 3000
user: (null)
namespace: test
set: s
startKey: 1
keys/records: 1000000
bins: 1
object spec: int
random values: false
workload: initialize 100% of records
threads: 1
max throughput: unlimited
read timeout: 0 ms
write timeout: 0 ms
max retries: 1
debug: true
latency: false
shared memory: false
read replica: sequence
read mode AP: one
read mode SC: session
write commit level: all
conn pools per node: 12
asynchronous mode: on
async max commands: 480
event loops: 1
auth mode: INTERNAL
2019-05-27 19:08:09 INFO Add node BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 DEBUG Node BB9020011AC4202 peers generation changed: 0
2019-05-27 19:08:09 DEBUG Node BB9020011AC4202 partition generation changed: 0
2019-05-27 19:08:09 INFO Initialize 1000000 records
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=516745 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=518828 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=520911 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=522994 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=525077 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=527160 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=529243 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=531326 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=533409 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=535492 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=537575 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=539658 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=541741 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=543824 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=545907 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=547990 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=550073 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=552156 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=554239 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=556322 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=558405 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=560488 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=562571 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=564654 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=566737 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=568820 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=570903 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=572986 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=575069 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=577152 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=579235 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=581318 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=583401 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=585484 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=587567 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=589650 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=591733 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=593816 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=595899 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=597982 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=600065 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=602148 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=604231 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=606314 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=608397 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=610480 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=612563 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=614646 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=616729 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=618812 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=620895 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=622978 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=625061 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=627144 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=629227 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=631310 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=633393 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=635476 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=637559 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
2019-05-27 19:08:09 ERROR Write error: ns=test set=s key=639642 bin=testbin code=-6 message=Connect failed: -1 BB9020011AC4202 127.0.0.1:3000
Please find here details info of my system:-
IBLMDVSHIKHIL:benchmarks shikhil.g$ sysctl -a | grep machdep.cpu
machdep.cpu.max_basic: 22
machdep.cpu.max_ext: 2147483656
machdep.cpu.vendor: GenuineIntel
machdep.cpu.brand_string: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
machdep.cpu.family: 6
machdep.cpu.model: 158
machdep.cpu.extmodel: 9
machdep.cpu.extfamily: 0
machdep.cpu.stepping: 10
machdep.cpu.feature_bits: 9221959987971750911
machdep.cpu.leaf7_feature_bits: 43804591
machdep.cpu.extfeature_bits: 1241984796928
machdep.cpu.signature: 591594
machdep.cpu.brand: 0
machdep.cpu.features: FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR PGE MCA CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE SSE3 PCLMULQDQ DTES64 MON DSCPL VMX EST TM2 SSSE3 FMA CX16 TPR PDCM SSE4.1 SSE4.2 x2APIC MOVBE POPCNT AES PCID XSAVE OSXSAVE SEGLIM64 TSCTMR AVX1.0 RDRAND F16C
machdep.cpu.leaf7_features: SMEP ERMS RDWRFSGS TSC_THREAD_OFFSET BMI1 AVX2 BMI2 INVPCID SMAP RDSEED ADX IPT SGX FPU_CSDS MPX CLFSOPT
machdep.cpu.extfeatures: SYSCALL XD 1GBPAGE EM64T LAHF LZCNT PREFETCHW RDTSCP TSCI
machdep.cpu.logical_per_package: 16
machdep.cpu.cores_per_package: 8
machdep.cpu.microcode_version: 150
machdep.cpu.processor_flag: 5
machdep.cpu.mwait.linesize_min: 64
machdep.cpu.mwait.linesize_max: 64
machdep.cpu.mwait.extensions: 3
machdep.cpu.mwait.sub_Cstates: 286531872
machdep.cpu.thermal.sensor: 1
machdep.cpu.thermal.dynamic_acceleration: 1
machdep.cpu.thermal.invariant_APIC_timer: 1
machdep.cpu.thermal.thresholds: 2
machdep.cpu.thermal.ACNT_MCNT: 1
machdep.cpu.thermal.core_power_limits: 1
machdep.cpu.thermal.fine_grain_clock_mod: 1
machdep.cpu.thermal.package_thermal_intr: 1
machdep.cpu.thermal.hardware_feedback: 0
machdep.cpu.thermal.energy_policy: 1
machdep.cpu.xsave.extended_state: 31 832 1088 0
machdep.cpu.xsave.extended_state1: 15 832 256 0
machdep.cpu.arch_perf.version: 4
machdep.cpu.arch_perf.number: 4
machdep.cpu.arch_perf.width: 48
machdep.cpu.arch_perf.events_number: 7
machdep.cpu.arch_perf.events: 0
machdep.cpu.arch_perf.fixed_number: 3
machdep.cpu.arch_perf.fixed_width: 48
machdep.cpu.cache.linesize: 64
machdep.cpu.cache.L2_associativity: 4
machdep.cpu.cache.size: 256
machdep.cpu.tlb.inst.large: 8
machdep.cpu.tlb.data.small: 64
machdep.cpu.tlb.data.small_level1: 64
machdep.cpu.address_bits.physical: 39
machdep.cpu.address_bits.virtual: 48
machdep.cpu.core_count: 6
machdep.cpu.thread_count: 12
machdep.cpu.tsc_ccc.numerator: 184
machdep.cpu.tsc_ccc.denominator: 2
IBLMDVSHIKHIL:benchmarks shikhil.g$
Unless there was more output, it appears that the client couldnāt connect to the server. Did you run the server with docker in the prior runs? Could you remove the ādockerā variable from this environment?
I am running aerospike server as docker container, what do you mean by removing docker variable from environment?
You are testing the server running in a Docker container in a VM on your laptop. For some reason the client cannot connect the the server in this environment, so Iām suggesting to try without Docker (i.e. removing the ādockerā variable). You can then confirm whether or not the simpler environment works. Once the simplified environment works, you may re-complect your environment with docker, but you will also need to determine why the client is unable to connect to the server behind docker.
Where are you running the client from? Is it running from the same docker container? from the host vm? or from another container?
lucien
May 30, 2019, 1:47am
16
Does the issue persist if you run docker with the host network (ie: --network host) ?
docker run -tid --network host --name aerospike -p 3000:3000 -p 3001:3001 -p 3002:3002 -p 3003:3003 aerospike/aerospike-server /usr/bin/asd --foreground
If above works, please also confirm connection by running the following:
docker exec -ti aerospike bash -c "asadm -e info"
docker exec -ti aerospike bash -c "asadm -e 'asinfo -v service'"
Also checkout this article on the error code -6 that you seem to be getting from the async mode. Try reducing the max async command.
Error code -6: ? Error Codes | Aerospike Documentation
For future write requests which specify 'BIN_CREATE_ONLY', request fail because any of the bin already exists. Are you running into that in your application code?
Shikhil,
It seems your issues would be better addressed by a conference call or gotomeeting. Please contact me so that we could perhaps set up a meeting sometime tomorrow at your convenience.
Bruce Parker
bparker@aerospike.com
Hi Bruce,
it would be great if we can setup meeting over hangout call so that i can quickly test complete performance for our use cases.
I work in IST timings. so when you said to setup a meeting tomorrow, do you mean friday or Saturday because today is friday in india and tomorrow is Saturday.
I think we cant run docker container with param --network host
IBLMDVSHIKHIL:benchmarks shikhil.g$ docker run -tid --network host --name aerospike -p 3000:3000 -p 3001:3001 -p 3002:3002 -p 3003:3003 aerospike/aerospike-server /usr/bin/asd --foreground
WARNING: Published ports are discarded when using host network mode
9ce6c2e20c65e0f9a56905c7a9e1a28b531b4e6d790a6d3a0d4856d53cc77c34
@Shikhil_Gupta can you please send me an email on dileep@aerospike.com with your contact details so that we can take this forward?
lucien
June 3, 2019, 5:56pm
21
ānetwork host should work if only one container in use for development. No need for published ports in that case since the container would be uisng the same interface and IP address as the host. Only suggested for development environments.
