Okay so now it seems that our hosting company gave us an stable environment, but we still can’t get the performance above 5k Writes per second.
We got 3 frontend servers, which processes logfiles and writes to Aerospike. These 3 servers are not maxed out on performance, and also the aerospike servers doesn’t looks to be maxed out on all other parameters but IOWait, which are at ~75% ?
So I was looking for what IOWait actually means and how it is related to Aerospike? Could our performance issue be hiding here?
# Aerospike database configuration file.
service {
user root
group root
paxos-single-replica-limit 1 # Number of nodes where the replica count is automatically reduced to 1.
pidfile /var/run/aerospike/asd.pid
service-threads 4
transaction-queues 4
transaction-threads-per-queue 4
proto-fd-max 15000
}
logging {
# Log file must be an absolute path.
file /var/log/aerospike/aerospike.log {
context any info
}
}
network {
service {
address any
port 3000
}
heartbeat {
mode multicast
address 239.1.99.222
port 9918
# To use unicast-mesh heartbeats, remove the 3 lines above, and see
# aerospike_mesh.conf for alternative.
interval 150
timeout 10
}
fabric {
port 3001
}
info {
port 3003
}
}
namespace audience {
replication-factor 2
memory-size 56G
default-ttl 0 # 30 days, use 0 to never expire/evict.
# storage-engine memory
# To use file storage backing, comment out the line above and use the
# following lines instead.
storage-engine device {
device /dev/sdb
# filesize 16G
# data-in-memory true # Store data in memory in addition to file.
}
}