# Aerospike Benchmark POC

**URL:** <https://discuss.aerospike.com/t/aerospike-benchmark-poc/2229>\
**Category:** Aerospike Server Benchmarks\
**Created:** [December 7, 2015, 10:57am UTC](https://discuss.aerospike.com/t/aerospike-benchmark-poc/2229 "2015-12-07T10:57:56Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![phanimv](https://avatars.discourse-cdn.com/v4/letter/p/958977/32.png) [@phanimv](https://discuss.aerospike.com/u/phanimv)\
**Post date:** [December 7, 2015, 10:57am UTC](https://discuss.aerospike.com/t/aerospike-benchmark-poc/2229/1 "2015-12-07T10:57:56Z")

</div>

Continuing the discussion from [How to test and benchmark Aerospike (vs. Couchbase)](https://discuss.aerospike.com/t/how-to-test-and-benchmark-aerospike-vs-couchbase/2114/2):

> [@How to test and benchmark Aerospike (vs. Couchbase)](https://discuss.aerospike.com/t/how-to-test-and-benchmark-aerospike-vs-couchbase/2114/2):
>
> I am not very familiar with Couchbase, but I can give you pointers on to benchmark against Aerospike (I work there). The YCSB (Yahoo! Cloud System Benchmark) would probably be the best tool to use against both. [Here is](https://github.com/aerospike/ycsb) the code to use against Aerospike.
> 
> Now for specifically benchmarking Aerospike, I would recommend using one of Aerospike’s benchmark tools - for example the [Java benchmark tool](http://www.aerospike.com/docs/client/java/benchmarks.html) - and use a String (-o S:) or a Java blob (-o B:) with a length comparable to your json document.
> 
> For example:
> 
> ```
> ./run_benchmarks -h 127.0.0.1 -p 3000 -n test -k 10000000 -b 1 -o B:1400 -w RU,80 -z 8
> 
> ```
> 
> This would run a workload of 80% reads and 20% writes (`-w RU,80`) and using 8 concurrent threads (`-z 8`).
> 
> Some important points to consider, though:
> 
> - Ideally, this should run against a 2+ node cluster (so you do check with replication factor 2).
> - You will probably need more than 1 client host as you will likely hit bottlenecks on the client side before saturating the cluster.
> - You will want to find out the optimal number of threads for your workload / object size. Typically, this is between 60 and 100.
> 
> Finally, I recommend that you read some of the [published documentation](http://www.aerospike.com/category/benchmarks/) related to benchmarking Aerospike and other NoSQL databases, particularly, [this one](http://www.aerospike.com/blog/aerospike-doubles-in-memory-nosql-database-performance/).
> 
> Hope this helps!

Hi, This post is related to what am doing, I have setup Aerospike and configured c and Java clients, but am not sure how to run benchmark tests against the DBs. First I didn’t understand what are the two different steps - load data and stress / performance test against the namespace using the AS clients.

How to get to know what is the exact performance being delivered - because AMC dashboard has Total TPS and successful TPS, which one should I consider,

Otherwise simply to try I ran the command shown in the benchmark example page, which ran for 15 mins and stopped with the below error:

“`2015-12-07 16:18:37 ERROR Server is currently in readonly mode. Shutting down...`” I couldn’t understand why? all of a sudden?

Same time I wanted to use NVMe SSD, which it was not allowing, when i insert a new namespace with the ssd device settings provided as below:

```auto
storage-engine device {
           device /dev/nvme0n1
            write-block-size 128K
            scheduler-mode noop
        }

```

DB doesn’t start properly, the cake message doesn’t appear in log file.

I badly need some help.

thanks

---

<div class="post-metadata">

**Author:** ![kporter](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/kporter/32/515_2.png) [@kporter](https://discuss.aerospike.com/u/kporter)\
**Post date:** [December 7, 2015, 8:19pm UTC](https://discuss.aerospike.com/t/aerospike-benchmark-poc/2229/2 "2015-12-07T20:19:31Z")

</div>

> [@phanimv](#):
>
> “2015-12-07 16:18:37 ERROR Server is currently in readonly mode. Shutting down…”

When you get this message, could you run `adadm -e "info"` and provide the output?

---

<div class="post-metadata">

**Author:** ![tibors](https://avatars.discourse-cdn.com/v4/letter/t/dbc845/32.png) [@tibors](https://discuss.aerospike.com/u/tibors)\
**Post date:** [December 8, 2015, 1:25am UTC](https://discuss.aerospike.com/t/aerospike-benchmark-poc/2229/3 "2015-12-08T01:25:44Z")

</div>

Before running a benchmark, data should be loaded into the database so that it can be read back during the load generation of the benchmark.

The following command will load 10 Million Objects into the database. Each object is 1400 bytes. The run\_benchmark tool will use 8 threads to insert the data.

```
./run_benchmarks -h 127.0.0.1 -p 3000 -n test -k 10000000 -b 1 -o B:1400 -w I -z 8

```

Running the load. The following command will run 80% reads and 20% writes against the database loaded with the previous command.

```
./run_benchmarks -h 127.0.0.1 -p 3000 -n test -k 10000000 -b 1 -o B:1400 -w RU,80 -z 8

```

**There are different ways to get performance data from the tools and the database.**

The AMC dashboard provides totals and the number of reads and writes per second. If you want more details on performance there are other tools that can be used as well.

The asadm tool can display some summaries of reads and write performance and latencies per node using the “show latency”. [http://www.aerospike.com/docs/tools/asadm/](http://www.aerospike.com/docs/tools/asadm/)

```
asadm -e "show latency"

```

The asloglatency tool can be used to provide performance histograms from the Aerospike log file on each node of a cluster. Below is an example use of the tool. [http://www.aerospike.com/docs/tools/asloglatency/](http://www.aerospike.com/docs/tools/asloglatency/)

```
 For write performance

asloglatency –t 10 –n 7 –e 1 –h writes_master  

For read performance

asloglatency –t 10 –n 7 –e 1 –h reads

```

The run\_benchmark also has an option to display a histogram data with the -latency option. These results will be from the client perspective.

**The error you are experiencing may occur because the database has reached a stop write limit.**

“2015-12-07 16:18:37 ERROR Server is currently in readonly mode. Shutting down…” I couldn’t understand why?

Use

```
asadm –e “info” 

```

to provide more details.

**Storage configuration:**

The storage-engine configuration looks fine. The aersopike.log may provide more details of the problem. It is also good to zero out your drive before using the drive.

```
# For devices that do not support TRIM:
dd of=/dev/zero if=/dev/nvme0n1 bs=1M

# For TRIM supported devices:
blkdiscard /dev/nvme0n1

```

---

<div class="post-metadata">

**Author:** ![phanimv](https://avatars.discourse-cdn.com/v4/letter/p/958977/32.png) [@phanimv](https://discuss.aerospike.com/u/phanimv)\
**Post date:** [December 8, 2015, 4:52am UTC](https://discuss.aerospike.com/t/aerospike-benchmark-poc/2229/4 "2015-12-08T04:52:45Z")

</div>

I shall do the things recommended and get back to you with more information from asadm

---

<div class="post-metadata">

**Author:** ![phanimv](https://avatars.discourse-cdn.com/v4/letter/p/958977/32.png) [@phanimv](https://discuss.aerospike.com/u/phanimv)\
**Post date:** [December 9, 2015, 3:03am UTC](https://discuss.aerospike.com/t/aerospike-benchmark-poc/2229/5 "2015-12-09T03:03:05Z")

</div>

Hi,

I tried to load data using the run\_benchmarks as suggested, for testing purpose initially I have selected file based which is in /opt/aerospike/data/ I used the below command:

./run\_benchmarks -h 127.0.0.1 -p 3000 -n asdb -k 6000000 -b 1 -o B:1400 -w I -z 8

the tool writes / inserts data to some extent, after which it doesnt complete the job, it kept on looping, at 2015-12-09 08:29:07.649 write(count=5483105 tps=0 timeouts=0 errors=0)

the same message comes after every second, when I checked the size of the data file, that is also not growing, it remained constant at 7.9GB,

my conf file content for the specific db are:

namespace asdb { replication-factor 1 memory-size 128G default-ttl 30d # 30 days, use 0 to never expire/evict.

```
    #storage-engine memory
    storage-engine device {
        file /opt/aerospike/data/asdb.dat
        filesize 60G
    }

```

}

I couldnt understand the issue, is this a bug ?

thanks

---

<div class="post-metadata">

**Author:** ![phanimv](https://avatars.discourse-cdn.com/v4/letter/p/958977/32.png) [@phanimv](https://discuss.aerospike.com/u/phanimv)\
**Post date:** [December 9, 2015, 1:03pm UTC](https://discuss.aerospike.com/t/aerospike-benchmark-poc/2229/6 "2015-12-09T13:03:04Z")

</div>

Hi,

I tried loading the data second and it was successful, still couldn’t understand why the tool hangs and repeats on the last step in case of any error?

Also I could see run\_benchmarks in java client only, is this not available with C client? One other thing which I wanted to check if there is any pre-built / compiled Java client available for download which doesn’t need Maven build / compile?

thanks

---

<div class="post-metadata">

**Author:** ![kporter](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/kporter/32/515_2.png) [@kporter](https://discuss.aerospike.com/u/kporter)\
**Post date:** [December 9, 2015, 7:08pm UTC](https://discuss.aerospike.com/t/aerospike-benchmark-poc/2229/7 "2015-12-09T19:08:27Z")

</div>

> [@phanimv](#):
>
> I tried loading the data second and it was successful, still couldn’t understand why the tool hangs and repeats on the last step in case of any error?

There is a similar benchmark utility distributed with the C client as well. [https://github.com/aerospike/aerospike-client-c/tree/master/benchmarks](https://github.com/aerospike/aerospike-client-c/tree/master/benchmarks)

---

<div class="post-metadata">

**Author:** ![phanimv](https://avatars.discourse-cdn.com/v4/letter/p/958977/32.png) [@phanimv](https://discuss.aerospike.com/u/phanimv)\
**Post date:** [December 10, 2015, 6:22pm UTC](https://discuss.aerospike.com/t/aerospike-benchmark-poc/2229/8 "2015-12-10T18:22:33Z")

</div>

Hi,

I shall take a look and try the c tool, otherwise currently am doing benchmark using Java client, I have done couple of tests, 1 AS namespace with NVMe configured as storage option with 1 x NVMe 400GB drive 1 AS namespace with HDD configures and file as storage option, I have loaded around ~50GB of data, followed be run against the data, the performance TPS delivered both storage options is same, I couldnt see any benefit of NVMe for Aerospike, not sure if am doing anything wrong,

storage configuration for NVMe looks as below:

storage-engine device { device /dev/nvme0n1 write-block-size 128K scheduler-mode noop }

not sure whats going wrong, are there tunings that have to be done to get the benefit of NVMe for Aerospike ?

thanks