artursocha » Tue Feb 25, 2014 2:23 am
Hi,
I was wondering if there were any load
tests conducted using java client (latest). What could be expected
performance drop compared to C api ? I was running some end2end test
and noticed around 5k tps (writes) whereas on the same boxes simple
‘restore from backup’ (even when ran from another box connected with
1Gbit network) generated ~60k tps (also migration when tuned a bit with
according to documentation was blazing fast). For now I am using SSDs
that are slow and eventually will be replaced with ones that passes
certification test.
My config was : community version, 2 x host ,
replication=2 , disk persistence + in memory data + fire and forget /
complete on master mode
Client config(WritePolicy) : between 300 - 1000 threads + defaults
I was using single Client instance per application(as recommended) if that matters.
I also tried 1 host with 1x replication and got around 8k tps using java client.
I
am aware that my Aerospike config might not be yet efficient and that
is why I am interested in what level performance I could look for with
java client.
There are many factors at work here. In general the Java client can be roughly as fast as the C client. However, you mentioned that you were using a large number of threads (300-1000). We generally find that 64-125 is optimal, depending on the machine. With the Java client, we often find that the bottleneck is the number of threads.
ok i found out that it’s the ClientPolicy setting for the maxThreads.
I’m doing tests myself and using memory-only storage with 1x replication, single server, java client and AS server on the same machine…machine is a 2014 8-core SSD Raid0 Dell T3610 (with Xeon processor) CentOS 7. with this setting inserting 12,000 objects barely hits 5K TPS…and I’m inserting really simple objects with 5 bins: 2 strings, 2 ints, 1 double…secondary index on one of the string bins (stringbins…lol).
i expect a LOT more throughput than this. a similar setup with MongoDB is about 10x faster (50K TPS)
Have you tried simply running the java benchmark tool first? Here is the documentation for it. I would definitely recommend trying with it first.
We typically do not recommend running the client and the server on the same host but I would definitely expect much higher numbers anyway. The number of threads, as stated already is one of the key parameter to tune from the client side.
Based on numbers using the java benchmark we could then decide whether to look at tuning the server side or focus on the client side code.
when i run the benchmarks it would be at least in the 60K TPS so i think the server is fine…and my code is very, very simple…pulled straight out of the documentation almost verbatim but i cannot get nowhere near that. here is the gist of what i’m trying to do: Code fragment fo inserting into Aerospike db · GitHub …i’m also using an in-memory db configuration setting only.
To try to narrow down why your code is running slower then the benchmark, I would suggest the following approach:
Try without creating the artist_index. (It shouldn’t have such an impact at all, but just to remove variable).
Another thing to try would be to write ‘dummy strings’ so you remove the part where you are getting the data from k2.Song and just write hardcoded strings (potentially post-fixed with some integer increment for example).
One other things would be to try with 100 max threads.
Let’s see if this gets the performance up. I will otherwise bring this up with our internal Java client specialist.
Actually just realizing that you are actually likely using just 1 thread which would explain the performance you are seeing.
The maxThreads only sizes the thread pool but does not create threads for you, so you would need to take care of that in your code. You can look at the java benchmark tool source code for example.