The max response time jumps to high when reach 32GB
Aerospike 3.3.8 with In-Memory (50GB) + Disk Persistence, on HP G6 server (24 CPU-threads, 72GB memory).
Running 70K TPS pure Insert traffic (1KB record size, Client with 4 threads), and have average response time 0.05ms, max average response time 1~2ms, very stable. That’s great!
It keeps such good situation during the period of growing from 0 to 32GB data. When reach and across 32GB data, the max average response time jumps to 100~180ms, and average response time 0.08ms.
Postby Hanson » Thu Jul 31, 2014 9:57 am
When reach 40GB data, I lower the Insert traffic to 18K TPS (Client with single thread), the max average response time drops to 20~70ms. I then let another thread to Delete the records previous Inserted (several minutes ago) ---- that is 50% Insert + 50% Delete, the max average response time of Insert drops to normal 1~2ms ! It’s very strange.
Postby Hanson » Thu Jul 31, 2014 10:03 am
Typo in my previous posts: “max average response time” should be “max response time”.
Tried again with the latest Aerospike 3.3.12. Got the same behavior when DB size > 32GB:
- With pure Insert traffic, the max latency keeps ~100ms, average latency 0.08ms.
- With balanced Insert and Delete traffic, the max latency keeps ~2ms, average latency 0.05ms. The statistics interval in above is 3s.
Seems it is related to the memory Allocate (for Insert) and Reuse (Delete and Insert).