Hi,
We use community version of aerospike 3.16.x with SSD as the underlying storage device. We are running some benchmarks to get an idea how much load our AS servers can take. We came to an interesting observation that we seek explanation to. We saw that, while we are generating a significant write load on the aerospike cluster (3K per machine), the write IOPs are around ~200 (can be seen the screenshot I have attached) . This could be explained by the fact that aerospike writes records into in-memory write block first and commits to device in the background as per default configurations in 3.16.x . But, we also observed that, there are good number of read IOPs on the same machine when we were generating write load. The read IOPs is in order of the write load we are generating (can be seen the same screenshot). Note that, the keys we were generating write load on were already present in the aerospike. Same behaviour was not observed when we generated fresh write load (when the keys were not already present in the aerospike).
We also observed something similar during Read only load. We saw read IOPs of the order of the Read only load we were generating.
I have following questions based on the observation :
-
How does read path in aerospike works? I have seen other docs in this forum which talks a lot of write path in bits and pieces but did not come across any doc that discusses about the read path.
-
Why writing a record with the key already present leading to a respective read IOP ?
-
Is there a way to bring down the read IOPs while both writing and reading? We saw a 1:1 ratio in the number of requests to number of read disk IOPs. This is directly impacting the performance of our cluster.
