We’ve been running on a rented root server. Now we want to roll our own hardware and can therefore choose the ideal components for running AS with a very write-heavy application. Ofc we know everything that has been said in the docs about the matter. However, we haven’t really understood yet what specs we should watch out for when it comes to IOPS.
So a general question: What is the most significant metric when selecting a model? Sustained 128kb (=write block size) write IOPS // 128kb random write bw or even 128kb sequential write bw ? (a.k.a. “What is the bottleneck with writes with AS”)
And a particular one: Does anybody have experience with running on the 845DC PRO? It’s specs are kinda promising:
- Enterprise/data center drive (10 DWPD for 5 years)
- 50,000 4kb write operations sustained - not sure if 4k perf is important with AS
- SATA3 (easy scale-out; pack more than 2 in a node)
- Latency seems to be ok too (avg 0.5ms, max 13ms, source: TheSSDReview.com)
- it became very cheap in Europe (~560 USD for the 800gb model)
It seems to be similiar or even better (33% more write IOPS) than the Intel S3700, but comes at half the price tag of such a drive.
Because our budget is limited, we would like to know in advance if there is any issue with this device before spending 600$ on atleast 1 drive to run ACT on.
Ofc we will run and publish ACT results once we purchased something.
Fortunately Aerospike has tested the 845DC Pro, although the results are not yet on our test results. The results were that they easily passed 3x tests, so it may go higher than that. This was without OP. Our test results at 3x show that using OP isn’t necessary. If you order one and try it out, please let us know what the results were like for you. We like to verify our results whenever we can.
In general, you should know that the 8xx line of disks has some very different lines. Anything with “DC” in it is much better than anything without. You should also steer away from the “EVO” line, unless you are doing very high read levels and these have much less durability than the “Pro” lines.
1 Like
Very promising to hear that. We will try to get our hands on atleast one 845DC Pro with 400gigs and one S3710 with 200gigs.
Looking at the papers, both drives should have similiar performance but we struggle to understand what exactly AS profits from: Hardware parallelism A.K.A. drive capacity, sequential vs. random 128k write perf, … Maybe doc’s could explain a bit more on AS’s use of the drives, helping with criterias to select good test candidates other than durability now that many vendors joined the high endurance enterprise segment.
Our goal is to achieve atleast 30k OPS/s (write-heavy) per 800gb capacity, so we might need to deploy more smaller drives than 845DC PRO offer to hit this goal on a tight budget.
Cheers from Germany, Manuel
Update: Drive’s have been ordered and will be tested within the next 4-6 weeks. Results will be posted once all benchmarks are done, if anybody is interested in early results just PM me and i’ll make sure to pass results as soon as they come in.
A few months passed and we finally got everything ready to benchmark these drives. We’ll post the results within the next weeks but from our first findings we can tell that: they certify for 3x easily but exceed latency SLAs (6-7% >1ms) at 6x without OP. At 1x you’ll find 97% completing below 0.128ms… At any load, there are hardly any requests > 8ms.
What we also found is that additional overprovisining of 21% still helps a lot with the latency and they pass at 6x and complete at 12x (stress test running atm) with slightly higher latencies. Atleast one of our drives manages to run at 12x for atleast 24 hrs without OP at reasonable latency increase. We are still testing to find a stable setting, since we’ve experienced some issues with ACT when we tested with Ubuntu. CentOS seems to be more stable.
The 845DC-drives have a periodic latency increase (some algo running every 5 to every 65 seconds) that is responsible for atleast 50% of the average latency. Not sure how common that is. However, we are still very happy with the drives especially due to their economics. They are rated at 10 DWPD and can be had for $0.68 per GB capacity (B2B-price). We’ll most likely go for a large namespace with these drives and a smaller one backed by NVMe’s for the ultra-hot dataset (if required at all).
EDIT: You can find our benchmark results here.