Aerospike's Database Manifesto: What a Good Benchmark Should Include (Aerospike Blog Post - December 18, 2015)


#1

On December 14, 2015, Aerospike published its Benchmark Manifesto, laying out the requirements that should be included in a benchmark so that it informs, rather than misleads, developers; we challenge our database peers to exercise the intellectual rigor that should underpin a true benchmark.

A good benchmark should include:

  • A large dataset (> 10 TB)
  • Object count from 100 million to 1 billion
  • Object size from 8 bytes to 10K
  • Object variation – a distribution of sizes
  • Latency and variation under load
  • 48-hour tests
  • Node failure / consistency results
  • Replication and persistence
  • Mixed reads and write
  • Scale out by adding nodes

Read Aerospike Co-Founder and CTO Brian Bulkowski’s (@bbulkow) blog post on the topic here, and view the full text of the Benchmark Manifesto here.