I have a version 4.2 database with a fairly large number number of records that are approaching the currently configured write-block-size of 1Mb. So, now I need to consider whether or not to shard my data or increase my configured write-block-size. I’ve seen that increasing the write-block-size has negative performance impacts, but I can’t find any specific benchmarks so I can’t tell if it’s better to raise it, or shard my data. Is there anywhere I could find concrete data to help me make this decision?
The answer will always be ‘benchmark it’. In my experience I think it more so comes down to a question of if you will ever be using parts of the data or always the whole piece.
If the answer on if you will ever only need to read/write part of the data is ‘yes’, such as “yes sometimes I only need part of the data”, or even “yes sometimes I will only write part of the data”, then the answer is usually that you will benefit from splitting up your records at the cost of extra index space.