Write: rejecting 432c0656878ba25a write size: 381952 - Record size exceeds SSD block size

Operations Monitoring
1

Hi!

From time to time I see in the logs:

Nov 16 09:56:52 as02 aerospike: Nov 16 2015 08:56:42 GMT: WARNING (drv_ssd): (drv_ssd.c::1568) write: rejecting 432c0656878ba25a write size: 381952
Nov 16 09:56:52 as02 aerospike: Nov 16 2015 08:56:42 GMT: WARNING (rw): (thr_rw.c::4591) {pa-exc} write_local: failed as_storage_record_close() <Digest>:0x5aa28b8756062c43caee7125cdc9c306e9002b59
Nov 16 10:08:23 as02 aerospike: Nov 16 2015 09:08:18 GMT: WARNING (drv_ssd): (drv_ssd.c::1568) write: rejecting abb73d821a5eece1 write size: 716416
Nov 16 10:08:23 as02 aerospike: Nov 16 2015 09:08:18 GMT: WARNING (rw): (thr_rw.c::4591) {pa-exc} write_local: failed as_storage_record_close() <Digest>:0xe1ec5e1a823db7ab3eb743cd6bb9f33e62f65b04

From the similar topic write_local: failed as_storage_record_close I got the idea that record size 381952 exceeds the SSD block size 128k.

Ok, I can see the mismatch, but what shall I do? I guess, those two records are gone now, but how to avoid such situations in the future?

Check the record size against some obtained limit from the server before putting it into the AS?

Increase block size for the storage? If so, can you do that by just changing config and restart or you need to re-deploy thr AS cluster, with cold start and zeroing all the storages?

Please, advise.

2

The server should have sent an error code back to the client when this write was rejected. Was this the case here? If you find yourself running into this frequently and you do not want to increase the write block size further than checking against some limit defined on the client before sending to the server could save bandwidth. Also take into account that as more records grow past the prior write block size your network utilization will increase.

Yes you can increase the block size, you cannot decrease the block size. The write block size should always be a power of 2 and when increasing the size needs to be a multiple of the prior size. More information can be found here: write-block-size.