FAQ - Write Block Size


#1

Details

The configuration write-block-size defines the size in bytes of each I/O block that are written to the disk. You can increase or decrease this depending on your record size. The default value is 1MB and the configured value of this parameter must be a power of 2, so the different options are: 128K, 256K, 512K, and 1M (up to 8M in as of version 4.2). To identify the optimal settings, we would recommend running a benchmark tool (ACT) or contact Aerospike support for guidance.

How do I change the write-block-size configuration?

To update the write-block-size setting:

  1. Open /etc/aerospike/aerospike.conf.

  2. Configure the namespaces write-block-size to the new desired size. Note that this configuration is placed inside the storage-engine device stanza. This is not applicable for storage-engine memory.

  3. Restart the server: /etc/init.d/aerospike restart.

  4. Continue with other nodes in the cluster.

Can I change the write-block-size configuration in a rolling manner on a cluster?

The write-block-size configuration is static but can be changed in a rolling manner across all nodes in a cluster. Here are a few points to be attentive to, though:

  • When increasing the write-block-size, records with the new ‘increased’ size should not be written until all nodes in the cluster have been re-configured with the increased write-block-size.
  • If decreasing the write-block-size, it is necessary to first delete all the records that are bigger than the new lower write-block-size and also zeroize the disks which will require to wait for migrations to complete between each node.
  • If running the older cluster protocol (versions 3.13 and older), it may also be necessary to wait for migrations to complete between each node (depending on the nature of the write workload).

Can I increase and decrease the value with only rolling service-restarts on a cluster?

Once the configuration is increased, it cannot be decreased without zeroizing the disks.

What happens when I write records bigger than the configured write-block-size?

This configuration upper-limits the maximum size of the record that can be written on the cluster. Any records with a size bigger than write-block-size will trigger an error to the client and a write-failure.

Server side logs / stats

  • Versions prior to 3.16:
Jan 22 2017 16:39:55 GMT: WARNING (rw): (thr_rw.c:write_local_ssd:4658) {namespace1} write_local: failed as_storage_record_write() <Digest>:0x88bb698a04a26517e5528hje57ed188e12ab29f4f
Jan 22 2017 16:39:59 GMT: WARNING (drv_ssd): (drv_ssd.c::1568) write: rejecting 1765a2048a69bb88 write size: 131328
  • Version 3.16 and above:
Feb 07 2018 01:16:46 GMT: INFO (info): (ticker.c:728) {test} special-errors: key-busy 1234 record-too-big 5678
  • 2 detail-level log lines are also available for record-too-big errors (‘rw’ context and ‘ssd’ context). Unlike the stat, the special-error log ticker and the ‘rw’ context line, the ‘ssd’ context line occurs on all oversized attempts, including replica writes, immigrations, and applying duplicate resolution winners.

  • The fail_record_too_big statistic will be incremented on each occurrence.

Error seen on the client

AS_PROTO_RESULT_FAIL_RECORD_TOO_BIG - Error code 13

Can I set the write-block-size lower than 128KB?

We do recommend a write-block-size of 128KB for SSD’s. Having smaller write-block-size causes more hit to SSD which in turn creates more I/O operations and increase the defragmentation process. This would in-turn also increase the write amplification factor of the SSD.

I have records larger than 1MB - what are my options?

For server versions prior to 4.2, the only option is to consider splitting the records and handling the merge on the client side. For server versions 4.2 and above, it is possible to increase the write-block-size. This could adversely impact the overall performance of the system, though.

Important considerations

A few considerations on the write-block-size parameter:

  1. This configuration is on a per-namespace basic and is only configurable if the storage-engine is device for the namespace.
  2. The value of this parameter must be a power of 2, so your options to decrease are: 128K, 256K, 512K, 1M etc.
  3. Performance characteristics of your cluster may change so careful monitor is necessary.

References

Link to the configuration reference: http://www.aerospike.com/docs/reference/configuration/#write-block-size

To identify the optimal configuration for your setup, we recommend testing your SSD’s with our certification tool (ACT).

Timestamp

06/08/2018

Keywords

write-block-size write block size


Data Modeling with records bigger than 128 KB
Are there any problems with increasing the write block size?
Particle flat size error in Aerospike.log
Record, Write Block Size & LDT
How can i implement one-to-many relationship on Aerospike?
How to store a binary file by php-client
LDT Allocation error
#2

Hi,

I tried putting write-block-size 1M to my config file like this:

namespace test {
    write-block-size 1M
    replication-factor 2
    memory-size 4G
    default-ttl 30d # 30 days, use 0 to never expire/evict.

    storage-engine memory
}

But aerospike failed to recognize this config:

May 28 2015 17:51:47 GMT: CRITICAL (config): (cfg.c:1189) line 49 :: unknown config parameter name 'write-block-size'
May 28 2015 17:51:47 GMT: WARNING (as): (signal.c::134) SIGINT received, shutting down
May 28 2015 17:51:47 GMT: WARNING (as): (signal.c::137) startup was not complete, exiting immediately

I also tried putting it inside storage-engine section like in Recipe for an SSD storage engine, but still doesn’t work:

namespace test {
    replication-factor 2
    memory-size 4G
    default-ttl 30d # 30 days, use 0 to never expire/evict.

    storage-engine memory {
        write-block-size 1M
    }
}

Could you please help me on this? I don’t know what I’ve done wrong here.


#3

Ok looks like write-block-size is only applicable to disk storage engine as I’ve read in configuration reference.


#4

That is correct, write-block-size only applies to storage-engine device.


#5

Hello @kporter, How can I set size to record value on RAM ?


#6

RAM has no record limit. The record size limit only applies if your namespace is configured with a persistent storage device @Jacky


#7

Thanks so much.


#8

What’s the difference between filesize and write-block-size as both of them seemed kind of similar to me?


#9

One of the possible storage engine options for a namespace can be in-memory with persistence. The persistence layer can be defined as (using one or more) raw device, and then the write-block-size applies. Alternatively, the persistence layer can be defined as a using one or more file on the filesystem, and then the filesize configuration applies.


#10

The write-block-size applies to both, normally it is left to 1MiB when using file.