FAQ - What is the max size of a records primary key in Aeropsike

Knowledge Base General
1

The Aerospike Knowledge Base has moved to https://support.aerospike.com. Content on https://discuss.aerospike.com is being migrated to either https://support.aerospike.com or https://docs.aerospike.com. Maintenance on articles stored in this repository ceased on December 31st 2022 and this article may be stale. If you have any questions, please do not hesitate to raise a case via https://support.aerospike.com.

FAQ - What is the max size of a records primary key in Aeropsike.

Details

Aerospike can handle strings, bytes (blobs) or integer values as its primary key. The original key will be hashed (using RIPEMD-160) to generate a unique record identifier known as the digest. (See http://www.aerospike.com/docs/architecture/data-distribution.html)

Answer

The original key is not stored in the database by default. Aeropsike will only store the 20 byte digest hash of the original key as well as the set name (if specified). This would imply that the original key could be of any size that the Client can handle. There are however some limit. Let’s first look at the Java client as an example.

The java client limits the key to slightly under 8 KiB. Here is the reason:

  • The computeDigest() method is used to create a 20 byte hash digest from a set name, user key type, and variable sized user key. This 20 byte digest will be sent to the server. The user key will not be sent to the server by default thus implying there is no limit.

  • The computeDigest() method uses a temporary thread local buffer initially sized at 8KiB to assemble bytes for use with the RIPEMD-160 hash function.

  • This temporary buffer size is the defacto limit in the java client.

For C based client: (python, php, C)

No limit defined. Actual limit is the size of the stack at around 10MiB.

The stack size can be determined by running:

ulimit -s
10240

or

ulimit -a

[vagrant@localhost ~]$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 7424
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 1024
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
[vagrant@localhost ~]$ ulimit -s
10240

Notes

  • It would have of course been possible to add code to estimate the size and dynamically create a buffer in the Java client thus allowing keys of any size, but this would impact performance. In general, 8KiB should be a very reasonable limit for the vast majority of use cases.

Keywords

Max userKey size client limit

Timestamp

12/15/2016