ArrayIndexOutOfBoundsException in client.put

Hi,

I am getting this error while inserting bins :

java.lang.ArrayIndexOutOfBoundsException: 70331 at com.aerospike.client.command.Buffer.stringToUtf8(Buffer.java:186) ~[aerospike-client-3.0.31.jar:na] at com.aerospike.client.Value$StringValue.write(Value.java:352) ~[aerospike-client-3.0.31.jar:na] at com.aerospike.client.command.Command.writeOperation(Command.java:524) ~[aerospike-client-3.0.31.jar:na] at com.aerospike.client.command.Command.setWrite(Command.java:83) ~[aerospike-client-3.0.31.jar:na] at com.aerospike.client.command.WriteCommand.writeBuffer(WriteCommand.java:49) ~[aerospike-client-3.0.31.jar:na] at com.aerospike.client.command.SyncCommand.execute(SyncCommand.java:47) ~[aerospike-client-3.0.31.jar:na] at com.aerospike.client.AerospikeClient.put(AerospikeClient.java:295) ~[aerospike-client-3.0.31.jar:na]

Am I reaching any limit here ?

Thanks

Hi Holmes,

Can you post the exact code you are using in the put(), and how many Bins you are writing?

There is a limit of 32K unique Bin names per namespace. How many have you written already?

Peter

Hi,

We have not reached that limit for sure. Its not related to unique binNames as we handled that case separately. Also I have tested that limit and the error message is not this.

This may have to to do with the amount of data being pushed in bins. I am inserting about 20 unique bins (all less than 13 characters) Here is the code snippet :

final Bin[] bins = new Bin[row.size()];//row is --> Map<String, Object> row
int i = 0;
for (Map.Entry<String, Object> entry : row.entrySet()) {
            String column = entry.getKey();
            bins[i++] = new Bin(column, (String) entry.getValue());
}
client.put(writePolicy, k, bins); // WritePolicy writePolicy = new WritePolicy();

While debugging I found that in this loop in Command class Line:83 (aerospike-client-3.0.31)

    for (Bin bin : bins) {
        writeOperation(bin, operation);
    }

its able to process few bins and then AIOBE is thrown.

Thanks.

More update :

There are a total of 29 bins here. In the same Command.setWrite method , there is first a estimation of buffer size. Here its estimate is 70331.

But on WriteCommand.writeOperation(Bin bin, Operation.Type operation) method , when it is updating the last bin (#29 in the above loop)

int valueLength = bin.value.write(dataBuffer, dataOffset + OPERATION_HEADER_SIZE + nameLength);

the value of dataOffset is 70320 and total value of dataOffset + OPERATION_HEADER_SIZE + nameLength = 70331.

So, next in Buffer.stringToUtf8(String s, byte[] buf, int offset) method, it tries to increment the buffer index to 70331+1 and fails.

for (int i = 0; i < length; i++) {
            int c = s.charAt(i);
            if (c < 0x80) {
                buf[offset++] = (byte) c; // <--- Error here
            }
            else if (c < 0x...

Thanks.

Hi Holmes,

In your code, where is i declared and initialized?

Peter

Hi @helipilot50,

Updated the code.
I saw the default size of buffer is 8192 defined in ThreadLocalData

Hi Holmes,

Your analysis is correct. The write process is:

  1. Calculate buffer size from command arguments.

  2. Find or allocate a suitable buffer.

  3. Write command to buffer.

If the buffer size is not calculated correctly and the size is greater than the default buffer size (8K), then you will get a buffer overflow. I suspect “Buffer.estimateSizeUtf8()” may be calculating a different size than is actually written, but it could be other estimations as well.

Is it possible to provide a small sample program that reproduces the problem?

Most strings will be calculated correctly, but I’m not sure about some multi-byte UTF8 characters.

We’re also getting the same error with client version 3.0.33.

client.put(asWritePolicy, new Key("main", "test1", keyValue), new Bin("data", data), new Bin("schema", "1"))

where keyValue is a 36 length string and data is a byte array of length 8208. Write policy has sendkey = true as well.

Update: After a little more testing. It seems that the issue doesn’t occur if we disable sendkey. For now, I’m adding the key to a bin, but this still seems like an issue that should get resolved.

Hi Ayush,

Thanks for the sample code. The “sendkey=true” was the crucial piece as that code had a bug in it. The fix will be in the next java client release.

-Brian

@ayush , thanks !
I too found out about sendkey feature causing this issue and have disabled it for now. Also, @helipilot50 suggested to use extra bin to store user key data in this post .

@Brian, thanks for looking into it. Is Java Client Library Release Note | Download | Aerospike the release as I can see a fix for user key send buffer byte size calculation.

Yes, 3.0.34 contains the sendKey fix. I recommend upgrading to the latest client, 3.0.35 to get other fixes.

1 Like