# Append bytes

**URL:** <https://discuss.aerospike.com/t/append-bytes/8161>\
**Category:** Python Client\
**Created:** [January 18, 2021, 8:09am UTC](https://discuss.aerospike.com/t/append-bytes/8161 "2021-01-18T08:09:34Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![avi](https://avatars.discourse-cdn.com/v4/letter/a/b9bd4f/32.png) [@avi](https://discuss.aerospike.com/u/avi)\
**Post date:** [January 18, 2021, 8:09am UTC](https://discuss.aerospike.com/t/append-bytes/8161/1 "2021-01-18T08:09:35Z")

</div>

is there any way to append bytes without a udf? i would expect something like this to work (using python client 4.0.0)

```auto
import aerospike

ac = aerospike.client({'hosts': [('localhost', 3000)]})
ac.connect()
key = ('test', None, 0)
ac.put(key, bins={'bytes': b''})
ac.append(key, bin='bytes', val=b'a')

```

but it raises `exception.ParamError: (-2, "Cannot concatenate 'str' and 'non-str' objects", 'src/main/client/operate.c', 1162, False)`

does using a one liner udf to append bytes with [https://www.aerospike.com/docs/udf/api/bytes.html#bytes-append\_bytes-](https://www.aerospike.com/docs/udf/api/bytes.html#bytes-append_bytes-) to a single bin cost significant overhead compared to a client operation?

---

<div class="post-metadata">

**Author:** ![Dylan\_W](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/dylan_w/32/1314_2.png) [@Dylan\_W](https://discuss.aerospike.com/u/Dylan_W)\
**Post date:** [January 19, 2021, 9:45pm UTC](https://discuss.aerospike.com/t/append-bytes/8161/2 "2021-01-19T21:45:27Z")

</div>

> [@avi](#):
>
> ```auto
> 
> import aerospike
> 
> ac = aerospike.client({'hosts': [('localhost', 3000)]})
> ac.connect()
> key = ('test', None, 0)
> ac.put(key, bins={'bytes': b''})
> ac.append(key, bin='bytes', val=b'a')
> 
> ```

Hi @aviWhat Python version are you using?

You can use a bytearray with the append.

```auto
import aerospike

ac = aerospike.client({'hosts': [('localhost', 3000)]})
ac.connect()
key = ('test', None, 0)
ac.put(key, bins={'bytes': b''})
ac.append(key, bin='bytes', val=bytearray('a', 'utf-8'))
print(ac.get(key)[2])

```

Output:

```auto
{'bytes': bytearray(b'a')}

```

---

<div class="post-metadata">

**Author:** ![Dylan\_W](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/dylan_w/32/1314_2.png) [@Dylan\_W](https://discuss.aerospike.com/u/Dylan_W)\
**Post date:** [January 19, 2021, 10:23pm UTC](https://discuss.aerospike.com/t/append-bytes/8161/3 "2021-01-19T22:23:17Z")

</div>

After looking at this some more, `ac.append(key, bin='bytes', val=b'a')` works in Python2 but `append()` bytes support was likely broken during the transition to Python3. `ac.append(key, bin='bytes', val=b'a')` should work on Python3. We will log an internal issue ticket to track the problem.

Thank you for bringing it to our attention.

---

<div class="post-metadata">

**Author:** ![avi](https://avatars.discourse-cdn.com/v4/letter/a/b9bd4f/32.png) [@avi](https://discuss.aerospike.com/u/avi)\
**Post date:** [January 20, 2021, 3:48am UTC](https://discuss.aerospike.com/t/append-bytes/8161/4 "2021-01-20T03:48:14Z")

</div>

hey @Dylan_W, thanks for helping me out!

I am using python 3.8

simply wrapping the bytes with `bytearray` works for me here, but it looks like a different issue is preventing the appends to run in queries, which is what I actually want; here’s a repro

namespace config:

```auto
namespace dram_general {
    storage-engine memory
    memory-size 2G
    replication-factor 2
    nsup-period 60
}

```

code:

```python
import time
import aerospike
from aerospike import exception, predicates
from aerospike_helpers.operations import operations

ac = aerospike.client({'hosts': [('localhost', 3000)]})
ac.connect()
key = ('dram_general', None, 0)

try:
    ac.index_string_create('dram_general', None, 'c', 'c_index')
except exception.IndexFoundError:
    pass

ac.put(key, bins={'bytes': b'', 'c': 'c0'})

b = bytearray(b'\n\x04\x12\x02\x10\x01')

ac.append(key, bin='bytes', val=b)
print(ac.get(key))
ac.append(key, bin='bytes', val=b)
print(ac.get(key)) # works fine

def append_query():
    query = ac.query('dram_general', None)
    query.where(predicates.equals('c', 'c0'))
    query.add_ops([operations.append('bytes', b)])
    job_id = query.execute_background()
    # print(ac.job_info(job_id, aerospike.JOB_QUERY))
    time.sleep(1)
    print(ac.job_info(job_id, aerospike.JOB_QUERY))
    print(ac.get(key))

for _ in range(4):
    append_query()

```

output:

```auto
(('dram_general', None, None, bytearray(b'\xb3F\x13{\xa9\x18\x95y\xdaR\x01b\xdf\xdc\xd6\x1a\x10\xfe7\xd6')), {'ttl': 4294967295, 'gen': 2}, {'bytes': bytearray(b'\n\x04\x12\x02\x10\x01'), 'c': 'c0'})
(('dram_general', None, None, bytearray(b'\xb3F\x13{\xa9\x18\x95y\xdaR\x01b\xdf\xdc\xd6\x1a\x10\xfe7\xd6')), {'ttl': 4294967295, 'gen': 3}, {'bytes': bytearray(b'\n\x04\x12\x02\x10\x01\n\x04\x12\x02\x10\x01'), 'c': 'c0'})
{'progress_pct': 0, 'records_read': 0, 'status': 2}
(('dram_general', None, None, bytearray(b'\xb3F\x13{\xa9\x18\x95y\xdaR\x01b\xdf\xdc\xd6\x1a\x10\xfe7\xd6')), {'ttl': 4294967295, 'gen': 4}, {'bytes': bytearray(b'\n\x04\x12\x02\x10\x01\n\x04\x12\x02\x10\x01\xff\xd8\xccO\t\xd9'), 'c': 'c0'})
{'progress_pct': 0, 'records_read': 0, 'status': 2}
(('dram_general', None, None, bytearray(b'\xb3F\x13{\xa9\x18\x95y\xdaR\x01b\xdf\xdc\xd6\x1a\x10\xfe7\xd6')), {'ttl': 4294967295, 'gen': 4}, {'bytes': bytearray(b'\n\x04\x12\x02\x10\x01\n\x04\x12\x02\x10\x01\xff\xd8\xccO\t\xd9'), 'c': 'c0'})
free(): double free detected in tcache 2
Aborted (core dumped)

```

it looks like despite the second query job status return 2 (success), the generation of the record is unchanged and the bytes were not appended, and then the client crashes out of python; is this a related issue? Thanks again!

edit: also just noticed that the bytes appended by the first query have been transformed somehow?

---

<div class="post-metadata">

**Author:** ![Dylan\_W](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/dylan_w/32/1314_2.png) [@Dylan\_W](https://discuss.aerospike.com/u/Dylan_W)\
**Post date:** [January 21, 2021, 12:05am UTC](https://discuss.aerospike.com/t/append-bytes/8161/5 "2021-01-21T00:05:15Z")

</div>

Hi @avi ,

I don’t have a lot of information to give yet but this might be related to an Issue with write operations on background queries that we are working on currently. I’m going to look into this and I’ll update you when I have more information.

---

<div class="post-metadata">

**Author:** ![Dylan\_W](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/dylan_w/32/1314_2.png) [@Dylan\_W](https://discuss.aerospike.com/u/Dylan_W)\
**Post date:** [January 22, 2021, 1:18am UTC](https://discuss.aerospike.com/t/append-bytes/8161/6 "2021-01-22T01:18:28Z")

</div>

After looking into the issue further, this is related to the write operations on query bug we are working on. A fix is in the works and will be in the next release. As a temporary work around, you should be able to use a UDF. Sorry for the inconvenience. I will update this thread with further developments.

---

<div class="post-metadata">

**Author:** ![avi](https://avatars.discourse-cdn.com/v4/letter/a/b9bd4f/32.png) [@avi](https://discuss.aerospike.com/u/avi)\
**Post date:** [January 22, 2021, 1:36am UTC](https://discuss.aerospike.com/t/append-bytes/8161/7 "2021-01-22T01:36:23Z")

</div>

no worries, thanks for addressing it so quickly!

---

<div class="post-metadata">

**Author:** ![Dylan\_W](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/dylan_w/32/1314_2.png) [@Dylan\_W](https://discuss.aerospike.com/u/Dylan_W)\
**Post date:** [February 18, 2021, 6:53pm UTC](https://discuss.aerospike.com/t/append-bytes/8161/8 "2021-02-18T18:53:35Z")

</div>

Hi @avi ,

Both issues should be fixed in Aerospike Python client 5.0.0. Please give it a try when you have a chance.

---

<div class="post-metadata">

**Author:** ![avi](https://avatars.discourse-cdn.com/v4/letter/a/b9bd4f/32.png) [@avi](https://discuss.aerospike.com/u/avi)\
**Post date:** [February 23, 2021, 12:48am UTC](https://discuss.aerospike.com/t/append-bytes/8161/9 "2021-02-23T00:48:10Z")

</div>

hi @Dylan_W

everything is working now thanks so much!

just to end this off with a related question: what is the time/space complexity of appending bytes? is it constant time and does it avoid the overhead of `memcpy` since appending doesn’t require reading before (or does it)?

---

<div class="post-metadata">

**Author:** ![Dylan\_W](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/dylan_w/32/1314_2.png) [@Dylan\_W](https://discuss.aerospike.com/u/Dylan_W)\
**Post date:** [March 1, 2021, 5:11pm UTC](https://discuss.aerospike.com/t/append-bytes/8161/10 "2021-03-01T17:11:13Z")

</div>

The time complexity for an append operation server side is O(m + n) where m are the bytes of the particle already present, and n are the bytes being appended. During an append operation, the existing particle is read, and a new particle created from the join of the old particle and the bytes to be appended.

---

<div class="post-metadata">

**Author:** ![system](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/system/32/2274_2.png) [@system](https://discuss.aerospike.com/u/system)\
**Post date:** [May 24, 2021, 5:12pm UTC](https://discuss.aerospike.com/t/append-bytes/8161/11 "2021-05-24T17:12:00Z")

</div>

Closed on May 24, 2021, 5:12pm UTC

This topic was automatically closed 84 days after the last reply. New replies are no longer allowed.