# LUA aggregation with byte array (AER-3589)

**URL:** <https://discuss.aerospike.com/t/lua-aggregation-with-byte-array-aer-3589/1229>\
**Category:** Aggregation\
**Created:** [April 30, 2015, 11:11am UTC](https://discuss.aerospike.com/t/lua-aggregation-with-byte-array-aer-3589/1229 "2015-04-30T11:11:55Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![del](https://avatars.discourse-cdn.com/v4/letter/d/bb73d2/32.png) [@del](https://discuss.aerospike.com/u/del)\
**Post date:** [April 30, 2015, 11:11am UTC](https://discuss.aerospike.com/t/lua-aggregation-with-byte-array-aer-3589/1229/1 "2015-04-30T11:11:55Z")

</div>

I would like to make a request in Aerospike similar than the following SQL query :

`SELECT ts,inst,blob from db.table where inst = 'VALUE' and ts > 12234 and ts < 12245`

bins are inserted according to this structure (in python):

`{ ts : integer for a timestamp`  
` inst : string`  
` blob : bytearray }`

I have a secondary index on “ts”. “blob” is a bytearray which is the result of a “zlib.compress” of a “json.dumps”.

As I understood, I need to create a LUA aggregation function. So here is the LUA code I wrote :

`local function map_profile(record)`  
`return map {ts=record.ts,inst=record.inst,blob=record.blob }`  
`end`

`function check_instrument(stream,instrument)`  
` local function filter_instrument(record)`  
` return record.inst == instrument`  
` end`  
` return stream : filter(filter_instrument) : map(map_profile)`  
`end`

Then, I make the following AQL query :

`aggregate profile.check_instrument('VALUE') on mynamespace.myset where ts between 12234 and 12245`

which give me this response :

`+-------------------------------------------+`  
`| check_instr... |`  
`+-------------------------------------------+`  
`| {"inst":"VALUE", "ts":12235} |`  
`| {"inst":"VALUE", "ts":12236} |`  
`+-------------------------------------------+`

where is my bytearray blob ? If a put a string in my blob instead of a bytearray, the query above give me this response :

`+-------------------------------------------+`  
`| check_instr... |`  
`+----------------------------------------------------------+`  
`| {"inst":"VALUE", "ts":12235, "blob" : "hello"} |`  
`| {"inst":"VALUE", "ts":12236, "blob" : "hello"} |`  
`+----------------------------------------------------------+`

with the blob !

What I did not understand ? Could we get a bytearray with an aggregation in LUA ? How can I have my bytearray blob in response of an aggregation query ?

Thank you

aerospike-server-community 3.5.9-1 ubuntu

---

<div class="post-metadata">

**Author:** ![sunil](https://avatars.discourse-cdn.com/v4/letter/s/5fc32e/32.png) [@sunil](https://discuss.aerospike.com/u/sunil)\
**Post date:** [May 2, 2015, 4:27pm UTC](https://discuss.aerospike.com/t/lua-aggregation-with-byte-array-aer-3589/1229/2 "2015-05-02T16:27:58Z")

</div>

I am able to get the blob back. Not sure what is going wrong in your case.

```
aql> select * from test.demo
+------+------------+----------------+
| ts | inst | blob |
+------+------------+----------------+
| 1234 | "VALUE" | 68 65 6C 6C 6F |
| 1236 | "NONVALUE" | 68 65 6C 6C 6F |
| 1236 | "VALUE" | 68 65 6C 6C 6F |
| 1235 | "VALUE" | 68 65 6C 6C 6F |
| 1235 | "NONVALUE" | 68 65 6C 6C 6F |
| 1234 | "NONVALUE" | 68 65 6C 6C 6F |
+------+------------+----------------+
6 rows in set (0.068 secs)

aql> aggregate simpleagg.check_instrument('VALUE') on test.demo where ts between 1233 and 1235
+----------------------------------------------------------+
| check_instr... |
+----------------------------------------------------------+
| Map("ts"->1234, "inst"->"VALUE", "blob"->68 65 6C 6C 6F) |
| Map("ts"->1235, "inst"->"VALUE", "blob"->68 65 6C 6C 6F) |
+----------------------------------------------------------+
2 rows in set (0.006 secs)

```

---

<div class="post-metadata">

**Author:** ![del](https://avatars.discourse-cdn.com/v4/letter/d/bb73d2/32.png) [@del](https://discuss.aerospike.com/u/del)\
**Post date:** [May 4, 2015, 7:45am UTC](https://discuss.aerospike.com/t/lua-aggregation-with-byte-array-aer-3589/1229/3 "2015-05-04T07:45:07Z")

</div>

Thank you.

What is your Aerospike’s version ? I see that in your case the values are returned in a “Map”. I have a dictionary. I will have a look on how I can get a Map. In the mean time, could you please post your LUA code ?

Best regards.

---

<div class="post-metadata">

**Author:** ![sunil](https://avatars.discourse-cdn.com/v4/letter/s/5fc32e/32.png) [@sunil](https://discuss.aerospike.com/u/sunil)\
**Post date:** [May 4, 2015, 4:48pm UTC](https://discuss.aerospike.com/t/lua-aggregation-with-byte-array-aer-3589/1229/4 "2015-05-04T16:48:15Z")

</div>

I am using the 3.5.9 enterprise edition. But this should not make any difference. Actually the result that you are getting is also a map. i am using an older version of aql which is displaying it differently. I am using the same lua file that you are using. So, I am wondering what is the difference.

Can you share the output of ‘select \* from test.demo where ts between 1233 and 1235’

---

<div class="post-metadata">

**Author:** ![del](https://avatars.discourse-cdn.com/v4/letter/d/bb73d2/32.png) [@del](https://discuss.aerospike.com/u/del)\
**Post date:** [May 5, 2015, 1:49pm UTC](https://discuss.aerospike.com/t/lua-aggregation-with-byte-array-aer-3589/1229/5 "2015-05-05T13:49:21Z")

</div>

Hi,

I have found my problem. This is because I use the python client which does not seem to handle the bytearray properly.

In python, when I insert “hello” in a bytearray, I get the following :

```
| "NONVALUE" | 1236 | 63 5F 5F 62 75 69 6C 74 69 6E 5F 5F 0A 62 79 74 65 61 72 72 61 79 0A 70 31 0A 28 56 68 65 6C 6C 6F 0A 53 27 6C 61 74 69 6E 2D 31 27 0A 74 52 70 32 0A 2E |

```

This decoded blob content is :

```
 "c __builtin__ \nbytearray\np1\n(Vhello\nS'latin-1'\ntRp2\n."

```

All work fine like you when I insert using the C client. There is something strange or something I do not understand with the python client.

```
aql> aggregate simpleagg.check_instrument('VALUE') on test.demo where ts between 1233 and 1236 ;

+----------------------------------------------------+
| check_instr... |
+----------------------------------------------------+
| {"inst":"VALUE", "ts":1234} |
| {"inst":"VALUE", "ts":1236} |
| {"inst":"VALUE", "ts":1235} |
| {"inst":"VALUE", "blob":68 65 6C 6C 6F, "ts":1236} |
+----------------------------------------------------+

```

Thanks for your help.

Best regards

---

<div class="post-metadata">

**Author:** ![sunil](https://avatars.discourse-cdn.com/v4/letter/s/5fc32e/32.png) [@sunil](https://discuss.aerospike.com/u/sunil)\
**Post date:** [May 5, 2015, 6:19pm UTC](https://discuss.aerospike.com/t/lua-aggregation-with-byte-array-aer-3589/1229/6 "2015-05-05T18:19:38Z")

</div>

Ahh, that explains things…I inserted data using node.js client. But can you share your python code snippet to insert blob. This could be a python client issue.

---

<div class="post-metadata">

**Author:** ![rbotzer](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/rbotzer/32/2114_2.png) [@rbotzer](https://discuss.aerospike.com/u/rbotzer)\
**Post date:** [May 5, 2015, 7:36pm UTC](https://discuss.aerospike.com/t/lua-aggregation-with-byte-array-aer-3589/1229/7 "2015-05-05T19:36:24Z")

</div>

Yes, it would be nice to get your snippet of Python, but I already see what’s going on, and have a JIRA ticket associated with it.

---

<div class="post-metadata">

**Author:** ![del](https://avatars.discourse-cdn.com/v4/letter/d/bb73d2/32.png) [@del](https://discuss.aerospike.com/u/del)\
**Post date:** [May 6, 2015, 7:17am UTC](https://discuss.aerospike.com/t/lua-aggregation-with-byte-array-aer-3589/1229/8 "2015-05-06T07:17:37Z")

</div>

Hi,

Here is my python code

```
import aerospike
config = { 'hosts': [("192.168.11.12", 3000)] }
client = aerospike.client(config).connect()
for ts,val in (( 1234 , "VALUE"),
               ( 1234 , "NONVALUE" ),
               ( 1235 , "VALUE"),   
               ( 1235 , "NONVALUE" ),
               ( 1236 , "NONVALUE"),
               ( 1236 , "VALUE") ) :

    key = ('test', 'demo' , "{}{}".format(ts,
                                           val))
    client.put(key, { "ts" : ts, 
                       "inst" : val , 
                       "blob" : bytearray('hello') })

client.close()

```

Thanks for your help.

Best regards

---

<div class="post-metadata">

**Author:** ![rbotzer](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/rbotzer/32/2114_2.png) [@rbotzer](https://discuss.aerospike.com/u/rbotzer)\
**Post date:** [May 6, 2015, 2:00pm UTC](https://discuss.aerospike.com/t/lua-aggregation-with-byte-array-aer-3589/1229/9 "2015-05-06T14:00:30Z")

</div>

Thanks! We’re in progress on this.

---

<div class="post-metadata">

**Author:** ![Mnemaudsyne](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/mnemaudsyne/32/200_2.png) [@Mnemaudsyne](https://discuss.aerospike.com/u/Mnemaudsyne)\
**Post date:** [May 7, 2015, 1:46am UTC](https://discuss.aerospike.com/t/lua-aggregation-with-byte-array-aer-3589/1229/10 "2015-05-07T01:46:17Z")

</div>

Hi @del,

The JIRA number in which we are tracking your issue is AER-3589.

---

<div class="post-metadata">

**Author:** ![rbotzer](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/rbotzer/32/2114_2.png) [@rbotzer](https://discuss.aerospike.com/u/rbotzer)\
**Post date:** [May 19, 2015, 7:48pm UTC](https://discuss.aerospike.com/t/lua-aggregation-with-byte-array-aer-3589/1229/11 "2015-05-19T19:48:37Z")

</div>

This is now part of the new [1.0.44 release](https://github.com/aerospike/aerospike-client-python/releases/tag/1.0.44) of the Python client. Please verify that it works for you.

---

<div class="post-metadata">

**Author:** ![del](https://avatars.discourse-cdn.com/v4/letter/d/bb73d2/32.png) [@del](https://discuss.aerospike.com/u/del)\
**Post date:** [May 21, 2015, 1:48pm UTC](https://discuss.aerospike.com/t/lua-aggregation-with-byte-array-aer-3589/1229/12 "2015-05-21T13:48:24Z")

</div>

Thank you, now it works fine.

Best Regards.