Secondary indexes over CDT (LIST/MAP) (3.8.1)

Feature Requests Delivered Requests
, , , ,
1

I’m using Aerospike 3.5.9 Enterprised edition, java client 3.1.1 (using within scala program).

Recently, I’m using the secondary index with LIST, and perform many extensive read with the secondary index. The current query i’m using is just like:

    val stmt = new Statement
    stmt.setNamespace(...)
    stmt.setSetName(...)
    stmt.setFilters(Filter.contains(..., IndexCollectionType.LIST, id))

    val rs = client.query(null, stmt)
    while(rs.next()) {
      count += 1
    }

The number of records to get is around 700.000 to 1million. Then the server just gone down with SIGSEGV (may be because of memory leak)

May 25 2015 15:39:11 GMT: WARNING (as): (signal.c::160) SIGSEGV received, aborting Aerospike Enterprise Edition build 3.5.9
May 25 2015 15:39:11 GMT: WARNING (as): (signal.c::162) stacktrace: found 10 frames
May 25 2015 15:39:11 GMT: WARNING (as): (signal.c::162) stacktrace: frame 0: /usr/bin/asd(as_sig_handle_segv+0x59) [0x47e038]
May 25 2015 15:39:11 GMT: WARNING (as): (signal.c::162) stacktrace: frame 1: /lib64/libc.so.6(+0x326a0) [0x7fe32da826a0]
May 25 2015 15:39:11 GMT: WARNING (as): (signal.c::162) stacktrace: frame 2: /usr/bin/asd(cf_buf_builder_reserve+0xc) [0x50f576]
May 25 2015 15:39:11 GMT: WARNING (as): (signal.c::162) stacktrace: frame 3: /usr/bin/asd(as_query__netio+0x257) [0x4b35cd]
May 25 2015 15:39:11 GMT: WARNING (as): (signal.c::162) stacktrace: frame 4: /usr/bin/asd(as_query__add_response+0xde) [0x4b36f5]
May 25 2015 15:39:11 GMT: WARNING (as): (signal.c::162) stacktrace: frame 5: /usr/bin/asd(as_query__io+0x264) [0x4b29b7]
May 25 2015 15:39:11 GMT: WARNING (as): (signal.c::162) stacktrace: frame 6: /usr/bin/asd(as_query__process_ioreq+0x109) [0x4b3140]
May 25 2015 15:39:11 GMT: WARNING (as): (signal.c::162) stacktrace: frame 7: /usr/bin/asd(as_query__worker_th+0x6a) [0x4b4a71]
May 25 2015 15:39:11 GMT: WARNING (as): (signal.c::162) stacktrace: frame 8: /lib64/libpthread.so.0(+0x79d1) [0x7fe32e8a79d1]
May 25 2015 15:39:11 GMT: WARNING (as): (signal.c::162) stacktrace: frame 9: /lib64/libc.so.6(clone+0x6d) [0x7fe32db388fd]

Does somebody have seen the same phenomenon before. I suspected that there are memory leak around query with secondary index with java client.

2

Huydx,

Secondary indexes over CDT (LIST/MAP) is NOT a supported feature yet !!!

We will notify when it is …

– R

3

@raj I see. What a pity to hear about that. There are some problems that I found out:

  • If the return record set is small enough (about 20000) , it was very fast (2seconds). However if it’s very big one (about 4m records) the speed is very fast at first, but very slow at the middle till it end (it takes more than 1 hour). It seems to be the keys are in the same server took very quick to archive, but the keys are in different server took very long to archive, so the scanning of secondary index in different server seem to have problem.
  • When it took too long, I kill the query client by hand, but the memory which is allocated at server is not released (definitely there a memory leaks here)

Hope those informations help you guy to figure out some problems.

4

Thanks for the detailed information …

– R

5

Officially supported since 3.8.1.