# C client crash on close idle connections cb

**URL:** <https://discuss.aerospike.com/t/c-client-crash-on-close-idle-connections-cb/7256>\
**Category:** C Client Library\
**Created:** [March 26, 2020, 6:11pm UTC](https://discuss.aerospike.com/t/c-client-crash-on-close-idle-connections-cb/7256 "2020-03-26T18:11:12Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![sfli](https://avatars.discourse-cdn.com/v4/letter/s/a88e4f/32.png) [@sfli](https://discuss.aerospike.com/u/sfli)\
**Post date:** [March 26, 2020, 6:11pm UTC](https://discuss.aerospike.com/t/c-client-crash-on-close-idle-connections-cb/7256/1 "2020-03-26T18:11:12Z")

</div>

My code doing async put. Running 16 independent processes.

- Cluster: 8 nodes, 256GB, HDD.
- Client version: 4.6.13.el7.x86\_64. I tried both libuv and libev. See the same problem.

Below is the trace back. See this before?

(gdb) bt #0 ck\_pr\_inc\_32 (target=0x191) at modules/common/target/Linux-x86\_64/include/aerospike/ck/gcc/x86\_64/ck\_pr.h:359 #1 as\_node\_reserve (node=0x191) at src/include/aerospike/as\_node.h:477

#2 as\_event\_close\_idle\_connections\_cb (event\_loop=0x25702a0, state=0x7ffab00008c0) at src/main/aerospike/as\_event.c:1311

#3 0x0000000000418b53 in as\_ev\_wakeup (loop=, wakeup=, revents=) at src/main/aerospike/as\_event\_ev.c:81

#4 0x000000000046a7cb in ev\_invoke\_pending ()

#5 0x000000000046b6a5 in ev\_run ()

#6 0x000000000041888b in ev\_loop (flags=0, loop=0x257c160) at /usr/local/include/ev.h:835

#7 as\_ev\_worker (udata=0x257c160) at src/main/aerospike/as\_event\_ev.c:98

#8 0x00007ffadf73cdc5 in start\_thread () from /lib64/libpthread.so.0

#9 0x00007ffadeb4773d in clone () from /lib64/libc.so.6

---

<div class="post-metadata">

**Author:** ![Brian](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.aerospike.com/brian/32/2211_2.png) [@Brian](https://discuss.aerospike.com/u/Brian)\
**Post date:** [March 29, 2020, 11:17pm UTC](https://discuss.aerospike.com/t/c-client-crash-on-close-idle-connections-cb/7256/2 "2020-03-29T23:17:48Z")

</div>

A new github C client branch “4.6.13.1” has been created with an important async retry fix.

[https://github.com/aerospike/aerospike-client-c/tree/4.6.13.1](https://github.com/aerospike/aerospike-client-c/tree/4.6.13.1)

Please try your application with this C client branch.

---

<div class="post-metadata">

**Author:** ![sfli](https://avatars.discourse-cdn.com/v4/letter/s/a88e4f/32.png) [@sfli](https://discuss.aerospike.com/u/sfli)\
**Post date:** [March 31, 2020, 3:28am UTC](https://discuss.aerospike.com/t/c-client-crash-on-close-idle-connections-cb/7256/3 "2020-03-31T03:28:07Z")

</div>

Hi Brian, Thank you very much for your quick response, and put out the fix in few days! I will give it a try.

---

<div class="post-metadata">

**Author:** ![sfli](https://avatars.discourse-cdn.com/v4/letter/s/a88e4f/32.png) [@sfli](https://discuss.aerospike.com/u/sfli)\
**Post date:** [March 31, 2020, 9:39pm UTC](https://discuss.aerospike.com/t/c-client-crash-on-close-idle-connections-cb/7256/4 "2020-03-31T21:39:53Z")

</div>

I reduced the number of processes from 8 to 4. Still crashed. For each process, I set max\_connections to 300. We have 6.3B records to insert. Crashed in the middle randomly.

I believe this is related to async\_put retry. When cluster is empty, the “fresh” put operation went through smoothly. Never seen any problem with “fresh” put. The performance of fresh put is awesome, see 1M+ puts per second.

Problem happened only when PK exists, i.e. updating the existing records. Performance decreased 3~5 times, and crash frequently.

Program terminated with signal 11, Segmentation fault. #0 ck\_pr\_inc\_32 (target=0x191) at modules/common/target/Linux-x86\_64/include/aerospike/ck/gcc/x86\_64/ck\_pr.h:359 359 CK\_PR\_GENERATE(inc)

(gdb) bt #0 ck\_pr\_inc\_32 (target=0x191) at modules/common/target/Linux-x86\_64/include/aerospike/ck/gcc/x86\_64/ck\_pr.h:359

#1 as\_node\_reserve (node=0x191) at src/include/aerospike/as\_node.h:477

#2 as\_event\_close\_idle\_connections\_cb (event\_loop=0x1d182a0, state=0x7f58e80008c0) at src/main/aerospike/as\_event.c:1379

#3 0x0000000000418e0d in as\_ev\_wakeup (loop=, wakeup=, revents=) at src/main/aerospike/as\_event\_ev.c:81

#4 0x000000000046ab0e in ev\_invoke\_pending ()

#5 0x000000000046b9e8 in ev\_run ()

#6 0x0000000000418b3b in ev\_loop (flags=0, loop=0x1d24160) at /usr/local/include/ev.h:835

#7 as\_ev\_worker (udata=0x1d24160) at src/main/aerospike/as\_event\_ev.c:98

#8 0x00007f592002bdc5 in start\_thread () from /lib64/libpthread.so.0

#9 0x00007f591f43673d in clone () from /lib64/libc.so.6 (gdb)

---

<div class="post-metadata">

**Author:** ![sfli](https://avatars.discourse-cdn.com/v4/letter/s/a88e4f/32.png) [@sfli](https://discuss.aerospike.com/u/sfli)\
**Post date:** [April 7, 2020, 11:00pm UTC](https://discuss.aerospike.com/t/c-client-crash-on-close-idle-connections-cb/7256/5 "2020-04-07T23:00:09Z")

</div>

I changed my code a lot. The problem could not be reproduce any more. So don’t worry about it for now. Thanks for the help Brian!
