May 11 2015 04:06:07 GMT: INFO (namespace): (namespace_warm.c::65) ns test can't warm restart if data-in-memory
May 11 2015 04:06:07 GMT: INFO (namespace): (namespace_warm.c::88) ns test beginning COLD start
May 11 2015 04:06:07 GMT: WARNING (cf:arenax): (arenax_warm.c::154) could not allocate 1073741824-byte arena stage 0: no memory
May 11 2015 04:06:07 GMT: CRITICAL (namespace): (namespace_warm.c:as_namespace_xmem_create:255) ns test can't create arena: error creating stage
May 11 2015 04:06:07 GMT: WARNING (as): (signal.c::93) SIGABRT received, aborting Aerospike Enterprise Edition build 3.5.15
May 11 2015 04:06:07 GMT: WARNING (as): (signal.c::95) stacktrace: found 10 frames
May 11 2015 04:06:07 GMT: WARNING (as): (signal.c::95) stacktrace: frame 0: /usr/bin/asd(as_sig_handle_abort+0x59) [0x47fbe5]
May 11 2015 04:06:07 GMT: WARNING (as): (signal.c::95) stacktrace: frame 1: /lib64/libc.so.6() [0x3acb032920]
May 11 2015 04:06:07 GMT: WARNING (as): (signal.c::95) stacktrace: frame 2: /lib64/libc.so.6(gsignal+0x35) [0x3acb0328a5]
May 11 2015 04:06:07 GMT: WARNING (as): (signal.c::95) stacktrace: frame 3: /lib64/libc.so.6(abort+0x175) [0x3acb034085]
May 11 2015 04:06:07 GMT: WARNING (as): (signal.c::95) stacktrace: frame 4: /usr/bin/asd(cf_fault_event+0x271) [0x513fe6]
May 11 2015 04:06:07 GMT: WARNING (as): (signal.c::95) stacktrace: frame 5: /usr/bin/asd(as_namespace_setup+0x4c9) [0x45d377]
May 11 2015 04:06:07 GMT: WARNING (as): (signal.c::95) stacktrace: frame 6: /usr/bin/asd(as_namespaces_init+0x79) [0x4752e4]
May 11 2015 04:06:07 GMT: WARNING (as): (signal.c::95) stacktrace: frame 7: /usr/bin/asd(main+0x2e9) [0x4605ff]
May 11 2015 04:06:07 GMT: WARNING (as): (signal.c::95) stacktrace: frame 8: /lib64/libc.so.6(__libc_start_main+0xfd) [0x3acb01ecdd]
May 11 2015 04:06:07 GMT: WARNING (as): (signal.c::95) stacktrace: frame 9: /usr/bin/asd() [0x450fe9]
Without context (full log file), it looks like an out of memory at a first glance. Would check the system capacity (maybe a ram module got a defect?).
Looks like @ManuelSchmidt is on the right track with his “out of memory” diagnosis. The Enterprise Edition requires that you have at least 1G (ideally, 1.5G) of physical memory for each node. Please be sure you’ve allocated enough memory to your node, and restart the service.
If this doesn’t solve your issue, let us know what was happening up until the crash, give us the hardware specs for your node, and let us know what OS you’re using. Also, please provide us with the config file so we can troubleshoot further.
Hi,
Thanks for the help! I did installed Aerospike on other node and it worked. I’ll check the old one to see why it happens. The memory is 4G and the crash happens once I do service aerospike start.
Sasi