My UDF make aerospike crash

Feature Discussion User Defined Functions (UDF)
1

I want some feature of a key-hash struct, support atomic increase and batch get, so I write the UDF like follow:

function increase(topRec, key, name, addvalue)
	local newRecord = false;
	if not aerospike:exists(topRec) then
	   local r = aerospike:create(topRec);
	   if (not r == 0) then
	           GP=F and warn("[ERROR]<%s> Record could not be created", meth);
		   error("Record could not be created");
	   end
	   newRecord = true;
	end
	
	if topRec["v"] == null then
		topRec["v"] = map()
		topRec["key"] = key
	end
	
	local m = topRec["v"]
	local idx = string.find(name, ":")
	while idx ~= null do
		local str = string.sub(name, 1, idx-1)
		
		if m[str] == null then
		   	m[str] = addvalue
		else
			m[str] = m[str] + addvalue	
		end
	
		idx = string.find(name, ":", idx+1)
	end

	if m[name] == null then
	   	m[name] = addvalue
	else
		m[name] = m[name] + addvalue	
	end
	topRec["v"] = m

	if not newRecord then
		record.set_ttl(topRec, record.ttl(topRec));
	end
	aerospike:update(topRec)
	return m[name]
end

function batch_get(topRec, ldtBinName, ...)
	if not aerospike:exists(topRec) then
		return "record not exist";
	end

	if topRec[ldtBinName] == null then
		error("bin not exist");
	end
	
	local kv = topRec[ldtBinName]
	local len = select('#', ...)
	if len == 0 then
		error("not enough args");
	end
	
	local ret = list()
	for i=1, len do
		local key = select(i, ...)
		if kv[key] == null then
			list.append(ret, 0)
		else
			list.append(ret, kv[key])
		end
	end
	return ret
end

in the code, I use bin “v” to store a map, bin “key” to store the raw user’s key.

when I insert “a3:p5:sc6”, another two item “a3:p5” and “a3” will also be inserted.

a record looks like this:

ascli udf-record-apply namespace set testkey frequence increase testkey a3:p5:sc6 1
ascli get namespace set testkey
{"v": {"a3:p5": 1, "a3": 1, "a3:p5:sc6": 1 }, "key": "testkey"}

the problem is, this udf sometimes make aerospike crash!

Oct 21 2014 09:34:44 GMT: WARNING (as): (signal.c::150) SIGSEGV received, aborting Aerospike Community Edition build 3.3.19
Oct 21 2014 09:34:44 GMT: WARNING (as): (signal.c::157) stacktrace: frame 0: /usr/bin/asd(as_sig_handle_segv+0x59) [0x4676d8]
Oct 21 2014 09:34:44 GMT: WARNING (as): (signal.c::157) stacktrace: frame 1: /lib64/libc.so.6(+0x326b0) [0x7f60c4e3e6b0]
Oct 21 2014 09:34:44 GMT: WARNING (as): (signal.c::157) stacktrace: frame 2: /usr/bin/asd() [0x51cb66]
Oct 21 2014 09:34:44 GMT: WARNING (as): (signal.c::157) stacktrace: frame 3: /usr/bin/asd() [0x51d004]
Oct 21 2014 09:34:44 GMT: WARNING (as): (signal.c::157) stacktrace: frame 4: /usr/bin/asd() [0x51e7a6]
Oct 21 2014 09:34:44 GMT: WARNING (as): (signal.c::157) stacktrace: frame 5: /usr/bin/asd() [0x51e8a0]
Oct 21 2014 09:34:44 GMT: WARNING (as): (signal.c::157) stacktrace: frame 6: /usr/bin/asd() [0x51ed98]
Oct 21 2014 09:34:44 GMT: WARNING (as): (signal.c::157) stacktrace: frame 7: /usr/bin/asd(lua_pushlstring+0x2d) [0x519b8d]
Oct 21 2014 09:34:44 GMT: WARNING (as): (signal.c::157) stacktrace: frame 8: /usr/bin/asd(mod_lua_pushval+0xef) [0x5103b4]
Oct 21 2014 09:34:44 GMT: WARNING (as): (signal.c::157) stacktrace: frame 9: /usr/bin/asd() [0x50a92f]
Oct 21 2014 09:34:44 GMT: WARNING (as): (signal.c::157) stacktrace: frame 10: /usr/bin/asd() [0x4b3e38]
Oct 21 2014 09:34:44 GMT: WARNING (as): (signal.c::157) stacktrace: frame 11: /usr/bin/asd() [0x50b011]
Oct 21 2014 09:34:44 GMT: WARNING (as): (signal.c::157) stacktrace: frame 12: /usr/bin/asd(udf_apply_record+0x120) [0x4b86fd]
Oct 21 2014 09:34:44 GMT: WARNING (as): (signal.c::157) stacktrace: frame 13: /usr/bin/asd(udf_rw_local+0x180) [0x4b978a]
Oct 21 2014 09:34:44 GMT: WARNING (as): (signal.c::157) stacktrace: frame 14: /usr/bin/asd() [0x4a56d9]
Oct 21 2014 09:34:44 GMT: WARNING (as): (signal.c::157) stacktrace: frame 15: /usr/bin/asd(as_rw_start+0x24f) [0x4a6b2e]
Oct 21 2014 09:34:44 GMT: WARNING (as): (signal.c::157) stacktrace: frame 16: /usr/bin/asd(process_transaction+0xd72) [0x4b0cfc]
Oct 21 2014 09:34:44 GMT: WARNING (as): (signal.c::157) stacktrace: frame 17: /usr/bin/asd(thr_tsvc_process_or_enqueue+0x3e) [0x4b1552]
Oct 21 2014 09:34:44 GMT: WARNING (as): (signal.c::157) stacktrace: frame 18: /usr/bin/asd(thr_demarshal+0x389) [0x478324]
Oct 21 2014 09:34:44 GMT: WARNING (as): (signal.c::157) stacktrace: frame 19: /lib64/libpthread.so.0(+0x79d1) [0x7f60c5a4d9d1]
Oct 21 2014 09:34:44 GMT: WARNING (as): (signal.c::157) stacktrace: frame 20: /lib64/libc.so.6(clone+0x6d) [0x7f60c4ef486d]

Would someone tell me what’s wrong with lua script or help me imporve it? Thanks in advance

2

In our tests we have noticed that passing a large number of variable arguments (in our case, more than 70) to a Lua function can sometimes result in memory corruption. If batch_get() is being called with a large number of arguments, this might be the source of the crashes.

You might try changing batch_get() to take only three parameters, with the third parameter being a list of keys. This approach has worked well for us.

Hope this helps,

Brad

3

Use a array instead of variable arguments solved my problem. Thanks, Brad