I am experimenting with maintaining a set as a list type in aerospike.
Firstly, I tried doing this purely using the Java client: read the entire record, check if element exists in list, append to list if not, then write back to aerospike. This method achieved ~3000 writes per second on my machine.
I expected the second way to at least perform as well as the first way, possibly better since the record doesn’t need to be sent to the client. But the UDF was miuch slower.
Does anyone know why this might be?
Not sure if it’s related but when I executed the UDF using ascli it works but always prints Segmentation fault to console.
Yes, unique_set_write is the only function I’m calling
The records are quite small. I am using an integer key and integer values in the set. (Set size varies between 1-100 elements). I have also tried with string values in the set and saw similar performance.
For both my tests I am using 20 threads in parallel each with an instance of the AerospikeClient (v3.0.33)
I am using the default configuration file with test namespace with data stored in memory as well as on disk
The CPU is around 20-30% when I’m running the test
How big is your keySpace is it only 1 key ?? If that is the case UDF in general are little heavier than doing just get and put operation and performing real work of checking and updating list at the client side. But on the other hand UDF simplify application logic where it need not worry about CAS using write with genCheck.
If keySpace is bigger, I would expect CPU utilization to be higher side. If this is the case you can paste the service section in your aerospike.conf file.
My keyspace is 10,000 keys (integers between 1 and 10,000)
I haven’t changed the default aerospike configurations:
service {
user root
group root
paxos-single-replica-limit 1 # Number of nodes where the replica count i
s automatically reduced to 1.
pidfile /var/run/aerospike/asd.pid
service-threads 4
transaction-queues 4
transaction-threads-per-queue 4
proto-fd-max 15000
}
Also, a bit of an unrelated question but do you know if the map type in aerospike is backed by hash table? Or does that only apply to the lmap type?
Thanks for the suggestions. I made a couple of changes: firstly I found that I was instantiating multiple instances of the Java client (1 per thread) and changed it so that each thread shares the same client instance (as recommended in the docs). secondly I increased the number of service threads as you said. It looks like performance is much better now, hitting nearly 2000 writes per second with the UDF. I guess the difference between UDF and clientside methods will be more visible when there’s network latency (I’m currently running client and aerospike on the same machine)
I’m still unsure about the map type though. I want to have a users set where the keys are user IDs, and for each user I want to store a map of counts. e.g. something like this:
ascli put test users 123 ‘{“a”:1, “b”:2, “c”:3}’
(is this using the map data type, or is it just using three bins of type integer?)
I would rather store the entire map in a single bin, however it looks like if I do this, it is not possible to read or write counts to the map without reading/writing the whole map. Any idea what the best way to do this is?
is syntax for inserting bins and its value. Use aql instead it accepts JSON strings
aql > insert into test.user(PK, bin) values ('123', 'JSON{"a":1, "b":2}')
OK, 1 record affected
aql > select * from test.user where PK='123'
+------------------+
| bin |
+ -----------------+
| {"a":1, "b":2 } |
+------------------+
1 row in set (0.382 sec)
Note that using aql you can only get and set map in entirety. You may have to use UDF to read and set specific map field. Example UDF would look like
function updateMap(record, binName, key, newValue)
local binVal = rec[binName];
if (binVal == nil) then
-- error jumps out of execution
error("Does not Exist");
end
binVal[key] = newValue;
aerospike:update(rec);
return 0;
end
Hi Raj, I have written a UDF (below) which increments a count in a map (or writes the count if it doesn’t exist already).
It seems to work completely fine when I run it with aql.
However, when I call it ~2000 times per second with the Java client, the aerospike UI shows that less than half of the writes are succeeding! (Total writes is more than twice successful writes)
I have tried adding trace logging to debug it, but everything looks fine the logs. (rc = 0 for every update).
I have also ran stat system in aql, and the stats show 100% success rate for writes and udf writes.
Any idea why Aerospike UI shows so many failures, and what might be going wrong?
function add_count( rec, binName, mapKey, valueToAdd )
if( not aerospike:exists(rec) ) then
rc = aerospike:create(rec);
if( rc == nil ) then
warn("[ERROR] Problem creating record");
error("ERROR creating record");
end
local m = map();
m[mapKey] = valueToAdd;
rec[binName] = m;
else
local binVal = rec[binName];
if( binVal == nil ) then
local m = map();
m[mapKey] = valueToAdd;
rec[binName] = m;
else
if( binVal[mapKey] == nil ) then
binVal[mapKey] = valueToAdd;
else
binVal[mapKey] = binVal[mapKey] + valueToAdd;
end
rec[binName] = binVal;
end
end
rc = aerospike:update(rec);
trace("<CHECK> Check Update Result(%s)", tostring(rc));
if( rc ~= nil and rc ~= 0 ) then
warn("[ERROR] Record update failed: rc(%s)", tostring(rc));
error("ERROR updating the record");
end
return 0;
end
No I’m not seeing any errors in the client or in the logs.
What causes the difference between total and success then?
I want to compare performance to the unique_set_write UDF which I was executing earlier - for that UDF success was always equal to total and it was consistently over 2000 successful writes per second.
For this UDF I’ve written for writing values to the map, AMC is showing around 1000 successful writes per second (although total is higher than this).
Looking at “udf_write_success” counter and calculating writes per second myself, I can see that the ‘success’ reported by AMC is correct - it is running at around 1000 writes per second. (which is less than expected, since I get 2000 writes per second with unique_set_write UDF) - Do you think there might be something wrong?
Thanks,
Josh
(this is the UDF I am currently running to write to the maps)
function write_val( rec, binName, mapKey, newValue )
if( not aerospike:exists(rec) ) then
rc = aerospike:create(rec);
if( rc == nil ) then
warn("[ERROR] Problem creating record");
error("ERROR creating record");
end
local m = map();
m[mapKey] = newValue;
rec[binName] = m;
else
local binVal = rec[binName];
if( binVal == nil ) then
local m = map();
m[mapKey] = newValue;
rec[binName] = m;
else
binVal[mapKey] = newValue;
rec[binName] = binVal;
end
end
rc = aerospike:update(rec);
if( rc ~= nil and rc ~= 0 ) then
warn("[ERROR] Record update failed: rc(%s)", tostring(rc));
error("ERROR updating the record");
end
return 0;
end
Well I’m not sure what’s going on with that. I’ve been changing the number of threads which are calling the UDF using the Java client and now it’s a lot better (success is around 90% of total). I’d still like to know why it’s not 100% though…
Yes I’m looking at udf_write_success on the server and that looks good (there are no failures)
unique_set_write isn’t a number that was just a UDF I was executing where I didn’t see the problem in AMC. The problem only occurs with the UDF I wrote to add values to map types.
It is just a 1 node cluster running on my local machine. Here are the results from those commands:
All udf request went fine. Your throughput of UDF on the server side is last column in the asloglatency output.
No errors recorded and success not incremented would mean somewhere the stats update code is not being called properly … I will have to invesigate. Can you share entire conf file I will try to reproduce …
I can see the issue using just one thead with the Java client, calling execute on the write_val UDF that I pasted earlier. (with one thread I get around 40 writes/second and AMC shows total > success)
