Adding object in LDT throws an error


#1

Hello,

I am getting the following error while adding an object to LLIST using C# client library.

Error Code 1432: /opt/aerospike/sys/udf/lua/ldt/lib_llist.lua:894 LDT-Bad Key Type (userdata)

The code snapshot is as follows:

Aerospike.Client.Key key = new Aerospike.Client.Key(clientObj.AerospikeNamespace, SetTradeData, newTradeOrderID);

Aerospike.Client.LargeList llistTradeEvents = clientObj.AerospikeClusterClient.GetLargeList(clientObj.AerospikeWritePolicy, key, tradeEvents, null);

TradeEvent newTradeEvent = new TradeEvent(“BUY”);

if(newTradeEvent != null && llistTradeEvents != null) {
llistTradeEvents.Add(Aerospike.Client.Value.Get(newTradeEvent)); // <------- Error here
}

I tried to test LDT using string data as follows and it works fine.

string testLargeList1 = “llistValue1”; string testLargeList2 = “llistValue2”;

llistTradeEvents.Add(Aerospike.Client.Value.Get(testLargeList1));

llistTradeEvents.Add(Aerospike.Client.Value.Get(testLargeList2));

I even tried integer values as follows and it works fine too. int value1 = 10; int value2 = 20;

llistTradeEvents.Add(Aerospike.Client.Value.Get(value1)); llistCntgtEvents.Add(Aerospike.Client.Value.Get(value2));

I tried to add object using GetAsBlob(…) method also and it doesn’t work. I tried to serialize object and store bytes. But that doesn’t work either. I tried using dictionary and that doesn’t work either.

            Dictionary<int, string> testDictionary = new Dictionary<int, string>();
            testDictionary.Add(value1, testLargeList1);
            testDictionary.Add(value2, testLargeList2);

            llistTradeEvents.Add(Aerospike.Client.Value.GetAsMap(testDictionary));

Please let me know how do I store object in LLIST. I have to store data in LLIST because I am expecting a lot of trade events.

Thanks, Mehul


#2

LargeList accepts the following types:

  • String
  • Integer
  • Map

The map type must contain a “key” and a variable number of column values. See AerospikeDemo/LargeList.cs in RunWithDistinctBins() for example on how to use map. Here is a snippet:

	// Write trades
	Dictionary<string,Value> dict = new Dictionary<string,Value>();

	DateTime timestamp1 = new DateTime(2014, 6, 25, 12, 18, 43);
	dict["key"] = Value.Get(timestamp1.Ticks);
	dict["ticker"] = Value.Get("IBM");
	dict["qty"] = Value.Get(100);
	dict["price"] = Value.Get(BitConverter.GetBytes(181.82));
	list.Add(Value.Get(dict));

Here is an example for your TradeEvent:

	TradeEvent newTradeEvent = new TradeEvent("BUY");
	Dictionary<string,Value> dict = new Dictionary<string,Value>();
	dict["key"] = Value.Get(newTradeEvent.key);
	dict["field1"] = Value.Get(newTradeEvent.field1);
	dict["field2"] = Value.Get(newTradeEvent.field2);
	list.Add(Value.Get(dict));

#3

Thank you for your reply. If I want to store serialized objects then do I have to create a map? Value class does have Get(object obj) method. Why can’t I use that method?

Aerospike.Client.Key key = new Aerospike.Client.Key(clientObj.AerospikeNamespace, SetTradeData, newTradeOrderID);

Aerospike.Client.LargeList llistTradeEvents = clientObj.AerospikeClusterClient.GetLargeList(clientObj.AerospikeWritePolicy, key, tradeEvents, null);

TradeEvent newTradeEvent = new TradeEvent("BUY");

if(newTradeEvent != null && llistTradeEvents != null) { 
 llistTradeEvents.Add(Aerospike.Client.Value.Get(newTradeEvent)); // <------- Error here
}

The code compiles fine but during run time it gives an error.

One more question I have is regarding Get method on Dictionary. Aren’t we supposed to use GetAsMap method of Aerospike.Client.Value ?


#4

Yes, you need to create a map. The map itself is not explicitly stored. It’s just used to specify the key (LargeList is an ordered list) and the bins for the list row. If you want to serialize the entire TradeEvent in one bin, your map could be:

	Dictionary<string,Value> dict = new Dictionary<string,Value>();
	dict["key"] = Value.get(key);
	dict["value"] = Value.get(<serialized byte[]>);

We recommend serializing to byte[] independent of the client because Value.Get(object obj) uses the very slow default C# serialization and this method also must introspect the object for its real type.

If you are using version < 3.1.1, then yes, you need to use Value.GetAsMap(). Version 3.1.1 supports Value.Get(IDictionary), so GetAsMap() is no longer needed.


#5

I use google protocol buffer to serialize objects. Thanks for the information. It worked for me. I understand that “key” is just to create a sorted list entry. Also new “key” value should be higher than the previous key value. If I am not wrong, key value should always be in an ascending order.


#6

Mehul,

Key has to be unique. System stores it in ascending order of key.

– R


#7

Yes key has to be unique. I am able to get it working but I noticed that with 100,000 records it takes 4 minutes to write data in Aerospike. The replication factor is 2. I am wondering how do I improve performance of the system as 4 minutes is just too much. I am using synchronous client not asynchronous client.


#8

Mehul,

Few quick questions

  1. Server version you are using
  2. How are you loading data llist.add / llist.add_all / llist.update / llist.upate_all
  3. What does your server configuration look like (data in memory / data on disk/ write_block_size), CPU core etc.
  4. How does TOP look like when you are loading data …
  5. What kind of data are you loading (Size of item number of items per records)

– R


#9

@mehul,

Thank you for posting about LDTs in our forum. Please see the LDT Feature Guide for current LDT recommendations and best practices.


#10

@mehul,

Effective immediately, we will no longer actively support the LDT feature and will eventually remove the API. The exact deprecation and removal timeline will depend on customer and community requirements. Instead of LDTs, we advise that you use our newer List and SortedMap APIs, which are now available in all Aerospike-supported clients at the General Availability level. Read our blog post for details.