Java POJO to Aerospike Design Pattern

Client Libraries Java Client
, , ,
1

I have a question about the best practice for storing/manipulating data for writing into and reading out of Aerospike.

I use the Java Client, and our app is written in Java 1.7.

My goal is to create a generic way of writing, reading an querying data so it may be used around the application

My original design was to take my POJO, for example a Person class with a name and surname field, convert that to JSON and store it in the database as a Bin as a String and visa versa for a read.

Subsequently I have read through your documentation and you say it is best practice not to write JSON as a plain string, which makes sense when you do queries on Bins etc.

I am now changing the code to save these POJO’s properties/fields as individual bins, in a generic way.

class Person{

name = Johnny surname = Jones

}

I think I will store this class as follows in a record:

Bin1 = name, “Johnny” Bin2 = surname, “Jones” Bin3 = json, (person class coverted to JSON string)

This should allow me to: Key/Value queries without returning JSON and then processing it Get straight JSON from db to send to Rest Service immediately JSON Bin allowing a simple conversion from Aerospike back into Java POJO for business logic.

Is this a good pattern, or have you seen better solution for this problem of simple and efficient conversion from Java Object to Bins and back?

2

There is no “best” way to do this, but there are merits to what you are doing.

  1. Using separate bins you can index them and perform secondary index queries and aggregations
  2. Storing as JSON allows you a standardized way to encode and decode the whole object.
  3. The cost of storing denormalized/duplicated data is quite in expensive if you are using SSDs.

You may also consider the following: Rather than storing a JSON string, store it as either a List or Map. (see Value.asList() or Value.asMap()). Lists and maps can be nested inside each other. So if you are using JSONSimple to encode/decode your JSON, a JSONObject implements the Map interface and a JSONArray implements the List interface, so they are easy to store as Lists or Maps. The advantage in doing this is they can be read and written by other programming language and map to native types.

I hope this helps

3

As helipilot50 has mentioned, you could store these kind of objects in separate Bin or in Bin as list/map/blob

It’s not a good idea to store entire POJO as JSON in Aerospike DB. You would lose the flexibility and other feature benefits of Aerospike DB.You would not get much of flexibility in your design when it comes to secondary index feature,UDF (user defined functions using lua) etc. because you will have to first retrieve JSON string then parse the JSON and then retrieve each fields/attributes out of JSON string might be slower and could degrade the performance.

Having separate bins for every attributes would be better design and you will get flexibility while retrieval based on the predicate/where clause for the secondary indexed bin and also in case of simple retrieval,it would be faster since Aerospike is basically a row store i.e. Key Value Store DB .

Also you could store bin value as binary ,list or map type as per your need. Example for list is -

Bin listBin = Bin.asList(String name, List<?> value) Key key2 = new Key(namespace, setName, Value.get(“mykey1”));

client.put(policy, key2, listBin);

4

Thanks anandprakash & helipilot50 for your suggestions and help.

After taking into consideration these points, I adopted for the following new pattern:

  • Save the pojo as a JSON String as one of the bins (now GZipped as well).
  • I made a custom annotation called @SecondaryIndex on a field in my pojo. During bean introspection I check for this annotation and add it as a an additional bin for secondary indexing.

Is this a good compromise?

5

Your idea is quite elegant. It gives you the ability to store the whole object, plus include a secondary index.

One enhancement I suggest is rather than using a JSON string, store the object as Map. This way the data can be read by any other language supported by Aerospike AND by Record and Stream UDFs.

If you are using Simple JSON from Google, as your JSON parser (or anything else that implements the Map interface) you can simply use Value.asMap(…) to convert your JSON to the Aerospike Map type.

Hope this helps