How to sort all records in a SET based on one BIN in Aerospike


#1

I have studied that aggregate functions can only be applied through UDFs in Aerospike. When you need to sort any bin value, so you need to apply UDF on that and it will return sorted bin values. Limitation is that it return only one value. I am afraid that my requirement cannot be fulfilled. My requirement is I have a set named impressions . It has following bins:

ImpressionId  |  DateOfImp                            | unxDateOfImp
0987432A      | 2015-09-15T15:47:57+0545              | 1442311377

I have to sort all records based on time. For this purpose, I change DateOfImp to UNIX format named unxDateOfImp and place index on it. I assume that through UDF, I give unxDateOfImp as a parameter to UDF and it returns me only that bin after sorting. I want that all records sort on the basis of unxDateOfImp and after sorting all not only unxDateOfImp returns but other bins also returns.

Kindly guide me about my requirement and if anyone has UDF for sorting and orderby, then kindly share with me. Well suggestions for alternative solutions are also highly welcomed.


#2

Hi,

An example of how to do small orderby and groupby is found on the Aerospike web site at http://www.aerospike.com/examples/ scroll down almost all the way.

I hope this helps


#3

Thank you for your response! I have now a lua script to sort records based on ‘unxDateOfImp’. The script is working fine at my local machine with nodeJS app. My lua file and nodeJS app call is as below:

orderByUDF.lua:

function orderby(touples,bin1,bin2)
  local function mapper(rec)
  local element = map()
    element[bin1] = rec[bin1];
  return element
  end

  local function accumulate(currentList, nextElement)
 local bin1 = nextElement[bin1]

  if currentList[bin1] == nil then
    currentList[bin1] = list()
  end
   return currentList
  end

  local function reducer(this, that)
    return map.merge(this,that)
  end
    return touples:map(mapper):aggregate(map{}, accumulate):reduce(reducer)
end

request.js

router.get('/udfTest', function(req,res,next){
    var aerospikeClient = aerospike.client({
     hosts: [ { addr: '127.0.0.1', port: 3000 } ],
        modlua: {systemPath: '/opt/aerospike/sys/udf/lua/', userPath: '/opt/aerospike/sys/udf/lua/'}

     }).connect(function(response) {
     if ( response.code === aerospike.status.AEROSPIKE_OK ) {
     console.log("Connection to Aerospike cluster succeeded!");
     }
     else {
     console.log("Connection to Aerospike cluster failed. Please check your cluster IP and Port settings and try again.");
     }
     });

    var statement = {};
    
    statement.aggregationUDF = {module: 'orderByUDF', funcname: 'orderby' , args: ["unxDateOfImp"]};
    var query = aerospikeClient.query('mgProduct', 'impression', statement);
    var stream = query.execute();

    stream.on('data', function(result) {
        console.log('Printing results')
        console.log(result)
    });
    stream.on('error', function(error) {
        console.log('error happend')
        console.log(error);
    });
    stream.on('end', function(end) {
        console.log('end streaming')
    });
});

I register the UDF through the following aql command:

aql> register module '/opt/aerospike/sys/udf/lua/orderByUDF.lua'

It was successfully registered. I checked it by using the aql command show modules

Its runs perfectly fine at my local machine where both NodeJS app and aerospike is running. When I tried to shift on my distributed system where NodeJS app and aerospike are both on separate servers,it gives error. NodeJS app is remotely calling aerospike.

Error

 { code: 100,  message: 
   'UDF: Execution Error 1',  
    func: 'as_query_aggregate',  
   file: 'src/main/aerospike/aerospike_query.c',  line: 704 }

I did everything to solve this issue but could not get rid of it. My modlua paths are same as on my local machine. I also tried to find its solution in your community through this link : Help debugging Error: code: 100, message: 'UDF: Execution Error 1 but could not find any help.


#4

Also gone through the link : Help debugging Error: code: 100, message: 'UDF: Execution Error 1 but still could not solve my issue


#5

It seems you have an error either in registering the UDF or in executing it. Do you have logs turned on for the UDF?