ResultSet Next() method seems to be taking a lot of processing time


#1

I have a lua file which filters the stream data based on multiple bins (bin1 and bin2 in the code below). The execution of lua seems to be taking no time at all, and control is quickly passed back to the client. But my ResultSet Next() iteration seems to be taking up a lot of time on the client, of around 150 ms. This number was lower when the stream (flitered on bin1 and bin2 in lua) it had to work on was smaller and gave a higher number when more data was filtered out in the stream.

I need to understand why is this call taking up so much time or am I doing something wrong here.

 //Aerospike client connection code goes here
   Statement stmt = new Statement();
   stmt.SetFilters(Filter.Equal("bin0", "abc"));
    sw = Stopwatch.StartNew();                              
    ResultSet rsinner = client.QueryAggregate(null, stmt, "udfanalytics", "aggregateFunction", Value.Get(bin1), Value.Get(bin2));
    File.AppendAllText(logfile, "end call to aerospike lua for, time = " + sw.ElapsedMilliseconds + "\n");
    sw.Stop();
                                
    sw = Stopwatch.StartNew();
    int searches = 0;                       
    if (rsinner.Next()) //This call seems to be taking up around 150ms
    {
        sw.Stop();
        File.AppendAllText(logfile, "ResultSet.Next() call, time = " + sw.ElapsedMilliseconds + "\n");
        sw = Stopwatch.StartNew();
        Dictionary<object, object> result = ((Dictionary<object, object>)rsinner.Object);  
        if(result.Count > 0)
            searches = int.Parse(result.Values.First().ToString()
    );

Please let me know if you need any more data in addition to this.

Thanks!


#2

The very first query aggegate call to a new lua module (udfanalytics) requires the lua module be read from disk on the client. This is responsible for your initial long latency. The client then caches udfanalytics, so future calls will perform more quickly.