Issue with Multiple threads executing queryAggregate

Feature Discussion Query & Indexing
1

Hi All,

I am just counting the number of records through UDF. For this I have created a Lua file as below

local function one(rec) return 1 end

local function add(a, b) return a + b end

function count_function(stream) return stream : map(one) : reduce(add); end

On Java Side, I am creating a statement of Secondary index and executing that with queryAggregate to count the number of records. Its like

Statement stmt = new Statement(); stmt.setNamespace(this.namespace); stmt.setSetName(this.set); stmt.setFilters(Filter.range(this.binName2, dataRangeLowerLimit, dataRangeUpperLimit)); stmt.setBinNames(this.binName2);

ResultSet rs = this.client.queryAggregate(null, stmt, “CountFile”, “count_function”); // if (rs.next()) { Object result = rs.getObject(); sum = ((Long)result).doubleValue(); System.out.println("Count = " + sum); }

If I run above code with 1 thread then it works fine and gives the count as per the number of records through filter range. But If I run it with Multiple threads and whether all threads are fetching different range of records or same range of records, it starts giving errors in “count” as well as all threads doesn’t complete execution and JVM remains in execution state.

Can somebody help me here ?

Thanks Nitin

2

You need to close the result set in a try/finally block.


ResultSet rs = this.client.queryAggregate(null, stmt, "CountFile", "count_function");

try {
    if (rs.next()) {
        Object result = rs.getObject();
        sum = ((Long)result).doubleValue();
        System.out.println("Count = " + sum);
    }
}
finally {
    rs.close();
}

Does the problem still happen with the latest client (3.0.30) after making this change?

3

Thanks Brian.

Yes I am closing the result set. I just didn’t paste that snippet.

and yes Its happening with the latest release.

4

We have been unable to duplicate your issue and we have some more questions about your use-case:

  1. Server version?

  2. Can you provide your aerospike.conf configuration file?

  3. How many nodes in your cluster?

  4. How many records are indexed by your secondary index?

  5. When querying, how many results match the filter when using the same range?

  6. Is rs.Close() called to cancel the query before all results have been retreived through rs.next()?

  7. How many threads is the client using?

  8. Does the problem manifest itself using one query per thread or are you running the queries in a loop on each thread (how many iterations)?

At this point, it would be very helpful if a full program was provided to duplicate the problem.

5

Aerospike Java client 3.0.31 has been released. This version fixes an aggregation query race condition which is probably the source of your problem.