Query with statement parameter doesn't work [Resolved]

Client Libraries Node.js Client
, ,
1

I have this line of code:

var q = client.query('ns1', 'set1', statement);

when run, gives the following error: Invalid query arguments: Pass UDF params to Query#apply instead.

I’m using v2.1.0 of the aerospike-client-nodejs library.

2

statement is defined as: 

var statement = {
  aggregationUDF : {
    module : 'test',
    funcname : 'filter_stream',
    args : [1, '1.0.0']
  },
  filters : [asdb.filter.equal('lang', 'en', asdb.indexType.LIST)]
};

If I renamed aggregationUDF into something else, i.e., myUDF, the execution goes through but the args don’t work as additional filter.

3

Hi @djhonjun,

The query interface has changed a bit between v1 and v2 of the client. As the error message suggests, you need to use the new Query#apply method to execute your stream UDF and aggregate the results:

var query = client.query('ns1', 'set1')
query.where(asdb.filter.equal('lang', 'en', asdb.indexType.LIST))
query.apply('test', 'filter_stream', [1, '1.0.0'], function (err, result) {
  if (err) throw err
  console.log('query result', result)
})

The new API documentation for the v2 client describes in detail how to use the Query functionality.

Hope this helps!

Cheers, Jan

4

Hi @Jan,

Thank you for the response.

I have rewritten my code as you’ve described. The extra filters seem to be working fine now. However, I am expecting it to return multiple records but it’s only returning one. Previously, I could get the stream of records via:

stream.on('data', (rec) => {
  console.log(rec);
});

With this new version, can you give an example of how it can be done?

Edit: Would appreciate it much if you can help with a corresponding Lua example of such a filter as well. Currently, this is how my Lua looks like:

local function map_filter(rec)
  return map {
    key = rec.key,
    lang = rec.lang,
    os = rec.os,
    ver = rec.ver,
  }
end

function filter_stream(stream, os, ver)
local function filter_rec(rec)
return rec[‘os’] == os and rec[‘ver’] == ver
end


return stream : filter(filter_rec) : map(map_filter)
end

Thanks!

5

@djhonjun, you are right. The v2 client currently does not support retrieving a record stream with a stream UDF applied. This was possible in the v1 client.

I am looking into adding this functionality back to v2 but with the caveat that the performance characteristics for this kind of query will likely be similar to the query functionality in v1. I.e. the client will have to do local buffering, which can lead to excessive memory consumption and has the potential to block the event queue. The query functionality currently existing in the v2 client performs much better. (See this recent blog post for further details.)

Another option is to use the Query#apply function and rewrite your UDF to return the results in a single list by using an extra reduce step. However, whether this is feasible or not depends on the result set in your particular use case.

6

@djhonjun,

Please check out v2.2 of the Node.js client that I just released today. I added back the capability to apply a stream UDF and fetch the results as a result stream using Query#foreach:

var query = client.query('ns1', 'set1')
query.where(asdb.filter.equal('lang', 'en', asdb.indexType.LIST))
query.setUdf('test', 'filter_stream', [1, '1.0.0'])
var stream = query.foreach()
stream.on('error', (err) => { throw err })
stream.on('data', (data) => console.log(data))
stream.on('exit', () => client.close())

Let me know if this works for you.

Cheers, Jan

7

Hi @Jan,

Thank you for this. Will try it out and let you know how it goes.

Cheers!

8

Hi @Jan,

I did a simple test with a small dataset and it works great so far!

Thank you for supporting this and more power to you!

Cheers! Jhonjun