Can't run QueryAggregate from client


#1

Hi,

I am trying to run a fairly simple UDF stream witch runs through the entire set and calculates some statistics (simple counter).

I se two errors in two different scenarios:

When doing register through client and running through AQL I get a “unexpected symbol” error. When opening the LUA file on the server, I can see it prepends “.” symbol in the file and when I remove it, save and running again through AQL it works. (are there some issues in the file reading part of the client? Encoding maybe?)

When above are fixed and the AQL method works I try to run the same UDF from the client, but keeps getting the below error message: Query Failed: Method not found: ‘Neo.IronLua.LuaGlobal Neo.IronLua.Lua.CreateEnvironment()’. I can’t find anything about that error?

Code for register:

var clientDirectory = Path.Combine(Path.GetDirectoryName(System.Reflection.Assembly.GetEntryAssembly().Location), "Aerospike\\LUA");
LuaConfig.PackagePath = clientDirectory + @"\?.lua";
var filename = "Statistics.lua";
var clientPath = Path.Combine(clientDirectory, filename);

RegisterTask task = Handler.Client.Register(null, clientPath, filename, Language.LUA);
task.Wait();

Code for execution:

var stmt = new Statement();
stmt.SetNamespace("nstest");
stmt.SetSetName("settest");

var resultSet = Aerospike.Client.QueryAggregate(null, stmt, "Statistics", "sum");
if (resultSet.Next())
...

I hope you are able to help me - thanks


#2
  1. Can you provide the exact AQL registration string command?

  2. NeoLua is required to run the lua aggregation function on the client for the final reduce. Therefore, you need to copy Neo.Lua.dll and Neo.Lua.pdb to the same directory as AerospikeClient.dll and AerospikeClient.pdb.


#3
  1. Like this?

Aerospike Query Copyright 2013-2015 Aerospike. All rights reserved.

aql> REGISTER MODULE ‘/opt/aerospike/usr/udf/lua/Statistics.lua’

OK, 1 module added.

aql> AGGREGATE Statistics.sum() ON audience.nodes 2015-11-09 10:21:23 ERROR Lua Create Error: error loading module ‘Statistics’ from file ’/opt/aerospike/usr/udf/lua/Statistics.lua’: /opt/aerospike/usr/udf/lua/Statistics.lua:1: unexpected symbol near ‘▒’ Error: (100) UDF: Execution Error 1

aql>

  1. Neo.Lua.dll exists in the same directory as AerospikeClient.dll (installed through NuGet)

#4

so no one has seen this error before?


#5

The encoding of Statistics.lua might be incorrect. Please email <>@aerospike.com with the file attached and I will use as a test case.


#6

Hi Lars,

We have identified the problem. Statistics.lua contains a “Byte Order Mark” (BOM) prefix that both Windows and Linux understand and skip over. Unfortunately, the lua interpreter does not parse and skip over these initial characters.

$ lua
Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
> require 'Statistics'
error loading module 'Statistics' from file './Statistics.lua':
	./Statistics.lua:1: unexpected symbol near '�'
stack traceback:
	[C]: ?
	[C]: in function 'require'
	stdin:1: in main chunk
	[C]: ?

$ file Statistics.lua 
Statistics.lua: UTF-8 Unicode (with BOM) text, with CRLF line terminators

$ xxd -p Statistics.lua  | head -1
efbbbf6c6f63616c2066756e6374696f6e206167677265676174655f7374

"efbbbf" is the BOM.
6c = 'l'
6f = 'o'
63 = 'c'
61 = 'a'
6c = 'l'

Windows is apparently creating BOM by default. Linux does not create BOM by default. If you remove the BOM from both the client and server file, everything should work.

$ xxd -p Statistics.lua > Statistics.hex
$ tail -c +7  Statistics.hex >StatisticsNew.hex
$ xxd -r -ps StatisticsNew.hex > StatisticsNew.lua
$ lua 
Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
> require 'StatisticsNew'
> 

#7

Hi again,

Thanks, that fixed the registration problem (used Notepad++ and changed the encoding to UTF8 without BOM).

I still gets the execution error: Query Failed: Method not found: ‘Neo.IronLua.LuaGlobal Neo.IronLua.Lua.CreateEnvironment()’

Both AerospikeClient.dll and Neo.Lua.dll exists in the same folder (installed through NuGet, running debug with Any CPU configuration)

Thanks in regards


#8

The error indicates NeoLua.dll can’t be found. I tried the following steps.

  1. Create a new Visual Studio project. I used Visual Studio 2010.

  2. Right Click project’s References and choose “Manage NuGet Packages”.

  3. Click “Online” and enter “Aerospike” in the search box.

  4. Click “Official Aerospike C# Client” and click Install button.

  5. Put aggregation query code into project and build. You should see the following files in the bin/Debug directory.

AerospikeClient.dll
AerospikeClient.xml
<program name>.exe
<program name>.pdb
<program name>.vshost.exe
<program name>.vshost.exe.manifest
Neo.Lua.dll
Neo.Lua.pdb
Neo.Lua.xml
  1. Run the program in this directory or in Visual Studio. It works for me either way.

#9

I did it … the neolua files (Neo.Lua.dll, Neo.Lua.pdb, Neo.Lua.xml) are on same folder as aerospike client files (AerospikeClient.dll, AerospikeClient.pdb, AerospikeClient.xml)

`

But I’m still facing same problem described above when runnign code like:

` AerospikeAuditClient.QueryAggregate(policy, statement, (record) => { var x = record; });


#10

Did you get any solution for this? I’m facing the same situation here …


#11

Do the example aggregation queries (Query Sum, Query Average) in AerospikeDemo work for you?

I would helpful if you could provide a sample solution (including lua functions) that just runs one aggregation query which results in the error.

It may not be applicable to your query, but records can’t be passed to the client in aggregation queries. The record bins must be copied to a map (which can be passed to the client).


#12

Hi Brian, can you pleae take a look at ‘Method Not Found’ During UDF QueryAggregateCall where I describe better what I’m facing …


#13

Since this topic comes us when searching for Method not found: 'Neo.IronLua.LuaGlobal Neo.IronLua.Lua.CreateEnvironment()', I would like to add this information:

Aerospike 3.4.4 (and previous versions) does not support Neo.Lua 1.0 and above. The last supported version from Nuget is 0.9.17. All newer versions will break LUA support in AerospikeClient.

Btw, it can be very easily fixed in the Aerospike client library, though just setting the upper limit for Neo.Lua dependency would be enough.


#14

I’ve always used Neo.Lua 0.9.14, so that is why it worked for me. I will try to resolve the problem with Neo.Lua 1.0 and above. If I can’t resolve, I will set the Neo.Lua package dependency to exactly [0.9.14].

Thank you!


#15

Np Brian. FYI, the majority of the code changes necessary to migrate are rather simple, but the new Neo.Lua is a .NET Portable library that targets .NET Framework 4.5, and the Aerospike client library target 4.0, which is a bigger change.


#16

The lastest Neo.Lua version 1.2.23 broke our tests that accessed our type class library (LuaList, LuaMap, etc…).

  map = clr.Aerospike.Client.LuaMap
  local m = map()

For some reason, Neo.Lua versions >= 1.0.0 can’t resolve them. I will create a new nuget dll that fixes the Neo.Lua version dependency at exactly 0.9.14 on Monday.