Lua UDF, 64 bit integers (AER-6669)

Feature Requests Delivered Requests
1

After a few wasted hours of debugging, it occurred to me that Lua does not support integers, and certainly not 64 bit ones. If I send a 64 bit integer as an argument to a UDF, it will be received as a 64-bit double floating point value.

For example, I send 799995950713014610, lua receives 7.9999595071301e+17, and saving the value to a bin (from lua) will actually save 799995950713014656.

This happened without any warnings and the only way to detect is was to print the value from the UDF and find it in the log file.

This behaviour is actually very dangerous, I can imagine many projects already sending integers to Lua and expecting them to be handled correctly. The dangerous part is that they will actually be handled correctly, but only until the values reach very high such as when a service is growing rapidly.

The alternative now is to send the large integer value as strings to Lua instead, which means they will be saved as strings in the bin and waste lots of space (~20 bytes instead of 8). This is a show stopper for us right now.

Now for the actual feature requests:

  • The documentation MUST be updated to mention this in the “Known Limitations” section. Lua number handling can NOT be trusted and it surprises me a lot that such a critical misbehaviour is mentioned nowhere

  • Support Lua 5.3 where 64 bit integers are supposedly supported (is it possible/safe to update manually? would Aerospike still send 64 bit integers as double floats to Lua?)

  • Add support for a more reliable UDF language. Again, in critical database environment implicit type casts are unacceptable. Is it possible to write a C UDF without a “lua wrapper”?

2

Apologies, thanks for the clear message.

  1. Documentation on “Known Limitations” is updated.
  2. Yes, definitely plans to support 5.3. We need to work out details on LuaJit, which is currently at 5.2. Community input welcome.
  3. Fully understand the concern. Right now, it is not possible to write C UDF without a lua wrapper.
1 Like
3

I would also like to be able to write UDFs in something other than Lua, besides Lua being slow (and apparently having other issues) I dont like the language at all, Lua is something game developers use. Give us a C UDF api and then we can wrap it in other languages, possibly Rust or anything else than can compile to a binary shared library. Also please make sure that I can debug that compiled UDF via GDB when it’s “inside” the Aerospike server. This should be doable when it’s only on one testserver on localhost.

4

7.0 has been released with Lua 5.4 support: Aerospike Server CE Release Note | Download | Aerospike.

5

This topic was automatically closed 84 days after the last reply. New replies are no longer allowed.