Java Async UDF Execute & Float in UDF (3.1.1)


#1

Java client: everywhere in my program I’m using asynchronous operations but I have large amounts of UDF operations to run on records at the same time. These are basically read/write operations. Not blocking is pretty important for me.

I read somewhere else on the forum that floating point is in the todo list for data types. If so, or even without support in the database itself, I request support for fractional numbers in UDF as arguments as well as inside map() and list() types. It looks like fractional numbers cannot currently be set in a map()? Currently I’m converting those to strings but I’m considering how to work around the limitation. Support for Lua 5.3 sting.pack would also be nice for dealing with unsupported types.


#2

Hi —

Are you looking for a type equivalent to Java’s BigDecimal and Python’s decimal.Decimal ?

Those are really the “sane” floating point similar to the now-standard but “decimal64” from IEEE 754. Do you need BigDecimal, or is decimal64 ok? They are different.

As you point out, weaving a type like this throughout the entire system, such as including maps and ordered lists, when some languages don’t have the type can be a bit difficult.

In C, for example, GCC supports decimal64 only starting at 4.5+ (which is fairly prevalent now but not ubiquitous). Clang doesn’t support it at all.

Have you found a similar type in Lua? That would make it easier for us.

The time-honored way to deal with this problem is to use fixed point, and simply do your calculations using integers and only convert on final display. Ugly, but your code will run anywhere.

I look forward to your response.


#3

Hi,

I actually just need a way to use Java double. which is IEEE754. I see there are some different formats. Here is my usage:

In java, I have some Java double I need to write into a record. I call execute UDF on this record. The lua code I’ve written reads a map() that contains a double from a bin and another map() from an LDT LLIST. It then uses the input double along with the numbers retrieved from the record and runs a small calculation then puts the new value in a map() and appends that onto the LLIST.

What I was thinking about, was making a C module for Lua that does some float/double to string packing like string.pack does. But I am new to this and haven’t determined if that is possible. My thoughts were that this would not limit my users on precision as I would if I used fixed point. But I can see that its so much simpler so I may just do that.


#4

My previous comment about lua map() and list() was just that. I noticed, if I create a map() object and put into it a lua number that has a fractional component, and then immediately read that number back out, I seem to get 0. As well as, If I call the udf with a java double: Value.get(myDouble), that value is 0 when the UDF executes.

So even If i did some conversions going into or out of the database, I still have to do more conversions to get into and out of lua.

But, it seems like this might be a lot of trouble, like you say, supporting many different languages.


#5

I’m sorry I misread your post about “fractional” data types, you’re just interested in floats, and as you say, other forum posts state we’re working on it (true enough). Part of that work will be to add support to the entire UDF system. We can’t really do the UDF arguments & list() and map() types before laying the fundamental double type. Sorry this won’t be a quick update.

On a related note, was there a question about async execution of UDFs? I saw you need execution to be async, but I didn’t get if there’s a question.


#6

No question about the async. Just that I feel it is important for the Java client to have that ability.


#7

Would adding an asynchronous execute(key) method for a single record suffice?

Asynchronous execute(key) is relatively easy to implement because the server returns after the command has completed.

Asynchronous execute(statement) for a group of records matching the statement filter would be much harder to implement. In this case, the server responds asynchronously (returns before command completed on all servers), so the client must poll the servers for true command completion. Asynchronous polling is not currently supported because it requires running asynchronous tasks at some future time (which is not currently allowed in AsyncClient) without blocking.


#8

Yes, async execute on key is exactly what I needed. At the moment I dont have any need of execute with a statement.

I see its been added. that is great.


#9

Indeed, the feature " Support asynchronous execute() for single records. Added asynchronous user defined function call example" is in the Java 3.1.1 release that came out just a few days ago.