How to throttle scan UDFs
Running scan UDFs can impact system resources and transaction queues and may require throttling their executions.
scan-max-udf-transactions setting can be tuned for such a purpose.
scan-max-udf-transactions setting can be tuned to globally throttle UDF executions.
This setting is dynamic and controls the number of transactions that is sent per running UDFs to the transaction-queues.
A throttling will occur when the number of active transactions for a particular UDF job is greater or equal to scan-max-udf-transactions.
This throttling will continue until the transaction has finished processing and the
udf-active (number of active transactions per scan UDF) count for that job has decremented below
udf-active count can be checked by running an jobs info call while the scan UDF is running:
Command to run:
asinfo -v 'jobs:module=scan'
$ asinfo -v 'jobs:module=scan' module=scan:trid=14506333510584022897:job-type=background-udf:ns=test:set=testset:priority=2:status=done(ok):job-progress=100.00:run-time=38:time-since-done=20095:recs-read=20:net-io-bytes=30:udf-filename=bg_scan_udf:udf-function=test_bin_add_1000:udf-active=0:udf-success=20:udf-failed=0
Monitor the udf-active number and set
asinfo -v "set-config:context=service;scan-max-udf-transactions=24"
The throttling is independent of the number of scan udfs.
udf-active: refers to the number of transactions that were sent to the transaction-queues but have not yet finished processing.
Throttling can also be implemeted at the lua level by implementing a sleep function:
http://lua-users.org/wiki/SleepFunction (os.sleep() )
UDF THROTTLING SCAN LUA SCAN-UDF