The Aerospike Knowledge Base has moved to https://support.aerospike.com. Content on https://discuss.aerospike.com is being migrated to either https://support.aerospike.com or https://docs.aerospike.com. Maintenance on articles stored in this repository ceased on December 31st 2022 and this article may be stale. If you have any questions, please do not hesitate to raise a case via https://support.aerospike.com.
How to throttle scan UDFs
Context
Running scan UDFs can impact system resources and transaction queues and may require throttling their executions.
The scan-max-udf-transactions
setting can be tuned for such a purpose.
Method
The scan-max-udf-transactions
setting can be tuned to globally throttle UDF executions.
This setting is dynamic and controls the number of transactions that is sent per running UDFs to the transaction-queues.
A throttling will occur when the number of active transactions for a particular UDF job is greater or equal to scan-max-udf-transactions.
This throttling will continue until the transaction has finished processing and the udf-active
(number of active transactions per scan UDF) count for that job has decremented below scan-max-udf-transactions
.
The udf-active
count can be checked by running an jobs info call while the scan UDF is running:
Command to run:
asinfo -v 'jobs:module=scan'
Example:
$ asinfo -v 'jobs:module=scan'
module=scan:trid=14506333510584022897:job-type=background-udf:ns=test:set=testset:priority=2:status=done(ok):job-progress=100.00:run-time=38:time-since-done=20095:recs-read=20:net-io-bytes=30:udf-filename=bg_scan_udf:udf-function=test_bin_add_1000:udf-active=0:udf-success=20:udf-failed=0
Monitor the udf-active number and set scan-max-udf-transactions
accordingly:
asinfo -v "set-config:context=service;scan-max-udf-transactions=24"
Notes
-
The throttling is independent of the number of scan udfs.
-
udf-active: refers to the number of transactions that were sent to the transaction-queues but have not yet finished processing.
Keywords
UDF THROTTLING SCAN LUA SCAN-UDF
Timestamp
07/16/2018