Information about threads created by the server

[mvk]

Hi all:

I am running Aerospike server in a pod with single core and default configuration as far as threading is concerned. I see 88 threads created by the server. I tried changing “service-threads” configuration. If I set it to anything more than 1, the difference gets added to 88 (i.e. if I set service-threads to 2, number of threads become 89). This shows that there is only 1 service thread out of 88 threads. Can someone explain or point me to documentation where I can understand why so many threads are generated for a single core pod?

[meher]

I don’t think there is a comprehensive doc that lists all the different threads. I am curious, though, on what those different metrics showed up for you:

• threads_detached
• threads_joinable
• threads_pool_active
• threads_pool_total

(under asadm, show stat like threads should give you those numbers).

There are definitely threads other than the service threads and I am not sure which ones and how many would be created by default and some of those wouldn’t be of any use in a single node cluster (fabric, migration are a couple that come up from the top of my head). I am thinking of query and batch related threads amongst other (XDR related ones for example).

[mvk]

Please find the stats that you asked for.

Admin> show stat like threads
~~~~~~~~~~~~Service Statistics (2023-09-20 05:49:29 UTC)~~~~~~~~~~~
...
threads_detached   |59              |59
threads_joinable   |8               |8
threads_pool_active|19              |19
threads_pool_total |19              |19
Number of rows: 5

~~~~~~~~~~test Namespace Statistics (2023-09-20 05:49:29 UTC)~~~~~~~~~~
...
nsup-threads           |1               |1
single-query-threads   |4               |4
truncate-threads       |4               |4
xdr-tomb-raider-threads|1               |1
Number of rows: 5

And one small correction in my original post. Server is creating 87 threads, not 88.

[meher]

Thanks, so all the threads in the pool seem active. I wasn’t sure whether that would be the case by default and I believe the thread pool would then also potentially grow. I just ran this on the sandbox and here is the output:

Admin> show stat like thread
~~~~~~~~~~~~~~~~~~~~Service Statistics (2023-09-20 17:04:01 UTC)~~~~~~~~~~~~~~~~~~~
Node               |jupyter-aerospike-2dexamp-2dctive-2dnotebooks-2drxrzz7i5:3000
threads_detached   |72                                                           
threads_joinable   |10                                                           
threads_pool_active|29                                                           
threads_pool_total |29                                                           
Number of rows: 5

~~~~~~~~~~~~~~~~~sandbox Namespace Statistics (2023-09-20 17:04:01 UTC)~~~~~~~~~~~~~~~~
Node                   |jupyter-aerospike-2dexamp-2dctive-2dnotebooks-2drxrzz7i5:3000
nsup-threads           |1                                                            
single-query-threads   |4                                                            
truncate-threads       |4                                                            
xdr-tomb-raider-threads|1                                                            
Number of rows: 5

~~~~~~~~~~~~~~~~~~test Namespace Statistics (2023-09-20 17:04:01 UTC)~~~~~~~~~~~~~~~~~~
Node                   |jupyter-aerospike-2dexamp-2dctive-2dnotebooks-2drxrzz7i5:3000
nsup-threads           |1                                                            
single-query-threads   |4                                                            
truncate-threads       |4                                                            
xdr-tomb-raider-threads|1                                                            
Number of rows: 5

The sandbox does kick off a workload to populate records and has two namespaces which would explain the higher numbers.

Any reason you were looking for the details on the number of threads? As mentioned, I would assume a variety of threads for the asd process and not all of them being configurable.

[mvk]

Thought that these are too many threads for a single core. Hence, wanted to make sure that there is no misconfiguration from my side.

Thanks for confirming that this is normal Aerospike behaviour.