We use Aerospike C# client API and since we started to use it, I noticed that our application’s (.net app) threads number duplicated many times and it keep growing (we usually utilize around 250 thread but now we are utilizing around 1,700 thread!!).
Is this an expected behaviour? or there is something wrong? and what is the healthy expected growth in the number of threads?
please advice?
Please note that we just use below APIs from Aerospike client API:
Out of the commands listed, AerospikeClient only spawns threads for “Batch Get” (to read records from multiple nodes in parallel). These threads are retrieved from the standard C# ThreadPool (ThreadPool.QueueUserWorkItem()). When ThreadPool finishes running the task, the thread is placed back into the pool. You can limit ThreadPool size via ThreadPool.SetMaxThreads().
Also, make sure AerospikeClient is only instantiated once and this instance should be shared across all threads that you application generates.
Keeping 1700 cached threads does seem excessive, but AerospikeClient does not control this. The C# runtime library class ThreadPool determines how many threads remain cached. You can control the max threads with the following command at your application startup.
int workerThreads;
int portThreads;
ThreadPool.GetMaxThreads(out workerThreads, out portThreads);
ThreadPool.SetMaxThreads(, portThreads);
Perhaps your platform or .NET framework ThreadPool version is not reusing threads properly?
Are you observing performance or memory consumption problems when the thread count is high?
How many concurrent threads (generated by your application) are running database commands?
Is your application using ThreadPool directly for some other reason?
You may want to replace “batch get” with multiple record get calls and observe the results. There are cases where single records gets will outperform a single “batch get” (especially with small batch requests).
Batch get maps keys to server nodes and sends requests to those nodes in parallel threads spawned from the transaction thread. The responses must be handled and assembled in a thread safe manner. This does add some overhead.
Single record gets can run in the same transaction thread because the request is sent to a single node.
Therefore, small batches and thread resource limited systems or limited thread pools can perform faster using single record gets.
Each system is different, so it pays to benchmark the performance differences.