AWS thread and block size setting recommendations


#1

Hi,

There are a number of thread settings in aerospike, the most important ones probably service-threads, transaction-threads-per-queue, and migrate-threads. In general, for some are recommended to be equal to the number of cores, while others are static (e.g. migrate-threads is 1 by default). A few questions about that:

  • in aws cores (aka vcpu-s) are actually virtual hyper threads so a 8 “core” box usually means a 4 core hyperthreaded box – do you recommend using the virtual core numbers or half of that?
  • should the total number of threads configured add up to the total number of cores or should they be overallocated - e.g. do i want 1 migrate thread + 7 service threads on a 8 core box?

Thanks!


#2

I will look into this and let you know what I find


#3

I spoke to a senior developer on this one and the response is that it is the number of visible cores that we are concerned with. So in your 4 core with hyperthreading example, we would look at that as an 8 core box.

The developer did comment though that this is only really of concern if you are at the extreme limit of performance benchmarking and that we would not advice you to run at that level of performance in a production scenario.

Does this clarify?


#4

@Ben_Bates, gotcha - thanks!


#5

@Ben_Bates,

If I can piggy back on this thread and ask about the recommended write-block-size on aws (and i2 instances in particular)?

Thanks!


#6

If you are using EBS and data in memory we would recommend 256k as Amazon writes to EBS in these multiples.

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-io-characteristics.html

If you have data on ephemeral devices there are no such restrictions 1Mb should be fine.


#7

@Ben_Bates - great, thanks!