Performance Degrades after restart and during migration


#1

Original Postby hqwider » Tue Jul 08, 2014 10:31 pm

Hi,

We have a two node cluster in an environment that has been launched recently. Currently we are using two namespaces configured “in-memory”.

We notice that after restarting Aerospike service on one of the servers, our application’s performance noticeably degrades for 5-6 minutes which is equivalent to the duration of migration although one server can comfortably handle the requests on its own.

Can you advice how to solve this? hqwider

Posts: 3 Joined: Mon Jul 07, 2014 8:30 am


#2

Postby devops02 » Wed Jul 30, 2014 1:37 pm

Hi,

I was wondering if you still need assistance with this issue? A few questions first, what version of Aerospike are you using? Are you seeing the latency from the reads or the write? After talking to the developers, there are a few ways to try and get your degrades down to a more acceptable level. First we can try to increase your fabric worker threads. What this does is it will increase the fabric threads for inter-node communication which should bring down the degrades. This Parameter is not dynamic so we will have to restart your node afterwards. Change the Fabric-worker under the Service Stanza from the default value to 24 and then restart.

If this doesn’t meet your expectation please let us know and we can try something else.

Jerry


#3

Postby hqwider » Thu Jul 31, 2014 5:09 am

Hi,

We are planning a Benchmark test using the Java Client benchmark tool sometime next week to make sure it is not related to the network environment or the application side. Ill keep you updated.