Network throughput issues with asd running

You mentioned the ping time going crazy as soon as you sent traffic. You went from a cluster of 4 nodes to a cluster of 2 nodes. Its possible that a network bandwidth is being reached on these aws instances when more connections are hitting two nodes instead of 4 nodes. Also could be a bandwidth issue in the aws shared link between clients and servers.

You may be able to use iperf tool to test bandwidth between a client and nodes on the new cluster. Please see:

On the server you may also be able to run

sar -n DEV

to see network utilization.

on aerospike side the number of open file descriptors can be set through proto-fd-max. Was there any warning in aerospike logs for connections limits being reached?

Did you go back to using 2 AZs or are the 4 new nodes on the same AZ?