The Aerospike Knowledge Base has moved to https://support.aerospike.com. Content on https://discuss.aerospike.com is being migrated to either https://support.aerospike.com or https://docs.aerospike.com. Maintenance on articles stored in this repository ceased on December 31st 2022 and this article may be stale. If you have any questions, please do not hesitate to raise a case via https://support.aerospike.com.
Benchmarking Throughput and Packet Count With Iperf3
Understanding the performance of a systems resources is important for managing the capacity and operations of Aerospike. This is a quick reference to understand the performance of your network resources. Included in this article is a benchmark setup for both throughput benchmarking and packet count benchmarking. Many benchmarks focus just on throughput. However it is specially important to understand the network packet count limits in a cloud environment like AWS because of the network throttling done by the hypervisors.
Installing Iperf3 on Centos and Ubuntu
You will need to install iperf3 on both of the servers used in the benchmark.
Install Iperf3 on Centos:
sudo yum install iperf3
Install iperf3 on Ubuntu 14.04
sudo apt-get install iperf3
Throughput Benchmark
Server Setup:
sudo iperf3 -s -p 3000
Client Setup:
sudo iperf3 -c 10.0.0.31 -i 1 -t 3600 -V -p 3000 -b 1000000000000
Where
-p port number
-c server IP address
-t is time for the test in seconds
-i option is the interval for output.
-b is the bandwidth in bits per second. This number should be set to the maximum bandwidth for the network interface.
Once the server and client are setup and running, you can use the scripts below to monitor the bandwidth utilization and the packet counts. Many monitoring tools will supply throughput statistics but no packet counts. The scripts below will supply packet count monitoring for the benchmark. When your test has finished running you will want to start the test a second time. However you will want to run the original server machine as a client and the orignal client machine as a server.
Packet Count Benchmark
Server Setup:
sudo iperf3 -s -p 3000
Client Setup:
sudo iperf3 -c 10.0.0.31 -i 1 -t 3600 -V -p 3000 -b 1000000000 -l 100 -M 89
-l option is the buffer size. Actually this doesn't matter too much.
-M option is the MTU size. The default is 1500. Here we set it low to get a high packet count. The throughput is set high.
-p port number
-c server IP address
-t is time for the test in seconds
-i option is the interval for output.
-b is the bandwidth in bits per second. This number should be set to the maximum bandwidth for the network interface.
Note: The –l option is the buffer size and the -M options are only used in the packet count test.
When your test has finished running you will want to start the test a second time. However you will want to run the original server machine as a client and the orignal client machine as a server.
Sample iperf3 Client output
sudo iperf3 -c 10.0.0.212 -i 1 -t 3600 -V -p 3000 -b 1000000000000
iperf 3.0.7
Linux ip-10-0-0-74 3.13.0-74-generic #118-Ubuntu SMP Thu Dec 17 22:52:10 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
Time: Mon, 28 Mar 2016 21:54:13 GMT
Connecting to host 10.0.0.212, port 3000
Cookie: ip-10-0-0-74.1459202053.406100.67077
TCP MSS: 8949 (default)
[ 4] local 10.0.0.74 port 51752 connected to 10.0.0.212 port 3000
Starting Test: protocol: TCP, 1 streams, 131072 byte blocks, omitting 0 seconds, 3600 second test
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 422 MBytes 3.54 Gbits/sec 23 953 KBytes
[ 4] 1.00-2.00 sec 82.8 MBytes 694 Mbits/sec 0 1.10 MBytes
[ 4] 2.00-3.00 sec 82.9 MBytes 695 Mbits/sec 0 1.25 MBytes
[ 4] 3.00-4.00 sec 82.9 MBytes 695 Mbits/sec 0 1.38 MBytes
[ 4] 4.00-5.00 sec 81.8 MBytes 686 Mbits/sec 1 1.21 MBytes
[ 4] 5.00-6.00 sec 83.9 MBytes 704 Mbits/sec 0 1.27 MBytes
[ 4] 6.00-7.00 sec 82.8 MBytes 694 Mbits/sec 0 1.41 MBytes
[ 4] 7.00-8.00 sec 82.9 MBytes 695 Mbits/sec 0 1.48 MBytes
[ 4] 8.00-9.00 sec 82.8 MBytes 694 Mbits/sec 1 1.19 MBytes
[ 4] 9.00-10.00 sec 82.9 MBytes 695 Mbits/sec 0 1.36 MBytes
[ 4] 10.00-11.00 sec 82.8 MBytes 694 Mbits/sec 0 1.45 MBytes
[ 4] 11.00-12.00 sec 82.9 MBytes 695 Mbits/sec 1 1.09 MBytes
[ 4] 12.00-13.00 sec 82.9 MBytes 695 Mbits/sec 0 1.29 MBytes
[ 4] 13.00-14.00 sec 82.8 MBytes 694 Mbits/sec 0 1.42 MBytes
[ 4] 14.00-15.00 sec 82.9 MBytes 695 Mbits/sec 0 1.48 MBytes
[ 4] 15.00-16.00 sec 82.8 MBytes 694 Mbits/sec 1 1.21 MBytes
[ 4] 16.00-17.00 sec 82.9 MBytes 695 Mbits/sec 0 1.37 MBytes
[ 4] 17.00-18.00 sec 82.8 MBytes 694 Mbits/sec 0 1.46 MBytes
[ 4] 18.00-19.00 sec 82.9 MBytes 695 Mbits/sec 1 1.11 MBytes
[ 4] 19.00-20.00 sec 82.9 MBytes 695 Mbits/sec 0 1.31 MBytes
[ 4] 20.00-21.00 sec 82.8 MBytes 694 Mbits/sec 0 1.43 MBytes
[ 4] 21.00-21.78 sec 64.5 MBytes 695 Mbits/sec 0 1.48 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
Test Complete. Summary Results:
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-21.78 sec 2.09 GBytes 825 Mbits/sec 28 sender
[ 4] 0.00-21.78 sec 0.00 Bytes 0.00 bits/sec receiver
CPU Utilization: local/sender 4.6% (0.9%u/3.7%s), remote/receiver 0.0% (0.0%u/0.0%s)
iperf3: interrupt - the client has terminated
Sample iperf3 Server Output
sudo iperf3 -s -p 3000
-----------------------------------------------------------
Server listening on 3000
-----------------------------------------------------------
Accepted connection from 10.0.0.74, port 51751
[ 5] local 10.0.0.212 port 3000 connected to 10.0.0.74 port 51752
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-1.00 sec 411 MBytes 3.45 Gbits/sec
[ 5] 1.00-2.00 sec 82.8 MBytes 695 Mbits/sec
[ 5] 2.00-3.00 sec 82.8 MBytes 695 Mbits/sec
[ 5] 3.00-4.00 sec 82.8 MBytes 695 Mbits/sec
[ 5] 4.00-5.00 sec 82.8 MBytes 695 Mbits/sec
[ 5] 5.00-6.00 sec 82.8 MBytes 695 Mbits/sec
[ 5] 6.00-7.00 sec 82.8 MBytes 695 Mbits/sec
[ 5] 7.00-8.00 sec 82.8 MBytes 695 Mbits/sec
[ 5] 8.00-9.00 sec 82.8 MBytes 695 Mbits/sec
[ 5] 9.00-10.00 sec 82.8 MBytes 695 Mbits/sec
[ 5] 10.00-11.00 sec 82.8 MBytes 695 Mbits/sec
[ 5] 11.00-12.00 sec 82.8 MBytes 695 Mbits/sec
[ 5] 12.00-13.00 sec 82.8 MBytes 695 Mbits/sec
[ 5] 13.00-14.00 sec 82.8 MBytes 695 Mbits/sec
[ 5] 14.00-15.00 sec 82.8 MBytes 695 Mbits/sec
[ 5] 15.00-16.00 sec 82.8 MBytes 695 Mbits/sec
[ 5] 16.00-17.00 sec 82.8 MBytes 695 Mbits/sec
[ 5] 17.00-18.00 sec 82.8 MBytes 695 Mbits/sec
[ 5] 18.00-19.00 sec 82.8 MBytes 695 Mbits/sec
[ 5] 19.00-20.00 sec 82.8 MBytes 695 Mbits/sec
[ 5] 20.00-21.00 sec 82.8 MBytes 695 Mbits/sec
[ 5] 20.00-21.00 sec 82.8 MBytes 695 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-21.00 sec 0.00 Bytes 0.00 bits/sec sender
[ 5] 0.00-21.00 sec 2.09 GBytes 853 Mbits/sec receiver
iperf3: the client has terminated
-----------------------------------------------------------
Server listening on 3000
-----------------------------------------------------------
Sample Output from Scripts
Packet Count
./nettp eth0
TX eth0: 9702 pkts/s RX eth0: 4558 pkts/s
TX eth0: 9745 pkts/s RX eth0: 4479 pkts/s
TX eth0: 9738 pkts/s RX eth0: 4493 pkts/s
TX eth0: 9693 pkts/s RX eth0: 4547 pkts/s
TX eth0: 9752 pkts/s RX eth0: 4497 pkts/s
TX eth0: 9745 pkts/s RX eth0: 4492 pkts/s
TX eth0: 9738 pkts/s RX eth0: 4488 pkts/s
Throughput
./nettp eth0
TX eth0: 85845 kB/s RX eth0: 289 kB/s
TX eth0: 85783 kB/s RX eth0: 289 kB/s
TX eth0: 85660 kB/s RX eth0: 290 kB/s
TX eth0: 85369 kB/s RX eth0: 295 kB/s
TX eth0: 85783 kB/s RX eth0: 288 kB/s
TX eth0: 85721 kB/s RX eth0: 288 kB/s
Network Packet Count Script
#!/bin/bash
INTERVAL="1" # update interval in seconds
if [ -z "$1" ]; then
echo
echo usage: $0 [network-interface]
echo
echo e.g. $0 eth0
echo
echo shows packets-per-second
exit
fi
IF=$1
while true
do
R1=`cat /sys/class/net/$1/statistics/rx_packets`
T1=`cat /sys/class/net/$1/statistics/tx_packets`
sleep $INTERVAL
R2=`cat /sys/class/net/$1/statistics/rx_packets`
T2=`cat /sys/class/net/$1/statistics/tx_packets`
TXPPS=`expr $T2 - $T1`
RXPPS=`expr $R2 - $R1`
echo "TX $1: $TXPPS pkts/s RX $1: $RXPPS pkts/s"
done
Network Bandwidth Script
#!/bin/bash
INTERVAL="1" # update interval in seconds
if [ -z "$1" ]; then
echo
echo usage: $0 [network-interface]
echo
echo e.g. $0 eth0
echo
exit
fi
IF=$1
while true
do
R1=`cat /sys/class/net/$1/statistics/rx_bytes`
T1=`cat /sys/class/net/$1/statistics/tx_bytes`
sleep $INTERVAL
R2=`cat /sys/class/net/$1/statistics/rx_bytes`
T2=`cat /sys/class/net/$1/statistics/tx_bytes`
TBPS=`expr $T2 - $T1`
RBPS=`expr $R2 - $R1`
TKBPS=`expr $TBPS / 1024`
RKBPS=`expr $RBPS / 1024`
echo "TX $1: $TKBPS kB/s RX $1: $RKBPS kB/s"
done