How to use ss or netstat to troubleshoot network issues

The Aerospike Knowledge Base has moved to https://support.aerospike.com. Content on https://discuss.aerospike.com is being migrated to either https://support.aerospike.com or https://docs.aerospike.com. Maintenance on articles stored in this repository ceased on December 31st 2022 and this article may be stale. If you have any questions, please do not hesitate to raise a case via https://support.aerospike.com.

FAQ How to use ss or netstat to troubleshoot network issues

Context

Using ss or netstat to trouble network issues in particular connection states. There are many network tools like sar or tcpdump but this article focuses on ss and netstat. The netstat command is actually deprecated and replaced by ss.

Method

ss -an
State       Recv-Q Send-Q                                                          Local Address:Port                                                            Peer Address:Port 
LISTEN      0      128                                                                        :::22                                                                        :::*     
LISTEN      0      128                                                                         *:22                                                                         *:*     
LISTEN      0      128                                                                         *:3000                                                                       *:*     
LISTEN      0      128                                                                         *:3001                                                                       *:*     
LISTEN      0      100                                                                       ::1:25                                                                        :::*     
LISTEN      0      100                                                                 127.0.0.1:25                                                                         *:*     
LISTEN      0      128                                                                         *:3003                                                                       *:*     
ESTAB       0      0                                                             192.168.120.219:22                                                           192.168.106.187:52691 
SYN-SENT    0      1                                                             192.168.120.219:41104                                                         104.36.112.167:3000  
To show PID (-p) and extents (-e)
netstat -pen --inet 
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State       User       Inode      PID/Program name   
tcp        0     40 192.168.120.219:22          192.168.106.187:52691       ESTABLISHED 0          739808     25870/sshd          
tcp        0      1 192.168.120.219:41552       104.36.112.169:3000         SYN_SENT    498        761695     2462/asd    
                                           
ss -anp
State       Recv-Q Send-Q                                                          Local Address:Port                                                            Peer Address:Port 
LISTEN      0      128                                                                        :::22                                                                        :::*      users:(("sshd",1498,4))
LISTEN      0      128                                                                         *:22                                                                         *:*      users:(("sshd",1498,3))
LISTEN      0      128                                                                         *:3000                                                                       *:*      users:(("asd",2462,71))
LISTEN      0      128                                                                         *:3001                                                                       *:*      users:(("asd",2462,56))
LISTEN      0      100                                                                       ::1:25                                                                        :::*      users:(("master",1577,13))
LISTEN      0      100                                                                 127.0.0.1:25                                                                         *:*      users:(("master",1577,12))
LISTEN      0      128                                                                         *:3003                                                                       *:*      users:(("asd",2462,84))
ESTAB       0      256                                                           192.168.120.219:22                                                           192.168.106.187:52691  users:(("sshd",25870,3),("sshd",25873,3))
SYN-SENT    0      1                                                             192.168.120.219:54988                                                         104.36.112.171:3000  
Watch for state change on syn-sent
watch -n 1 "ss -n -t4 state syn-sent"
Check default ports used by Aerospike
ss -nt '( dst :3000 or dst :3001 or dst :3002 or dst :3003 )'
State      Recv-Q Send-Q                                      Local Address:Port                                        Peer Address:Port 
ESTAB      0      0                                               127.0.0.1:39618                                          127.0.0.1:3000  
ESTAB      0      0                                           192.168.33.13:41276                                      192.168.33.16:3001  
ESTAB      0      0                                           192.168.33.13:41274                                      192.168.33.16:3001  
ESTAB      0      0                                           192.168.33.13:55236                                      192.168.33.16:3000  
ESTAB      0      0                                               127.0.0.1:39620                                          127.0.0.1:3000  
SYN-SENT   0      1                                         192.168.120.219:48444                                      10.10.112.167:3000  

The last line SYN-SENT implies that there is no reply from remote server (configured for destination XDR).

connection states

State Description
LISTEN accepting connections
ESTABLISHED connection up and passing data
SYN_SENT TCP; session has been requested by us; waiting for reply from remote endpoint
SYN_RECV TCP; session has been requested by a remote endpoint for a socket on which we were listening
LAST_ACK TCP; our socket is closed; remote endpoint has also shut down; we are waiting for a final acknowledgement
CLOSE_WAIT TCP; remote endpoint has shut down; the kernel is waiting for the application to close the socket
TIME_WAIT TCP; socket is waiting after closing for any packets left on the network
CLOSED socket is not being used
CLOSING TCP; our socket is shut down; remote endpoint is shut down; not all data has been sent
FIN_WAIT1 TCP; our socket has closed; we are in the process of tearing down the connection
FIN_WAIT2 TCP; the connection has been closed; our socket is waiting for the remote endpoint to shut down

Notes

Keywords

network socket netstat

Timestamp

11/17/2016