The netstat command on Solaris is a fantastic gateway to all your TCP statistics of your system.

You would have used the netstat -an command in the past. However the netstat -s -P tcp can give you up-to-date statistics about what your TCP stack has been up to…

$ nestat -s -P tcp
TCP
  tcpRtoAlgorithm     =     4     tcpRtoMin           =   400
  tcpRtoMax           = 60000     tcpMaxConn          =    -1
  tcpActiveOpens      =7624114    tcpPassiveOpens     =7084624
  tcpAttemptFails     =1896763    tcpEstabResets      =193326
  tcpCurrEstab        =    74     tcpOutSegs          =21843389688
  tcpOutDataSegs      =3328351751 tcpOutDataBytes     =3235412917
  tcpRetransSegs      =41967918   tcpRetransBytes     =2212890976
  tcpOutAck           =853704065  tcpOutAckDelayed    =247961090
  tcpOutUrg           =     1     tcpOutWinUpdate     =477772
  tcpOutWinProbe      = 12412     tcpOutControl       =33045410
  tcpOutRsts          =5285917    tcpOutFastRetrans   =  8210
  tcpInSegs           =11491393189
  tcpInAckSegs        =1158661729 tcpInAckBytes       =1102332654
  tcpInDupAck         =142544351  tcpInAckUnsent      =     0
  tcpInInorderSegs    =1884725886 tcpInInorderBytes   =1286627563
  tcpInUnorderSegs    =1912668    tcpInUnorderBytes   =2409325298
  tcpInDupSegs        =34780066   tcpInDupBytes       =1415828491
  tcpInPartDupSegs    =  3626     tcpInPartDupBytes   =1693311
  tcpInPastWinSegs    =2269167    tcpInPastWinBytes   =126796354
  tcpInWinProbe       =  1057     tcpInWinUpdate      = 11758
  tcpInClosed         =3426232    tcpRttNoUpdate      =445365299
  tcpRttUpdate        =706469685  tcpTimRetrans       =1305409
  tcpTimRetransDrop   =  2328     tcpTimKeepalive     = 75510
  tcpTimKeepaliveProbe= 28677     tcpTimKeepaliveDrop =   193
  tcpListenDrop       =    11     tcpListenDropQ0     =     0
  tcpHalfOpenDrop     =     0     tcpOutSackRetrans   =40485027

As you can see this is a head of data that you now have at your finger tips, however the man pages have no explanation what each of the items are in this list.

 $ man netstat
 -s
     Show per-protocol statistics.

The source code has a little bit more data (http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/inet/mib2.h#mib2_tcp) to get to started…

Sockets

tcpRtoAlgorithm: algorithm used for transmit timeout value

tcpRtoMin: minimum retransmit timeout (ms)

tcpRtoMax: maximum retransmit timeout (ms)

tcpMaxConn: maximum # of connections supported

tcpActiveOpens: # of direct transitions CLOSED -> SYN-SENT

tcpPassiveOpens: # of direct transitions LISTEN -> SYN-RCVD

tcpAttemptFails: # of direct SIN-SENT/RCVD -> CLOSED/LISTEN

tcpEstabResets: # of direct ESTABLISHED/CLOSE-WAIT -> CLOSED

tcpCurrEstab: # of connections ESTABLISHED or CLOSE-WAIT

tcpInSegs: total # of segments recv’d

tcpOutSegs: total # of segments sent

tcpRetransSegs: total # of segments retransmitted

tcpConnTableSize: Size of tcpConnEntry_t in ip

tcpOutRsts: # of segments sent with RST flag

Sender

tcpOutDataSegs: total # of data segments sent

tcpOutDataBytes: total # of bytes in data segments sent

tcpRetransBytes: total # of bytes in segments retransmitted

tcpOutAck: total # of acks sent

tcpOutAckDelayed: total # of delayed acks sent

tcpOutUrg: total # of segments sent with the urg flag on

tcpOutWinUpdate: total # of window updates sent

tcpOutWinProbe: total # of zero window probes sent

tcpOutControl: total # of control segments sent (syn, fin, rst)

tcpOutFastRetrans: total # of segments sent due to “fast retransmit”

Receiver

tcpInAckSegs: total # of ack segments received

tcpInAckBytes: total # of bytes acked

tcpInDupAck: total # of duplicate acks

tcpInAckUnsent: total # of acks acking unsent data

tcpInDataInorderSegs: total # of data segments received in order

tcpInDataInorderBytes: total # of data bytes received in order

tcpInDataUnorderSegs: total # of data segments received out-of-order

tcpInDataUnorderBytes: total # of data bytes received out-of-order

tcpInDataDupSegs: total # of complete duplicate data segments received

tcpInDataDupBytes: total # of bytes in the complete duplicate data segments received

tcpInDataPartDupSegs: total # of partial duplicate data segments received

tcpInDataPartDupBytes: total # of bytes in the partial duplicate data segments received

tcpInDataPastWinSegs: total # of data segments received past the window

tcpInDataPastWinBytes: total # of data bytes received part the window

tcpInWinProbe: total # of zero window probes received

tcpInWinUpdate: total # of window updates received

tcpInClosed: total # of data segments received after the connection has closed

Others

tcpRttNoUpdate: total # of failed attempts to update the rtt estimate

tcpRttUpdate: total # of successful attempts to update the rtt estimate

tcpTimRetrans: total # of retransmit timeouts

tcpTimRetransDrop: total # of retransmit timeouts dropping the connection

tcpTimKeepalive: total # of keepalive timeouts

tcpTimKeepaliveProbe: total # of keepalive timeouts sending a probe

tcpTimKeepaliveDrop: total # of keepalive timeouts dropping the connection

tcpListenDrop: total # of connections refused due to backlog full on listen

tcpListenDropQ0: total # of connections refused due to half-open queue (q0) full

tcpHalfOpenDrop: total # of connections dropped from a full half-open queue (q0)

tcpOutSackRetransSegs: total # of retransmitted segments by SACK retransmission

tcp6ConnTableSize: Size of tcp6ConnEntry_t

Getting Netstat output into Graphite

What would blog post able statistics be if you can’t graph them into graphite using python… ?

from sysadm import graph
data={}
for line in os.popen("/usr/bin/netstat -s -P tcp").readlines():
    netstat = line.replace('=','').replace('TCP','').split()
    if len(netstat) == 2:
        (k,v) = netstat
        data["tcp.%s" % k] = float(v)
        continue
    if len(netstat) == 4:
        (k,v,k2,v2) = netstat
        data["tcp.%s" % k2] = float(v2)
        data["tcp.%s" % k] = float(v)
        continue
graph.logBulkData(data)