Network Diagnosis tools and utils
Network Diagnosis tools and utils.
Dear AO. Anyone who is an admin on any network should be familiar with a few basic tools and utils that will help make administration a bit easier. Today I would like to cover some of these tools used and how to use them. I hope this comes in handy to some of you and off course this is a must for any newbie who would like to maintain any network. Like always, please excuse any typos and/or bad English grammar. My tutorial is as always based and tested for SuSE Linux, but I’m sure that most of it will work on any *nix system.
Diagnosis Tools.
Any network out there consists of many different network components, such as routers, hubs, bridges, etc which are responsible for the correct forwarding of data to the correct locations. The more components you add to any network, the more installation and configuration is needed to maintain it correctly, and there is an increase of failures and errors in many parts of the network. Incase of an error or a failure it is vital to be able to analyze and fix it ASAP. In order to do that successfully you need the basic tools that are present on most systems. I have seen many people looking for external 3rd party software to do simple tasks, since they did not now that very simple and basic tools that are normally already present will do the trick.
1 – ping
Ping is a simple diagnostics tool to test and study a network connection. It is present on any host which has a TCP/IP stack, and should always be used as the first choice to examine network errors or failures.
The tool ping allows in simple ways to test a connection between 2 hosts in the same network or on different networks. If the command ping is successful, it means that there is a correct physical as well as a logical connection between 2 hosts in any network. (Unless if there is a firewall interfering somewhere in between.)
The easiest way to use ping is:
ping <targethost or ip>
That will send an ICMP datagram to a remote host with the message “echo request”. If the target host can receive this ICMP datagram, then it will try to answer with an ICMP datagram called “echo reply”. The sending of an ICMP protocol has the advantage that it does not use any protocols which are on a higher level such as TCP or UDP, so any misconfiguration within the higher protocols will not cause any errors based on the connection between 2 hosts. A simple example of what a ping along with its reply looks like is:
instronics@ns1:~> ping ns.esn.gr
PING ns.esn.gr (212.89.163.211) from 212.89.163.212 : 56(84) bytes of data.
64 bytes from ns.esn.gr (212.89.163.211): icmp_seq=1 ttl=255 time=2.337 msec
64 bytes from ns.esn.gr (212.89.163.211): icmp_seq=2 ttl=255 time=1.361 msec
64 bytes from ns.esn.gr (212.89.163.211): icmp_seq=3 ttl=255 time=968 usec
64 bytes from ns.esn.gr (212.89.163.211): icmp_seq=4 ttl=255 time=664 usec
--- ns.esn.gr ping statistics ---
4 packets transmitted, 4 received, 0% loss, time 3046ms
rtt min/avg/max/mdev = 0.664/1.332/2.337/0.631 ms
instronics@ns1:~>
As you can see above, 4 ICMP datagrams were sent to ns.esn.gr (echo request), which were all answered with an (echo reply). The information retrieved here provides us with the following:
A – The IP-address from our target host resolved from ns.esn.gr to 212.89.196.211
B – The length of an ICMP-echo-request-Datagram is 56 bytes.
C – The length of an ICMP-echo-replay-Datagram is 64 bytes.
D – The increase of the sequence number with every reply datagram with one number up.
E – The TTL (Time to live) from the datagrams. When a Linux system sends an ICMP, the default TTL is 255 which every times it passes a router, it is decremented. If that number reaches 0, then the packet will be dropped. This information tells us how many routers the packet has gone through before reaching its target.
F – The time which every round-trip takes from the sending of an “echo-request” until the recieval of its “echo-reply” datagram passes.
G – A statically result from the round-trip times: Minimum, regular, maximum.
In case of a network with errors, or the ICMP packets do not reply, the result from a ping would look like this:
instronics@ns1:~> ping www.runsecure.com
PING runsecure.com (66.152.128.86) from 212.89.163.212 : 56(84) bytes of data.
--- runsecure.com ping statistics ---
5 packets transmitted, 0 received, 100% loss, time 4013ms
instronics@ns1:~>
This attempt to send 5 ICMP packets with the “echo request” datagram to runsecure.com failed because there was no “echo reply” from the target host.
1.a – Ping extended.
Ping also has many parameters that can be very useful. The most important parameters and options available, I will try to explain to you. For more deeper information though on the ping command, please do a:
man 8 ping.
Parameters:
ping [-c <count>] [-I <interface address>] [-i <wait>] [-f ] [-l <preload>] [-n] [-t <ttl>] [-w <maxwait>]
Parameter: Meaning:
-c <count> You can define the amount of pings to send. After that ping will end.
-I <interface address> If you have more than 1 network interface, you can choose which one to use.
-i <wait> Waits in seconds how long to wait before the next ping is sent. Default is 1 sec.
-l <preload> <preload> datagrams are send as fast as possible before ping continues with its
default configurations.
-n Numeric results. The ip addresses of hostnames will not be resolved.
-t <ttl> Define the time to live for “echo request” datagrams.
-w <maxwait> You can define here how many seconds to wait for each “echo reply”. Default
is 10 seconds.
1.b – Ways to analyze problems.
If you want to test the communication of 2 hosts, then you should always start with an internal network. (ping localhost.) Then the next step should be to ping you local ip address on which your interface sits on. (ping <ip-address of your interface>.) If these tests are successful, then you can proceed to test the network segment up to the next router, switch, or hub which are in the path to your target. After that test the network path to the next segment. And so on until your target host. This way you can define where the problem might be, on which network, which router etc. This will help you minimize the area of where the error might be.
2 – Traceroute.
Traceroute is a diagnosis tool to find out which path a packet takes to reach its destination. It is a nice way to see which routers it passes and which networks it crosses to reach its destination. Traceroute sends 3 UDP datagrams to the target host, whereas the first datagram is a TTL (time to live) with a value of 1. Every following value of a datagram will increase by 1. Since every time when it passes a router (HOP), it will decrease by 1 again, and once it reaches 0, it will drop the packet, so the gateway will send a message “time exceeded” back to the sender. If the packet reaches the target host, then it will send back a message “port unreachable” meaning that the target host has been reached. With this collection of information, “traceroute” then gives you information about its results to an stdout (standard output). The easiest way to use trace route is by simply typing:
Traceroute <targethost>
A typical result from traceroute is:
instronics@ns1:~> traceroute ns.esn.gr
traceroute to ns.esn.gr (212.89.163.211), 30 hops max, 40byte
packets
1 bg.esn.gr (212.89.0.254) 0 ms 0 ms 0 ms
2 web.esn.gr (212.89.163.254) 14 ms 18 ms 14 ms
3 ns.esn.gr (212.89.163.211) 19 ms * 26 ms
This tells us that there were 3 nods in the network before the datagram was routed to the target host.
The information provided here was:
A – The IP address from the target host was resolved (ns.esn.gr to 212.89.163.211).
B – The UDP datagrams are send with a default of TTL=30 hops max.
C – A UDP datagram has a length of 40 bytes.
D – The time for a round-trip on every router along the way is displayed in milliseconds.
E – The second UDP datagram which reached ns.esn.gr was not answered within the given TTL which consisted of 3 seconds. (the * )
2.b – Traceroute extended.
Just like ping, there are many parameters for traceroute. Again I will try to explain the most important parameters here, but for more detail also do a:
Man 8 traceroute
Parameters:
traceroute [-m <max_ttl>] [-n] [-p <port>] [-q <nqueries>] [-r] [-s <src_addr>] [-w <waittime>] host
Traceroute: Meaning:
-m <max_ttl> Define the maximum value for the TTL. Default is 30.
-n Only displays the IP address from each HOP. Does not resolve them into
hostnames.
-p <port> Defines the target port number from the target machine. This will increase
with every datagram send. Default starting port is 33434
-r Sends a UDP datagram directly to a target host without taking notice from
the routing tables.
-q <nqueries> Defines the amount of UDP datagrams for every host. Default is 3 datagrams.
-s <src_addr> Defines the sender’s address. This is done if you have more than one network interface. The given value here must be an existing host. (heh, no spoofing here :p )
-w <waittime> Defines the timeout for the answer from each HOP. Default are 3 seconds.
-v Verbose mode. ICMP datagrams which are not types from “time exceeded” or port unreachable” will be also shown here.
-o Displays the mail address from the administrators on each tested HOP. This information is displayed by the DNS entries.
2.c – Ways to analyze problems.
If there is a communication error between 2 networks, then the problem is most probably a mistake or failure in the transport way between 2 networks. Traceroute is a wonderful tool to discover the transport ways and their routers. It offers detailed information about the path it takes, which routers it passes. It will tell you up to which HOP is ok or has errors.
3 – tcpdump
tcpdump is a diagnosis tool which analyses data packets which pass through the network interface which you are connected to. tcpdump is an excellent tool to study many problems which arise within networks. From the analysis if a simple broadcast up to the analysis of complicated data connections between 2 or more hosts.
The easy form of tcpdump, to be executed as user “root” looks like:
tcpdump –i <network-interface>
This command puts the defined network interface into “promiscuous mode” (this means that in this mode the interface shows us all network data that passes the interface as well as a lot of information about the data itself).
instronics@ns1:~> tcpdump –i eth0
User level filter, protocol ALL, datagram packet socket
tcpdump: listening on eth0
18:54:43.666301 ns1.esn.gr > ns.esn.gr.ssh:
S 4057372301:4057372301 (0)
win 32120 <mss 1460, sackOK, timstamp 359463 0, nop, wscale 0> (DF)
18:54:43.666836 ns.esn.gr.ssh > ns1.esn.gr.1023:
S 4047727479:4047727479 (0)
ack 4057372302 win 32120 <mss 1460, sackOK, timestamp 327365500 359463, nop, wscale
0> (DF)
…………
This shows us the following:
A – The information gathered came from network interface eth0.
B – We see here 2 datagrams.
C – Each datagram includes the time as well as its sequence number from each datagram.
D – Every datagram shows its source address, source port, destination address and destination port.
E – The given flags inside the protocol headers are (S=SYN, F=FIN, P=PUSH, R=RST).
F – Sequence numbers from the data.
G – A set ACK-bit with its sequence number for the next estimated data packet.
H – The size from the receiver buffer for data packets in the other direction.
I – A few other TCP-options.
The first thing we saw here was a connection attempt between ns.esn.gr and ns1.esn.gr (the syn bit, not an ack bit). The connection attempts to connect to destination port 22 (SSH) on ns.esn.gr, and our source port from ns1.esn.gr is 1023. The second packet returns the answer (the ack-bit) from source port 22 (SSH) on ns.esn.gr to port 1023 on ns1.esn.gr.
tcpdump extended.
tcpdump knows many parameters. I will explain the most important ones, but please also do a :
man 1 tcpdump
Parameters:
tcpdump [ -elnqvx ] [ -c count] [-i interface] [-r file] [-w file] [expression]
Parameter: Meaning:
-e The header on the link-level will be displayed.
-l The output on stdout will be verified.
-n Addresses (hosts, ports) will not be converted into names (22=ssh etc)
-q Little output with little information.
-v | -vv Displays a lot of details (verbose | very verbose).
-x The ouput will result in HEX format.
-c <count> End tcpdump when <count> value is reached.
-i <interface> Which interface to listen on.
-r <file> Read the data packet from defined file.
-w <file> Write in the data packet of a defined file (hehehehe)
<expression> Define a filter for packets to be shown.
Since this can be a bit complicated, allow me to provide you with some examples:
Example 1:
tcpdump –nv –i eth0 –c 64 –w /tmp/tcpdump.net192 net 192
Here we write 64 packets into the file /tmp/net192. Addresses are not being converted, and the output is in verbose (a lot of detail). The information comes from interface eth0, and is only shown to us if the source or destination address is within the 192.0.0.0/255.0.0.0 network.
Example 2:
tcpdump –r /tmp/tcpdump.net192
Here tcpdump gets the data from the file /tmp/tcpdump.net192 and shows it to us via stdout.
Example 3:
tcpdump –e –i eth0 src 192.168.0.1 and dst 192.168.0.2 or host 192.168.0.3
(hmmm, this is really interesting, is it not ;) ).
Here we see all packets that come from the IP address 192.168.0.1 and is sent to 192.168.0.2. In addition to this, we can see all data packets which have the source or/and destination IP 192.168.0.3
Remember that tcpdump can be abused to read clear text passwords and usernames. It is a packet sniffer.
4 – Netstat.
Netstat is a tool which with its help you can determine the status of all network connections to or from your computer.
Netstat shows you all open sockets (network connections) and gives you details about source/destination IP and port. Sometimes it even converts the ports to services names (if they are known in /etc/services). The easiest way to use this tool is:
Netstat
A typical netstat result looks like:
instronics@ns1:~> netstat
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 212.89.163.212:ssh 212.89.163.:sso-service ESTABLISHED
Active UNIX domain sockets (w/o servers)
Proto RefCnt Flags Type State I-Node Path
unix 7 [ ] DGRAM 498 /dev/log
unix 2 [ ] DGRAM 14952
unix 2 [ ] DGRAM 7240
unix 2 [ ] DGRAM 2163
unix 2 [ ] DGRAM 1352
unix 2 [ ] DGRAM 1037
instronics@ns1:~>
The information we see here shows us all connections on ns1.esn.gr. As you can see the only connection is on port 22 (SSH) which is me connecting from this windoze box here where I am writing this tutorial on Microsoft office, and I’m ssh’ing to ns1.esn.gr to be able to copy paste an example of the netstat command for you. Apart from that, the information we have gathered is the following.
A – The first block shows us every socket on its own line.
B – The first socket tells us that the protocol used is TCP, the local port (on ns1.esn.gr) is port 22 (SSH) and the connection comes from 212.89.163.:ss0-service which is my windows box. The state of this connection is established, meaning that this connection is still alive.
C – The information under Recv-Q tells us that all data within the receiver queue (I hope this is correct L ) has no pending jobs/transfers. That means that the network conversation has completed successfully. The protocol will always be displayed as UNIX.
D – The RefCnt shows us the amount of active applications and to which socket it is combined with (could not find a better word than “combine” here, help me out someone.)
E – The special flags (SO_ACCEPTION = ACC, SO_WAITDATA =W, SO_NOSPACE =N) are not set here.
F – The type is for all listed sockets STREAM, meaning that its go to do with connection orientated sockets.
G – Non of the listed sockets are connected.
H – Also you can see the I-nodes as well as the path names for each “combined” (grrr, again) process.
Netstat also has a few parameters that are very useful.
Please do not forget to do a :
man 8 netstat
A few parameters are :
A – netstat –rn
Lists the entries within the routing table. There is no name resolution.
B – netstat –a
Lists all passive and active sockets.
C – netstat –u
Lists only the sockets for UDP datagrams.
D – netstat –t
Lists only the sockets for TCP packets.
E – netstat –r
Lists only the raw-sockets.
(I recommend personally the combination of)
netstat –patune
This is the one I personally use most.
5 – Log-Files.
Log files are also an excellent source of information to get an idea of what might be wrong if you have problems. The syslog daemon can help you a lot with this. http://www.antionline.com/showthread...ght=monitoring will help you on the syslog daemon.
Also do not forget the /var/log/messages, since a vay lot of information is stored there.
I hope that this information here comes in handy. The tools I listed are very basic, but they do get the job done often very well. Ofcourse there are many alternative applications, one of them would be ethereal instead of tcpdump. Anyways….good luck and happy administering your networks.
Cheers.
