Troubleshooting OpenVPN
From TroubleshootingWiki
| Official Page |
| Project Documentation |
| Download |
|
In this tutorial, we will learn how to use tools to debug and monitor our VPN tunnels. We will also learn how to scan and test the connectivity of a (VPN) server with standard networking tools.
Contents |
[edit] Testing the Network Connectivity
In our typical OpenVPN setup, we have connected two networks (192.168.250.0/24 and 172.16.76.0/24) via two Linux servers that are connected to the Internet via a default gateway. Between the two Linux servers is a tunnel that uses the virtual IPs 10.179.10.1 and 10.179.10.2.
In the connected local networks there are two Linux machines that we will use to test our tunnels (perhaps by conveniently accessing them remotely with Secure Shell). We will now use the tools ifconfig, route, and ping to show and test the network settings.
In our first step, we will check the local system's network address, default route, and if the default router is pingable. The command ifconfig will print statistics of all active network interfaces:
root@sydney:~ #ifconfig eth0 Link encap:Ethernet HWaddr 00:0C:29:AE:8C:D7 inet addr:192.168.250.128 Bcast:192.168.250.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:2640 errors:0 dropped:0 overruns:0 frame:0 TX packets:2290 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:250738 (244.8 KiB) TX bytes:273328 (266.9 KiB) Interrupt:10 Base address:0x1080 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:57 errors:0 dropped:0 overruns:0 frame:0 TX packets:57 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:7907 (7.7 KiB) TX bytes:7907 (7.7 KiB) root@sydney:~ #
This system has the IP address 192.168.250.128, and its network interface is up and running. Obviously this machine is located in Sydney, Australia.
Now let's look at its routing entries. The command route prints all routing entries, including the router to the Internet. A default gateway is a router that is supposed to handle all traffic not specified by any other routing entries. In our networks, the OpenVPN server is the only router from the internal network and is therefore configured as default gateway for the local network.
Type route -n to receive a numeric output of the routing table of your system. Simply typing route will work in most cases, but the command will try to resolve the IPs via DNS, which might take a little time.
root@sydney:~ #route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 192.168.250.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 0.0.0.0 192.168.250.251 0.0.0.0 UG 0 0 0 eth0 root@sydney:~ #
We see a table where destinations, gateways, netmasks, and interfaces are listed. Every line is a routing entry that can be read like a real sentence. An entry 0.0.0.0 simply matches every address (source or destination, depending on the context) and is e. g. used for the default gateway.
Line three means that all traffic to the network 192.168.250.0 is sent directly to the network interface eth0, no matter which gateway is to be used.
Line four indicates that all the traffic to any destination will be sent over the default gateway 192.168.250.251 via interface eth0.
This setup is perfectly OK for a typical network client. Let's now test if the default gateway is reachable by pinging it from the client:
root@sydney:~ #ping 192.168.250.251 PING 192.168.250.251 (192.168.250.251): 56 data bytes 64 bytes from 192.168.250.251: icmp_seq=0 ttl=64 time=1.3 ms 64 bytes from 192.168.250.251: icmp_seq=1 ttl=64 time=0.6 ms 64 bytes from 192.168.250.251: icmp_seq=2 ttl=64 time=0.4 ms --- 192.168.250.251 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.4/0.7/1.3 ms root@sydney:~ #
It works. The default gateway (our OpenVPN server) answers the ping requests from our client. If it doesn't in your setup, check the firewall rules on this server as to whether they allow traffic from the internal network to the firewall itself. If you are unsure, it may be a good idea to temporarily stop the firewall services.
Now let's try the same on the client in the other network (obviously in Germany):
root@munich:~ #ifconfig eth0 Link encap:Ethernet HWaddr 00:0C:29:21:07:FC inet addr:172.16.76.128 Bcast:172.16.76.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:2399 errors:0 dropped:0 overruns:0 frame:0 TX packets:2715 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:345146 (337.0 KiB) TX bytes:271839 (265.4 KiB) Interrupt:10 Base address:0x1080 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:8 errors:0 dropped:0 overruns:0 frame:0 TX packets:8 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:772 (772.0 B) TX bytes:772 (772.0 B) root@munich:~ #route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 172.16.76.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 0.0.0.0 172.16.76.251 0.0.0.0 UG 0 0 0 eth0 root@munich:~ #ping 172.16.76.251 PING 172.16.76.251 (172.16.76.251): 56 data bytes 64 bytes from 172.16.76.251: icmp_seq=0 ttl=64 time=2.0 ms 64 bytes from 172.16.76.251: icmp_seq=1 ttl=64 time=0.5 ms 64 bytes from 172.16.76.251: icmp_seq=2 ttl=64 time=0.5 ms --- 172.16.76.251 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.5/1.0/2.0 ms root@munich:~ #
Network configuration and routing are correct, and pinging the VPN server works.
[edit] Checking Interfaces, Routing, and Connectivity on the VPN Servers
In our next step we will have a close look at the network settings on the VPN servers. We will use the same tools as above, but the output will be a little more complex:
opensuse01:~ # ifconfig eth0 Protokoll:Ethernet Hardware Adresse 00:0C:29:13:EC:48 inet Adresse:172.16.103.2 Bcast:172.16.103.255 Maske:255.255.255.0 inet6 Adresse: fe80::20c:29ff:fe13:ec48/64 G ltigkeitsbereich:Verbindung UP BROADCAST NOTRAILERS RUNNING MULTICAST MTU:1500 Metric:1 RX packets:2900 errors:0 dropped:0 overruns:0 frame:0 TX packets:4790 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 Sendewarteschlangenl nge:1000 RX bytes:759578 (741.7 Kb) TX bytes:666545 (650.9 Kb) Interrupt:10 Basisadresse:0x1080 eth1 Protokoll:Ethernet Hardware Adresse 00:0C:29:13:EC:52 inet Adresse:172.16.76.251 Bcast:172.16.76.255 Maske:255.255.255.0 inet6 Adresse: fe80::20c:29ff:fe13:ec52/64 G ltigkeitsbereich:Verbindung UP BROADCAST NOTRAILERS RUNNING MULTICAST MTU:1500 Metric:1 RX packets:797 errors:0 dropped:0 overruns:0 frame:0 TX packets:421 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 Sendewarteschlangenl nge:1000 RX bytes:77682 (75.8 Kb) TX bytes:42404 (41.4 Kb) Interrupt:9 Basisadresse:0x1400 lo Protokoll:Lokale Schleife inet Adresse:127.0.0.1 Maske:255.0.0.0 inet6 Adresse: ::1/128 G ltigkeitsbereich:Maschine UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:109 errors:0 dropped:0 overruns:0 frame:0 TX packets:109 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 Sendewarteschlangenl nge:0 RX bytes:8380 (8.1 Kb) TX bytes:8380 (8.1 Kb) tunVPN0 Protokoll:UNSPEC Hardware Adresse 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 inet Adresse:10.179.10.2 P-z-P:10.179.10.1 Maske:255.255.255.255 UP PUNKTZUPUNKT RUNNING NOARP MULTICAST MTU:1500 Metric:1 RX packets:1337 errors:0 dropped:0 overruns:0 frame:0 TX packets:1547 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 Sendewarteschlangenl nge:100 RX bytes:470725 (459.6 Kb) TX bytes:181397 (177.1 Kb) opensuse01:~ #
OK, this server seems to have to network interface cards eth0 and eth1 (with two networks 172.16.103.0/24 and 172.16.76.0/24 in addition to the OpenVPN tunnel network tunVPN0 with the network address 10.179.10.2 and the point-to-point partner's IP 10.179.10.1. How about routing?
opensuse01:~ # route -n Kernel IP Routentabelle Ziel Router Genmask Flags Metric Ref Use Iface 10.179.10.1 0.0.0.0 255.255.255.255 UH 0 0 0 tunVPN0 172.16.103.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 172.16.76.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1 192.168.250.0 10.179.10.1 255.255.255.0 UG 0 0 0 tunVPN0 127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo 0.0.0.0 172.16.103.1 0.0.0.0 UG 0 0 0 eth0 opensuse01:~ #
Routing is a little more complicated here. We have two subnets connected to eth0 and eth1, and two entries for our tunnel; everything to the virtual address 10.179.10.1 is routed via the interface tunVPN0, likewise traffic to the subnet 192.168.250.0/24, but this is routed via the gateway 10.179.10.1. Last but not least, the default gateway of this router has the IP 172.16.103.1. Obviously there is another network between this firewall and the Internet.
Let's now ping the point-to-point partner of this machine. We could see from the aforementioned interface list that this machine has the virtual IP 10.179.10.2, and the VPN partner has the IP 10.179.10.1. If our tunnel is working, it should be possible to ping through the tunnel:
opensuse01:~ # ping 10.179.10.1 PING 10.179.10.1 (10.179.10.1) 56(84) bytes of data. 64 bytes from 10.179.10.1: icmp_seq=1 ttl=64 time=1.77 ms 64 bytes from 10.179.10.1: icmp_seq=2 ttl=64 time=1.50 ms 64 bytes from 10.179.10.1: icmp_seq=3 ttl=64 time=1.42 ms 64 bytes from 10.179.10.1: icmp_seq=4 ttl=64 time=1.44 ms --- 10.179.10.1 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3013ms rtt min/avg/max/mdev = 1.425/1.535/1.770/0.141 ms opensuse01:~ #
It's working. Please note that the time taken to answer a ping will be significantly higher through the tunnel than for a local or direct ping.
Now let's do the same tests the other way around. We will analyze the network and routing of the Sydney server and try to ping to Munich through the tunnel:
debian01:~# ifconfig eth0 Link encap:Ethernet HWaddr 00:0C:29:99:7B:CA inet addr:172.16.247.2 Bcast:172.16.247.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:7735 errors:0 dropped:0 overruns:0 frame:0 TX packets:11012 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:924335 (902.6 KiB) TX bytes:1714169 (1.6 MiB) Interrupt:18 Base address:0x1080 eth1 Link encap:Ethernet HWaddr 00:0C:29:99:7B:D4 inet addr:192.168.250.251 Bcast:192.168.250.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:490 errors:0 dropped:0 overruns:0 frame:0 TX packets:468 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:47652 (46.5 KiB) TX bytes:43728 (42.7 KiB) Interrupt:19 Base address:0x1400 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) tunVPN0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 inet addr:10.179.10.1 P-t-P:10.179.10.2 Mask:255.255.255.255 UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1 RX packets:1849 errors:0 dropped:0 overruns:0 frame:0 TX packets:1489 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:100 RX bytes:206765 (201.9 KiB) TX bytes:483493 (472.1 KiB) debian01:~# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 10.179.10.2 0.0.0.0 255.255.255.255 UH 0 0 0 tunVPN0 172.16.247.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 172.16.76.0 10.179.10.2 255.255.255.0 UG 0 0 0 tunVPN0 192.168.250.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1 0.0.0.0 172.16.247.1 0.0.0.0 UG 0 0 0 eth0 debian01:~# ping 10.179.10.1 PING 10.179.10.1 (10.179.10.1) 56(84) bytes of data. 64 bytes from 10.179.10.1: icmp_seq=1 ttl=64 time=0.221 ms 64 bytes from 10.179.10.1: icmp_seq=2 ttl=64 time=0.069 ms 64 bytes from 10.179.10.1: icmp_seq=3 ttl=64 time=0.059 ms --- 10.179.10.1 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2005ms rtt min/avg/max/mdev = 0.059/0.116/0.221/0.074 ms debian01:~#
It worked. We have now made sure that:
- The VPN servers are reachable in their local networks.
- The OpenVPN tunnel is up and running.
- The OpenVPN tunnel is working in both directions.
Let's now enter another level of testing. We will now test if the Sydney network is reachable from our VPN server in Munich still using ICMP packets only. Furthermore, the program traceroute will help us follow the route the packets take:
opensuse01:~ # ping 192.168.250.128 PING 192.168.250.128 (192.168.250.128) 56(84) bytes of data. 64 bytes from 192.168.250.128: icmp_seq=1 ttl=63 time=1.90 ms 64 bytes from 192.168.250.128: icmp_seq=2 ttl=63 time=1.26 ms 64 bytes from 192.168.250.128: icmp_seq=3 ttl=63 time=1.57 ms --- 192.168.250.128 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2009ms rtt min/avg/max/mdev = 1.261/1.577/1.900/0.264 ms opensuse01:~ # traceroute -n 192.168.250.128 traceroute to 192.168.250.128 (192.168.250.128), 30 hops max, 40 byte packets 1 10.179.10.1 1.874 ms 8.949 ms 20.241 ms 2 192.168.250.128 24.911 ms 35.618 ms 40.988 ms opensuse01:~ #
Again, pinging worked fine. This indicates correct routing on the Sydney side and on the Munich VPN server. The output of the program traceroute lists all servers the packets passed on their way to Sydney; they were thrown into the tunnel immediately and arrived at the VPN server in Sydney 10.179.10.1, which passed them on to the local machine, which took forty milliseconds. Of course we can also "traceroute" our packets that go the other way, provided that the administrator of the Debian server has installed traceroute (apt-get install traceroute).
Another very handy tool is "My traceroute", or mtr. Called with mtr -n 192.168.250.128, mtr keeps running traceroute -n 192.168.250.128 command until you type q or Ctrl+C. The output is displayed in a clear table. With this command, we can easily switch routing entries and control the effect interactively.
[edit] Debugging with tcpdump and IPTraf
Another very handy tool to control traffic is tcpdump. As a network sniffer, tcpdump is often used by administrators or hackers to collect the data exchanged on the network. tcpdump prints all traffic that passes the interface given as a parameter. The following example shows the usage of tcpdump. When called with the options -n and -i eth1, tcpdump will listen on interface eth1 and give a numeric output (without resolving DNS):
debian01:~# tcpdump -n -i eth1 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 96 bytes 21:00:16.640142 IP 192.168.250.128 > 172.16.76.128: ICMP echo request, id 55298, seq 0, length 64 21:00:16.648116 IP 172.16.76.128 > 192.168.250.128: ICMP echo reply, id 55298, seq 0, length 64 21:00:17.678429 IP 192.168.250.128 > 172.16.76.128: ICMP echo request, id 55298, seq 256, length 64 21:00:17.680701 IP 172.16.76.128 > 192.168.250.128: ICMP echo reply, id 55298, seq 256, length 64 21:00:18.668565 IP 192.168.250.128 > 172.16.76.128: ICMP echo request, id 55298, seq 512, length 64 21:00:18.670722 IP 172.16.76.128 > 192.168.250.128: ICMP echo reply, id 55298, seq 512, length 64 21:00:19.688618 IP 192.168.250.128 > 172.16.76.128: ICMP echo request, id 55298, seq 768, length 64 21:00:19.690836 IP 172.16.76.128 > 192.168.250.128: ICMP echo reply, id 55298, seq 768, length 64
As we can see, there were four ICMP echo request messages sent from 192.168.250.128 to 172.16.76.128. All of them were answered by the machine 172.16.76.128 with the appropriate "echo reply" message.
Now we can use tcpdump on every machine in the chain of routers between the two clients in order to track the ICMP packets. For example, if a firewall is blocking the ICMP messages, then no PC behind this firewall will receive any the requests or replies, whereas the machines before the firewall will do.
debian01:~# tcpdump -ni tunVPN0 tcpdump: WARNING: arptype 65534 not supported by libpcap - falling back to cooked socket tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on tunVPN0, link-type LINUX_SLL (Linux cooked), capture size 96 bytes 21:07:53.800707 IP 172.16.76.128 > 192.168.250.128: ICMP echo request, id 19971, seq 9472, length 64 21:07:53.801608 IP 192.168.250.128 > 172.16.76.128: ICMP echo reply, id 19971, seq 9472, length 64 21:07:54.799266 IP 172.16.76.128 > 192.168.250.128: ICMP echo request, id 19971, seq 9728, length 64 21:07:54.800531 IP 192.168.250.128 > 172.16.76.128: ICMP echo reply, id 19971, seq 9728, length 64 21:07:55.800302 IP 172.16.76.128 > 192.168.250.128: ICMP echo request, id 19971, seq 9984, length 64 21:07:55.801296 IP 192.168.250.128 > 172.16.76.128: ICMP echo reply, id 19971, seq 9984, length 64 21:07:56.752248 IP 172.16.76.128 > 192.168.250.128: ICMP echo request, id 19971, seq 10240, length 64 21:07:56.752876 IP 192.168.250.128 > 172.16.76.128: ICMP echo reply, id 19971, seq 10240, length 64 8 packets captured 16 packets received by filter 0 packets dropped by kernel debian01:~#
You see, tcpdump runs also on the tunnel interfaces, but some features won't work with TUN or TAP interfaces. Also because the network interface will be run in promiscuous mode, tcpdump will need root privileges. Furthermore, the information returned will be scarce in most switched networks, where only local packets can be displayed.
Another helpful tool is IPTraf (on Debian installed with apt-get install iptraf). IPTraf collects and displays packets and statistical data on selected interfaces. IPTraf comes with many options, but we will only focus on its list view.
Enter iptraf and hit return four times. You will get a window as depicted in the following screenshot:
In the upper half of the window, TCP connections are displayed. UDP, ICMP, and other connections can be found in the lower half. In the example above, we can recognize an SSH session (from which IPTraf was started), ICMP packages between the Sydney and Munich client PCs, and the UDP packages encapsulating these ICMP packages.
Hit X twice and Enter once to quit IPTraf.
[edit] Using OpenVPN Protocol and Status Files for Debugging
A very convenient method to watch tunnel traffic is setting the verbosity of OpenVPN to the fifth level. This is simply done with the entry verb 5 in its configuration file. The following output shows an excerpt of OpenVPN's protocol file (as specified in the OpenVPN configuration file):
Fri Dec 9 21:05:15 2005 us=51912 Data Channel Encrypt: Cipher 'AES-256-CBC' initialized with 256 bit key Fri Dec 9 21:05:15 2005 us=51944 Data Channel Encrypt: Using 160 bit message hash 'SHA1' for HMAC authentication Fri Dec 9 21:05:15 2005 us=51962 Data Channel Decrypt: Cipher 'AES-256-CBC' initialized with 256 bit key Fri Dec 9 21:05:15 2005 us=52033 Data Channel Decrypt: Using 160 bit message hash 'SHA1' for HMAC authentication Fri Dec 9 21:05:15 2005 us=131924 Control Channel: TLSv1, cipher TLSv1/SSLv3 DHE-RSA-AES256-SHA, 2048 bit RSA WRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrW RwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwr WRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrWRwrW (...)
In the last lines we find the detailed statistics of all tunnel traffic. Upper cased letters stand for TCP or UDP datagrams on the real interface, encapsulating OpenVPN traffic, and lower case letters indicate traffic on the TUN/TAP interface. Unsurprisingly, r is for read and w is for write. Thus a successful ping command through the tunnel will always cause an entry like WRwr or vice versa.
Another file that our sample setup writes information to is the status file. Depending on the time period given as a parameter, OpenVPN will update the information in this file on a regular basis. In the example the file was /var/log/openvpn/feilner-it.status; the command cat can show us the content of this file:
debian01:~# cat /var/log/openvpn/feilner-it.status OpenVPN STATISTICS Updated,Fri Dec 9 21:26:53 2005 TUN/TAP read bytes,1102504 TUN/TAP write bytes,806453 TCP/UDP read bytes,1302857 TCP/UDP write bytes,1588558 Auth read bytes,808809 pre-compress bytes,55193 post-compress bytes,53110 pre-decompress bytes,1449 post-decompress bytes,2076 END debian01:~#
We find detailed statistical data. If you run into problems with OpenVPN, it may be a good idea to check this file to find out if the values make sense, or if there is either too much or missing traffic on either side, for example, if it gets lost or the routing is wrong.
Depending on your system and logging setup, there may also be entries in your system protocol, like those here on this SuSE system:
opensuse01:~ # tail /var/log/messages Dec 2 17:50:09 opensuse01 openvpn[11661]: Local Options String: 'V4,dev-type tun,link-mtu 1545,tun-mtu 1500, proto UDPv4,ifconfig 10.179.11.1 10.179.11.2,comp-lzo,cipher BF-CBC,auth SHA1,keysize 128,secret' Dec 2 17:50:09 opensuse01 openvpn[11661]: Expected Remote Options String: 'V4,dev-type tun,link-mtu 1545, tun-mtu 1500,proto UDPv4,ifconfig 10.179.11.2 10.179.11.1,comp-lzo,cipher BF-CBC,auth SHA1,keysize 128,secret' Dec 2 17:50:09 opensuse01 openvpn[11661]: Local Options hash (VER=V4): '59c313f6' Dec 2 17:50:09 opensuse01 openvpn[11661]: Expected Remote Options hash (VER=V4): '36b1f115' Dec 2 17:50:09 opensuse01 openvpn[11661]: Output Traffic Shaping initialized at 20000 bytes per second Dec 2 17:50:09 opensuse01 openvpn[11674]: Socket Buffers: R=[113664->131072] S=[113664->131072] Dec 2 17:50:09 opensuse01 openvpn[11674]: UDPv4 link local (bound): [undef]:5001 Dec 2 17:50:09 opensuse01 openvpn[11674]: UDPv4 link remote: 172.16.247.2:5001
This shows that another VPN tunnel has been created; OpenVPN is listening on UDP port 5001.
[edit] Scanning Servers with Nmap
Nmap is a port scanner that can be used to determine whether a UDP or TCP port on a machine is open, and whether there is a server process accepting connections. Nmap can also find out if a firewall is protecting the machine scanned, and Nmap can scan whole networks. Let's scan the local client PC (which is obviously not protected by a firewall...):
opensuse01:~ # nmap 172.16.76.128 Starting nmap 3.81 ( http://www.insecure.org/nmap/ ) at 2005-12-02 18:02 CET Interesting ports on localhost (172.16.76.128): (The 1661 ports scanned but not shown below are in state: closed) PORT STATE SERVICE 22/tcp open ssh 68/tcp open dhcpclient MAC Address: 00:0C:29:21:07:FC Nmap finished: 1 IP address (1 host up) scanned in 1.773 seconds
There are two ports open on this system; port 1661 and other scanned ports are closed. If there were a firewall on this system, then scanning would not be that easy, because most firewalls detect scans and can prevent them. But there are many options to Nmap, including stealth scans, altering sender IPs, and many more the manual page is really good.
We will now scan one of our OpenVPN servers to find out if our VPN port (5000) can be reached. The command nmap -sU <IP> -p <Port> will make Nmap scan only if the UDP port on the machine with the given IP address is open:
opensuse01:~ # nmap -sU 172.16.247.2 -p 5000 Starting nmap 3.81 ( http://www.insecure.org/nmap/ ) at 2005-12-02 18:06 CET Note: Host seems down. If it is really up, but blocking our ping probes, try -P0 Nmap finished: 1 IP address (0 hosts up) scanned in 2.067 seconds opensuse01:~ # nmap -P0 -sU 172.16.247.2 -p 5000 Starting nmap 3.81 ( http://www.insecure.org/nmap/ ) at 2005-12-02 18:06 CET Interesting ports on debian01.feilner-it.home (172.16.247.2): PORT STATE SERVICE 5000/udp open|filtered UPnP Nmap finished: 1 IP address (1 host up) scanned in 2.039 seconds opensuse01:~ #
You saw how our Shorewall firewall did not reveal information about the port when we scanned it in the first try. However, Nmap already gave us a hint: add the parameter -P0 to act even more stealthily. With this option, Nmap does not ping the hosts it scans before really scanning them. Some firewalls recognize this as a typical behavior of port scanners and block it. The second try, however revealed that the UDP port 5000 is filtered (by a firewall). This means: firewall rules may be protecting and limiting access to this port, but it is open.
[edit] Monitoring Tools
There are many tools that provide detailed statistics on network interfaces. Two very easily installed monitoring tools with great functions are ntop and Munin.
[edit] ntop
ntop monitors a network and may in some states be illegal because it creates detailed records of connections between IP addresses. Furthermore, it offers a nice browser GUI and does not need a running web server. ntop installs easily on Debian.
Enter apt-get install ntop and choose the interface you want to monitor. After software installation, type ntop -A, and enter an administrator password for ntop's admin account. Now type /etc/init.d/ntop start and point a browser to the http://IP:3000 of this system (ntop is running on port 3000). You will get a feature-rich window with a growing amount of information, especially if ntop has been for running some time:
ntop offers many possibilities. We can save the data to a database, access to a database can be secured and monitored, interfaces can be switched online, and many more possibilities.
[edit] Munin
Another helpful statistic tool is Munin. Munin consists of a client and a server process that collect data that is provided from an almost arbitrary source on Linux (or even Windows) systems. The example below shows the standard Munin interface after installation as documented on http://munin.sf.net. Unfortunately, Munin needs a web server like Apache, but apart from this, the installation is very easy. Munin is configured from files in /etc/munin/, and makes use of a great number of plug-ins; even more can be downloaded.
Since there are only a few requirements for a Munin plug-in, we can easily create our own OpenVPN monitoring plug-in. Such a plug-in must be executable, and return data in the format of:
router:/usr/share/munin/plugins # /etc/munin/plugins/if_eth0 down.value 1777836059 up.value 94615124 router:/usr/share/munin/plugins #
As an example, on http://rodolphe.quiedeville.org/hack/openvpn there is a simple plug-in that reports the number of users connected to an OpenVPN server. I leave it up to you to imagine the possibilities of such plug-ins when combined with samba, iptables, OpenVPN, and more. Just think of the OpenVPN status file and the information it provides.
[edit] Hints to Other Tools
There is an abundance of networking tools concerning monitoring, sniffing, and scanning. Two of my favorites are Cacti and Nagios. Cacti is a monitoring tool similar to Munin, but it seems more powerful. Nagios is a tool designed to monitor machines and services.
With Nagios you can not only determine if a server is still answering pings, but can also check for services by accessing them (using e.g. the samba or HTTP protocols) and trigger actions when the service is not available. You can have your Nagios machine send you an SMS if your OpenVPN tunnel is down, or if the management interface is not reacting.
[edit] Additional References
For instructions on Installing OpenVPN, click here
[edit] Source
The source of this content is Chapter 11: Troubleshooting and Monitoring of OpenVPN: Building and Integrating Virtual Private Networks by Markus Feilner (Packt Publishing, 2007).
