Friday, November 02, 2012

Windows Virtual PC IPv6: Part 3

The original title of the first post in the this series was "Windows Virtual PC Wireless IPv6", but dropped the "Wireless" before publishing. At that point, I had Part 1 and Part 2 written and knew the problem was with the wireless card versus the wired card, but didn't yet know the root cause. And since I had exhausted my troubleshooting tools (tcpdump and network based analysis) and wasn't about to dive into wireless driver software reverse engineering, I figured I may never have the exact answer. Instead, I focused the posts on discussing the problem and how it relates to the question, "is 'X' IPv6 ready?"

I did want to close the loop and exhaust all possible angles. So I called my friend and colleague Randy who had experience with wireless bridging / routing and virtual machines from a previous client. I remember he would lament about the issues he faced and custom drivers and wireless cards needed to get things only partially working. Since I was only an observer, I didn't have the intimate details, so now I sought them out over a phone call.

I explained my set up and issue and Randy immediately knew the problem. It wasn't IPv6 specific nor was it related to Windows Virtual PC. In fact, it wasn't even the wireless card or its respective drivers (per se), it was the 802.11 protocol itself. Randy explained the technical issues and gave me the proper Google terms to search to flush out the entire explanation.

And sure enough, after our call I started finding links such as:

that detail the IPv6 neighbor discovery problem I encountered when using wireless bridging. In my case, it wasn't a standalone bridge device, but the virtual network bridge when using a virtual machine. Note that in my testing the issue happened with Windows Virtual PC, but Randy's previous work involved VMware, which also had the issue. Again, this is protocol / driver related, not specific to IPv6 (network) or application layer software.

Further complicating the matter was the "incorrect" 'tcpdump' decodes. While the Ethernet header in the packets I posted showed source and destination MAC addresses, it wasn't the full picture. Wireless 802.11 has a different frame format than wired 802.3 Ethernet. However, the *pcap (winpcap in my case) libraries can't decode the 802.11 header if the wireless driver software doesn't pass it along. With my stock Broadcom wireless on a Dell laptop running Windows 7, I was seeing the "fake" header.

This final link from the OpenWRT project does a good job visualizing what is happening. Again, the link details physical bridges versus the virtual bridge introduced with machine virtualization software, but the same holds true.

So what does all this mean? Depending on how you look at this problem, it could be hardware, protocol or software related. For me, it manifested when trying IPv6 on Windows Virtual PC - so that was my first thought. It turns out the answer was much more involved and complicated than I imagined. However, broken IPv6 connectivity was the symptom and the casual user experiencing that will probably just say, "the network is down." No amount of planning is going to identify these unique use cases until they occur - hopefully during a pilot, more likely during production. You're going to need some smart people when you start rolling out IPv6.

Thursday, November 01, 2012

Windows Virtual PC IPv6: Part 2

In the previous post we saw the wireless card was responsible for some IPv6 connectivity issues between a Windows XP Virtual PC and the Internet due to it rewriting the VPC virtual MAC address to the MAC address of the HOST.

Suspecting the wireless card may be the culprit, I physically connected the HOST to the switch on the router and changed the VPC settings to use the wired NIC instead of the wireless one. A quick reset and back to testing - this time in somewhat reverse order.

I first checked the router to see what the MAC address for the VPC was. It was correct!

ROUTER# arp -a
? (192.168.10.102) at C4:17:FE:12:7D:75 [ether]  on br0
? (192.168.10.104) at 00:03:FF:13:7D:75 [ether]  on br0

So I started my 'tcpdump' captures and launched the pings from the VPC.

HOST> tcpdump -i2 -nevvXX icmp or icmp6
00:03:ff:13:7d:75 > 58:6d:8f:78:ad:40, ethertype IPv6 (0x86dd),
length 94: (hlim 128, next-header: ICMPv6 (58), length: 40)
2001:db8:192:168:203:FFFF:FE13:7D75 > 2607:F8B0:400C:C03::68
[icmp6 sum ok] ICMP6, echo request, length 40, seq 553
  0x0000:  586d 8f78 ad40 0003 ff13 7d75 86dd 6000  Xm.x.@....}u..`.
  0x0010:  0000 0028 3a80 2001 0db8 0192 0168 0203  ...(:....p...|..
  0x0020:  ffff fe13 7d75 2607 f8b0 400c 0c03 0000  ....}u&...@.....
  0x0030:  0000 0000 0068 8000 9120 0000 0229 6162  .....h.......)ab
  0x0040:  6364 6566 6768 696a 6b6c 6d6e 6f70 7172  cdefghijklmnopqr
  0x0050:  7374 7576 7761 6263 6465 6667 6869       stuvwabcdefghi
00:03:ff:13:7d:75 > 58:6d:8f:78:ad:40, ethertype IPv4 (0x0800),
length 74: (tos 0x0, ttl 128, id 25380, offset 0, flags [none], 
proto: ICMP (1), length: 60)
192.168.10.104 > 74.125.228.80
ICMP echo request, id 512, seq 9728, length 40
  0x0000:  586d 8f78 ad40 0003 ff13 7d75 0800 4500  Xm.x.@....}u..E.
  0x0010:  003c 6324 0000 8001 ddbe c0a8 0a68 4a7d  .< c$.........hJ}
  0x0020:  e450 0800 255c 0200 2600 6162 6364 6566  .P..%\..&.abcdef
  0x0030:  6768 696a 6b6c 6d6e 6f70 7172 7374 7576  ghijklmnopqrstuv
  0x0040:  7761 6263 6465 6667 6869                 wabcdefghi

The two packets show the IPv6 and IPv4 pings (echo requests) respectively. Notice the first line of each packet has the VPC source MAC address - '00:03:ff:13:7d:75' - not the HOST MAC address as seen in the previous test on the wireless card.

And the satisfying result:

VM> ping 2607:f8b0:400c:c03::68
Reply from 2607:f8b0:400c:c03::68: time=50ms

VM> ping 74.125.228.80
Reply from 74.125.228.80: bytes=32 time=166ms TTL=52

Just to make sure this whole testing regime wasn't an anomaly, I rebooted and reset to my wireless configuration and tested again with same failed results described in the previous post.

Why do the wireless and wired cards behave differently? I haven't answered that yet. Certainly, they are from different manufacturers and have different settings as verified in the "Properties" page, "Configure..." button, "Advanced" tab of each adapter. I did verify the common settings were matched between cards. I have some more research to do to get to the bottom of this one.

So, "does Windows Virtual PC support IPv6?" It's complicated. As we've seen, it depends on the application, the host and virtual machine operating systems, and in this case, the hardware used.

Wednesday, October 31, 2012

Windows Virtual PC IPv6: Part 1

I have a Windows XP Virtual PC for testing - among other things - all the Perl experimenting I've been doing lately around IPv6. Windows XP supports IPv6 if you install it - it's not on by default. However, I've had some issues with IPv6 connectivity from my VPC to the rest of the Internet.

I run in bridge mode as Shared Networking (NAT) doesn't make much sense with IPv6; there is no NATv6. With bridge mode, the VPC should act as another computer on the network with it's own IP(v6) address(es) and a unique MAC address. I should also note at this time, that I use wireless throughout the house and never connect over the wired LAN.

The setup is pretty simple. Note: I've changed the IPv6 prefix to the documentation prefix for this example but was using global unicast addressing for the actual troubleshooting tests.

+-----------------------------------------+
| (HOST) Windows 7                        |
| c4:17:fe:12:7d:75                       |
| 192.168.10.102                          |
| 2001:db8:192.168:c417:feff:fe12:7d75    |
|                                         |
| +-------------------------------------+ |
| | (VPC) Windows XP                    | |
| | 00:03:ff:13:7d:75                   | |
| | 192.168.10.104                      | |
| | 2001:db8:192.168:203:FFFF:FE13:7D75 | |
| +-------------------------------------+ |
+-----------------------------------------+
                    |
                    |
     +-----------------------------+
     | ROUTER                      |
     | 58:6d:8f:78:ad:40           |
     | 192.168.10.1                |
     | FE80::5A6D:8FFF:FE78:AD40   |
     | ('radvd' running for SLAAC) |
     +-----------------------------+
                    |
                  __|_
               __/    \__
              /          \
              | Internet |
              \__      __/
                 \____/

When I try to ping the IPv6 address returned by 'dig' for "www.google.com" from the VPC, I get 'destination unreachables'. I can ping the same IPv6 address from the HOST. To be sure, I also tried the IPv4 address returned by 'dig' for "www.google.com" from the VPC and was successful.

VPC> ping 2607:f8b0:400c:c03::68
Destination host unreachable.

VPC> ping 74.125.228.80
Reply from 74.125.228.80: bytes=32 time=22ms TTL=52

So the VPC has network connectivity all the way to the Internet for IPv4, but not for IPv6. Why? A packet capture may help.

HOST> tcpdump -i3 -nevvXX icmp or icmp6
c4:17:fe:12:7d:75 > 58:6d:8f:78:ad:40, ethertype IPv4 (0x0800),
length 74: (tos 0x0, ttl 128, id 19700, offset 0, flags [none], 
proto: ICMP (1), length: 60)
192.168.10.104 > 74.125.228.80
ICMP echo request, id 512, seq 19713, length 40
  0x0000:  586d 8f78 ad40 c417 fe12 7d75 0800 4500  Xm.x.@....}u..E.
  0x0010:  003c 4cf4 0000 8001 f3ee c0a8 0a68 4a7d  .< L..........hJ}
  0x0020:  e450 0800 fe5a 0200 4d01 6162 6364 6566  .P...Z..M.abcdef
  0x0030:  6768 696a 6b6c 6d6e 6f70 7172 7374 7576  ghijklmnopqrstuv
  0x0040:  7761 6263 6465 6667 6869                 wabcdefghi
58:6d:8f:78:ad:40 > c4:17:fe:12:7d:75, ethertype IPv4 (0x0800),
length 74: (tos 0x20, ttl  52, id 62188, offset 0, flags [none], 
proto: ICMP (1), length: 60)
74.125.228.80 > 192.168.10.104
ICMP echo reply, id 512, seq 19713, length 40
  0x0000:  c417 fe12 7d75 586d 8f78 ad40 0800 4520  ....}uXm.x.@..E.
  0x0010:  003c f2ec 0000 3401 99d6 4a7d e450 c0a8  .<....4...J}.P..
  0x0020:  0a68 0000 065b 0200 4d01 6162 6364 6566  .h...[..M.abcdef
  0x0030:  6768 696a 6b6c 6d6e 6f70 7172 7374 7576  ghijklmnopqrstuv
  0x0040:  7761 6263 6465 6667 6869                 wabcdefghi

c4:17:fe:12:7d:75 > 58:6d:8f:78:ad:40, ethertype IPv6 (0x86dd),
length 86: (hlim 255, next-header: ICMPv6 (58), length: 32)
FE80::203:FFFF:FE13:7D75 > FE80::5A6D:8FFF:FE78:AD40
[icmp6 sum ok] ICMP6, neighbor solicitation, length 32, 
who has FE80::5A6D:8FFF:FE78:AD40
source link-address option (1), length 8 (1): 00:03:ff:13:7d:75
  0x0000:  586d 8f78 ad40 c417 fe12 7d75 86dd 6000  Xm.x.@....}u..`.
  0x0010:  0000 0020 3aff fe80 0000 0000 0000 0203  ....:...........
  0x0020:  ffff fe13 7d75 fe80 0000 0000 0000 5a6d  ....}u........Zm
  0x0030:  8fff fe78 ad40 8700 55bb 0000 0000 fe80  ...x.@..U.......
  0x0040:  0000 0000 0000 5a6d 8fff fe78 ad40 0101  ......Zm...x.@..
  0x0050:  0003 ff13 7d75                           ....}u
c4:17:fe:12:7d:75 > 58:6d:8f:78:ad:40, ethertype IPv6 (0x86dd),
length 94: (hlim 128, next-header: ICMPv6 (58), length: 40) 
2001:db8:192:168:203:FFFF:FE13:7D75 > 2607:F8B0:400C:C03::68
[icmp6 sum ok] ICMP6, echo request, length 40, seq 498
  0x0000:  586d 8f78 ad40 c417 fe12 7d75 86dd 6000  Xm.x.@....}u..`.
  0x0010:  0000 0028 3a80 2001 0db8 0192 0168 0203  ...(:....p...|..
  0x0020:  ffff fe13 7d75 2607 f8b0 400c 0c03 0000  ....}u&...@.....
  0x0030:  0000 0000 0068 8000 9157 0000 01f2 6162  .....h...W....ab
  0x0040:  6364 6566 6768 696a 6b6c 6d6e 6f70 7172  cdefghijklmnopqr
  0x0050:  7374 7576 7761 6263 6465 6667 6869       stuvwabcdefghi

The first two packets show the IPv4 ping (echo request) and reply (echo reply). The next two packets are more interesting (although, if you were paying attention, you could see it in the first two also).

The second two packets are an IPv6 ICMPv6 neighbor solicitation from the VPC asking for the router's MAC address - the IPv6 version of address resolution protocol (ARP), and an IPv6 ICMPv6 ping request (echo request). Let's look closer at the neighbor solicitation message.

You can see "who has FE80::5A6D:8FFF:FE78:AD40" (the router IPv6 link-local address) and the source of the request in line that follows, "source link-address option (1), length 8 (1): 00:03:ff:13:7d:75" - the MAC address of the VPC. However, look at the first line of that packet, "c4:17:fe:12:7d:75 > 58:6d:8f:78:ad:40, ethertype IPv6 (0x86dd)". The source MAC address is the HOST MAC, not the VPC MAC! WHAT?!?!

Is this normal behavior for bridge mode, to rewrite the source MAC of the VPC with the source MAC of the HOST? Why the virtual MAC address on the VPC in the first place? The HOST operating systems sees the virtual MAC:

HOST> arp -a
Interface: 192.168.10.102 --- 0xb
  Internet Address      Physical Address      Type
  192.168.10.104        00-03-ff-13-7d-75     dynamic

Could this be just a 'tcpdump' anomaly, perhaps capturing at a point in the stack before/after a rewrite? Apparently not, because the router confirms the MAC address for the VPC IPv4 address (192.168.10.104) is in fact the MAC address of the HOST, not the virtual MAC of the VPC:

ROUTER# arp -a
? (192.168.10.102) at C4:17:FE:12:7D:75 [ether]  on br0
? (192.168.10.104) at C4:17:FE:12:7D:75 [ether]  on br0

This touches on a problem that will manifest in the years to come regarding IPv6 support - "does 'X' support IPv6". The answer isn't really binary (yes or no). In this case, the VM can get IPv6 addresses and ping the HOST over IPv6; however, the wireless card (and I say that because in the next post, I'll show how the wired card does not exhibit this behavior) doesn't handle the virtual and host MAC addresses properly and thus breaks IPv6 ICMPv6 neighbor solicitation.

Thursday, October 25, 2012

More on IPv6 in Perl Modules

Recently, I went on a tear updating my Perl modules on CPAN to support IPv6. Since my modules (and scripts) rely on other core modules like Socket, IO::Socket and some non core modules like Net::SNMP and Net::Telnet(::Cisco), I had a task to make sure the underlying modules supported IPv6.

I already dealt with IPv6 in the core Socket module for Windows in a previous post. So now to deal with a higher level of code written by other developers. Would they be responsive?

Turns out ... YES! I worked with the author of Net::TFTPd a while back to get an ASCII / BIN mode bug fix submitted and he was again very responsive with my IPv6 ask and patch. So IPv6 capable TFTP server - Perl has one!

I also rely on Net::Telnet::Cisco which uses Net::Telnet to do all the heavy lifting. Net::Telnet looked like it hadn't been updated since 2002 but I submitted the query / patch and contacted the author. In a few days he got back to me and we had a pretty detailed dialogue about adding IPv6 support. There may be a coming version which does this "natively", but imagine my surprise when I found Net::Telnet was already IPv6-capable!

The "workaround" - and as workarounds go, this is the least kludgey I've seen - goes like this:

Open a socket using an IPv6 capable module - like IO::Socket::IP:

use IO::Socket::IP -register;

my $socket = IO::Socket::IP->new(
    PeerHost => '192.168.10.1' # or 2001:db8:192:168::10:1
    PeerPort => 23,
    Family   => AF_INET # or AF_INET6
) or die "not connected\n";

Then, use the $socket handle as an argument to the 'fhopen' parameter in the Net::Telnet new() constructor:

my $session = Net::Telnet->new(
    fhopen => $socket
);

Jay Rogers (Net::Telnet author) is obviously a genius providing IPv6 support in Net::Telnet before Perl core even supported IPv6! Ok, maybe that's a stretch, but certainly the openness of the Net::Telnet API allows for this kind of "plug-and-play" with IPv4/v6 sockets and enables IPv6 right now, while in the background "native" support may be coming.

Thanks Jay!

Monday, September 17, 2012

Reach the Beach Relay - 2012

Massakruliks - Mixed Open

147/425 - 28/117

Back row from left: Bill (leg 3), Randy (leg 7), Gavin (leg 5), Dave (leg 1), Dan (leg 9)

Middle row from left: Patty (leg 12), Jen (leg 6), Jodi (leg 11), Suzi (leg 10), Ashley (leg 8), Jackie (leg 4)

Front row: Vince [aka: Me] (leg 2), Krulik [aka: Mascot] (on my leg)

Vert
Distance Difficulty Gain Loss Net Time Pace
Leg 2 8.96 Hard 285 127 158 69:48 7:47
Leg 14 7.57 Moderate 492 502 -10 54:12 7:10
Leg 25 * 7.1 (wild card) Moderate 56:05 7:54
Leg 30 * 3.15 Easy 101 173 -72 21:52 6:57
26.78 3:21:57 7:32

* Due to an injury to Leg 1 runner, I doubled-up on our last set running his leg (Leg 25) and the last leg for our van (Leg 30) - Jen took my original Leg 26.

Monday, August 20, 2012

IPv6 in Perl on Windows

The state of IPv6 support in Perl has long been lamented but there are those trying to change this including Paul Evans who's updating many of the networking modules - including the Socket core module - to support IPv6. It's a shame the C compiler shipping with Strawberry Perl still doesn't accommodate IPv6 on Windows.

I've been using Strawberry Perl after switching from ActiveState since version 5.10. I found myself needing more and more XS modules and the ActiveState PPM process was always lagging behind. I had a copy of MinGW gcc - the free Windows port of the 'gcc' compiler - and with 'dmake', I could build my own modules directly from CPAN. However, I had some issues and when I learned about Strawberry Perl including the gcc compiler in the distribution, I switched. I've used Strawberry Perl on Windows XP 32-bit and now Windows 7 64-bit.

I've also recently been doing a lot of IPv6 work and using Perl as a quick development platform to test the IPv6 protocol proved easy with the Net::Frame modules detailed in an early series of blog posts. However, when looking to create sockets with Perl, I found errors, most notably:

Socket::inet_ntop not implemented on this architecture

The lack of this function prevented the display of IPv6 addresses and it's missing complement - inet_pton - prevents the resolution of IPv6. These functions are certainly present in Windows 7.

The issue lies in the header and library files distributed with MinGW gcc compiler in the Strawberry Perl release. They simply don't identify the functions in the headers, nor provide linkage to them in the libraries. Fortunately, tools shipped with the MinGW gcc compiler in Strawberry Perl can be used to remedy the issue and get full IPv6 functionality. Here's how.

First off, we need to update the header file with the function prototypes. I did this by creating a new header file called 'ws2tcpip-win.h':


#ifndef WS2TCPIP_WIN_H
#define WS2TCPIP_WIN_H

typedef USHORT ADDRESS_FAMILY;

#if (NTDDI_VERSION >= NTDDI_VISTA)
WINSOCK_API_LINKAGE INT WSAAPI inet_pton(INT, PCSTR, PVOID);
WINSOCK_API_LINKAGE INT WSAAPI InetPtonW(INT, PCWSTR, PVOID);

WINSOCK_API_LINKAGE PCSTR WSAAPI inet_ntop(INT, PVOID, PSTR, size_t);
WINSOCK_API_LINKAGE PCWSTR WSAAPI InetNtopW(INT, PVOID, PWSTR, size_t);

#define InetPtonA       inet_pton
#define InetNtopA       inet_ntop

#ifdef UNICODE
#define InetPton        InetPtonW
#define InetNtop        InetNtopW
#else
#define InetPton        InetPtonA
#define InetNtop        InetNtopA
#endif

#if INCL_WINSOCK_API_TYPEDEFS
typedef INT (WSAAPI * LPFN_INET_PTONA)(INT Family, PCSTR pszAddrString, PVOID pAddrBuf);
typedef INT (WSAAPI * LPFN_INET_PTONW)(INT Family, PCWSTR pszAddrString, PVOID pAddrBuf);

typedef PCSTR (WSAAPI * LPFN_INET_NTOPA)(INT Family, PVOID pAddr, PSTR pStringBuf, size_t StringBufSize);
typedef PCWSTR (WSAAPI * LPFN_INET_NTOPW)(INT Family, PVOID pAddr, PWSTR pStringBuf, size_t StringBufSize);

#ifdef UNICODE
#define LPFN_INET_PTON          LPFN_INET_PTONW
#define LPFN_INET_NTOP          LPFN_INET_NTOPW
#else
#define LPFN_INET_PTON          LPFN_INET_PTONA
#define LPFN_INET_NTOP          LPFN_INET_NTOPA
#endif

#endif  //  TYPEDEFS
#endif  //  (NTDDI_VERSION >= NTDDI_VISTA)

#endif

Next, copy that file to the appropriate location:

copy ws2tcpip-win.h c:\strawberry\c\x86_64-w64-mingw32\include

and update the existing 'ws2tcpip.h' file in that location with the following:

#include <ws2tcpip-win.h>

Now we need to create the library. This is done by first exporting the symbols from the appropriate DLL file - in this case, ws2_32.dll - and then creating the link library. The version of the MinGW gcc compiler with Strawberry Perl has 'pexports' included, so simply create the DEF file:

pexports c:\windows\system32\ws2_32.dll > ws2_32.def

Now create the library:

dlltool --as-flags=--64 -m i386:x86-64 -k --output-lib libws2_32-new.a --input-def WS2_32.def

Then put the new library in the appropriate location:

copy libws2_32-new.a c:\strawberry\c\x86_64-w64-mingw32\lib

Now we're ready to go. Get the latest release of Socket from CPAN, unzip and extract the tar file. Change into the directory and start the build process:

perl Makefile.PL

Now, before doing the 'dmake', edit the generated Makefile. Find the "DEFINE =" and "LDLOADLIBS =" lines and update them as per below:

DEFINE = -DHAS_GETADDRINFO -DHAS_GETNAMEINFO -DHAS_SOCKADDR_IN6 -DHAS_IP_MREQ -DHAS_IP_MREQ_SOURCE -DHAS_IPV6_MREQ -DHAS_INETNTOP -DHAS_INETPTON -DAF_INET6

LDLOADLIBS = -lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32 -lws2_32-new

Finally, 'dmake', 'dmake test' and 'dmake install' and you're good to go with IPv6!

Friday, August 17, 2012

The Adventures of Rich and Vince




Characters courtesy of SP Studio
 

Copyright © VinsWorld. All Rights Reserved.