Qemu (kvm) internal network setup

flattr this!

I got a new notebook, a nice Thinkpad T61 with virtualization technology. I need to run some Windows system for development so I’ve decided to use the Kernel based Virtual Machine (KVM). The VMs should communicate over an internal network but should have access to the internet and I want access via network to them. So I setup a bridge with TUN/TAP devices masqueraded to my normal interface.


               HOST            QEMU GUEST1
        +---------------+   +--------------+
        | 10.10.5.158   |   |              |
 LAN ---+---- eth0      |   |              |
        |               |   |              |    QEMU GUEST2
        |   +------+ +--+---+---- nic0     |   +--------------+
        |   | tap0---+  |   |192.168.100.5 |   |              |
        |   | tap1---+  |   +--------------+   |              |
        |   +------+ |  |                      |              |
        |     br0    +--+----------------------+---- nic0     |
        |192.168.100.254|                      |192.168.100.1 |
        +---------------+                      +--------------+


Needed packages:

tunctl (uml-utilities)
bridge-utilities
kvm

Setup the network:

Create a file call kvm-network with the following content and make it executeable.

#!/bin/bash

KVMNET_UID=1000
KVMNET_GID=$(grep kvm /etc/group | cut -d ':' -f 3)

# number of TUN/TAP devices to setup
NUM_OF_DEVICES=3

case $1 in
        start)
                modprobe kvm
                modprobe kvm_intel

                modprobe tun
                echo "Setting up bridge device br0"
                brctl addbr br0
                ifconfig br0 192.168.100.254 netmask 255.255.255.0 up
                for ((i=0; i < NUM_OF_DEVICES ; i++)); do
                        echo -n "Setting up "
                        tunctl -b -g ${KVMNET_GID} -t kvmnet$i
                        #tunctl -b -u ${KVMNET_UID} -t kvmnet$i
                        brctl addif br0 kvmnet$i
                        ifconfig kvmnet$i up 0.0.0.0 promisc
                done
                SuSEfirewall2 stop
                SuSEfirewall2
        ;;
        stop)
                for ((i=0; i < NUM_OF_DEVICES ; i++)); do
                        ifconfig kvmnet$i down
                        brctl delif br0 kvmnet$i
                        tunctl -d kvmnet$i
                done
                ifconfig br0 down
                brctl delbr br0
                SuSEfirewall2 stop
                SuSEfirewall2

                rmmod kvm_intel
                rmmod kvm
        ;;
        *)
                echo "Usage: $(basename $0) (start|stop)"
        ;;
esac

br0 is the gateway to the external network.

Setting up the firewall:

Edit /etc/sysconfig/SuSEfirewall and set the following variables:

FW_DEV_INT="br0 qtap0 qtap1 qtap2 qtap3 qtap4"
FW_ROUTE="yes"
FW_MASQUERADE="yes"
FW_MASQ_NETS="192.168.100.0/24"
FW_PROTECT_FROM_INT="no"

If you don't run a SUSE system use the following lines to setup masquerading:

echo "1" > /proc/sys/net/ipv4/ip_forward
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

On the guest you have to set the default gateway to 192.168.100.254 which is the bridge br0 and take a look in /etc/resolv.conf to get the name servers. I run a Windows 2003 Server as a guest which is the dhcp and name server for the other guests (Vista, several Linux installations).

Setting up qemu

Guest 1:

#!/bin/bash

qemu-kvm /path/to/vm.img \
          -net nic,model=rtl8139,macaddr=52:54:00:12:34:56 \
          -net tap,ifname=qtap0,script=no \
          -m 256 \
          -smp 1 \
          -usb \
          -usbdevice tablet \
          -localtime

Guest 2:

#!/bin/bash

qemu-kvm /path/to/vm2.img \
          -net nic,model=rtl8139,macaddr=52:54:00:12:34:57 \
          -net tap,ifname=qtap1,script=no \
          -m 256 \
          -smp 1 \
          -usb \
          -usbdevice tablet \
          -localtime

Note that the VMs have different MAC addresses. It took me a long time to find why I couldn't ping from one guest to another ;) By the way, one of the guests is running Vista, which runs smoothly on my machine with KVM.

33 thoughts on Qemu (kvm) internal network setup

  1. Hi, Andreas !

    Thanks a LOT for KVM networking setup advice. Can you please tell me if this will work with Kernel 2.6.22, and not SuSE 2.6.18 ? Something has changed in the bridging stuff starting from 2.6.20 kernel.

    Thanks in advance for any suggestion(s)

  2. Hi

    I’ve spent some time looking for he reason why I could not ping between guests. When I check the bridge from the host brctl showmacs br0 it shows different addresses. Your wink prompted me to run ifconfig on each guest, sure enough, they are the same mac. Thank you!

  3. Hey,

    I couldn’t make it work. My guests (both Ubuntu & WinXP) haven’t got connection. I think everything was setup exactly like the instruction. I just slightly changed the UserID as 0 since I ran as root.

    I wonder if is that correct when I assign NIC_0 (guest_1) as the same value as TAP_0 (Tap)?

    Thanks,

  4. I’m sorry for last post since my firewall system wasn’t on as I thought. Using 2-line script to setup the masquerading is great. I got everything works now. Thanks!

  5. I’m trying to get this working in Windows 2003, but the machine hangs dead when installing the kvmnet.sys driver. Here is my startup script:
    qemu-kvm \
    -M pc \
    -cdrom /ISO/kvm-driver-disc-20080318.iso \
    -m 512 \
    -name janus1 \
    -smp 2 \
    -net nic,model=virtio \
    -net tap,ifname=tap0,script=no \
    -no-acpi \
    /dev/nunez/janus

    Thanks for great guide.

  6. Nice tutorial but i can’t make it work on Opensuse 11.
    Everything seems fine except an “commit failed on table filter: No chain/target/match by that name” error with firewall (standard setup).
    VM can ping the host but can’t go on the internet.
    It seems my problem came from MASQUERADE (unable to activate it) but i haven’t found how to solve it.
    Any help appreciated.

  7. @Mic:
    edit /etc/sysconfig/SuSEfirewall2 instead of /etc/sysconfig/SuSEfirewall.
    FW_FORWARD_ALWAYS_INOUT_DEV is deprecated, so don’t use it.

    @GlaDiaC:
    Great work. You could have avoid some trouble with MAC adresses by setting them in kvm-network.sh with something like this:
    ip link set dev qtap$i up promisc on address 52:54:00:12:34:5$i

  8. “Note that the VMs have different MAC addresses. It took me a long time to find why I couldn’t ping from one guest to another”

    This did the trick for me as well. I was able to ping one or the other, but not both back to back. It was weird. In retrospect it must have been having a hard time figuring out which “NIC” to route to.

    Now I can contact multiple KVM instances over the network without issue. Thanks for the tip!

  9. Hi, Andreas !

    Thanks a lot, this is perfect!

    I repeat if you cannot communicate between both VM, BEWARE of MAC addresses, I looked for hours before find out this tutorial ;)

  10. The br/tap script is awesome, it brings the net devices right up. I can assign the correct IP/netmask/gateway to each VM via the mac address with dhpcd3. The VM’s can ping out to the internet just fine. However I can not ping to the VMs. If I try assigning an ip to qtap1 (ifconfig qtap1 192.168.100.1), then I can ping it, however the network on that VM stops. ifconfig qtap1 0.0.0.0 revives it again. It makes sense that it can’t be pinged because that IP does not show up under ifconfig. This is what a qtap interface looks like that works outbound only:

    qtap1 Link encap:Ethernet HWaddr 00:ff:2e:0b:24:af
    inet6 addr: fe80::2ff:2eff:fe0b:24af/64 Scope:Link
    UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
    RX packets:111 errors:0 dropped:0 overruns:0 frame:0
    TX packets:23 errors:0 dropped:27 overruns:0 carrier:0
    collisions:0 txqueuelen:500
    RX bytes:15056 (14.7 KiB) TX bytes:2322 (2.2 KiB)

    I’m running the 2.6.26 kernel, did something change?

    P.S. Your ascii art diagram of the network is the best illustration of bridged networking I’ve come across.

  11. The only IP that can ping 192.168.100.1 is the XP VM itself. Adding eth0 to br0 changes the IP of br0 to what eth0 was; 192.168.192.125. I have dhcpcd giving the VM on qtap1 an IP of 192.168.192.75. Again, the only IP that can ping 192.168.192.75 is 192.168.192.75. If I could get the Host Linux machine to be able to ping the guests I could set up port forwarding on iptables to resolve the rest.

  12. Ok, I can’t ping 192.168.192.75, however nmap detects it:
    All 1715 scanned ports on 192.168.192.75 are filtered
    MAC Address: 52:54:00:12:34:51 (QEMU Virtual NIC)

    I guess this infers that it’s a firewall issue after all?

  13. I was overlooking the obvious – yes it was a firewall issue – but from the XP guest. I guess I’m not used to the XP firewall stopping ICMP ping packets but this fresh install did.

    Again – thanks for the great how-to.

  14. Can someone help me . I have a Fedora 10 guest running on a KVM with Fedora9 host .
    I see vibr0 up and running with a valid ip already . But all my attempts to get the networking going on the guest fails .
    what should be the right argument to the -net when i start the guest so that it works .

  15. Pingback Stephan Kulow: Qemu network speed
  16. Thanks very much for this article. Concise and precise. Requires basic networking knowledge in Linux but that is a fair assumption.

  17. Pingback Emile “iMil” Heitor ’s home » Blog Archive » sous les pavés, NetBSD
  18. Pingback andy.edmonds.be › links for 2010-01-31
  19. Wow. That is a great guide. It cleared lot of my doubts. The script worked great. Great work.

  20. hi
    how to connect OS “ubuntu 9:10″ as the host, with Qemu as a guest.?
    I can not connect from the host to geust, how to connect?
    I use Samba to connect the host to the guest.
    I was able to ping from guest to host, but from host to guest can not ping .. why?

    the answer to my email please
    thanks.

  21. Why you start the guest with `ifname=qtap0` while your created interface was `kvmnet0`?
    My guest didn’t ping the host, but as soon as I start that with `ifname=kvmnet0`, it works!

    I don’t know the real solution and I don’t know what kind of problem may occurred by this change. Would you please explain a little about it.

    Thanks

Leave a Reply


*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Post comment

What is Persona?