Qemu (kvm) internal network setup
I got a new notebook, a nice Thinkpad T61 with virtualization technology. I need to run some Windows system for development so I’ve decided to use the Kernel based Virtual Machine (KVM). The VMs should communicate over an internal network but should have access to the internet and I want access via network to them. So I setup a bridge with TUN/TAP devices masqueraded to my normal interface.
HOST QEMU GUEST1
+---------------+ +--------------+
| 10.10.5.158 | | |
LAN ---+---- eth0 | | |
| | | | QEMU GUEST2
| +------+ +--+---+---- nic0 | +--------------+
| | tap0---+ | |192.168.100.5 | | |
| | tap1---+ | +--------------+ | |
| +------+ | | | |
| br0 +--+----------------------+---- nic0 |
|192.168.100.254| |192.168.100.1 |
+---------------+ +--------------+
Needed packages:
tunctl (uml-utilities)
bridge-utilities
kvm
Setup the network:
Create a file call kvm-network with the following content and make it executeable.
#!/bin/bash
KVMNET_UID=1000
KVMNET_GID=$(grep kvm /etc/group | cut -d ':' -f 3)
# number of TUN/TAP devices to setup
NUM_OF_DEVICES=3
case $1 in
start)
modprobe kvm
modprobe kvm_intel
modprobe tun
echo "Setting up bridge device br0"
brctl addbr br0
ifconfig br0 192.168.100.254 netmask 255.255.255.0 up
for ((i=0; i < NUM_OF_DEVICES ; i++)); do
echo -n "Setting up "
tunctl -b -g ${KVMNET_GID} -t kvmnet$i
#tunctl -b -u ${KVMNET_UID} -t kvmnet$i
brctl addif br0 kvmnet$i
ifconfig kvmnet$i up 0.0.0.0 promisc
done
SuSEfirewall2 stop
SuSEfirewall2
;;
stop)
for ((i=0; i < NUM_OF_DEVICES ; i++)); do
ifconfig kvmnet$i down
brctl delif br0 kvmnet$i
tunctl -d kvmnet$i
done
ifconfig br0 down
brctl delbr br0
SuSEfirewall2 stop
SuSEfirewall2
rmmod kvm_intel
rmmod kvm
;;
*)
echo "Usage: $(basename $0) (start|stop)"
;;
esac
br0 is the gateway to the external network.
Setting up the firewall:
Edit /etc/sysconfig/SuSEfirewall and set the following variables:
FW_DEV_INT="br0 qtap0 qtap1 qtap2 qtap3 qtap4" FW_ROUTE="yes" FW_MASQUERADE="yes" FW_MASQ_NETS="192.168.100.0/24" FW_PROTECT_FROM_INT="no"
If you don't run a SUSE system use the following lines to setup masquerading:
echo "1" > /proc/sys/net/ipv4/ip_forward iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
On the guest you have to set the default gateway to 192.168.100.254 which is the bridge br0 and take a look in /etc/resolv.conf to get the name servers. I run a Windows 2003 Server as a guest which is the dhcp and name server for the other guests (Vista, several Linux installations).
Setting up qemu
Guest 1:
#!/bin/bash
qemu-kvm /path/to/vm.img \
-net nic,model=rtl8139,macaddr=52:54:00:12:34:56 \
-net tap,ifname=qtap0,script=no \
-m 256 \
-smp 1 \
-usb \
-usbdevice tablet \
-localtime
Guest 2:
#!/bin/bash
qemu-kvm /path/to/vm2.img \
-net nic,model=rtl8139,macaddr=52:54:00:12:34:57 \
-net tap,ifname=qtap1,script=no \
-m 256 \
-smp 1 \
-usb \
-usbdevice tablet \
-localtime
Note that the VMs have different MAC addresses. It took me a long time to find why I couldn't ping from one guest to another 😉 By the way, one of the guests is running Vista, which runs smoothly on my machine with KVM.
on ubuntu 7.04, the package “bridge-utilities” is called “bridge-utils”
Hi, Andreas !
Thanks a LOT for KVM networking setup advice. Can you please tell me if this will work with Kernel 2.6.22, and not SuSE 2.6.18 ? Something has changed in the bridging stuff starting from 2.6.20 kernel.
Thanks in advance for any suggestion(s)
Hello,
I’ve done this on my notebook which is running openSUSE 10.3 Beta with Kernel 2.6.22.
Hi
I’ve spent some time looking for he reason why I could not ping between guests. When I check the bridge from the host brctl showmacs br0 it shows different addresses. Your wink prompted me to run ifconfig on each guest, sure enough, they are the same mac. Thank you!
Hey,
I want the guest OS’s to join the other networks windows work group, how is this possible?
Dave, please be more precise!
Hey,
I couldn’t make it work. My guests (both Ubuntu & WinXP) haven’t got connection. I think everything was setup exactly like the instruction. I just slightly changed the UserID as 0 since I ran as root.
I wonder if is that correct when I assign NIC_0 (guest_1) as the same value as TAP_0 (Tap)?
Thanks,
I’m sorry for last post since my firewall system wasn’t on as I thought. Using 2-line script to setup the masquerading is great. I got everything works now. Thanks!
I’m trying to get this working in Windows 2003, but the machine hangs dead when installing the kvmnet.sys driver. Here is my startup script:
qemu-kvm \
-M pc \
-cdrom /ISO/kvm-driver-disc-20080318.iso \
-m 512 \
-name janus1 \
-smp 2 \
-net nic,model=virtio \
-net tap,ifname=tap0,script=no \
-no-acpi \
/dev/nunez/janus
Thanks for great guide.
Nevermind, figured out that part, seems that the version 1.0.0.0 doesn’t play nice. Upgraded to 1.2.0.0 and work like a charm – 1Gbps in the system tray…:))
I’ve set up bridging using the guide at: http://www.linux-kvm.com/content/tip-how-get-maximum-network-performance-using-paravirtual-drivers-and-bridged-networking but doesn’t work, my virt machine can’t get na ip from dhcp. Help greatly appreciated
Nice tutorial but i can’t make it work on Opensuse 11.
Everything seems fine except an “commit failed on table filter: No chain/target/match by that name” error with firewall (standard setup).
VM can ping the host but can’t go on the internet.
It seems my problem came from MASQUERADE (unable to activate it) but i haven’t found how to solve it.
Any help appreciated.
@Mic:
edit /etc/sysconfig/SuSEfirewall2 instead of /etc/sysconfig/SuSEfirewall.
FW_FORWARD_ALWAYS_INOUT_DEV is deprecated, so don’t use it.
@GlaDiaC:
Great work. You could have avoid some trouble with MAC adresses by setting them in kvm-network.sh with something like this:
ip link set dev qtap$i up promisc on address 52:54:00:12:34:5$i
thanks, helped me to setup guest networking 🙂
“Note that the VMs have different MAC addresses. It took me a long time to find why I couldn’t ping from one guest to another”
This did the trick for me as well. I was able to ping one or the other, but not both back to back. It was weird. In retrospect it must have been having a hard time figuring out which “NIC” to route to.
Now I can contact multiple KVM instances over the network without issue. Thanks for the tip!
Hi, Andreas !
Thanks a lot, this is perfect!
I repeat if you cannot communicate between both VM, BEWARE of MAC addresses, I looked for hours before find out this tutorial 😉
The br/tap script is awesome, it brings the net devices right up. I can assign the correct IP/netmask/gateway to each VM via the mac address with dhpcd3. The VM’s can ping out to the internet just fine. However I can not ping to the VMs. If I try assigning an ip to qtap1 (ifconfig qtap1 192.168.100.1), then I can ping it, however the network on that VM stops. ifconfig qtap1 0.0.0.0 revives it again. It makes sense that it can’t be pinged because that IP does not show up under ifconfig. This is what a qtap interface looks like that works outbound only:
qtap1 Link encap:Ethernet HWaddr 00:ff:2e:0b:24:af
inet6 addr: fe80::2ff:2eff:fe0b:24af/64 Scope:Link
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:111 errors:0 dropped:0 overruns:0 frame:0
TX packets:23 errors:0 dropped:27 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:15056 (14.7 KiB) TX bytes:2322 (2.2 KiB)
I’m running the 2.6.26 kernel, did something change?
P.S. Your ascii art diagram of the network is the best illustration of bridged networking I’ve come across.
If you want to connect to the machines from external, you have to add your network card to the bridge.
The only IP that can ping 192.168.100.1 is the XP VM itself. Adding eth0 to br0 changes the IP of br0 to what eth0 was; 192.168.192.125. I have dhcpcd giving the VM on qtap1 an IP of 192.168.192.75. Again, the only IP that can ping 192.168.192.75 is 192.168.192.75. If I could get the Host Linux machine to be able to ping the guests I could set up port forwarding on iptables to resolve the rest.
Ok, I can’t ping 192.168.192.75, however nmap detects it:
All 1715 scanned ports on 192.168.192.75 are filtered
MAC Address: 52:54:00:12:34:51 (QEMU Virtual NIC)
I guess this infers that it’s a firewall issue after all?
I was overlooking the obvious – yes it was a firewall issue – but from the XP guest. I guess I’m not used to the XP firewall stopping ICMP ping packets but this fresh install did.
Again – thanks for the great how-to.
Can someone help me . I have a Fedora 10 guest running on a KVM with Fedora9 host .
I see vibr0 up and running with a valid ip already . But all my attempts to get the networking going on the guest fails .
what should be the right argument to the -net when i start the guest so that it works .
Thanks very much for this article. Concise and precise. Requires basic networking knowledge in Linux but that is a fair assumption.
Dude, I’ve been looking for a guide like this. Thanks.
I use gentoo with kernel 2.6.31 and have kvm-88 build in the kernel so I had to comment the load modules.
thanks for the guild, as my wiki (http://en.gentoo-wiki.com/wiki/KVM#Networking) used it 😉
Wow. That is a great guide. It cleared lot of my doubts. The script worked great. Great work.
hi
how to connect OS “ubuntu 9:10” as the host, with Qemu as a guest.?
I can not connect from the host to geust, how to connect?
I use Samba to connect the host to the guest.
I was able to ping from guest to host, but from host to guest can not ping .. why?
the answer to my email please
thanks.
Hello,
try different driver for KVM VGA, like curris. Check Qemu-kvm page, works better on Win7/Vista VMs.
Why you start the guest with `ifname=qtap0` while your created interface was `kvmnet0`?
My guest didn’t ping the host, but as soon as I start that with `ifname=kvmnet0`, it works!
I don’t know the real solution and I don’t know what kind of problem may occurred by this change. Would you please explain a little about it.
Thanks
Typo 🙂
Very useful and with beautiful ASCII diagram 😉