KVM on a Gentoo home router host with non-nat bridged netwk

mondjef

ok, I have been all over the net and back again and it is apparent that my knowledge of networking knowledge needs to be improved. I have a gentoo linux home router that I setup using the following guide to setup right down to the "T" http://www.gentoo.org/doc/en/home-router-howto.xml. Now I would like to set up KVM virtualization on this machine (the host).

Through numerous guides I have managed to configure the kernel as necessary and install the needed packages via emerge. Now my problem is getting the network set up so that the guest OS can receive an IP address on the local network and are able to communication with the outside and on the LAN. For understanding how to go about setting up the network I have looked at the two following links for setting up Bridged networking between the host and the guest with no success http://en.gentoo-wiki.com/wiki/KVM and http://www.linux-kvm.org/page/KvmOnGentoo. When I set up the bridge I get no internet access from other clients connected to this Gentoo home router and these clients can no longer SSH into the Gentoo home router, in other words it seems the clients connected to this router no longer receive IP addresses via DHCP. Is there something that I am missing that would be unique to my setup...do I need to alter dnsmasq and/or iptables to get this to work? Any help would be greatly appreciated.

Let me know if any config files are needed for understanding.

Hu · Administrator Joined: 06 Mar 2007 Posts: 23062

If I understand correctly, you are already using NAT for your real machines, but you want to bridge the guest. Is this correct? If so, why do you want to do this? Mixing bridging with NAT can be done, but is often more complex than just using NAT consistently.

Please post the output of emerge --info app-emulation/qemu-kvm ; ps -efwwww | grep qemu ; iptables-save -c | cat -n; brctl show; ip addr show; grep -E '^[^#]' /path/to/dnsmasq.conf.

mondjef · Posted: Sat Jan 28, 2012 8:56 pm Post subject:

Hu · Administrator Joined: 06 Mar 2007 Posts: 23062

Please use code tags to wrap output from commands. It groups the output nicely and ensures a font that is often more suitable for large data dumps.

If you want other machines on the LAN to have access to the guest, then bridging is best. You can do tricks with NAT/port forwarding to expose selected guest services to the LAN, but bridging will be cleaner in the long term.

Placing both NICs in a single subnet is rarely wise. I suspect it only works for you now because of the use of PPP for your upstream. It would help if you could show the output as it was when the setup was broken. I have not run dnsmasq on an interface enslaved to a bridge, but I expect that it needs to be reconfigured to listen on br0. I know that your firewall rules are written in such a way that the LAN clients will fail when you switch to the bridge. Packets which arrive on an interface enslaved to a bridge use the name of the bridge, not the name of the enslaved interface, when performing matching. Similarly, packets leaving through an enslaved interface will use the bridge name. If you need to write a rule which knows which enslaved interface received the packet, you can use the physdev match to inspect that. Thus, to use the bridge, you need to s/eth0/br0/ all your firewall rules. Of course, if you change them in place, then they will work only when you use the bridge and will fail if you switch back to an unbridged setup. Using a bridge with only a single port enslaved is fine, so after the rules are converted, everything should work independent of whether a guest is actually running at the time.

mondjef · Posted: Sat Jan 28, 2012 9:51 pm Post subject:

Sorry about the code tags, should have known as I read enough of the posts but rarely get the chance to post and help someone else out...one day.

Ok, at one point in my configuration I had changed dnsmasq to use the br0 interface instead of eth0. I had wondered about the iptables rules having something to do with it. I will reconfigure the bridge, have dnsmasq to use br0 instead of eth0, and last but not least change all my iptables rules to use br0 instead of eth0 and report back whether I can have beer yet or not. Thanks again for taking the time to trouble shout this.

NeddySeagoon · Posted: Sat Jan 28, 2012 10:06 pm Post subject:

mondjef,

You get to have a beer anyway.

I use Virtual Machine Manager on a remote system for managing my KVMs

The real hardware has four NICs all bridged to a VM router/firewall done with shorewall.
The NICs are for the Internet, the DMZ, wireless and protected wired. I don't use PPPoE as my VDSL 'modem' does that.

I had to draw out the networking several times, with IP numbers, to get it right as I wanted minimal downtime when I switched from a Smoothwall box.
I also have KVMs on that system for a media server and mail server.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

mondjef · Posted: Sat Jan 28, 2012 10:11 pm Post subject:

Success! I can ssh into the box from a client on same LAN and access internet. I will continue with getting the VM running now. Thank you very much for your help.

mondjef · Posted: Sat Jan 28, 2012 10:14 pm Post subject:

NeddySeagoon · Posted: Sat Jan 28, 2012 10:30 pm Post subject:

mondjef,

I had been using a 4 way Smoothwall for years for security and the time came when it wasn't fast enough to handel my downlink.

I didn't intend to use bridging - I wanted to use PCI passthrough but the 4 way NIC I bought did not support it. Ooops.
The bare metal install is a minimal hardened install for supporting KVM, which is what I intended. It has its own Physical Volume in a lvm set.

I had to fall back to bridging when PCI passthrough would not work for me, or buy another 4 way NIC.
The KVMs all share a different Physical Volume and have one or more logical volumes each.
I use the virtio drivers as the performace is better then the emulated hardware plus drivers. Using logical volumes for the KVMs cuts out the overhead of a filesystem in a file too.

I did start writing it up but its by no means complete.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

mondjef · Posted: Sun Jan 29, 2012 7:42 pm Post subject:

ok, finally had time to install a VM but now I am having network problems with clients/Guest OS connected to this machine. It seems no clients (DHCP) or Guest OS on the same LAN (receive IP via DHCP also) can communicate with each other. Anyone have any suggestions on where and how to trouble shout this?

jamapii · l33t Joined: 16 Sep 2004 Posts: 637

I guess most problems will result from lack of doing

jamapii · l33t Joined: 16 Sep 2004 Posts: 637

Hu · Administrator Joined: 06 Mar 2007 Posts: 23062

mondjef · Posted: Fri Feb 03, 2012 2:58 am Post subject:

ok, finally had some more time to trouble shoot this as far as I am capable of given my current linux abilities.

here is how things look:

Host:
ppp0 (eth1)--> wan assigned public ip address by ISP via DHCP
br0 (192.168.0.1)--> bridged with eth0, all LAN clients and VMs connected to this bridge. Both lan clients and vm assigned ips via DHCP from the host so everyone is on the same subnet.

NeddySeagoon · Posted: Fri Feb 03, 2012 7:01 pm Post subject:

mondjef,

This has to be wrong. You may not have two interfaces in the same subnet