| View previous topic :: View next topic |
| Author |
Message |
dkasak
n00b
Joined: 10 Feb 2012
Posts: 61
|
 Posted: Tue Jun 10, 2014 6:43 am Post subject: Bridged eth0, KVM, access to guest from network [solved] |
 |
|
Hi all. I'm having an interesting problem. I have to run a bunch of VMs, that should sit on the same network as other PCs on my local network. I've set up my bridged networking ( bridging an ethernet device, not wireless ) as per the instructions for systemd users, at: http://wiki.gentoo.org/wiki/Network_bridge#Host_Configuration_.28using_systemd.29. ie I have:
| Code: | # bridge ports defined empty to avoid DHCP being run for their configuration
config_eth0="null"
# Bridge-config
config_br0="192.168.1.128"
routes_br0="192.168.1.254"
brctl_br0="setfd 0 sethello 10 stp on"
bridge_br0="eth0" |
... and the /etc/systemd/system/bridge@.service file as per the instructions.
I've set up a couple of VMs now. I can ssh into them from the host, and also ssh to the host from the guest. But I can't ssh to guests from other PCs on the network, or visa-versa. I can, however, ping things. eg:
Pinging a VM from another PC on network:
| Code: | ultrabook ~ # ping 192.168.1.130
PING 192.168.1.130 (192.168.1.130) 56(84) bytes of data.
64 bytes from 192.168.1.130: icmp_seq=1 ttl=64 time=2.21 ms
64 bytes from 192.168.1.130: icmp_seq=2 ttl=64 time=1.94 ms
^C
--- 192.168.1.130 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 1.941/2.078/2.215/0.137 ms
ultrabook ~ #
|
Attempting to ssh into it:
| Code: | ultrabook ~ # ssh 192.168.1.130
ssh: connect to host 192.168.1.130 port 22: No route to host
ultrabook ~ #
|
Why would that be? I'm admittedly not so hot on networking 
Oh - on the VM host:
| Code: | tesla ~ # brctl show
bridge name bridge id STP enabled interfaces
br0 8000.d050991071d2 no eth0
vnet0
virbr0 8000.000000000000 yes
virbr1 8000.525400af33dd yes virbr1-nic
vnet1
vnet2
tesla ~ #
|
Any help greatly appreciated 
Last edited by dkasak on Fri Jun 13, 2014 12:09 am; edited 1 time in total |
|
| Back to top |
|
 |
schorsch_76
Guru
Joined: 19 Jun 2012
Posts: 452
|
| Posted: Tue Jun 10, 2014 6:57 am Post subject: |
|
|
You need to add tap devices to the bridge. These tap devices are used by the qemu/kvm VMs to access the network.
I do it the same. Here is my config:
| Code: |
#################################################
# the bridge br0 contains eth0 and tap0
# tap0: virtual machine
# tap1: VPN
# eth0: Network
#################################################
config_eth0="null"
tuntap_tap0="tap"
config_tap0="null"
tunctl_tap0="-t tap0 -u me"
config_br0="192.168.178.19/24"
routes_br0="default via 192.168.178.1"
brctl_br0="setfd 0 sethello 10 stp off"
bridge_br0="eth0 tap0"
depend_br0() {
need net.eth0
need net.tap0
}
|
|
|
| Back to top |
|
|
dkasak
n00b
Joined: 10 Feb 2012
Posts: 61
|
| Posted: Tue Jun 10, 2014 7:40 am Post subject: |
|
|
| Thanks for the response. Alas, this config doesn't work for me, and in fact it breaks connectivity completely. Maybe it's because I'm using systemd? I end up with BOTH eth0 and br0 'active' and with IP 192.168.1.128. I've never seen eth0 get an IP address when using bridging. I guess for now, I can try to revert to 'vanilla' networking, and use KVM's "macvtap" thing. It's a bit annoying that you can access the host from the guest, but I'm looking at workarounds, and they're not TOO bad. |
|
| Back to top |
|
|
schorsch_76
Guru
Joined: 19 Jun 2012
Posts: 452
|
| Posted: Tue Jun 10, 2014 8:02 am Post subject: |
|
|
systemd should not have effect on the network. I am not sure how systemd manages "depends_br0()". I use plain openrc without networkmanager/wicd/whatever.
In bridge setup only the bridge interface has an ip address. Maybe you need to shut the network complete down and start it again. The important thing is, that the tap device and eth0 are in the same bridge and only the br0 has the ip adress. The tap device get assigned to the QEMU VM's. If you use systemd or openrc to setup this config doesn't matter. |
|
| Back to top |
|
|
dkasak
n00b
Joined: 10 Feb 2012
Posts: 61
|
| Posted: Tue Jun 10, 2014 9:45 am Post subject: |
|
|
Yeah I rebooted multiple times while attempting to figure things out.
I'm pretty sure systemd is involved in all this, as it's the init scripts that parse the config files, bring up the interfaces, etc. The WIKI ( http://wiki.gentoo.org/wiki/Network_bridge#Host_Configuration_.28using_systemd.29 ) includes a new service file /etc/systemd/system/bridge@.service that has 'simple' support for bridging, but doesn't appear to bring up the tap0 interface. So I guess this isn't "strictly" a systemd issue, but an issue with the much simpler systemd service file that I'm using, compared to the more complex networking scripts that Gentoo has had up until systemd arrived. |
|
| Back to top |
|
|
schorsch_76
Guru
Joined: 19 Jun 2012
Posts: 452
|
| Posted: Tue Jun 10, 2014 9:52 am Post subject: |
|
|
| Ok, then it is most probably a issue with the systemd service file. |
|
| Back to top |
|
|
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Tue Jun 10, 2014 2:55 pm Post subject: |
|
|
Lovely tip schorsch. I've saved it off in my tips thread.
Would be cool to see the setup for the other VPN interface as well.
Thanks,
steveL. |
|
| Back to top |
|
|
Aiken
Apprentice
Joined: 22 Jan 2003
Posts: 239
Location: Toowoomba/Australia
|
| Posted: Tue Jun 10, 2014 10:23 pm Post subject: |
|
|
| schorsch_76 wrote: | You need to add tap devices to the bridge. These tap devices are used by the qemu/kvm VMs to access the network.
|
I do it differently. Instead of setting up tapX in /etc/conf.d/net I let quemu configure the tap device at runtime. At boot time my bridge consists of eth0, eth1 and wlan0. The tap devices are created by qemu at runtime and it has options to name scripts to be run when it brings up and takes down the tap device. Qemu passes the name of the tap device to the script. I use simple scripts for when I wanted to start qemu and for qemu-ifup. This adds the tapX devices to br0 on demand instead of preallocating them at boot time.
| Code: |
qemu-system-x86_64 -enable-kvm -cpu host \
-drive file=${1},if=virtio \
-net nic -net tap,script=qemu-ifup,downscript=qemu-ifdown \
-m 2000M \
-daemonize \
-vnc :$2 -smp 2 \
-pidfile pid-$2 \
-usbdevice tablet
|
| Code: |
#!/bin/bash
#echo Interface $1
ifconfig $1 0.0.0.0 promisc up
brctl addif br0 $1
|
_________________
Beware the grue. |
|
| Back to top |
|
|
toralf
Developer
Joined: 01 Feb 2004
Posts: 3940
Location: Hamburg
|
| Posted: Tue Jun 10, 2014 10:50 pm Post subject: |
|
|
| I'm just curious about qemu-ifdown script ... |
|
| Back to top |
|
|
Aiken
Apprentice
Joined: 22 Jan 2003
Posts: 239
Location: Toowoomba/Australia
|
| Posted: Tue Jun 10, 2014 11:15 pm Post subject: |
|
|
My qemu-ifdown script only had an echo in it to see what arguments were passed to it. Thinking I forgot to do anything with it as when that qemu instance stops the tap device is destroyed and removed from the bridge.
| Code: |
#!/bin/bash
#echo Interface ${1}
brctl delif br0 ${1}
|
While writing this post I changed my qemu-ifdown to that. That removes the tapX device from the bridge instead of hoping it is automatically removed when qemu stops.
_________________
Beware the grue. |
|
| Back to top |
|
|
dkasak
n00b
Joined: 10 Feb 2012
Posts: 61
|
| Posted: Wed Jun 11, 2014 9:37 am Post subject: |
|
|
OK I've changed my approach. virt-manager, and quite possibly libvirt and other related things, are flakey shit. I had a massive number of iptables rules, most of them totally invalid, which were responsible for not only 75% of my VM issues, but also prevented the system from booting properly, and eventually Gnome from starting at all. This seems to be related to me passing br0 to virt-manager and telling it to set up VMs using this interface.
I ended up deleting /etc/libvirt and starting again, passing in my disk images, and avoiding using gentoo's "kinda half-way-there" bridged support with systemd. I now have my normal eth0 active, and I've set up macvtap bridging. I then created a host-only VPN and attached a 2nd network interface to it. So now:
- PCs on the LAN can access the guest via the macvtap interface
- the host can access the guest via the host-only VPN
<phew> 
Incidentally, it would be great if this wasn't necessary - I don't remember this being necessary with other VMs, but then I guess I can't complain - it's open-source.
If I abandoned virt-manager and used the command-line, I might have a better experience, but I unfortunately don't have time for that now - this is for work, required yesterday. If I have to abandon virt-manager, I might as well go back to virtualbox / vmware, and run with an 'evaluation' license or whatever  |
|
| Back to top |
|
|
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Thu Jun 12, 2014 5:49 pm Post subject: |
|
|
| dkasak wrote: | I now have my normal eth0 active, and I've set up macvtap bridging. I then created a host-only VPN and attached a 2nd network interface to it. So now:
- PCs on the LAN can access the guest via the macvtap interface
- the host can access the guest via the host-only VPN
|
Great :) Can you post (redacted-for-work) configs to show the setup, and how the various things tie together?
Even if it's only two files, or you just refer to above, it helps to have the specific solution to the problem asked, as a guide to others in the future. |
|
| Back to top |
|
|
dkasak
n00b
Joined: 10 Feb 2012
Posts: 61
|
| Posted: Fri Jun 13, 2014 12:08 am Post subject: |
|
|
OK sure. It's pretty simple. I'd taken some screenshots and gone to upload them, but ... the forums don't let you post images. WTF? Whatever. I use virt-manager to configure my VMs. Those who want to use the command-line can post the command-line alternatives ...
First I go into virt-manager, and double-click on 'localhost (QEMU). This opens a 'Connection Details' window. Click the 'Virtual Networks' page / tab / whatever. Click on the 'add' button in the bottom-left of the window. Fill in the wizard. It doesn't really matter what network you define, I guess as long as it's not the same network as your LAN.
Typically people would use the built-in DHCP server, but for my purpose I disabled this, as one of the VMs that I'm using actually needed a VPN for guest-to-guest communication, and it provides it's own DHCP server. If you have something similar going on in your guests, note that your host will create a network interface that sits inside this VPN, and you should be careful that you don't have multiple DHCP servers, or IP address clashes between host and guests.
Next I go to the guest configuration, and define ...
1st network interface:
| Code: | Source device: Host device eth0: macvtap
Device model: virtio
Source mode: VEPA |
2nd network interface:
| Code: | Virtual network 'nznet' : Isolated network, internal and host routing only
Device model: virtio |
The 'Device model' you choose must be supported in the guest(s) you're running. 'virtio' is recommended if your guests have kernel support for this device. Otherwise use something else. The 'Source mode' in the device running in 'macvtap' mode doesn't appear to make much difference for what I'm using it for.
After you've configured this way, everything on the LAN except for the host itself ( and possibly other guests ) can communicate with the guest via the 1st network interface, which will have an IP address in your LAN. Your host can communicate with the guest via the 2nd network interface, which sits in the VPN.
---
It's not quite perfect, but it's an acceptable hack, and much less hassle than doing bridging 'properly'. It also doesn't freak out network-manager, which can have flow-on effects, with other network-manager-dependent services not starting up. I've actually had bridging working properly before with both VirtualBox ( before it supported doing it via it's own kernel module ... and also before everything depended on network-manager ) and KVM / QEMU ... but as noted above, it doesn't seem to work so well these days :/ Such is life. |
|
| Back to top |
|
|
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3432
|
| Posted: Fri Jun 13, 2014 3:43 pm Post subject: |
|
|
My way to make it work with command line. Just in case someone might find it useful.
| Code: | cat ./qemu_launcher.sh
#! /bin/bash
# HELP: This script is intended to be ran by user with limited priviliges.
# HELP: User must be allowed to run 'sudo tunctl', 'sudo brctl' and 'sudo ifconfig' without password in order to work.
# HELP: Intended invocation of this script is:
# HELP: "cmd=<params you want to append to qemu command line> <THIS FILE'S NAME> <virtual machines name> [name of the bridge to bind]"
# HELP: Multiple bridges are allowed. Bridges must already exist and be configured before this script is started (e.g. configured by init scripts).
# HELP: Feel free to edit contents of "static_params" near the end of this file to define set of options you want always active.
add_ifaces ()
{
i=1
USER=$(whoami)
while [ ${#1} -gt 0 ]
do sudo /usr/bin/tunctl -u $USER -t "${name}-${i}" > /dev/null
sudo /sbin/brctl addif $1 "${name}-${i}" > /dev/null
sudo /bin/ifconfig "${name}-${i}" up promisc > /dev/null
# HINT: VIRTIO NETWORK IS DISABLED BY DEFAULT, ADJUST THE FOLLOWING LINE IF YOU NEED IT
params="${params} -net nic,macaddr=$( printf 'DE:AD:BE:EF:%02X:%02X\n' $((RANDOM%256)) $((RANDOM%256)) ),vlan=$i -net tap,ifname=$name-$i,script=no,downscript=no,vlan=$i"
shift
i=$(($i+1))
done
echo $params
}
del_ifaces ()
{
i=1
while [ ${#1} -gt 0 ]
do sudo /bin/ifconfig "${name}-${i}" down > /dev/null
sudo /sbin/brctl delif $1 "${name}-${i}" > /dev/null
sudo /usr/bin/tunctl -d "${name}-${i}" > /dev/null
shift
i=$(($i+1))
done
}
main ()
{
name=$1
shift
bridges="$@"
networking="$(add_ifaces ${bridges})"
RUN="qemu-system-x86_64 -name ${name} ${static_params} ${networking} ${cmd}"
echo starting machine $name with command
echo $RUN
$RUN
del_ifaces ${bridges}
}
# HINT: PUT YOUR FAVOURITE OPTIONS HERE
static_params="-enable-kvm -m 1024"
main $@
|
|
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Networking & Security
