| View previous topic :: View next topic |
| Author |
Message |
duxsco
n00b
Joined: 05 Jun 2010
Posts: 45
|
 Posted: Fri Oct 04, 2013 3:27 pm Post subject: HOWTO setup dual wan |
 |
|
Hi,
I have setup a dual wan network on my Gentoo Linux server and would like to discuss this solution, which I have come up with, with the help of various websites. Unfortunately, I am not able to recall the various website that have come in handy as my research lies some time back. But, anyway thanks a lot all of you. In the end, someone, who is experienced with the gentoo wiki, can create an entry.
1. This is the exemplary network used (based on: http://lartc.org/howto/lartc.rpdb.multiple-links.html):
| Code: |
________
+--------------+ /
192.168.0.0/24 | router0 | |
+-------------+IP: +-------
| | 192.168.0.254| /
eth2 IP: _ +------+-------+ +--------------+ |
192.168.2.1 \ | eth0 | |
\ |IP: | |
\ | 192.168.0.1 | |
__ \ |Default GW IP:| |
___/ \_ \ | 192.168.0.254| |
_/ \__ \ | | /
/ \ eth2| | |
| Local network -----+ server | | Internet
\_ __/ | | |
\__ __/ | | |
\___/ | eth1 | \
192.168.2.0/24 |IP: | |
| 192.168.1.1 | |
|Default GW IP:| |
| 192.168.1.254| |
+------+-------+ +--------------+ |
| | router1 | \
+-------------+IP: +-------
192.168.0.1/24 | 192.168.1.254| |
+--------------+ \________
|
The server will provide dual wan access with round-robin load balancing and failover. router0 and router1 can be a DD-WRT, OpenWrt or FritzBox router. Other router platforms are currently not supported for this failover approach. But, you can just google around to get it running.
2. First, "emerge -av sys-apps/iproute2" for routing. Then, add the following code to "/etc/iproute2/rt_tables" in order to create two new routing tables. One for each wan network at eth0 and eth1:
| Code: |
100 first
101 second
|
3. These are basic iptables routes. They protect the local network from the internet, provide NAT and mark packets passing eth0 and eth1 in order to keep a network session running on the same wan port. In this example, I save it at "/root/fw.sh".
| Code: |
#!/bin/bash
#tools
IPTABLES="/sbin/iptables"
IP6TABLES="/sbin/ip6tables"
###interfaces
WAN0="eth0"; WAN0_IP="192.168.0.1"
WAN1="eth1"; WAN1_IP="192.168.1.1"
LAN="eth2"
###clearing iptables
$IPTABLES -F
$IPTABLES -t nat -F
$IPTABLES -t mangle -F
$IPTABLES -X
$IPTABLES -t nat -X
$IPTABLES -t mangle -X
$IPTABLES -P INPUT DROP
$IPTABLES -P OUTPUT ACCEPT
$IPTABLES -P FORWARD DROP
$IP6TABLES -F
$IP6TABLES -t mangle -F
$IP6TABLES -X
$IP6TABLES -t mangle -X
$IP6TABLES -P INPUT DROP
$IP6TABLES -P OUTPUT ACCEPT
$IP6TABLES -P FORWARD DROP
###nat
$IPTABLES -t nat -A POSTROUTING -o $WAN0 -j SNAT --to-source $WAN0_IP
$IPTABLES -t nat -A POSTROUTING -o $WAN1 -j SNAT --to-source $WAN1_IP
###mangle (used for load balancing)
$IPTABLES -t mangle -A POSTROUTING -o $WAN0 -m conntrack --ctstate NEW -j CONNMARK --set-mark 0x100
$IPTABLES -t mangle -A POSTROUTING -o $WAN1 -m conntrack --ctstate NEW -j CONNMARK --set-mark 0x101
$IPTABLES -t mangle -A PREROUTING -i $WAN0 -m conntrack --ctstate NEW -j CONNMARK --set-mark 0x100
$IPTABLES -t mangle -A PREROUTING -i $WAN1 -m conntrack --ctstate NEW -j CONNMARK --set-mark 0x101
$IPTABLES -t mangle -A PREROUTING -m conntrack --ctstate ESTABLISHED,RELATED -j CONNMARK --restore-mark
$IPTABLES -t mangle -A OUTPUT -m conntrack --ctstate ESTABLISHED,RELATED -j CONNMARK --restore-mark
###must have rules
$IPTABLES -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
$IPTABLES -A INPUT -p icmp -j ACCEPT
$IPTABLES -A INPUT -i lo -j ACCEPT
$IPTABLES -A FORWARD -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
$IP6TABLES -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
$IP6TABLES -A INPUT -p icmpv6 -j ACCEPT
$IP6TABLES -A INPUT -i lo -j ACCEPT
$IP6TABLES -A FORWARD -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
###internet access
$IPTABLES -A FORWARD -i $LAN -o $WAN0 ! -d 192.168.0.0/16 -j ACCEPT
$IPTABLES -A FORWARD -i $LAN -o $WAN1 ! -d 192.168.0.0/16 -j ACCEPT
###default reject
$IPTABLES -A INPUT -j REJECT
$IPTABLES -A FORWARD -j REJECT
$IP6TABLES -A INPUT -j REJECT
$IP6TABLES -A FORWARD -j REJECT
|
Now, do the following as root in order to apply the iptables rules:
| Code: |
# chmod 700 /root/fw.sh
# /root/fw.sh
# /etc/init.d/iptables save
# /etc/init.d/ip6tables save
# rc-update add iptables boot
# rc-update add ip6tables boot
|
4. With the mangle rules, iptables marks packets for iproute2. Now, we need to tell iproute what to do with packets that have been marked. We execute the following code as root:
| Code: |
# ip rule add fwmark 0x100 table 100
# ip rule add fwmark 0x101 table 101
|
And, place the code in "/etc/local.d/local.start" to get it executed at boot time, which should look like:
| Code: |
#!/bin/bash
ip rule add fwmark 0x100 table 100
ip rule add fwmark 0x101 table 101
|
Above code tells iproute to lookup table 100 when getting packets marked with 0x100 and table 101, if getting 0x101 marked packets.
5. We need to fill table 100 and 101 with routing rules. For this purpose, I created the file "/usr/local/bin/dualwan_routes.sh" with this code:
| Code: |
#!/bin/bash
ip r flush table 0x100
ip r flush table 0x101
ip r | grep -v default | grep -v nexthop | while read I; do ip r add table 0x100 $I; ip r add table 0x101 $I; done
ip r add default via 192.168.0.254 dev eth0 table 0x100
ip r add default via 192.168.1.254 dev eth1 table 0x101
|
I made it executable "chmod 700 /usr/local/bin/dualwan_routes.sh", ran "emerge -av sys-process/daemontools" to get setlock and added this line to cron:
| Code: |
*/1 * * * * /usr/bin/setlock -n /tmp/dualwan_routes.lock /usr/local/bin/dualwan_routes.sh > /dev/null 2>&1
|
6. The last thing to do is setting up failover. The approach depends on the router you have in use. I explain the steps I took in order to get it running on DD-WRT, FritzBox and OpenWrt.
For DD-WRT and OpenWrt, you have to create an ssh keypair without password as normal user on the Gentoo server by executing:
6.a) DD-WRT:
- Go to "Administration" -> "Management" -> "Remote Access" and enable "SSH Management"
- Go to "Services" -> "Services" -> "Secure Shell", enable "SSHd" and add the following code to "Authorized Keys" (ssh-... is the ssh public key, e.g. id_rsa.pub, created in step 6.):
| Code: |
command="nvram get wanup",no-port-forwarding,no-X11-forwarding,no-pty ssh-...
|
Save and apply.
6.b) OpenWrt:
Go to "System" -> "Administration" -> "SSH Access" -> "SSH-Keys" and save this in the field (ssh-... is the ssh public key, e.g. id_rsa.pub, created in step 6.):
| Code: |
command="ifstatus wan",no-port-forwarding,no-X11-forwarding,no-pty ssh-...
|
Save and apply.
6.c) FritzBox:
Unfortunately, I only have the German version. Perhaps, someone can get the English names.
Go to "Heimnetz" -> "Netzwerk" -> "Programme" and enable "Statusinformationen über UPnP übertragen (empfohlen)".
English names might be:
"Home network" -> "network" -> "programs" and enable "status information over UPnP"
Save and apply.
7. You need to create the failover script "/usr/local/bin/dualwan_failover.sh" and adjust it to your needs (IP adresses, interface names and non-root username on the Gentoo server). I assume router0 being OpenWrt and router1 FritzBox:
| Code: |
#!/bin/bash
#DD-WRT
#wan0=1
#if [ $(sudo -u david ssh root@192.168.0.254 2>/dev/null) -eq 1 ]; then
# wan0=0
#fi
#OpenWrt
wan0=1
if [ $(sudo -u david ssh root@192.168.0.254 2>/dev/null | grep -q '"up": true,'; echo $?) -eq 0 ]; then
wan0=0
fi
#FritzBox
sudo -u david curl -s "http://192.168.1.254:49000/upnp/control/WANIPConn1" \
-H "Content-Type: text/xml; charset="utf-8"" \
-H "SoapAction:urn:schemas-upnp-org:service:WANIPConnection:1#GetStatusInfo" \
-d "<?xml version='1.0' encoding='utf-8'?> \
<s:Envelope s:encodingStyle='http://schemas.xmlsoap.org/soap/encoding/' xmlns:s='http://schemas.xmlsoap.org/soap/envelope/'> \
<s:Body> <u:GetStatusInfo xmlns:u='urn:schemas-upnp-org:service:WANIPConnection:1' /> \
</s:Body> </s:Envelope>" |\
sudo -u david grep -q "<NewConnectionStatus>Connected</NewConnectionStatus>"
wan1=$?
ip r | sudo -u david grep -q ^default[[:space:]]\*$
both=$?
ip r | sudo -u david grep -q "default via 192.168.0.254 dev eth0"
first=$?
ip r | sudo -u david grep -q "default via 192.168.1.254 dev eth1"
second=$?
if [[ $wan0 -eq 0 ]] && [[ $wan1 -eq 0 ]] && [[ $both -ne 0 ]]; then
ip r del default
ip r add default nexthop via 192.168.0.254 dev eth0 weight 1 nexthop via 192.168.1.254 dev eth1 weight 1
elif [[ $wan0 -eq 0 ]] && [[ $wan1 -ne 0 ]] && [[ $first -ne 0 ]]; then
ip r del default
ip r add default via 192.168.0.254 dev eth0
elif [[ $wan0 -ne 0 ]] && [[ $wan1 -eq 0 ]] && [[ $second -ne 0 ]]; then
ip r del default
ip r add default via 192.168.1.254 dev eth1
fi
|
Make it executable as root:
| Code: |
# chmod 700 /usr/local/bin/dualwan_failover.sh
|
And, add this to cron:
| Code: |
*/1 * * * * /usr/bin/setlock -n /tmp/dualwan_failover.lock /usr/local/bin/dualwan_failover.sh > /dev/null 2>&1
|
Adjust the weight to your needs. In above code, half of the traffic is distributed to each wan port (both "weight 1").
Last edited by duxsco on Fri Oct 04, 2013 3:56 pm; edited 1 time in total |
|
| Back to top |
|
 |
duxsco
n00b
Joined: 05 Jun 2010
Posts: 45
|
| Posted: Fri Oct 04, 2013 3:40 pm Post subject: |
|
|
| I don't know why, but somehow the Gentoo forum kicks me out regularly. I don't have this problem on other websites yet. |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54588
Location: 56N 3W
|
| Posted: Fri Oct 04, 2013 8:04 pm Post subject: |
|
|
Moved from Networking & Security to Documentation, Tips & Tricks.
by request of OP
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
SlashBeast
Retired Dev
Joined: 23 May 2006
Posts: 2922
|
|
| Back to top |
|
|
duxsco
n00b
Joined: 05 Jun 2010
Posts: 45
|
| Posted: Sat Oct 05, 2013 4:49 pm Post subject: |
|
|
From larc.org:
| Code: |
________
+------------+ /
| | |
+-------------+ Provider 1 +-------
__ | | | /
___/ \_ +------+-------+ +------------+ |
_/ \__ | if1 | /
/ \ | | |
| Local network -----+ Linux router | | Internet
\_ __/ | | |
\__ __/ | if2 | \
\___/ +------+-------+ +------------+ |
| | | \
+-------------+ Provider 2 +-------
| | |
+------------+ \________
|
| Quote: |
Let us first set some symbolical names. Let $IF1 be the name of the first interface (if1 in the picture above) and $IF2 the name of the second interface. Then let $IP1 be the IP address associated with $IF1 and $IP2 the IP address associated with $IF2. Next, let $P1 be the IP address of the gateway at Provider 1, and $P2 the IP address of the gateway at provider 2. Finally, let $P1_NET be the IP network $P1 is in, and $P2_NET the IP network $P2 is in.
|
The solution, provided at larc.org, won't work. The Linux router only knows of the networks $P1_NET and $P2_NET. It is not able to assign incoming packets with source IPs, that are not part of the subnet $P1_NET and $P2_NET, to if1 and if2. Let's say you are currently not connected to the local network, but via UMTS, and try to ssh into your linux router. The laptop you use got the ip 123.123.123.123 assigned. $P1_NET is 123.123.150.0/24 and $P2_NET is 123.123.151.0/24. Your linux router gets a packet with source IP 123.123.123.123 and tries to answer, but doesn't know which WAN NIC to use, because the routing tables contain only the $P1_NET and $P2_NET subnets.
You can use larg.org solution if you do the following:
You need to place a router between linux router and provider1. And, one between linux router and provider2.
Then, you have the following setttings:
| Code: |
________
+--------------+ /
192.168.0.0/24 | router0 | |
+-------------+IP: +-------
| | 192.168.0.254| /
eth2 IP: _ +------+-------+ +--------------+ |
192.168.2.1 \ | eth0 | |
\ |IP: | |
\ | 192.168.0.1 | |
__ \ |Default GW IP:| |
___/ \_ \ | 192.168.0.254| |
_/ \__ \ | | /
/ \ eth2| | |
| Local network -----+ server | | Internet
\_ __/ | | |
\__ __/ | | |
\___/ | eth1 | \
192.168.2.0/24 |IP: | |
| 192.168.1.1 | |
|Default GW IP:| |
| 192.168.1.254| |
+------+-------+ +--------------+ |
| | router1 | \
+-------------+IP: +-------
192.168.0.1/24 | 192.168.1.254| |
+--------------+ \________
|
On router0 you need to execute:
| Code: |
iptables -t nat -A POSTROUTING -d 192.168.0.1 -j SNAT --to-source 192.168.0.254
|
And on router1:
| Code: |
iptables -t nat -A POSTROUTING -d 192.168.1.1 -j SNAT --to-source 192.168.1.254
|
I don't like this solution, because all packets from the internet get the source address 192.168.0.254 or 192.168.1.254 assigned. |
|
| Back to top |
|
|
duxsco
n00b
Joined: 05 Jun 2010
Posts: 45
|
| Posted: Sat Oct 05, 2013 4:56 pm Post subject: |
|
|
Or, lets say you are in the local net and try to connect to google (173.194.112.136). Your packet gets forwarded correctly because of lartc.org code:
| Code: |
ip route add default scope global nexthop via $P1 dev $IF1 weight 1 \
nexthop via $P2 dev $IF2 weight 1
|
We assume the packet got out on the first wan NIC
Then, google responds and sends a packet over the first wan NIC. This packet has the source IP 173.194.112.136. Your laptop now tries to respond to this packet, but doesn't know which NIC to use, because only 123.123.150.0/24 and 123.123.151.0/24 are in the routing tables. |
|
| Back to top |
|
|
duxsco
n00b
Joined: 05 Jun 2010
Posts: 45
|
| Posted: Sat Oct 05, 2013 5:01 pm Post subject: |
|
|
Another solution to this problem is splitting all non-private subnets up between the two wan NICs.
Very simple (and wrong) example:
-first wan gets IP ranges 0.0.0.0-127.255.255.255
-second wan gets IP ranges 128.0.0.0-255.255.255.255
But, I don't like this solution, because it's not round-robin. |
|
| Back to top |
|
|
SlashBeast
Retired Dev
Joined: 23 May 2006
Posts: 2922
|
| Posted: Sat Oct 05, 2013 5:02 pm Post subject: |
|
|
I use lartc.org soluton with boxes where I have multiple wans and everything works great, either I have by far other usecase or we have great misunderstanding.
At this moment whatever goes from eth0 reply goes out thru eth0, same goes for eth1, eth2 and eth3. |
|
| Back to top |
|
|
duxsco
n00b
Joined: 05 Jun 2010
Posts: 45
|
| Posted: Sat Oct 05, 2013 5:08 pm Post subject: |
|
|
What do you use for $P1_NET and $P2_NET?
Larc.org says:
| Code: |
Finally, let $P1_NET be the IP network $P1 is in, and $P2_NET the IP network $P2 is in.
|
In both cases, the network is the internet. So $P1_NET == $P2_NET. |
|
| Back to top |
|
|
SlashBeast
Retired Dev
Joined: 23 May 2006
Posts: 2922
|
| Posted: Sat Oct 05, 2013 5:42 pm Post subject: |
|
|
The networks of ip addresses I use, so if I have 1.0.0.10 and 2.0.0.20 with 24bit subnet my networks are 1.0.0.0/24 and 2.0.0.0/24. And in my case both ip addresses are the public one, but still I do have 'upstream' which is ISP, I still have ip network and gateway I go thru on each link.
to make it crystal clear, your isp gave you ip address, which is in specified network, this is this network.
edit: something with real world examples https://kindlund.wordpress.com/2007/11/19/configuring-multiple-default-routes-in-linux/ |
|
| Back to top |
|
|
duxsco
n00b
Joined: 05 Jun 2010
Posts: 45
|
| Posted: Sat Oct 05, 2013 6:38 pm Post subject: |
|
|
If I am not mistaken, your routing tables are like the following:
- table T1:
| Code: |
default via 1.0.0.254 dev eth0 proto static
1.0.0.0/24 dev eth0 proto kernel scope link src 1.0.0.10
|
- table T1:
| Code: |
default via 2.0.0.254 dev eth1 proto static
2.0.0.0/24 dev eth1 proto kernel scope link src 2.0.0.10
|
- table main:
| Code: |
1.0.0.0/24 dev eth0 proto kernel scope link src 1.0.0.10
2.0.0.0/24 dev eth1 proto kernel scope link src 2.0.0.10
default
nexthop via 10.0.0.254 dev eth0 weight 1
nexthop via 20.0.0.254 dev eth1 weight 1
|
- ip rules:
| Code: |
0: from all lookup local
32762: from 1.0.0.10 lookup T1
32763: from 2.0.0.20 lookup T2
32766: from all lookup main
32767: from all lookup default
|
I think it's working for you because your Gentoo server is directly connected to the internet (running pppoe). Unfortunately, I cannot connect mine directly, because I use IP-TV and Dell's iDrac:
| Code: |
________
+--------+ +--------------+ /
192.168.0.0/24 | | | router0 | |
+-----------------+ switch +-------+IP: +-------
| | | | 192.168.0.254| /
eth2 IP: _ +------+-------+ +---+----+ +--------------+ |
192.168.2.1 \ | eth0 | | |
\ |IP: | | |
\ | 192.168.0.1 | | |
__ \ |Default GW IP:| | |
___/ \_ \ | 192.168.0.254| | |
_/ \__ \ | | +--+------+ /
/ \ eth2| | iDrac | | |
| Local network -----+ server +----------+ router2 + | Internet
\_ __/ | | | | |
\__ __/ | | +--+------+ |
\___/ | eth1 | | \
192.168.2.0/24 |IP: | | |
| 192.168.1.1 | | |
|Default GW IP:| | |
| 192.168.1.254| | |
+------+-------+ +---+----+ +--------------+ |
| | | | router1 | \
+-----------------+ switch +-------+IP: +-------
192.168.0.1/24 | | | 192.168.1.254| |
+---+----+ +--------------+ \________
|
|
+---+----+
| |
| IP-TV |
| |
+--------+
|
router2 has openvpn running. If my server crashes, I can still connect to it over iDrac over both wan connections. I could place the TVs in the local network, if I setup an IGMP proxy on my Gentoo server. But, I need router1 anyway in order to provide router2 with an internet connection. Therefore, I avoid the hassle and just connect the TVs to router1. |
|
| Back to top |
|
|
666threesixes666
Veteran
Joined: 31 May 2011
Posts: 1248
Location: 42.68n 85.41w
|
| Posted: Sat Oct 05, 2013 7:07 pm Post subject: |
|
|
you guys should wiki this brew with no iptv, routers, or switches, server acting as the router.
eth1 192.168.0.1/24 should be 192.168.1.0/24 representing the network. if it was wiki, id fix that typo that has been plaguing this thread.
| Code: |
________
/
192.168.0.0/24 |
+----------------------
| /
eth2 IP: _ +------+-------+ |
192.168.2.1 \ | eth0 | |
\ |IP: | |
\ | 192.168.0.1 | |
__ \ |Default GW IP:| |
___/ \_ \ | 192.168.0.254| |
_/ \__ \ | | /
/ \ eth2| | |
| Local network -----+ server | | Internet
\_ __/ | | |
\__ __/ | | |
\___/ | eth1 | \
192.168.2.0/24 |IP: | |
| 192.168.1.1 | |
|Default GW IP:| |
| 192.168.1.254| |
+------+-------+ |
| \
+----------------------
192.168.1.0/24 |
\________
|
only switching 192.168.0.0, and 192.168.1.0 to random dhcp addresses from isp, and their default gw.... |
|
| Back to top |
|
|
duxsco
n00b
Joined: 05 Jun 2010
Posts: 45
|
| Posted: Sat Oct 05, 2013 8:14 pm Post subject: |
|
|
| There might be use cases where a direct connection of the server to the internet is not possible, e.g. IP-based telephony, IP-TV, restrictions of TV-cable based connections (KabelDeutschland), VDSL. The advantage of my solution is that it is applicable under more use cases. As long as you have OpenWRT, DD-WRT or FritzBox between the linux server and the internet, failover works. You don't need to think of dynamically changing wan IPs. |
|
| Back to top |
|
|
duxsco
n00b
Joined: 05 Jun 2010
Posts: 45
|
| Posted: Sun Oct 06, 2013 1:13 am Post subject: |
|
|
| I think you should use the larc.org tutorial and ping based fail-over in order to get a dual wan setup where the server is directly connected to the internet. |
|
| Back to top |
|
|
Nreal
Apprentice
Joined: 06 Jan 2009
Posts: 289
|
| Posted: Sat Jun 16, 2018 12:26 pm Post subject: |
|
|
| How could you bond two connections - wifi and wired to one? Wired 10 mbs from router enp4s0 and wifi from adroid phone 300mbs/s. |
|
| Back to top |
|
|
duxsco
n00b
Joined: 05 Jun 2010
Posts: 45
|
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54588
Location: 56N 3W
|
| Posted: Sat Jun 16, 2018 9:13 pm Post subject: |
|
|
Nreal'
Both ends of the bond have to go to the same place so that they can be unbonded.
If you got your phone and landline from the same ISP, in theory it might be possible but your ISP would need to support it.
You can do failover but that's not a bond.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
Nreal
Apprentice
Joined: 06 Jan 2009
Posts: 289
|
| Posted: Sat Jun 16, 2018 9:20 pm Post subject: |
|
|
| Ok thank You for the answers. It sure would be nice, to be able to combine all 4gs in a house + landline.. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Documentation, Tips & Tricks
