| View previous topic :: View next topic |
| Author |
Message |
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3432
|
 Posted: Wed Mar 08, 2023 2:03 pm Post subject: Testing L2 routing with BATMAN |
 |
|
Been looking for things that could help deal with small networks easily, and wanted to share one of the things I've ran across.
BATMAN. This name pops up in the kernel config, so most of people here probably know it exists, but I doubt anyone tried using it, even though it seems pretty cool. And I wanted to give it a shot.
BATMAN is a protocol for creating a self-organizing, self-healing mesh. It runs in kernel, and batman interfaces act as both, entry point to the overlay network and routers for their batman neighbours.
Don't have a lot of hardware lying around me right now, so went with namespaces, veth, iperf and top on a quad-core laptop.
Tests done:
iperf on veth pair (no batman)
cross-talk between batman and veth (the flock of 1 batman doesn't fly, must be a least a pair)
iperf on 2 node batman pair
iperf on 3 node batman loop
iperf on 3 node batman chain
varying number of hops in 16-node long loop:
1 hop
5 hops
8 hops
Also:
8 hops across across 16-node ladder (loop 1-2-3-4 ...15-16-1 + shortcuts 2-16;3-15;4-14 and so on. 1 and 9 are the test points and all shortcuts are in the wrong direction)
A simple script (which admittedly could have been written in a better way) used to setup the loop fast:
| Code: | # cat battest_setup
#! /bin/bash
for i in $( seq 1 $1 )
do ip netns add ns${i}
done
j=$1
ip link add dev eth${j}a type veth peer name eth${j}b
for i in $( seq 1 $(($1-1)) )
do ip link add dev eth${i}a type veth peer name eth${i}b
ip link set eth${i}a netns ns${i}
ip link set eth${j}b netns ns${i}
ip netns exec ns${i} ip link add name bat${i} type batadv
ip netns exec ns${i} ip link set bat${i} up
ip netns exec ns${i} ip a add 10.1.1.${i}/24 dev bat${i}
ip netns exec ns${i} ip link set dev eth${i}a master bat${i} up
ip netns exec ns${i} ip link set dev eth${j}b master bat${i} up
j=${i}
done
i=$1
ip link set eth${i}a netns ns${i}
ip link set eth${j}b netns ns${i}
ip netns exec ns${i} ip link add name bat${i} type batadv
ip netns exec ns${i} ip link set bat${i} up
ip netns exec ns${i} ip a add 10.1.1.${i}/24 dev bat${i}
ip netns exec ns${i} ip link set dev eth${i}a master bat${i} up
ip netns exec ns${i} ip link set dev eth${j}b master bat${i} up
|
Iperf test results:
| Code: |
### Direct connect via veth:
Connecting to host 10.1.1.1, port 5201
[ 5] local 10.1.1.2 port 40528 connected to 10.1.1.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 8.76 GBytes 75.2 Gbits/sec 0 523 KBytes
[ 5] 1.00-2.00 sec 8.82 GBytes 75.7 Gbits/sec 0 837 KBytes
[ 5] 2.00-3.00 sec 8.80 GBytes 75.6 Gbits/sec 0 882 KBytes
[ 5] 3.00-4.00 sec 8.81 GBytes 75.7 Gbits/sec 0 1.05 MBytes
[ 5] 4.00-5.00 sec 8.87 GBytes 76.2 Gbits/sec 0 1.10 MBytes
[ 5] 5.00-6.00 sec 8.53 GBytes 73.2 Gbits/sec 0 1.62 MBytes
[ 5] 6.00-7.00 sec 8.49 GBytes 72.9 Gbits/sec 0 1.70 MBytes
[ 5] 7.00-8.00 sec 8.47 GBytes 72.8 Gbits/sec 0 1.87 MBytes
[ 5] 8.00-9.00 sec 8.79 GBytes 75.5 Gbits/sec 0 1.87 MBytes
[ 5] 9.00-10.00 sec 8.80 GBytes 75.6 Gbits/sec 0 1.96 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 87.1 GBytes 74.8 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 2.00 GBytes 1.72 Gbits/sec receiver
### BATMAN 1 hop
Connecting to host 10.1.1.1, port 5201
[ 5] local 10.1.1.2 port 56408 connected to 10.1.1.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 520 MBytes 4.36 Gbits/sec 32 392 KBytes
[ 5] 1.00-2.00 sec 519 MBytes 4.35 Gbits/sec 409 501 KBytes
[ 5] 2.00-3.00 sec 525 MBytes 4.41 Gbits/sec 548 369 KBytes
[ 5] 3.00-4.00 sec 527 MBytes 4.42 Gbits/sec 288 445 KBytes
[ 5] 4.00-5.00 sec 529 MBytes 4.44 Gbits/sec 86 492 KBytes
[ 5] 5.00-6.00 sec 420 MBytes 3.53 Gbits/sec 115 380 KBytes
[ 5] 6.00-7.00 sec 523 MBytes 4.38 Gbits/sec 362 269 KBytes
[ 5] 7.00-8.00 sec 529 MBytes 4.44 Gbits/sec 301 402 KBytes
[ 5] 8.00-9.00 sec 534 MBytes 4.48 Gbits/sec 586 293 KBytes
[ 5] 9.00-10.00 sec 527 MBytes 4.41 Gbits/sec 216 329 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 5.03 GBytes 4.32 Gbits/sec 2943 sender
[ 5] 0.00-10.00 sec 2.00 GBytes 1.72 Gbits/sec receiver
### BATMAN 1 hop; loop of 3
Connecting to host 10.1.1.1, port 5201
[ 5] local 10.1.1.3 port 50604 connected to 10.1.1.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 432 MBytes 3.62 Gbits/sec 753 222 KBytes
[ 5] 1.00-2.00 sec 438 MBytes 3.67 Gbits/sec 847 197 KBytes
[ 5] 2.00-3.00 sec 437 MBytes 3.67 Gbits/sec 375 283 KBytes
[ 5] 3.00-4.00 sec 261 MBytes 2.19 Gbits/sec 586 297 KBytes
[ 5] 4.00-5.00 sec 350 MBytes 2.94 Gbits/sec 696 417 KBytes
[ 5] 5.00-6.00 sec 348 MBytes 2.92 Gbits/sec 446 313 KBytes
[ 5] 6.00-7.00 sec 436 MBytes 3.66 Gbits/sec 258 428 KBytes
[ 5] 7.00-8.00 sec 279 MBytes 2.34 Gbits/sec 335 346 KBytes
[ 5] 8.00-9.00 sec 338 MBytes 2.83 Gbits/sec 311 1.20 MBytes
[ 5] 9.00-10.00 sec 325 MBytes 2.73 Gbits/sec 778 1.16 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 3.56 GBytes 3.06 Gbits/sec 5385 sender
[ 5] 0.00-10.00 sec 2.00 GBytes 1.72 Gbits/sec receiver
### BATMAN 2 hops; chain of 3
Connecting to host 10.1.1.1, port 5201
[ 5] local 10.1.1.3 port 43520 connected to 10.1.1.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 428 MBytes 3.59 Gbits/sec 382 243 KBytes
[ 5] 1.00-2.00 sec 436 MBytes 3.65 Gbits/sec 465 322 KBytes
[ 5] 2.00-3.00 sec 346 MBytes 2.91 Gbits/sec 364 226 KBytes
[ 5] 3.00-4.00 sec 432 MBytes 3.63 Gbits/sec 321 300 KBytes
[ 5] 4.00-5.00 sec 440 MBytes 3.69 Gbits/sec 147 301 KBytes
[ 5] 5.00-6.00 sec 355 MBytes 2.97 Gbits/sec 22 253 KBytes
[ 5] 6.00-7.00 sec 348 MBytes 2.93 Gbits/sec 788 267 KBytes
[ 5] 7.00-8.00 sec 432 MBytes 3.62 Gbits/sec 600 286 KBytes
[ 5] 8.00-9.00 sec 445 MBytes 3.73 Gbits/sec 45 286 KBytes
[ 5] 9.00-10.00 sec 445 MBytes 3.73 Gbits/sec 185 464 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 4.01 GBytes 3.45 Gbits/sec 3319 sender
[ 5] 0.00-10.00 sec 2.00 GBytes 1.72 Gbits/sec receiver
### BATMAN 1/16 hops
Connecting to host 10.1.1.1, port 5201
[ 5] local 10.1.1.2 port 59162 connected to 10.1.1.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 520 MBytes 4.36 Gbits/sec 255 300 KBytes
[ 5] 1.00-2.00 sec 514 MBytes 4.31 Gbits/sec 41 402 KBytes
[ 5] 2.00-3.00 sec 523 MBytes 4.39 Gbits/sec 979 284 KBytes
[ 5] 3.00-4.00 sec 530 MBytes 4.44 Gbits/sec 460 433 KBytes
[ 5] 4.00-5.00 sec 539 MBytes 4.53 Gbits/sec 323 457 KBytes
[ 5] 5.00-6.00 sec 535 MBytes 4.48 Gbits/sec 134 457 KBytes
[ 5] 6.00-7.00 sec 532 MBytes 4.47 Gbits/sec 99 467 KBytes
[ 5] 7.00-8.00 sec 534 MBytes 4.47 Gbits/sec 34 522 KBytes
[ 5] 8.00-9.00 sec 315 MBytes 2.64 Gbits/sec 353 199 KBytes
[ 5] 9.00-10.00 sec 422 MBytes 3.55 Gbits/sec 6 393 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 4.85 GBytes 4.16 Gbits/sec 2684 sender
[ 5] 0.00-10.00 sec 2.00 GBytes 1.72 Gbits/sec receiver
### BATMAN 5/16 hops
Connecting to host 10.1.1.1, port 5201
[ 5] local 10.1.1.6 port 52260 connected to 10.1.1.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 258 MBytes 2.16 Gbits/sec 147 803 KBytes
[ 5] 1.00-2.00 sec 244 MBytes 2.04 Gbits/sec 573 246 KBytes
[ 5] 2.00-3.00 sec 165 MBytes 1.38 Gbits/sec 44 1.56 MBytes
[ 5] 3.00-4.00 sec 258 MBytes 2.16 Gbits/sec 40 1.93 MBytes
[ 5] 4.00-5.00 sec 206 MBytes 1.73 Gbits/sec 283 553 KBytes
[ 5] 5.00-6.00 sec 256 MBytes 2.15 Gbits/sec 1 1.72 MBytes
[ 5] 6.00-7.00 sec 202 MBytes 1.70 Gbits/sec 188 980 KBytes
[ 5] 7.00-8.00 sec 258 MBytes 2.16 Gbits/sec 0 1.03 MBytes
[ 5] 8.00-9.00 sec 256 MBytes 2.15 Gbits/sec 45 1.08 MBytes
[ 5] 9.00-10.00 sec 155 MBytes 1.30 Gbits/sec 247 700 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 2.20 GBytes 1.89 Gbits/sec 1568 sender
[ 5] 0.00-10.00 sec 2.00 GBytes 1.72 Gbits/sec receiver
### BATMAN 8/16 hops in the ring
Connecting to host 10.1.1.1, port 5201
[ 5] local 10.1.1.9 port 58916 connected to 10.1.1.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 180 MBytes 1.51 Gbits/sec 457 273 KBytes
[ 5] 1.00-2.00 sec 178 MBytes 1.50 Gbits/sec 208 491 KBytes
[ 5] 2.00-3.00 sec 179 MBytes 1.50 Gbits/sec 68 520 KBytes
[ 5] 3.00-4.00 sec 177 MBytes 1.49 Gbits/sec 6 468 KBytes
[ 5] 4.00-5.00 sec 142 MBytes 1.20 Gbits/sec 95 436 KBytes
[ 5] 5.00-6.00 sec 180 MBytes 1.51 Gbits/sec 50 578 KBytes
[ 5] 6.00-7.00 sec 143 MBytes 1.20 Gbits/sec 105 795 KBytes
[ 5] 7.00-8.00 sec 179 MBytes 1.50 Gbits/sec 232 574 KBytes
[ 5] 8.00-9.00 sec 180 MBytes 1.51 Gbits/sec 38 382 KBytes
[ 5] 9.00-10.00 sec 180 MBytes 1.51 Gbits/sec 2 438 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 1.68 GBytes 1.44 Gbits/sec 1261 sender
### BATMAN 8/16 hops on ladder shape
Connecting to host 10.1.1.1, port 5201
[ 5] local 10.1.1.9 port 34524 connected to 10.1.1.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 151 MBytes 1.26 Gbits/sec 220 414 KBytes
[ 5] 1.00-2.00 sec 147 MBytes 1.23 Gbits/sec 1342 383 KBytes
[ 5] 2.00-3.00 sec 155 MBytes 1.30 Gbits/sec 530 298 KBytes
[ 5] 3.00-4.00 sec 162 MBytes 1.36 Gbits/sec 266 402 KBytes
[ 5] 4.00-5.00 sec 161 MBytes 1.35 Gbits/sec 404 300 KBytes
[ 5] 5.00-6.00 sec 160 MBytes 1.34 Gbits/sec 3 395 KBytes
[ 5] 6.00-7.00 sec 158 MBytes 1.32 Gbits/sec 6 329 KBytes
[ 5] 7.00-8.00 sec 160 MBytes 1.35 Gbits/sec 4 355 KBytes
[ 5] 8.00-9.00 sec 118 MBytes 991 Mbits/sec 14 286 KBytes
[ 5] 9.00-10.00 sec 158 MBytes 1.32 Gbits/sec 3 270 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 1.49 GBytes 1.28 Gbits/sec 2792 sender
[ 5] 0.00-10.00 sec 1.49 GBytes 1.28 Gbits/sec receiver
|
The bigger the network, the slower it gets, not very surprising, however the connections not on the path also influence transfer speed (the ladder was slower than ring, 3 node loop was slower than a pair)
It seems that speed on the long chains degrades due to latency, since running iperf did not cause big load spikes. Also, while BATMAN is capable of running on wires, it's designed for highly dynamic networks, like ad-hoc wifi with mobile nodes. Wifi is WAY slower than veth, so the processing latency might not be noticeable in its intended use case scenario.
Could also be some attempt at bandwidth sharing.
Although I haven't found explicit information about applicability, I noticed e.g. multicast optimizations which appear to target networks with a few dozens of nodes.
The longest chain that works by default is 32 nodes, batctl does not seem to have any way to change it, throughput went down do 50MBps though.
Bonus: there are 2 shortest paths in my ring test and batman picks them more or less randomly:
| Code: | ### BATMAN changing active chain
~ # batctl meshif bat9 tr 10.1.1.1
traceroute to 10.1.1.1 (46:52:77:89:25:b4), 50 hops max, 20 byte packets
1: a2:48:a1:d3:33:99 0.028 ms 0.002 ms 0.001 ms
2: fa:a1:d3:6b:be:30 0.002 ms 0.002 ms 0.002 ms
3: e6:8c:45:d3:4c:5b 0.001 ms 0.001 ms 0.001 ms
4: ba:6c:d8:53:bb:8c 0.002 ms 0.001 ms 0.001 ms
5: ce:66:9c:e2:2b:88 0.001 ms 0.001 ms 0.001 ms
6: 56:d9:d4:9c:19:bc 0.001 ms 0.001 ms 0.001 ms
7: be:ae:36:15:a9:31 0.001 ms 0.001 ms 0.001 ms
8: 46:52:77:89:25:b4 0.002 ms 0.001 ms 0.001 ms
~ # batctl meshif bat9 tr 10.1.1.1
traceroute to 10.1.1.1 (46:52:77:89:25:b4), 50 hops max, 20 byte packets
1: 76:c6:91:b7:3e:be 0.006 ms 0.002 ms 0.002 ms
2: 92:8c:8e:9f:26:9a 0.002 ms 0.001 ms 0.001 ms
3: a6:85:a4:2a:bd:7e 0.002 ms 0.002 ms 0.001 ms
4: 6a:4b:05:1c:23:0e 0.002 ms 0.001 ms 0.001 ms
5: 82:8f:04:90:28:1a 0.002 ms 0.002 ms 0.001 ms
6: 7a:b1:3a:98:c7:a7 0.002 ms 0.001 ms 0.001 ms
7: 62:1d:2f:dc:5e:97 0.002 ms 0.002 ms 0.001 ms
8: 46:52:77:89:25:b4 0.002 ms 0.002 ms 0.001 ms
~ # batctl meshif bat9 tr 10.1.1.1
traceroute to 10.1.1.1 (46:52:77:89:25:b4), 50 hops max, 20 byte packets
1: a2:48:a1:d3:33:99 0.006 ms 0.002 ms 0.001 ms
2: fa:a1:d3:6b:be:30 0.002 ms 0.001 ms 0.016 ms
3: e6:8c:45:d3:4c:5b 0.002 ms 0.001 ms 0.001 ms
4: ba:6c:d8:53:bb:8c 0.001 ms 0.002 ms 0.001 ms
5: ce:66:9c:e2:2b:88 0.001 ms 0.001 ms 0.001 ms
6: 56:d9:d4:9c:19:bc 0.001 ms 0.001 ms 0.001 ms
7: be:ae:36:15:a9:31 0.022 ms 0.001 ms 0.001 ms
8: 46:52:77:89:25:b4 0.002 ms 0.001 ms 0.001 ms
|
Wrapping up, it exists, it is simple to use, if anyone runs it in real life I'm curious how it performs. |
|
| Back to top |
|
 |
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Documentation, Tips & Tricks
