Newer
Older
MTU for Mesh-VPN
================
What is a good MTU on the mesh-vpn?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Setting the MTU on the transport interface requires careful consideration, as
setting it too low will cause excessive fragmentation and setting it too high
may leave peers with a broken tunnel due to packet loss.
Consider these key values:
- Payload: Allow for the transport of IPv6 packets, by adhering to the minimum MTU
of 1280 Byte specified in RFC 2460
- and configure `MSS clamping`_ accordingly,
- and announce your link MTU via Router Advertisements and DHCP
.. _MSS clamping: https://tldp.org/HOWTO/Adv-Routing-HOWTO/lartc.cookbook.mtu-mss.html
- Encapsulation: Account for the overhead created by the configured mesh protocol
encapsulating the payload, which is up to 32 Byte (14 Byte Ethernet + 18 Byte
batadv).
- PMTU: What MTU does the path between your gateway and each of its peers support?
For reference, the complete MTU stack looks like this:
.. image:: mtu-diagram_v5.png
Example for Minimum MTU
-----------------------
Calculate the minimum transport MTU by adding the encapsulation overhead to the
minimum payload MTU required. This is the lowest recommended value, since going
lower would cause unnecessary fragmentation for clients which respect the announced
link MTU.
.. editorconfig-checker-disable
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
Example: Our network currently uses batman-adv v15, it therefore requires up
to 32 Bytes of encapsulation overhead on top of the minimal link MTU required for
transporting IPv6.::
\ 1312 1294 1280 0
\---------+-----------------+-------------+----------------------------------+
\TAP | batadv v15 | Ethernet | Payload |
\-------+-----------------+-------------+----------------------------------+
\ ^
|
MTU_LOW = 1280 Byte + 14 Byte + 18 Byte = 1312 Byte
Example for Maximum MTU
-----------------------
Calculating the maximum transport MTU is interesting, because it increases the
throughput, by allowing larger payloads to be transported, but also more difficult
as you have to take into account the tunneling overhead and each peers PMTU, which
varies between providers.
The underlying reasons are mostly PPPoE, Tunneling and IPv6 transition technologies
like DS-Lite.
Example: The peer with the smallest MTU on your network is behind DS-Lite and can
transport IPv4 packets up to 1436 Bytes in size. Your tunnel uses IPv4 (20 Byte),
UDP (8 Byte), Fastd (24 byte) and you require TAP (14 Byte) for Layer 2 (Ethernet)
Tunneling.::
1436 1416 1408 1384 1370 \
+-------------------+--------+-----------------------+-------------+------\
| IP | UDP | Fastd | TAP | bat\
+-------------------+--------+-----------------------+-------------+--------\
^ \
|
MTU_HIGH = 1436 Byte - 20 Byte - 8 Byte - 24 Byte - 14 Byte = 1370 Byte
.. editorconfig-checker-enable
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
Tables for Different VPN Providers
----------------------------------
VPN Protocol Overhead (IPv4)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Overhead of the VPN protocol layers in bytes on top of an Ethernet frame.
+----------+-------+--------------+-----------+
| | fastd | Tunneldigger | Wireguard |
+==========+=======+==============+===========+
| IPv4 | 20 | 20 | 20 |
+----------+-------+--------------+-----------+
| UDP | 8 | 8 | 8 |
+----------+-------+--------------+-----------+
| Protocol | 24 | 8 | 32 |
+----------+-------+--------------+-----------+
| TAP | 14 | 14 | / |
+----------+-------+--------------+-----------+
| Sum | 66 | 50 | 60 |
+----------+-------+--------------+-----------+
Intermediate Layer Overhead
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Overhead of additional layers on top of the VPN packet needed for different VPN
providers.
+------------+-------+--------------+-----------+
| | fastd | Tunneldigger | Wireguard |
+============+=======+==============+===========+
| IPv6 | / | / | 40 |
+------------+-------+--------------+-----------+
| vxlan | / | / | 16 |
+------------+-------+--------------+-----------+
| Ethernet | / | / | 14 |
+------------+-------+--------------+-----------+
| Batman v15 | 18 | 18 | 18 |
+------------+-------+--------------+-----------+
| Ethernet | 14 | 14 | 14 |
+------------+-------+--------------+-----------+
| Sum | 32 | 32 | 102 |
+------------+-------+--------------+-----------+
Minimum MTU
^^^^^^^^^^^
Calculation of different derived MTUs based on a 1280 byte payload to
avoid fragmentation.
Suggestions:
- This configuration is only suggested for fastd and Tunneldigger.
- For WireGuard, this configuration is **unsuitable**. To obtain a 1280 byte
payload with our protocol stack (see below), the Ethernet frame payload would
be 1442 bytes long (for IPv4). As we assume that the WAN network might have
a (worst case) MTU of only 1436 (with DSLite), this packet would be too long
for the WAN network.
+-------------------------------+-------+--------------+-----------+
| | fastd | Tunneldigger | Wireguard |
+===============================+=======+==============+===========+
| max unfragmented payload\* | 1280 | 1280 | 1280 |
+-------------------------------+-------+--------------+-----------+
| intermed layer overhead | 32 | 32 | 102 |
+-------------------------------+-------+--------------+-----------+
| VPN MTU\*\* | 1312 | 1312 | 1382 |
+-------------------------------+-------+--------------+-----------+
| protocol overhead (IPv4) | 66 | 50 | 60 |
+-------------------------------+-------+--------------+-----------+
| min acceptable WAN MTU (IPv4) | 1378 | 1362 | **1442** |
+-------------------------------+-------+--------------+-----------+
| min acceptable WAN MTU (IPv6) | 1398 | 1382 | 1462 |
+-------------------------------+-------+--------------+-----------+
\* Maximum size of payload going into the bat0 interface, that will not be
fragmented by batman.
\*\* This is the MTU that is set in the site.conf.
Maximum MTU
^^^^^^^^^^^
Calculation of different derived MTUs based on a maximum WAN MTU of 1436.
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
- This configuration can be used for fastd and Tunneldigger.
- For WireGuard, this is the recommended configuration. batman-adv will
fragment larger packets transparently to avoid packet loss.
+-------------------------------+-------+--------------+-----------+
| | fastd | Tunneldigger | Wireguard |
+===============================+=======+==============+===========+
| min acceptable WAN MTU (IPv4) | 1436 | 1436 | 1436 |
+-------------------------------+-------+--------------+-----------+
| protocol overhead (IPv4) | 66 | 50 | 60 |
+-------------------------------+-------+--------------+-----------+
| VPN MTU\*\* | 1370 | 1386 | 1376 |
+-------------------------------+-------+--------------+-----------+
| intermed layer overhead | 32 | 32 | 102 |
+-------------------------------+-------+--------------+-----------+
| max unfragmented payload\* | 1338 | 1354 | 1274 |
+-------------------------------+-------+--------------+-----------+
| min acceptable WAN MTU (IPv6) | 1398 | 1382 | 1462 |
+-------------------------------+-------+--------------+-----------+
\* Maximum size of payload going into the bat0 interface, that will not be
fragmented by batman.
\*\* This is the MTU that is set in the site.conf.
Suggested MSS Values
^^^^^^^^^^^^^^^^^^^^
It is highly advised to use MSS clamping for TCP on the gateways/supernodes in
order to avoid the fragmentation mechanism of batman whenever possible.
Especially on small embedded devices, fragmentation costs performance.
As batmans fragmentation is transparent to the TCP layer, clamping the MSS
automatically to the PMTU does not work. Instead, the MSS must be specified
explicitly. In iptables, this is done via :code:`-j TCPMSS --set-mss X`,
whereby :code:`X` is the desired MSS.
Since the MSS is specified in terms of payload of a TCP packet, the MSS is
different for IPv4 and IPv6. Here are some examples for different max
unfragmented payloads:
+---------------------------------+------+------+------+------+
| max unfragmented payload | 1274 | 1280 | 1338 | 1354 |
+=================================+======+======+======+======+
| suggested MSS (IPv4, -40 bytes) | 1234 | 1240 | 1298 | 1314 |
+---------------------------------+------+------+------+------+
| suggested MSS (IPv6, -60 bytes) | 1214 | 1220 | 1278 | 1294 |
+---------------------------------+------+------+------+------+
Conclusion
^^^^^^^^^^
Determining the maximum MTU can be a tedious process, especially since the PMTU
of peers could change at any time. The general recommendation for maximized
compatibility is therefore an MTU of 1312 bytes (for fastd and tunneldigger)
and 1376 bytes (for WireGuard).