I was listening to a recent Packet Pushers Priority Queue podcast called Meet ZeroTier – Open Source Networking and decided to give ZeroTier a go, to see if it really worked as described. TL;DR: Yes it really does work as described! If you’re unfamiliar with ZeroTier I highly recommend listening to this podcast to understand why, what and how. If you’re unsure on the why, what and how, this blog post probably isn’t for you (yet!).
One of the obvious issues with ZeroTier is that for devices to talk to each other, they need a ZeroTier client installed. For laptops, mobile devices and even some NAS devices this isn’t a problem because there are clients in existence already. But what about using ZeroTier to extend an entire Layer 2 network? This is what the ZeroTier Edge device will do but it is entirely possible to configure a Linux host as an edge gateway device to do exactly this.
The following notes are based on a Linux host running Debian 8 and some of the notes will be specific to Debian and other such derivatives (e.g. Ubuntu).
If you’ve not done so already start by installing the Linux ZeroTier client. ZeroTier have also made this stupidly easy to do (ensure you have curl and gpg installed):
curl -s 'https://pgp.mit.edu/pks/lookup?op=get&search=0x1657198823E52A61' | gpg --import && \ if z=$(curl -s 'https://install.zerotier.com/' | gpg); then echo "$z" | sudo bash; fi
Now install the Linux bridge utilities:
apt-get install bridge-utils
Edit the network configuration (/etc/network/interfaces ) to bridge the Ethernet adapter and ZeroTier adapter together, giving the bridge interface (br0) the IP configuration from the Ethernet adapter. For example:
auto eth0 iface eth0 inet manual auto br0 iface br0 inet static address 192.168.0.3 netmask 255.255.255.0 network 192.168.0.0 broadcast 192.168.0.255 gateway 192.168.0.254 dns-nameservers 127.0.0.1 dns-search home.lan bridge_ports eth0 zt0 bridge_fd 0 bridge_maxage 0 post-up echo 1 > /proc/sys/net/ipv4/ip_forward
Note: The post-up directive is only required if you have other Layer 3 routable networks on your LAN. If you don’t, you don’t need this.
One gotchya is that the network service will start before the ZeroTier service meaning that the zt0 interface will not be present when the bridge interface is formed. A quick and dirty way around this is to add an entry to root’s crontab (sudo crontab -e or edit you could edit /etc/crontab ) to restart networking after a reboot:
@reboot /etc/init.d/networking restart
Next move on to your ZeroTier network. Login to ZeroTier Central. If you’ve not done so already create a new network. Your ZeroTier network wants to use the same Layer 3 addressing as the network you’re bridging to. In my example this means the ZeroTier network will use 192.168.0.0/24 because my LAN is 192.168.0.0/24. Add this in to the Managed Routes section of your ZeroTier network and enable the Auto-Assign from Range option. Select Advanced and in the Auto-Assign Pool, set a range of IPs to assign to your ZeroTier clients that’s not used on your Layer 2 network, e.g. a range of IPs outside of your DHCP scope range so that no IP conflicts arise.
Back on the Linux host, use the zerotier-cli command to request to join the host to your ZeroTier network:
zerotier-cli join <network id>
Note: When the Linux host joins the ZeroTier network it will likely drop off your LAN until all the configuration is completed. Ensure you have console access.
Now edit the /var/lib/zerotier-one/networks.d/<network id>.local.conf file and set allowManaged to 0.
On ZeroTier Central the host should now appear as a Member of your network and will need to be authorised to join it. In addition you’ll need to edit the properties of this member and enable Allow Ethernet Bridging and Do Not Auto-Assign IPs. Delete any IPs associated with the member and set a manual IP. This should be the same IP as used on the br0 interface on the Linux host (in my example this would be 192.168.0.3).
Reboot the Linux host. After it has rebooted you should see br0 is up with the correct IP configured (ifconfig br0 ) and the bridge should contain your Ethernet adapter (eth0 in this example) and the ZeroTier adapter (zt0). Check using brctl show .
If everything is correct your Linux host will be now be acting as an Ethernet (Layer 2) bridge between your LAN and your ZeroTier network. Install tcpdump to see what’s happening (e.g. tcpdump -n -i br0 ‘not host 192.168.0.3’ ) and from another ZeroTier client on your ZeroTier network attempt to access a non-ZeroTier client on your LAN (like your toaster, because you know, everything is IoT now).
If you’ve got other Layer 3 (routable) networks on your LAN you can easily make this accessible from your ZeroTier network simply by adding the network to the Managed Routes section of your ZeroTier network via ZeroTier Central. This requires IP forwarding to be enabled on the Linux bridge so if you removed the post-up directive mentioned earlier, you’ll need to add this back in.
Thanks, I listened to the same podcast and also thought this is awesome. Great article and I now have it all working and have managed to ditch some of my port forwards into my network, which I always try to minimise. Thanks again
Hey. Just wanted to pop in and say thanks for posting this. There’s lots of info out there about bridging with 2 NICs, which give some useful info, but yours is the first I’ve found to use a single NIC, and to include some important details that others leave out, like the post-up statement and the allowManaged setting. So, Thanks!
In my case, I’m using Raspbian Stretch on a Raspberry Pi 3. Due to the differences with using DHCPD, I had to make some adjustments, but your post definitely pointed me in the right direction. I now have the RPI3 providing a bridge to one of my subnets, and routing traffic beyond to other more central subnets.
Thanks again!
Hi Corey – interested in trying this on a RPi3. What adjustments did you need to make?
Hi Corey — can you provide any details on what adjustments to this method were necessary to get everything working in Raspbian? You mention differences with using DHCPD, but I’m not sure what exactly I’m doing wrong with the current approach. Thanks!
https://0wned.it/2017/12/04/building-a-zerotier-bridged-network/
Hi, this is great. Looking for this to have my cam and nas remotely available but only have a raspberry available for this.
Can anyone explain in detail how to? Corey can you help me out ?
Many thanks!!
Just wanted to reach out and say that your instructions are the best!
Had trouble accessing any local network resources other than the bridge. Turns out I needed to add iptables rules to allow network traffic to flow through the bridge. For anyone else who has torn out their hair for at least two days:
iptables -A INPUT -i eth0 -j ACCEPT
iptables -A INPUT -i br0 -j ACCEPT
iptables -A FORWARD -i br0 -j ACCEPT
Hi having the same issue as you had, Debian 9 VM can ping the gateway but nothing else.
Tried your fix, iptables -A INPUT -i eth0 -j ACCEPT
iptables -A INPUT -i br0 -j ACCEPT
iptables -A FORWARD -i br0 -j ACCEPT
using sudo in cli was not sure what to do with them seemed to accept no errors however no joy pinging anything other than bridge.
hi, can i know wat are the modifications u did for rpi3
After
Trying to do thew first command, but receive (using Ubuntu 18.04) this
-bash: syntax error near unexpected token `then’
One extra comment when originally using DHCP with the Ethernet interface:
It is possible to continue using the same DHCP setup with the bridge by setting eth0 to “manual”, and setting br0 to “dhcp”, then use the “bridge_hw” interfaces entry to set the br0 MAC address to the eth0 MAC address – works like a charm.
interfaces:
auto br0
iface br0 inet dhcp
bridge_hw 11:22:33:44:55:66
bridge_ports eth0 zt1abcdefg
bridge_fd 0
bridge_maxage 0
Take out the \ character and the space before the “if”. See here for more details: https://stackoverflow.com/a/57525484/649497