ZeroTier is a “zero trust” solution, meaning it always uses end-to-end encryption. It works like DNS, with root servers (a.k.a ZeroTier “moons”) for establishing the initial connection. Afterwards, all clients communicate directly to each other, like a giant mesh network, as long as they have IPv6, a public IPv4 address, or can use NAT “hole punching”. Otherwise, traffic is relayed for that specific client. You can connect to several networks at once or set up site-to-site networking.
ZeroTier is super easy to set up and use with the standard service. It’ll take you just a couple of minutes to configure ZeroTier on the Mikrotik router and a couple more minutes to download and install the ZeroTier client on your laptop or phone. If you want, you can of course run your own ZeroTier root server and admin controller, but that’s way more hands-on and technical.
Larsa has completely skipped over the multiple ISP thing.
Zerotier uses UDP hole punching.
The system makes IP connections via the best route it can. Once the service knows where it’s clients are… It makes connections from IP to IP then encrypts traffic between.
If you set up bridging… Zerotier can do full on later 2 broadcasts over those links… However…
It completely ignored the issue of having multiple ISPs. Shakey performance from one link or the other. Load balancing. Or rollover.
ZeroTier supports all of that, like most other SD-WANs do. Performance-wise, it all depends on the platform. There’s not much any other SD-WAN solution can do about it, be it Bigleaf or others..
Right now… I have 3 ISPs connected to the service I am paying a S–t TON for.
2 ISPs are public. the 3rd is CGNAT.
I am provided ONE public IP. I entered that into my Tik as my WAN connection and did the SRC-NAT to match.
Now ALL my traffic goes back to a data center over the 3 connections using various tricks to strype or push. When it hits the data center… the service combines the connections together and sends traffic where it needs to go. The data comes back to the data center when it makes decisions on how to send the traffic back over the 3 feeds. Then the data is reassembled and sent into my router.
Now if I am doing some sort of live time connection, and the connection from the ISP to the data center gets a little shakey, traffic will be shifted to the next feed that had a better connection. As the service is checking the connectivity down ALL feeds several times per second. So as the shift happens… more traffic goes over one ISP than the other to the data center. Where it then goes on to the interwebz. With NO change in the requesting IP.
When a circuit or ISP is performing better… traffic shifts to the better connections. Without SEVERING connections.
I have watched as ALL out going traffic went UP one circuit. But was returning down another circuit.
How EXACTLY… can I do this with Zerotier and not pay hundreds of dollars a month?
Most SD-WAN solutions does offer support for different kind of aggregation types. ZeroTier has several Standard Policies listed below but also offer Custom Policies as well as Segmentation. This allows you to aggregate multiple links of different types into different “circuits” using various policies.
active-backup: Use only one primary link at a time and failover to another designated link.
broadcast: Duplicate traffic across all available links at all times.
balance-rr: Stripe packets across multiple links (not for use with TCP.)
balance-xor: Hash flows to specific links.
balance-aware: Auto-balance flows across links.
The big difference between for example Bigleaf and Zerotier is how link aggregation is administered. Bigleaf has a significantly simpler and more powerful user interface, whereas with Zerotier, one has to edit everything manually which can be challenging with complex configurations. Bigleaf also offers more granular built-in control of traffic shaping, QoS, etc.
I’d say SD-WAN solutions like Netmaker, ZeroTier, Tailscale and similar, pretty much cover everything you need for small businesses, let’s say up to 10-20 branch offices with people on the move or working from home. They’re very easy to install and get going with great bang for your buck, with solid and reliable services and minimal need for administration. They don’t offer all the bells and whistles for network administration which are seldom needed in smaller organizations.
When it comes to larger corporations and service providers there is a significantly greater need to granularly manage capacity allocation, link aggregation and segmentation with technologies like MPLS, QoS, etc, as well as security management, monitoring and troubleshooting. However, the underlying network principle remains the same. These services are often delivered as black box sw/hw ‘appliances’ with agreements for guaranteed uptime and service commitments which comes with a price tag. In combination with the fact that this usually requires dedicated IT staff, it’s something smaller organizations often cannot afford.
As they say, you get what you pay for but if you don’t know what you’re doing you often pay way too much.
EDIT
Sorry, but I don’t really know how to respond to your “So, it can’t”. Could you be a bit more specific about the details of the use case?
Okay, got it. Agregating 3 somewhat (intermittently) shaky wan links to a datacenter. Seems like load balansing using asymetict links tweaked with quality and capacity settings should do it. Check out Multipath Balance-Aware and beyond.
If you want to set up a testbed, it’s not as fancy to configure as Bigleaf so you have to manually edit the policies config file.
And that’s the reason you’re paying a couple hundred bucks. Someone else has built the solution, hosts stuff in a datacenter, and has bandwidth/power/development costs associated with doing so.
I’d view ZeroTier as the tool to build something akin to what BigLeaf does. If you have another router or server in a well-connected datacenter, then ZeroTier would help your office/branch router find and connect (and stay connected) to that DC-hosted router.
The question is whether building the solution and hosting your own router in such a location is any cheaper than paying someone else to do it.
ZeroTier is software that creates and manages dynamic wireguard tunnels between endpoints, and, on occasion, relays traffic between those endpoints if they can’t find each other directly.
If your devices have public IP’s that don’t change, making direct wireguard tunnels is ideal. ZeroTier helps with getting around NAT and some firewall issues, and makes adding new sites easier than manually creating a full mesh of VPN tunnels between sites.
@gotsprings, since the description of your use case is limited and you haven’t specified what type of business it’s used for, it’s basically impossible to assess the best solution. It seems more like you need load balancing with redundancy to a central server solution and for that you don’t really need SD-WAN as @Sirbryan indicated. A high-end SD-WAN solution is only really necessary if it involves very large volumes of WAN connections with a complex network topology which is where the solution works best and pays off financially.
If it’s a small business with a few offices, Netmaker, Zerotier, etc. work just fine. If we assume that you have a service on a server you’re already paying for there’s no additional server costs to install a simple SD-WAN endpoint like Zerotier. Are you the dedicated resource to manage and administer the solution?
Generally, you have to pick two of the following: good, fast and cheap for it to reflect reality. You’ve only stated that you find the current solution expensive which indicates that you want to save money. What about the other factors?
I like Zerotier for connecting and reaching back into systems.
But and this is a big one…
I have a lot of systems that rely on having lots of connections that when your IP changes $#!+ Goes sideways.
Like for instance… My bars and restaurants rely on video services now. If your IP address is different from one device to another… You get flagged and one feed or the other stops working. So you need to get all your devices to use one Public IP as it presents to the video service provider. Big Leaf aggregates the ISPs into one IP at the data center. So you can use both feeds at once and one or the other goes out… No IP change.
In my business environment VoIP and wifi calling are a big deal. If some calls are going over one link and some over the other… If one feed “gets Shakey”… Too bad. You have to wait for the link to fail completely. Then wait for the client device to connect back to the cellular gateway and re-establish it’s IPSec keys. Losing calls and so forth.
A small broadcast site where they live stream to the national provider… Yeah I need that connection to keep going live.
CCR2004 bought and provisioned, everything in the network is setup and ready for moving in. VPN tested and getting somewhere around 700mbps of throughput with a speed test. When disconnecting one ISP cable, it takes about 2s to get the connection to work again (but not always because sometimes obviously I’m not disconnecting the link I’m using). Thanks everyone for the input!
Great choice! IPsec should get you closer to 1 Gbps with a CCR2004 at both ends. With OSPF + BFD, you should be able to switch redundant routes within 5-10 milliseconds, depending on the settings. Btw, OSPF and BFD are very easy to set up. Additionally, you might consider ZeroTier as an easy way to provide office access for people on the go.
Thanks! Just a quick question, my manager seems to think that we overpaid for the configuration of the router. Would someone be so kind to estimate the work amount for what I’ve described (number of hours), testing and deployment, and the usual fee you would ask for that (if you are a professional working with networks)? This goes to all the members participating in the thread or forum. Bear in mind that the location was less then 10 minutes on foot from the person providing the services (a one-man-show company), and that the average salary in my country is around 700 euros per month, an average developer ears about 2000-2500 e/m.
I can’t give an estimation anyways as I am not a professional network engineer.
Regarding salary, as you already pointed out: a developer (what kind of actually?) earns 3 times as much as average joe. This sounds crazy and unfair to me. Either the average is too way too low or the salary for a developer is way too high. But this is a different discussion.
Regarding the country: the rate per hour varies highly by country. For my country I would not say that rate per hour is much different across, so it does not really matter much if it’s a local guy or from a distant big city. They all need to make a living.
