I had an adapter fail on me during a rather large installation last week, so I scrambled to go there and replace it… I have a big box of them, just in case. (I think 20-30 adapters. )
Then I started reading about PoE input and realized that some of the devices are built in such a way that if you provide PoE in AND an adapter, then the two will act as a redundant power supply. Even if a device reboots while switching, this is fine with me; the only question is how to implement it correctly. (@Anav? )
I was thinking of setting the PoE in port as a power-in dummy + management, just connected to the bridge. I have a number of 309 and 326 models that I can set this way. It would provide power + access to the router if something breaks (so I don’t have to use the serial port in).
Is this a good approach? I would also set dummy ports on the PoE switch just for this purpose.
Any input is appreciated. I can make this work by myself, but I would really like to skip doing something dumb.
(I could even just use PoE injectors and two power supplies. I really do have a lot of power supplies and injectors left from projects. LOL)
I am doing exactly what you are wanting to do on a RB4011, RB750Gr3, and RB750r2 routers, and two CSS326 switches in my data cabinet. It works very well. All of these devices use passive (or dumb) POE input on port 1. In the case of the RB4011 it is getting 48V POE and the others are getting 24V POE. In both cases I am using an external POE injector that has port(s) for LAN and for POE and a power inlet. The LAN port goes to the whatever port 1 of the switch or router is currently connected to and the POe port connects to port 1 of the switch or router. In a couple cases, port 1 of the switch or router is unused except for the POE and in some cases, port 1 is being actively used.
So you chose an option to have external dedicated poe backup.
I am thinking about this too, in a way that I would have a separate injector and an adapter for each device if I wont find a way to configure excess PoE switch ports to be redundancies.
Have you maybe checked what happens when you plug/unplug ports? Do devices reset or gracefully transition power supply?
I am thinking about this too, in a way that I would have a separate injector and an adapter for each device if I wont find a way to configure excess PoE switch ports to be redundancies.
I don’t have POE switches, so that is not an option. Therefore the external POE injectors.
In my case (today), each of the routers and switches is using the included AC powered wall wort as one power source. Those are all powered from a UPS. The two external POE injectors are also powered from the same UPS. This protects from a failure of a wall wort, or power inlet in each device. Somewhere later this year, that is going to change. I will be doing a fairly major change to power distribution in my data cabinet. I will end up with a fairly large 48 volt battery plant that will feed the 48V POE injector for the RB4011. For 24 volts DC, there will be two sources of 24 volts (referred to as 24-A and 24-B). 24-A will be an AC operated power supply that will be powered from a 48V DC to 120 Volt AC inverter that in turn is powered from the 48 volt battery plant. The other (24-B) will be a separate AC operated power supply that will be powered from a separate 48V DC to 120 Volt AC inverter that in turn is powered from the 48 volt battery plant. 24-A will power the DC inlet to the two CSS326 switches and the two RB750 routers. The four port 24 volt POE injector will be powered from the 24-B volt DC source. There will be a third 24 volt DC source (referred to as 24-C) that is a sum (diode ORed for you electronics techs) of 24-A and 24-B. 24-C will power the DC inlet to the RB4011 router, and some air circulating fans. Note that the two AC inverters will also be powering a bunch of other AC load (computers for example), and there will also be three 12 volt DC distributions arranged similar to the three 24 volt DC distributions.
Have you maybe checked what happens when you plug/unplug ports? Do devices reset or gracefully transition power supply?
Yes. Completely seamless. As long as either power source is present to the device, it will continue to run. Confirmed on all five devices.
#2 Jesus. … Don’t you think that s a bit of an overkill? That system is really complex, have multiple fail points and is expensive to maintain. If I my suggest maybe a more sensible approach, given that you like to thinker - get an Smart UPS 750, or 1000 (they don’t have a fan), or if you are money constrained APC Back-UPS - someting. You can make external ports and connect really big batteries to the system in parallel. You can feed 12V devices directly (MikroTiks take 12V in) and 220V/110V through a converter from UPS directly. If If you are a big spender, you can get APC XL units + external battery packs, but those cost a lot. There are a plenty of videos on this online.
#3 Appreciated. I will also test it, including diff voltages. Adapter is 24V and PoE Switch is 48 if I am not mistaken.
Only a little. Think of it as public safety grade. I run a large regional public safety 2-way radio system for a living. We operate with the understanding that under the wrong set of circumstances, failure of the radio system can result in people dying. Therefore, we take system reliability VERY seriously. That mentality rubs off at home. Pretty much everything that is important to operation is redundant. And yes, there is monitoring and alarming if something fails…
As a useful note, although the APC XL units are designed for extended (many hour operation), the non-XL UPSes simply will not survive a 12 hour run under anywhere near full load without overheating (without substantial external cooling). Right now, I can (and have) run on battery for eight hours, and when I am completed with the power rebuild, that will be more like 24 hours on battery. Add to that if the sun is shining (there is some solar). I also can hook up a portable generator to the data cabinet if needed.
Thanks for info. Out of curiosity, how do you solve monitoring and reporting of MIkroTik systems and external sensors/batteries and such?
Also re UPS, fullcapacity is a no-no in my opinion, one should always leave a large power margin when UPSs are concerned. The hack I mentioned regarding Back-UPS is because it is used extensively in homes by people to run water heating pumps where I am in vacation houses and such. It is a low load and those can work for a very long time. I even found there are specific UPSes that use car batteries and such (12V). So, if you cool it well, and use, say, 30% of power I presume you could be quite OK for MikroTik load.
Why do you need such redundancy? Are you located on a remote area?
EDIT: ¿Maybe I didn’t understand - you said you are running public safety radio.. We are talking about securing MT equipment on the site also?
There are a couple things that I am doing. For environmental monitoring, I have a box called a Watchdog 1000 from IT Watchdogs (now owned by Vertiv). It has multiple temperature/airflow sensors in a few key places in the cabinet. It also has voltage monitors and contact closures for door closures and power failures. It can be set up for Email or text notifications or SNMP alerts. To check on network connectivity, my RB4011 has scheduled scripts to ping a bunch of various devices and will notify me if things fail. For devices that are accessible from the Internet, I use a monitoring service from UptimeRobot.com to monitor a bunch of different network points.
Why do you need such redundancy? Are you located on a remote area?
Need and want are not necessarily the same thing. No, I’m in the suburbs of the Los Angeles, Calif metro area.
I have three personal websites that run off my server, and there is some ham radio linking that goes through my house. I really want to keep stuff working. Also why I have both fiber and cable based Internet services - although I’m thinking about killing the cable.
EDIT: Maybe I didn’t understand - you said you are running public safety radio.. We are talking about securing MT equipment on the site also?
That’s for work, but the mentality rubs off at home. And yes, we use some of the same equipment and monitoring services at work.
These monitoring tips are great. For me the most important one would be the temperature - sometimes AC units fail in server rooms and then things can get toasty.
As for the other stuff, maybe keep coax as a backup? Out of curiosity, how much to you pay for Internet access? Coax here is about 25 USD for 200/20 and I think 500/50 is something like 30 USD. Fiber is about 30 USD for 1000/500. 5G wireless uplink is about 35-40 USD a month with 1 TB included.
StarLink is available, and TBH I would really like to put that one up. Don’t need it, but would like to… I was thinking of putting that into some companies…
These days, there are lots of home brew solutions for monitoring. Many off the shelf products as well. Just depends on your needs.
As for the other stuff, maybe keep coax as a backup? Out of curiosity, how much to you pay for Internet access? Coax here is about 25 USD for 200/20 and I think 500/50 is something like 30 USD. Fiber is about 30 USD for 1000/500. 5G wireless uplink is about 35-40 USD a month with 1 TB included.
StarLink is available, and TBH I would really like to put that one up. Don’t need it, but would like to… I was thinking of putting that into some companies…
For the time being, I am keeping the cable Internet as a backup. However I am retiring next year, so I may have to look harder if it’s worth the extra cashflow.
My 1000/1000 fiber is 75 USD per month. That has been available in my neighborhood for just under two years and I ordered the service as soon as we got notice that the fiber service was coming. Been very happy with it. The cable Internet (now 300/10 - it’s gone up a few times since I got it) is a bundle with phone and TV so I’m not really sure what the cable Internet by itself would be.
I have looked at Starlink as an “Oh my God” backup since it is totally not dependent on any local infrastructure. I have a couple friends who have it because it’s the only thing available where they live.
As a little background and history. Besides working in Public Safety radio, I have also been a Communications Reserve Officer with the California state equivalent of the Federal Emergency Management Agency (FEMA) for around 35 years. That also gives more of an interest in keeping things working. For history, I have lived in my house for 32 years. When we bought the house I got a second phone line for use on a dial up BBS that I ran on an Amiga computer. When this new Internet thing pretty much killed dial up BBSes almost 30 years ago, that line was used for dial up Internet. Somewhere around 2000, I was able to get DSL over that same phone line and I had the DSL until two years ago. About eight or so years ago I added the cable TV and Internet and that became the primary with the DSL as a backup. The cable started off as officially 100/10, but I normally got a bit better than that. Shortly thereafter it bumped up to 200/10 and a few years later it went up to 300/10. When I got the fiber just under two years ago, I dropped the DSL and the last copper phone line. The fiber is primary and the cable is the backup for the most part. The fiber is more reliable than the cable.
I sometimes use 24V battery packs on either PoE injector or as the DC IN, with the battery charge always plugged in to the grid. No AC/DC conversion and Mikrotik’s don’t really use all that much power, so $50-100 battery pack is often longer lasting/cheaper than APC-like AC UPS. The 24V lithium packs are pretty common for CPAP medical machines, so some batteries even have certs.
One point on redundancy… it mostly about reducing downtime… So I always like pointing out having even an older router with VRRP to main/big router may be helpful since these Mikrotik have to reboot. Power does fail, and a lot of Mikrotik do allow some form of “dual powering” but upgrade are more common than power failures IMO. So if you get a 2nd internet connection, get a 2nd Mikrotik with it – any failover/LB can happen with VLANs and with VRRP one will take over if hardware (not power) fails.
Yup, someone can hit an above-ground pole with fiber and cable on it. I appreciate starlink is newer/generally faster… But GEO sat may be better if “shit hits the fan” – lot of complexity in starlink’s network & imagine that the viasat/hughes/etc likely have some good deals since no doubt there under pressure from Starlink. Starlink always requires a rather wide view of sky, while GEO’s is a narrow beam. More a thought, in all ways starlink is faster. But in a disaster, I’d worry more about them…
I have done similar for certain small installations. for larger installations, a more “centralized” battery plant makes more sense.
One point on redundancy… it mostly about reducing downtime… So I always like pointing out having even an older router with VRRP to main/big router may be helpful since these Mikrotik have to reboot. Power does fail, and a lot of Mikrotik do allow some form of “dual powering” but upgrade are more common than power failures IMO. So if you get a 2nd internet connection, get a 2nd Mikrotik with it – any failover/LB can happen with VLANs and with VRRP one will take over if hardware (not power) fails.
I sort of have that. When I upgraded my RB750Gr3 to the current RB4011, I retained the RB750Gr3 and it is configured as a backup. It’s not automatic - I have to turn on four ports in a switch, and move the cable from the fiber ONT from the RB4011 to the RB750Gr3. So it’s not something I would use for a firmware update on the RB4011, but if the RB4011 died, I could restore service in a few minutes.
Yup, someone can hit an above-ground pole with fiber and cable on it.
Our neighborhood is above ground utilities for power, phone, cable, data. We did have a drunk driver take out a pole around the corner a few years back. 19 hour outage, but I lucked out. I was the last house that was fed from the other direction, so lost power and cable for less than an hour.
I appreciate starlink is newer/generally faster… But GEO sat may be better if “shit hits the fan” – lot of complexity in starlink’s network & imagine that the viasat/hughes/etc likely have some good deals since no doubt there under pressure from Starlink. Starlink always requires a rather wide view of sky, while GEO’s is a narrow beam. More a thought, in all ways starlink is faster. But in a disaster, I’d worry more about them…
Plusses and minuses both ways. So far, I have not seen that Hughesnet is lowering prices due to Starlink competition - but I also have not looked. The biggest downside to a Geo solution is latency. In my CalOES life we deploy satellite based communications via Geo satellites and have to explain to people the latency. Interactive stuff such as phone calls are just plain ugly over Geo.
If I hadn’t had a starlink with some power supply issue at site, that luckily failed over to a viasat (that customer was going to disconnect)… I certainly would not ever remotely suggest viasat or hughes… But given the power topic…
Now queues help a ton with the latency (e.g. drop locally before you hit RF channel limit) and client OS’s TCP congestion control has improved dramatically over the years to help with low error, high latency cases… but TCP is tough at >500ms.
Can you please tell me more about this? For common 12V packs - you hook them up to MT device, and for charging what you use? A common battery charger with trickle-charge function? This is quite interesting to me, as I need to power mostly MikroTik devices with this.
Also, arent CPAP batteries expensive? And why would one need 24V, 12V is enough, and you can put them in series…
I haven’t ever encountered VRRP setup. How does it work? Is some central device for used for switching from one to another? If that is so, then you again have a point of failure. My approach is just to have a backup and another physical device and enough power supplies. If something fails, I restore backup and replace the failed power supply/router.
Shouldn’t StarLink be great for low latency? From what I read online it is from 20-40 ms.
I doubt traditional services will lover prices as they offer different kind of service. They will remain as a niche option and that’s it. In a few years there will be at least 3 new online services, so geostationary internet links will be used for specialty purposes or backup only.
Re Batteries, I’ve used 24V lithium batteries from TalentCell on Amazon generally. The power supply is 24v1a, so if plugged in router draws from that and excess goes to charging (or if usage is >1A for a period, batteries allow higher draw). For a single device, the works very well. Mikrotik rarely even get close the max watts, so this last for many hours & both smaller and longer lasting than a “classic” lead-acid UPS. Now, if you have one of the bigger routers, certainly this doesn’t work .
Starlink is certainly better than consumer GEO options – latency is 30-60ms typically with LEO starlinks vs 550ms+ with GEO. I was more being a contrarian since I have seen a couple starlink fail and had to be replaced. Starlink only takes AC power, so you need a real UPS for these.
I just mention VRRP since I’d put in the “practical redundancy”. Since redundancy is based on underlying goal of reducing downtime (achieving some 99.xyz%), having a 2nd router and using a backup internet on that add a lot more redundancy than just dual powering one . So if you have a backup internet connection, one topology is just having internet using a new/2nd router for the 2nd internet connect. In this model, the device/user VLANs use VRRP e.g. adding VRRP interface, add a /32 in /ip/address for it, and using that a gateway on VLAN dhcp-server network, and VRRP priority set higher on router with “primary” internet. For example, say RB5009 with some cable/fiber and hAPax2 that goes to starlink – both are trunked with all client VLANs between the two routers and the VRRP running on each VLAN. So if the main router loses power, the 2nd router will take over & if the 2nd router has Wi-Fi, even if main Wi-Fi install failed, there still be some Wi-Fi from the hAPax2 for emergency use. Mikrotik’s are relatively inexpensive, so having two solves a lot of backup/redundancy needs was more my point e.g. the internal power of a router can fail too - after the extra PoE+DC jack split in the OP
BATTERY: But how do you prevent overcharging?? I supposed you have circuitry to monitor both upper and lower voltages and stop charging, or you rely on internal battery circuitry? (Apart from that, this is rather smart setup..)
STARLINK: I was actually thinking of using starlink as a backup in a number of companies. I also heard that starlink antenna array uses a lot of power… Have you had experience of how it feels using it in real life?
VRRP: Your comments on VRRP are really good. I think I could use them. Thanks a lot for this. I knew that at one point I will have to deal with this, and you pushed me there to consider the application.
The AC/DC adapter that comes with battery has a voltage sensor.
I haven’t monitored it closely, but SL typically use 50-100W, more with heater/higher usage. I want to say premium dish is slightly higher, but range is pretty variable hard to know. It is that Starlink runs the relatively high-power over longer cable to dish, that looks like beefer USB-C cable, that worries me. IMO it’s a lot of power going through what looks like a USB-C jack on dish. Since I’ve seen them fail, like due to bad power, my recommendation is using them with a decent UPS for sure.
Thanks. So the cable is not PoE? Do you happen to know how long is the cable and is it custom made / pre-cut or can be adapted. Max-length?
Re USB-C connector seems weird as there is a 3m? limitation for USB 3.1.
Also, the power of 50-100W is quite a lot, but USB C seems to be rated for it, but not for X meters needed for the long cable… It is like they are sending AC to the antenna. From what I see PoE++ (IEEE 802.3bt (Type 4)) is limited to 70W practical power which is not enough.
(Disclamer: I would really like to set up a StarLink…)
Yeah starlink works fine. They are easy-to-use. Speed is variable, but it’s pretty comparable to a decent LTE modem (with good coverage).
My advice be to download the starlink app, and check to see if you have enough visibility. GEO sat typically need point toward equator and need narrow beam… Starlink’s LEO is opposite, so needs a view towards to the north (or south) pole & ~120º view of sky. So a spot to mount it may be the biggest problem… You can use the starlink app without a starlink account, it uses AR to scan the sky and report if it a good location or not.
In starlink’s scheme, cables are pre-terminated and you drill a bigger hole for jack to pass. They sell mounts, drill bits, and longer cables…as extras. e.g. like a “masonry kit” that includes a drill bit, cover for hole, and a cute little starlink-brand tube of silicone.
My main grip is their cables between router (or power supply on premium) and dish are custom & fixed length. They do however look very similar to USB, whether it us runs “real” USB, IDK. Just we had to replace at least 3 cables of them for various issues - enough that we order a spare cable with them. e.g. you cannot just “terminate” a new starlink cable.
But the dish ain’t PoE, and draws at lower-end ~50W, with higher watts posisble. So in power outage, you’d need more way more power for starlink, than for an LTE as a backup.
)
)
That system is really complex, have multiple fail points and is expensive to maintain. If I my suggest maybe a more sensible approach, given that you like to thinker - get an Smart UPS 750, or 1000 (they don’t have a fan), or if you are money constrained APC Back-UPS - someting. You can make external ports and connect really big batteries to the system in parallel. You can feed 12V devices directly (MikroTiks take 12V in) and 220V/110V through a converter from UPS directly. If If you are a big spender, you can get APC XL units + external battery packs, but those cost a lot. There are a plenty of videos on this online.