MikroTik

sirbryan

You can either have hardware-accelerated connection tracking (FW/NAT) or routing, but not both at the same time.

Try disabling HWoffload on all the ports, but leave it on for the switch, and see if that fixes anything performance-wise.

sirbryan

He said 30km or longer, guys. I had a pair of dual-band LHG XL's at 32km, from the valley floor to a mountain top, but both the 5GHz and 2.4GHz links were pretty weak and we only got about 10-20Mbps out of it after maxing everything that we could (for US region). I don't think the noise floor helped...

sirbryan

I haven't dug into that myself. I'm still figuring out how to best leverage all these BGP knobs.

sirbryan

Without posting some of your config, it's hard to know how you're doing things. But it sounds like what's happening is if you don't put the /24 somewhere on your router, it stops announcing it to your upstream provider. You need to at least blackhole route the /24 to the router itself so the BGP ann...

sirbryan

Specificity always trumps almost anything else. If provider Z doesn't give you anything but the default route, then you'll need to filter out everything from the other providers except for the default route. With your use case, unless you're saturating one of your providers on outbound, slurping in ...

sirbryan

One thing is missing. You haven't marked the blue links as a LAG with an MLAG ID (it can be the same for both sides if you like).

The blue links themselves are a LAG between the MLAG pairs. To the switches, they will need to be configured just like the links going to the PVE stacks.

sirbryan

The short answer to your question is that it should work and your diagram looks good. I did a similar configuration in the core of my network, with two CRS317's in one MLAG setup feeding six routers, and another MLAG pairing with a 312 and 354 feeding my home/office router (a 2116). Then I connected...

sirbryan

Short answer (from what Raimond pointed to):

For NAT to work, the traffic has to hit the CPU initially, and then it pushes those NAT sessions to the ASIC. You therefore can't enable L3HW Offload on the WAN ports or else it'll never hit the firewall. You can only enable it on the LAN ports.

sirbryan

to get a maximum throughput of 500mbit on freakin masquerading is simply surprising for what's otherwise a 2.5G device. The CRS310's are great routers, if you don't need the CPU to do anything (queueing, firewall, NAT, etc.). I have a few of the 5SFP/4SFP+ CRS310's at customer-facing sites, paired ...

sirbryan

If all ports are part of a bridge, and you enable IP firewall on the bridge, then, unless hardware offload (switching) is enabled on the individual ports (which it is by default), it should send all traffic through the firewall. At that point, you can use whatever Layer 3 rules you want to keep devi...

sirbryan

I don't see how anyone but Mikrotik devs can really help, it's not a configuration issue... Sounds like a good reason to open a ticket, then, if you haven't already. The forum is for users to share what works and what doesn't with each other, with only the slight possibility that a dev might see it.

sirbryan

I am using a 5009 with several ports with VLANs (all on a common VLAN-aware bridge) and I have not yet observed such a problem... I have set the STP mode to "none", as I always do in places where there is no need for STP. Unfortunately, STP is needed on this router due to some uplink redu...

sirbryan

I have an RB5009 where I've started noticing it randomly stop talking to devices on one of the ports. It takes a reboot to fix it. No amount of port-bouncing or bridge tinkering works. I suspect it's seeing occasional route loops or some other packet it doesn't like and it silently shuts down the p...

sirbryan

Mikrotik has "general rule" about not touching existing configs, except during major upgrades where a config update is necessary. Usually, changing connection tracking settings falls under that "not a major upgrade" category. Except they did just this with VRFs and firewall rule...

sirbryan

Well, among other things I just found and fixed the UDP timeout (which is amazing, Mikrotik changing it for new setups but not changing it for existing installations where the user has not changed the default value - talk about breaking systems) which fixed SOME of the issues (RDP it seems). Now I ...

sirbryan

7.14.x reverts/restores BGP as-path filter behavior on egress to previous behavior. Why isn't this mentioned in the release notes? Before and up to 7.7, any outgoing as-path would be filtered based on an incoming AS's number (their ASN was first in the match list). Sometime after 7.7 (not sure which...

sirbryan

The 2116 is a beauty, with plenty of resources for homelabbing.

sirbryan

do you still have use cases where the ARM devices work as full-fledged routers, so using any ROS feature, and not only as APs. If second, why not offer a slimed main package for the "as AP only mode"? I have situations where I use dynamic routing protocols and VPNs on low-power devices li...

sirbryan

Hi, any another experience with 60 Ghz for long distances? Thank you That's kind of a generic question for a year-old post. What exactly are you needing? MikroTik's 60GHz distance limitations have been discussed in several threads. For close-range (<500m for PTMP, <800m for PTP, Cubes don't go as f...

sirbryan

Guys, remember that most of the lab devices they are developing on have nothing else on the disk, so 7.14/7.15 etc. all fit and work fine because likely the CI/CD setup is wiping those test devices with a fresh netinstall every time. You can't expect a lab device to load up all the cruft many of us ...

sirbryan

Is there ISP gear that can be powered via PoE in? I'm not aware of that (at least in Italy). Yes, my antenna was powered by port1 of the ax3, so there are a real use cases. I use AX3's when my customer's wireless CPE is capable of 500-2500Mbps. If it's a MikroTik 60GHz radio, the router automatical...

sirbryan

I have OOB access to the serial console so it isn't a big deal to actually recover if I make a mistake. I guess I'm going to have to test it on site, sadly I don't have OOB connection T_T. Kind regards, Set up ROMON between the routers. All routers connected via Layer 2 will see each other and be m...

sirbryan

L3HW offload is hit and miss on the 2116's. I have it disabled for now on all of mine. There are known issues with ECMP (multiple identical-cost routes) and L3HW offload. They fixed some of it (for plain routing) a release or two ago, but I found new issues with it when NAT is enabled on a 2116. Wit...

sirbryan

Yeah, POE + 2.5G + 24V to me doesn't make sense. It seems one can only use two out of the three features of that port. We use Tachyon radios in parts of our network, and it is nice to see that 2.5G port light up. But they want too much juice to use the router's POE, as do Wave radios. I use them wit...

sirbryan

I ran into this when trying to install a customer's connection. It was at 5GHz, not 60GHz, but concept was the same. I was trying to align it down my side of the street, straight towards the AP, but it was going through a couple of trees. I was watching my phone while turning the radio and noticed t...

sirbryan

Remember that in other posts, MikroTik pointed out that it would actually make file sizes larger to split some things out. Over half what the OP requested be removed is part of the stock Linux kernel (ppp, ipsec, wireguard, etc.). ppp, for example, is required for pppoe, a standard still used by ISP...

sirbryan

Individual ports are added to the bond, and the bonds are added to the bridge.

sirbryan

Back to 7.15b6: My home/office CCR2116 stalled overnight. Log says kernel panic. I had to power-cycle it to get it to come back. It had been running for two or three days (since b6 came out) just fine. Aside from OSPF & BGP to external network, it's got a few containers (piHole, homeassistant, o...

sirbryan

Only one strange thing, after reboot my active interfaces without POE devices were red, "PoE out status: short circuit"... Then you probably upgraded from a quite old version. Always mention your previous version. You can go to interfaces->ethernet and open each interface and set PoE to &...

sirbryan

After messing around with it as bare metal on the Honeycomb (extra-nics was installed), I booted back into Ubuntu and got it working through QEMU/KVM. With 2 or 8 cores, I can get it to receive 1.5Gbps and transmit 700Mbps, so I'm guessing there's a bit of optimization yet to be had. iperf3 on the h...

sirbryan

Huzzah! Just like with QEMU, two NVMe drives works. But it doesn't see any of the NICs or USB interfaces. That would probably explain why the install kernel doesn't see the USB drive after it takes over from UEFI. In System/Resources/PCI it lists just the NVMe drives. I wonder what cards in Ampere s...

sirbryan

I made a tiny bit of headway trying to boot the installer on my Honeycomb LX2. Using a UEFI image from SolidRun, the ISO boots via USB. It looks for an NVMe drive (SATA are ignored), and then burps because it can't find the CD-ROM. Based on what we saw with QEMU, I wonder if it expects the CD-ROM to...

sirbryan

I'm personally running ARM64 variant on my Rock 5B now via KVM virtualization, and it achieves maximum throughput! I'll have to try it via KVM on my SolidRun Honeycomb LX2 that's collecting dust. It doesn't support UEFI boot and their UEFI shim is out of date. Otherwise I'd have spent some more tim...

sirbryan

Sweet. I got it installed and booted.

Now to figure out how to boot it under Apple's Virtualization Framework to get closer to the CPU.

sirbryan

I'm excited to see what people come up with now that the Ampere ISO is out. I tried messing around with it but I can't get things to boot. I tried on Equinox Metal, but couldn't get it work. I do not think it's Mikrotik fault... Metal uses iPXE with netboot.xyz for custom OSes. I use VMWare on X86....

sirbryan

I said quite the opposite. I said we listen to all users, not just the forum In what language? What you said was very clear, and you made no mention of listening to all users. In fact, it seemed to be, if anything, stating that home users, ME included, dont come to the forums, nor all the people I ...

sirbryan

I'm excited to see what people come up with now that the Ampere ISO is out. I tried messing around with it but I can't get things to boot.

sirbryan

*) media - added support for DLNA;

I was happy to see SMB leave the default system package, but DLNA is even worse.

SMB didn't leave. It was replaced with ROSE's version of SMB.

sirbryan

Regarding the "Handshake for peer did not complete" log messages from WireGuard... I confirmed that these were coming from my passive peer entries where I had configured a persistent keepalive. The keepalive configuration was the "trigger". Removing that keepalive stopped the me...

sirbryan

Is this the Fasttrack inactive the intended behavior? According to https://forum.mikrotik.com/viewtopic.php?t=182658 FastPath should be supported with VLAN filtering since 7.2 so this condition should still be met for Fasttrack, shouldn't it? RB5009 doesn't qualify for hardware-offloaded routing, j...

sirbryan

VLAN MTU Issue We have reproduced multiple issues regarding VLAN MTU not applying correctly or resetting to default after reboot. Unfortunately, it is too late to incorporate the fixes into 7.14, so those will be available in the upcoming 7.15beta. Is this an issue introduced in 7.14? I see heavy d...

sirbryan

Should more of us file support/feature requests for generic UEFI support in CHR?

sirbryan

I had a site where this was happening. I don't remember whether it was warm or cold when it happened, but I eventually replaced it with a 5009. All ports (except maybe one) were being used. I have other 4011's still out there with only 3-4 ports being used, mostly on 1-5, and they haven't exhibited ...

sirbryan

IP WAN ports (like eBGP Transit, IXP port, PNI port, residential broadband DHCP, PPPoE etc) are meant to be independent PHY ports outside any bridge, if they need VLAN tagging on egress, you directly create layer 3 sub-interface VLAN on top of the port. This has been discussed ad nauseam. In all of...

sirbryan

Something seems wrong in the code, i do not think that it is a setup problem. If it where a setup problem, i think that the MTU value would be rejected during setup, not silently after a random time. I've seen this on a number of releases over the past year or so, but in my case it can take weeks o...

sirbryan

I tried this tonight WITHOUT using UTM, only Apple. I used some swift from an Apple sample project that used VZEFIBootLoader() & another sample with the serial console window. To confirm, This window is not the VGA graphics from the CHR VM, but a SwiftUI window connecting to the serial port of ...

sirbryan

I can't speak for the 309 as a NAT router, but as L3HW offloaded router, I've got a pair doing it right now as my core aggregation routers, talking to CRS310's and NetPower 16's out in the field. I have two CCR2116's doing CGNAT, and with L3HW offload, NAT load drops to near zero. Unfortunately, it ...

sirbryan

Outdoor AP's cover both WISP and WiFi needs. Better to get those out and on the market because it reaches a wider audience. Likewise, a WISP can start upgrading their sites (since it's backwards compatible) in preparation for CPEs to come. Am I missing something? The new ax hardware is not backward...

sirbryan

Outdoor AP's cover both WISP and WiFi needs. Better to get those out and on the market because it reaches a wider audience. Likewise, a WISP can start upgrading their sites (since it's backwards compatible) in preparation for CPEs to come.

sirbryan

When mixing MTU sizes on VLAN interfaces on the bridge, anything that's not 1500 starts up as 1500, breaking OSPF adjacencies and causing PMTU issues. And now, even if I run my script at startup to "fix" the VLAN interface's MTU (by disabling and enabling all of them >1500), something post...

sirbryan

Short answer: Yes. On AP side put all 60GHz and 5GHz radios in the same Layer 2 network (bridge/LAN/VLAN) and set up CPE bond to be active/passive (or active/backup, whatever it is). This will also allow you to pass VLAN tags through, so long as the bridge/switch/router at the AP side has all VLANs ...

sirbryan

I use these in PTMP deployments as well, with OmniTik 5AC's as the 5GHz AP and its POE out powering a couple of wAP60's at half of my MikroTik-based sites. The rest are 60GHz-only to wAP60's. The bond on the CPE side is active/passive, with the 60GHz as primary, and all of the AP's (5GHz + 60GHz) in...

sirbryan

7.13 crashing when winbox is left open is a known bug

OK. Makes me feel better. Sort of. I thought I had seen/read it before.

sirbryan

So, people with reboots must send their supout files to Mikrotik, please. 🙂 After posting that I had had none, I had two within the past 72 hours, one a 1036 and the other a CRS317. The only difference between the weeks (and months) of them running without reboots and the last couple of days was my...

sirbryan

It is an advantage of RouterOS that MikroTik releases new versions for old devices for a very long time. Other manufacturers have a separate image for every device, and they simply stop releasing updates for older devices, except sometimes for security issues. It could be considered to have a "...

sirbryan

So it's not entirely random, it's tied to some particular configuration. I'd say that's an useful input for mikrotik devs who will try to chase the problems down. And I agree with @whatever, none of my devices (running 7.13) suffer from reboots either. Indeed I'm not using neither openvpn nor wireg...

sirbryan

Multiply that by all routing protocols and on 16MB devices most likely you will not be able to install routing package at all. That's the point: almost nobody will want to run routing protocols (none but static routing, which should not require any executable to run) on devices with only 16MB flash...

sirbryan

I will add that even though it's single-core bound, the throughput on 2004 was still 8Gbps full duplex, or 9Gbps one-way (due to testing on 10Gbps ports, of course) with the router bridging an entire interface with the VPLS, routing it over the second to the second router, which passed the traffic b...

sirbryan

That was me. :D Well, I was one ISP who spent some time tonight testing from a 2116 to a 1036 through two 2004's (as well as with another ISP's two 2004's from a 1036 the other night) and determined it was single-core on egress. (All boxes under test are on 7.13.2.) According to the WISP Talk Facebo...

sirbryan

In a past life (job) we had some devices with mixed AC/DC power supplies in a -48V environment and I don't recall any issues.

sirbryan

Yes it will work.

In pretty much all redundant PSU situations, each PSU steps down to board voltage (for most of MikroTik that's 12V) and the board draws from one or all online PSU's.

sirbryan

See this post: viewtopic.php?p=1043459#p1043459

Try swapping the A/B pins (+/-) between the DRS and KNOT. Also, I couldn't get the /iot/modbus/transceive commands to work, but using a TCP app to connect to the KNOT works just fine.

sirbryan

For lack of a better spot to put this, I'm dropping it in general. I'm having a heck of a time getting a KNOT to talk to a mean well DRS-240 unit that has modbus exposed on an rj45/8x8 connector. pins 6(+) and 7(-) on the mean well running to A+ and B- on the KNOT. .... I've also tried hitting this...

sirbryan

No single device would get better bandwidth than one of the links, That is accurate and NOT bonding, and thus as stated it is not possible, just another form of load sharing. You can bond two EoIP links using LACP, if you really wanted to. It's not as good as using Layer 3 techniques, though.

sirbryan

Yes, you can--sort of. If you build two tunnels over the different connections back to your own office, you can bond over the tunnels. No single device would get better bandwidth than one of the links, as the current bonding technologies assign individual flows to one path or the other. Simply put d...

sirbryan

Vendors are competing for your business. Lots of people get hung up on only using one vendor/manufacturer because of "reasons." But, as a provider, it's up to you to find and use the best tool for your business and customers, and if a vendor isn't making that tool, they're not holding you ...

sirbryan

I'm doing CGNAT for 500+ people on a 1036 with 2.5-3Gbps of traffic at peak and only 5-10% of CPU load running RouterOS 7.11.2. Minimal firewall rules (to protect the router itself) and minimal shaping (about 30 of the 500+ customers).

sirbryan

I don't know how many filter rules you have on that firewall, but that's probably why your CPU is higher. I know I can get about 800Mbps on the CRS300's I've tested without any firewall rules (maybe one or two), so you could try testing to see which rule (or set of rules) is causing the CPU load. Or...

sirbryan

The log says there's no version of the Unifi container that will load on that particular CPU. From https://gist.github.com/jasco/2d39fdc808a1c482ed3c295d0e09c116 : "The instructions for setting up the Unifi Controller on ARM do not cover ARM64. The documentation states that ARM64 is not support...

sirbryan

Pardon the "me too" post, but I observed an OSPF regression between 7.13Beta 2 and 7.13RC2. The beta ran fine on my home/office CCR2116 for a number of days (a week or two I believe). Loaded the RC for a few minutes (it peers via OSPF and BGP to the core of my ISP network) and adjacencies ...

sirbryan

ZeroTier is Wireguard, with their additional relay tech. And now MikroTik is doing their own thing with Back to Home. I'm not disputing the benefits, especially for the double NAT scenarios you speak of (both ends), where there is no choice but to relay. ZT and BTH make that easy. But I also know my...

sirbryan

Fwiw, ZeroTier is already integrated into ROS v.7. As for remote management, I’d choose ZeroTier anytime over WireGuard but the latter might serve well as backup access. ZeroTier is a great tool/service for automagically creating Wireguard tunnels between peers, especially for management & tele...

sirbryan

I know I'm coming in on this conversation late, but I'd like to throw in a couple ideas that you can look at once you get Starlink squared away. 1) I would use Wireguard (if possible) for the management VPN. If one end (your office) has a static IP, the remote end(s) (the router(s) behind Starlink) ...

sirbryan

You might have luck by enabling L3HW offload in (webfig/winbox) Switch -> (select the only switch) -> L3 HW offloading. That only works presuming you've followed all the steps in the L3HW offloading documentation. Since you're doing NAT, do not enable it on any of the switch ports too. Otherwise you...

sirbryan

While I agree with this, there's also a good side to it. I'm thankful for all the people who do run the beta and report issues - I don't have the time to do it - even when they run it in production. It will result in a better final version. Alpha and Beta testing have their place on production netw...

sirbryan

The CRS310's CPU is quite a bottleneck for anything but the simplest of tasks. hAP AX3 or RB5009 would be far more powerful (and cost-effective) options for USB3 drive hosting. If you really need some kind of all-in-one solution, CCR1036 and CCR2116 have M.2 slots for PCIe NVME drives, plus plenty o...

sirbryan

In another case, we have a BGP router with Full Internet (and we have around 300.000 connections), so I have some doubts that L3HW work. Until today we didn't have the courage to activate it (also due to bad previous experiences). Anybody have experiences in these cases? L3HW can give some relief w...

sirbryan

If mikrotik releases a poe variant of the CRS310-8G+2S+IN we will buy hundreds of them. @Normis do you want to share if this might be in the works?

If you open up the new CRS310, you'll see lots of interesting stuff related to POE on the circuit board...

IMG_1262.jpeg

sirbryan

Your configuration looks pretty straightforward, i.e. no routing, just switching, which is good. What speeds are the ports reporting? Do they show up as 10G? Are all the devices in the same LAN subnet?

sirbryan

@gotsprings I haven't used the SA's at this point. I'm using wAP60's and Cube60's as AP's with up to 8 CPE at a time and have over 100 customers deployed on them. Cube 60 Pro's as CPE work very well, along with the older Cubes and LHG's. I use Cubes up to about 500m and LHG60 up to 750m. For redunda...

sirbryan

Also, the 2004 uses less CPU if the traffic flow goes from the SFP28 to an SFP+ port (or vice versa). And I've noticed the machine on the receiving end (all other things being equal) ends up using more CPU than the sender, at least for TCP.

sirbryan

One of my 8010 links is fiber with FS.com modules into a CRS317 at one end and copper into a UF-RJ45-10G in a CRS309 at the other. The second 8010 link uses a DAC I found on Amazon into a CCR2116 at the mountaintop side, and FS.com modules into a CCR2116 at the data center side. The CRS309 is on 7.7...

sirbryan

Yes. As strictly a DHCP server, almost anything (including a Raspberry Pi) would work with DHCP relay configured on the routers back to the DHCP server. If you're asking if the 1072 can act as a router for 7000 clients, that's a different story, and depends on how much bandwidth you want to route th...

sirbryan

If you remove the ethernet ports from the RouterOS bridge, then you can assign IP addresses to the ether1 interface and the radio becomes a router. You can do it on the CPE, the AP, or both. Leave the AP as a bridge (unless you plan to add more CPE) and the CPE side as station bridge, and leave them...

sirbryan

You have to disable Fasttrack in IP -> Firewall.

sirbryan

Don't create a bond on the AP side. Set it to AP bridge as well. The CPE side will handle the failover. This is how I do it in PTMP setups, with a wAP 60 AP and an OmniTik 5AC at the AP end. I connect Cube60's and Cube60 Pro's to both AP's, but configure the 60GHz radio as primary in the bond and 5G...

sirbryan

If you have the original circle starlink dish, those you could bypass as well, and don't need the ethernet adapter... but still need the router. I have an original Dishy kit and you can use it without the router. You plug your Mikrotik into the white port on the power supply (which is just a fancy/...

sirbryan

Because some people don't like or need the complexity of RouterOS's bridge/VLAN configuration
Because there are a few smaller chips they use that don't have the capacity to run RouterOS, but make a great platform for creating cost-effective switches

sirbryan

Yeah, like I care about what some random dude on the internet thinks about it. You can think what you want, we're never crossing paths in real life, I never worked with and will never work with Latin America network operations, mainly due to financial issues in your region, compared to North Americ...

sirbryan

I can confirm it is the same on CCR2216, although the CLI command still works, so it looks like it's some sort of web ui specific issue.

Yeah, support acknowledged it as well. I did what I needed to with the CLI, but wanted to report it here.

sirbryan

I opened a support ticket [SUP-129558], but I'm going to post it here, too. On both builds 7 and 9, when attempting to add any kind of dynamic interface (bridge, VLAN, bonds, IPIP/EOIP tunnels, etc.) from within Webfig, I get an error that the interface type is not supported. Screenshot 2023-09-29 a...

sirbryan

It's included/installed at the factory for rack-mount versions, and included in the kit of screws and mounting hardware for the outdoor version. Phoenix Contact part number MC 1,5/ 2-ST1-5,08 - PCB connector (1900772). It's a 5.08mm pitch 2-position Mini COMBICON (MC) connector. https://www.phoenixc...

sirbryan

It says it's off by 5 degrees. How did you align them? At 78m you should be in the -40's or higher, not -60's.

Personally, I place my phone against the face of the Cube and center it on the remote end.

sirbryan

Somebody doesn't know that RouterOS pushes switching (and now Layer 3 routing) to the switch CPU. I have several of these in production routing (and switching) traffic at wire speed.

sirbryan

If you had an option to switch between 2-pair or 4-pair passive POE, then you could feed the switch with 27V and still run AirMax/MikroTik 24V devices alongside LTU, Wave, etc.

sirbryan

I had issues with 7.11.2 on my CCR2116; had to back it off to 7.11. My CRS310/318's are running 7.10, and my CRS317 and 2216's are on 7.11. Whenever I have issues with L3HW offload, even for devices that have been running for hours or months, I end up bouncing it just to ensure it transfers everythi...

sirbryan

I have a hard time seeing the problem you're trying to solve. You could set up the RB5009 with a CRS3XX in a router-on-a-stick configuration, with the RB5009's CPU handling the Internet and VPN traffic, and the CRS3XX handling inter-VLAN traffic and switching. One of the CRS3XX's SFP+ ports would go...

sirbryan

Can you guys clarify the use case of 2.5G ports but with PoE output? I thought this kind of switch was great for high end PC's, not for plugging in more routers? Access points. Indoor and outdoor. Indoor you've got new WiFi 6 AP's that have 2.5G ports and more powerful radios. Outdoors, as mentione...

sirbryan

What version are you running? Have you disabled/enabled L3HW offload after you added all the VLAN interfaces and IP addresses? Whenever my CRS300's stop routing properly, I have to disable/enable L3HW offload on the switch to get it to resync the bridge and routing tables from the CPU to the switch ...

sirbryan

VPN traffic is handled by the CPU, not by the ASICs. Many of the CPU's have encryption acceleration, so in this case you want the CPU to take that load.

sirbryan

I have this exact same setup. For L3HW offload to work properly (for any CRS3xx or CCR2xxx device): There must only be one bridge on the switch, and all VLANs must belong to it. VLANs cannot belong to ports. All ports participating in switching (or routing between VLANs) must belong to that bridge. ...

sirbryan

I upgraded one partition of my CCR2116 office router from 7.11 to 7.11.2 and OSPF had all kinds of issues keeping adjacencies with a couple of its peers (via VLAN interfaces) and so half of the network's routing table was missing or not installed properly. (Log was full of OSPF up/down entries). I'm...

sirbryan

(OT) I like how Ubiquiti has configured their forum software to accept private uploads on the first post of a thread. Well, I HATE that and I refuse to use it! My point is less about the means or methods, and more about the idea that the issues and feedback are exposed to both internal and external...

sirbryan

(OT) I like how Ubiquiti has configured their forum software to accept private uploads on the first post of a thread. This allows both the support teams to get the files they need, as well as allowing other members of the community to see that a particular issue has been reported. Others can then pi...

sirbryan

7.11rc2 broke iBGP for my office CCR2116 as an rr-client to my two RR's (2011's running 7.10rc4). It was working on 7.11b6 and 7.11rc1. Fortunately I had 7.11b6 saved to disk. Unfortunately there's not a way to get 7.11rc1 back from the website.

sirbryan

I have a second bridge used as "loopback0" on all of my CRS300 switches and CCR2116's. All switch ports belong to "bridge", meaning no ports belong to "loopback0". L2 switching, CPU routing, and L3HW offload all work fine on all of them. I think the undefined behavior c...

sirbryan

I upgraded a RB4011iGS+5HacQ2HnD to 7.11rc1 and MTU's were no longer being applied to vlans on a bridge, they were all at 1500. Rolling back to 7.10, the MTUs were correctly set. It looks like this has been broken for a while, and I didn't notice. I have played around with it a bit, and it looks li...

sirbryan

Is it an NVMe SSD in an M.2 form factor?

sirbryan

I upgraded one of our BGP core router : a CCR1072 to 7.10.2 (included firmware) and the situation was problematic. All CPUs continuously going from 60 to 100% utilization. Anecdotally, when I loaded down a couple of my 7.10 CCR2116's using a 10Gbps speed test after having just inserted over 300K ro...

sirbryan

Another CCR2116 device running a full BGP table on 7.10 is causing sluggishness with 100% load on one CPU after enabling an input route filter. Has this issue been fixed using 7.11b5 or newer? What's the filter? I've got three inbound filters and they work fine: if (dst in 0.0.0.0/0 && dst-...

sirbryan

6.49.x has been the best for me on 60GHz. 7.10 has fixes for 802.11ay devices, but I've seen little to no difference for the original 802.11ad devices. (I have half my AP's on 7.10 and half on 6.49.x). The drops most often occur due to low signal levels, interference, reflections/multipath, and, of ...

sirbryan

What happens if you change it via the web interface (webfig) instead of Winbox? I have over a hundred 60GHz radios (wAP 60, LHG 60) in service and haven't seen this issue, but then I'm always managing them via Webfig. ROS versions range from 6.47 to 7.10.

sirbryan

Yes. The CCR2116 is designed to support NVME M.2 SSD's. I have several such setups and they work beautifully.

sirbryan

I found issues with full BGP tables and high throughput causing sluggishness etc. on 7.10. I loaded 7.11b5 the other day (or 6, whatever the version was before the 24th) and I saw significant improvements in my 2116's. You might give that a try.

sirbryan

On the 2216's I manage, RTSP is enabled, VLAN filtering is enabled on the bridge, with the EtherType set to 0x8100. PVID on bridge and on connected ports is 1, and Frame Types is set to "all" on both as well. Priority of both bridges is also identical.

sirbryan

What does your bridge, bridge VLAN, and port/connectivity look like? Is VLAN filtering enabled on the bridge, and what is the EtherType for the bridge?

sirbryan

What is the affinity set at? Should be "alone" and "alone" (which puts BGP processes on their own core). Also, what is your filter? A list of dozens of bogons, or a regex of some kind? I've got a couple of 2116's pulling in full tables and filtering out everything beyond a single...

sirbryan

Ok... I do not use MPLS. But I think that the priority with v7 should have been to make everything work at least as well as it did in v6. But it seems they got carried away in implementing new features (probably requested by important customers) before finishing that. I think some of the new featur...

sirbryan

Unless there are some other tricks in FT or /interface/wifiwave2/steering we need option to change minimum rates so we can improve roaming decisions made by clients. Reason im asking for this because my devices get stuck for days on 2ghz even if im 1m from AP, on other WIFI6 vendor APs this just wo...

sirbryan

L3HW offload has to be enabled on the switch before enabling it on the ports makes any difference. If it's not enabled at the switch level, then none of the HW QoS marking will apply.

sirbryan

@normis What are the chances that the relay server could be self-hosted, i.e. for service providers? For example, I have a lot of customers with hAP's behind CGNAT. If I could host a relay on a CHR or CCR2116, their app/device could be configured to use that relay, which then forwards their tunnels ...

sirbryan

There was a lot of hype about Terragraph a few years ago. Today 802.11ay devices still limited to 8 stations per AP and still no mesh capablities while Cambium and IgniteNet already have shipping products. I can't speak for MikroTik's plans with 802.11ay devices, but as long as they make stuff usin...

sirbryan

FYI, and also reported via support, the new date/timestamp is off (behind) by one day. With all clocks synced via NTP, the date is showing as yesterday on all devices upgraded to 7.10rcX, both on logs and under System/Clock.

sirbryan

Then they added "Use BFD" flag to OSPF in Webfig (7.10rc6). That is not written in the change log, just: !) route - added BFD; Same as in 7.10rc3 I know that, which is why I shared what I noticed in each of those releases where something related to BFD was changed. Whoever updated the cha...

sirbryan

Same with: 7.10beta8 added BFD (CLI only) 7.10rc1 added BFD (CLI only) Not sure why MT repeat the same stuff in various releases. But I guess: !) route - added BFD; is telling that its also added to gui. But why two times? They added BFD to CLI for BGP first (7.10b8). Then they added BFD via CLI fo...

sirbryan

Please revert to the old WebFig Style or at least give the user the choice. - The new one is confusing - Everything needs more clicks. - Traffic stats now have the size of a stamp - There is no (really, 0) advantage - The width of columns is not saved which is annoying. I agree 100% I trust there m...

sirbryan

I have successfully connected a hub and a couple of drives to a 5009 or 2004. If the drives weren't bad in the first place (which turned out to be the case for a couple), they came right up. So ROS handles multiple drives without an issue.

sirbryan

Much thanks to the MikroTik team for getting this implemented. I've been running BFD for a while via CLI and was pleased to see it added to the GUIs. I have a handful of 2116's running BGP + OSPF to each other and they've all been solid for the past week or so.

sirbryan

What you want would have to pass through a CPU. hAP AX3 has a 2.5Gbps uplink port and four gigabit ports, with a quad-core ARM64 CPU. It should handle Cake/FQ-Codel on the interfaces pretty easily. Next up would be RB5009, with SFP+ and 2.5Gbps ports, similar quad-core ARM64 CPU, but capable of rout...

sirbryan

A CSS610-8G-2S+IN would probably be your best bet. It has two SFP+ ports you could use with a fiber module in one and a copper module (S+RJ10) in the second one, to hand off to AX3 (and eight copper gigabit ports to hand off to anything else if you want). For it's price, it's not a bad deal compared...

sirbryan

It depends on your objectives. It also depends on factors such as throughput to the ISP's, whether you're currently running BGP with them, if you're NATting anywhere, etc. OSPF alone, when configured correctly, will ensure each router's path to the next one will take the shortest path, and if one (o...

sirbryan

@antonsb It would be nice to know why they dropped, if that's possible to glean from the driver (example: signal too weak, stopped transmitting, too many errors, etc.). Perhaps the AP and/or the station could also log the last known signal level as they leave. I have loaded this onto a number of wAP...

sirbryan

OK, so after updating several routers to 7.9.1 during tonight's maintenance window, and then losing two AF60LR's this morning to a bad/failed firmware update procedure, I had to use my phone while in the field to get the replacement Wave LR's online. The routers at the two sites involved in the outa...

sirbryan

But because comments are now inline, I have no choice but to widen my browser to see longer comments. The problem isn't a matter of taste when comments exceed 10-15 chars. It's a matter of usability/readability. Why do you have "no choice to widen"? Does this mouse-hover not work in your ...

sirbryan

As you can see from other commenters, who like it ... This is a matter of taste. How is "left aligned" in your big monitor not wasting space, but centering is wasting space? It is the same amount of wasted space, just in a different place. Do not use webfig in a maximised window on a ultr...

sirbryan

Ran a couple IPV4 tests from my 2116 in a data center in Utah on HE.net (10Gbps pipe). TCP hovers around 1Gbps, as high as 1400Mbps. CPU load on the 2116 is pretty minimal compared to the CHR. With UDP I'm able to saturate the full 4Gbps, smacking your CHR's CPU in the face at 100%, while the 2116 m...

sirbryan

Bug in webfig still persists: Toggle WiFi Interface off, then on. It stays greyed out even if it got enabled. This applies to all toggable rows in any config area.

This is also present in 7.10b. Just noticed this the other day after upgrading a couple routers to 7.9.

sirbryan

You're unlikely to get that kind of detail. It's most likely some kind of driver update or tweak at a really low level. I loaded it on some of my lightly-loaded AP's a couple hours ago and so far haven't seen the same kind of split-second disconnects I'm used to seeing. So now I'm loading it up acro...

sirbryan

I posted a reply on Facebook, but try slightly altering your timers. For example, I have hello at 1s and dead at 4s.

sirbryan

Your best bet (performance-wise) is to: Put them all into the same bridge. Your LACP config seems to be correct already. Create two (or more) VLANs, one for the first group of ports, one for the second group (your IPMI ports). (Or just create one VLAN for the IPMI ports and leave the others unassign...

sirbryan

If you can't get them to connect in the first place, try placing the wAP 60 x3 on a non-metallic mast or pole, or use a plastic bracket like the ones MikroTik makes to get the antennas (they're in the top 25mm of the box) away from the metal. I had issues when I swapped a couple wAP 60's with wAP 60...

sirbryan

SMB containers work very well. ROSE with NFS also works well. I haven't tried ROSE SMB yet on macOS since they improved it (7.9/7.10) but look forward to doing so.

sirbryan

Tell that to the average home user who buys an AX2.

I think a good default (for the next year or so) would be to leave UNII-4 out of the "auto" or default config mix.

sirbryan

I do like the attempt to clean up/modernize Webfig. I've used Webfig almost exclusively for the past three years because it's 1) universally accessible and 2) I can get to and see all the settings in one place, unlike Winbox, where it takes more effort to see/change information & settings. Since...

sirbryan

I have additional bridges for loopback purposes on multiple CCR2116's and CRS300's and it doesn't seem to affect L3HW offload. This is probably because they don't have any physical interfaces associated with them, so there's nothing there to confuse the offload process.

sirbryan

I'd love being able to at least choose the old behavior and having the comments on a different line instead of inline. The previous form of comments on a separated line wasted way more screen space. And the default column width is quite narrow, but you can still decrease it. If your comments are sh...

sirbryan

Being a person that uses the webfig much more than winbox, this change: Why? Genuinely curious as to why anyone would use WebFig over Winbox if they have the option to use Winbox. I use the web interface from multiple Macs and my iPhone all day long. Winbox has its place, but can be finicky at time...

sirbryan

Summary is a VXLAN does work. And, yes, it does get unnecessarily fragmented but "works" nonetheless. Basically you (from a winbox POV): - take wifi2 out of bridge on both - one becomes get set to "station", other remains "ap" - set SSID "AX 2488 Mesh" on bot...

sirbryan

@sirbryan can you share partially your config ? I have created on a AX 3: cfg1 for wlan1 (5ghz) cfg2 for wlan2 (2ghz). that is pretty default, only the DFS channels set to 10min on wlan1 and to ALL to wlan2 I start with the defaults, add a management VLAN to the WAN port and associated firewall rul...

sirbryan

Being a person that uses the webfig much more than winbox, this change: . *) webfig - added inline comments; . is terrible. I'd love being able to at least choose the old behavior and having the comments on a different line instead of inline. Please consider having this as an option and not fixed i...

sirbryan

On my CCR2116s, L3HW offload wasn't working at all on 7.8. I had to revert them to 7.7.

sirbryan

I have probably over a dozen AX3's installed in customer homes by now, with 2-3x that for AC3 and hundreds of AC2. What you're seeing posted here comes from a lot of tinkerers trying to do everything they can on an AX3, certainly not combinations MikroTik has test cases for. I have received zero com...

sirbryan

Saw the topic and just had to come to say this:

Get the AX3.

sirbryan

MikroTik has suggested the CCR2004 as a replacement for the CCR1009. (The RB4011 and RB5009 also would foot the bill in all honesty.)

The CCR2116 is a good trade for the CCR1036.

sirbryan

I want to say I saw this too when trying to format individual partitions, but formatting the whole disk (without a partition table) worked fine. I don't remember the exact details, other than the fact that partitions were the culprit.

sirbryan

If I configure radius accounting will RouterOS be able to get the bits per lease back from the switch chip and pass it up the stack to the radius client or will I just stop seeing radius accounting data when hardware is abled, or will it simply revert back to the CPU? Also, I'm assuming if I assign...

sirbryan

The need for speed... I still don't get it. Why is good enough never good enough ? It's less about the "speed" and more about the amount of data transmitted in a given window of time, thus maximizing the efficiency of the medium. And it's not always about having full access to all of your...

sirbryan

Anecdotally, after installing hundreds of hAP AC2 and AC3, and now some AX3's: All tests run with iPhone 11 Pro and M1 13" MacBook Pro (both 802.11ax capable). I typically max out at around 300Mbps on stock AC drivers (6 & 7), but have seen as high as 450 on upload. If I upgrade the drivers...

sirbryan

"Considerably slower" is relative to the hardware. My ARM, ARM64, and Tile boxes have seen significant improvements. Under 6.48.x my CCR1036 was showing 2-3% on 2Gbps of traffic. Now it shows 0% on the same traffic. ARM64 devices will see an improvement because they're not running 32-bit s...

sirbryan

The 2004 will be fine for 1Gbps with NAT for a home router, but it won't switch at wirespeed. Everything will be bridged through the CPU. On the bench, I got about 20Gbps bridged through the CPU and the CPU got up to 95%. So unless you're planning to max out the 10G ports all the time, it'll be fine...

sirbryan

Or MikroTik could develop a new product using the same Peraso chip as Tachyon and Ubiquiti, but with RouterOS under the hood. Each endpoint could be routed if you wanted, which would work very nicely for PTMP backhaul. With an LHG version for CPE, they could go a bit further than Tachyon and yet uti...

sirbryan

This would be nice as an ISP to be able to push this DHCP option to managed routers, having them redirect to a server on the network that reminds them they are overdue on payment, or to advise them of an outage, etc. Good find on NodeRED.

And boo to all the container nay-sayers.

sirbryan

Sometimes you have to force refresh your browser's cache. There are subtle differences between the versions of Webfig. Other times I've found with certain devices on my network (for whatever unknown reason), I have to reload/refresh the page to get routes (or DHCP leases, or other large tables) to l...

sirbryan

If it can't, can the mikrotik Poe splitters be used in reverse? And then I connect the DC side to it According to docs, POE out on RB5009-power only works if source is 2-pin (maybe DC jack too). POE-in is for router only. But yes, you could take an injector, cut the DC plug off, and feed the wires ...

sirbryan

Switches move traffic around at wire speed because of the switch chip (ASIC), provided they are configured correctly (one bridge, all ports in the bridge, VLAN filtering enabled and VLANs assigned to their respective ports). The CPU should show hardly any usage except when you browse into it, run te...

sirbryan

Yeah, the CPU's of those things can barely handle 1Gbps of routing, let alone generating/receiving a speed test.

sirbryan

Only one bridge can belong to the hardware switch. Additional bridges are all routed through the CPU, which explains the slower speeds. Yes, in this scenario, individual LAG (LACP) ports need to be removed from the bridge, added to the LAG, and the LAG (LACP bond) needs to be added to the bridge. Th...

sirbryan

If you know the IP's they're advertising, you could mark all packets to/from their netblocks and queue based on that mark, or queue based on the source/destination IP's directly.

There might be a fancier way of mangling using BGP filters and/or attributes, too, in RouterOS 7.

sirbryan

There are significant improvements for Tile CPU's in 7.x that may help improve things, if you haven't already upgraded. My CPU on just plain routing 2.5Gbps went from 3% to 0% utilization. I imagine if you are shaping, then fq-codel will be CPU friendly and pretty effective. I don't do any PPPoE, so...

sirbryan

D'oh!

sirbryan

Looks good. The only thing I'd personally do is to bolt the radio higher up on the tilt mount so there's zero metal behind the patches (the top inch or two).

I've never locked the sectors on any radios. And I likewise haven't noticed any difference with mdmg fix enabled.

sirbryan

Use the QME/QMP mount for the wAP shaped devices. Put the included metal bracket right up flust with the top of the mount so the top of the wAP is above it when it's clicked into the bracket. This is dramatically better than direct to a pole of any kind, even a wood pole. Noticed the issue putting ...

sirbryan

I am looking for a 2 gig to 10 gig wireless link for short distances (50 meters) between buildings so we do not have to trench and run fiber. Ubiquiti has several 60GHz options. AF60HD - 2-3Gbps aggregate up to 6Gbps aggregate (3Gbps each way "full duplex") ~ $360 apiece (your best option...

sirbryan

Hi, which beta can I test for more stable w60g ptmp? I have a lot of trouble stabilizing wap60gx3 with three Cube60gAC attached. Using fw 6.49. Thanks! How is your wAP60gx3 mounted? I found they don't work when mounted on a metal pipe/mast/pole/wall. The signal leaks out through the back enough tha...

sirbryan

You do have an explicit accept rule in the filters, correct? Does it show any routes are being filtered (/routing/route/print, look for F in front of the routes)? Do the clients show that they are advertising any routes (/routing/bgp/advertisements print where dst in 2000::/3)?

sirbryan

Are you on 7.8? I had issues with L3HW offload on 7.8 on some 2116's this morning, but it works on my CRS310's. My 317 is on 7.7 and it works very well. I haven't tried 7.8 on it yet.

sirbryan

L3HW offload on CCR2116 doesn't seem to work on 7.8. Reverting back to 7.7 fixes that problem. But it did seem to fix issues on CRS310 (fingers crossed).

sirbryan

If your OPNSense box is doing all the firewall/gateway work, then there should be no need for firewall filters in the 317 (except for input to the router itself, i.e. for trust/ACL rules), and there should be no NAT rules. You can have L3HW-offloading for NAT/firewall or for interVLAN routing, but t...

sirbryan

Actually, what's missing is all the ports in the switch have L3HW turned OFF. The firewall settings only matter if you want L3HW-accelerated NAT (instead of interVLAN routing). /interface ethernet switch port set 0 l3-hw-offloading=no set 1 l3-hw-offloading=no set 2 l3-hw-offloading=no set 3 l3-hw-o...

sirbryan

It's working for me. My connections to the RR's are dual-AFI on a single BGP connection, with a route filter to change the gateway of IPv6 routes to the IPv6 loopback of the router. Otherwise it tries to use the IPv4 literal (::ffff:x.x.x.x), which could also work if you add it as a loopback on the ...

sirbryan

On all my routers running 7.7, I switched all my OSPFv3 types to broadcast from PTP. Most are dedicated VLANs used for PTP links between routers. After that was done, they all came up have been stable for a week or so. I also had IGMP snooping enabled on one switch, which was causing problems. Had t...

sirbryan

You mean "Protocol Mode" in bridge set to "None"? I have this set to None on all my devices including the RB1100, all my reported 7.8 problems were with it set to None. I also tried disabling Fast Track as this was also mentioned somewhere up this thread and changelog. Didn't he...

sirbryan

Routes show the HW flag if they're eligible for offloading, whether or not they're actually offloaded. I find HW offload for IPv6 to be quite buggy, still. I have a variety of CRS300's and 2116's all running 7.7. IPV6 + IPV4 L3HW offload on the 317 has been working fine, but on the CRS310's, with IP...

sirbryan

What is the reason to have such short lease-times? Maybe it is useful in a guest wifi in a restaurant or similar, but in "normal" networks I set the lease time to 1d or 7d. I guess you could flip the question around and ask yourself whats the advantage to having a longer lease time? Is th...

sirbryan

There are ways to help people see a point or understand specific advice without constantly calling to attention their lack of expertise in an area (this goes for anybody and everybody). There is also nothing wrong with conceding a point when valid counter-arguments or corrections are provided. For t...

sirbryan

No, you're essentially hitting the maximums of that CPU, since PPPoE is CPU-bound.

Depending on what your Internet connection is, you could front it with something like an RB5009.

sirbryan

What version are you running?

For reference, my CCR2116's (same 16-core processors) are hitting 2.5Gbps, 200Kpps, at 12%. So it could be a PPS issue more than a throughput issue. You want to figure out how to push as much as you can towards L3HW offload.

sirbryan

Thats fine but I have a single bridge with multiple VLANS. So you are saying create a separate vlan for the docker?? A VETH is kind of like an EOIP interface. If you make it a member of a bridge (and tag it to a particular VLAN's PVID), then you can assign it an IP in the subnet for that bridge (or...

sirbryan

Hi @sirbryan Are all of the ports on your switch enabled for L3HW offloading? I had a configuration when I had a port with disabled L3HW, and if any VLANs defined on the port with the disabled L3HW are present on the other ports, those ports would be silently disabled for L3HW either. I asked on th...

sirbryan

you just loose flexability because you can only really shape 'downloads' to the individual AP. What if your uploads get overwhelmed? You don't have an interface shaper that will handle that if you have more than 1 AP. To date that hasn't been a problem. At some future point I'll load queueing + sha...

sirbryan

I don't like how queue trees behave when explicit directionality isn't set via a packet mark. Am I extrapolating that you have a vlan per customer from this tree? Each of those VLANs goes to an AP. I have 10-25 customers per AP. The queueing is on the VLAN's egress. My hope was to take just enough ...

sirbryan

If you want POE, the CRS112-8P-4S-IN isn't too bad, and you can get a few SFP-RJ45 adapters for the SFP's.

The RB5009 with power would give you eight POE ports plus an SFP+, but you'd still need a second (cheap) switch for the remaining ports.

sirbryan

Thanks @sirbryan for clarification, waiting for support to know if mine is a bug.
Are you using MSTP on your devices? Or RSTP?

RSTP.

sirbryan

I emailed support after upgrading a couple of RB4011's to 7. Their response was, with the hardware bridge support added to 4011, the CPU has to be used to bridge the two switch chips for VLANs that span both switches. It's as simple as tagging the "bridge" itself in all VLANs that you want...

sirbryan

What is working for me: As AP's: - wAP 60 - wAP 60 180 - Cube 60G At close range, despite their 60° pattern, wAP 60's mounted to a round mast almost act like they have a 180° pattern. I have one customer with a Cube 60 90° to the side of the wAP 60 at 200m and it's working great. On the flip side, w...

sirbryan

I did it with both IP's on the interface as well as all ports bridged with VLAN's. In both cases, it's all done in CPU, so neither method has any hardware offload mechanism to leverage. I also realized that the 19Gbps limitation was more about the fact that I was using two 10Gbps cables from upstrea...

sirbryan

Your “uplink” port should be an independent Ethernet interface with plain L3 VLAN on top of it, not in the bridge. Remove it from the bridge. This is the issue right here... Um, no, it's not. And MikroTik recommends against that, especially for hardware offload to work on switch chips (which is lit...

sirbryan

I'm 99% sure you did your bridge configuration incorrectly for your specific topology. See here: https://forum.mikrotik.com/viewtopic.php?t=183142#p987793 I'm 100% sure I didn't. I use this same bridge/VLAN config at 15 sites on 20+ routers, including hAP AC2, CCR1009, CCR2004, CCR2116, RB3011, RB4...

sirbryan

Here's an odd one. I've spent hours overnight and this morning trying to figure out why a newly-deployed 310 won't properly offload routed traffic. I migrated the config from an RB4011 to the 310, similar to what I've done at other sites, which are working fine. The only difference is that this one'...

sirbryan

Out of the box, it should come configured as a "dumb switch", with all ports in a single bridge, which is then hardware-offloaded. The MTU is usually set at 1500, so you might have to go in and change the L2MTU on all the ports. You might have to disable RSTP on the bridge, but be careful ...

Search found 437 matches