MikroTik

IPANetEngineer

an iBGP peering ensures that both border routers have a full view of the routes of the other router and also the interior of the network (if you're iBGP based on the inside) In most cases, an iBGP peering is desired between two routers in the same ASN that take in full tables. Otherwise, traffic cou...

IPANetEngineer

OpenFlow is still very much in use! ISPs, Internet Exchanges and Data Centers all have use cases for OpenFlow. There is a fantastic open source controller called Faucet that is easy to get up and running. https://faucet.nz/ We have been working on a project to use OpenFlow to manage traffic engineer...

IPANetEngineer

I'm waiting on my CCR2004 to get here. Then i'm going to benchmark it with our iperf3 and BGP full table performance lab that we've used for MUM Presentations in the past.

We maxed out the CCR1072 when it first came out with 80Gbps of traffic, so we should be able to make the CCR1004 fall over

IPANetEngineer

It's also worth pointing out, that many IP transit providers (not all) will allow you to announce a prefix length greater than a /24 (i.e /25 through /32) over multiple connections to the same provider & will aggregate the prefix to /24 or less when announcing it to peers in the DFZ.

IPANetEngineer

I'm curious to know the answer to this as well.

Does the long term image load and run properly?

IPANetEngineer

However for WISP networks, hands down EIGRP. And since it's quite simple i'd wager its a lot easier to program and implement than IS-IS As consultants, we design and build a *lot* of WISP networks globally and I can probably count on one hand the number of times i've seen EIGRP used in a WISP in th...

IPANetEngineer

I don't disagree that /31 would be useful but i'd rather see the time spent on improving IPv6 support. The number of IPv6 networks we've been doing consulting work on in the last 12 months has skyrocketed. Once dual stack is in place, IPv4 public requirements don't disappear but are certainly dimini...

IPANetEngineer

when Router C receive the route it is not complete and it will not choose as best path, this is the issue

If a route is not valid or active, it often means the next hop is unreachable. Check that first.

IPANetEngineer

Are you going to put all of the 1Gbps links into the same data center?

IPANetEngineer

This is normal for most BGP routes and does not indicate a problem. This is typically due to a route being redistributed into BGP. Here are the origin types from the BGP-4 RFC (EGP is no longer used so it's listed but not important anymore) https://tools.ietf.org/html/rfc4271 a) ORIGIN (Type Code 1)...

IPANetEngineer

Does you upstream provider support BGP communities? If so, I'd use those instead of prepending which doesn't work as well anymore.

Can you post the config and routing table for each router?

IPANetEngineer

Yes, please add this to the list. More and more people are asking for it.

IPANetEngineer

Can you post configs for all 4 routers in the A,B,C,D chain and the output of the following on A and D?

mpls ldp neighbor print detail

mpls forwarding-table print detail

interface vpls print detail

IPANetEngineer

Ok - next question is re-adjacency – with OSPF can take up to 5mins will iBGP be the same

How many routes and routers do you have and what OSPF network types are you using - boradcast, point-to-point, etc?

IPANetEngineer

Is stubbing and range/summary mandatory for take advantage of the benefits? It definitely helps. The topology information is what contributes to CPU overhead and thus convergence time. When you're summarizing, you're only sending routes and not the detailed area topology info. At what level (# of r...

IPANetEngineer

You can definitely run an additional area and connect it at more than one point. This is why the backbone area exists and when there are redundant paths, they will be used. Areas will definitely help with scaling OSPF when applied to areas of the network that aren't core transit. CPU should be a bit...

IPANetEngineer

Yes, OSPF (or any IGP) is needed to provide loopback reachability for an iBGP AS so issues in OSPF that cause routes to flap can affect BGP and cause the peerings go down.

If the network is well built and designed, this shouldn't happen very often.

IPANetEngineer

You can't separate them. OSPF no matter how much you want to try and pretend differently, it has certain design restrictions that are very restrictive for service providers. Those restrictions make sense in many enterprise environments because of the vastly different topologies and real world condi...

IPANetEngineer

But we have no other protocol to use, we have OSPF and thats it. So rather than picking OSPF to 'solve problems' we have to figure out ways to solve the problems of an OSPF design. Problems like summarization at key points, but spanning multiple areas. I've had to use multiple OSPF area 0 instances...

IPANetEngineer

I have not tried to replicate this exact issue, but I did publish an article a few months ago about Juniper to MIkroTik MPLS with VPNv4 which works well. You can at least use it to verify MPLS between MikroTIk and Juniper. https://stubarea51.net/2020/01/22/juniper-to-mikrotik-mpls-and-vpnv4-interop/

IPANetEngineer

OSPF works fine for corporate/enterprise IS-IS works far better for 'service provider' environments EIGRP works in both This is **NOT** the way to look at routing protocols. Routing protocols solve problems. We have to stop looking at them as enterprise vs. service provider. OSPF and ISIS are link ...

IPANetEngineer

I would much rather have IS-IS or OSPF segment routing for WISP environments. Nobody wants to run MPLS on EIGRP when you have SR available

EIGRP while released to open standards, wasn't completely opened up so most of the best features are missing anyway.

IPANetEngineer

This is more of a general WISP post than anything but there are some specific MikroTik references in here. i've been wanting to do this for a long time but it needed some depth and attention so it's been on the back burner for a while. I finally got a basic comparison written of all the major WISP r...

IPANetEngineer

We've run into this with a few clients that run the 4011 and have used the following as a workaround while we work on a ticket with MikroTik. It seems to be more stable in the few we have tested.

Set the CPU frequency to 1200mhz

system routerboard settings set cpu-frequency=1200

IPANetEngineer

We've run into this with a few clients that run the 4011 and have used the following as a workaround while we work on a ticket with MikroTik. It seems to be more stable in the few we have tested.

Set the CPU frequency to 1200mhz

system routerboard settings set cpu-frequency=1200

IPANetEngineer

We use Unimus for our Data Center and also for our clients. It can backup, notify about config changes and execute automated commands on multiple devices. It supports MikroTik and lots of other vendors as well. It is not expensive and is an excellent tool. https://unimus.net/ https://unimus.net/imag...

IPANetEngineer

Having VxLAN support in HW on the CRS326-24S+2Q+RM would be just amazing.

IPANetEngineer

A couple of questions:

Can you ping the subnets from within your ASN? Also can you post your routing table?

IPANetEngineer

I completely get that it's not trivial. I spend a lot of time consulting in disaggregated networks where we deploy IP Infusion or Cumulus Linux on ONIE switches and sometimes we have to contend with issues in getting the FIB pushed down into the ASIC. My point is that it's possible in software and t...

IPANetEngineer

There is an important point the OP is asking about that is not being addressed. The marvell prestera ASIC that is in the CRS317 and CRS309 is capable of routing in hardware (not the CPU). The question he is asking is if MIkroTik is considering enabling that functionality (which already exists in the...

IPANetEngineer

Confederation is really designed to allow eBGP style policy enforcement within an iBGP AS and break up smaller sub-ASNs within one logical ASN. I've done migration and merging of Public ASNs a numer of times and what you really want to look at is running different instances of BGP on routers that ne...

IPANetEngineer

Now that VxLAN is supported in v7, EVPN would be fantastic to have as well to use as a control plane for VxLAN and for multihoming.

IPANetEngineer

Can you post the full config and active routing tables for the MikroTik and PF Sense device?

IPANetEngineer

So the MikroTik router is receiving the ICMP traffic from the host behind the PF Sense firewall and sending it back. Have you performed a capture on the PF Sense FW to see if it receives the traffic? It seems like your issue is in PF Sense based on the data you sent. The MikroTik is routing correctly.

IPANetEngineer

If you look correctly, he need to route 192.168.0.0/16, as 192.168.5.1 is a part of this network

You're correct, I missed that it's a /16 mask, but I believe the route will still need to be on PF Sense.

IPANetEngineer

Hello, You'll need to add a route : ip route add dst-address=192.168.0.0/16 gateway=10.100.0.60 distance=1 He shouldn't need a route on the MikroTik as both of these will be directly connected networks. More than likely (making an assumption from the drawing), the default GW for the 10.0.0.0/8 netw...

IPANetEngineer

Netwatch would probably be the easiest tool.

You can ping a VPN endpoint and then take an action when it is up or down...like enabling a firewall rule to block all traffic out a certain port to the Internet or changing the routes.

https://wiki.mikrotik.com/wiki/Manual:Tools/Netwatch

IPANetEngineer

If you are going to be moving a lot of traffic between ports at different speeds , you may be exceeding the buffer (which is required for moving traffic between ports of different speeds.

IPANetEngineer

In the lab, I just lowered the VxLAN MTU to 1400 because it was quick and easy. But in production, I would raise the IP MTU of the routed network to something larger like 9000 so that 1500 byte frames can pass without issue.

IPANetEngineer

The CCRs will be too slow until RouterOSv7 is out as a stable option with BGP. The example below is 17 minutes of convergence on CCRs. Here are some examples of convergence time for CCR vs. CHR using Free Range Routing as the RR. The full presentation from MUM US 2019 is here: PDF https://mum.mikrot...

IPANetEngineer

You're welcome! I'm not sure if an IPv6 underlay is supported, but I may try that later today

EDIT: Looks like the VTEPs and VxLAN interface group multicast address are IPv4 only for now.

IPANetEngineer

Here is a blog post I did with a VxLAN lab in EVE-NG between 3 routers and 3 linux servers:

The blog and configurations are available here:
https://stubarea51.net/2020/02/15/mikro ... ook-vxlan/

IPANetEngineer

Here is an example working lab in EVE-NG between 3 routers and 3 linux servers from an article I wrote:

Configs are in the link here:
https://stubarea51.net/2020/02/15/mikro ... ook-vxlan/

IPANetEngineer

Agreed. If we see EVPN added to BGP to support VxLAN in the future, MikroTik will see a massive increase in sales from people who need inexpensive devices to act as a VTEP.

IPANetEngineer

This meas TCP port 179 for BGP is not answering on the remote end which can be one of several things

1) The source of your peering
2) Firewall rules
3) Issue with the router you are peering with.

IPANetEngineer

Can you post your config? What type of AirFibers are you using?

IPANetEngineer

This article I wrote will probably be very helpful since you're familiar with Cisco. It translates cisco switching configs for VLANs to the MikroTik equivalent.

https://stubarea51.net/2019/02/06/cisco ... and-vlans/

IPANetEngineer

Yes, you can do this. Here is an example witjh two Loopback/bridge interfaces

[admin@lab1.iparchitechs.com] > ip route vrf add routing-mark=vrf-1 interfaces=lo1,lo2

IPANetEngineer

Typically when everything else is identical, the oldest route wins

IPANetEngineer

If you have 200 sites, then OSPF is not the right protocol anyway.

I would look at BGP and filtering to solve this problem

IPANetEngineer

I really prefer BGP for what you're trying to do as I really dislike filtering in OSPF. That said, you could filter the routes for each HQ on the ospf-out chain at the Branch. That should prevent the HQs from transiting the branch in an outage but let the branch still talk to each HQ. Probably somet...

IPANetEngineer

I agree, i'd like to see buffer numbers on all the switching chips to be able to plan. The Marvell Prestera chipset this switch is based on (98DX3236) is used in a few other platforms and the listed buffer space is 12MB for the 24 port model. https://eltex-co.com/upload/iblock/9af/MES_2324P_2348P_da...

IPANetEngineer

This is happening because you have them in separate instances. The BGP best path algorithm is only used for routes within the same instance. Otherwise MikroTik's general route selection algorithm is used between instances. From the wiki: "Best path algorithm compares routes received by a single BGP ...

IPANetEngineer

Cool....thanks for all the hard work!

IPANetEngineer

I would look at this platform on either side if you want to solve the window problem.

http://wanos.co/wan-optimization/how-wa ... ion-works/

IPANetEngineer

Agreed, this is great work.

IPANetEngineer

I think a routed Data Center Interconnect (DCI) is better than Layer 2, so yes I would route between them and then use VPLS if you must extend VLANs.

But, to be clear, this will not solve the TCP window problem.

IPANetEngineer

Here is an article we did on high availability for PPPoE servers. It has example configs in it. Hope it helps!

https://stubarea51.net/2018/04/23/pppoe ... atorsbras/

IPANetEngineer

I would recommend using OSPF for transit subnets/loopbacks and iBGP (peering to an RR with loopbacks) for customer subnets. Add MPLS if you want to deliver L2VPN or L3VPN services. Here is a topology overview: https://stubarea51.net/wp-content/uploads/2020/02/WISP-Routing-Arch.png Take a look at thi...

IPANetEngineer

But how can it help if we cannot change the window size on the tunneled clients connections ? The only way I've ever been able to deal with this is by having a WAN optimization box between the DCs like Riverbed or WanOS. We had a DC migration between DCs with 50ms latency and had to use a Riverbed ...

IPANetEngineer

iBGP is a little easier. OSPF can support ECMP and since iBGP uses a loopback to peer - advertised in OSPF as a recursive next hop, it works well.

The only caveat is if you also run MPLS - the LDP implementation that MikroTik uses is not ECMP capable.

IPANetEngineer

Are the data centers in production? How much traffic is normally on the link while you are testing? The CRS326 most likely has 12 MB of packet buffer memory (based on other Marvell Prestera DX3236 board specs) while the CRS 317 packet buffer memory is unknown. As primarily a 10 gig switch, i'd assum...

IPANetEngineer

The only way this works without using routing marks (which i'd avoid) is if you have an eBGP multi-hop peering.

https://wiki.mikrotik.com/wiki/Manual:B ... interfaces

IPANetEngineer

I would much rather see stacking. MLAG is fine for Layer 2, but it's a nightmare for Layer 3. Stacking works well for both. Considering a lot of the CRS3XX chipsets have L3 HW offload that has yet to be taken advantage of, it would be nice to be able to form an LACP chanel across two or more switche...

IPANetEngineer

This is the way.

Indeed

It would be nice to be able to run either fq_codel or cake in RouterOS for better shaping options. Please consider adding this MikroTik.

IPANetEngineer

I am anxious to test this as well. There are a number of things in BGP that v7 is supposed to fix/improve.

Hopefully we'll see a new RouterOS v7 in February with BGP added

IPANetEngineer

LISP is a protocol that Cisco helped to develop and it never really saw widespread use.

https://orhanergun.net/is-lisp-locator- ... ocol-dead/

What's the specific use case you have in mind to implement in MikroTIk?

IPANetEngineer

One idea I had was to change everything to internal addressing and set up MPLS then have a VPLS instance to each base station and run a central PPPoE concentrator in our datacenter. The problems with this setup is loss of the data-center or the concentrator would cause a 100% outage everywhere. It ...

IPANetEngineer

I sure hope so! MIkroTIk WIFI6 would be amazing!

IPANetEngineer

I would consider the CGN NAT range for this as it's not publicly routable 100.64.0.0/10 We use it extensively in our designs for PTP addressing where public addresses would otherwise be used. It works very well. You can then use a public loopback to NAT the traffic to. Here is an example: https://st...

IPANetEngineer

Intel server cards like the DA520 require Intel chipped optical SFPs or coaxial DACs.

If you are using generic SFPs (either optical or DAC), they must be chipped for Intel on the NIC side.

The CRS326 will use a generic SFP without issue.

IPANetEngineer

Yes, it is a great platform. We have used it to test the CHR for each of these presentations. CHR as a BGP Border Router https://mum.mikrotik.com/presentations/EU18/presentation_5188_1524562405.pdf CHR as an MPLS router https://mum.mikrotik.com/presentations/EU19/presentation_6291_1554448059.pdf CHR...

IPANetEngineer

What error is the PPPoE client showing in the log for not connecting?

IPANetEngineer

It's a performance issue. Hyperthreading is helpful for certain types of applications, but not for virtualized routing. There is a good explanation by @TomjNorthIdaho in the linked thread https://forum.mikrotik.com/viewtopic.php?t=138549 Why disable hyper-threading ? Hyper-Threading is a CPU trick t...

IPANetEngineer

I prefer 9.9.9.9 / 2620:fe::fe

It has malware protection and is very transparent about not storing or tracking user data.

https://www.quad9.net/policy/

IPANetEngineer

But the privacy/restriction problem will only move from the ISP resolver to the DoH resolver chosen. Whether that is an improvement, depends on the local situation. That's a great point, but at least the user has the choice of which DNS resolver to trust and it's obscured to the transit providers.

IPANetEngineer

I might be a minority here, but all this DNS over https/TLS,etc, in my opinion, has nothing to do with user's privacy at all, but it has everything to do with making ad blocking and corporate filtering obsolete. I would tend to disagree. The case you mention is only one possible application of DNS ...

IPANetEngineer

Have you done a packet capture of your video to ensure that all ports/protocols and src/dst have been properly identified to mark?

What does your config look like?

IPANetEngineer

The Maximum MTU on that model is 10218 bytes for ports 1-24 and the sfp+ ports ref: https://wiki.mikrotik.com/wiki/Manual:Maximum_Transmission_Unit_on_RouterBoards The following commands should raise the MTU for what you need. select the interface numbers to apply interface ethernet set l2mtu=10218 ...

IPANetEngineer

Are you using the same version on both sides?

IPANetEngineer

I believe in version 7, MikroTik is using the native VRF capability of the Linux Kernel, so it will be interesting to see if this is resolved once BGP is released in the beta

IPANetEngineer

Just a warning on using two /32 addresses as a workaround for the /31 problem.

We've had a number of problems with OSPF stability using this approach if you're planning on using OSPF for a dynamic routing protocol.

IPANetEngineer

What does your current config look like?

IPANetEngineer

Can you also post the routing tables for each node and the IPs of the VPLS endpoints?

IPANetEngineer

I wouldn't redistribute into OSPF for reachability. It would be better to build iBGP in your AS and then build a full table route reflector so that you can deploy a PE router to peer with customers. Here is an example and the presentation I did last year: https://mum.mikrotik.com/presentations/US19/...

IPANetEngineer

Currently, it is not possible, but because of a lot of customer requests for it, we are strongly considering to implement it! :]

This is the single most important switching feature our customers for MikroTik consulting are asking for.

IPANetEngineer

So it may be helpful to clarify that you have two options to connect the server to the MikroTik switch 1) You had originally asked about a DAC cable which is a coaxial copper cable with SFPs molded and permanently attached to the ends. You can either purchase one from Lenovo, one that is compatible ...

IPANetEngineer

Here is a sample config that may be helpful for a subscriber router from an article I did on IPv6 for ISPs It sounds like you are almost there. Make sure you have assigned a subnet from the IPv6 prefix that's delegated to the router. It needs to be on the interface/bridge that yourwired/wireless net...

IPANetEngineer

I especially enjoyed reading about the network in Serbia to detect hail. Good stuff!

IPANetEngineer

It should be the real one...VRRP only answers ARP for the common MAC address and passes the traffic to the real interface. IIRC outbound traffic should originate from the physical interface.

IPANetEngineer

Is there any reason you wouldn't use DHCP relay instead of bridging a Layer 2 tunnel segment?

IPANetEngineer

Still, SNMP monitoring of BGP parameters (peer up, number of prefixes received, number of active routes to that peer) would be welcome, e.g. for graphing.

I agree, I've been dying to get SNMP monitoring of BGP peers for a long time.

IPANetEngineer

A few things to look at

1) Cost can be set on both sides of the link, so make sure you've set the cost on the correct side
2) Cost does not cross areas, if one of the links is in a different area, that could also contribute to the problem

Can you post the MikroTik and Cisco configs?

IPANetEngineer

I’ve just received an answer from MikroTik: Thank you for the report. There is a known problem with enabled connection tracking and fragmented packets. If OSPF packet is being fragmented, then connection tracking passes it to OSPF twice, causing an error. Currently as a workaround I can suggest to ...

IPANetEngineer

It looks like you're applying this on routes learned inbound. Have you tried applying a similar policy to the routes as they are advertised out to the RR clients?

IPANetEngineer

If the routes are received by RIP from a RIP neighbor, they are not "redistributed" but rather learned routes in RIP. You can however filter the routes with a prefix list by interface. First create the prefix lists with the routes you want to permit/deny in and then another prefix list for the outbo...

IPANetEngineer

A few questions:

1) What version or versions of RouterOS are involved?
2) Are the RR clients all MikroTIk/RouterOS or are there other NOS?

IPANetEngineer

This is a pretty simple solution. You can purchase DAC cables that are flashed on one end for Lenovo and are generic at the other end (since MikroTik accepts almost all types) https://www.fs.com/c/10g-sfp-dac-1114 We've had a lot of success with FS.com when connecting hypervisors, servers, storage, ...

IPANetEngineer

It's only a matter of time before RPKI becomes a standard for most IP transit peerings. This is a feature we desperately need to keep using MikroTik routers to peer into the DFZ.

BGP FlowSpec is also important for DDoS mitigation.

IPANetEngineer

You need to use iBGP throughout your AS - redistributing in and out of OSPF for this kind of use case is something i'd avoid. Here are config examples from a presentation I did: https://mum.mikrotik.com//presentations/US13/kevin.pdf Ideally you want a full table Route Reflector to take full tables a...

IPANetEngineer

A few questions

1. How many PPPoE sessions do you have?
2. What is the total throughput?
3. Did you recently add more clients?
4. What do the CPU/Memory resources look like?

IPANetEngineer

OSPF is not an ideal routing protocol for hub/spoke tunnels, issues with one spoke can affect all the others - it's not scalable. You'll find similar guidance in Cisco as well. BGP is really the way to go as it has much better filtering options and you can assign a different AS for each spoke site t...

IPANetEngineer

Glad you found the transit fabric helpful!

One of the largest WISPs in the US who is a client of ours (2000+ towers) uses it in certain areas of their network to increase capacity between towers.

You should file a bug report with MikroTik since you found a repeatable error so that work isn't lost

IPANetEngineer

Kicking off the new year of blog posting with an article on MPLS and VPNv4 interop between Juniper and MIkroTik. Here is the article link and overview of the design with the configs from EVE-NG. https://stubarea51.net/2020/01/22/juniper-to-mikrotik-mpls-and-vpnv4-interop/ https://stubarea51.net/wp-c...

IPANetEngineer

I've used quite a few x520s in Hypervisors and they definitely need Intel-chipped DAC or SFPs. That's most likely your issue.

IPANetEngineer

Awesome, now all we need is stacking on the 40G ports. MikroTik was working on it when I talked to them in Vienna last year at MUM Europe. I hope we see that as a feature this year. It would be nice to be able to build LACP channels across multiple switches.

IPANetEngineer

This is what you're after....

https://wiki.mikrotik.com/wiki/Manual:R ... ting_rules

IPANetEngineer

add address=2001:44b8:2159:cf01::1 interface=ether2 add address=2001:44b8:2159:cf00::1 advertise=no interface=ether1 add address=2001:44b8:2159:cfff::1 advertise=no interface=pppoe-out1 Are these all supposed to be /128? I don't see a mask assigned. I wrote a brief primer for designing and deployin...

IPANetEngineer

https://apps.apple.com/us/app/openvpn-c ... d590379981

IPANetEngineer

If you want more rapid and reliable failover than Layer 2 MSTP, a routed network with OSPF or BGP would be more resilient and could failover in milliseconds using BFD.

IPANetEngineer

Take a look at OpenVPN on RouterOS. You can put it into the ethernet mode and hand out IPv4 and IPv6.

https://wiki.mikrotik.com/wiki/OpenVPN#Bridge_mode

There is a client for Android and Apple.

IPANetEngineer

edit: forgot the most important bit, thanks IPANetEngineer for chiming in! No prob....the other thing you might want to consider for mgmt if it only allows outbound connections is an SSTP outbound tunnel (which traverses NAT with no issue) to a MIkroTik CHR in Digital Ocean or AWS. That way you can...

IPANetEngineer

There is a good thread on it here.

viewtopic.php?t=136465

IPANetEngineer

This is a pretty broad topic. What features do you need and what kind of network/locations will this serve?

IPANetEngineer

The behavior you described sounds a lot like an MTU problem. Have you tried pinging from a user computer with the DF bit set to see if you can get 1500 bytes through?

IPANetEngineer

Well shucks. That adds about 23 grand to the project. I guess I'll need to rethink the whole thing. Would I be able to pay you to go over this design in a little more detail with me? Charles No problem, shoot an e-mail to consulting@iparchitechs.com with your contact info. We've designed and built ...

IPANetEngineer

Instead of using the interface list on the NAT statement, try creating two NAT rules with explicit out interface statements - one for the fiber WAN and one for the LTE interface

I've done a similar setup numerous times without issues switching from fiber to LTE on the private LAN side.

IPANetEngineer

VRF Lite in MIkroTik would be using the routing marks as a standalone without MPLS/VPNv4. This is supported.

Are you trying to use routing protocols in a VRF as well?

IPANetEngineer

The problem you are having is more than one default route. With a few exceptions that prob don't apply here, you can't have more than one active. The solution depends on whether you use the router for more than just this VPN or of it's dedicated to only the VPN. If yes, then you may want to create a...

IPANetEngineer

Your other option would be to use the RJ45 serial port, console in with the proper serial cable and USB adapter and re-enable mac-telnet from the CLI.

It's not Winbox, but you wouldn't have to wipe the config. If the config is very brief, that may be the easiest though.

IPANetEngineer

If that's the case, you'll probably want to use each IP individually as a loopback to NAT the traffic to and advertise them in iBGP or just in OSPF. Here is a blog I did that has some of the config you need. You can skip the VPLS/MPLS sections and just use the OSPF/Loopback portion to do this. https...

IPANetEngineer

Technically you could use both with unequal load balancing if you use OSPF and the method outlined below. We've done this on a number of networks with a lot of success. It's certainly not the simplest way to go, but it's reliable and will use all the bandwidth available. https://stubarea51.net/2016/...

IPANetEngineer

The only solution is to NOT BUY this model is a scam. I wouldn't say that. we have a lot of clients that use it successfully and when it first came out, we were able to sustain 80 Gbps of iperf traffic without issue. I am curious about the config and conditions that are causing the reboots. From re...

IPANetEngineer

Are you trying to VPN over IPv6 and get IPv4 and IPv6 or over IPv4?

IPANetEngineer

What type of links are you connecting to the ports in the bridge? copper/fiber/RF? If i understood correctly the OP says that there are 2 Bondings inside the same Bridge. If those 2 Bondings inside the same Bridge are 802.3ad there is packet loss, when different there is no packet loss .. so why sh...

IPANetEngineer

If you want to test and fallback to the previous config you can always use safe mode. Alternatively or in conjunction with safe mode, you can put a /32 static route in that goes back to the Internet connection you're accessing from. That way it will always be the most specific route in the table for...

IPANetEngineer

We work on a lot of WISPs with MikroTik using the design you mentioned with MPLS/VPLS. I would not use the CRS hardware for anything other than Layer 2 in this type of network. The CPU is not designed for heavy Layer 3 workloads. I'd look at using the CCR family along with a stackable Layer 2 or 3 s...

IPANetEngineer

If IPv6 is enabled, you can connect using the link local address.

IPANetEngineer

You need to use 'limit-at' to guarantee bandwidth instead of 'max-limit' try that and see if you get different results.

IPANetEngineer

Awesome! Can't wait to try it

IPANetEngineer

How this needs to be done depends on how many routers you have. Is it just the one, or do you have routers at towers as well?

IPANetEngineer

I would also love to see ZeroTier implemented.

We use it for remote access to DCs as well as an encrypted transport path between DCs. It's one of my favorite SDWAN implementations.

IPANetEngineer

vpls one side is up and other end is down help please, i have checked LDP and MTU on both routers but still its the same and i m stuck Be sure that the transport address is set on the LDP interface or global LDP options. VPLS requires dynamic targeted LDP sessions which must have a consistent trans...

IPANetEngineer

What type of links are you connecting to the ports in the bridge? copper/fiber/RF?

IPANetEngineer

Can you be a little more specific?

Do you want to filter by prefix/length with a community and by prefix/length without a community?

IPANetEngineer

Here is a guide on HA PPPoE BRAS with MIkroTik that may be helpful.

https://www.stubarea51.net/2018/04/23/p ... atorsbras/

IPANetEngineer

CHR is super unstable product (at least on vmware). We use it only for "the dude" monitoring and 3 ppptp servers for managment. Total throuput is not even 10mbit/s And Even in that case, it locks up. Needs reboot every week. It looses its winbox port and ssh accesibility. We reboot it from the vm c...

IPANetEngineer

Are you trying to get PIM or IGMP working or both?

IPANetEngineer

Thanks for the update....glad you got it working.

Please mark this as solved

IPANetEngineer

If I read the OP correctly, he does not have control of the PPPoE server.

IPANetEngineer

Sometimes an onsite visit to see what issues people are having can make all the difference in the world.

Glad you figured it out

IPANetEngineer

Here are some example configs and a diagram that should point you in the right direction. A few notes on this - I tried to keep the config as simplistic as possible since you're dealing with very low speeds. There are more efficient ways to do this, but it requires more config. Also, I used 10M down...

IPANetEngineer

I hope so...it would be very useful!

IPANetEngineer

If you're coming from Cisco, this may also be helpful for bridge VLAN configuration in MIkroTik.

https://www.stubarea51.net/2019/02/06/c ... and-vlans/

IPANetEngineer

It definitely would be nice to reference a prefix list directly and MikroTik has stated they are working on it. However, from a CPU standpoint, Cisco and Juniper do the same thing mostly Cisco has a route-map that references a prefix list for BGP peers - which requires two different constructs Junip...

IPANetEngineer

Here is an overview of IPv6 in MikroTik and how it goes from the BGP edge to the last mile with a customer handoff....it might be helpful. The configs for the entire network are in the article. :D https://www.stubarea51.net/2018/09/14/wisp-design-an-overview-of-adding-ipv6-to-your-wisp/ https://www....

IPANetEngineer

Here is a performance comparison I did between ESXi, ProxMox and HyperV

https://www.youtube.com/watch?v=xcgdGA1W_0o

IPANetEngineer

I wouldn't celebrate yet, there's udp in v7, but it wasn't the only missing feature. So it's great step for own use, but not much changed for interoperability with someone else's service using standard OpenVPN. We've been able to interop with non-MikroTik OpenVPN linux builds. It takes a little wor...

IPANetEngineer

What version of RouterOS is running on each device?

IPANetEngineer

The route cache is set here

[admin@R1] > ip settings set route-cache=no

IPANetEngineer

Something is definitely off - we don't normally see high latency in the CHR like that.

Have you licensed it with a trial license? If not it's restricted to 1 Mbps throughput and that can def cause latency

IPANetEngineer

In the last couple of weeks, whenever I forward port tcp 443 I get internet issues. All my other port forwardings work fine, except this one. And it's disabled in IP SERVICES before someone asks :) Seems to be on the latest versions, because today I messed around with an 6.43.16 RB and it worked fi...

IPANetEngineer

The last rule appears to be an IPv6 ipsec issue.

Are you trying to terminate the tunnel on IPv4 or IPv6?

IPANetEngineer

Now that OpenVPN has UDP support in ROSv7, I expect we'll see a large migration to that once ROSv7 is prod and stable.

We've scaled OpenVPN to more than 100,000 clients with MikroTik for IoT solutions....it works really well

IPANetEngineer

I worked on an IPTV network recently using CCRs in bridge mode with IGMP Snooping and PIM Sparse and I don't think we ran into any issues. Maybe it was a problem in the older ROS version?

IPANetEngineer

The MIkroTik implementation of MPLS/LDP does not have fast reroute, so first OSPF timers must expire and then LDP timers have to also expire before a path is moved over. Sometimes this happens under a minute and sometimes it takes longer. In, general though, we've deployed a large number of MPLS bas...

IPANetEngineer

You'll probably need a bit of trial and error with this one but I think this is a fairly close translation into RouterOS from iptables

/ip firewall filter
add action=accept chain=input dst-port=1701 in-interface=ether1 protocol=udp src-port=500

IPANetEngineer

I've done this for a number of clients with Radius DHCP over VPLS except for the dynamic VLAN assignment which can be done one of two ways with radius: via dot1x or on wireless using CAPSMAN. If those don't work for you, I would probably use the API to set the VLAN based on the radius response. Also...

IPANetEngineer

What does the vswitch config look like in ESXI?

IPANetEngineer

This often depends on what the clients will be... Phones, Laptops, servers, firewalls?? You also need to consider the type of Internet connection the clients will be coming from - in some cases, TCP/443 behind NAT is all you are allowed to use. UDP based VPNs are great if they meet the requirements ...

IPANetEngineer

I would strongly recommend the use of a switch-centric design for the core as shown below for several reasons: - Scale - Operational flexibility - Ease of migration and adding capacity - Simplifies failover https://www.stubarea51.net/wp-content/uploads/2018/09/Switch-centric-IPA.png Then utilize Mi...

IPANetEngineer

Typically the issue you are describing indicates a problem with MTU somewhere.

What are your MTU settings for.

L2
L3
MPLS

?

IPANetEngineer

Also, if you use the following page, it will tell you what features are supported in 'hw-offload'

https://wiki.mikrotik.com/wiki/Manual:I ... Offloading

IPANetEngineer

Awesome....can you please change the thread to "solved"

IPANetEngineer

The option to do this is there, you just have to build more than one filter. First build the prefixes common to multiple peers /routing filter add action=accept chain=as-65000-prefixes prefix=203.0.113.0/29 prefix-length=29-32 add action=accept chain=as-65000-prefixes prefix=203.0.113.8/29 prefix-le...

IPANetEngineer

Here is an example of forwarding port 80 on two different IPs using 203.0.113.0/24 "public" network and the 192.168.0.0/24 private network. /ip firewall nat add chain=dstnat dst-address=203.0.113.11 dst-port=80 action=dst-nat to-addresses=192.168.0.11 to-port=80 /ip firewall nat add chain=dstnat dst...

IPANetEngineer

OSPF is very easy to setup MPLS with LDP.

Do you have a specific reason why static label distribution is required?

IPANetEngineer

If you're doing PPPoE, this means that you're working with /32 networking , i'd probably use OSPF to get started and then move to iBGP with OSPF advertising the transit and loopback subnets. You'll have to break OSPF into areas to summarize the users and OSPF is happiest in MIkroTik when you keep th...

IPANetEngineer

Can you share your config?

IPANetEngineer

Out of interest is there an expected release date of new revisions, e.g v7.0beta2 every couple weeks? Or would we only get updates every quarter? Thanksk There are no set release schedules for the next beta releases. We will release beta2 once were done fixing current bugs that were found in beta1 ...

IPANetEngineer

Although not specifically a MikroTik article, there are a lot of WISP operators in here and thought this might be helpful for some of you (and MikroTik is mentioned :wink: ) If you're struggling to figure out how to migrate a WISP from Bridged to Routed, here is a guide to help you get started. http...

IPANetEngineer

I would suggest running a long term packet capture only for IP Protcol 89 on each side of the link so that we can look at the packets as it's failing. You may need to let it run for a while so be sure to give it enough memory and only allow OSPF packets which are small. That's how I normally trouble...

IPANetEngineer

AWS gives you full control over the FW. To permit IP Protocol 50, you need this type of rule

IPANetEngineer

What you're trying to do is a little unclear...do you have a drawing of what you want to do or can you make one?

IPANetEngineer

Here is a good tutorial on how to open ports in AWS. And I do not agree that you should just open all ports. Unless you are an ISP or Hosting Data Center that has other security appliances deployed, You should only allow the ports that you need and deny the rest. AWS Has great security appliances th...

IPANetEngineer

You'll never get 10 Gbps through the CRS317 when routing L3 packets. It maxes out at 2 to 3 Gbps because L3 is handled by a CPU not an ASIC. https://mikrotik.com/product/crs317_1g_16s_rm#fndtn-testresults Here is a guide I did on MikroTik and VLANs to convert from Cisco that may be helpful (testing ...

IPANetEngineer

I literally finished setting this up myself this morning. Absolutely as above. You get a private LAN which is 1:1 NAT with a real public IP. You need NAT traversal and the key for me was IPSEC-ESP being open in the firewall. My CHR at home connects no problem as well as parents RB750 but I had to d...

IPANetEngineer

You don't need to use a public subnet between routers, you can use 10.x.x.x or 100.64.x.x ranges.

Then you can put a single IP on the Router that has the PCs using a new bridge as a loopback and NAT to the single IP. Just replace the /30 route with a /32 route.

IPANetEngineer

AWS Doesn't normally pass a true public to the guest VM so you need to make sure that you have NAT Traversal enabled for IPSEC. What are the settings you are using? Can you share your config?

IPANetEngineer

The CRS317 is intended for 10G L2 switching and not routing.

What kind of throughput do you need through the Hypervisors?

IPANetEngineer

I'm trying to configure OSPF for IPv6 to test recursive routing in v7 to route ipv6 you should use OSPFv3. as far as i see, you have set this under /routing ospf instance using the new 'version' attribute [admin@hgw] /routing/ospf/instance> print Flags: D - dynamic, X - disabled, I - inactive 0 ver...

IPANetEngineer

I'm trying to configure OSPF for IPv6 to test recursive routing in v7 Since this is a limited release, should it be working? Everything seems in order and I can ping on the /64 between routers, but there is no neighbor adjacency R1 - RB3011 /routing ospf instance add name=ospf-instance-1 router-id=1...

IPANetEngineer

Is it what you expect or what you're afraid of? Because it's like this by design: if you broke access 'forever', it will be rolled back. But generally ssh is quite tolerant to network instability. Well, it is certainly a weak point in the RouterOS "safe mode" that it immediately rolls back all chan...

IPANetEngineer

Recursive nexthops in v7 works without any scripts.

Woahhhhhhhhhhhh!!!!

+1000000000000

IPANetEngineer

I believe this problem was discussed here:

viewtopic.php?f=14&t=151650

The workaround was to run two OSPF instances since there is no way to change the link local

IPANetEngineer

IPANetEngineer

When you say that you need to "account" for it, what does that mean exactly?

Seems like you could use netflow for this purpose and then pull the data out of a netflow collector instead of adding the CPU overhead of marking every single packet

IPANetEngineer

We came up with a design to solve this problem for a client WISP and then published the solution which is here: https://www.stubarea51.net/2016/10/27/wisp-design-using-ospf-to-build-a-transit-fabric-over-unequal-links/ We took it a step further and paired it with BGP and presented the design at the ...

IPANetEngineer

I actually wrote an article to help people who have learned Layer 2 in Cisco migrate to MikroTik VLAN configuration.

https://www.stubarea51.net/2019/02/06/c ... and-vlans/

IPANetEngineer

Try using 'aggregate' instead of 'network' and check the include IGP option, that should fix the issue you're seeing.

IPANetEngineer

Any chance you can grab a packet capture from BGP / MPLS when the routes are going in and out? Might provide a clue as to what it's unhappy about

IPANetEngineer

Are you trying to create a Rendezvous Point for the headend feed from your upstream provider?

IPANetEngineer

Turn synchronization off and announce the prefixes you need. Unlike Cisco, MikroTik will announce whatever prefix length you specify in networks if synchronization is turned off. You don't need a static route for the BGP advertisement.

IPANetEngineer

I agree with pe1chi , if you're using your border routers as a stateful firewall for traffic to customers and BGP full tables, you need to redesign the way you are doing things and break out security devices into a separate box - independent of the border router.

IPANetEngineer

Prepending doesn't work very well these days so i'd choose another strategy. If your ISPs support communities and most large ISPs do, then you can set communities on your routes to either prioritize or deprioritize them via a specific peer. You can also split the prefixes up and advertise specific r...

IPANetEngineer

Why not put the subnet into a VPLS tunnel? Then it will exist as one L2 segment at each switch and you can have a routed network with failover underneath it. here is an example....you don't have to use S-Tag though, you could set it to untagged or a standard 802.1q tag https://www.stubarea51.net/201...

IPANetEngineer

OSPF is not going to help you when the network is bridged. You need to convert to independently routed subnets between the routers and at the towers. Here is an example of migrating from bridged to routed in a presentation I did last year Bridged https://www.stubarea51.net/wp-content/uploads/2019/08...

IPANetEngineer

Watch this podcast (which I am a guest on) and you can listen to us discuss the issues with extending Layer 2

https://thenetworkcollective.com/2017/0 ... nd-layer2/

IPANetEngineer

Extending Layer 2 is complicated and not recommended across countries for data centers. I strongly recommend against this.

Is there a specific reason why each datacenter can't have a routable subnet assigned to it?

IPANetEngineer

You have an MTU Mismatch and it appears to be inheriting it from the bridge. You need to get the MTUs to match even if you have to lower the whole segment MTU

192.168.30.15
actual-mtu=1380

10.14.100.81
actual-mtu=1458

Search found 1180 matches