MikroTik

IPANetEngineer

CHR is super unstable product (at least on vmware). We use it only for "the dude" monitoring and 3 ppptp servers for managment. Total throuput is not even 10mbit/s And Even in that case, it locks up. Needs reboot every week. It looses its winbox port and ssh accesibility. We reboot it from the vm c...

IPANetEngineer

Are you trying to get PIM or IGMP working or both?

IPANetEngineer

Thanks for the update....glad you got it working.

Please mark this as solved

IPANetEngineer

If I read the OP correctly, he does not have control of the PPPoE server.

IPANetEngineer

Sometimes an onsite visit to see what issues people are having can make all the difference in the world.

Glad you figured it out

IPANetEngineer

Here are some example configs and a diagram that should point you in the right direction. A few notes on this - I tried to keep the config as simplistic as possible since you're dealing with very low speeds. There are more efficient ways to do this, but it requires more config. Also, I used 10M down...

IPANetEngineer

I hope so...it would be very useful!

IPANetEngineer

If you're coming from Cisco, this may also be helpful for bridge VLAN configuration in MIkroTik.

https://www.stubarea51.net/2019/02/06/c ... and-vlans/

IPANetEngineer

It definitely would be nice to reference a prefix list directly and MikroTik has stated they are working on it. However, from a CPU standpoint, Cisco and Juniper do the same thing mostly Cisco has a route-map that references a prefix list for BGP peers - which requires two different constructs Junip...

IPANetEngineer

Here is an overview of IPv6 in MikroTik and how it goes from the BGP edge to the last mile with a customer handoff....it might be helpful. The configs for the entire network are in the article. :D https://www.stubarea51.net/2018/09/14/wisp-design-an-overview-of-adding-ipv6-to-your-wisp/ https://www....

IPANetEngineer

Here is a performance comparison I did between ESXi, ProxMox and HyperV

https://www.youtube.com/watch?v=xcgdGA1W_0o

IPANetEngineer

I wouldn't celebrate yet, there's udp in v7, but it wasn't the only missing feature. So it's great step for own use, but not much changed for interoperability with someone else's service using standard OpenVPN. We've been able to interop with non-MikroTik OpenVPN linux builds. It takes a little wor...

IPANetEngineer

What version of RouterOS is running on each device?

IPANetEngineer

The route cache is set here

[admin@R1] > ip settings set route-cache=no

IPANetEngineer

Something is definitely off - we don't normally see high latency in the CHR like that.

Have you licensed it with a trial license? If not it's restricted to 1 Mbps throughput and that can def cause latency

IPANetEngineer

In the last couple of weeks, whenever I forward port tcp 443 I get internet issues. All my other port forwardings work fine, except this one. And it's disabled in IP SERVICES before someone asks :) Seems to be on the latest versions, because today I messed around with an 6.43.16 RB and it worked fi...

IPANetEngineer

The last rule appears to be an IPv6 ipsec issue.

Are you trying to terminate the tunnel on IPv4 or IPv6?

IPANetEngineer

Now that OpenVPN has UDP support in ROSv7, I expect we'll see a large migration to that once ROSv7 is prod and stable.

We've scaled OpenVPN to more than 100,000 clients with MikroTik for IoT solutions....it works really well

IPANetEngineer

I worked on an IPTV network recently using CCRs in bridge mode with IGMP Snooping and PIM Sparse and I don't think we ran into any issues. Maybe it was a problem in the older ROS version?

IPANetEngineer

The MIkroTik implementation of MPLS/LDP does not have fast reroute, so first OSPF timers must expire and then LDP timers have to also expire before a path is moved over. Sometimes this happens under a minute and sometimes it takes longer. In, general though, we've deployed a large number of MPLS bas...

IPANetEngineer

You'll probably need a bit of trial and error with this one but I think this is a fairly close translation into RouterOS from iptables

/ip firewall filter
add action=accept chain=input dst-port=1701 in-interface=ether1 protocol=udp src-port=500

IPANetEngineer

I've done this for a number of clients with Radius DHCP over VPLS except for the dynamic VLAN assignment which can be done one of two ways with radius: via dot1x or on wireless using CAPSMAN. If those don't work for you, I would probably use the API to set the VLAN based on the radius response. Also...

IPANetEngineer

What does the vswitch config look like in ESXI?

IPANetEngineer

This often depends on what the clients will be... Phones, Laptops, servers, firewalls?? You also need to consider the type of Internet connection the clients will be coming from - in some cases, TCP/443 behind NAT is all you are allowed to use. UDP based VPNs are great if they meet the requirements ...

IPANetEngineer

I would strongly recommend the use of a switch-centric design for the core as shown below for several reasons: - Scale - Operational flexibility - Ease of migration and adding capacity - Simplifies failover https://www.stubarea51.net/wp-content/uploads/2018/09/Switch-centric-IPA.png Then utilize Mi...

IPANetEngineer

Typically the issue you are describing indicates a problem with MTU somewhere.

What are your MTU settings for.

L2
L3
MPLS

?

IPANetEngineer

Also, if you use the following page, it will tell you what features are supported in 'hw-offload'

https://wiki.mikrotik.com/wiki/Manual:I ... Offloading

IPANetEngineer

Awesome....can you please change the thread to "solved"

IPANetEngineer

The option to do this is there, you just have to build more than one filter. First build the prefixes common to multiple peers /routing filter add action=accept chain=as-65000-prefixes prefix=203.0.113.0/29 prefix-length=29-32 add action=accept chain=as-65000-prefixes prefix=203.0.113.8/29 prefix-le...

IPANetEngineer

Here is an example of forwarding port 80 on two different IPs using 203.0.113.0/24 "public" network and the 192.168.0.0/24 private network. /ip firewall nat add chain=dstnat dst-address=203.0.113.11 dst-port=80 action=dst-nat to-addresses=192.168.0.11 to-port=80 /ip firewall nat add chain=dstnat dst...

IPANetEngineer

OSPF is very easy to setup MPLS with LDP.

Do you have a specific reason why static label distribution is required?

IPANetEngineer

If you're doing PPPoE, this means that you're working with /32 networking , i'd probably use OSPF to get started and then move to iBGP with OSPF advertising the transit and loopback subnets. You'll have to break OSPF into areas to summarize the users and OSPF is happiest in MIkroTik when you keep th...

IPANetEngineer

Can you share your config?

IPANetEngineer

Out of interest is there an expected release date of new revisions, e.g v7.0beta2 every couple weeks? Or would we only get updates every quarter? Thanksk There are no set release schedules for the next beta releases. We will release beta2 once were done fixing current bugs that were found in beta1 ...

IPANetEngineer

Although not specifically a MikroTik article, there are a lot of WISP operators in here and thought this might be helpful for some of you (and MikroTik is mentioned :wink: ) If you're struggling to figure out how to migrate a WISP from Bridged to Routed, here is a guide to help you get started. http...

IPANetEngineer

I would suggest running a long term packet capture only for IP Protcol 89 on each side of the link so that we can look at the packets as it's failing. You may need to let it run for a while so be sure to give it enough memory and only allow OSPF packets which are small. That's how I normally trouble...

IPANetEngineer

AWS gives you full control over the FW. To permit IP Protocol 50, you need this type of rule

IPANetEngineer

What you're trying to do is a little unclear...do you have a drawing of what you want to do or can you make one?

IPANetEngineer

Here is a good tutorial on how to open ports in AWS. And I do not agree that you should just open all ports. Unless you are an ISP or Hosting Data Center that has other security appliances deployed, You should only allow the ports that you need and deny the rest. AWS Has great security appliances th...

IPANetEngineer

You'll never get 10 Gbps through the CRS317 when routing L3 packets. It maxes out at 2 to 3 Gbps because L3 is handled by a CPU not an ASIC. https://mikrotik.com/product/crs317_1g_16s_rm#fndtn-testresults Here is a guide I did on MikroTik and VLANs to convert from Cisco that may be helpful (testing ...

IPANetEngineer

I literally finished setting this up myself this morning. Absolutely as above. You get a private LAN which is 1:1 NAT with a real public IP. You need NAT traversal and the key for me was IPSEC-ESP being open in the firewall. My CHR at home connects no problem as well as parents RB750 but I had to d...

IPANetEngineer

You don't need to use a public subnet between routers, you can use 10.x.x.x or 100.64.x.x ranges.

Then you can put a single IP on the Router that has the PCs using a new bridge as a loopback and NAT to the single IP. Just replace the /30 route with a /32 route.

IPANetEngineer

AWS Doesn't normally pass a true public to the guest VM so you need to make sure that you have NAT Traversal enabled for IPSEC. What are the settings you are using? Can you share your config?

IPANetEngineer

The CRS317 is intended for 10G L2 switching and not routing.

What kind of throughput do you need through the Hypervisors?

IPANetEngineer

I'm trying to configure OSPF for IPv6 to test recursive routing in v7 to route ipv6 you should use OSPFv3. as far as i see, you have set this under /routing ospf instance using the new 'version' attribute [admin@hgw] /routing/ospf/instance> print Flags: D - dynamic, X - disabled, I - inactive 0 ver...

IPANetEngineer

I'm trying to configure OSPF for IPv6 to test recursive routing in v7 Since this is a limited release, should it be working? Everything seems in order and I can ping on the /64 between routers, but there is no neighbor adjacency R1 - RB3011 /routing ospf instance add name=ospf-instance-1 router-id=1...

IPANetEngineer

Is it what you expect or what you're afraid of? Because it's like this by design: if you broke access 'forever', it will be rolled back. But generally ssh is quite tolerant to network instability. Well, it is certainly a weak point in the RouterOS "safe mode" that it immediately rolls back all chan...

IPANetEngineer

Recursive nexthops in v7 works without any scripts.

Woahhhhhhhhhhhh!!!!

+1000000000000

IPANetEngineer

I believe this problem was discussed here:

viewtopic.php?f=14&t=151650

The workaround was to run two OSPF instances since there is no way to change the link local

IPANetEngineer

IPANetEngineer

When you say that you need to "account" for it, what does that mean exactly?

Seems like you could use netflow for this purpose and then pull the data out of a netflow collector instead of adding the CPU overhead of marking every single packet

IPANetEngineer

We came up with a design to solve this problem for a client WISP and then published the solution which is here: https://www.stubarea51.net/2016/10/27/wisp-design-using-ospf-to-build-a-transit-fabric-over-unequal-links/ We took it a step further and paired it with BGP and presented the design at the ...

IPANetEngineer

I actually wrote an article to help people who have learned Layer 2 in Cisco migrate to MikroTik VLAN configuration.

https://www.stubarea51.net/2019/02/06/c ... and-vlans/

IPANetEngineer

Try using 'aggregate' instead of 'network' and check the include IGP option, that should fix the issue you're seeing.

IPANetEngineer

Any chance you can grab a packet capture from BGP / MPLS when the routes are going in and out? Might provide a clue as to what it's unhappy about

IPANetEngineer

Are you trying to create a Rendezvous Point for the headend feed from your upstream provider?

IPANetEngineer

Turn synchronization off and announce the prefixes you need. Unlike Cisco, MikroTik will announce whatever prefix length you specify in networks if synchronization is turned off. You don't need a static route for the BGP advertisement.

IPANetEngineer

I agree with pe1chi , if you're using your border routers as a stateful firewall for traffic to customers and BGP full tables, you need to redesign the way you are doing things and break out security devices into a separate box - independent of the border router.

IPANetEngineer

Prepending doesn't work very well these days so i'd choose another strategy. If your ISPs support communities and most large ISPs do, then you can set communities on your routes to either prioritize or deprioritize them via a specific peer. You can also split the prefixes up and advertise specific r...

IPANetEngineer

Why not put the subnet into a VPLS tunnel? Then it will exist as one L2 segment at each switch and you can have a routed network with failover underneath it. here is an example....you don't have to use S-Tag though, you could set it to untagged or a standard 802.1q tag https://www.stubarea51.net/201...

IPANetEngineer

OSPF is not going to help you when the network is bridged. You need to convert to independently routed subnets between the routers and at the towers. Here is an example of migrating from bridged to routed in a presentation I did last year Bridged https://www.stubarea51.net/wp-content/uploads/2019/08...

IPANetEngineer

Watch this podcast (which I am a guest on) and you can listen to us discuss the issues with extending Layer 2

https://thenetworkcollective.com/2017/0 ... nd-layer2/

IPANetEngineer

Extending Layer 2 is complicated and not recommended across countries for data centers. I strongly recommend against this.

Is there a specific reason why each datacenter can't have a routable subnet assigned to it?

IPANetEngineer

You have an MTU Mismatch and it appears to be inheriting it from the bridge. You need to get the MTUs to match even if you have to lower the whole segment MTU

192.168.30.15
actual-mtu=1380

10.14.100.81
actual-mtu=1458

IPANetEngineer

What does your MPLS config look like?

IPANetEngineer

Route lookups and routing table convergence is significantly faster when using the CHR as a border router.

Here is a presentation I did at MUM Europe in 2018 on that topic.
https://www.youtube.com/watch?v=xcgdGA1W_0o

IPANetEngineer

OSPF, by design will learn and advertise all subnets that it learns to every other router in the area.

What problem are you trying to solve by limiting the subnets that are advertised?

IPANetEngineer

What is the TTL set for on the multicast source? If it's too low, it won't work. If you can't see or set it on the multicast source, then do a packet capture at the PIM RP and see what it's set to.

Run into this issue more than once.

IPANetEngineer

Be sure that you are explicitly defining LDP interfaces and setting the transport to the loopback address. In your configs, it doesn't appear to be set.

VPLS requires targeted LDP sessions and needs the transport IP explicitly set.

IPANetEngineer

Are you saying the interfaces are physically up but show as down in OSPF passive?

IPANetEngineer

I'd use a 4011...compact and 10 gig capable if needed. It's a fantastic tower router and can even be a border router.

IPANetEngineer

Also, in the world of ever increasing security threats, IS-IS runs at Layer 2 and not Layer 3 to form IGP adjacencies, so it is much harder to DDoS the control plane when it doesn't use L3.

IPANetEngineer

The other reason for this would be for ECMP as MikroTik does not implement ECMP for LDP.

IPANetEngineer

The backbone area is a fundamental part of OSPF.

What kind of network is this? ISP, Data Center?

IPANetEngineer

Normis...i'm pretty confident we have replicated the conditions of one of the CVEs from doing some digging on our own for this issue. Without the rules, the router crashed. When we added the rules the router stayed online. Meanwhile CVE-2018-19299 still needs fixing, because even with those perform...

IPANetEngineer

We believe that we have now recreated the conditions of both CVEs and have been able to cause a memory leak and router crash in both of the conditions listed below using software from a common offensive linux security tool for IPv6. soft lockup when forwarding IPv6 packets (CVE-2018-19299); soft loc...

IPANetEngineer

Until we release the next beta with memory exhaustion fix, this firewall config should stop any attack even with small amount of RAM: admin@MikroTik] /ipv6 firewall> export /ipv6 firewall filter add action=drop chain=forward connection-mark=drop connection-state=new /ipv6 firewall mangle add action...

IPANetEngineer

Would somebody please post some additional information about this. I need to understand what is the problem, the potential impact and what vulnerabilities are possible. Where can I find information to read/learn about this? I am not aware of any workarounds or mitigations any of us can use. I belie...

IPANetEngineer

That's normal standard from Mikrotik when they are faced with a problem to resolve That is not true at all! We have always reacted to issues quickly, all the previous vulnerabilities have been fixed within hours or days time. Even in this case, we did reproduce and acknowledge the issue. In this ca...

IPANetEngineer

Even if there is no way to firewall it on a MikroTik, i'm assuming that once we know what is being set in the packet header, it can be mitigated with another solution based on flow detection and dropping the traffic in a switch. That won't work for everyone obviously, but it would work for a lot of ...

IPANetEngineer

This is also a new one for me...will be digging into it

IPANetEngineer

AS Path prepending is not a technology used for outbound route selection (it controls inbound) and it has limited use in the BGP Global Table these days due to provider traffic engineering with localpref overriding it. My question before commenting would be what are you trying to achieve? Equal load...

IPANetEngineer

Hyper-V is hands down the best hypervisor for using a CHR as a BGP edge router. Mostly this is because MikroTIk spent a lot of time building the Hyper-V drivers for the CHR and they used off the shelf drivers for KVM/ESXi The single biggest impact is to get a CPU with a higher clock speed and fewer ...

IPANetEngineer

This MikroTik to Cisco article has been on my to-do list for a while. If you know Cisco and want to understand how to work with VLANs in MikroTik's CRS3xx series of switches, here is a guide to get you started. https://www.stubarea51.net/2019/02/06/cisco-to-mikrotik-switching-and-vlans/ https://www....

IPANetEngineer

If you want absolute control over your traffic paths, eBGP will work well, otherwise OSPF will be better than static routes but is more limited in its ability to manipulate traffic. Here is a presentation I did at the US MUM in 2017 which covers a lot of the questions you have in the first half. htt...

IPANetEngineer

Nothing weird about this. R2 is going to pick the best route for prefix 3.3.3.0/24 to put into the FIB and it has two choices. 1) An iBGP route with an admin distance of 200 2) An OSPF intra-area route with an admin distance of 110 It's going to pick the OSPF route which means there will be no activ...

IPANetEngineer

What are your MTU settings for Layer 2 and Layer 3 on each side of the link OSPF is trying to form a neighbor on? Normally getting stuck in two way indicates MTU, have also seen it occur as the result of a network type mismatch. What is the config for the other router? Here is a presentation I did o...

IPANetEngineer

The VPLS MTU is the size of the MTU you want to hand off to your customer. The interface MTU should be set to accommodate the overhead of VPLS. You need at least 1530 to send an 802.1q tagged frame through a VPLS tunnel. https://wiki.mikrotik.com/images/3/35/MTUVPLS.png Take a look at this MUM prese...

IPANetEngineer

Can you post your configuration?

IPANetEngineer

Take a look at this if you want an example of a production deployment with HA for VPLS and public subnets. Configs are in this post - https://www.stubarea51.net/2018/04/23/wisp-design-building-highly-available-vpls-for-public-subnets/ https://dev.stubarea51.net/wp-content/uploads/2018/08/vpls-1.png

IPANetEngineer

Thanks for doing the testing MIke! I'm looking forward to putting a 4011 in our lab and benchmarking it against a hardware router.

I'm excited about where MikroTik is headed with more ARM based routers

IPANetEngineer

Can you post your config minus sensitive information?

export compact hide-sensitive

IPANetEngineer

This would be really nice to have

IPANetEngineer

To clarify what you are trying to do here... Are you trying to advertise a summary route out of the stub area? If so, the area range command is what you're looking for. /routing ospf area range add advertise=yes area=backbone cost=default disabled=no range=192.168.88.0/24 add advertise=yes area=area...

IPANetEngineer

I would consider a design like this with multiple CCRs so that you have redundancy and the ability to add resources. This can also be done in a CHR very well. We've deployed this for clients with a lot of success. https://www.stubarea51.net/2018/04/23/pppoe-high-availability-design-incorporating-mul...

IPANetEngineer

Thanks! Just tried it on a CRS317-1G-16S+ and it worked perfectly for the RouterOS and Firmware upgrades

IPANetEngineer

It might be more helpful to understand why you want to filter OSPF?

IPANetEngineer

We've worked with a number of clients that have had compromised routers. As others have suggested, the two best things you can possibly do are

1) Netinstall
2) Restore config from text

When we have done this, we have not seen any further issues with the routers

IPANetEngineer

Can you post the output of the MPLS forwarding tables for the Juniper and MIkroTIk routers?

IPANetEngineer

Many many thanks for your great help and idea! It did exactly what I wanted!

Perfect, glad I could help...just update the status of the thread to 'Solved'

IPANetEngineer

No problem! OSPF can be very complicated and takes a while to learn :-) To makes things much easier for yourself, here is what I would do. 1. Put all subnets that connect routers together into the backbone area aka transit links like your PPTP interfaces/subnets 2. Put all subnets that aren't involv...

IPANetEngineer

Where???? I don't see it

IPANetEngineer

At first glance, the main issue is with your design. If you look at the MIkroTik Wiki on OSPF area design, you'll see that multiple areas are all attached to the backbone area.But not Area 1 going to Area 2 https://wiki.mikrotik.com/images/c/cf/Image6006.gif OSPF is not intended for areas other than...

IPANetEngineer

Just wrote an article on how to add IPv6 to your WISP using MIkroTik, complete with configs. It covers adding IPv6 at the Core, the Tower and the subscriber as well as a subscriber device. Hope this is helpful for someone! https://www.stubarea51.net/2018/09/14/wisp-design-an-overview-of-adding-ipv6-...

IPANetEngineer

There are a lot of stable MIkroTik networks with BGP, OSPF, MPLS/VPLS on the CCR series.

Can you post your config? Do you have a network diagram?

IPANetEngineer

Can you post your config? That might be helpful.

Also please post the output of these commands on each router:

routing ospf interface print detail
routing ospf network print detail
routing ospf lsa print detail

IPANetEngineer

You definitely don't want to do it with a small CRS. Look at using a 3011 at smaller sites and CCR at others.

IPANetEngineer

Thanks! will test it in our MirkoTik lab

IPANetEngineer

On the BGP routers, turn on bgp redistribution then add only the wanted routes into ospf-out filter denying the others, so it will redistribute only those? Done this? I fear as the BGP process is single threaded and eats a whole cpu of the CCR it may cause problems. You have to get into hundreds of...

IPANetEngineer

Thanks. On the MTU size, I see some people set it to 1530 for MPLS, some 1580, 1600, and 2000. Is there any downside to setting it to 2000 across the board? Also I am having issues getting the MPLS working out in the field through the various wireless links even though on my lab it works fine. I ha...

IPANetEngineer

How many routes do you want to put in OSPF?

IPANetEngineer

Here's an example of OSPF/MPLS/VPLS for a WISP with HA DCs and with configs....I'll see what I can dig up for BGP

https://www.stubarea51.net/2018/04/23/w ... c-subnets/

IPANetEngineer

This depends on your use case. Some ISPs may use LDP signalled VPLS for private transport circuits. Other ISPs may use BGP signalled VPLS. In most cases, it's helpful to have iBGP to advertise public subjects and /32 loopbacks even if the majority of traffic is in VPLS. In short, having BGP on the i...

IPANetEngineer

I've been wanting to see this as well, but i'd rather have recursive routing in IPv6 for BGP fixed first.

IPANetEngineer

What model and type of routers are you using?

IPANetEngineer

I would consider a design like this and use local pref to prefer the IX routes.

IPANetEngineer

Everyone's use case is different, but I'm actually happy they stripped some things out. I look at this a different way - now you have a router capable of routing 10 Gbps peak throughput which is very close to CCR1009 number for half the cost. All of the bells and whistles are nice, I agree, but i'll...

IPANetEngineer

This largely depends on your BGP edge design and whether or not the peers are fully meshed inside your AS. If the routes learned from your upstreams are only present at the border routers then you'll need aggregate routes injected form the border routers to draw traffic towards one of those peers. A...

IPANetEngineer

The may be some options using policy routing. Can you post a diagram of what you have? It would be much easier to comment with some context as to the layout.

IPANetEngineer

If you run it on a switch, you can ask your upstream provider if you can maintain dual peerings and that way you'll have edge router redundancy if you lose a router or need to upgrade the RouterOS code, it can be done without an outage by failing traffic from one to the other. It still doesn't help ...

IPANetEngineer

I've done a lot of Enterprise networks and there are some key things when you're trying to migrate and the network is a mess. 1) If you don't have detailed documentation of how the current network is laid out, take the time to create it. Layer 1 - Document physical connections to all network equipme...

IPANetEngineer

We've been incredibly successful with switch-centric designs over the years and have deployed it on every continent except Antarctica. Switch stacks are not a single point of failure as they form a pair of HA switches that are logically a single switch from sa spanning tree perspective. Using a swit...

IPANetEngineer

We work on this type of design frequently. I would suggest a switch-centric architecture where all of the links terminate in the switch stack and you use LACP to connect the MikroTik routers and hypervisors. Then connect the internet circuits on different switches in the stack for redundancy. Switch...

IPANetEngineer

This is what you're looking for...hope it helps!

/routing bgp instance
set default as=200
/routing bgp network
add network=10.1.1.0/24
/routing bgp peer
add name=peer1 out-filter=PREPEND remote-address=192.168.1.5 remote-as=100
/routing filter
add action=accept chain=PREPEND set-bgp-prepend=3

IPANetEngineer

I'm actually interested to test this router with a full BGP table given the high clock speed and 10 gig port.

Who knows? Could be a diamond in the rough for a border router

IPANetEngineer

I would like to see TACACS+ support as well. Being able to restrict the commands that a user can execute is incredibly important. Especially with all of the attacks against MikroTik devices - it provides another layer of protection in addition to the firewall if a lower level user account is comprom...

IPANetEngineer

That's interesting, I haven't heard of this behavior yet, but will certainly look for it.

IPANetEngineer

Don't use the edge-port type for 802.1q trunking to another switch. Also you might try MSTP as it can sometimes solve STP interop issues with other switches. Setup 1 This works great. The VLAN is accessible on the second CSS326-24G-2S+RM 1GbE. VLAN1 : (meraki switch) over 1GbE -> (1)CSS326-24G-2S+RM...

IPANetEngineer

@IPANetEngineer If it would be important for them, they would have fixed this issue years ago. Just proceed with FRRouting :-) It's better anyways. Depends on your use case. I like FRR and talk to a number of the developers at FRR on a regular basis. However, it's still software that's go to go on ...

IPANetEngineer

We could really use an update on this MikroTik.

We are seeing IPv6 adoption move at a much faster pace in 2018 and are having to modify the routing architecture or use other brand routers for our clients to solve this problem.

This has been an issue for a long time but we could really use a fix

IPANetEngineer

Short answer is no, it's not possible...please see my reply in this thread.

viewtopic.php?f=14&t=138551

IPANetEngineer

In MikroTik, you cannot duplicate the transit IP or subnet inside a VRF the same way you can in Cisco.

Duplicated prefixes inside of a BGP route are fine, but the subnets used for peering or an IGP must be discrete and separate.

This is a limitation of the 6.x kernel from what I've been told.

IPANetEngineer

You're close but you need to add the WAN interface to the VRF interfaces as well.

Also remove the routing-table=INTERNET on the NAT rule and just match on routing mark

IPANetEngineer

Start a continuous ping to the peer address...do you see packet loss or bouncing of the peer? Also start a packet capture on that peering and filter for BGP, that way you can review the BGP updates and messages between your router and the upstream peer to see if your router is signalling a withdraw ...

IPANetEngineer

+1 for this feature

IPANetEngineer

One of the best ways is to use a public route server. From there you can see what your prefixes look like advertised into the BGP global table

http://routeserver.org/

Also, BGPLay allows you to visualize subnets and connected ASes

https://stat.ripe.net/special/bgplay

IPANetEngineer

Have you tired setting explicit null?

In the topology you have it looks like the PHP routers would be MikroTik and Juniper. We've seen issues before when the PHP routers are different vendors.

IPANetEngineer

Just a tip, MikroTik recommends using src-nat instead of masquerade when possible as the performance is much better than masquerade.

IPANetEngineer

*) bridge - added support for DHCP Option 82 (disables hardware offloading, CLI only); *) bridge - added support for DHCP Snooping (disables hardware offloading, CLI only); Could we please get some examples of how to use these features on the Wiki ? I cannot see any of the options I would expect, e...

IPANetEngineer

I think Metarouter running something else than RouterOS is a long abandoned concept...

Unfortunately I think yo're right...we all got so excited when CCR came out that it could be an inexpensive hypervisor but I don't think it will happen.

IPANetEngineer

Feature requests - SNMP OID Ethernet link speed It would be great to have SNMP OIDs for Ethernet link speeds. (if they are there , I have not spotted them yet). These could be very useful to detect when an Ethernet link changes link speed. Such as when what is/was supposed to be a 1-Gig link change...

IPANetEngineer

So we need to understand whether you're trying to influence traffic coming in from your upstream (normally a "download" for a user) or traffic from your network going outbound (normally an "upload" for a user)

There are different ways to influence BGP depending on what you're trying to do.

IPANetEngineer

So is VLAN 959 tagged towards the MikroTik router from the ISP Cisco Distribution Switch or not?

Can you validate tagged or untagged by using the MikroTik to perform a packet capture using /tool sniffer?

IPANetEngineer

If you want a single public IP on each tower, the easiest thing to do is create a loopback using a public IP and then advertise it in OSPF. That way you can keep using non-public IPs for the subnets that connect the towers. Even though this presentation I did is on OSPF troubleshooting, you can see ...

IPANetEngineer

What's your CPU load look like? CHR does a pretty good job of terminating PPPoE...you might consider a design like this for HA and scaling. http://www.stubarea51.net/2018/04/23/pppoe-high-availability-design-incorporating-multiple-access-concentratorsbras/ http://www.stubarea51.net/wp-content/upload...

IPANetEngineer

As a start take a look at this: http://www.stubarea51.net/2018/08/07/mikrotik-isp-design-building-an-802-1q-trunk-between-sites-using-vpls-and-s-tag/ Also if you need professional help, we can certainly assist as we design, build and troubleshoot ISP networks all around the world. Visit iparchitechs...

IPANetEngineer

My latest blog post on MikroTik design...thought i'd share

http://www.stubarea51.net/2018/08/07/mi ... and-s-tag/

IPANetEngineer

Thanks for the update!

IPANetEngineer

If you're trying to influence traffic out of an AS (which is what you appear to be doing) then I would start with marking localpref higher on the route you want to be preferred. localpref, unlike weight will be learned by every router in the AS (unlike weight)

IPANetEngineer

If you can go into more detail on what "failover" is needed exactly, we can probably give you a more accurate answer. Are you trying to use the same public subnet at two different locations? This article I wrote may be helpful: http://www.stubarea51.net/2018/04/23/wisp-design-building-highly-availab...

IPANetEngineer

Doing MPLS/VPLS and VXLAN in Hardware on switches is a HUGE opportunity for Mikrotik to carve out a niche. I know if this was supported, I wouldn't be buying 1 or 2 switches per year, it would be hundreds if not thousands of switches per year. Couldn't agree with you more...nobody has VXLAN in anyt...

IPANetEngineer

Currently Hyper-V performs way better for CHR than either ESXi or ProxMox

We did extensive performance testing on CHR in our lab and the results are in this presentation from Berlin, Germany at MUM Europe 2018

https://www.youtube.com/watch?time_cont ... cgdGA1W_0o

IPANetEngineer

OSPF is incredibly resilient to routing loops unless you're redistributing prefixes and have a config or design issue. It is the backbone of almost every network we work on for MikroTik - whether by itself or as the transport for iBGP and MPLS/LDP. It sounds like you may have a config or other issue...

IPANetEngineer

I moved my lab to Hyper-V Core 2012 R2, and can confirm that MPLS runs fine on that. That's great info...we did a bunch of CHR testing on different hypervisors for BGP and presented the results in Berlin at MUM Europe 2018. Hyper-V was way better than ESXi and ProxMox (KVM) by a significant margin.

IPANetEngineer

IS-IS can scale much larger than OSPF due to the way it designs the hierarchy of flooding domains and by using Incremental SPF. This is why it's used as the IGP of choice for most large ISPs and Data Centers I have an ISP customer with around 200 POP's and OSPF scalability is a real problem. We hav...

IPANetEngineer

No problem...glad you connected with someone. A large part of the IPA sales team has been travelling for the European, US and Mexico MUMs so that could be why you had an issue via phone - although it still shouldn't happen and I'll pass it along to the sales team :-) There are two issues that plague...

IPANetEngineer

Unfortunately IPv6 routing recursion has been broken for a while and is a limitation of the kernel being used.

viewtopic.php?f=14&t=42268&hilit=ipv6+recursive

IPANetEngineer

This is what BGP is designed for. OSPF doesn't manage traffic very well when you're trying to take a path that isn't the "shortest" While this design may not be exactly what you need, it will give you some ideas on the limitations of OSPF and how you can use BGP communities to set up traffic enginee...

IPANetEngineer

Its just sooooo coooooooooool protocol... I'd really like to know where hell I can use it in real life, so please tell the truth :) So to say, I have neither ISPs to establish ISIS with, nor software/hardware within the LAN to use it internally. But the proto is nice, really. IS-IS can scale much l...

IPANetEngineer

Since you're using VPLS already, I would probably use MPLS TE to define paths for traffic rather than OSPF cost. OSPF cost doesn't scale well and can create suboptimal traffic scenarios. This is assuming that traffic engineering is your most important requirement. I normally recommend the bugfix ver...

IPANetEngineer

There is a project on Github that worked on this concept (link below) and there are a number of examples of config synch scripts out there.

https://github.com/svlsResearch/ha-mikrotik

IPANetEngineer

I don't think it will ever show up in v6 but we may see it in v7 whenever that comes out.

IPANetEngineer

without knowing all of your requirements, I would probably recommend using something like a CCR1036-8G-2S+ or CCR1072 in a ring topology and configure OSPF for routing and possibly MPLS if you have a need for Layer2 VLANs to go between sites.

IPANetEngineer

Any updates on this MikroTik? I've been holding off on deploying CHR for MPLS because of this and would love to see this fixed.

IPANetEngineer

Your route reflectors should also be peered with each other (but not reflected) is that the way you have it setup? Even if you don't set the cluster-id or set a different cluster id, you're still going to see one route active and one inactive because BGP in MIkroTik doesn't support ECMP. There is a ...

IPANetEngineer

Your BGP peer remote-addresses are wrong. These should be the peer's address on the /30 links.

Not true...in an iBGP design you want the peering address to use loopbacks that are advertised by OSPF so that if an interface goes down and another path is available, the peering will stay online.

IPANetEngineer

Here are a couple of presentations I've done at different MUMs that may help you out with design for this type of network. Hope this helps! BGP as an IGP for Carrier/Enterprise Networks https://mum.mikrotik.com//presentations/US13/kevin.pdf ISP Architecture – MPLS Overview, Design and Implementation...

IPANetEngineer

It can be changed if made a filter to discard half of ISP1 prefixes? If you do that, and ISP2 goes down, then any IP within those prefixes not being advertised to ISP1 will not have Internet connectivity. No, here I mean the route prefixes that I receive from ISP1 and not my advertised networks pre...

IPANetEngineer

Has anyone got experience on hiring people online to write/edit/support a Mikrotik config? We have a single office router that needs to load share and prioritise traffic over three adsl links with the usual firewall protection and a couple of pinholes for SSH and VPN We have enough in house experti...

IPANetEngineer

+1 to add IS-IS to Router OS. I think we would be able to to build larger IGP flooding domains with IS-IS due to features like incremental SPF - especially since the Tilera processor doesn't do as well under a heavy computational load like what we have seen in large BGP table sizes and slow converge...

IPANetEngineer

Increase local preference for prefixes you get from ISP2 thanks for your replay, sorry but I don't have experience on tuning BGP I made a filter accept role with BGP Local Pref = 200 And I applied to peer ISP2(less used) in-filter. That bgp-peers prefixes now have 200 local pref value. But I don't ...

IPANetEngineer

This is great work! thanks for posting

IPANetEngineer

Will be in Berlin in April exhibiting and presenting with IP ArchiTechs...can't wait to see everyone. Haven't been back to MUM Europe since Slovenia

IPANetEngineer

Wrote another post in a series I started on Cisco to MikroTik command translation to make it easier for engineers that know Cisco to get into MikroTik.

http://www.stubarea51.net/2018/01/05/ci ... tion-ospf/

IPANetEngineer

Hello, I have a network bridget together. It looks like on the attached image (don't pay attention to the addresses at this time). Should I change it into a routed network? For example a RIP routing with addresses like in the image? Or should i go into a OSPF routing? https://thumb.ibb.co/dtwOpw/ne...

IPANetEngineer

Is the L3 transport over a connection that you control or is it a private circuit?

IPANetEngineer

I have the CRS setup as a P router in the middle of two CCR1036 routers acting as PE routers. It only shows one path as hardware offloaded [admin@MikroTik] > mpls forwarding-table print Flags: H - hw-offload, L - ldp, V - vpls, T - traffic-eng # IN-LABEL OUT-LABELS DESTINATION INTERFACE NEXTHOP 0 ex...

IPANetEngineer

Have you tried setting the next hop to the link local address instead of the global unicast?

IPANetEngineer

Dear Santa Claus,

I would like RouterOS v7 for Christmas
of which year?

2017...I'm hoping for a Christmas Miracle

IPANetEngineer

I've been trying to test MPLS hw offload in my lab based on the following entry in the release notes *) crs317 - added initial support for HW offloaded MPLS forwarding; I've built an MPLS network using two CCR1036 routers connected at 10 gig with the CRS317 in the middle (6.41rc47). When i perform a...

IPANetEngineer

Try disabling the gateway check on the static default route

IPANetEngineer

You have a static route set for the gateway of 2a01:4a0:4a::1/128 istead of using the directly connected route. That forces the MT to use routing recursion which is not yet supported in RouterOS for IPv6. Try removing the static route and see if the defualt route for global unicast goes active.

IPANetEngineer

For some reason, the route you have for 2000::/3 isn't active. Is the next hop reachable?

IPANetEngineer

Dear Santa Claus,

I would like RouterOS v7 for Christmas

IPANetEngineer

There have been a few attempts to work around this before. Here is one of them:

viewtopic.php?t=111215

IPANetEngineer

I would consider a packet scrubber in between the upstream peer and the border router. This is what they are designed for.

We've used these successfully with both CCRs and CHRs to defend against DDoS attacks and use RTBH

http://www.serveru.us/en/

IPANetEngineer

I don't think MRZ is suggesting a loopback per subscriber but rather a pair of loopbacks so that you can set different policy for the path of the VPLS tunnel.

IPANetEngineer

what is the output of /ipv6 route print ?

IPANetEngineer

Will be interesting to see what comes of this. I've had the exact same issue of having to clear UDP/5060 sessions manually when there is a failover or outage. I've not put much time into troubleshooting as it doesn't happen very often, but it seems to be the same issue everyone else is having.

IPANetEngineer

The CPU of the CRS is very underpowered and struggles to reach anything close to a gig. This is a very common complaint and scenario on the CRS. You really need an RB3011, 100AHx2 or 4 or CCR1009 for the routing part.

IPANetEngineer

SSTP works pretty well as a VPN that branches can "dial" back into a central location. Benefits are: 1) AES256 encryption - can be cert based 2) Uses TCP/443 so having ports blocked for VPN is rarely an issue 3) Is a native protocol on Microsoft operating systems and can be setup very easily in wind...

IPANetEngineer

There are some issues with MPLS throughput in the CHR. I don't believe MikroTik has fixed this yet

viewtopic.php?t=122446

IPANetEngineer

It is always a best practice to use the same routeros version on each end of the tunnel.

Have you tried matching the ros version and retesting?

IPANetEngineer

Two questions: 1) Have you run an RFC2544 test on the base RF link without MPLS to see if it passes? 2) What are your MTU settings for: L2MTU, L3 MTU and MPLS MTU on the MikroTik routers? This is a presentation I did for WISPs that want to use MPLS at the 2016 US MUM https://mum.mikrotik.com//presen...

IPANetEngineer

any update on this MikroTIk?

IPANetEngineer

So with the 6.41rc releases Mikrotik have added "MPLS Forwarding" to the CRS317. In my opinion just being able to forward MPLS packets based on the label is pointless. It wont make me buy the CRS317 over any other plain Layer2 switch. The MPLS features that would make me buy a CRS317 over another d...

IPANetEngineer

Definitely modular PS units. In most cases, we will do all AC or all DC.

IPANetEngineer

Great troubleshooting work guys! I'm anxious to see the results of this fix as we have been planning to use CHR for a number of MPLS applications.

IPANetEngineer

Which VNIC are you guys using in VMWARE. VMXNET3 or something else?

IPANetEngineer

Nice work and thanks for the info. We've been doing more and more with BFD in MikroTik now that it's been patched.

Search found 1044 matches