MPLS L3 VPN CCR 2216 as PE

RouterOS Forwarding Protocols
1

Hello MikroTik community,

I am building an MPLS L3VPN network for a small ISP and would love to hear from operators who have deployed CCR2216 as a PE router in production. I have already purchased some units and am planning to use them in our Points of Presence (POPs) to terminate customer VRFs.

Our network context:

  • Total traffic: <5 Gbps (Internet handled separately by Cisco ASR)

  • ~300 customers distributed across main city and regional sites

  • Target: ~10 VRFs per POP, each VRF for a corporate/enterprise customer

  • CE–PE routing: static or light eBGP per VRF

  • MP-BGP VPNv4 between PEs, centralized Route Reflector

  • OSPF + LDP in the MPLS core

  • CCR2216 used only as PE — not for Internet, CGNAT or PPPoE

Questions I would like your feedback on:

  1. Stability of MPLS L3VPN on CCR2216 in production — Have you seen LDP/IGP desync, VRF blackholes, or label table corruption in long-running deployments? How frequent and how did you recover?

  2. L3HW offload with MPLS active — Most reports suggest keeping L3HW OFF when MPLS is running. Have any of you found a stable way to run both simultaneously (e.g., with suppress-hw-offload per route)?

  3. RouterOS version — Which ROS v7.x build are you running on your MPLS PE boxes and which ones should be avoided? (I have read about VPNv4 freeze bugs in 7.13 and BGP session wipeouts in 7.16–7.18.)

  4. VRF and MP-BGP scale — How many VRFs and VPNv4 prefixes are you running per CCR2216 PE without issues? We plan ~10 VRFs with 10–30 prefixes each.

  5. CE–PE routing — Are you using static routes, eBGP, or OSPF between CE and PE inside VRFs? Any tips for avoiding the CE–PE BGP session "not on shared network" error?

  6. Mixed vendor environment — Has anyone successfully run CCR2216 PEs peering with Cisco ASR or Juniper MX as a Route Reflector or hub PE? Any interoperability gotchas?

  7. CPU core monitoring — We understand MPLS decapsulation is pinned to a single CPU core. What monitoring tools or thresholds do you use to detect when cpu0 is approaching saturation?

  8. Would you recommend CCR2216 as PE at our scale, or do you regret it? — Honest opinions welcome. If you switched to another platform (Juniper MX, Nokia SR, Cisco ASR), what was the trigger?

Any configuration snippets, design tips or cautionary tales are very welcome. We are still in the pre-deployment lab phase so there is time to adjust our design.

Thank you in advance for sharing your experience. This community has already helped us avoid several pitfalls.

2

For eBGP you need to set multihop=yes if session is not on connected network.

3

Any further inputs please need your advice on this point guys

4

Hello,

Your request seems very specific. To give you more details about our use cases, we have a number of MikroTik CCR2004 devices; it is not the model you are planning to deploy, but I think it could help you with your planning. Our network operates at less than 1 Gbps (95th percentile). We primarily use L2VPNs (previously with VPLS and now with VXLAN, as we found it offered better throughput), as well as a few VRFs for certain clients, but that remains marginal.

Here are some of the behaviors we have observed:

  1. We haven't seen any LDP or IGP desynchronization. Most of the time, it works. The same goes for label corruption: nothing triggers an alert.
  2. You may have encountered bugs regarding this. We do not yet rely on hardware offloading. Sometimes it works, sometimes it doesn't. We have given up on it.
  3. We use ROS 7.20+. We have noticed issues with certain route reflectors for VPNv4. I still don't quite understand why certain routes are not redistributed or, even worse, why redistribution might not work at all. To bypass this, we use a full mesh with PE routers.
  4. We only have 2 VRFs, so I cannot explain how this would scale to a larger size.
  5. Static and BGP works perfectly.
  6. Peering with Juniper MX and ACX works well. We did some experiments in the past and we found nothing that could block us.
  7. Actually, for us, it’s not yet a problem. We do not have enough traffic to speak of.
  8. At your scale, the main advice I could give is to try. At our scale, we are satisfied with it, but the more we grow, the more we see problems coming. I would say it depends on the bandwidth you sell to your customers. If you plan to rapidly exceed 1 Gbit/s (95th percentile), it can become a bottleneck very quickly. We used the CCR2004 because it’s cheaper than any similar box, but I have doubts about the scaling we may experience with it. For growth, I think an Arista would be better compared to pricing (sorry MikroTik).