Hello to everyone,
I'm new to routing and Mikrotik and I've decided to pull the trigger and start my journey with ROS 7.
I didn't have any particular issue with setting everything up apart from the max-prefix-limit which seems to be missing.
I've tried searching in Mikrotik's documentation for ROS 7 without any luck, it just seems to be gone.
In all best practice guides that I've found, everyone suggests to set a limit for the number of prefixes for each BGP session to protect your router from potential leaks.
Am I missing something?

currently session does not show prefix count, but you can get route count either by printing routes from the routing table with print count-only or use /routing/stats/origin which shows total route count for each process

currently session does not show prefix count, but you can get route count either by printing routes from the routing table with print count-only or use /routing/stats/origin which shows total route count for each process

So there's no way to limit the number of prefixes pushed by the other peer in BGP?
Since this function was available in ROS 6, do you plan to reintroduce it at a later time in ROS 7?

sorry misread the original post. max-prefix-limit most likely will not be added.

sorry misread the original post. max-prefix-limit most likely will not be added.

You got to be kidding, right?

sorry misread the original post. max-prefix-limit most likely will not be added.

Seriously????????
This is not an optional feature! max-prefix limit is extremely important.

This is a legacy parameter that initially was used to not overflow available memory. Then somebody thought that it is a good idea to use it as a protection from bad announcements. But actually, it is the worst case you can do for protection. Flapping connections simply add unnecessary load to the router. Think of it like this, what if we start to flap BGP sessions just because the received prefix after RPKI validation is considered invalid.
Instead, proper filtering should be used. RouterOS v7 has a very good option to filter incoming NLRIs, before they get processed (see input.accept-* properties). This is an easy and flexible way to allow specific LSRIs from remote peers, that way it handles improper route leaking, and nothing is flapping.


What we consider is adding feature to log warning message when certain prefix limit is reached (like juniper action warn), but thing that flaps bgp session will not be added.

sorry misread the original post. max-prefix-limit most likely will not be added.

Seriously?

So any peer could crash my routers by flooding me with prefixes? No, wait, should we even EXPLAIN you why max-prefix-limit is important?

This is a legacy parameter that initially was used to not overflow available memory. Then somebody thought that it is a good idea to use it as a protection from bad announcements. But actually, it is the worst case you can do for protection. Flapping connections simply add unnecessary load to the router.

You should shutdown the session and have it disabled permanently or for a long time, not just close and restart immediately.

Flapping connections simply add unnecessary load to the router.

It can be solved with a long timeout after hitting a max-prefix limit. Even 12 hours could do. But it's dangerous to not have it. Full-view leaks are more common that what everyone could think.

You should shutdown the session and have it disabled permanently or for a long time, not just close and restart immediately.

Exactly. It's better to shut it down and require manual intervention to reactivate it than not having the feature.

Decision is not written in stone if somebody could describe situation where it is absolutely necessary and cannot be done in any other way.
At this point mentioned arguments are not enough.
If you are not receiving full BGP feeds, then typically you know what prefixes should be advertised and as mentioned before proper setup of input.accept-* parameters will not allow flooding of unwanted prefixes and session is never dropped.
IF you are receiving full bgp feed from the router then setting max-prefix-limit does not make any sense at all.

sorry misread the original post. max-prefix-limit most likely will not be added.

What? Is anyone using BGP in real enviroment there?

If you are not receiving full BGP feeds, then typically you know what prefixes should be advertised and as mentioned before proper setup of input.accept-* parameters will not allow flooding of unwanted prefixes and session is never dropped.
IF you are receiving full bgp feed from the router then setting max-prefix-limit does not make any sense at all.

You've obviously never seen a typical setup at a NAP or other peering facilities.

Obviously not, that is why I am asking to explain when flapping bgp session is more beneficial than just setting up properly bgp input.

Obviously not, that is why I am asking to explain when flapping bgp session is more beneficial than just setting up properly bgp input.

In the typical case, when you have hundreds of heterogeneous peers, filtering is not always feasible for a number of them. The last line of defense against someone else's configuration mistake becoming a complete disaster for you is to drop (not flap!) a peer if they hit a max-prefix limit.
That usually triggers alarms and someone has to look at it, and clear it manually.
I know about RPKI and stuff in an ideal world, but it's still not possible to get rid of max-prefix-limit circuit breakers, today.

Or, think of a sizeable network, with reflectors and clients, some with a full view, and most with not enough memory/cpu to handle the whole thing. max-prefix-limit is what prevents a configuration mistake from bringing the network down.

If you are not receiving full BGP feeds, then typically you know what prefixes should be advertised and as mentioned before proper setup of input.accept-* parameters will not allow flooding of unwanted prefixes and session is never dropped.

There are some conditions that may be appropriate:
1) I want to accept prefixes up to /48 from a peer who can maliciously send me all the /48s contained in a /32 or in a /29, thus generating cpu and memory issues with 65k or 512k routes... in this case it is best to shutdown the peer and solve the issue (and an "highwater mark per peer" could be an useful feature, indeed, to show problematic peerers). A slow atuomatic restart could be useful to handle "transient" leaks due to configuration errors.
2) I agree with you that on transits sessions this is less useful since network count numbers are so high that a "leak" is too small to trigger any max-prefix (eg. 65k leaked routes in 900k).
3) It could be useful in a iBGP route-reflectors scenario where you don't have RPKI filters on iBGP sessions

Anyway avoiding triggering limits that may scale up to cpu or memory exhaustion could cause a not so good scenery similar to what happened in the "longer is not always better" event.
My 2 cents, GP

So MikroTik, can you please explain me, how to filter out a fulltable leaked to me by a peer? With RPKI? For sure not!
Max-Prefix-Limit is a missing feature. And it's needed. Even if you don't like it.

There's a simple solution to prevent peer flapping: Disable the session until someone clears it. Like cisco etc.

There is as safety pin mostly for IX.

You know that IX has 100-2500 routes. If this goes over 5000 something is wrong and its better ti drop.

There is as safety pin mostly for IX.

You know that IX has 100-2500 routes. If this goes over 5000 something is wrong and its better ti drop.

That and not only for IXs.
Think about what happened yesterday with Cloudflare leaking all Akamais routes, in this case you would have simply shut down the session.

I would like to put my +1 to this setting.

Setting max-prefix is often crucial in operating peering routers at IXes to protect your own network. Although you should build routing filters to all of your peers. This is often impossible when you do not rely on automatically building them from things like RIPE-DB etc. Things are changing fast and you can not manually adjust those filters. To customer side this might be different.
If you have a look at the entries in the PeeringDB (https://peeringdb.com) you see that almost all networks out there are publishing their suggested max-prefix values when peering with them. From time to time they are changing this value when they gain more prefixes in their announcements. Recently Google did this. When such a change arises, those networks are even actively announcing those changes over the IX mailing lists. So this functionality is no "nice to have" but a de facto industrial standard when you do "real" BGP. Every major vendor supports this.

I would also like to have a flag to indicate that a session has to stay down (in addition to max prefix restart time), when max-prefix has been hit, until it is manually cleared. That way you would prevent flapping of the session which might lead to overload in BGP processing when a big peer hits max-prefix.

Considering how many people intervened and explained why having a max-prefix-limit is important, is there an official decision on Mikrotik's side?
I honestly don't understand the resistance on implementing such a simple function that was already present in ROS 6 and that every competing product already has.
If you go and have a look at PeeringDB to just a few IX and big companies, they all have suggested values for the max-prefix-limit when peering with them and some also require you to do the same so they can create automatic filters based on your PeeringDB page.

This is a legacy parameter that initially was used to not overflow available memory.

Is overflowing available memory truly a non-issue? I get that with software packet forwarding, the only real limitation in the lookup table is the size of RAM and that is becoming --- or is already --- a nonissue on most devices that are worthwhile to put in a peering role.

On the other hand, Mikrotik seems to be going the direction of building more and more products with hardware offloading. To my understanding, these products very much do have hardware routing table limitations and much lower than the boxes with 10-100x the price tag from the big vendors. According to the documentation [1], none of them have enough resources to fit a full table. In the best case scenario this means the CPU picks up the slack but in the worst case, it seems the DX8000 and DX4000 series devices will simply drop prefixes that don't fit into the hardware tables. There's a big red warning (for those chips, at least) that the operator needs to make sure the table fits within the limit. What exactly is our real-world defense to meet this advice if not a BGP session prefix limit? I can guarantee that it's not going to be the IGP routes or well-behaved BGP peers that will cause this number to jump.

A lot of the examples so far have been from IXs, but I have encountered real world scenarios working on a regular, multihomed, non-transit AS where maximum-prefix would have saved us (and eventually did). The specific example is a circumstance where we accept default and a limited selection (~1K) of more specific routes from our upstream provider. There was really no commonality in terms of AS path, path length, or other intrinsic way for us to filter what this subset was supposed to be and it was entirely subject to the traffic engineering whims of our upstream provider. Of course, this subset was tagged with an agreed-upon community and we filtered on that. Nonetheless, one bad route map from one of their routers was enough to accidentally give us full tables with that community. Our closest edge router crashed and then our others learned the routes from iBGP and crashed too. After that incident we implemented maximum-prefix. It happened again about a year later in another maintenance window and that time we only lost one BGP session and our AS stayed online.

The amount of mileage you get out of a feature as simple as counting prefixes is extraordinary.

[1] https://help.mikrotik.com/docs/display/ ... Offloading

+1 for all the reason above for having max-prefix-limit.

We also have this requirment at our Peering INX's and have this requirement from our IPT providers for this Parameter

I think the reluctance (or refusal) to implement this is related to the new architecture of the BGP service.
In the past, the BGP service did all the peer communication and filtering. And in the end, it had a count for the number of prefixes received from each peer (and passed the filtering).
Such a counter can of course be easily compared to a limit, and some action can be taken.

However, in the new BGP things appear to be different. The BGP peer communication happens in a separate process per peer (which is an improvement for routers with a lot of cores) and all received prefixes are always added to a routing table. Only after that, the routes are filtered and an active route is selected and put in the FIB.
There is no such thing as "number of prefixes received from a peer" anymore, it would have to be counted in the routing table after the filtering has been performed.
Which mrz suggests can be done, however the "routing table with print count-only" lacks the proper "where" clause to filter by peer. Maybe that still has to be added?
("belongs to" is a column that can be displayed, but cannot be used in "where")

The advantage is of course that it improves the possibility to multithread the entire thing. But I also see the disadvantages, like not having an overview over the number of received prefixes, and no way to control them.
Plus, in a case where you filter much of the received prefixes (probably not a good thing to have, but it can occur) you will still store all the filtered prefixes. That was not happening in v6, it would simply drop them.

Anyway, the whole BGP implementation is not finished. Hopefully serious work is being done on it right now.

Doing BGP peer communication in a separate process per peer makes things easier, at least from my point of view. As prefix-limit is per peer too.
And there must be some internal BGP table for each peer, how otherwise recalculation would be done when a peer drops down? But who knows? At least I don't
Also, there is "where belongs-to=..." clause.

Ok I did not check that for v7.1, but v7.2rc4 (which I expected to be only better) for sure has no "where belongs-to=" clause.
But it displays this info in winbox. Maybe you mean you can filter it in winbox? That is something unrelated.

What should work is: /ip/route/print count-only where belongs-to="BGP IP routes from 1.2.3.4"
For sure this does not work.

I mean
/routing/route/print count-only where belongs-to="BGP IP routes from A.B.C.D"

I would be really surprised if there is a technical reason that it is difficult to implement maximum-prefix but I don’t know enough about the new BGP architecture in 7.0 to say. AFAIK maximum-prefix needs to kick in far earlier than any of the parts of BGP processing that are performance critical. If it’s a process per peer or per peer-group/template now, the logic for checking how many prefixes you are ingesting from a peer only needs to live there and I wouldn’t necessarily expect that it would be reflected in the final BGP RIB-IN datastructure. If we’ve got this wrong I’m sure mrz or someone will point it out.

It sounds more like planning to omit it was a well-intentioned feature clean-up on Mikrotik’s part that most of us here think is a step too far.

I mean
/routing/route/print count-only where belongs-to="BGP IP routes from A.B.C.D"

Ah yes, that works. I wasn't aware that /routing/route/print was different from /ip/route/print, as there is no /routing/route menu in winbox and what is shown in /ip/route seems equivalent to what is shown in /routing/route in winbox.
Confusing... also /ip/route/rule in winbox is /routing/rule in commandline. Why??

Well, now I also could make /routing/stats/origin/print work, and it shows the requested counts. I don't know if these are gathered using a walk of the routing table when this command is executed, or of they are kept in memory all the time. It has been a repeated request to show the "prefix count" in /routing/bgp/sessions but it remains at zero in 7.2rc4, both in winbox and in commandline (where it does not appear at all).

Still lots of work to do!

Guys, this is critical in IX peering environments.

Please add it !

sorry misread the original post. max-prefix-limit most likely will not be added.

It most definitely will be added or RouterOS version 7 will not be used by any competent network administrators.

This is not a negotiation, but a demand.

Must have feature in my opinion for reasons stated above.

sorry misread the original post. max-prefix-limit most likely will not be added.

i will just leave this here:
https://datatracker.ietf.org/doc/html/rfc7454#page-18

Well, I already noticed it also does not support route aggregation. Not only can the router itself not aggregate routes, but when it is in the network receiving aggregated routes created by another router it does not forward them correctly further down the path.
And, it does not handle refresh correctly. When you change something in the filters and use refresh the routes elsewhere in the network will not be updated.
(this, too, applies both to the router itself and to other routers further down)
So after a filter change you need to disable/enable all peers to reconfigure the network.

And, it does not support BFD. Also not convenient when using it in a network with "unstable" links like WiFi links.

I'm not claiming they do not understand this, but the product is not finished, there is no warning about that in the announcements, and it takes them ages to fix it.

I'd like to share my perspective on the feature. My work does not use MikroTik gear (wrong market segment), however I do use it personally and am a fan. I'm hoping I can at least provide some illumination around what it provides and why we use it. Since my work is probably not the target market for MikroTik, it'll be up to the people here and MikroTik to determine if this feature is important and how the feature applies. I'm just providing an outside perspective.

A little bit about our network -- On the customer side of things, we have BGP neighbor adjacencies with around 200 ASNs (customer-of-customer ASN count is around 1000). On the peer side, we have around 500 settlement-free BGP adjacencies (note: I'm talking with our peering architect to get better numbers, these were a guess on my part). Some are peer networks are cloud/content/cdn providers, some are consumer ISPs, some are networks in other geographies with a similar organization structure.

There are two different kinds of max-prefix-limit settings that we've used in the past, so I'm going to mention both here. The most common one applies to routes that are accepted -- this is the one we use today. The other setting applies to all prefixes, regardless of whether they are accepted or not (as far as I know, this is a JunOS specific feature). Unless stated otherwise, I'm talking about a max-prefix-limit that applies to accepted routes only.

In either case, these features act as a circuit breaker, both on the control plane and data plane. I want to emphasize the circuit breaker aspect word. it works a lot like an electrical circuit breaker or fuse -- you have fuses/breakers in smaller devices like your PC, you might have 15/20A circuits for appliances, and 30-50A circuits for heavy appliances like ranges, ovens, air conditioning, etc. They're sized for the use case and protects the wires, the appliances, humans (if it's an RCD/GFCI protected circuit), and sometimes upstream components like the breaker panel itself. Even the breaker-box itself might have a 100-150A fuse or circuit breaker for the whole house. The point being is that this feature is there to protect things locally and also to isolate faults in a system before that fault can cascade.

max-prefix-limit protects the local router, the entire SP network, and even neighboring networks. Accidental route leaks do happen, RPKI is making those happen less often, but they still do happen and RPKI is not widely deployed - indeed in some network scenarios, RPKI may not be available.

We often have prefix-limits tripped (either at the warning level, or actually shutting down BGP sessions). These are the cases I can think of:

• Misconfiguration that leaks neighbor prefixes - usually this is corrected fairly fast (within an hour). Resetting the BGP session functions to protect our network -- both local router control plane, but also other routers in the network and even backbone links -- Route leaks can shift 100+Gbps of traffic. As a nice side-effect, it's a clear signal to the neighboring network that "something went wrong with your BGP policies".
• Misconfiguration that leaks BGP optimizer injected routes. These can be particularly dangerous because they're often more-specific prefixes that will override every BGP selection criteria.
• Legitimate gradual network growth - We usually hit some alarm threshold (75%) and will increase the session size. This often happens without disruption.
• Legitimate, but significant network growth - Sometimes a merger/acquisition or peering agreement can mean that the neighboring network grew significantly and trips the reset/shutdown threshold. This is one is rare, but it would give us a chance to look at link speeds and ensure that there is sufficient capacity. We've had to drop a % of routes (usually done via routing policy) as we arrange for capacity increases. In one specific case we actually had to use that to convince someone we could saturate the capacity of a link for a period of time before they'd let us upgrade to a faster peering connection.

In all cases we let the BGP session retry after ~15 minutes or whatever the vendor default is. If a neighbor goes over, the BGP neighbor is held in a down state for 15 minutes then allowed to retry. We've found this to be a use-case.

On the private circuit side of things, we provide a small truck load of L3VPNs to customers, and by default we limit it to 50 prefixes. We occasionally see a customer accidentally (try to) leak a full internet routing table into their private L3VPN. Many of the other tools to protect the routing table like RPKI are unavailable since the L3VPN also needs to carry RFC1918 addresses. Here it's absolutely vital to have those protections because of the collateral damage that a flood of routes could cause.

max-prefix-limit may not be as applicable to every network, but as a network moves from sitting at the edge of a network to sitting more at the center, these protections start to become more and more important. It protects the local router's control-plane, other router control-planes, but also protects upstream links by preventing an uncontrolled deluge of traffic being directed in the wrong direction.

Finally, from a dollars and cents perspective even though my work is not in the right market segment for MikroTik, the lack of this feature would most certainly disqualify a vendor from being considered as part of an RFP or purchase. And that's not any single engineer/architect making a unilateral decision. That's just how it is for networks like us.

References:

• https://www.juniper.net/documentation/u ... s-bgp.html
• https://www.juniper.net/documentation/u ... s-bgp.html
• https://spaces.at.internet2.edu/display/RS/Max+Prefixes

Small update.

We are working on this feature. Currently there will be two options:
* do not break session but throw the warning that limit was reached;
* break the session on the limit, reconnect will require the administrator actions, otherwise the session will stay down indefinitely.
Also by not going too much in the details max-prefix will represent the approximate value at which event will be triggered due to the nature of BGP implementation.

Thanks for the update mrz; glad to hear that the use cases have been persuasive to Mikrotik product engineering. The two options would meet the requirement for me at least, not sure about others' use cases. Presumably anybody desiring a retry behavior wants it to be pretty slow to avoid flapping and can therefore accomplish this with a script.

Thanks mrz!

I hope this will also bring back the Prefix Count field in the Routing->BGP->Sessions window.
It is an invaluable indicator for quick status overview of the routing.

Also please make the Routing->BGP->Sessions window in winbox auto-refreshing like the Routing->BGP->Peers window in v6.

Small update.

We are working on this feature. Currently there will be two options:
* do not break session but throw the warning that limit was reached;
* break the session on the limit, reconnect will require the administrator actions, otherwise the session will stay down indefinitely.
Also by not going too much in the details max-prefix will represent the approximate value at which event will be triggered due to the nature of BGP implementation.

This is great news!
Thank you for listening to the community.

Small update.

We are working on this feature. Currently there will be two options:
* do not break session but throw the warning that limit was reached;
* break the session on the limit, reconnect will require the administrator actions, otherwise the session will stay down indefinitely.
Also by not going too much in the details max-prefix will represent the approximate value at which event will be triggered due to the nature of BGP implementation.

Hi Maris,

Thank you for the update.

Can your team please consider "Feature Parity" with RouterOS v6 for the max-prefix-limit feature.

e.g.

- Support max-prefix-limit
- Support max-prefix-restart-time

with the same behavior as RouterOS v6 e.g. https://wiki.mikrotik.com/wiki/Manual: ... s_accepted

I have tens of thousands of peers configured using these settings with RouterOS v6. The max-prefix-restart-time is also useful, as if a peer makes and accidental filter change then realizes the error of their ways and corrects it the peer will automatically reconnect and continue working.

Small update.

We are working on this feature. Currently there will be two options:
* do not break session but throw the warning that limit was reached;
* break the session on the limit, reconnect will require the administrator actions, otherwise the session will stay down indefinitely.
Also by not going too much in the details max-prefix will represent the approximate value at which event will be triggered due to the nature of BGP implementation.

Hi Maris,

Thank you for the update.

Can your team please consider "Feature Parity" with RouterOS v6 for the max-prefix-limit feature.

e.g.

- Support max-prefix-limit
- Support max-prefix-restart-time

with the same behavior as RouterOS v6 e.g. https://wiki.mikrotik.com/wiki/Manual: ... s_accepted

I have tens of thousands of peers configured using these settings with RouterOS v6. The max-prefix-restart-time is also useful, as if a peer makes and accidental filter change then realizes the error of their ways and corrects it the peer will automatically reconnect and continue working.

I agree with this

Just released 7.3beta33, has input prefix limit feature.

Thanks! Are there any docs available somewhere? Btw, regarding "bgp - added initial support for prefix limit": is "initial support" limited somehow?

I cant find option max-prefix-limit ir RoS v7.7, please any tip ?

input.limit-process-routes-ipv4

https://help.mikrotik.com/docs/display/ ... mplateMenu

So glad this was implemented, and confirm it works really well - thanks for implementing Mikrotik Team. It's not a replacement for filtering what the peers are sending of course viewtopic.php?p=997165

I can't figure how do I set prefix limit.

Please give console command or winbox capture for it.

Thanks

input.limit-process-routes-ipv4

https://help.mikrotik.com/docs/display/ ... mplateMenu

This is a poor implementation of a function that should be hard limiting prefixes received.

On many prominent Internet Exchanges, they configure a hard limit of routes received from peers based on PeeringDB entries. Peers generally do the same when configuring bilateral sessions.

Let's use Microsoft AS8075 as an example. On their PeeringDB entry (https://www.peeringdb.com/net/694) they specify that peers should see no greater than 2000 v4 and 500 v6 routes and to assume there is an issue if a peer receives anything greater than this. If a peer hits a limit it should do an administrative shutdown of the session until the issue has been resolved with the relevant parties where it would simply require the session to be re-enabled. This function should be a clearly defined option when configuring a peer, as this value will be different between all peers and no two are the same.

Further, this is a RECOMMENDATION by the IETF in RFC 7454 when configuring peering sessions, as previously explained. For such a prominent hardware vendor to completely disregard an RFC is baffling. Especially when no support for filtering prefixes by IRR objects natively exists without using external systems.

@mrz

/routing/stats/origin--> what is the flag YZ?

Let's use Microsoft AS8075 as an example. On their PeeringDB entry (https://www.peeringdb.com/net/694) they specify that peers should see no greater than 2000 v4 and 500 v6 routes and to assume there is an issue if a peer receives anything greater than this. If a peer hits a limit it should do an administrative shutdown of the session until the issue has been resolved with the relevant parties where it would simply require the session to be re-enabled. This function should be a clearly defined option when configuring a peer, as this value will be different between all peers and no two are the same.

input.limit-process-routes-ipv4=2000 .limit-process-routes-ipv6=500 does exactly that.

Let's use Microsoft AS8075 as an example. On their PeeringDB entry (https://www.peeringdb.com/net/694) they specify that peers should see no greater than 2000 v4 and 500 v6 routes and to assume there is an issue if a peer receives anything greater than this. If a peer hits a limit it should do an administrative shutdown of the session until the issue has been resolved with the relevant parties where it would simply require the session to be re-enabled. This function should be a clearly defined option when configuring a peer, as this value will be different between all peers and no two are the same.

input.limit-process-routes-ipv4=2000 .limit-process-routes-ipv6=500 does exactly that.

Am I wrong or these parameters are only available in templates?
This would mean creating a template for each peer since each one might have a different number of routes they might advertise.
I think everyone was hoping for a more granular support at the Connection level.

No, template parameters are exposed to connection, so there is no need to create template for each connection.

how about the flag what YZ does?