Joined: Sat Jan 07, 2006 5:15 am Posts: 413
Karma: 1
Location: Brazil
Hi, Every now and them we got that problem. Today another AS network engineer called me becouse his IP prefix couldnt reach our prefixes. After troubleshooting he told me that they lost they connection in a national wide IXP and when doing a traceroute from our bgp router, the next hop showed was our IXP ip address. After checking the routes, we're receiving their routes via another service provider but RouterOS were trying to get to them via a cached route that doesnt even exists anymore.
To fix the problem we have to disable the IXP interface on our router and them reenable it. After that RouterOS finnaly forgot the route and them everything started to work.
Anyone more having that problem? Is there any command to clear routing cache?
Hi, Every now and them we got that problem. Today another AS network engineer called me becouse his IP prefix couldnt reach our prefixes. After troubleshooting he told me that they lost they connection in a national wide IXP and when doing a traceroute from our bgp router, the next hop showed was our IXP ip address. After checking the routes, we're receiving their routes via another service provider but RouterOS were trying to get to them via a cached route that doesnt even exists anymore.
To fix the problem we have to disable the IXP interface on our router and them reenable it. After that RouterOS finnaly forgot the route and them everything started to work.
Anyone more having that problem? Is there any command to clear routing cache?
Thanks!
I don't think there is a command to flush the route list/cache. It generally does this automatically. You should contact support@mikrotik.com and request such a feature or workaround.
Joined: Sat Jan 21, 2006 6:54 pm Posts: 63
Karma: 3
Location: Johannesburg, South Africa
Hi,
This specific bug is causing some frustrations for me as well. It appears that the stale route remains in cache for as long as any IP sessions are trying to make use of the path. In this regard think of any TCP connections that might need to time out etc - the route could therefore stay in cache for far longer than one would reasonably expect. From past reading it looks like items typically expire from the cache after around 10 minutes - but not for 'in-use' routes.
The forcible shutting of an interface has on occasion not solved it for us - and we actually had to perform a reboot.
This is one of the frustrations I hope gets addressed on next RouterOS builds. While being able to forcibly manually clear the cache would help it would be better for TIK to remove 'in-use' routes if the routing entry supporting it is removed from the main routing table.
Joined: Sat Jan 07, 2006 5:15 am Posts: 413
Karma: 1
Location: Brazil
Thank for your statement about that bug. Please open a support ticket. So way the MikroTik trama will see that more ppl have the same stuck route problema.
The same problem. I have a test that pings the remote server. Sometimes the ping test fails but all the devices on the way are UP and running. Disabling and enabling route to this destination on mikrotik solve the problem.
Im experiencing this problem on 3 x86 (5.7) and 1 RB433 (5.14).
I have exaclty same problem reported at top of this thread (dec 2011?!?). Two routers connected to IXP and when a peering partner stops sending a prefix my router removes old route from table but still trying to use that route to forward packets... and worst still sendind that prefix through ibgp to my core router.
I`m wondering if that problem still the same problem reported in 2011 or its is new. Hope it`s new because one year to solve a routing problem is too much if you have in mind we are talking about ROUTERS.
Joined: Sat Jan 07, 2006 5:15 am Posts: 413
Karma: 1
Location: Brazil
Hi,
Still present, yesterday I have the same issue again with a router with a low route count ( about 300 routes). I had to disable and reenable the interface that had the stale stuck route and everything back to work..
I don't know why RouterOS guys fix this or create a tool to clear the route cache.
Joined: Sat Jun 16, 2012 12:42 pm Posts: 44
Karma: 0
Hi...
You should take in consider that BGP need time to be updated.. its not like of other routing protocol.. update the route DB as soon ad there is change apear in network..!!
So you have to wait even my take up to 5min..!!!! till most of Upstream provider got the update..!!
Also there is another option is that you can refresh you peer update / routing bgp peer refresh-all afi=ip
Joined: Sat Jan 07, 2006 5:15 am Posts: 413
Karma: 1
Location: Brazil
MikroTikIQ wrote:
Hi...
You should take in consider that BGP need time to be updated.. its not like of other routing protocol.. update the route DB as soon ad there is change apear in network..!!
So you have to wait even my take up to 5min..!!!! till most of Upstream provider got the update..!!
Also there is another option is that you can refresh you peer update / routing bgp peer refresh-all afi=ip
Regards Ali Sami
The bug related here is that even if you wait a month, if you dont reboot the router or disable and reenable the interface that the related bgp route were using, the traffic destinated to that destination will use the stale route for ever..
As I said before, route disappear from routing table... Looks like bgp receive routing update from neighbor, remove it from routing table but it still present on route-cache. What I dont know is why the removed prefix keep advertised to other routers.
As far I know, bgp use messages to update routes, it don`t have a holdown timer to wait before removing a route, like RIP does. (I`m talking about updates and not neighbors timers)
On my case routers a connected to IXPs and prefixes are exchanged by MLPA on those IXPS, what give me only 11.000 prefixes and updates occour only inside that group.
I`m going to downgrade to version 3.27 and see what happens. Support told me that problem is known since 4.x.
But I did it only on RB1000 hardware because RB1100/RB1200 dont support that version.
Also... I downgraded to 3.30 and I`m using default routing package, looks like routing-test is the same used as default routing package on versions > 3.30.
Don`f forget to check if 3.30 can do all tasks that you need on your router, mikrotik did a lot of improvements 5.x versions.
Joined: Wed Feb 15, 2012 2:55 am Posts: 32
Karma: 0
Here in Brazil, the biggest IXP in South America (PTT-SP) there is a mailing list in which several users posted they are suffering from the same problem (with different ROS versions) and also complaning support tickets are left unanswered. Some companies migrated to Quagga because of that issue (and mainly due the lack of consistent response). I strongly suggest this issue to be reviewed with a higher priority.
I`m one of those users connected to PTT-SP and saw a thread on [GTER] list about that problem just after my first post here. Also... I definitely agree with you about mikrotik must increase priority to solve this bug.
C`mon Mikrotik help us to keep supporting your products! give us an workaround at least!
I`m one of those users connected to PTT-SP and saw a thread on [GTER] list about that problem just after my first post here. Also... I definitely agree with you about mikrotik must increase priority to solve this bug.
C`mon Mikrotik help us to keep supporting your products! give us an workaround at least!
+1 member! ( and BR too)
Gustkiller, anything on your support ticket about this issue?
Joined: Sat Jan 07, 2006 5:15 am Posts: 413
Karma: 1
Location: Brazil
I gave up...
lyma wrote:
jthiele wrote:
Hello Kurt,
I`m one of those users connected to PTT-SP and saw a thread on [GTER] list about that problem just after my first post here. Also... I definitely agree with you about mikrotik must increase priority to solve this bug.
C`mon Mikrotik help us to keep supporting your products! give us an workaround at least!
+1 member! ( and BR too)
Gustkiller, anything on your support ticket about this issue?
Joined: Wed Oct 19, 2011 2:13 am Posts: 28
Karma: 0
Location: Outside
gustkiller wrote:
MikroTikIQ wrote:
Hi...
You should take in consider that BGP need time to be updated.. its not like of other routing protocol.. update the route DB as soon ad there is change apear in network..!!
So you have to wait even my take up to 5min..!!!! till most of Upstream provider got the update..!!
Also there is another option is that you can refresh you peer update / routing bgp peer refresh-all afi=ip
Regards Ali Sami
The bug related here is that even if you wait a month, if you dont reboot the router or disable and reenable the interface that the related bgp route were using, the traffic destinated to that destination will use the stale route for ever..
same problem with OSPF routes... I sure it is a basic problem with routing package I send many bug report to support and each time answer is "it will be fix later"... time after time...
Joined: Wed Feb 15, 2012 2:55 am Posts: 32
Karma: 0
The lack of official and public response from Mikrotik, for me, is an evidence they don't have a clue on why this happens and how to fix it. So, no fix soon, even in the first v6.0. If such severe problem were about to be fixed in the next v6 stable release, they would be annoucing it heavily.
OSPFv3 just doesn't work, BGP has stale routes... They've been too busy devoting all their time to building the CCR which is great and all, but the adage "learn to walk before you try to run" springs to mind. It makes absolutely no difference to me how many packets I can push over a device, how many queues I can have on it (I don't personally use any - we only trade in wirespeed), if I can't rely on it having a valid/complete routing table in the first place and know that the packets will actually get where they're supposed to be going.
I kept getting told that the problems I have will be fixed in "the new routing package" but that they have no real information on when it will be available. There's no talk at all in the v6 Changelogs of a new routing package, so it presumably still hasn't been addressed. Like others, I gave up asking support because it was a waste of my time.
I'd have been game for grabbing a pair of CCR1036's but after buying two of their former flagship tins (RB1100AHx2) to use in a testbed for customer access routing and getting - mostly - no useful support it seems like a waste of time (which is in shorter supply than money) - and I have no reason to believe that support will get any better. By support, I include rolling out software fixes in less than a couple of weeks for major functionality defects rather than things being broken for over a year. I even offered to buy additional licences to bribe them into fixing their product.
It's actually pretty hilarious to me that the MUM in the US had a presentation from one of the guys at Hurricane about how IPv6 is no longer optional whilst current builds of RouterOS can't even maintain adjacency in OSPFv3.
I *want* to back the underdog, but I *can't* when there are major problems with the way the product's firmware is supported. MikroTik don't seem to get that I and others will spend more money to get a product that just WORKS and they are limiting their own growth potential - refurb/grey Cisco and Juniper is not so expensive as to be an unattractive alternative, because Cisco et al without support is better than MikroTik with support right now.
This strikes me as a dumb position to be in, when you're proposing to have a box that can shift 24mpps at $1000 and you're still not coming out on top simply because the software and support is bad - I might need a router, I might want a <$5000 router, but I certainly don't need - or want - a $1000 paperweight that's going to tie up hours of my time (which has a very real cost).
The worst of it is that nobody really seems to care. There is no sensible response to any of this. Presumably the WISPs et al are buying enough of the smaller bits of kit to keep the lights on, so why change the model, right?
Joined: Sat Jan 07, 2006 5:15 am Posts: 413
Karma: 1
Location: Brazil
Great comment!
Here we have to buy a Juniper MX router that cost here about U$30K dollars, but in the last 8 months since we change our x86 mikrotik border router with the Juniper, we have yet to see any kind of issue or hicup. Sometimes I really forget that we have a router running here.
I really "loved" mikrotik router os and products, since 2005 . But when you go large ( full tables, and more routing complexity) it lacks "software" reliability.
I'm really looking forward to try the CCR product, but we will not be able to deploy as a backup router for our Juniper or even in our most crowed PoPs because of these routing issue.
I suppose it's fortunate that because I started from a more complex environment to begin with - dual stack full table access routing - that I uncovered the flaws before we got too involved with MT.
If I had spent time getting a certification, and had a pile of their kit, I would definitely be in the CCR announcement thread making myself unwelcome asking if the CCR actually did BGP/OSPFv3 properly or if it was broken (and likely to remain broken for a year+) in them too.
In the absence of any meaningful response from MikroTik, has anyone found a workaround short of rebooting the router every X hours ? I have the same problem with two routers (out of several hundred). Neither do any advanced routing (OSPF, BGP), but both use bridges to terminate VLANs. Each will invariably stop processing IP traffic once the route cache fills up. The rate is dependent upon traffic, one takes several days, the other just hours. Interestingly, you can MAC telnet into the router which at least allows me to remotely reboot from an adjacent device.
Joined: Sat Jan 07, 2006 5:15 am Posts: 413
Karma: 1
Location: Brazil
deimos wrote:
In the absence of any meaningful response from MikroTik, has anyone found a workaround short of rebooting the router every X hours ? I have the same problem with two routers (out of several hundred). Neither do any advanced routing (OSPF, BGP), but both use bridges to terminate VLANs. Each will invariably stop processing IP traffic once the route cache fills up. The rate is dependent upon traffic, one takes several days, the other just hours. Interestingly, you can MAC telnet into the router which at least allows me to remotely reboot from an adjacent device.
Disable and Reenable the affected interface do the trick here. But as I changed our border router from MK to Juniper, I don't have this issue anymore.
I would try that if the router was not an hour's drive away. I suppose I could experiment with a script to accomplish this, but it is a little scary if something goes awry and customers are down for hours. They just hate that.
I actually solved my problem by splitting the functionality between two MT routers. One does the VLAN tagging, the other does NAT. Stupid, but what else am I going to do with all those RB532's ....
Joined: Sat Nov 24, 2012 10:26 am Posts: 4
Karma: 0
Reading this thread is very disappointing. Route manipulation in any dynamic routing protocol needs to function reliably. I was hoping to deploy MikroTik devices at the edge. This thread just killed any future plans with MikroTik with exception for use as a wireless home IP access product.
Staff i have been trying to fix a little issue i had i was basicly adding static routes and removing them and re creating nats and so forth, and some where along the line one of the routes i am certain of got stuck, but it was not in the route table
it took a reboot to clear the mysterious issue.
I really dont have much in my router but i have rb450g i dont think you need my config in this case
Just checking but you mean the issue WILL be fixed right? Do you know something about when that new package will be avaliable?
tks
Hi jthiele, I have no idea when this will be fixed, I am guessing the answer is "When it's ready". Maybe Maris(MRZ) or Janis from Mikrotik can provide a better answer
During some testing of a dual border router, dual provider setup with OSPF re-distributing a default route between the two border, I discovered that the default route being re-distributed in to the OSPF is being received according to the OSPF routes tab, but does not appear in the main routing table under /ip routes. Oddly, it still appears to be active as I can still reach the router, but it certainly is not appearing in the winbox output!
can you give a roadmap how to repeat bgp-cache bug with CCR or 1100AHx2 in LAB? if necessary i have several copies of 440k routing table to share in iBGP
Isn`t necessary to have many routes in the table for the problem appears. Here I have ~ 20.000 routes on router connected to an IXP (NAP).
Some information that can help:
- My peers are IXP Route Servers running Quagga (not 100% sure if quagga still). - There are routing filters implemented, nothing special. Discarding bogons, and setting community on input filter. - Mikrotik redistritute learned bgp routes from IXP to a local quagga working a Looking Glass server
What happen: - If a IXP member stop announcing a prefix it desapears from Mikrotik routing table but Mikrotik keep announcing it to My LG Server - Despite route not part of Mikrotik routing table, router keep using it to send traffic - Refresh ou reset on bgp session does not help, reboot needed
Last time it happened to me I had two prefixes from an ASn, one was 186.x.y.0/21 learned from IXP and the other was a 186.x.y.0/20 learned from my transit peer. When prefix owner removed /21 from his announce router ignored /20 and keep sending traffic to /21 gateway.
Joined: Sat Jan 21, 2006 6:54 pm Posts: 63
Karma: 3
Location: Johannesburg, South Africa
Hi Kurt,
To the best of my knowledge v6 is still currently running the old routing code base - I'm not willing to put v6 into my main production network at this point to test it.
I do know there were plans from Mikrotik on a new routing-test package but haven't seen or heard anything of late. Maybe support@mikrotik.com can assist in this case?
Joined: Wed Oct 19, 2011 2:13 am Posts: 28
Karma: 0
Location: Outside
Chupaka wrote:
not telepathy, but laboratory mode
Cool! So many useless work for finding hidden surprises... We need to test ALL releases instead reading changelogs... Like Russians say: "Find something, do not know what".
[irony]Very very important know about LCD displays improvements instead routing fixes [/irony]
you should test only current release for your needs - they don't fix old releases. if something is not working - you write to support@mikrotik.com and wait for response. they will say when they fix what you need
_________________ For every complex problem, there is a solution that is simple, neat, and wrong.
Users browsing this forum: No registered users and 3 guests
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot post attachments in this forum
Login |
HOME
