MikroTik

timberwolf

High quality stainless steel is non-magnetic.

timberwolf

The example "InterVLAN Routing by Bridge" has been updated:
https://wiki.mikrotik.com/wiki/Manual:I ... _Bridge.29

Thanks!

timberwolf

...snip...
Ker-blamo, you have a 2 VLAN network with an access port in each and a trunk port on ether4 to uplink to your network.

Thanks, will give it a shot.

timberwolf

How would one realize inter-vlan routing or generally vlan interfaces with the new bridge implementation?
Can't find anything on this except some emtpy placeholder topic in the Wiki.

timberwolf

Did anyone ever revisit or verify this with newer versions?
I had some issues with reordering with 6.34(if I'm not mistaken) which could be worked around by disabling one core in RouterBoot.
Didn't retry this later on, as I then shelved the RB850Gx2.

timberwolf

Fix for muticore packet reordering comes to mind, if this isn't already silently fixed. I didn't have the motivation of retesting this since a few months. Last time I stumbled upon this(6.34 I think) I disabled one core of the RB850Gx2 under test. Im referring to https://forum.mikrotik.com/viewtopic...

timberwolf

Problem with router cache owerflow caused by sending traffic over IPsec tunnels still persists in this release, despite the fact the Router Cache feature is turned off.

Ticket number #2015081766000633

In which constelation? Pleain IPSec or in combination with L2TP maybe?

timberwolf

As MikroTik products are more targeted to the (W)ISP sector
I am not sure if that's the case anymore.
...

Anyhow, they (MikroTik) are nevertheless not aimed at the DC sector.

timberwolf

As MikroTik products are more targeted to the (W)ISP sector, there is not much point in integrating a purely datacenter centric feature like OpenStack Neutron into the product line.

Just my two cent.

timberwolf

Cha0s Seems to be something about how the onfiguration system works, as I often came across the same behaviour during operation of a single version. Something breaks and is only fixable with config reset or netinstall. As I have no clue how the config system is implemented, I can only guess that so...

timberwolf

*) fixed async. ppp server;
PPTP Client seems broken after upgrade to 6.29.

Every few minutes it disconnects.Screenshot: http://prntscr.com/7apgti
This keeps on since yesterday's upgrade.

Working flawless here, 6.29 on client(RB450G)&server(x86) though.

timberwolf

RB750 after upgrade from 6.28 DHCP client on ether1 don't work - status "searching". Settings are default.
I downgraded to 6.28 - works fine.

Stumbled across this more then once. In my cases reinstalling with netinstall always solved the issue in case config reset didn't help.

timberwolf

Did anyone else notice the L2MTU reduction on RB450G? With switch-all-ports=no I previously had 1526 now its 1522 for ether1. Noticed this with 6.29rc22 and 6.29, not sure if it was reduced before those versions

timberwolf

Hi all, I couldn't find a positive report about anyone running (RouterOS) KVM instances inside a RouterOS system which itself is a KVM guest. I'm running multiple nested KVM instances for testing purposes using plain Ubuntu 12.04 LTS, so nested virtualization with KVM is definitely working in my env...

timberwolf

Noyo, why is a changed button location a bug?

timberwolf

As far as I know, you can't use any STP on a CRS unless you use software bridges, which you for sure don't wan't to.

timberwolf

Just to let you now, I couldn't get the lastest 6.15 netinstall working, used the 6.13 netinstall without any problems.
Of course local firewall was off in both cases.

timberwolf

Why no acl on the crs125? Sent from my SCH-I545 using Tapatalk CRS125 does not have hardware capability for ACL, CRS226 has it because of more advanced switch-chip. So L3 forwarding in hardware will also only be available for the CRS226, maybe? Any further info on the planned L3 support? Like to be...

timberwolf

Looks like a hardware defect to me.

timberwolf

Hello, I just would like some more configuration examples such as a basic L3 switch with inter-VLAN routing. The way how a VLAN interface interacts with the switch is pretty unclear for me. Thanks! AFAIK you can only route via the embedded CPU, so inter-VLAN routing is achieved by switching the aff...

timberwolf

GSXR You could try to netinstall the failing RB if you haven't done so already. I had many errors which could only be resolved by netinstalling the whole board, a configuration reset also didn't help in many cases. I guess this is somehow caused by the configuration system screwing up in the backgr...

timberwolf

GSXR You could try to netinstall the failing RB if you haven't done so already. I had many errors which could only be resolved by netinstalling the whole board, a configuration reset also didn't help in many cases. I guess this is somehow caused by the configuration system screwing up in the backgr...

timberwolf

What puzzles me is the RAM size. What do we need 512MB of RAM for when we AFAIK can't use Metarouters on it, because it is a dual core architecture?

timberwolf

I was hoping for some info from MT, as the board seems to be ready judging from the pictures in the presentation and brochure.
But somehow that seems to be asking for too much.

timberwolf

The Rack mount cases sold by Varia are very good, I used one of this cases fitted with a RB450G and an ALIX board, made a nice routing and generic x86 platform in 1U.

timberwolf

Price $129 HW acceleration HW acceleration? Where did you get this info? Cheers, Jan It was mentioned in another thread here: http://forum.mikrotik.com/viewtopic.php?f=3&t=78907#p410563 but you can also check the specs for CPU in this link, under accelerators you will find security engine: http...

timberwolf

Any more details about the RB850Gx2?
I started a thread here: http://forum.mikrotik.com/viewtopic.php?f=3&t=82684

timberwolf

Any more details about the RB850Gx2?
I started a thread here: http://forum.mikrotik.com/viewtopic.php?f=3&t=82684

timberwolf

I'm not talking about mechanical features rextended...
As barkas said, a block diagram including switch chip specs is what I'd like to see.

timberwolf

Hi MT Team, any more info you can reveal fpr this upcoming board? I really like the network interface design of the RB450G, especially the separatable first interface and the switch chip features. Im also using some VLAN tag rewriting functions of the AR8316 for some neat tricks. Any info on what to...

timberwolf

The device is in full production for thousands of people so no supout and further analysis is possible, this is a severe incident, email incidents like this gives calls out 24 / 7 with responce time 1 hour! I will now have to sit writing a report to upper management on what have happened. I really ...

timberwolf

Hi, I am missing certain features in the AR8327 switch ruleset which are present in the AR8316 ruleset. To be specific, matching on VLAN IDs and setting new VLAN IDs as an action. Both features are supported by the AR8327, at leat the datasheet states this in section "2.7.2 Action Definition&qu...

timberwolf

Cisco products only supports their own SFPs, other SFPs don't work.
But you may have a look here, maybe it helps: http://www.flexoptix.net

timberwolf

I'd say 20Mbit/s is okay for the small CPU in the RB750GL. Did you have a look on the CPU usage during such a file tranfer? Do you have encryption enabled?

timberwolf

A lot of stuff is possible in pure theory, however, until somebody has actually done it, there is no point in scaring people.

That wasn't my intention but stating "RouterOS can not be infected." isn't really better.

Anyway, I hope the report from gcs was bogus.

timberwolf

RouterOS can not be infected. ... Heard that one before. :lol: Even if you prohibit writing to disk, infected code can still be run from RAM. ;-) I would love to see some actual evidence for this. Evidence for what? That ROS can be infected or that it's possible to run code from RAM? ;-) Seriously ...

timberwolf

I highly doubt, that you will anything even near to 1Gbps, if the CRS design was done like all aother RBs with integrated switch chip, then there is a single 1G link from the switch chip to the CPU. So if you add all ports including the uplink to a software bridge you trying to push the traffic of ...

timberwolf

RouterOS can not be infected. ...

Heard that one before.

Even if you prohibit writing to disk, infected code can still be run from RAM.

timberwolf

/interface bridge filter add action=drop chain=forward in-interface=!ether1 out-interface=!ether1 I'd suspect this kills performance for the CRS, cause as soon as you use interfaces as non switch port, everything is done in software.... I suspect your right but if I still get close too 1gbps out of...

timberwolf

I put all interfaces in same bridge group then did this. /interface bridge filter add action=drop chain=forward in-interface=!ether1 out-interface=!ether1 Beautiful, yes ... I misunderstood you. I'd suspect this kills performance for the CRS, cause as soon as you use interfaces as non switch port, ...

timberwolf

A new routing engine was announced for v6, but this has been posponed till v7, if I recall this correctly.

timberwolf

so, currently there's no way to limit total amount of huge traffic on MikroTik devices?.. Queue tree - Interface HTB is the fastest one, HTB Global requires little bit more processing. regardless of whether htb is on interface or global, CCR cannot police/shape more than 600-800Mbps per queue inclu...

timberwolf

Won't help him, as the user changes his MAC. Like normis said, only thing is WPA2 setup or key change...

timberwolf

Yes, you should open a ticket for this in any case. MT might be able to suggest a workaround, I don't have a good idea, you can't pin a NIC IRQ to a CPU on the CCR/Tilear architecture, can you?

timberwolf

Looks like a classical concurrency programming error to me, and not a DHCP server error. I guess the ARP code isn't synchronized correctly.

timberwolf

My guess: They are working on a Routerboard based on a chip with hardware acceleration like the Cavium Octeon. That would me really nice :) I hope they do a better job than Ubiquiti with offloading tasks to the hardware. Actually ubnt did a nice job on it, at least by now. There were some problems ...

timberwolf

Have a look here http://wiki.mikrotik.com/wiki/Manual:Ma ... uterBoards

timberwolf

Won't help you much, especially if the user, which seems to be no noob, finds this theread.

timberwolf

A broadcast storm could easily congest the RB2011 CPU, this may cause your problem. However the real question is, what causes a broadcast storm in your network?

timberwolf

Like say there's normal forwarding (process switching), fast path (fast-switching), and then hardware fast path (CEF equivalent).

Just for accuracy, CEF is software fastpath on lower end devices and true hardware TCAM based forwarding on higher end devices.

timberwolf

Call it a teaser.

I was "teased" by the CRS as this option showed up a a little bit earlier then 6.5.

timberwolf

it is the same thing. you simply have an option to turn it off. it will turn on by default, if the mentioned conditions are met. however, if for some reason you need it off, there is the option I think you misunderstood the question, there is a aoption for "Fastpath" and one for "Hw....

timberwolf

I suspect it's not the software upgrade which is problematic but more their configuration system which goes haywire.
Many bugs I've experienced without an update could be solved with netinstall...

timberwolf

I never tried web caching on those systems as I was sure about the fact, that they are to weak for this anyway.

I would recommend using a cheap x86 box with lots of ram for caching. RB750, 450 and 493 are in my oppinion not suitable for this task.

timberwolf

Based on my experience the updates work best, when applied during a clean netinstall. I never had problems following this procedure, only when I updatet a running or configured system itself.

timberwolf

YES!!! And I ask you - stop releasing features (i do not need management Curiosity :lol: ), bug fix errors first, make a very stable product. I can't agree more! Either that or have two really seperated streams of development, not the current "we have a beta right before realease and that's it...

timberwolf

Well NAT requires SPI aka connection tracking. But until MT comments on what SPI really means, we both stumble in the dark.

Your web proxy doesn't use a microSD card for storage by any chance?

timberwolf

You read my mind. when SPI eating CPU .... I was looking on our Border Firewall for Attack Kind of Sessions. I found NONE. So in my experience SPI joust activate himself .... without any exterior reason. I believe is something wrong in software, or something missing in software. (software means 5.6...

timberwolf

SPI can stand for Serial Peripheral Interface, that would match to LCD/Touchscreen on the RB2011 or SDcard for units with an appropriate slot, maybe even for the system flash.
It may also be Statefull Packet Inspection...

timberwolf

Tilera is a quite experimental platform, you have to rewrite your whole software to take adavantage of the small cores. NIC drivers, IP stack and firewall implementation are only the tip of the iceberg. I don't know a single company which succesfully dealt with this type of software. The only one w...

timberwolf

... So anybody who expected to use the CCR in a production environment in such a short timeframe can't seriously been working in networking for a long time. Or just hasn't the slightest idea of concurrency or software design in general. To defend MT, they told everybody, that the CCR is still in de...

timberwolf

Tilera is a quite experimental platform, you have to rewrite your whole software to take adavantage of the small cores. NIC drivers, IP stack and firewall implementation are only the tip of the iceberg. I don't know a single company which succesfully dealt with this type of software. The only one wh...

timberwolf

I get my info from checking on which devices are layer 3 capable in ASICs vs which devices are layer 3 capable through a general purpose CPU. Look at the price differences between the two and you'll see that ASICs that route are NOT cheap. I will agree that the ASICs themselves aren't anywhere NEAR...

timberwolf

Well normis, I really was hoping that the CRS would have some basic L3 routing capacity in hardware. I don't expect MT to come up with a full featured L3 high performance switch with enterprise features.

timberwolf

Unfortunately, layer 3 routing ASICs are still.....super inflated in cost as one can make a LOOOOOOT of money off of them. That and they're pretty difficult to make. Sorry if I seem harsh, but where the hell do you get your infos from?! They aren't expensive anymore and also they aren't that diffic...

timberwolf

Thanks for the info mrz. Hardware routing recently got very affordable with devices in the 100-200$ range, but only with 3 port devices, so I hoped MT would come up with something similar.

timberwolf

Hi MT,

does the CRS support any form of hardware based routing for IPv4/IPv6? Or is everything which isn't L2 switching handled by the ROS software as we are used to from the existing boards.

Cheers,
Jan

timberwolf

Availabillity for distributors and customer is one thing, the absolute lack of any information since the official anouncement in the newsletter another. I do not know if it will translate properly, but one wise man sad: "it better to spend time doing "the thing", that to spend time t...

timberwolf

Availabillity for distributors and customer is one thing, the absolute lack of any information since the official anouncement in the newsletter another.

timberwolf

And still no sign of the once advertised Cloud Router Switch....

I guess the CCRs are binding to many development ressources for anything else to be done.

timberwolf

Well there is an admin login via web, just before your settings magically get changed...
You did notice this, didn't you?

timberwolf

Timberwolf is right..... OSPF issue is happening with us also...... getting flapped automatically. Downgraded to ROS 5.24... I couldn't narrow down the cause by now, but it seem to be caused by short lineflaps or packet loss. So be carefull on unstable links. I am guessing that its either an error ...

timberwolf

We are not talking about 50%, we are talking about a factor of 10x or more. BIG DIFFERENCE. If what you say were true, then a task which would normally take about 1 second without a MetaROUTER guest running would take roughly 2 seconds with a MetaROUTER guest. I'm seeing 10-15 seconds! -- Nathan I ...

timberwolf

NTP Client status isn't reflected in Winbox, always shows stopped although CLI states synchronized, 6.0 on x86

timberwolf

I'm running a testing setup with OSPF and BGP all over L2TP, with 5.25 no problems. With 6.0 OSPF is behaving quite strange with messages like may/21 20:27:24 route,ospf,info OSPFv2 neighbor 10.42.1.104: state change from ExStart to 2-Way may/21 20:28:05 route,ospf,info OSPFv2 neighbor 10.42.1.104: ...

timberwolf

New issue: presence of a MetaROUTER on 5.25 can cause host in certain cases to take much longer to complete tasks than it normally would. Example: I added a new MetaROUTER to a 450G that a customer had. Instantly, CPU on 450G shot up to 100% and stayed there until I disabled the MetaROUTER. Profile...

timberwolf

Nathan, just to be sure, you are talking about a OpenWRT MetaROUTER or a ROS MetaROUTER?
I'll have to try and reproduce this with a ROS MetaROUTER.

timberwolf

Just a quick update on the ROS MR front.
I didn't have a single freeze or other issue since I installed 5.25rc1 on my RB450G and RB2011.
Although I didn't conduct any further tests, this feels pretty stable as for now. I will try to do some more elaborate tests if my time allows.

timberwolf

I find it interesting, though, that in that thread, RB2011 are talked about as if they don't have any problems with MetaROUTER, even though I can easily reproduce the problem myself on my RB2011L-IN running 5.24. Different CPU? Im not into MIPS assembly but locking/pipelining might be a totally dif...

timberwolf

Thanks NathanA for linking to this thread, I too think this is a interesting topic. Lots of thanks to majbthrd , as it really seems that your minimal two to three instruction images helped MikroTik to finally get a grip on their MetaROUTER hypervisor code. Nathan, although I concur with you opinion,...

timberwolf

Still looking good for me too, will upgrade another system soon. Keeping my fingers crossed, but it looks very promising this time.

timberwolf

I started tests with 5.25rc1 on RB2011 a few hours ago, I am using the same test setup like before, so time will tell. It didn't hnag in the last 5h but I got that result before. So lets wait a little before we declare this fixed. Right now I was brave enough to also update a more important RB450G t...

timberwolf

please try these new buiilds: http://www.mikrotik.com/download/share/routeros-mipsbe-5.25rc1.npk http://www.mikrotik.com/download/share/all_files_5.25rc1.zip http://www.mikrotik.com/download/share/routeros-mipsbe-6.0rc14.npk http://www.mikrotik.com/download/share/all_files_6.0rc14.zip they contain ...

timberwolf

Hi Nathan,

thanks for also testing this. Using only MT Metarouter instances wasn't as bad as you described, but also practically unuasable. I also would have expected MT to test this...
And as this topic gets ignored by MT again...well, I won't repeat myself.

timberwolf

I wonder why I'm not seeing this issue so far? ... ... Full disclosure: I still distrust the interrupt-crazy health monitoring subsystem on the 450G, so even on 5.22 and 5.23, I have been ripping out voltage.ko on any systems we deploy.... Well that is because it is most likely a faulty ISR which k...

timberwolf

Has anyone tried to change power supply for RB2011UAS-2HnD-IN and tested its impact to MetaROUTER stability? Is it even possible to change it without losing warranty? This isn't even worth a try, as the MR issues clearly aren't powersupply dependent. But if you want to waste your time, like we all ...

timberwolf

I guess this topic is an general hold again? Mikrotik, care to comment on current status?

timberwolf

Hi all, before I take the step and open a bug with MikroTik support, I'd like to get forum feedback as it only concerns a testing setup and may therefore also be an user error. ;-) If I setup a L2TP link with mtu and mru set to 1452, I can pass packets(icmp ping) over that link with the expected siz...

timberwolf

Well some fresh optimism in this thread can't do any harm.

timberwolf

You can run KVM guests in ROS which istself is running as a VMware guest. Depending on your VMware product (ESXi, Workstation, Fusion) you have to enable nested virtualization. I can't dig up a link right now, but google should be useful given the keywords "nested virtualization" or "...

timberwolf

@payday please also refer to http://forum.mikrotik.com/viewtopic.php?f=15&t=60746
This is the place where you can provide help, we consolidated all the threads to this one.
Right now it's MTs turn.

@MT
Is it possible to lock down this thread, while pointing to the consolidated one?

timberwolf

For mor uptodate info please refer to: http://forum.mikrotik.com/viewtopic.php?f=15&t=60746
Metarouter isn't really stable, neither on MIPS-BE nor on PPC.

timberwolf

Metarouter freezes when loading the base system more than 40-50 per cent.
...

We are not even at the point, where Metarouter works in an idle situation. So I guess it's no suprise that a freezes with load.

timberwolf

@MT
Any christmas presents planned?

timberwolf

Ok bridge fastpath isn't supported in conjuction with VLAN interfaces. MT Support just got back to me about this.

timberwolf

Maybe , but who knows?

What kind of post is this supposed to be?

timberwolf

I have also noticed that short (about 10 seconds) freeze of whole device (not only MetaRouters), many times yesterday, when using console a lot. Traffic is not forwarded during that break and console does not respond. I monitor some hosts behind MT router. Normally they are all green and when freez...

timberwolf

I can confirm this on RB450G, RB750G, RB751G and RB2011. But I didn't notice any problems, maybe a display glitch?

timberwolf

Any chance for an answer? normis, sergejs, janisk, anyone?
Next step would be to ask support, but I thought this simple question could be answered on the fly in the forum.

timberwolf

I am still trying to get some more info about fastpath, especially if the bridge handler still allows bridge fastpath if the source and/or destination interface is a VLAN interface "below" a supported interfaces regarding this list: http://wiki.mikrotik.com/wiki/Manual:Fast_Path#List_of_Ro...

timberwolf

There is no change on mine, also. Problem persists.

Thanks for your verification effort!

timberwolf

Ok janisk here come the results for the 5.22 release version. RB450G Got a freeze after about 5 hours, this time 11,9seconds, affecting the host and both MR instances, so no significant change, just -1second(luck?). RB2011UAS Still the same strange behaviour I got with 5.22rc1. Host froze for about ...

timberwolf

Good sounds promising. I will retest with 5.22 as soon as it is available, as I happen to have some spare time at the moment.

timberwolf

Thank you for the additional information.
I have one question left for now, you talk about crashes, what about those freezes which I've even seen on a RB2011UAS?

timberwolf

atm - I'd say - less likely to happen. No problematic events on debug metarouter have happened for a long time. At le

Hmm "less likely", what's the developers point of view? No offense, but this sounds like "we have no idea if we did fix it", don't you agree?

timberwolf

Are you indicating, that you might have fixed it?

timberwolf

@MT / normis
Any feedback?

timberwolf

we are working on "fast path" for firewall too. we will publish more detailed tests in a few weeks
This is this
but tell me its not only for Cloud Core Router?

I don't think so

see http://wiki.mikrotik.com/wiki/Manual:Fast_Path

timberwolf

timberwolf - during weekend and national holiday i have generated quite a lot of useful stuff for devs to work on regarding metarouter.

Thanks, I must admit, right now I'm quite disappointed that RB2011 isn't working either.

timberwolf

I'm trying to repeat my question regarding fastpath support here:
@MT
Does the fastpath bridge handler still get utilized if using VLAN interfaces from a supported ethernet interface in the bridge?

timberwolf

Will it be fixed? Or am I doing something wrong?

To keep the answer short, we all hope it will be fixed anytime soon as it is an open issue since 2 years(if I'm not mistaken). And no I'am quite sure you aren't doing anything wrong.

timberwolf

Does the fastpath bridge handler still get utilized if using VLAN interfaces from a supported ethernet port in the bridge?

timberwolf

In the meantime I started conducting the exact same tests with a RB2011UAS-RM but with the following modifications: 1.) ROS 5.21 2.) Host an MRs use static IPs 3.) Host runs NTP client On first sight, I got the same freezes after just 4 hours but with differing packet losses and timeouts,to rule out...

timberwolf

EDIT: Merged post with next post.

timberwolf

Yes, firewall fastpath is planned as well.

Yes I understood. I was more thinking about you sharing some of you plans with us, like other features which may be planned beside firewall or maybe even which firewall functionality is planned.

timberwolf

Please don't just rely on the forum when reporting bugs. Also send an error report and supout to the MikroTik support.

timberwolf

Question for MikroTik Team:

Is there a Roadmap available for the advanced fastpath features? For example planned availabillity for fastpath firewall/acl and set of other planned features.

timberwolf

Hmm your are right with your 38k ticks, I made a mistake in my calculation. Never do math aber 10:00pm ;-) I will come back to you, as soon as get this 32bit counter thing figured out correctly. EDIT: Is there any chance, that you use a 32bit integer as a timestamp with microsecond accuracy? Cause w...

timberwolf

The connectors are to small to attache a simple wire, and without any deeper knowledge you wouldn't get teh impedance right anyway.
Sorry, you really must use an antenna, otherwise you will definitely damage your card within seconds(depending on output power).

timberwolf

off, to test some metarotuers. Thank you for details of your setup. I wish you luck. The bug seems to be related to the scheduling of the hypervisor I think. An overflow (maybe with interrupt) seems to "wake" the scheduler up after the normal tick has been somehow blocked. Your devs shoul...

timberwolf

how often these happen? It now happened once in 24h . can you describe configuration, mine was like this: 1 metarouter running with 1 dynamic vif and 1 static vif interface. both bridged with physical port. in network only usual broadcast chatter. No freezes so far. The complete config without any ...

timberwolf

I am sorry, but i have to report the 12 second hang as still NOT fixed. Still exactly the same behaviour like in 5.21rc2 and 5.21, whole system freezes for 12 seconds and resumes afterwards like nothing happened. Therefore I request MT to finally provide an image without hardware monitoring for test...

timberwolf

first ppc metarouter has to start work, and resolve issues there, maybe afterwards someone will make this cosmetic change.

Understandable

timberwolf

unfortunately due to how metarouter operates (one of my older posts in this thread) health monitoring values can be off the mark.

Yes I know, that's why me and other people recommended disabling health monitoring completely.

timberwolf

over last week a lot of work went into metarotuer on mips. It would be really nice if you could try and see if this still hangs/freezes for you. I ran tests over the weekend (55hours) and did not get eny problems. So if any of you can put it somewhere, that would be highly appretiated. Here the lin...

timberwolf

Bad part is - it took week to manifest itself. Anyway, "blackout period was those 12 seconds.

I took about 3-4 hours on my setup.

Did you have one or several guests running?

Two, as I wrote earlier.

timberwolf

about mips metarouter - i had a moment where response time was 12129.041ms so we will look into it. As we now know that there still is some issue that can happen rarely

Oh really? Of course there is an issue! What cause do you think I would have for making this up?!

timberwolf

Ok I see where this is heading, again.
Well that's it, I'm done with MetaROUTER, MikroTik do whatever you want. I'm not paid for testing.

timberwolf

Just got an 12second hang :-( The host and all MR(which seems logical) froze in that time. Still the same simple test setup with no load. EDIT: All ICMP requests got answered after this time, however. this happened once or you could make it happen more than once? Also, could you post build date and...

timberwolf

I see.. maybe those are spurious interrupts ? Could you test regular load as I said? If those interrupts were noise induced, maybe you can somehow get the same results without metarouter ? It is possible that removing the voltage.ko module actually means that interrupts on some gpio line(s) are mas...

timberwolf

I've startetd some simple tests right now. One thing I did notice immediatly: The health values now are a little crazy, my voltage reading now jumps from 12 to 14 to 20 Volts randomly and the temperature readings jump from 58 to 68 randomly ;-) janisk, could you provide a little insight on how the ...

timberwolf

Just got an 12second hang :-( The host and all MR(which seems logical) froze in that time. Still the same simple test setup with no load. EDIT: All ICMP requests got answered after this time, however. MT I beg you to give us the option to turn off health monitoring so we could at least try without v...

timberwolf

I've startetd some simple tests right now. One thing I did notice immediatly: The health values now are a little crazy, my voltage reading now jumps from 12 to 14 to 20 Volts randomly and the temperature readings jump from 58 to 68 randomly ;-) janisk, could you provide a little insight on how the h...

timberwolf

NathanA,

I admire your optimism, but lets wait a few days.

Lets hope the effects barkas is seeing aren't somehow related to MR fixes, but just the side product of some other RC1 changes.

Jan

timberwolf

Version 5.21 RC1 is this correct?
The last version also was RC1.

timberwolf

you should consider how many steps can be done when ICMP echo travels from one guest to another. And if packet is not processed in one time-slice (it arrives at the end) how long it can be delayed. Ok, then lets try some math: I have running 3 tasks: 1 host and 2 guests Each task gets arround 3,33m...

timberwolf

janisk Thank you very much for sharing those details with us. With this knowledge we shouldn't see that much jitter in an unloaded system if each task gets a 1/300s timeshare. Your devs might whant to check if the health monitoring ISR code disables interrupts globally and if there is a race conditi...

timberwolf

Thank you for your investigation Nathan, I can't afford time for tests right now.
This once again shows, that the core problem seems to be interrupt handling.

timberwolf

Fair enough, then they don't use tun/tap. As you say, getting <10ms latency with that would be impossible with HZ_100. But metarouter still needs to run inside a thread at least. That means it needs to sleep and can 'wake up' at 100Hz at most. If you send it icmp request just before it wakes up, th...

timberwolf

Well guy1with1mr1problems, some comments then: 1.) I am getting 1ms latency to a MR, which wouldn't be possible with 100Hz polling frequency at all. 2.) Ever heard of of interrupt controllers which might be able to trigger software interrupts? Have a look at the "active" interrupts when yo...

timberwolf

I got those 60ms while running two idling MR instances, most of the pings where at arroung 1-2ms. Also the host responded in about 1ms to external pings. I don't think this variance is caused by load but by the same problem which causes the system to freeze up. I'm still hilding my theorie that this...

timberwolf

my explanation is about variable ping times when ICMP echo is sent from MR to itself or other MR. Of course, 60ms is excessive, but as i understand, that happens rarely.

Well it happens frequently, 60ms is an extreme value but 14-30ms seem to happen quite often.

timberwolf

AFAIK since a lot of stuff is happening in OS all the time, and guest and host has to do these things all the time, so each gets time-slice when to do stuff. If ping comes in wrong time it can miss current time-slice and get to the other one, hence the deviation. Could be, but 60ms seems to big, I ...

timberwolf

Nathan, did you test if those freezes also happen when running ROS based MetaROUTERS? I didn't notice them by now, but I only run 2 ROS instances. EDIT: Ok, I can confirm your observation, I got random hangs of 4-5 seconds which recovered themself withou watchdog intervention. So I also conclude: mu...

timberwolf

I noted two things: 1.) The MR Console in Winbox freezes after a while with no impact on the MR itself. 2.) Ping roundtrip time is a little strange: -Host to MR is usually 1ms -MR to Host also usually 1m -MR to MR is usually 2ms -Sometimes all those roundtrip times goe up to 10,30 or sometimes even...

timberwolf

I started the tests yesterday at arround 18:00 CEST. They are still running(I hope).

timberwolf

I conducted some simple tests, so far it looks very promising. I did a very simple setup for now: 1 bridge in host with a single IPv4 address. 2 Metarouters, each with a dynamic interface on that bridge and single IPv4 address. I then pinged: - Each MR from the host in a Winbox Terminal - The host f...

timberwolf

Thanks for the update, I will test as soon as my time allows.

timberwolf

PS.: I just read this thread from the beginning...I had no idea MetaROUTER can affect the whole router. Does Atheros SoC have a MMU? Why is it possible to crash hypervisor with a guest? Because MetaROUTER isn't a hypervisor based virtualization environment. Judging from the patches, it's more like ...

timberwolf

Is anyone still believing that MikroTik has NOT given up on MetaRouter?

timberwolf

Thats why they talked about "soon" and are ignoring any request for comment in this thread.
Sure...

timberwolf

In 3 days exactly one month passed without any news on progress. So its relatively safe to assume that MT is back to the old habit of ignoring this problem altogether for maybe another year or so.
What a shame.

timberwolf

Lets not get to OT in this thread, which is about stabillity issues with the MetaROUTER feature itself.
Which by now still aren't fixed even on some PPC boards.

So MT staff, what about news? You claimed to be able to provide a fix "soon" on june 19.

timberwolf

MT claims to be close to a fix, have a look here: http://forum.mikrotik.com/viewtopic.php?f=15&t=60746

timberwolf

janisk
Any news? Are any fixes/changes implemented in ROS 5.18? The changelog doesn't mention MR related fixes, so I didn't try on my own.

timberwolf

hopefully soon enough you (obviously not days) will not need to do that any more to run MetaROUTER on RB450G

Any update or rough estimate you could give us?

timberwolf

For future readers: that means mounting the yaffs2 filesystem of router os (I used openwrt running from RAM booted over network.) and renaming the 'voltage.ko' module. This worked like a charm for some reason ! It is now running for about a day and requesting phpinfo() from an embedded apache + php...

timberwolf

currently rRB540G status is unchanged - there is unresolved problem. Ok, so you don't think it would be worth gathering some results from the field, with disabled hardware monitoring? RB2011 can be added here if you wish so. I have added similar setup as on RB1100AH for RB2011 without any packet lo...

timberwolf

Actually I assume yes, MIkrotik should IF they test test against new versions now, too. See, thep roblem is: this is the release candidate. When do you find out oyou have a problem - once the **it hits the fan (i.e. the release is out?). Sounds like a really piss the customer off attitude - IF this...

timberwolf

Now you tell me it is not valid to test RouterOS on a new coming Hyper-Visor generation? THAT Is a new one, seriously. In my eyes NOW is the time to nail that down and see who is at fault here - so that possibly a ticket can be opened with Microsoft. You assume that MikroTik tests against this new ...

timberwolf

Sanity Well first of all stop insulting people who try to help; the only one acting religious here is you. From my point of view you have two options: 1.) Stop using prerelease version of microsoft operating systems for production use. Then maybe one might be willing and able to help you. I assume ...

timberwolf

janisk
Two questions, should we include RB2011 in this thread and whats the status for RB450G/MIPSBE?

timberwolf

On PPC it somehow sounds like some receive DMA function goes crazy, maybe some setup error or memory overflow/allocation error? janisk Still no updates for MIPSBE/RB450G? How hard can it be to at least build a testing release for us to try and provide feedback to you? By now already two forum member...

timberwolf

ferywu
Thanks for your effort, but modifying npk files is outside the scope of this thread and not in MTs interest I guess.
We want an official supported MT solution not an unsupported work-around.

timberwolf

janisk What about packages for a RB450G with hardware monitoring(voltage.ko) disabled? I would invest some time to test those on my RB450G boards, maybe barkas would also try, AFAIK he has still a ticket open about this issue. You would then get some more feedback outside your own test setup. I agr...

timberwolf

So a short summary at this point in the thread. MIPSBE ------- We did various tests on RB450G boards, most of them were conducted by NathanA. The conclusions for this board so far are: 1.) The powersupply does have an influence but isn't the cause. 2.) Disabling hardware monitoring by a hack seems t...

timberwolf

I am always open to discussions, You can keep the topic title as you like, I just thought it was a better name. The goal of this technical forum is solve issues and keep to the technical aspects of networking. I did choose the title very carefully and the around 4000 views and 4 pages of posts seem...

timberwolf

I just received the following warning from normis: This is a warning regarding the following post made by you: viewtopic.php?f=15&p=319494#p319494 I'm sorry but the thread title is misleading, JanisK is trying to help, and the issue is nearly resolved now. As I am not allowed to answer this mess...

timberwolf

It's possible that the general interrupt servicer was actually written by the SoC manufacturer, or a contractor of the manufacturer, and not MikroTik directly...I could be wrong. But I note that the IRQ code for the RB500 SoC has a copyright notice by an embedded systems software company on it. Per...

timberwolf

First I would like to state, that I am not pleased by the fact that MikroTik changed the Subject line of this thread! EDIT: And I therefore restored the original subject. NathanA Great work, I think I know what you did. ;-) I am a little disappointed that you had to come up with this and that MT cou...

timberwolf

i will try locally without monitoring turned off, as it is not that easy to make npk with that change.

Sorry, I don't understand what you are saying. How could you try without turning monitoring off, when the test would be to actually turn monitoring off?

timberwolf

some time there where a question about what GPIO is - it is used for health monitoring (voltage and temperature) janisk Have you and the devs tried disabling all functions/drivers using GPIO, to see if it makes a difference? There have been some reports of strange voltage and temperature readings i...

timberwolf

So why not try to work on the problem? I totally agree with you on the potential of MR. But to keep my answer short, WE are not working on the problem, we are fumbling arround, providing information to MT without getting anywhere. And as long as I don't get any proof that MT is working on the probl...

timberwolf

NathanA , thank you very much for putting that much effort into this. I may try 400MHz@12V if my time allows, but I am not very tempted to do so at the moment. This is because, even if the behavior changes it still won't help me or others having problems with this board. The only who can contribute...

timberwolf

I will pull a couple more RB450Gs out of stock and replace the one I'm currently using with a different one, and see if stability varies from board to board. Thanks, last idea I got. :( Without any further words from MT, I by now doubt that I will ever run a MR on one of my RB450G boards, which is ...

timberwolf

lordzar I am a bit puzzled, you say you have MR running stable for a year, yet you talk about ROS 5.11 and 5.16. ROS 5.11 was released december 2011, ROS 5.16 just a few weeks ago. Yould please go into detail about this debugging feature a little more also? With all this workarounds, I wouldn't cal...

timberwolf

Ok, I can confirm the behaviour barkas is seeing, still reboots. :-( And that even with nothing more than pings running from the MR to the outside. So maybe barkas and I got boards which are more on the weak side than that from NathanA. That would be 2 boards for me and one for barkas that won't be ...

timberwolf

I had to rebuild my system located at the datacenter, so I just setup the RB450G located there with a very minimal configuration. The powersupply at this location outputs 12V to the RB450G and an Alix 2c3, the RB450G reports 13V. At this moment there is only a single MR running with one dynamic vif....

timberwolf

The SFPs datasheet might provide some insight, as there are modules which act as a media translator.
I didn't work with those, mainly with 1/10 Gigabit SX and LX SFPs and some "little" larger OC-768/STM-256 tranceivers(1OC768-POS-SR ).

timberwolf

describe weird. I have metarotuer running on RB2011L w/o weird problems. I was referring to the report of ropebih . As long as noone else reports in this issue, it might be a special problem for his configuration. I won't try it myself, as no RB2011 with enough RAM worth trying is available. Slight...

timberwolf

I disagree - SerDes have nothing to do with the actual link establishment between 2 modules, it kicks in only after. SerDes 8B/10B is the reason why 1,25Gbps SFP only handles 1Gbps of actual traffic 1.25 GBps is just the electrical bandwidth of the tranceiver electronics. 1) RJ45 modules can be 10/...

timberwolf

Link speed negotiation is up to transceivers not the cage implementation on the board. SerDes functionality and therefore linkspeed negotiation usually is the business of the host system, not the optical tranceiver. The SFP MSA doesn't specify any data rates AFAIK. So are you sure about this, any s...

timberwolf

When Metarouter on this new RB with a new generation CPU also acts weird, then maybe not everything is lost, because this more sound like a software issue then.
Or, what would be very unsatisfying, the powersupply or other hardware on this board is also insufficient for higher loads.

timberwolf

his sytem has ROS 5.6 and a single additional metarouter environment operating, with a non-ROS, OpenWRT system running our application in it. i do not remember an unexplained wathcog restart (or detected freeze) of this configuration. This particular environment seems quite stable for us. It has sh...

timberwolf

janisk
Thank you for the update. Please keep us posted.

since rb433AH and RB540G has the same CPU and one is supposed to crash and other is not

all Can you confirm this? I can't recall it exactly, but I think there where negative reports for RB433AH too.

timberwolf

This is what I'm concerned about, but even moreso, that it is only happening (so far) on the AH. Truly makes me wonder if there is something wrong with this specific AH. Has anybody ever tried replacing the stock heat pad on the CPU with something like Arctic Silver 5? Is it worth the hassle? I don...

timberwolf

sergejs janisk
Any news to report from the MT developers?

timberwolf

You mean this? http://img.routerboard.com/pricelist/272_hi_res.png Or you mean block diagram? I mean a block diagram, like we have for the RB450G switch "features" and topology. :-) Where did you see a block diagram for RB450G ? Sorry, I must have mixed that up somehow. I was thinking abo...

timberwolf

You mean this? http://img.routerboard.com/pricelist/272_hi_res.png

Or you mean block diagram?

I mean a block diagram, like we have for the RB450G switch "features" and topology.

timberwolf

The 5 Gigabit Ports and the SFP ports are connected to an external switch chip, Atheros 83xx?
My RB2011 tells that it has two switches: ar8327 and ar8227. Not sure how those are wired however.

I am still hoping that there will someday be a picture in the Wiki.

timberwolf

All of them are real interfaces! When I say "real interface" I usually talk about a dedicated MAC+PHY combination or NIC. Not necessarily the logical representation inside ROS. CPU has 1GBit channel to the 100Mbit interfaces, and one 1Gbit channel to the switch chip. So the CPU itself has...

timberwolf

you assumption is wrong, SFP and Gigabit ports (total of 6 ports) are all connected through switch chip. 100Mbit ports are connected directly to CPU. This is because of CPU design. Thank you for your answer, a few questions remain however. ;-) Ok so we have 6 Gigabit Ports on a switch chip with an ...

timberwolf

Looking at the PCB pictures, it seems like the FastEthernet ports are connected to a switch chip, and the GigabitEthernet ports directly to the CPU. Could you please outline the topology a bit more? Are those GigabitEthernet ports connected to dedicated CPU interfaces or an CPU internal switch? How...

timberwolf

the difference is - hardware watchdog will reboot device always, while software watchdog can lock up,.That is why all recent product series use hardware watchdog.

If I get this correct, then Nathans RB1100AH shouldn't have locked up completely, as he had the watchdog enabled, right?

timberwolf

...but recalling my initial tests with MR on this RB450G board, I got crashes every minute(!). ...how do the capacitors look on your board? I conducted the tests with the board fresh out of the bag, the caps all looked good, like one would expect for a brand new board. I can check them friday if I ...

timberwolf

running tests on RB450G - with 28V PSU and 800MHz and 680MHz it crashes if metarouter is enabled. running tcp BT to itself at full speed. Metarouter has static, dynamic and hardware port assigned. Older boards are working without any problem. Without MR, it ran without problems, even at 28V and 800...

timberwolf

janisk if I would know more about the dc/dc converter design and topology MT implemented at the RB450G, than I could possibly help. Otherwise it is impossible to come to any conclusion on the relationship of input voltage and the capacitors, as those should sit at the output side of the dc/dc conve...

timberwolf

That might be that they "don't know how to fix it", but I don't believe it is from a lack of effort on their part. Given the past and recent posts and their "tone" from janisk I really can't be sure about that. We don't know what they actually did, that's all one can say. I woul...

timberwolf

I would read my posts more carefully. What I said was that I suspected my PPC crashes on the 1100AH were due to me overclocking the CPU; I suspected this because the behavior of PPC when crashing/rebooting was different than what I was seeing on MIPS/450G. I have since put the 1100AH back down to t...

timberwolf

Thanks to all, for joining in on this topic! I am in a big project right now, so I can't contribute that much. As it shows MR isn't even stable on PPC and it seems Nathan is right about the powersupply. What still doesn't fit, is the fact that we can't get a RB crash with high load not related to MR...

Search found 274 matches