Community discussions

 
gmsmstr
Trainer
Trainer
Topic Author
Posts: 940
Joined: Fri Jun 04, 2004 2:22 am
Location: St. Louis, MO
Contact:

VPLS 1500 MTU working then stops

Fri Feb 10, 2012 5:10 pm

A large network that is running VPLS has been working quite well. We currently are not worried about packet fragmentation, however, we can not push the 1500 byte MTU packets though the VPLS in some instances.

VPLS circuits are set to 1508 in MTU in case of VLAN information, the VPLS tunnels do come up. In most cases, we have had the VPLS tunnels up for several weeks or longer, then in the morning, we come in and have calls about customer issues. We check, the VPLS circuit is up, but the largest ping we can get though is 1442. In many cases we can simply remove the MPLS interfaces, and readd them, upon readding the VPLS comes back up and we can ping 1500 byte packets across the link.

The question is WHY does this stop working? We have tried BGP VPLS as well, but still 1442 pings only. Whats even more confusing, is we have another VPLS tunnel that goes THOUGH the affected site, and it does not have the MTU issue!

Suggestions?
Dennis Burgess, MCTCE, MTCNA, MCTCTE, MTCWE, MTCNIE, A+, N+, MCP, MTCSE Mikrotik Certified Consultant / Trainer
Need Mikrotik Support: http://www.linktechs.net -- Link Technologies, Inc.
-- Author of "Learn RouterOS: Second Edition"
 
User avatar
Bernardo
newbie
Posts: 27
Joined: Wed Nov 02, 2011 12:05 pm

Re: VPLS 1500 MTU working then stops

Fri Feb 24, 2012 8:34 pm

A large network that is running VPLS has been working quite well. We currently are not worried about packet fragmentation, however, we can not push the 1500 byte MTU packets though the VPLS in some instances.

VPLS circuits are set to 1508 in MTU in case of VLAN information, the VPLS tunnels do come up. In most cases, we have had the VPLS tunnels up for several weeks or longer, then in the morning, we come in and have calls about customer issues. We check, the VPLS circuit is up, but the largest ping we can get though is 1442. In many cases we can simply remove the MPLS interfaces, and readd them, upon readding the VPLS comes back up and we can ping 1500 byte packets across the link.

The question is WHY does this stop working? We have tried BGP VPLS as well, but still 1442 pings only. Whats even more confusing, is we have another VPLS tunnel that goes THOUGH the affected site, and it does not have the MTU issue!

Suggestions?
Sorry for not being of much help here, but I thought at least I should tell you that I see the same behavior for quite some time now.
On the one affected tower I even replaced the RB450G with an RB1100, and the problem was gone for months.
Thought I had it tackled. But now it is back. No changes have been made recently.
As you stated, tunnels to other sites, going through this one, are NOT affected.
Really strange.

saludos
Bernardo
 
Inssomniak
Member
Member
Posts: 326
Joined: Fri Apr 13, 2007 11:21 pm

Re: VPLS 1500 MTU working then stops

Mon Mar 05, 2012 5:11 am

Yes!

I have this problem too but only so far with 5.12 and 5.13

5.7 or earlier I never saw this. Had to downgrade.

A while can go by and everything is OK, but then suddenly the packets size that fits is suddenly smaller.
I couldn't ever find a way to hot fix it like you did.

Yes! I even have the same issue, packets will go THRU the router to one farther up and it works!

I saw this on a RB600 and RB493 so far.
 
Inssomniak
Member
Member
Posts: 326
Joined: Fri Apr 13, 2007 11:21 pm

Re: VPLS 1500 MTU working then stops

Thu Mar 08, 2012 4:25 pm

I had this problem again today on v5.8
 
User avatar
Bernardo
newbie
Posts: 27
Joined: Wed Nov 02, 2011 12:05 pm

Re: VPLS 1500 MTU working then stops

Fri Mar 09, 2012 9:34 pm

Still the same here.
The one affected RB1100 is on 4.17 right now, and the problem persists, but only on ether6.
Re-routing (OSPF) over to ether1 and problem is gone.

saludos
Bernardo
 
Inssomniak
Member
Member
Posts: 326
Joined: Fri Apr 13, 2007 11:21 pm

Re: VPLS 1500 MTU working then stops

Fri Mar 09, 2012 10:29 pm

It comes and goes for me, doing what was mentioned above, removing the MPLS interface and re-adding fixed it.

This was on an RB433 this time.

So it appears not to be specific to any build. mipsbe and ppc affected for me so far.
 
User avatar
mrz
MikroTik Support
MikroTik Support
Posts: 5913
Joined: Wed Feb 07, 2007 12:45 pm
Location: Latvia
Contact:

Re: VPLS 1500 MTU working then stops

Tue Mar 13, 2012 9:41 am

make sure you have added all interfaces used by label switching in "/mpls interface" menu, for some customers thatwas the problem, they for got to add one interface in one router in MPLS cloud.
 
Inssomniak
Member
Member
Posts: 326
Joined: Fri Apr 13, 2007 11:21 pm

Re: VPLS 1500 MTU working then stops

Sat Apr 14, 2012 1:34 am

This was all correct network wide.

It is definitely a bug.
 
WirelessDSL
newbie
Posts: 29
Joined: Thu Nov 24, 2011 12:43 pm
Location: Germany
Contact:

Re: VPLS 1500 MTU working then stops

Mon Aug 06, 2012 4:17 pm

Friday the same thing here.

Suddenly the VPLS-Interface stops working with 1500 Bytes. Checked it with a Customer. The Maximum was 1442 Bytes (+ PPPoE-Overhead) which goes through the tunnel.
Hardware between Customer and Core: 3xRouterboard 1100AHx2 and 1xRouterboard 1100AH (all with Version 5.14)

Double checked everything (all used Interfaces included in MPLS-Interface with MTU 1530).

Nothing has changed from our side. Only our carrier switched our leased line on another fibre-pair in the night before, so that the line was offline for about 3 hours and then the ospf-routing and mpls/vpls comes up again but with a smaller MTU within the VPLS.

We just started to implement a pure MTIK MPLS environement in our core network so please Mikrotik give us a hint how to solve this.
MTCNA MTCRE MTCTCE MTCUME MTCWE MTCINE
 
User avatar
mrz
MikroTik Support
MikroTik Support
Posts: 5913
Joined: Wed Feb 07, 2007 12:45 pm
Location: Latvia
Contact:

Re: VPLS 1500 MTU working then stops

Mon Aug 06, 2012 4:30 pm

Is VPLS tunnel running over your carriers network?
 
WirelessDSL
newbie
Posts: 29
Joined: Thu Nov 24, 2011 12:43 pm
Location: Germany
Contact:

Re: VPLS 1500 MTU working then stops

Mon Aug 06, 2012 5:48 pm

There are several leased lines from data center to locations over the carrier ethernet. It´s SDH and they say they give us a MTU-Size of 9600.
On friday as we noticed it, we made a ticket and they measured the line with no success.
All of our other leased lines work with VPLS 1500.
after the affected leased line is one mpls-router (with vpls-termination). this one works with 1500 bytes. After that there is a licensed radio to the affected routerboard which dont get the 1500 bytes.
There are three vpls-tunnels. I checked the mtu on all tunnels. one can handle 1446 bytes (including icmp-header), one with 1478 and one 1470. that´s really strange.

In the core there is vpls-terminator and from there are several vpls tunnels to the locations.

I readded the mpls interfaces, rebooted all routers to the customers and recreated the vpls tunnels.
MTCNA MTCRE MTCTCE MTCUME MTCWE MTCINE
 
WirelessDSL
newbie
Posts: 29
Joined: Thu Nov 24, 2011 12:43 pm
Location: Germany
Contact:

Re: VPLS 1500 MTU working then stops

Mon Aug 06, 2012 6:44 pm

sorry for double post.

the first vpls tunnel with 1446 is because of a vpn through which i checked with a server. so this is ok.

for the one with 1478 bytes it looks like this:

server -> vpls-bridge (rb1100AHx2) -> mpls router (rb 1100AHx2) -> vpls-bridge (rb1100AH) -> wireless ap -> wireless cpe

for the one with 1470 bytes:

server -> pppoe-server (mtu/mrru 1492) -> vpls-bridge (rb1100AHx2) -> mpls router (rb 1100AHx2) -> vpls-bridge (rb1100AH) -> dslam -> cpe

i know it´s hard to give some advice for this, but everything worked like a charm before and the other users in the forum noticed this strange behavior before.
MTCNA MTCRE MTCTCE MTCUME MTCWE MTCINE
 
WirelessDSL
newbie
Posts: 29
Joined: Thu Nov 24, 2011 12:43 pm
Location: Germany
Contact:

Re: VPLS 1500 MTU working then stops

Tue Aug 07, 2012 11:48 am

Update:

changed Routerboard at customer-site ->nothing changed
changed the middle Routerboard -> nothing changed

tested the MTU-Size more accurate:

1478 Bytes is what i get on all vpls-tunnels at the customer-site. In the middle between data center and customer-site i get 1500 Bytes on all vpls-tunnels, so the leased line is definitly ok.
Between the customer-site and the middle Routerboard is a licensed radio which is capable of jumbo frames so i think thats not the problem.

Now, where are my 22 Bytes :(

This is getting me crazy, because it worked before without any problems and suddenly it stops.
The other vpls-tunnels on the other sites work like a charm so i dont think the routerboard within the data center is the problem.

Any ideas?
MTCNA MTCRE MTCTCE MTCUME MTCWE MTCINE
 
User avatar
mrz
MikroTik Support
MikroTik Support
Posts: 5913
Joined: Wed Feb 07, 2007 12:45 pm
Location: Latvia
Contact:

Re: VPLS 1500 MTU working then stops

Tue Aug 07, 2012 12:11 pm

Between the customer-site and the middle Routerboard is a licensed radio which is capable of jumbo frames
What kind of radio is this.
 
WirelessDSL
newbie
Posts: 29
Joined: Thu Nov 24, 2011 12:43 pm
Location: Germany
Contact:

Re: VPLS 1500 MTU working then stops

Tue Aug 07, 2012 12:12 pm

It´s a NEC iPasolink 400 and on the other side a NEC iPasolink 200
MTCNA MTCRE MTCTCE MTCUME MTCWE MTCINE
 
WirelessDSL
newbie
Posts: 29
Joined: Thu Nov 24, 2011 12:43 pm
Location: Germany
Contact:

Re: VPLS 1500 MTU working then stops

Fri Aug 10, 2012 10:27 am

Next update.

Replaced all Routers on this line to the customers including the one in the data center. (now 3x RB1100AHx2 with RouterOS 5.19)
All routers are configured manually from the scratch.

Same problem. The VPLS-Tunnel to place 1 works with 1500 Bytes and the VPLS-Tunnel to place 2 only with 1478 Bytes.

This is not working:
VPLS_NOT_Working1.png
Now I changed the config in this way that I create a new VPLS-Tunnel between place 1 and place 2 and attach both tunnels to a new bridge.
The OSPF interface mode are set to PTMP on the used interfaces.

This is working:
VPLS_Working.png
So where is my mistake? Why is it working with two tunnels and a bridge between and not with one tunnel directly from the data center?
So the licensed radio doesn´t cause the problem. I checked it with the support of our dealer, it´s capable of jumbo frames.

In another part of the network a have the same constellation (3x RB 1100AHx2 with version 5.14 and 5.19 in a row). The only thing which is different is instead of a licensed radio there is a 5GHz Ubiquiti Link (with the new mac address allowing frames bigger than 1518 Bytes for MPLS). And there is no problem. Same config except the routing part. Between data center and place 1 there is ospf network mode ptmp and between place 1 and 2 there is nbma.


Thank you for some hints. This problems causes a lot of headaches :(
You do not have the required permissions to view the files attached to this post.
MTCNA MTCRE MTCTCE MTCUME MTCWE MTCINE
 
Inssomniak
Member
Member
Posts: 326
Joined: Fri Apr 13, 2007 11:21 pm

Re: VPLS 1500 MTU working then stops

Mon Aug 13, 2012 12:45 am

Yea this happened again for me but went long tine without seeing this bug.

It just quits, not changing any settings, just stops passing traffic, but only in one direction.
 
WirelessDSL
newbie
Posts: 29
Joined: Thu Nov 24, 2011 12:43 pm
Location: Germany
Contact:

Re: VPLS 1500 MTU working then stops

Mon Aug 13, 2012 12:26 pm

Mikrotik any suggestions on this issue?

Do you need more information?
MTCNA MTCRE MTCTCE MTCUME MTCWE MTCINE
 
Inssomniak
Member
Member
Posts: 326
Joined: Fri Apr 13, 2007 11:21 pm

Re: VPLS 1500 MTU working then stops

Mon Aug 13, 2012 3:32 pm

It was so bad for me this time that I couldn't fix it. I had to abandon it for now in favor of l2tp
 
User avatar
IPANetEngineer
Trainer
Trainer
Posts: 985
Joined: Fri Aug 10, 2012 6:46 am
Location: Jackson, MS, USA
Contact:

Re: VPLS 1500 MTU working then stops

Tue Aug 14, 2012 3:59 am

saw this post and thought I would throw my two pesos in as I do a lot of MPLS work in Cisco and Mikrotik.

Not sure if any of you guys have tried this, but if you want to do EoMPLS/VPLS and pass a 1500 byte IP packet between the two connected hosts on the CE routers, then the MPLS MTU (and hence the max L2MTU) has to be at least 1526 for untagged frames and 1530 for tagged frames. You should be able to ping with the fragment bit set at 1472 bytes to test passing a full 1500 byte packet (you lose 20 bytes for the IP header and 8 bytes for ICMP hence 1472)

Basically with VPLS you have to have additional mtu overhead because you are encapsulating a Layer 2 frame inside of an MPLS packet and it has to have an ethernet header to deliver once it gets there so there's 14 bytes plus a 4 byte VPLS header and the standard 8 bytes for MPLS....which gets you to 1526. So whatever the max L2 MTU is on your interface is, subtract 26 bytes and you will get the VPLS payload

Also, if you're setting different MPLS mtus on each router, then you can potentially run into issues. it's better to set them all to the lowest common denominator in the path.
Global - MikroTik Support & Consulting - English | Francais | Español | Portuguese +1 855-645-7684
https://iparchitechs.com/services/mikro ... l-support/ mikrotiksupport@iparchitechs.com
 
Inssomniak
Member
Member
Posts: 326
Joined: Fri Apr 13, 2007 11:21 pm

Re: VPLS 1500 MTU working then stops

Tue Aug 14, 2012 4:13 am

The problem we have with Mikrotik is that its ticking along fine for even months at a time, then quits, without change.

Yesterday it spread like a virus thru my whole network, 14 routers all stopped passing even a 1480 byte packet.

All routers set to the lowest supported MTU.
 
Inssomniak
Member
Member
Posts: 326
Joined: Fri Apr 13, 2007 11:21 pm

Re: VPLS 1500 MTU working then stops

Tue Aug 14, 2012 4:55 am

Perhaps I have been mis-understanding the MTU of my x86 router after doing some reading.

It reports an IP MTU of 1500, great but does not report an L2MTU and I read that you should assume then in this case the IP MTU is the L2MTU, and therefore my MPLS MTU is 1500.?

Ive always had it at 1512 (or at one time 1508).

I dont know what the max size of packets my switch can handle, its not mentioned anywhere (FS726T)

The lowest L2MTU routerboard I have on the network is 1522.
 
User avatar
IPANetEngineer
Trainer
Trainer
Posts: 985
Joined: Fri Aug 10, 2012 6:46 am
Location: Jackson, MS, USA
Contact:

Re: VPLS 1500 MTU working then stops

Tue Aug 14, 2012 7:18 am

An MPLS MTU of 1512 means that your router won't pass any more than 1512 bytes (including headers via MPLS)

So if you take 1512 and subtract the 26 bytes needed for VPLS, that leaves you with a payload of 1486 (1482 if you're using dot1q tags) which is 14 bytes short of being able to carry a standard 1500 byte IP packet without fragmenting it (fragmenting the packet is bad juju and you never want to do it if you can help it)

If the maximum L2MTU (which is a completely different setting than IP or MPLS mtu) is only 1522 bytes, then you won't be able to carry a 1500 byte IP packet over MPLS/VPLS. Likewise, all the Layer 2 links in the path need to have a 1526 or higher L2MTU.

As to the issue of MPLS forwarding stopping on all routers, here are some things i've seen in production:

1) CPU utilization - this can kill MPLS quickly if it tears down the routing adjacencies
2) Routing - MPLS relies on an IGP (typically OSPF) to build the LDP forwarding table, if the routing dies or something changes it, MPLS can go bye bye
3) MTU differences in the label switched path - if there is an LSR switching traffic that is on a lower MTU segment than the endpoints, it can cause issues.
4) Routing summarization - if you summarize routes with a low enough mask, it can cause issues with an LSR that sits in the summarized subnet. MPLS needs knowledge of specific subnets to build the LDP forwarding information base
Global - MikroTik Support & Consulting - English | Francais | Español | Portuguese +1 855-645-7684
https://iparchitechs.com/services/mikro ... l-support/ mikrotiksupport@iparchitechs.com
 
WirelessDSL
newbie
Posts: 29
Joined: Thu Nov 24, 2011 12:43 pm
Location: Germany
Contact:

Re: VPLS 1500 MTU working then stops

Tue Aug 14, 2012 12:45 pm

saw this post and thought I would throw my two pesos in as I do a lot of MPLS work in Cisco and Mikrotik.

Not sure if any of you guys have tried this, but if you want to do EoMPLS/VPLS and pass a 1500 byte IP packet between the two connected hosts on the CE routers, then the MPLS MTU (and hence the max L2MTU) has to be at least 1526 for untagged frames and 1530 for tagged frames. You should be able to ping with the fragment bit set at 1472 bytes to test passing a full 1500 byte packet (you lose 20 bytes for the IP header and 8 bytes for ICMP hence 1472)

Basically with VPLS you have to have additional mtu overhead because you are encapsulating a Layer 2 frame inside of an MPLS packet and it has to have an ethernet header to deliver once it gets there so there's 14 bytes plus a 4 byte VPLS header and the standard 8 bytes for MPLS....which gets you to 1526. So whatever the max L2 MTU is on your interface is, subtract 26 bytes and you will get the VPLS payload

Also, if you're setting different MPLS mtus on each router, then you can potentially run into issues. it's better to set them all to the lowest common denominator in the path.

Thanks for your reply.

As I´m using only RB1100AHx2 in the core the Layer 2 MTU on the interfaces is set to the defaults on 1598 Bytes. The path to the other routers is capable of Jumbo Frames (9600 Bytes).
All MPLS Interfaces are set to 1530 Bytes (tagged frames planned in the future). The VPLS is set to 1500 Bytes. It works all over the network except the part seen above.

I know that a ping must be 1472 Bytes + 28 Bytes for the header without fragmentation. This works within the network.
But in the one part there is only the chance to ping with 1478 Bytes (including the 28 Bytes overhead).

What I don´t understand:
Why is the VPLS-Tunnel isn´t working from core over another MPLS-Router to the customer site with 1500 Bytes. But if I create two VPLS-Tunnels (one from core to the firste router and another one from the first router to the customer site) and bridge them together it works with 1500 Bytes.
Last week everything was fine with the first constellation and then it suddenly stops working. This must be a bug. I didn´t change anything.

What I´ve done:
- Changed all routers from core to the customer site
- created a new configuration from the scratch
- updated all routers on the path from 5.14 to 5.19

Now with the two tunnels it works, but this isn´t a solution for me. Just a workaround.

@ Inssomniak

Your switch (FS726T) should support jumbo frames. This is what i read in the manual.
http://www.manualowl.com/m/Netgear/FS72 ... 78?page=25

With the x86 hardware i don´t know how the MTU behaves.
But as IPANet Engineer said. MPLS MTU 1512 Bytes is too low to get VPLS with 1500 Bytes. And also 1522 Bytes L2MTU is too low.
MTCNA MTCRE MTCTCE MTCUME MTCWE MTCINE
 
Inssomniak
Member
Member
Posts: 326
Joined: Fri Apr 13, 2007 11:21 pm

Re: VPLS 1500 MTU working then stops

Tue Aug 14, 2012 3:54 pm

Hmm my switch doesn't have that setting. :(. I did find that it was not on their list of supported list of jumbo frame capable switches.

As for not passing 1500 byte packets I was ok with it. I only wanted it to pass a 1480 byte packet. The VPLS tunnel just carried pppoe traffic with max MTU of 1480 bytes.

So if all I care about is 1480 bytes, I should be able to get away with an MTU of 1506 without fragmenting the pppoe traffic?

1510 if I had tagged traffic?

EDIT:

Ive been doing some experimenting since the total un-called-for collapse of my MPLS network a few days ago. Why it failed like a virus I dont know.

The ONLY way I can get MPLS to work properly between my core x86 router and a directly adjacent mikrotik router is if I set the MPLS MTU to 1500. Then it works, fragmenting or not.

So Im leading to believe the reported IP MTU of the gigabit interface in the x86 router is actually also the L2 MTU.

Increasing this value leads to this: http://forum.mikrotik.com/viewtopic.php?f=2&t=58329

EDIT: more testing.

The intel pro/1000 cards cant do it, their L2MTU is 1504 tops, any attempt to change the MTU in mikrotik crashes the router. This card is essentially useless for MPLS networks with mikrotik.
I took a 10 year old national semiconductor based FE trendnet card and it works fine, go figure.

I ordered another card, unfortunately its a realtek 8169 card and the wiki mentions it might lock up the router, but there isnt a lot of pickings for gigabit ethernet cards, I could not find a low profile Via based one, or a marvell based one which I read is not supporting well past 2.6.31 kernel.
 
User avatar
IPANetEngineer
Trainer
Trainer
Posts: 985
Joined: Fri Aug 10, 2012 6:46 am
Location: Jackson, MS, USA
Contact:

Re: VPLS 1500 MTU working then stops

Wed Aug 15, 2012 11:13 pm

That's too bad about your network card. MPLS is very dependent on MTU and is often the source of much heartache in MPLS networks. To do PPPoE you'll have to account for another 8 bytes of MTU overhead IIRC (2 bytes PP and 6 bytes PPPoE)

Baltic networks sells a MIkroTik router that is a multi-core and does jumbo frames called RouterMAXX. They work very well for MPLS.

http://www.balticnetworks.com/manufactu ... rotik.html
Global - MikroTik Support & Consulting - English | Francais | Español | Portuguese +1 855-645-7684
https://iparchitechs.com/services/mikro ... l-support/ mikrotiksupport@iparchitechs.com
 
Inssomniak
Member
Member
Posts: 326
Joined: Fri Apr 13, 2007 11:21 pm

Re: VPLS 1500 MTU working then stops

Wed Aug 15, 2012 11:21 pm

From what I understand though, no intel based pro/1000 card can work. How do you get that routerMAXX to work? (they are based on intel pro/1000 chips from what I read)

The 3 Intel NIC cards I have all support jumbo frames, but mikrotik either wont pass the large packets or when trying to raise the MTU just crashes the router.
 
User avatar
IPANetEngineer
Trainer
Trainer
Posts: 985
Joined: Fri Aug 10, 2012 6:46 am
Location: Jackson, MS, USA
Contact:

Re: VPLS 1500 MTU working then stops

Thu Aug 16, 2012 3:58 am

Do you have a link for the issue with raising the MTU and that cardset?

The L2MTU on that particular router can be set to 9214 and is what shows up in the router as the L2MTU. I'd have to do some further testing on that platform as I don't have one in front of me but have worked on them before.
Global - MikroTik Support & Consulting - English | Francais | Español | Portuguese +1 855-645-7684
https://iparchitechs.com/services/mikro ... l-support/ mikrotiksupport@iparchitechs.com
 
Inssomniak
Member
Member
Posts: 326
Joined: Fri Apr 13, 2007 11:21 pm

Re: VPLS 1500 MTU working then stops

Thu Aug 16, 2012 4:01 am

Yea a few posts above.

I have reproduced the problem on 2 separeate x86 mikrotik routers with 2 different intel pro/1000 chipsets.

The L2MTU is not reported at all.
 
User avatar
IPANetEngineer
Trainer
Trainer
Posts: 985
Joined: Fri Aug 10, 2012 6:46 am
Location: Jackson, MS, USA
Contact:

Re: VPLS 1500 MTU working then stops

Thu Aug 16, 2012 6:15 am

You might want to contact Baltic networks and ask them about the RouterMAXX line in regard to this specific issue. I know that I have implemented MPLS and L2 MTUs above 1504 without any issues on that gear, but I don't think i've tried a higher IP MTU.
Global - MikroTik Support & Consulting - English | Francais | Español | Portuguese +1 855-645-7684
https://iparchitechs.com/services/mikro ... l-support/ mikrotiksupport@iparchitechs.com
 
User avatar
nz_monkey
Forum Guru
Forum Guru
Posts: 1810
Joined: Mon Jan 14, 2008 1:53 pm
Location: Straya
Contact:

Re: VPLS 1500 MTU working then stops

Thu Aug 16, 2012 6:38 am

We have many implementations using Intel Pro1000 cards and 1530 byte and larger frames, they work brilliantly.
http://thebrotherswisp.com/ | Mikrotik MTCNA, MTCRE, MTCINE | Fortinet FTCNA, FCNSP, FCT | Extreme Networks ENA
 
Inssomniak
Member
Member
Posts: 326
Joined: Fri Apr 13, 2007 11:21 pm

Re: VPLS 1500 MTU working then stops

Thu Aug 16, 2012 3:20 pm

We have many implementations using Intel Pro1000 cards and 1530 byte and larger frames, they work brilliantly.
Well then how come mine dont work???
Are you also using MPLS VPLS too?
 
User avatar
nz_monkey
Forum Guru
Forum Guru
Posts: 1810
Joined: Mon Jan 14, 2008 1:53 pm
Location: Straya
Contact:

Re: VPLS 1500 MTU working then stops

Thu Aug 16, 2012 10:33 pm

Yes we are running vpls circuits between data centres. Each vpls tunnel is carrying vlan tagged packets. We are running RouterOS on Dell R210-II with Pro1000 quad port NIC and booting from Kingston 8GB SSD.

We have a lot of these in production all doing MPLS/VPLS with no issues.
http://thebrotherswisp.com/ | Mikrotik MTCNA, MTCRE, MTCINE | Fortinet FTCNA, FCNSP, FCT | Extreme Networks ENA
 
Inssomniak
Member
Member
Posts: 326
Joined: Fri Apr 13, 2007 11:21 pm

Re: VPLS 1500 MTU working then stops

Fri Aug 17, 2012 2:11 am

Yes we are running vpls circuits between data centres. Each vpls tunnel is carrying vlan tagged packets. We are running RouterOS on Dell R210-II with Pro1000 quad port NIC and booting from Kingston 8GB SSD.

We have a lot of these in production all doing MPLS/VPLS with no issues.
Does that network card report an L2MTU in Mikrotik?
 
User avatar
mrz
MikroTik Support
MikroTik Support
Posts: 5913
Joined: Wed Feb 07, 2007 12:45 pm
Location: Latvia
Contact:

Re: VPLS 1500 MTU working then stops

Fri Aug 17, 2012 9:37 am

If the card does not report L2MTU you must set MTU to desired value.
 
Inssomniak
Member
Member
Posts: 326
Joined: Fri Apr 13, 2007 11:21 pm

Re: VPLS 1500 MTU working then stops

Fri Aug 17, 2012 3:22 pm

If the card does not report L2MTU you must set MTU to desired value.
The cards do not report L2MTU and it cannot be manually set. The values are greyed out in winbox, and at the command line you can enter the command, but it does nothing.
 
User avatar
mrz
MikroTik Support
MikroTik Support
Posts: 5913
Joined: Wed Feb 07, 2007 12:45 pm
Location: Latvia
Contact:

Re: VPLS 1500 MTU working then stops

Fri Aug 17, 2012 4:58 pm

Again if L2MTU is not detected you must set MTU (Layer3 MTU) to desired value.
/interface set ether1 mtu=1530
 
Inssomniak
Member
Member
Posts: 326
Joined: Fri Apr 13, 2007 11:21 pm

Re: VPLS 1500 MTU working then stops

Fri Aug 17, 2012 5:01 pm

Again if L2MTU is not detected you must set MTU (Layer3 MTU) to desired value.
/interface set ether1 mtu=1530
Yea, but when I do that, and I try to put a larger packet thru that interface with MPLS and a VPLS tunnel, the router crashes.

Reproducible on 2 separate x86 machines with 2 separate types of intel pro/1000 cards.
 
User avatar
mrz
MikroTik Support
MikroTik Support
Posts: 5913
Joined: Wed Feb 07, 2007 12:45 pm
Location: Latvia
Contact:

Re: VPLS 1500 MTU working then stops

Fri Aug 17, 2012 5:03 pm

It is a driver problem. Try v6beta3 there should be newer drivers. If it crashes on beta3 too then get better ethernet card.
 
Inssomniak
Member
Member
Posts: 326
Joined: Fri Apr 13, 2007 11:21 pm

Re: VPLS 1500 MTU working then stops

Fri Aug 17, 2012 5:06 pm

It is a driver problem. Try v6beta3 there should be newer drivers. If it crashes on beta3 too then get better ethernet card.
Is beta 3 out? Can I get a copy?

I didnt think an intel pro/1000 GT card was a "bad" card. I dont know what to get thats better than that for a PCI bus.
 
User avatar
mrz
MikroTik Support
MikroTik Support
Posts: 5913
Joined: Wed Feb 07, 2007 12:45 pm
Location: Latvia
Contact:

Re: VPLS 1500 MTU working then stops

Fri Aug 17, 2012 5:10 pm

You can get beta3 if you contact support. Well the card is not bad, but it is quite new and drivers are not optimized in older kernel.
 
Inssomniak
Member
Member
Posts: 326
Joined: Fri Apr 13, 2007 11:21 pm

Re: VPLS 1500 MTU working then stops

Fri Aug 17, 2012 5:17 pm

You can get beta3 if you contact support. Well the card is not bad, but it is quite new and drivers are not optimized in older kernel.
Ok thanks emailed support.

I did order a realtek 8169, and I believe that does the job but comment in wiki scares me: "not recommended, may lock up router"
 
Inssomniak
Member
Member
Posts: 326
Joined: Fri Apr 13, 2007 11:21 pm

Re: VPLS 1500 MTU working then stops

Sat Aug 18, 2012 11:20 pm

I got in a RTL8169 and it worked fine. It reports an l2mtu 16383

Now Id like to make a call out to all that are using intel pro/1000 cards, that have the l2mtu properly reporting and have VPLS working with MPLS MTU=1526 --
And also that have pro/1000 cards that dont report l2mtu, that adjust the IP MTU past 1500 and have it working without crash.

Models, bus type, etc..

What I have that doesnt work: Pro/1000 GT PCI, Pro/1000 MT Dual port, PCI-X

Anyone else?
 
User avatar
nz_monkey
Forum Guru
Forum Guru
Posts: 1810
Joined: Mon Jan 14, 2008 1:53 pm
Location: Straya
Contact:

Re: VPLS 1500 MTU working then stops

Sun Aug 19, 2012 12:05 am

Pro1000PT Quad port PCIe

Why are you using old PCI card?
http://thebrotherswisp.com/ | Mikrotik MTCNA, MTCRE, MTCINE | Fortinet FTCNA, FCNSP, FCT | Extreme Networks ENA
 
Inssomniak
Member
Member
Posts: 326
Joined: Fri Apr 13, 2007 11:21 pm

Re: VPLS 1500 MTU working then stops

Sun Aug 19, 2012 12:35 am

Pro1000PT Quad port PCIe

Why are you using old PCI card?
I couldn't get the PCI-e card in the PCI slot, lol

Can u give me more details? Does it report l2mtu and you are running MPLS MTU 1526 or better?
 
User avatar
nz_monkey
Forum Guru
Forum Guru
Posts: 1810
Joined: Mon Jan 14, 2008 1:53 pm
Location: Straya
Contact:

Re: VPLS 1500 MTU working then stops

Sun Aug 19, 2012 9:13 am

I couldn't get the PCI-e card in the PCI slot, lol
Haha, fair enough ;)

Screenshot of a working deployment:
You do not have the required permissions to view the files attached to this post.
http://thebrotherswisp.com/ | Mikrotik MTCNA, MTCRE, MTCINE | Fortinet FTCNA, FCNSP, FCT | Extreme Networks ENA
 
Inssomniak
Member
Member
Posts: 326
Joined: Fri Apr 13, 2007 11:21 pm

Re: VPLS 1500 MTU working then stops

Sun Aug 19, 2012 2:43 pm

Hah Thanks! that image is impossible for me to reproduce in the lab without a router crash.


Anyone else?

Maybe pic-e cards are the difference.
 
User avatar
nz_monkey
Forum Guru
Forum Guru
Posts: 1810
Joined: Mon Jan 14, 2008 1:53 pm
Location: Straya
Contact:

Re: VPLS 1500 MTU working then stops

Mon Aug 20, 2012 2:50 am

Get a Dell R210-II with Xeon E3 CPU, Intel Pro1000PT Quad Port NIC (I350 should work in later 5.x), then just add a small 8GB SSD for RouterOS.

You should be able to get this configuration from Dell Direct for around USD $800 - $1000 depending on where you are located you can probably get same day warranty on it too.

This is known working config and is being deployed by quite a few people on these forums with success.


I would probably go for the new RouterBoard RB44Ge http://routerboard.com/RB44Ge instead of an Intel NIC on any new builds. This way if you encounter any issues Mikrotik will be more obliged to support you since you are using their hardware ;)
http://thebrotherswisp.com/ | Mikrotik MTCNA, MTCRE, MTCINE | Fortinet FTCNA, FCNSP, FCT | Extreme Networks ENA
 
Inssomniak
Member
Member
Posts: 326
Joined: Fri Apr 13, 2007 11:21 pm

Re: VPLS 1500 MTU working then stops

Mon Aug 20, 2012 6:01 am

Get a Dell R210-II with Xeon E3 CPU, Intel Pro1000PT Quad Port NIC (I350 should work in later 5.x), then just add a small 8GB SSD for RouterOS.

You should be able to get this configuration from Dell Direct for around USD $800 - $1000 depending on where you are located you can probably get same day warranty on it too.

This is known working config and is being deployed by quite a few people on these forums with success.


I would probably go for the new RouterBoard RB44Ge http://routerboard.com/RB44Ge instead of an Intel NIC on any new builds. This way if you encounter any issues Mikrotik will be more obliged to support you since you are using their hardware ;)
I think I might have to do this, further testing of a realtek card shows it can just quit working, only a disable and re-enable fixes it, I dont like that.
There is a local r210-II for sale here, more than I need (dual SAS 300 gig raid, core i3 chip, DRAC 6 enterprise), new for 800 bux no tax, I can probably strip out the drives and raid controller and sell them and stick in a small sata drive, but I just bought a controller level 6 license and it sits on an IDE drive, that was only a couple months ago and I dont want to buy a new one again :(
 
User avatar
nz_monkey
Forum Guru
Forum Guru
Posts: 1810
Joined: Mon Jan 14, 2008 1:53 pm
Location: Straya
Contact:

Re: VPLS 1500 MTU working then stops

Mon Aug 20, 2012 6:12 am

I think I might have to do this, further testing of a realtek card shows it can just quit working, only a disable and re-enable fixes it, I dont like that.
There is a local r210-II for sale here, more than I need (dual SAS 300 gig raid, core i3 chip, DRAC 6 enterprise), new for 800 bux no tax, I can probably strip out the drives and raid controller and sell them and stick in a small sata drive, but I just bought a controller level 6 license and it sits on an IDE drive, that was only a couple months ago and I dont want to buy a new one again :(
Maybe email support@mikrotik.com and see if they will do a licence transfer (doubtful, but worth a try)
http://thebrotherswisp.com/ | Mikrotik MTCNA, MTCRE, MTCINE | Fortinet FTCNA, FCNSP, FCT | Extreme Networks ENA

Who is online

Users browsing this forum: No registered users and 8 guests