Community discussions

 
cmurrayis
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 98
Joined: Fri May 15, 2009 4:31 am

Queues Cisco > Mikrotik

Tue Apr 16, 2013 1:59 am

Guys,

Having a little issue with a fibre service for a customer that is being terminated on on a Rb2011.

A little background, The service is 10mbps/10mbps delivered via fibre to the providers NTU then Ethernet to the Rb2011, we are having issue with the upstream traffic not being able to get the full 10mbps.

If we use old faithful speedtest.net the service gets 10mbps down and between 2 - 3.5mbps up.

The provider has said on a cisco device this is what is required to resolve the issue:

policy-map EAS-SHAPE-10MB
class class-default
shape average 9800000 40000


Mikrotik:

name="eas-10m-out" parent=ether1-fibre packet-mark="" limit-at=9900k queue=ethernet-default priority=8 max-limit=9900k burst-limit=0 burst-threshold=0 burst-time=0s

We've translated this as best as possible in Queues however the service still isn't able to achieve the 10mbps from inside the network. We tested FTP directly from a file hosted on their mikrotik and it gets the full 10mbps straight off the bat!

This is driving me nuts - Any pointers?
 
SwissWISP
Member Candidate
Member Candidate
Posts: 181
Joined: Fri Sep 23, 2011 12:16 pm

Re: Queues Cisco > Mikrotik

Tue Apr 16, 2013 12:12 pm

Well, I don't see a reason for traffic shaping. To me it sounds a bit like a duplex problem between the RB2011 and the NTU...
 
cmurrayis
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 98
Joined: Fri May 15, 2009 4:31 am

Re: Queues Cisco > Mikrotik

Tue Apr 16, 2013 1:05 pm

Yes it does however this was the first thing we checked as the services are ordered as the Ntu interface set to 100 full duplex. We tested and with the interface set to auto it is worse.
 
CelticComms
Forum Guru
Forum Guru
Posts: 1766
Joined: Wed May 02, 2012 5:48 am

Re: Queues Cisco > Mikrotik

Tue Apr 16, 2013 4:08 pm

If you force the PC connection to the routerboard to 10 Mb do you see any difference?
Interlynx | Networking and Information Security Consultants & Trainers | Email: routerlynx@gmail.com
BGP | EIGRP | OSPF | MPLS | Firewall | VPN | IPsec | Multicast | QOS | IPv4/6 | STP | VLAN | PON | AE | M2M | and more!

 
spite
just joined
Posts: 7
Joined: Sat Jun 16, 2012 5:23 am

Re: Queues Cisco > Mikrotik

Tue Apr 16, 2013 4:56 pm

Hi MT Forum,

This post is a fairly significant one in the big picture of network supply/provision within Australia. The largest carrier in the country has recently implemented their MEF certified offering and as a result the EVC performance metrics are extremely 'strict'. They police the NTU and the ISP ingress aggregation port to ensure that ethernet service metrics are maintained. The result is if you exceed the interface parameters for the EVC the circuit will be clipped and TCP performance suffers greatly.

The service cmurrayis referring to is a 10Mbit/s Fibre access ethernet service that requires the following 'Cisco' shaping to adhere to the correct inbound rates etc.
  Service-policy output: SHAPE-10MB

    Class-map: class-default (match-any)
      34643190 packets, 38824984971 bytes
      30 second offered rate 29000 bps, drop rate 0 bps
      Match: any 
      Traffic Shaping
           Target/Average   Byte   Sustain   Excess    Interval  Increment
             Rate           Limit  bits/int  bits/int  (ms)      (bytes)  
          9800000/9800000   5000   40000     0         4         5000     
The correct configuration in Cisco land is.
policy-map SHAPE-10MB
 class class-default
  shape average 9800000 40000 0
This results in perfect performance of a service in both directions when using a Cisco on each end. Obviously there are customers who choose a Mikrotik CPE and as a result the MT shapers/burst parameters need to function within the MEF standards else the service will not perform at the bandwidth levels they have subscribed.

I have tried significant tinkering with simple queues, queue trees, bfifo interface queues to try and emulate the behavior however the Mikrotik configuration does NOT translate directly. The MT queue methods all 'drop' traffic, which clips TCP and causes transfer rate issues. Packets need to be delayed/queued in order for them to fit inside the burst time/size increments of the EVC.

Can someone either provide a Queue configuration for a Mikrotik that will function with exactly the same parameters as the Cisco shaping configuration above or someone from Mikrotik comment on whether this is even possible or not.

With the growing prevalence of MEF standards, performance metrics and definitions, this will become increasingly pressing as an issue for the MikroTik community and we would like to work out a way for our customer base to continue using the feature rich RouterOS platform at their edge if they so choose.
 
cmurrayis
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 98
Joined: Fri May 15, 2009 4:31 am

Re: Queues Cisco > Mikrotik

Tue Apr 16, 2013 11:50 pm

Hi,

The wholesale provider did do this today and set the Ntu port to 10mbps full and we saw the upstream come up to a consistent 5mbps but still not the 10mbps
 
CelticComms
Forum Guru
Forum Guru
Posts: 1766
Joined: Wed May 02, 2012 5:48 am

Re: Queues Cisco > Mikrotik

Wed Apr 17, 2013 12:09 am

If a PC that is only connected to the routerboard at 10 Mbps and running speedtest isn't able to show upload close to 10 Mbps then it seems unlikely that any form of queuing on the routerboard is going to help much. speedtest.net uses TCP so as packets are dropped a proportion of the available bandwidth is given over to retries - that is what needs to be minimized. It would be worth testing right at the provider's demarc.
Interlynx | Networking and Information Security Consultants & Trainers | Email: routerlynx@gmail.com
BGP | EIGRP | OSPF | MPLS | Firewall | VPN | IPsec | Multicast | QOS | IPv4/6 | STP | VLAN | PON | AE | M2M | and more!

 
cmurrayis
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 98
Joined: Fri May 15, 2009 4:31 am

Re: Queues Cisco > Mikrotik

Wed Apr 17, 2013 9:39 am

I'd appreciate a reply from the mikrotik guys Normis?
 
CelticComms
Forum Guru
Forum Guru
Posts: 1766
Joined: Wed May 02, 2012 5:48 am

Re: Queues Cisco > Mikrotik

Wed Apr 17, 2013 10:39 pm

Try emailing support directly. The Mikrotik guys do not see every thread on the user forums.
Interlynx | Networking and Information Security Consultants & Trainers | Email: routerlynx@gmail.com
BGP | EIGRP | OSPF | MPLS | Firewall | VPN | IPsec | Multicast | QOS | IPv4/6 | STP | VLAN | PON | AE | M2M | and more!

 
cmurrayis
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 98
Joined: Fri May 15, 2009 4:31 am

Re: Queues Cisco > Mikrotik

Thu Apr 18, 2013 1:06 am

Already 2 steps ahead of you :)
 
CelticComms
Forum Guru
Forum Guru
Posts: 1766
Joined: Wed May 02, 2012 5:48 am

Re: Queues Cisco > Mikrotik

Thu Apr 18, 2013 6:02 am

The reason I suggested checking at the demarc is that the Cisco shaping described shouldn't be exceeded significantly by a 10 Mbps source - not significantly enough to kill half the bandwidth.

The Cisco<>Mikrotik interfaces I deal with generally run far faster than that but I can set up a test if the matter isn't resolved quickly.
Interlynx | Networking and Information Security Consultants & Trainers | Email: routerlynx@gmail.com
BGP | EIGRP | OSPF | MPLS | Firewall | VPN | IPsec | Multicast | QOS | IPv4/6 | STP | VLAN | PON | AE | M2M | and more!

 
cmurrayis
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 98
Joined: Fri May 15, 2009 4:31 am

Re: Queues Cisco > Mikrotik

Thu Apr 18, 2013 6:18 am

Ok thanks, I've asked support to reference this thread and reply if possible regarding the issue.
 
CelticComms
Forum Guru
Forum Guru
Posts: 1766
Joined: Wed May 02, 2012 5:48 am

Re: Queues Cisco > Mikrotik

Thu Apr 18, 2013 6:49 am

Do you happen to know what the round-trip time is the the site that speedtest.net was using? That might help rule some things in/out as regards TCP windowing.
Interlynx | Networking and Information Security Consultants & Trainers | Email: routerlynx@gmail.com
BGP | EIGRP | OSPF | MPLS | Firewall | VPN | IPsec | Multicast | QOS | IPv4/6 | STP | VLAN | PON | AE | M2M | and more!

 
cmurrayis
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 98
Joined: Fri May 15, 2009 4:31 am

Re: Queues Cisco > Mikrotik

Thu Apr 18, 2013 6:59 am

Round trip is around 70ms

Customer site:
http://www.speedtest.net/result/2653117065.png
 
spite
just joined
Posts: 7
Joined: Sat Jun 16, 2012 5:23 am

Re: Queues Cisco > Mikrotik

Thu Apr 18, 2013 8:02 am

The RTT is definitely exacerbating the issue which is what is making it more frustrating.

The RTT latency will be 5ms in a month or so but for now, this link is direct across Australia WEST to EAST (yes, it is that big ;) ).
 
User avatar
janisk
MikroTik Support
MikroTik Support
Posts: 6283
Joined: Tue Feb 14, 2006 9:46 am
Location: Riga, Latvia

Re: Queues Cisco > Mikrotik

Thu Apr 18, 2013 1:54 pm

just wondering have you resolved the issue.

i have configuration like this (as simple as it can be):

/queue simple add max-limit=10M/10M target=192.168.88.2

and i get 9.6M/9.5M using speedtest.net
 
cmurrayis
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 98
Joined: Fri May 15, 2009 4:31 am

Re: Queues Cisco > Mikrotik

Thu Apr 18, 2013 1:59 pm

Unfortunately not, I believe it is an issue with the type of connection and a set of standards the queues need to comply to.
 
CelticComms
Forum Guru
Forum Guru
Posts: 1766
Joined: Wed May 02, 2012 5:48 am

Re: Queues Cisco > Mikrotik

Thu Apr 18, 2013 2:19 pm

I think the RTT is probably having a significant effect especially if there is any packet loss at all. Trying a UDP test would show what the link is doing without the effects of RTT and TCP windows.
Interlynx | Networking and Information Security Consultants & Trainers | Email: routerlynx@gmail.com
BGP | EIGRP | OSPF | MPLS | Firewall | VPN | IPsec | Multicast | QOS | IPv4/6 | STP | VLAN | PON | AE | M2M | and more!

 
spite
just joined
Posts: 7
Joined: Sat Jun 16, 2012 5:23 am

Re: Queues Cisco > Mikrotik

Fri Apr 19, 2013 4:53 am

UDP is 10mbit/s flatout, no problems.

The problem still seems to be that the shaper from the Mikrotik doesn't work with the metrics of the MEF EVC.

Specifying burst in the cisco command allows the shaper to function within the metrics and thus it works fine.
 
CelticComms
Forum Guru
Forum Guru
Posts: 1766
Joined: Wed May 02, 2012 5:48 am

Re: Queues Cisco > Mikrotik

Sat Apr 20, 2013 10:07 pm

UDP is 10mbit/s flatout, no problems.

The problem still seems to be that the shaper from the Mikrotik doesn't work with the metrics of the MEF EVC.

Specifying burst in the cisco command allows the shaper to function within the metrics and thus it works fine.
The Cisco settings shown do *not* permit bursting above the CIR of 9.8 Mbps (less than 10 Mbps to account for overhead). You will see a similar effect on TCP tests with the Cisco settings on a Cisco router. At 3-4ms RTT I would expect to see the throughput at >9 Mbps but every 10s added to the RTT will be noticeable on the upload speed of a speedtest.net test. While 5 Mbps seems a touch low for 80 ms RTT (~7-8 Mbps would have been my guess) it depends on many factors and speedtest.net servers are anything but consistent.

Incidentally, the "shaping" implied by the Cisco config above *will* drop traffic once the queue buffers are full - as do comparable RouterOS queues. What this shaping *can* do is buffer a certain amount of traffic and then release it from the queue if the source drops below 9.8 Mbps. RouterOS queues can do the same. However if the source is constantly attempting to drive higher than 9.8 Mbps upload then both systems will be forced to drop traffic once the queue buffers are full.
Interlynx | Networking and Information Security Consultants & Trainers | Email: routerlynx@gmail.com
BGP | EIGRP | OSPF | MPLS | Firewall | VPN | IPsec | Multicast | QOS | IPv4/6 | STP | VLAN | PON | AE | M2M | and more!

 
pjulian
Member Candidate
Member Candidate
Posts: 267
Joined: Mon May 31, 2004 12:16 pm
Location: Sydney, Australia

Re: Queues Cisco > Mikrotik

Mon Feb 24, 2014 8:39 am

Hi, I am assuming that you are talking about Telstra Ethernet Access services as they are particularly bad in dropping packets which exceed the maximum service speed, nothing like BDAS services for bursting but a lot cheaper so you need to learn to work with them, even on Cisco kit they can be tricky to get working right, we have spent countless hours with our partners and Telstra getting these to work right.

We have done a lot of work with these and resell them so it was worth our while to try and get it right, the only real option was to use very minimal bursting because the burst threshold is only 4ms and Mikrotik can't support that.

The settings which we find work the best are as follows, this example is for a 20M service, but settings can be adjusted accordingly, just use the on-boarding guide for reference values depending on the service you are installing.

Work out values like this....

Burst speed = L2 service speed x 0.98 (Max realistic L3 speed with overheads)
Burst time = 1s
Shaping Burst (EBS) = 320kbit/s (from Telstra on-boarding guide and is different for each speed and traffic class)
Limit At = Burst Speed - Shaping Burst
Max Limit = Burst Speed - Shaping Burst

So settings for a 20M service are as follows:
Burst Speed = 19600k
Burst Time - 1s
Shaping Burst = 320kbit/s
Limit At = 19280k
Max Limit = 19280k

We also use a PFIFO queue with a size of 200 packets which is just under the 320k EBS size.

Obviously you need this on each end of the vlan, but we find that this flat lines comfortably just a smidge under 20M which doesn't cause the carrier to drop packets for us :-)

Good luck !

Regards
Paul
 
spite
just joined
Posts: 7
Joined: Sat Jun 16, 2012 5:23 am

Re: Queues Cisco > Mikrotik

Mon Feb 24, 2014 2:49 pm

Paul,

Yes, you are correct - EthernetAccess. Thanks for your replies. What you're proposing makes sense and I will work with cmurrayis to test a couple of these.

Our's show 200,000cps for 20Mbit/s services as PREMIUM COS we don't utilise the standard CoS service types which show 320,000 like you suggest.
 
rplant
just joined
Posts: 1
Joined: Fri Sep 29, 2017 11:42 am

Re: Queues Cisco > Mikrotik

Fri Sep 29, 2017 11:54 am

Hi,
A bit late, but perhaps useful for people finding this thread later.

Current Mikrotiks have a Bucket Size option in their Queue Settings.
We had this problem and were able to fix it using this option.
We set it to a small value 0.005, 0.01 also seemed ok in our instance

See https://wiki.mikrotik.com/wiki/Manual:H ... _Algorithm
for details.

Roger

Who is online

Users browsing this forum: No registered users and 86 guests