Guys,

Having a little issue with a fibre service for a customer that is being terminated on on a Rb2011.

A little background, The service is 10mbps/10mbps delivered via fibre to the providers NTU then Ethernet to the Rb2011, we are having issue with the upstream traffic not being able to get the full 10mbps.

If we use old faithful speedtest.net the service gets 10mbps down and between 2 - 3.5mbps up.

The provider has said on a cisco device this is what is required to resolve the issue:

policy-map EAS-SHAPE-10MB
class class-default
shape average 9800000 40000

Mikrotik:

name="eas-10m-out" parent=ether1-fibre packet-mark="" limit-at=9900k queue=ethernet-default priority=8 max-limit=9900k burst-limit=0 burst-threshold=0 burst-time=0s

We've translated this as best as possible in Queues however the service still isn't able to achieve the 10mbps from inside the network. We tested FTP directly from a file hosted on their mikrotik and it gets the full 10mbps straight off the bat!

This is driving me nuts - Any pointers?

Well, I don't see a reason for traffic shaping. To me it sounds a bit like a duplex problem between the RB2011 and the NTU...

Yes it does however this was the first thing we checked as the services are ordered as the Ntu interface set to 100 full duplex. We tested and with the interface set to auto it is worse.

If you force the PC connection to the routerboard to 10 Mb do you see any difference?

Hi MT Forum,

This post is a fairly significant one in the big picture of network supply/provision within Australia. The largest carrier in the country has recently implemented their MEF certified offering and as a result the EVC performance metrics are extremely 'strict'. They police the NTU and the ISP ingress aggregation port to ensure that ethernet service metrics are maintained. The result is if you exceed the interface parameters for the EVC the circuit will be clipped and TCP performance suffers greatly.

The service cmurrayis referring to is a 10Mbit/s Fibre access ethernet service that requires the following 'Cisco' shaping to adhere to the correct inbound rates etc.
The correct configuration in Cisco land is.
This results in perfect performance of a service in both directions when using a Cisco on each end. Obviously there are customers who choose a Mikrotik CPE and as a result the MT shapers/burst parameters need to function within the MEF standards else the service will not perform at the bandwidth levels they have subscribed.

I have tried significant tinkering with simple queues, queue trees, bfifo interface queues to try and emulate the behavior however the Mikrotik configuration does NOT translate directly. The MT queue methods all 'drop' traffic, which clips TCP and causes transfer rate issues. Packets need to be delayed/queued in order for them to fit inside the burst time/size increments of the EVC.

Can someone either provide a Queue configuration for a Mikrotik that will function with exactly the same parameters as the Cisco shaping configuration above or someone from Mikrotik comment on whether this is even possible or not.

With the growing prevalence of MEF standards, performance metrics and definitions, this will become increasingly pressing as an issue for the MikroTik community and we would like to work out a way for our customer base to continue using the feature rich RouterOS platform at their edge if they so choose.

Hi,

The wholesale provider did do this today and set the Ntu port to 10mbps full and we saw the upstream come up to a consistent 5mbps but still not the 10mbps

If a PC that is only connected to the routerboard at 10 Mbps and running speedtest isn't able to show upload close to 10 Mbps then it seems unlikely that any form of queuing on the routerboard is going to help much. speedtest.net uses TCP so as packets are dropped a proportion of the available bandwidth is given over to retries - that is what needs to be minimized. It would be worth testing right at the provider's demarc.

I'd appreciate a reply from the mikrotik guys Normis?

Try emailing support directly. The Mikrotik guys do not see every thread on the user forums.

Already 2 steps ahead of you

The reason I suggested checking at the demarc is that the Cisco shaping described shouldn't be exceeded significantly by a 10 Mbps source - not significantly enough to kill half the bandwidth.

The Cisco<>Mikrotik interfaces I deal with generally run far faster than that but I can set up a test if the matter isn't resolved quickly.

Ok thanks, I've asked support to reference this thread and reply if possible regarding the issue.

Do you happen to know what the round-trip time is the the site that speedtest.net was using? That might help rule some things in/out as regards TCP windowing.

Round trip is around 70ms

Customer site:
http://www.speedtest.net/result/2653117065.png

The RTT is definitely exacerbating the issue which is what is making it more frustrating.

The RTT latency will be 5ms in a month or so but for now, this link is direct across Australia WEST to EAST (yes, it is that big ).

just wondering have you resolved the issue.

i have configuration like this (as simple as it can be):

/queue simple add max-limit=10M/10M target=192.168.88.2

and i get 9.6M/9.5M using speedtest.net

Unfortunately not, I believe it is an issue with the type of connection and a set of standards the queues need to comply to.

I think the RTT is probably having a significant effect especially if there is any packet loss at all. Trying a UDP test would show what the link is doing without the effects of RTT and TCP windows.

UDP is 10mbit/s flatout, no problems.

The problem still seems to be that the shaper from the Mikrotik doesn't work with the metrics of the MEF EVC.

Specifying burst in the cisco command allows the shaper to function within the metrics and thus it works fine.

UDP is 10mbit/s flatout, no problems.

The problem still seems to be that the shaper from the Mikrotik doesn't work with the metrics of the MEF EVC.

Specifying burst in the cisco command allows the shaper to function within the metrics and thus it works fine.

The Cisco settings shown do *not* permit bursting above the CIR of 9.8 Mbps (less than 10 Mbps to account for overhead). You will see a similar effect on TCP tests with the Cisco settings on a Cisco router. At 3-4ms RTT I would expect to see the throughput at >9 Mbps but every 10s added to the RTT will be noticeable on the upload speed of a speedtest.net test. While 5 Mbps seems a touch low for 80 ms RTT (~7-8 Mbps would have been my guess) it depends on many factors and speedtest.net servers are anything but consistent.

Incidentally, the "shaping" implied by the Cisco config above *will* drop traffic once the queue buffers are full - as do comparable RouterOS queues. What this shaping *can* do is buffer a certain amount of traffic and then release it from the queue if the source drops below 9.8 Mbps. RouterOS queues can do the same. However if the source is constantly attempting to drive higher than 9.8 Mbps upload then both systems will be forced to drop traffic once the queue buffers are full.

Hi, I am assuming that you are talking about Telstra Ethernet Access services as they are particularly bad in dropping packets which exceed the maximum service speed, nothing like BDAS services for bursting but a lot cheaper so you need to learn to work with them, even on Cisco kit they can be tricky to get working right, we have spent countless hours with our partners and Telstra getting these to work right.

We have done a lot of work with these and resell them so it was worth our while to try and get it right, the only real option was to use very minimal bursting because the burst threshold is only 4ms and Mikrotik can't support that.

The settings which we find work the best are as follows, this example is for a 20M service, but settings can be adjusted accordingly, just use the on-boarding guide for reference values depending on the service you are installing.

Work out values like this....

Burst speed = L2 service speed x 0.98 (Max realistic L3 speed with overheads)
Burst time = 1s
Shaping Burst (EBS) = 320kbit/s (from Telstra on-boarding guide and is different for each speed and traffic class)
Limit At = Burst Speed - Shaping Burst
Max Limit = Burst Speed - Shaping Burst

So settings for a 20M service are as follows:
Burst Speed = 19600k
Burst Time - 1s
Shaping Burst = 320kbit/s
Limit At = 19280k
Max Limit = 19280k

We also use a PFIFO queue with a size of 200 packets which is just under the 320k EBS size.

Obviously you need this on each end of the vlan, but we find that this flat lines comfortably just a smidge under 20M which doesn't cause the carrier to drop packets for us

Good luck !

Regards
Paul

Paul,

Yes, you are correct - EthernetAccess. Thanks for your replies. What you're proposing makes sense and I will work with cmurrayis to test a couple of these.

Our's show 200,000cps for 20Mbit/s services as PREMIUM COS we don't utilise the standard CoS service types which show 320,000 like you suggest.

Hi,
A bit late, but perhaps useful for people finding this thread later.

Current Mikrotiks have a Bucket Size option in their Queue Settings.
We had this problem and were able to fix it using this option.
We set it to a small value 0.005, 0.01 also seemed ok in our instance

See https://wiki.mikrotik.com/wiki/Manual:H ... _Algorithm
for details.

Roger