Community discussions

 
Zod
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 91
Joined: Mon Apr 23, 2012 11:02 pm

RB1100 Drops packets when Queue Tree enabled ?

Tue Nov 26, 2013 2:50 pm

RB1100AHx2 drops packets when Queue Tree is enabled even if Queue is not used ??

Interface Ether 1 (10.1.1.1/24)
Private LAN Ether 5 (10.3.1.1/24)
Server LAN Ether 10 (10.3.3.1/24)

Create one Queue Tree entry on Interface 5. Parent=Ether5 (or Global-Out, Or Global-Total)
/queue tree
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=0 name=Ether5-QOS packet-mark="" parent=ether5 priority=1
I've used Ethernet-default Queue Type here but problem exists regardless of queue type used. Do NOT create any Mangles so No traffic should actually be going through this queue correct ?

Ping from any device on the Server LAN through Ether 10 to any device on either Ether 5 subnet and anywhere from 1 in 1000 to 5 in 1000 packets will be dropped.

With queue disabled = no loss
 ping interval=.050 count=1000 10.3.1.100   
sent=1000 received=1000 packet-loss=0% min-rtt=1ms avg-rtt=3ms max-rtt=28ms
With queue 'enabled' (still no Mangle used) and parent set for Global-Total
 ping interval=.050 count=1000 10.3.1.100  
    sent=1000 received=999 packet-loss=0% min-rtt=1ms avg-rtt=2ms max-rtt=36ms
With queue 'enabled' (still no Mangle used) and parent set for Ether5
 ping interval=.050 count=1000 10.3.1.100  
    sent=1000 received=999 packet-loss=0% min-rtt=1ms avg-rtt=3ms max-rtt=23ms
With queue 'enabled' (still no Mangle used) and parent set for Global-Out
 ping interval=.050 count=1000 10.3.1.100  
    sent=1000 received=999 packet-loss=0% min-rtt=1ms avg-rtt=3ms max-rtt=42ms
These are all 100Mbps FD connections. Problem does not exist if the Queue Tree entry is disabled. There are NO mangles involved.

Why is this ??
 
scampbell
Trainer
Trainer
Posts: 458
Joined: Thu Jun 22, 2006 5:20 am
Location: Wellington, NZ
Contact:

Re: RB1100 Drops packets when Queue Tree enabled ?

Tue Nov 26, 2013 9:56 pm

You have packet-mark="" in your queue configuration.

Try changing that to packet-mark="something" and see if that helps.
 
Zod
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 91
Joined: Mon Apr 23, 2012 11:02 pm

Re: RB1100 Drops packets when Queue Tree enabled ?

Tue Nov 26, 2013 10:01 pm

I will... This dropped packet problem does not seem to exist on a RB750G
 
Zod
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 91
Joined: Mon Apr 23, 2012 11:02 pm

Re: RB1100 Drops packets when Queue Tree enabled ?

Tue Nov 26, 2013 10:15 pm

No luck the RB1100 still has loss with queue tree regardless of packet mark
/queue tree
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=0 name=Ether5-QOS packet-mark=something parent=ether5 priority=1
 sent=1000 received=998 packet-loss=0% min-rtt=1ms avg-rtt=2ms max-rtt=19ms 
RB750G similar setup, no loss with queue tree enabled
/queue tree
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 \
    max-limit=0 name=ether1 packet-mark="" parent=ether1-To_DMZ priority=8
sent=1000 received=1000 packet-loss=0% min-rtt=1ms avg-rtt=2ms max-rtt=17ms 
 
scampbell
Trainer
Trainer
Posts: 458
Joined: Thu Jun 22, 2006 5:20 am
Location: Wellington, NZ
Contact:

Re: RB1100 Drops packets when Queue Tree enabled ?

Wed Nov 27, 2013 12:19 am

What version RoS and FW are you using on the RB1100AHx2 ?
 
rjickity
Member Candidate
Member Candidate
Posts: 212
Joined: Sat Jul 17, 2010 10:40 am
Location: Perth, Australia

Re: RB1100 Drops packets when Queue Tree enabled ?

Wed Nov 27, 2013 12:05 pm

i can confirm i'm seeing this on rb1100ahx2 with queue trees also.

getting between 1-5/1000 drops with tree's enabled. disable the trees and its fine.

ROS 6.6 and routerboot 3.02
 
Zod
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 91
Joined: Mon Apr 23, 2012 11:02 pm

Re: RB1100 Drops packets when Queue Tree enabled ?

Wed Nov 27, 2013 1:55 pm

I've confirmed it on ROS 5.21, 5.24 and 5.26. Using the most current firmware with each ROS. Currently ROS 5.26 Firmware 3.02. I have half a dozen 1100's in production. Same problem on any one of them. Problem does not exist on the RB750GL or on the RB450G. Just rebooted the one that I am testing on (Uptime was 192 days) and the problem continues.
 
guille4
Frequent Visitor
Frequent Visitor
Posts: 93
Joined: Tue Sep 29, 2009 9:23 pm

Re: RB1100 Drops packets when Queue Tree enabled ?

Wed Nov 27, 2013 5:23 pm

Same problem here. With packet marks in mangle and queue tree enabled I lost at least 5% of packets with no load. BUT if I pass more traffic in fordward acorss the bridge the losts packets up to 8%.

RB1100. ROS 6.6. Bridge and of course ip firewall activated.

This issue IS NOT present in other RBs.
 
Zod
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 91
Joined: Mon Apr 23, 2012 11:02 pm

Re: RB1100 Drops packets when Queue Tree enabled ?

Wed Nov 27, 2013 7:47 pm

Possible you are running RED or default small queueing and packet drops are by design or due to buffer over run ?

I'm seeing loss even with no packet marks in Mangle. There are No packets are going through the Queue Tree entry. Packets in and out on Queue entry = 0 but the RB1100 drops ~.01 to .2 % of packets unless the queue is disabled.

I'll try a ticket.
 
Zod
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 91
Joined: Mon Apr 23, 2012 11:02 pm

Re: RB1100 Drops packets when Queue Tree enabled ?

Thu Nov 28, 2013 3:30 am

I found this post http://forum.mikrotik.com/viewtopic.php?f=2&t=68995 started in Jan 2013 and last updated Oct 2013 that indicates there might be a cure by changing the interface queue type. I tried Global-total and that didn't work for me but maybe only-hardware will work ?

Maybe Mikrotik support will respond to the email I sent them today as this seems to be a repeatable problem affecting many users.
 
Zod
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 91
Joined: Mon Apr 23, 2012 11:02 pm

Re: RB1100 Drops packets when Queue Tree enabled ?

Thu Nov 28, 2013 4:07 pm

Following the tips in the linked post above did NOT resolve the problem

RB1100AHx2 is STILL dropping packets when QueueTree is enabled even when queue entry is NOT used.

There are NO Mangle rules on this box so no packets will match the packet-mark.

With queue tree entry disabled:
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=yes limit-at=0 max-limit=0 name=Ether5-test packet-mark=fake parent=ether5 priority=1 queue=only-hardware-queue
sent=1000 received=1000 packet-loss=0% min-rtt=0ms avg-rtt=0ms max-rtt=4ms
With queue tree entry enabled:
add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=Ether5-test packet-mark=fake parent=ether5 priority=1 queue=only-hardware-queue
 sent=1000 received=997 packet-loss=0% min-rtt=0ms avg-rtt=0ms max-rtt=4ms
This looks like a common problem. Many customers with the 1100AHx2 are NOT able use the device properly. And no response from support on the email I submitted....
 
scampbell
Trainer
Trainer
Posts: 458
Joined: Thu Jun 22, 2006 5:20 am
Location: Wellington, NZ
Contact:

Re: RB1100 Drops packets when Queue Tree enabled ?

Thu Nov 28, 2013 8:18 pm

Looks like it may well be an issue. I've got an RB1100AHx2 in production and if I get a chance I'll try and replicate your test :-) No promises though asI have several other jobs on.......
 
Zod
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 91
Joined: Mon Apr 23, 2012 11:02 pm

Re: RB1100 Drops packets when Queue Tree enabled ?

Thu Nov 28, 2013 9:34 pm

I have two more coming in today that I can bench test on... I need a Queue Tree solution for shaping so I'm hoping to find a work around but it would be REALLY nice is support would respond to email.
 
Zod
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 91
Joined: Mon Apr 23, 2012 11:02 pm

Re: RB1100 Drops packets when Queue Tree enabled ?

Fri Nov 29, 2013 2:17 pm

No response from Support.

RB1100AHx2 cannot be used as anything more than a simple router or a switch and this is not a serious problem ? This is easily repeatable.

Emailed again.
 
Zod
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 91
Joined: Mon Apr 23, 2012 11:02 pm

Re: RB1100 Drops packets when Queue Tree enabled ?

Fri Nov 29, 2013 3:32 pm

On the bench with new 1100AHx2 ROS 5.25 FW 3.02.

Reset configuration to defaults.

Setup subnets on two ports. I tried 1, 10 and 1,5 with the same results.
Connected a RB433AH on each side of the 1100AHx2.
Ping from one RB433AH through the 1100AHx2 to the other RB433AH with no mangle or QueueTree and 0 loss on 5000 packets

Started a Bandwidth test on one of the RB433AH, 20Mbps bidiretional, UDP. CPU on the 433AH's is not more than 30% with BW test running
Ping from one RB433AH through the 1100AHx2 with no mangle or queuetree and 0 loss on 5000 packets

Created a single Queutree entry in the 1100AHx2 using defaults plus packet mark "thisisnotused". Placed on any one of the interfaces being tested) results were similar regardless).

Ping from one RB433AH through the 1100AHx2 with no mangle or queuetree and >25 packets lost on 5000 packets sent
No packets are going through the QueueTree entry according to the stats. CPU's are at ~8% on 1100AHx2 and <40% on the RB433AH's

Created a mangle rule in prerouting chain and marked packets to direct them to the QueuTree entry.
Ping from one RB433AH through the 1100AHx2 with no mangle or queuetree and >100 packets lost on 5000 packets sent!!

Changed mangle rule to postrouting chain and marked packets to direct them to the QueuTree entry.
Ping from one RB433AH through the 1100AHx2 with no mangle or queuetree and >100 packets lost on 5000 packets sent!!

These are all brand new RB's from stock. Brand new cables. Conditioned power. I did this process a total of 16 times and the results are consistent.

QueueTree cannot be used on RB1100AHx2 !!

THIS IS A PROBLEM.
 
Zod
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 91
Joined: Mon Apr 23, 2012 11:02 pm

Re: RB1100 Drops packets when Queue Tree enabled ?

Sat Nov 30, 2013 2:53 pm

This problem exists from version ROS 5.12 FW 2.39 - which is as far back as I could downgrade due to the NAND changes at 5.11>5.12 I presume - right through to ROS 6.6

QueueTree does not work properly on the RB1100AHx2
 
Zod
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 91
Joined: Mon Apr 23, 2012 11:02 pm

Re: RB1100 Drops packets when Queue Tree enabled ?

Sat Nov 30, 2013 9:19 pm

Sent to support : Maybe they can find time to duplicate the problem in their lab.

------------------------------------------------------------------------------------
The only way that queuetree can be used on the RB1100AHx2 is on the global interface. If it is applied to a physical interface which passes traffic, even if it is NOT queueing ANY traffic, there will be packet loss. This loss is NOT due to drops in the queuetree - 'Dropped' counter does NOT increment.

Attached 4 supout files.

In all cases:
RB1100AHx2 Netinstall ROS 6.6 with current routerboot. Configured from default
- Ether1 2.2.2.1/24
Ether1 directly connected to RB433AH with 6' Cat5e cable

RB433AH Configured from default
-Ether1 2.2.2.2/24

Control Test
No other configuration on either device for control test
On RB1100AHx2:
- Bandwidth test configured for UDP, both directions, random data, 50Mbps local and remote.
- ping 2.2.2.2 timeout 50ms
Result NO packets lost

Case1 Mangle and queue on physical interface
Rebooted both RB433 and RB1100AHx2
created 1 mangle rule with packet mark fake
created 1 queuetree entry on ether1 with packet mark 'fake'
- Bandwidth test configured for UDP, both directions, random data, 50Mbps local and remote.
- ping 2.2.2.2 timeout 50ms
Result >3% packet loss on ping
Supoutfile = supout-mangleandqueue.rif

Case2 No Mangle, queue only on physical interface
Rebooted both RB433 and RB1100AHx2
Disabled and removed: 1 mangle rule with packet mark fake
Still exists: 1 queuetree entry on ether1 with packet mark 'fake'
- Bandwidth test configured for UDP, both directions, random data, 50Mbps local and remote.
- ping 2.2.2.2 timeout 50ms
Result 1% packet loss on ping
Supoutfile = supout-nomangleonlyqueue.rif

Case3 No Mangle, and No queue on physical interface
Rebooted both RB433 and RB1100AHx2
Disabled and removed: 1 mangle rule with packet mark fake (in case2 above)
Disabled and removed: 1 queuetree entry on ether1 with packet mark 'fake'
- Bandwidth test configured for UDP, both directions, random data, 50Mbps local and remote.
- ping 2.2.2.2 timeout 50ms
Result NO packet loss on ping
Supoutfile = supout-nomangleoorqueue.rif

Case4 Mangle All but No queue on physical interface
Rebooted both RB433 and RB1100AHx2
created 1 mangle rule with packet mark fake
Disabled and removed: 1 queuetree entry on ether1 with packet mark 'fake' (in Case3 above)
- Bandwidth test configured for UDP, both directions, random data, 50Mbps local and remote.
- ping 2.2.2.2 timeout 50ms
Result No packet loss on ping
Supoutfile = supout-mangleallnoqueue.rif
 
Zod
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 91
Joined: Mon Apr 23, 2012 11:02 pm

RESOLVED(?) : Re: RB1100 Drops packets when Queue Tree enabl

Wed Dec 04, 2013 12:44 am

Downloaded and installed 6.7

Recreated test bed and the issue appears to be resolved. Even with QueueTree Active (Mangles directing traffic to QueueTree) it is performing as expected.

In production with ~40Mbps live traffic running through 20 mangles and queuetree with 1 parent and 10 leafs, ping from an adjacent system; traffic is traversing the queuetree:

>ping count=5000 interval=00:00:00.3 xx.xx.xx.xx (outside IP of 1100AHx2)

sent=5000 received=4996 packet-loss=0% min-rtt=0ms avg-rtt=0ms max-rtt=25ms

Not Perfect but may be attributed to buffer overflow ?
 
Zod
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 91
Joined: Mon Apr 23, 2012 11:02 pm

Re: RB1100 Drops packets when Queue Tree enabled

Fri Dec 27, 2013 4:40 pm

Looks like I was mistaken about this being resolved. The problem still exists with the 1100AHx2 on 6.7. After a 12 days in production it started dropping ~2% packets for no reason. Reboot didn't fix it. Disabling all queues didn't fix it. Disabling All mangles didn't fix it. Rebooted it again with all the above disabled and it passes traffic 'normally' but with no traffic shaping rules in place.
 
guille4
Frequent Visitor
Frequent Visitor
Posts: 93
Joined: Tue Sep 29, 2009 9:23 pm

Re: RB1100 Drops packets when Queue Tree enabled ?

Sat Jan 18, 2014 12:07 am

Hi Zod, can you solve this issue?

I´m still waiting for a solution from MK, I didnt see anything until now... and I cant find it by my own experiments in my net lab.

The loss packet still present in Rb1100ahx2 with ROS 6.7.

:( :?

Thank you
 
Zod
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 91
Joined: Mon Apr 23, 2012 11:02 pm

Re: RB1100 Drops packets when Queue Tree enabled ?

Sat Jan 18, 2014 12:25 am

I first thought it was resolved in 6.7 but found out it is not after a couple weeks in production

I have three 1100AHx2 in production that are Running Mangles and they all drop packets when I enabled Mangle rules. They are all Internet facing in different cities. All on different upstream providers. 1 has proxy ARP and N:1 NAT on the Internet Interface. 1 is just N:1 NAT and 1 is native routing with PPPOE server running. They all do the same thing - latency spikes up to +200ms and packet drops when load goes above about 20Mbps with Mangles enabled. It's not a pipe problem.

I've been to busy to follow up on it, but it is a real problem. I do not have a solution except to disable Mangles.

Z
 
rjickity
Member Candidate
Member Candidate
Posts: 212
Joined: Sat Jul 17, 2010 10:40 am
Location: Perth, Australia

Re: RB1100 Drops packets when Queue Tree enabled ?

Wed Feb 05, 2014 1:25 pm

Zod do you see this in 6.9 still ? i am not able to upgrade the 1100's i have at the moment so i cannot check

Who is online

Users browsing this forum: MSN [Bot] and 110 guests