I have a RB3011 in a location with Cable Internet. We just had the location upgraded from 300mbit download and 20mbit upload to 400mbit download and 20mbit upload. Only the download speed was upgraded.
Over recent weeks I have done periodic speed tests and hit about 120mbit over WiFi. In past times I’ve been able to hit the full 300mbit and actually beyond up to about 350mbit. For some reason speeds haven’t quite been as great but I attributed this to just WiFi issues.
Today when we got the speed upgraded, we did some tests and I had hard wired my laptop to the router and only got an average of about 215mbit, 250mbit maybe at best.
I did a test by going directly to the modem and got full speed, in fact, hitting up to 480mbit down. Quite a difference!
I’m doing some tests I disabled some firewall rules which improved some, but not great.
I’m doing further tests I disabled all my mangle rules, and what do you know, I was able to hit the full 400+mbit behind the 3011.
Simply enabling one mangle rule, any rule, reduced the speed test to an average of about 215mbit.
The 3011 is well capable of beyond 200mbit. What is making the 3011 only perform at this lower speed?
The mangle rules are simply marking packets for our queue tree and QoS.
Why is this having such a huge impact on our speed?
While performing a speed test I’ve checked CPU Usage and it’s hardly ever at 30%. So much more power available.
The 3011 is on the latest software and firmware currently available.
We have about 5 VLANs with DHCP servers for each. Some firewall rules to drop packets between certain VLANs.
Other simple firewall rules to allow IPSec/LT2P.
Some other NAT rules for our PBX and web server.
Nothing super complicated.
Any help is appreciated.
-
-
vecernik87
Forum Veteran
- Posts: 882
- Joined:
Re: Severe Performance Drop RB3011
what kind of speed test is it? If it is TCP based, how many paralel streams/connections? If only one, it might be the issue as such test is strongly affected by latency. Adding mangle rule is going to introduce slight delay as the packet must be processed in another block of code.
Just to make sure - do you have fasttrack rule set up?
If you share your /export hide-sensitive it might be easier to spot any possible issue straight away. Feel free to use find&replace in your favourite text editor to systematically replace all occurrences of each public IP address potentially identifying you by a distinctive pattern such as my.public.ip.1. (credit for this sentence goes to sindy)
Just to make sure - do you have fasttrack rule set up?
If you share your /export hide-sensitive it might be easier to spot any possible issue straight away. Feel free to use find&replace in your favourite text editor to systematically replace all occurrences of each public IP address potentially identifying you by a distinctive pattern such as my.public.ip.1. (credit for this sentence goes to sindy)
Re: Severe Performance Drop RB3011
The test is SpeedTest.net. I’m not sure if they are UDP or TCP.
Regardless of this though, the latency doesn’t change when I enable/disable the mangle rules (we’re only talking about less than 10 of them in total).
Latency remains real well at about 9-14ms depending on the test.
Overall throughout changes big time.
With rules disabled I get all we are allowed to get at about 480mbit down.
With even just one mangle rule enabled a huge drop to about 215mbit.
Both times latency is excellent and seemingly unaffected.
I disabled fastrack in IP -> Settings last night when I was trying some stuff but it made no difference. I don’t have any fastrack rules either, though.
I’m not at my computer right now but I will see what I can do about piston the config.
Regardless of this though, the latency doesn’t change when I enable/disable the mangle rules (we’re only talking about less than 10 of them in total).
Latency remains real well at about 9-14ms depending on the test.
Overall throughout changes big time.
With rules disabled I get all we are allowed to get at about 480mbit down.
With even just one mangle rule enabled a huge drop to about 215mbit.
Both times latency is excellent and seemingly unaffected.
I disabled fastrack in IP -> Settings last night when I was trying some stuff but it made no difference. I don’t have any fastrack rules either, though.
I’m not at my computer right now but I will see what I can do about piston the config.
Re: Severe Performance Drop RB3011
They are multi-stream TCP, testing the download direction first using four streams and then the upload one using another four streams.The test is SpeedTest.net. I’m not sure if they are UDP or TCP.
This is due to one type of optimization - if there are no rules at all in the firewall, the firewall processing is skipped completely.With rules disabled I get all we are allowed to get at about 480mbit down.
Fasttracking is another kind of optimization, where you skip most of firewall processing in a controlled way for most mid-connection packets, so only the packets establishing the connection and every n-th mid-connection packet are handled by all stages of the firewall. Without fasttracking, the CPU may be insufficient to handle the traffic, depending on the RB model. The bad news is that fasttracking is incompatible with mangling (and IPsec policy matching) but the other way round - setting up a mangle rule does not disable fasttracking for all (which is good), you just get unexpected behaviour if you use both without taking additional measures. The correct way to make the two coexist is described here; if the /ip route rules with their limited number of match conditions are sufficient to cover your policy routing needs, you can use them instead of mangle rules and fasttracking will still work without any extra measures to take.With even just one mangle rule enabled a huge drop to about 215mbit.
I disabled fastrack in IP -> Settings last night when I was trying some stuff but it made no difference. I don’t have any fastrack rules either, though.
Re: Severe Performance Drop RB3011
Thank you Sindy. I will take a look at that and do some testing.
I'm still not sure though why the performance is dropping so badly. CPU usage is not above 30% during that test. Even if the CPU was processing all those packets, there is plenty of processing power available. I would expect slow downs if the CPU usage was at 100%, right? Please correct me if my understanding of this is incorrect.
At the end of the day, I am trying to use the mangle to mark packets from specific groups of nodes within the network to prioritize and shape bandwidth.
My understanding to do this correctly is:
Add mangle rule to mark connections FROM IP (Upload traffic), passthrough = yes
Add mangle rule to mark packets matching connection above, passthrough = no
Add mangle rule to mark connections TO IP (Download traffic), passthrough = yes
Add mangle rule to mark packets matching connection above, passthrough = no
Then going to the queue tree, lets say we're starting from scratch, we add:
Global Upload queue
Global Download queue
Then we add:
Upload queue for marked packets from above (Of course setting any max limits etc) to the Global Upload queue
Download queue for marked packets from above (Of course setting any max limits etc) to the Global Download queue
Essentially this method works great. This way I can assign a priority accordingly as well as any bandwidth limitations.
In the same manner for our public WiFi which is on a separate VLAN, I mark connections from the public IP range, say 172.31.0.0/24, for example. Then mark their packets from that connection.
The queue is similar as described earlier but I now have the bandwidth throttle for the entire group to have a max and set it to a PCQ type of queue.
What I have described here is all I am trying to accomplish but unfortunately lose half our available bandwidth. So I'm trying to figure out why. There is nothing I can think of that should be affecting bandwidth this much, right? Or are the 3011's just not as good as I think they are?
I am considering a complete factory reset and starting over. It seems that something is just not right here. I appreciate your help in trying to dig in to this.
I'm still not sure though why the performance is dropping so badly. CPU usage is not above 30% during that test. Even if the CPU was processing all those packets, there is plenty of processing power available. I would expect slow downs if the CPU usage was at 100%, right? Please correct me if my understanding of this is incorrect.
At the end of the day, I am trying to use the mangle to mark packets from specific groups of nodes within the network to prioritize and shape bandwidth.
My understanding to do this correctly is:
Add mangle rule to mark connections FROM IP (Upload traffic), passthrough = yes
Add mangle rule to mark packets matching connection above, passthrough = no
Add mangle rule to mark connections TO IP (Download traffic), passthrough = yes
Add mangle rule to mark packets matching connection above, passthrough = no
Then going to the queue tree, lets say we're starting from scratch, we add:
Global Upload queue
Global Download queue
Then we add:
Upload queue for marked packets from above (Of course setting any max limits etc) to the Global Upload queue
Download queue for marked packets from above (Of course setting any max limits etc) to the Global Download queue
Essentially this method works great. This way I can assign a priority accordingly as well as any bandwidth limitations.
In the same manner for our public WiFi which is on a separate VLAN, I mark connections from the public IP range, say 172.31.0.0/24, for example. Then mark their packets from that connection.
The queue is similar as described earlier but I now have the bandwidth throttle for the entire group to have a max and set it to a PCQ type of queue.
What I have described here is all I am trying to accomplish but unfortunately lose half our available bandwidth. So I'm trying to figure out why. There is nothing I can think of that should be affecting bandwidth this much, right? Or are the 3011's just not as good as I think they are?
I am considering a complete factory reset and starting over. It seems that something is just not right here. I appreciate your help in trying to dig in to this.
Re: Severe Performance Drop RB3011
Sadly, queueing is incompatible with fasttracking as well, for obvious reasons. So if you need to shape the traffic, pour in more horsepower, which means to replace the 3011 with something that can handle the requirements.
Re: Severe Performance Drop RB3011
What would you suggest?
Going off of MikroTik's test results I figured the 3011 would be more than sufficient:
Their test in routing with 25 ip filter rules shows a result of 2,453.1 Mbps. Granted, this is not with queuing. But is queuing really this hard on the available horsepower?
Would Simple Queuing be less intensive? I don't like it though since I can't use address lists. But if thats the solution then I'd have to make it work.
Going off of MikroTik's test results I figured the 3011 would be more than sufficient:
Their test in routing with 25 ip filter rules shows a result of 2,453.1 Mbps. Granted, this is not with queuing. But is queuing really this hard on the available horsepower?
Would Simple Queuing be less intensive? I don't like it though since I can't use address lists. But if thats the solution then I'd have to make it work.
Re: Severe Performance Drop RB3011
I've had my eye on the 4011, sadly it's not quite available just yet.
Re: Severe Performance Drop RB3011
For test results that approximate real world performance, look at 25 ip filter rules with 512 byte packet size (not 1518 byte packet size). With those test results, the 3011 is only capable of 836.0 Mbps with essentially a default config plus maybe a few more rules. Adding more rules etc are going to drop that rate from the 836 Mbps figure even lower.Going off of MikroTik's test results I figured the 3011 would be more than sufficient:
Their test in routing with 25 ip filter rules shows a result of 2,453.1 Mbps. Granted, this is not with queuing. But is queuing really this hard on the available horsepower?
Re: Severe Performance Drop RB3011
But his problem is getting only 120 Mbps - about one eight of the 512 byte/ 25 ip rules scenario. I know one can always use something CPU heavy, and get this result. But seems... excessive, in this case. Don't You think?For test results that approximate real world performance, look at 25 ip filter rules with 512 byte packet size (not 1518 byte packet size). With those test results, the 3011 is only capable of 836.0 Mbps with essentially a default config plus maybe a few more rules. Adding more rules etc are going to drop that rate from the 836 Mbps figure even lower.Going off of MikroTik's test results I figured the 3011 would be more than sufficient:
Their test in routing with 25 ip filter rules shows a result of 2,453.1 Mbps. Granted, this is not with queuing. But is queuing really this hard on the available horsepower?
Re: Severe Performance Drop RB3011
That is exactly my point. Adding one simple mangle rule throws everything off.
I’ve had better results using an older 450G! Although I did use Simple Queues back then.
Something just isn’t right but I can’t put my finger on it. CPU usage never really goes above 30%.
I’ve had better results using an older 450G! Although I did use Simple Queues back then.
Something just isn’t right but I can’t put my finger on it. CPU usage never really goes above 30%.
Re: Severe Performance Drop RB3011
I think i’m just going to have to do a factory reset and start over.
Re: Severe Performance Drop RB3011
I have a 3011 and do not have this issue. Can you export your mangle rule? I can see if I can test it.
Re: Severe Performance Drop RB3011
A single mangle rule may just activate some heavy processing which is not done without that rule, so yes, without seeing the actual configuration and pinpoint to that rule it is hard to guess.
Plus if it is as you describe, there is no guarantee that the 4011 will not fall into the same trap, except that it would probably perform better in absolute figures. It uses the same architecture. And so does e.g. hAP ac², so it would be easy to test e.g. for me if I had a fat enough pipe to the net.
Plus if it is as you describe, there is no guarantee that the 4011 will not fall into the same trap, except that it would probably perform better in absolute figures. It uses the same architecture. And so does e.g. hAP ac², so it would be easy to test e.g. for me if I had a fat enough pipe to the net.
Re: Severe Performance Drop RB3011
Sorry for the delay with this guys.
So here is what I have:
The Mangle Rules:
Essentially the first rule is marking packets coming from IP addresses on our Public WiFi network, and being marked as Upload Packets, and the second rule is marking packets going to those Public WiFi network IP addresses and being marked as Download Packets.
From here under the Queue Tree I have a Global Download and a Global Upload queue with sub items
So this essentially shares a 150mbit bandwidth pool for download speed for our 2 Public WiFi subnets and essentially shares a 5mbit bandwidth pool for upload speed for our 2 Public WiFi subnets.
Very simple.
At the end of the day, activating just one of these mangle rules from above takes a very big hit on our overall throughput. Running a test from a server which has no restrictions on the router and should get the full 480mbit we can hit, only gets about 200 - 215 mbit, and CPU usage never hitting above 30% during this test. With or without the queue tree options enabled.
Creating a random mangle rule with dummy information that wouldn't ever hit an actual packet produces the same result. So clearly something is wrong here, and again concluding that I will factory reset the router and start over, as something is terribly wrong. I never had this problem before and have had our mangle rules in place for months without a problem. I only identified an issue now that our pipe got upgraded and I wasn't getting the expected results during our tests. But had noticed the WiFi not really performing as it should over recent weeks but didn't clearly identify the problem until now.
I am guessing that perhaps a recent RouterOS update may have caused a problem, but I don't know.
So here is what I have:
Code: Select all
/ip firewall address-list
add address=172.31.0.0/24 list=Public-WiFI
add address=172.31.1.0/24 list=Public-WiFI
Code: Select all
/ip firewall mangle
add action=mark-packet chain=prerouting new-packet-mark="Public-WiFi Upload" passthrough=no src-address-list=Public-WiFI
add action=mark-packet chain=prerouting dst-address-list=Public-WiFI new-packet-mark="Public-WiFi Download" passthrough=noFrom here under the Queue Tree I have a Global Download and a Global Upload queue with sub items
Code: Select all
/queue tree
add name="Public WiFi Upload" parent="Global Upload" max-limit=150M packet-mark="Public-WiFi Upload" queue=pcq-upload-default
add name="Public WiFi Download" parent="Global Download" max-limit=5M packet-mark="Public-WiFi Download" queue=pcq-download-defaultVery simple.
At the end of the day, activating just one of these mangle rules from above takes a very big hit on our overall throughput. Running a test from a server which has no restrictions on the router and should get the full 480mbit we can hit, only gets about 200 - 215 mbit, and CPU usage never hitting above 30% during this test. With or without the queue tree options enabled.
Creating a random mangle rule with dummy information that wouldn't ever hit an actual packet produces the same result. So clearly something is wrong here, and again concluding that I will factory reset the router and start over, as something is terribly wrong. I never had this problem before and have had our mangle rules in place for months without a problem. I only identified an issue now that our pipe got upgraded and I wasn't getting the expected results during our tests. But had noticed the WiFi not really performing as it should over recent weeks but didn't clearly identify the problem until now.
I am guessing that perhaps a recent RouterOS update may have caused a problem, but I don't know.
Re: Severe Performance Drop RB3011
Another note here, I have tried different mangle rules by first marking the new connections from and to these address lists, and then marking the connections with the packet marks, its no different.
Re: Severe Performance Drop RB3011
Well, the next test should be to leave the mangle rules in place but disable the queues.
But again, as you have posted only an excerpt from your configuration, it is hard to guess what all happens.
I can imagine that if you have no other firewall rules than these two, and you remove/disable these two, RouterOS says "hooray, no firewall rules, no need to push packets via firewall, so switching on fastpath" (I don't know whether it takes care about existence of configuration items under /queue tree when taking this decision). But as soon as you add a single mangle rule, fastpath gets disabled, and as you cannot activate it using fasttrack (because you need mangling), you see the performance drop, and further performance drop may come from the queue handling as these mangle rules not only are present but also let at least part of the traffic be handled by queues.
So if you enable the mangle rules but disable the queues, you may see yet another speedtest results.
But again, as you have posted only an excerpt from your configuration, it is hard to guess what all happens.
I can imagine that if you have no other firewall rules than these two, and you remove/disable these two, RouterOS says "hooray, no firewall rules, no need to push packets via firewall, so switching on fastpath" (I don't know whether it takes care about existence of configuration items under /queue tree when taking this decision). But as soon as you add a single mangle rule, fastpath gets disabled, and as you cannot activate it using fasttrack (because you need mangling), you see the performance drop, and further performance drop may come from the queue handling as these mangle rules not only are present but also let at least part of the traffic be handled by queues.
So if you enable the mangle rules but disable the queues, you may see yet another speedtest results.
Re: Severe Performance Drop RB3011
EDIT: Verbiage updated. Thanks!
I've already tried that actually. Disabling the queues and enabling even just one mangle rule, bandwidth drops substantially. Disabling all mangle rules and leaving all the queues on, full bandwidth.
I will work to get the full configuration on here as I understand there could be an underlying issue besides the simple mangle rules, and obviously as something really strange is going on here.
I've already tried that actually. Disabling the queues and enabling even just one mangle rule, bandwidth drops substantially. Disabling all mangle rules and leaving all the queues on, full bandwidth.
I will work to get the full configuration on here as I understand there could be an underlying issue besides the simple mangle rules, and obviously as something really strange is going on here.
Last edited by ChildOTK on Wed Oct 17, 2018 1:46 am, edited 1 time in total.
Re: Severe Performance Drop RB3011
What is the difference between Leaving the queue tree enabled and simply disabling the mangle rules and Disabling all mangle rules and leaving the queues on cases? To me it is the same but you report different impact on bandwidth.
Re: Severe Performance Drop RB3011
The queues are essentially useless without the mangle as the queue is looking for packets with a certain mark to initiate it's job. So yes, I would completely agree, useless.
Re: Severe Performance Drop RB3011
I do understand that the packet mark directs the packet to the queue, but what I had in mind was whether the very presence of queues in configuration alone, although the mangle rules mark no packets for them, affects the throughput or not. I know that empty firewall activates fastpath behaviour, but I don't know whether non-empty queue configuration prevents it even if firewall is empty. And from your description it seemed that the answer was yes and no at the same time 
Re-read that post and fix it, please. Maybe there should have been one "raises" instead of "drops"?
Re-read that post and fix it, please. Maybe there should have been one "raises" instead of "drops"?
Re: Severe Performance Drop RB3011
Here is my config with some bits and pieces removed but are not important or are sensitive information:
Some items here I don't recognize from before and can only assume were added as a result of updating over the last few months, such as the IPv6 Firewall rules. I also am only supposed to have one bridge interface but notice there are 2. Guessing this is the result of an update again.
Code: Select all
# oct/16/2018 15:57:27 by RouterOS 6.43.2
# software id = REMOVED
#
# model = RouterBOARD 3011UiAS
# serial number = REMOVED
/interface bridge
add admin-mac=REMOVED auto-mac=no comment=defconf name=bridge
add fast-forward=no mtu=1500 name=internal-bridge
/interface ethernet
set [ find default-name=ether1 ] mac-address=REMOVED
set [ find default-name=sfp1 ] advertise=\
10M-half,10M-full,100M-half,100M-full,1000M-half,1000M-full
/interface vlan
add interface=internal-bridge name=vlan1 vlan-id=REMOVED
add interface=internal-bridge name=vlan2 vlan-id=REMOVED
add interface=internal-bridge name=vlan3 vlan-id=REMOVED
add interface=internal-bridge name=vlan5 vlan-id=REMOVED
add interface=internal-bridge name=vlan6 vlan-id=REMOVED
add interface=internal-bridge name=vlan4 vlan-id=REMOVED
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
add name="Public Network"
add name="Admin Networks"
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip firewall layer7-protocol
add name=activedirectory regexp="\\x06\\x5Fmsdcs\\x06itsoft\\x02by"
/ip ipsec peer profile
add dh-group=modp1024 lifetime=1h name=profile_1
add dh-group=modp1024 dpd-interval=disable-dpd enc-algorithm=aes-256 name=\
profile_2 nat-traversal=no
/ip ipsec proposal
set [ find default=yes ] enc-algorithms=3des
/ip pool
add name=vlan6-2 ranges=172.31.1.2-172.31.1.254
add name=VPN ranges=192.168.0.32/29
add name=vlan1-dynamic-2-old ranges=192.168.1.2-192.168.1.254
add name=vlan1-dynamic-old next-pool=vlan1-dynamic-2-old ranges=\
192.168.0.150-192.168.0.254
add name=vlan6-1 next-pool=vlan6-2 ranges=172.31.0.2-172.31.0.254
add name=vlan2 ranges=172.16.253.2-172.16.253.254
add name=vlan1 ranges=172.16.255.2-172.16.255.254
add name=vlan3 ranges=172.16.0.2-172.16.0.254
add name=vlan4 ranges=172.16.254.2-172.16.254.254
/ip dhcp-server
add address-pool=vlan1-dynamic-old disabled=no interface=internal-bridge \
lease-time=1d name=vlan1-old
add address-pool=vlan6-1 disabled=no interface=vlan6 lease-time=\
2h name=vlan6
add address-pool=vlan2 disabled=no interface=vlan2 lease-time=\
1d name=vlan2
add address-pool=vlan1 disabled=no interface=vlan1 lease-time=1d \
name=vlan1
add address-pool=vlan3 disabled=no interface=vlan3 lease-time=3d \
name=vlan3
add address-pool=vlan4 disabled=no interface=vlan4 lease-time=\
1d name=vlan4
/ppp profile
set *FFFFFFFE local-address=192.168.0.1 remote-address=vpn
/queue tree
add comment="Everything Else 1k" disabled=yes max-limit=1k name="No Mark" \
packet-mark=no-mark parent=global
/queue tree
add name="Total Download" parent=global queue=pcq-download-default
add name="Total Upload" parent=global queue=pcq-upload-default
add limit-at=6M max-limit=25M name="02 Live Stream Upload" packet-mark=\
"Upload Live Stream" parent="Total Upload" priority=2 queue=\
pcq-upload-default
add limit-at=10M max-limit=100M name="02 Live Stream Download" packet-mark=\
"Download Live Stream" parent="Total Download" priority=2 queue=\
pcq-download-default
add name="03 Giving Stations, Kiosks, etc Upload" packet-mark=\
"Upload Giving Stations, Kiosks, etc" parent="Total Upload" priority=3 \
queue=pcq-upload-default
add name="03 Giving Stations, Kiosks, etc Download" packet-mark=\
"Download Giving Stations, Kiosks, etc" parent="Total Download" priority=\
3 queue=pcq-download-default
add max-limit=5M name="08 Guest Network Upload" packet-mark=\
"Upload Guest Network" parent="Total Upload" queue=pcq-upload-default
add max-limit=150M name="08 Guest Network Download" packet-mark=\
"Download Guest Network" parent="Total Download" queue=\
pcq-download-default
add disabled=yes name="01 Unassigned Download" parent="Total Download" \
priority=1 queue=pcq-download-default
add disabled=yes name="04 Unassigned Download" parent="Total Download" \
priority=4 queue=pcq-download-default
add disabled=yes name="05 Unassigned Download" parent="Total Download" \
priority=5
add disabled=yes name="06 Unassigned Download" parent="Total Download" \
priority=6
add disabled=yes name="07 Unassigned Download" parent="Total Download" \
priority=7 queue=pcq-download-default
add disabled=yes name="01 Unassigned Upload" parent="Total Upload" priority=1 \
queue=pcq-upload-default
add disabled=yes name="04 Unassigned Upload" parent="Total Upload" priority=4 \
queue=pcq-upload-default
add disabled=yes name="05 Unassigned Upload" parent="Total Upload" priority=5 \
queue=pcq-upload-default
add disabled=yes name="06 Unassigned Upload" parent="Total Upload" priority=6 \
queue=pcq-upload-default
add disabled=yes name="07 Unassigned Upload" parent="Total Upload" priority=7 \
queue=pcq-upload-default
/snmp community
set [ find default=yes ] addresses=0.0.0.0/0
/system logging action
set 1 disk-file-count=1 disk-lines-per-file=4000
/interface bridge port
add bridge=internal-bridge comment=defconf interface=ether2
add bridge=internal-bridge comment=defconf interface=ether3
add bridge=internal-bridge comment=defconf interface=ether4
add bridge=internal-bridge comment=defconf interface=ether5
add bridge=internal-bridge comment=defconf interface=ether6
add bridge=internal-bridge comment=defconf interface=ether7
add bridge=internal-bridge comment=defconf interface=ether8
add bridge=internal-bridge comment=defconf interface=ether9
add bridge=internal-bridge comment=defconf interface=sfp1
add bridge=internal-bridge hw=no interface=ether10
/ip firewall connection tracking
set enabled=yes
/ip neighbor discovery-settings
set discover-interface-list=LAN
/ip settings
set rp-filter=strict tcp-syncookies=yes
/interface l2tp-server server
set default-profile=default enabled=yes use-ipsec=yes
/interface list member
add comment=defconf interface=bridge list=LAN
add comment=defconf interface=ether1 list=WAN
add interface=vlan6 list="Public Network"
add interface=vlan1 list="Admin Networks"
add interface=vlan2 list="Admin Networks"
add interface=vlan3 list="Admin Networks"
add interface=vlan5 list="Admin Networks"
add interface=vlan4 list="Admin Networks"
/ip address
add address=192.168.0.1/24 interface=internal-bridge network=192.168.0.0
add address=172.31.0.1/24 interface=vlan6 network=172.31.0.0
add address=172.31.1.1/24 interface=vlan6 network=172.31.1.0
add address=192.168.1.1/24 interface=internal-bridge network=192.168.1.0
add address=192.168.88.2/24 interface=internal-bridge network=192.168.88.0
add address=172.16.0.1/24 interface=vlan3 network=172.16.0.0
add address=172.16.255.1/24 interface=vlan1 network=172.16.255.0
add address=172.16.254.1/24 interface=vlan4 network=172.16.254.0
add address=172.16.253.1/24 interface=vlan2 network=172.16.253.0
add address=172.16.250.1/24 interface=internal-bridge network=172.16.250.0
add address=172.16.251.1/24 interface=internal-bridge network=172.16.251.0
add address=172.16.252.1/24 interface=internal-bridge network=172.16.252.0
add address=REMOVED interface=ether1 network=REMOVED
/ip cloud
set ddns-enabled=yes
/ip dhcp-client
add comment=defconf dhcp-options=hostname,clientid interface=ether1 \
use-peer-dns=no
add dhcp-options=clientid,clientid,hostname interface=ether1 use-peer-dns=no
/ip dhcp-server lease
LEASES REMOVED
/ip dhcp-server network
add address=172.16.0.0/24 dns-server=172.16.0.1 \
domain=COTH.local gateway=172.16.0.1
add address=172.16.253.0/24 dns-server=\
1.1.1.1,1.0.0.1,8.8.8.8,8.8.4.4 gateway=172.16.253.1
add address=172.16.254.0/24 dns-server=\
1.1.1.1,1.0.0.1,8.8.8.8,8.8.4.4 gateway=172.16.254.1
add address=172.16.255.0/24 dns-server=172.16.255.1 \
domain=COTH.local gateway=172.16.255.1
add address=172.31.0.0/24 dns-server=\
1.1.1.1,1.0.0.1,8.8.8.8,8.8.4.4 gateway=172.31.0.1
add address=172.31.1.0/24 dns-server=\
1.1.1.1,1.0.0.1,8.8.8.8,8.8.4.4 gateway=172.31.1.1
add address=192.168.0.0/24 dns-server=\
1.1.1.1,1.0.0.1,8.8.8.8,8.8.4.4 gateway=192.168.0.1
add address=192.168.1.0/24 dns-server=\
1.1.1.1,1.0.0.1,8.8.8.8,8.8.4.4 gateway=192.168.1.1
add address=192.168.88.0/24 gateway=192.168.88.1
/ip dns
set allow-remote-requests=yes servers=\
172.16.0.4,1.1.1.1,1.0.0.1,8.8.8.8,8.8.4.4
/ip dns static
REMOVED
/ip firewall address-list
add address=192.168.0.32/29 list="VPN Users"
add address=172.31.0.0/24 list="Guest Network"
add address=172.31.1.0/24 list="Guest Network"
add address=0.pool.ntp.org list="NTP Servers"
add address=1.pool.ntp.org list="NTP Servers"
OTHERS REMOVED
/ip firewall filter
add action=reject chain=forward comment="Dynamo Printer Connections" \
dst-address=128.30.52.100 protocol=tcp reject-with=tcp-reset
add action=drop chain=forward comment=\
"Drop Traffic from the Free WiFi VLAN to Primary VLAN" in-interface-list=\
"Public Network" out-interface-list="Admin Networks"
add action=drop chain=forward comment=\
"Drop Traffic from the Free WiFi VLAN to Primary VLAN" in-interface-list=\
"Admin Networks" out-interface-list="Public Network"
add action=drop chain=input comment="Drop blocked IP addresses" in-interface=\
ether1 src-address-list="Blocked IPs"
add action=accept chain=input comment="Always Allow List" in-interface=ether1 \
src-address-list="Always Allow List"
add action=accept chain=input comment="Allow NTP Servers" dst-port=123 \
in-interface=ether1 protocol=udp src-address-list="NTP Servers"
add action=accept chain=input comment=VPN dst-port=500,1701,4500 \
in-interface=ether1 protocol=udp
add action=accept chain=input comment=VPN in-interface=ether1 protocol=\
ipsec-esp
add action=drop chain=input comment="Drop ICMP period" in-interface=ether1 \
protocol=icmp
add action=drop chain=input comment="Drop invalid connections" \
connection-state=invalid in-interface=ether1
add action=accept chain=input comment="Accept Established Connections" \
connection-state=established in-interface=ether1
add action=accept chain=input comment="Accept Related Connections" \
connection-state=related in-interface=ether1
add action=drop chain=input comment="Drop Everything Else" in-interface=\
ether1
/ip firewall mangle
add action=mark-packet chain=prerouting disabled=yes dst-address=192.168.0.1 \
dst-port=53 layer7-protocol=activedirectory new-packet-mark=\
activedirectory passthrough=yes protocol=udp
add action=mark-packet chain=forward comment="Live Stream Upload" disabled=\
yes new-packet-mark="Upload Live Stream" passthrough=no src-address-list=\
"Live Stream"
add action=mark-packet chain=forward comment="Live Stream Download" disabled=\
yes dst-address-list="Live Stream" new-packet-mark="Download Live Stream" \
passthrough=no
add action=mark-packet chain=forward comment=\
"Giving Stations, Kiosks, etc Upload" disabled=yes new-packet-mark=\
"Upload Giving Stations, Kiosks, etc" passthrough=no src-address-list=\
"Giving Stations, Kiosks, etc"
add action=mark-packet chain=forward comment=\
"Giving Stations, Kiosks, etc Download" disabled=yes dst-address-list=\
"Giving Stations, Kiosks, etc" new-packet-mark=\
"Download Giving Stations, Kiosks, etc" passthrough=no
add action=mark-packet chain=forward comment="Guest Network Upload" disabled=\
yes new-packet-mark="Upload Guest Network" passthrough=no \
src-address-list="Guest Network"
add action=mark-packet chain=forward comment="Guest Network Download" \
disabled=yes dst-address-list="Guest Network" new-packet-mark=\
"Download Guest Network" passthrough=no
add action=mark-connection chain=prerouting disabled=yes new-connection-mark=\
"Live Stream Connection" passthrough=yes src-address-list="Live Stream"
add action=mark-packet chain=prerouting connection-mark=\
"Live Stream Connection" disabled=yes new-packet-mark=\
"Upload Live Stream" passthrough=no
add action=mark-packet chain=input disabled=yes new-packet-mark=\
"Upload Live Stream" passthrough=no src-address-list="Live Stream"
add action=mark-packet chain=output disabled=yes dst-address-list=\
"Live Stream" new-packet-mark="Download Live Stream" passthrough=no
/ip firewall nat
REMOVED
/ip firewall service-port
set sip disabled=yes sip-direct-media=no
/ip ipsec peer
add address=0.0.0.0/0 comment="VPN" exchange-mode=main-l2tp \
generate-policy=port-override passive=yes profile=profile_1 \
send-initial-contact=no
/ip ipsec policy
add dst-address=0.0.0.0/0 src-address=0.0.0.0/0 template=yes
/ip route
add distance=1 gateway=REMOVED
/ip service
set telnet address=0.0.0.0/0 disabled=yes
set ftp address=0.0.0.0/0
set www address=0.0.0.0/0
set ssh address=0.0.0.0/0
set www-ssl address=0.0.0.0/0
set api address=0.0.0.0/0 disabled=yes
set winbox address=0.0.0.0/0
set api-ssl address=0.0.0.0/0 disabled=yes
/ip upnp
set enabled=yes
/ip upnp interfaces
add interface=ether1 type=external
add interface=vlan1 type=internal
add interface=vlan3 type=internal
/ipv6 firewall address-list
add address=::/128 comment="defconf: unspecified address" list=bad_ipv6
add address=::1/128 comment="defconf: lo" list=bad_ipv6
add address=fec0::/10 comment="defconf: site-local" list=bad_ipv6
add address=::ffff:0.0.0.0/96 comment="defconf: ipv4-mapped" list=bad_ipv6
add address=::/96 comment="defconf: ipv4 compat" list=bad_ipv6
add address=100::/64 comment="defconf: discard only " list=bad_ipv6
add address=2001:db8::/32 comment="defconf: documentation" list=bad_ipv6
add address=2001:10::/28 comment="defconf: ORCHID" list=bad_ipv6
add address=3ffe::/16 comment="defconf: 6bone" list=bad_ipv6
add address=::224.0.0.0/100 comment="defconf: other" list=bad_ipv6
add address=::127.0.0.0/104 comment="defconf: other" list=bad_ipv6
add address=::/104 comment="defconf: other" list=bad_ipv6
add address=::255.0.0.0/104 comment="defconf: other" list=bad_ipv6
/ipv6 firewall filter
add action=accept chain=input comment=\
"defconf: accept established,related,untracked" connection-state=\
established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
invalid
add action=accept chain=input comment="defconf: accept ICMPv6" protocol=\
icmpv6
add action=accept chain=input comment="defconf: accept UDP traceroute" port=\
33434-33534 protocol=udp
add action=accept chain=input comment=\
"defconf: accept DHCPv6-Client prefix delegation." dst-port=546 protocol=\
udp src-address=fe80::/16
add action=accept chain=input comment="defconf: accept IKE" dst-port=500,4500 \
protocol=udp
add action=accept chain=input comment="defconf: accept ipsec AH" protocol=\
ipsec-ah
add action=accept chain=input comment="defconf: accept ipsec ESP" protocol=\
ipsec-esp
add action=accept chain=input comment=\
"defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec
add action=drop chain=input comment=\
"defconf: drop everything else not coming from LAN" in-interface-list=\
!LAN
add action=accept chain=forward comment=\
"defconf: accept established,related,untracked" connection-state=\
established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" \
connection-state=invalid
add action=drop chain=forward comment=\
"defconf: drop packets with bad src ipv6" src-address-list=bad_ipv6
add action=drop chain=forward comment=\
"defconf: drop packets with bad dst ipv6" dst-address-list=bad_ipv6
add action=drop chain=forward comment="defconf: rfc4890 drop hop-limit=1" \
hop-limit=equal:1 protocol=icmpv6
add action=accept chain=forward comment="defconf: accept ICMPv6" protocol=\
icmpv6
add action=accept chain=forward comment="defconf: accept HIP" protocol=139
add action=accept chain=forward comment="defconf: accept IKE" dst-port=\
500,4500 protocol=udp
add action=accept chain=forward comment="defconf: accept ipsec AH" protocol=\
ipsec-ah
add action=accept chain=forward comment="defconf: accept ipsec ESP" protocol=\
ipsec-esp
add action=accept chain=forward comment=\
"defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec
add action=drop chain=forward comment=\
"defconf: drop everything else not coming from LAN" in-interface-list=\
!LAN
/lcd
set read-only-mode=yes time-interval=hour
/lcd pin
set hide-pin-number=yes pin-number=3492
/ppp secret
REMOVED
/snmp
set contact="REMOVED" enabled=yes location=\
"REMOVED" trap-generators=\
interfaces,temp-exception trap-interfaces=all trap-version=2
/system clock
set time-zone-name=America/Chicago
/system identity
set name=MikroTik
/system logging
set 0 action=disk
set 1 action=disk
set 2 action=disk
set 3 action=disk
/system ntp client
set enabled=yes primary-ntp=89.111.54.85 secondary-ntp=217.147.223.78 \
server-dns-names=0.pool.ntp.org,1.pool.ntp.org
/system routerboard settings
set silent-boot=yes
/system scheduler
add comment="Daily Reboot" interval=1d name="Daily Reboot" on-event=\
"/system reboot" policy=\
ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=\
mar/16/2017 start-time=04:00:00
add interval=1h name=Update_NTP on-event=Update_NTP policy=\
ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon \
start-time=startup
/system script
add dont-require-permissions=no name=Update_NTP owner=admin policy=\
ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon source=":\
delay 60s\r\
\n/system ntp client set primary-ntp=[:resolve 0.pool.ntp.org]\r\
\n/system ntp client set secondary-ntp=[:resolve 1.pool.ntp.org]"
add dont-require-permissions=no name="Bandwidth Test Both" owner=admin \
policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon \
source=":log info \"------------ UDP Bandwidth Test Start: BOTH ----------\
----\";\r\
\n\r\
\n:local btUser \"btest\";\r\
\n:local btPass \"btest\";\r\
\n:local btHost \"207.32.194.24\";\r\
\n\r\
\n:local avrRX 0;\r\
\n:local avrTX 0;\r\
\n\r\
\n# SPEED TEST\r\
\n:log info \"----> <--- Measuring Bandwidth (30 second segment)........\"\
;\r\
\n:do {/tool bandwidth-test duration=30s user=\$btUser password=\$btPass p\
rotocol=udp address=\$btHost direction=both do={\r\
\n :set \$avrRX (\"rx-10-second-average: \" . (\$\"rx-10-second-averag\
e\" / 1048576) . \".\" . (\$\"rx-10-second-average\" % (1048576) / 1024) .\
\_\" Mbps\" );\r\
\n :set \$avrTX (\"tx-10-second-average: \" . (\$\"tx-10-second-averag\
e\" / 1048576) . \".\" . (\$\"tx-10-second-average\" % (1048576) / 1024) .\
\_\" Mbps\" );\r\
\n }\r\
\n} on-error={:log error message=\"Bandwidth Test Failed\"}\r\
\n\r\
\n:log info message=\$avrRX;\r\
\n:log info message=\$avrTX;\r\
\n\r\
\n:log info \"-------- UDP Bandwidth Test End: BOTH ------------\";\r\
\n\r\
\n\r\
\n################# SAVING RESULTS WITH DATE ######################\r\
\n:local filename2 \"LOG_BW_TEST.txt\"\r\
\n:local ds [/system clock get date];\r\
\n:local months (\"jan\",\"feb\",\"mar\",\"apr\",\"may\",\"jun\",\"jul\",\
\"aug\",\"sep\",\"oct\",\"nov\",\"dec\");\r\
\n:local month [ :pick \$ds 0 3 ];\r\
\n:local mm ([ :find \$months \$month -1 ] + 1);\r\
\n:if (\$mm < 10) do={ :set mm (\"0\" . \$mm); };\r\
\n:set ds ([:pick \$ds 7 11] . \$mm . [:pick \$ds 4 6]);\r\
\n\r\
\n:if ( [:len [/file find name=\$filename2]] = 0) do={\r\
\n:log info \"Log file does not exist. Creating a new one.....\";\r\
\n/file print file=\$filename2 where name=\"\";\r\
\n}\r\
\n\r\
\n:log info \"Adding result to the end of the lof file......\";\r\
\n/file set \$filename2 contents=([get \$filename2 contents] .\"\\n\".\$d\
s.\"-->\" . \$avrRX);\r\
\n/file set \$filename2 contents=([get \$filename2 contents] .\" \". \$\
avrTX);\r\
\n}\r\
\n"
/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=LAN
/tool romon
set enabled=yes
Some items here I don't recognize from before and can only assume were added as a result of updating over the last few months, such as the IPv6 Firewall rules. I also am only supposed to have one bridge interface but notice there are 2. Guessing this is the result of an update again.
Re: Severe Performance Drop RB3011
I just caught that fast-forward was not enabled on my main bridge. I wonder if this is what is causing my problem. I'm going to try again with this on.
Re: Severe Performance Drop RB3011
OK. First, are you sure your layer7 rule doesn't reveal anything about you?
Second, as you have tons of other firewall rules there, the fastpath is not enabled for sure even if you disable the two ones in mangle, So I would assume that the mangle rules affect the throughput only because they actually enable the use of the queues.
Second, as you have tons of other firewall rules there, the fastpath is not enabled for sure even if you disable the two ones in mangle, So I would assume that the mangle rules affect the throughput only because they actually enable the use of the queues.
Re: Severe Performance Drop RB3011
The layer7 rule was there from testing in the past, I don't actually use that anymore although it is still in the system. I can certainly remove that.
How would you recommend making changes to the rules right now?
I enabled Fast Forward on the bridge and rebooted the router. I got better results (now hitting about 280mbit down) with a mangle rule turned on. With them still off I hit a little over the 480mbit.
I would really like to use this as a learning experience as I obviously messed something up somewhere or am not doing things right as I thought I was. So when I ask what you would recommend it's from a view of wanting to learn to do things better and learn from my mistakes here. So I appreciate any input you could provide right now. I intend to use the 3011 to its full potential and don't mean to create a bottle neck.
How would you recommend making changes to the rules right now?
I enabled Fast Forward on the bridge and rebooted the router. I got better results (now hitting about 280mbit down) with a mangle rule turned on. With them still off I hit a little over the 480mbit.
I would really like to use this as a learning experience as I obviously messed something up somewhere or am not doing things right as I thought I was. So when I ask what you would recommend it's from a view of wanting to learn to do things better and learn from my mistakes here. So I appreciate any input you could provide right now. I intend to use the 3011 to its full potential and don't mean to create a bottle neck.
Re: Severe Performance Drop RB3011
The layer 7 protocol was removed and the rule along with it, and that made no difference, just FYI.
Re: Severe Performance Drop RB3011
"Fast Forward" shouldn't make a difference, it is only for two port bridges.
You are doing a lot of stuff on that router, ex. the PCQ, but I would not expect it to be hitting a throughput limit at 30% CPU. Are you sure you have been upgrading the firmware along with the RouterOS version?
You are doing a lot of stuff on that router, ex. the PCQ, but I would not expect it to be hitting a throughput limit at 30% CPU. Are you sure you have been upgrading the firmware along with the RouterOS version?
Re: Severe Performance Drop RB3011
Yes, both software and firmware are on the same version and both on the latest 6.43.2.
Re: Severe Performance Drop RB3011
Layer 7 was already disabled in the config you've posted, so I didn't have in mind that it should affect performance but that it contained a domain name which may have been linked to you.
To the rest, at 1:15 AM I'm unable to provide any clever ideas You may have a look at some optimization of the rules but the sad truth is that every single packet, regardless the direction, must pass through the mangle rules, and that the queueing does require some resources.
So yes, you may e.g. split the mangle prerouting into two chains, one for download direction and the other one for the upload direction so that each packet traverses just 1/2 of the total count of the mangle rules (i.e. you place a single action=jump rule as the first one, with jump-target=my-mangle-prerouting, and move all rules for one direction to this new chain=my-mangle-prerouting and keep the remaining ones (for the other direction) unchanged in the default chain=prerouting. I'm sure it will provide an improvement but I doubt it will be a significant one.
To the rest, at 1:15 AM I'm unable to provide any clever ideas
You may have a look at some optimization of the rules but the sad truth is that every single packet, regardless the direction, must pass through the mangle rules, and that the queueing does require some resources.So yes, you may e.g. split the mangle prerouting into two chains, one for download direction and the other one for the upload direction so that each packet traverses just 1/2 of the total count of the mangle rules (i.e. you place a single action=jump rule as the first one, with jump-target=my-mangle-prerouting, and move all rules for one direction to this new chain=my-mangle-prerouting and keep the remaining ones (for the other direction) unchanged in the default chain=prerouting. I'm sure it will provide an improvement but I doubt it will be a significant one.
Re: Severe Performance Drop RB3011
One last idea for today, if you disable all mangle rules and add one which has a different action than mark-packet, is the impact on bandwidth the same like with a packet-marking rule or a different one?
Re: Severe Performance Drop RB3011
I'll give that a try.
Another note, I reverted to simple queues and its working just fine. So I may just resort to that. I don't have a lot I need to do but I am able to do PCQ queues there too, and it works as I need it to.
Another note, I reverted to simple queues and its working just fine. So I may just resort to that. I don't have a lot I need to do but I am able to do PCQ queues there too, and it works as I need it to.
Re: Severe Performance Drop RB3011 [SOLVED]
Interesting update here...I upgraded to 6.43.4 today, still had the same issues. However this evening I decided to disable the IPv6 package (I don’t need it anyway) as I noticed in my export I had IPv6 firewall rules going on too. Rebooted the router, and tada! Mangle rules work just fine as before with my queue tree. Who would have thought...
Who is online
Users browsing this forum: Bing [Bot], yonutm and 58 guests