Community discussions

MikroTik App
 
noven
just joined
Topic Author
Posts: 14
Joined: Mon Jul 05, 2010 6:44 am

Understanding switch performance

Tue Feb 07, 2012 10:51 pm

I'm trying to figure out exactly *why* I'm getting a non-intuitive performance result.
I have an RB493G (5.12) servicing a small office. Ether 1 is WAN. Trying to get all 8 other ports + wifi operating as a single subnet. So I switched 3,4,5 to 2 and 7,8,9 to 6. Then I created a bridge and added 2, 6 and wifi.

Everything could talk but throughput was terrible. Browsing the NAS had noticeable lag, during a big copy stats showed ~30-60kbps moving through the switch port - copy was between 2 ports in same switch group.

So I removed all ports from being switched and added them all to the bridge directly. Immediately NAS performance was instant to browse and the same copy showed over 75Mbps throughput.

I would have thought the switch would operate faster than the bridge, which must go through the cpu. But CPU maybe doesn't matter so much - switched scenario had CPU around 2%, bridged mode hiked it up to ~14% - so no problems dealing with load. Maybe adding a switched port to the bridge creates some doubleup in packet processing? Or maybe the switch chip is just a bit broken? Insights appreciated.
 
brandonrossl
Frequent Visitor
Frequent Visitor
Posts: 61
Joined: Wed Jun 08, 2011 10:09 pm

Re: Understanding switch performance

Tue Feb 07, 2012 11:31 pm

Even though the ports are switched, they still need to be added to the bridge.
 
scampbell
Trainer
Trainer
Posts: 487
Joined: Thu Jun 22, 2006 5:20 am
Location: Wellington, NZ
Contact:

Re: Understanding switch performance

Wed Feb 08, 2012 1:04 am

According to http://wiki.mikrotik.com/wiki/Manual:Sw ... p_Features you appear to be doing this correctly. :?


Ether 1 Master Port=None
Ether 2 Master Port=None
Ether 3-5 Master port = ether2
Ether 6 Master port=none
ether 7-9 Master port =Ether6

Bridge Ports=ether2,ether6,wlan1


How about your Masquerade rule ? Is that set with Out Interface = ether1

any other firewall or nat rules ?
 
noven
just joined
Topic Author
Posts: 14
Joined: Mon Jul 05, 2010 6:44 am

Re: Understanding switch performance

Wed Feb 08, 2012 1:52 am

I was pretty confident it was set up correctly, and from a pure ping test point of view it 'worked', its the performance I was concerned about.

All else being equal, how can a software bridge perform *far far* better than a hardware switch chip?

Ether1 has masquerade and a pretty light set of firewall rules. Bridge has no ip firewall enabled, so the actual path in question has no rules being applied.
 
scampbell
Trainer
Trainer
Posts: 487
Joined: Thu Jun 22, 2006 5:20 am
Location: Wellington, NZ
Contact:

Re: Understanding switch performance

Wed Feb 08, 2012 2:15 am

Are you running DHCP ? If so I assume you have the DHCP server attached to the Bridge ?

Any chance of sharing the /export compact ? (Req RoS v5.12)

I've a RB493 here and a NAS so i can do a quick test for you.
 
noven
just joined
Topic Author
Posts: 14
Joined: Mon Jul 05, 2010 6:44 am

Re: Understanding switch performance

Wed Feb 08, 2012 2:41 am

Indeed I do have DHCP enabled on the bridge.

Heres the export:

[admin@MikroTik] > export compact
# feb/08/2012 11:37:30 by RouterOS 5.12
# software id = PIXE-94VZ
#
/interface bridge
add l2mtu=1520 name=Local
/interface wireless
set 0 band=2ghz-b/g/n disabled=no frequency=2452 l2mtu=2290 mode=ap-bridge \
name=WLAN ssid=AWC wireless-protocol=802.11
/interface ethernet
set 0 arp=proxy-arp name=ether1 speed=1Gbps
set 1 name="2 - LAN" speed=1Gbps
set 2 name="3 - LAN" speed=1Gbps
set 3 name="4 - LAN" speed=1Gbps
set 4 name="5 - LAN" speed=1Gbps
set 5 name="6 - LAN2" speed=1Gbps
set 6 name="7 - LAN2" speed=1Gbps
set 7 name="8 - LAN2" speed=1Gbps
set 8 name="9 - LAN2" speed=1Gbps
/interface wireless security-profiles
set [ find default=yes ] authentication-types=wpa-psk,wpa2-psk eap-methods="" \
group-ciphers=tkip,aes-ccm mode=dynamic-keys unicast-ciphers=tkip,aes-ccm \
wpa-pre-shared-key=XXXX wpa2-pre-shared-key=XXXX
/ip pool
add name=dhcp_pool1 ranges=192.168.1.100-192.168.1.200
/ip dhcp-server
add address-pool=dhcp_pool1 disabled=no interface=Local name=dhcp1
/interface pppoe-client
add add-default-route=yes disabled=no interface=ether1 name=Telstra password=\
XXXX profile=default-encryption user=XXXX
/interface bridge port
add bridge=Local interface="2 - LAN"
add bridge=Local interface="6 - LAN2"
add bridge=Local interface=WLAN
add bridge=Local interface="3 - LAN"
add bridge=Local interface="4 - LAN"
add bridge=Local interface="5 - LAN"
add bridge=Local interface="7 - LAN2"
add bridge=Local interface="8 - LAN2"
add bridge=Local interface="9 - LAN2"
/ip address
add address=192.168.1.254/24 interface=Local
/ip dhcp-server network
add address=192.168.1.0/24 dns-server=192.168.1.254,208.67.222.222 gateway=\
192.168.1.254 netmask=24
/ip dns
set allow-remote-requests=yes cache-size=512KiB max-udp-packet-size=50 \
servers=208.67.220.220,208.67.222.222
/ip dns static
add address=192.168.88.1 name=router
/ip firewall filter
add chain=input comment=LAN src-address=192.168.1.0/24
add chain=forward src-address=192.168.1.0/24
add chain=input comment="Accept established connections" connection-state=\
established
add chain=input comment="Accept related connections" connection-state=related
add action=drop chain=input comment="Drop invalid connections" \
connection-state=invalid
add chain=input comment=UDP protocol=udp
add chain=input comment="Winbox in" dst-port=8291 protocol=tcp
add action=add-src-to-address-list address-list=ssh_blacklist \
address-list-timeout=1w3d chain=input connection-state=new dst-port=22 \
protocol=tcp src-address-list=ssh_stage3
add action=add-src-to-address-list address-list=ssh_stage3 \
address-list-timeout=1m chain=input connection-state=new dst-port=22 \
protocol=tcp src-address-list=ssh_stage2
add action=add-src-to-address-list address-list=ssh_stage2 \
address-list-timeout=1m chain=input connection-state=new dst-port=22 \
protocol=tcp src-address-list=ssh_stage1
add action=add-src-to-address-list address-list=ssh_stage1 \
address-list-timeout=1m chain=input connection-state=new dst-port=22 \
protocol=tcp
add chain=input comment="SSH In" dst-port=22 protocol=tcp
add chain=input comment="Allow limited pings" limit=50/5s,2 protocol=icmp
add action=drop chain=input comment="Drop excess pings" protocol=icmp
add action=log chain=input comment="Log everything else" log-prefix=\
"DROP INPUT"
add action=drop chain=input comment="Drop everything else"
/ip firewall nat
add action=masquerade chain=srcnat src-address=192.168.1.0/24
/ip smb shares
set [ find default=yes ] directory=/pub
/system clock
set time-zone-name=Australia/Sydney
/system health
set fan-mode=manual use-fan=auxiliary
/system ntp client
set enabled=yes mode=unicast primary-ntp=192.189.54.17 secondary-ntp=\
202.158.218.239

Thanks for any input :)
 
noven
just joined
Topic Author
Posts: 14
Joined: Mon Jul 05, 2010 6:44 am

Re: Understanding switch performance

Wed Feb 08, 2012 1:59 pm

That of course is the config with bridge no switch which works great. Modifying ONLY the section to remove from brisge all LAN interfaces except 2 / 6 and adding other ethernets to use master port for switch causes hideous performance drop.
 
brandonrossl
Frequent Visitor
Frequent Visitor
Posts: 61
Joined: Wed Jun 08, 2011 10:09 pm

Re: Understanding switch performance

Wed Feb 08, 2012 3:41 pm

Switch chip ports are 2 and 9 8)
 
noven
just joined
Topic Author
Posts: 14
Joined: Mon Jul 05, 2010 6:44 am

Re: Understanding switch performance

Wed Feb 08, 2012 4:14 pm

Does it even matter which port you assign as master? assuming ports on same switch chip of course. I had:

Ether 1 Master Port=None
Ether 2 Master Port=None
Ether 3-5 Master port = ether2
Ether 6 Master port=none
ether 7-9 Master port =Ether6

Are you suggesting

Ether 1 Master Port=None
Ether 2 Master Port=None
Ether 3-5 Master port = ether2
Ether 9 Master port=none
ether 6-8 Master port =Ether9

Would make a difference to performance - or anything else for that matter?
 
brandonrossl
Frequent Visitor
Frequent Visitor
Posts: 61
Joined: Wed Jun 08, 2011 10:09 pm

Re: Understanding switch performance

Wed Feb 08, 2012 4:16 pm

Worth a shot since the switch chip page in the wiki names ports 2 and 9 as the switch ports.
 
scampbell
Trainer
Trainer
Posts: 487
Joined: Thu Jun 22, 2006 5:20 am
Location: Wellington, NZ
Contact:

Re: Understanding switch performance

Wed Feb 08, 2012 11:11 pm

I did a quick test based on your config with some minor changes to suit my environment. I used a DHCP Client on E1 instead of PPP and different IP's due to a conflict with yours :-)

Basically it seemed to work much as would be expected.

I did a copy from two ports on the same switch group and a copy between the two switches with the master ports bridged.

Both gave about the same speed (43MBps) but the CPU load was different.

A CPU load of 2% when copying between ports in the same switch:
transfer - P6-P9.jpg
and 40+% when copying between the two switches via the Bridge:
transfer - P2-P9.jpg
You should also note the throughput on the port speeds as well. The traffic does not show at all when copying between ports in the same switch but does when you go through the bridge.

Here are the settings I used :-


/interface bridge
add l2mtu=1520 name=Local
/interface ethernet
set 0 arp=proxy-arp name=ether1 speed=1Gbps
set 1 name="2 - Master" speed=1Gbps
set 2 master-port="2 - Master" name="3 - LAN" speed=1Gbps
set 3 master-port="2 - Master" name="4 - LAN" speed=1Gbps
set 4 master-port="2 - Master" name="5 - LAN" speed=1Gbps
set 5 name="6 - Master" speed=1Gbps
set 6 master-port="6 - Master" name="7 - LAN2" speed=1Gbps
set 7 master-port="6 - Master" name="8 - LAN2" speed=1Gbps
set 8 master-port="6 - Master" name="9 - LAN2" speed=1Gbps
/interface wireless
set 0 band=2ghz-b/g/n disabled=no frequency=2452 l2mtu=2290 mode=ap-bridge \
name=WLAN ssid=AWC wireless-protocol=802.11
/ip pool
add name=dhcp_pool1 ranges=192.168.11.100-192.168.11.200
/ip dhcp-server
add address-pool=dhcp_pool1 disabled=no interface=Local name=dhcp1
/interface bridge port
add bridge=Local interface="2 - Master"
add bridge=Local interface="6 - Master"
add bridge=Local interface=WLAN
/ip address
add address=192.168.11.254/24 interface=Local
/ip dhcp-client
add disabled=no interface=ether1
/ip dhcp-server network
add address=192.168.11.0/24 dns-server=192.168.11.254,208.67.222.222 gateway=\
192.168.11.254 netmask=24
/ip dns
set allow-remote-requests=yes cache-size=512KiB max-udp-packet-size=50 \
servers=192.168.10.1
/ip dns static
add address=192.168.11.254 name=router
/ip firewall filter
add chain=input comment=LAN src-address=192.168.11.0/24
add chain=forward src-address=192.168.11.0/24
add chain=input comment="Accept established connections" connection-state=\
established
add chain=input comment="Accept related connections" connection-state=related
add action=drop chain=input comment="Drop invalid connections" \
connection-state=invalid
add chain=input comment=UDP protocol=udp
add chain=input comment="Winbox in" dst-port=8291 protocol=tcp
add action=add-src-to-address-list address-list=ssh_blacklist \
address-list-timeout=1w3d chain=input connection-state=new dst-port=22 \
protocol=tcp src-address-list=ssh_stage3
add action=add-src-to-address-list address-list=ssh_stage3 \
address-list-timeout=1m chain=input connection-state=new dst-port=22 \
protocol=tcp src-address-list=ssh_stage2
add action=add-src-to-address-list address-list=ssh_stage2 \
address-list-timeout=1m chain=input connection-state=new dst-port=22 \
protocol=tcp src-address-list=ssh_stage1
add action=add-src-to-address-list address-list=ssh_stage1 \
address-list-timeout=1m chain=input connection-state=new dst-port=22 \
protocol=tcp
add chain=input comment="SSH In" dst-port=22 protocol=tcp
add chain=input comment="Allow limited pings" limit=50/5s,2 protocol=icmp
add action=drop chain=input comment="Drop excess pings" protocol=icmp
add action=log chain=input comment="Log everything else" log-prefix=\
"DROP INPUT"
add action=drop chain=input comment="Drop everything else"
/ip firewall nat
add action=masquerade chain=srcnat src-address=192.168.11.0/24
/ip smb shares
set [ find default=yes ] directory=/pub
/system clock
set time-zone-name=Australia/Sydney
/system gps
set set-system-time=no
/system health
set fan-mode=manual use-fan=auxiliary
/system lcd
set contrast=0 enabled=no port=parallel type=24x4
/system lcd page
set time disabled=yes display-time=5s
set resources disabled=yes display-time=5s
set uptime disabled=yes display-time=5s
set packets disabled=yes display-time=5s
set bits disabled=yes display-time=5s
set version disabled=yes display-time=5s
set identity disabled=yes display-time=5s
set "9 - LAN2" disabled=yes display-time=5s
set "8 - LAN2" disabled=yes display-time=5s
set "7 - LAN2" disabled=yes display-time=5s
set Local disabled=yes display-time=5s
set "6 - Master" disabled=yes display-time=5s
set "5 - LAN" disabled=yes display-time=5s
set "4 - LAN" disabled=yes display-time=5s
set "3 - LAN" disabled=yes display-time=5s
set "2 - Master" disabled=yes display-time=5s
set ether1 disabled=yes display-time=5s
set WLAN disabled=yes display-time=5s
/system ntp client
set enabled=yes primary-ntp=192.189.54.17 secondary-ntp=202.158.218.239
/system routerboard settings
set force-backup-booter=yes
/tool user-manager customer
add backup-allowed=yes disabled=no login=admin parent=admin password="" \
paypal-accept-pending=no paypal-allowed=no paypal-secure-response=no \
permissions=owner signup-allowed=no time-zone=-00:00
You do not have the required permissions to view the files attached to this post.
 
scampbell
Trainer
Trainer
Posts: 487
Joined: Thu Jun 22, 2006 5:20 am
Location: Wellington, NZ
Contact:

Re: Understanding switch performance

Thu Feb 09, 2012 12:12 am

Be sure to also check /system routerboard and ensure latest FW is running :-)
 
brandonrossl
Frequent Visitor
Frequent Visitor
Posts: 61
Joined: Wed Jun 08, 2011 10:09 pm

Re: Understanding switch performance

Thu Feb 09, 2012 3:02 pm

So the lesson to be learned for today is that if you really need a switch, buy a dedicated one. :lol:
 
noven
just joined
Topic Author
Posts: 14
Joined: Mon Jul 05, 2010 6:44 am

Re: Understanding switch performance

Tue Feb 14, 2012 6:05 am

Hmm, interesting results. CPU is same as what I see. Actual throughput is not. Using latest firmware.

Will try same test on a different board and see if itv is something uniquely flaky with that one.

Who is online

Users browsing this forum: godel0914, RobertsN, TheCat12 and 102 guests