Community discussions

MUM Europe 2020
 
bbs2web
Member Candidate
Member Candidate
Topic Author
Posts: 202
Joined: Sun Apr 22, 2012 6:25 pm
Location: Johannesburg, South Africa
Contact:

300 Mbps Full Duplex 100m link - Quest - Solved

Sun Dec 02, 2012 11:15 pm

I posted an article in April (http://forum.mikrotik.com/viewtopic.php?t=61336), after upgrading a 100 metre hop between offices using R433AH with R52Hn cards on dual polarised dishes. Whilst I was expecting to loose half of the 300Mbps to protocol overheads and can transmit either way at 100Mbps (ethernet port limitation on R433AH), the link performs terribly in full duplex (http://forum.mikrotik.com/viewtopic.php?t=61340).

There are IP camera servers on either side recording the opposite side, so have a constant full duplex requirement. Am hoping for 150Mbps full duplex 'real world' transfers by using two 300Mbps links.

Got some new gear to play with:
2 x R433GL
4 x R52Hn
4 x 5GHz 11dbi dual polarised omni (current dishes are running with tx-power=0)

Plan was to setup OSPF on two dual chained links or nStream Dual but I can not get two links working concurrently. Equipment is not mounted yet so I can easily provide feedback and fiddle with things.

PS: Even tried dual single chained links where 1st card connected to H on omni 1 and 2nd card to V on omni 2. Tried with cards on opposite ends of the available spectrums but CCQ and link problems the moment both are enabled.

NB: Links work properly when run individually.

Any recommendations welcome.
Last edited by bbs2web on Mon Dec 10, 2012 11:06 am, edited 2 times in total.
 
InoX
Forum Guru
Forum Guru
Posts: 1966
Joined: Tue Jan 09, 2007 6:44 pm

Re: 300 Mbps Full Duplex 100m link - Quest

Sun Dec 02, 2012 11:40 pm

Very bad setup.
Two cards in Rb433 will almost always interfere with each other.
Omni near omni, another problem. Not to say 4 omni's...
And another thing is that a link set indoors will perform much worse than outdoor, especially nv2 links. You should bought 4xSXT, two rb750/g/gl/450g etc for settings , lower the power and make ospf. Even in this case, shielding will be an very good ideea.
Anyway, any dual chain link with mikrotik and nv2, with an -50 signal and no interference should get easy 150mbps both ways.
From what I have read in your posts, you lack a lot of wireless basics. This is the first problem.
This is an 11km link and TCP traffic.
You do not have the required permissions to view the files attached to this post.
 
bbs2web
Member Candidate
Member Candidate
Topic Author
Posts: 202
Joined: Sun Apr 22, 2012 6:25 pm
Location: Johannesburg, South Africa
Contact:

Re: 300 Mbps Full Duplex 100m link - Quest

Mon Dec 03, 2012 9:53 am

Would welcome any suggestions on improving my knowledge but don't think the problem here was necessarily a configuration problem. I'm able to transmit UDP data at 150Mbps full duplex using the RouterOS bandwidth test tool and about 110Mbps when sending UDP data between two Linux systems, after upgrading the devices to RouterOS 5.22.

2 x RB433GL
4 x R52Hn
4 x 10dBi Ubiquiti AirMax MiMo 5GHz Omni (UBAM-O-5G10)

Routing separated over chained (MiMo) links using OSPF. Screen shot shows transfer rate when using RouterOS's bandwidth test tool. AP1 has wlan1 as AP and wlan2 as station, AP2 has wlan1 as station and wlan2 as AP. Connected chains as follows on both APs:
wlan1 - chain 0 - omni 1 vertical
wlan1 - chain 1 - omni 1 horizontal
wlan2 - chain 0 - omni 2 horizontal
wlan2 - chain 1 - omni 2 vertical

Distance between omnis is about 1 metre, mounted above and below weather proof enclosure. Also using official Gigabit PoE injectors (http://routerboard.com/RBGPOE).

AP1:
5ghz_working_dual-chain - 1.gif
AP2:
5ghz_working_dual-chain - 2.gif
You do not have the required permissions to view the files attached to this post.
 
bbs2web
Member Candidate
Member Candidate
Topic Author
Posts: 202
Joined: Sun Apr 22, 2012 6:25 pm
Location: Johannesburg, South Africa
Contact:

Re: 300 Mbps Full Duplex 100m link - Quest

Mon Dec 03, 2012 11:01 am

Measured 'real' concurrent UDP transfer rate between Linux systems, connected to either AP.

AP1:
real-udp-1.gif
AP2:
real-udp-2.gif
Linux system 1 (attached to AP1):
eth0 - 192.168.254.1/30
  route add -net 192.168.254.0/24 gw 192.168.254.2
  # Limit sending speed to 200Mbps to prevent it drowning out LAN/RouterBoard
  ip link set dev eth0 qlen 100
  tc qdisc del dev eth0 root
  tc qdisc del dev eth0 ingress
  tc qdisc add dev eth0 root handle 1: htb r2q 1 default 10
  tc class add dev eth0 parent 1: classid 1:1 htb rate 500mbit
  tc class add dev eth0 parent 1:1 classid 1:10 htb rate 300mbit ceil 500mbit prio 1
  tc class add dev eth0 parent 1:1 classid 1:20 htb rate 200mbit ceil 200mbit prio 2
  tc qdisc add dev eth0 parent 1:10 handle 10: pfifo limit 20
  tc qdisc add dev eth0 parent 1:20 handle 20: pfifo limit 20
  tc filter add dev eth0 parent 1: protocol ip prio 10 u32 match ip dst 192.168.254.14/32 flowid 1:20
  tc filter add dev eth0 parent 1: protocol ip prio 11 u32 match ip dst 0.0.0.0/0 flowid 1:10
Linux system 2 (attached to AP2):
eth0 - 192.168.254.14/30
  route add -net 192.168.254.0/24 gw 192.168.254.13
  # Limit sending speed to 200Mbps to prevent it drowning out LAN/RouterBoard
  ip link set dev eth0 qlen 100
  tc qdisc del dev eth0 root
  tc qdisc del dev eth0 ingress
  tc qdisc add dev eth0 root handle 1: htb r2q 1 default 10
  tc class add dev eth0 parent 1: classid 1:1 htb rate 500mbit
  tc class add dev eth0 parent 1:1 classid 1:10 htb rate 300mbit ceil 500mbit prio 1
  tc class add dev eth0 parent 1:1 classid 1:20 htb rate 200mbit ceil 200mbit prio 2
  tc qdisc add dev eth0 parent 1:10 handle 10: pfifo limit 20
  tc qdisc add dev eth0 parent 1:20 handle 20: pfifo limit 20
  tc filter add dev eth0 parent 1: protocol ip prio 10 u32 match ip dst 192.168.254.1/32 flowid 1:20
  tc filter add dev eth0 parent 1: protocol ip prio 11 u32 match ip dst 0.0.0.0/0 flowid 1:10
Linux system 1 - Listen for data from system 2:
nc -l -u 5000 > /dev/null
Linux system 2 - Send data to system 1:
dd if=/dev/zero | nc -u 192.168.254.1 5000
Linux system 2 - Listen for data from system 1:
nc -l -u 6000 > /dev/null
Linux system 1 - Send data to system 2:
dd if=/dev/zero | nc -u 192.168.254.14 6000
You do not have the required permissions to view the files attached to this post.
 
bbs2web
Member Candidate
Member Candidate
Topic Author
Posts: 202
Joined: Sun Apr 22, 2012 6:25 pm
Location: Johannesburg, South Africa
Contact:

Re: 300 Mbps Full Duplex 100m link - Quest

Mon Dec 03, 2012 11:14 am

The problem: 40Mbps/15Mbps on full duplex TCP

Full duplex TCP only yields:
problem-full_duplex.gif
Linux system 1 - Listen for data from system 2:
nc -l 5000 > /dev/null
Linux system 2 - Send data to system 1:
dd if=/dev/zero | nc 192.168.254.1 5000
Linux system 2 - Listen for data from system 1:
nc -l 6000 > /dev/null
Linux system 1 - Send data to system 2:
dd if=/dev/zero | nc 192.168.254.14 6000
You do not have the required permissions to view the files attached to this post.
 
bbs2web
Member Candidate
Member Candidate
Topic Author
Posts: 202
Joined: Sun Apr 22, 2012 6:25 pm
Location: Johannesburg, South Africa
Contact:

Re: 300 Mbps Full Duplex 100m link - Quest

Mon Dec 03, 2012 11:18 am

Testing TCP in either direction separately provides excellent results:

AP1 to AP2:
real-tcp-1.gif
AP2 to AP1:
real-tcp-2.gif
You do not have the required permissions to view the files attached to this post.
 
Lakis
Forum Veteran
Forum Veteran
Posts: 701
Joined: Wed Sep 23, 2009 7:52 pm

Re: 300 Mbps Full Duplex 100m link - Quest

Mon Dec 03, 2012 11:36 am

I posted an article in April (http://forum.mikrotik.com/viewtopic.php?t=61336), after upgrading a 100 metre hop between offices using R433AH with R52Hn cards on dual polarised dishes. Whilst I was expecting to loose half of the 300Mbps to protocol overheads and can transmit either way at 100Mbps (ethernet port limitation on R433AH), the link performs terribly in full duplex (http://forum.mikrotik.com/viewtopic.php?t=61340).

There are IP camera servers on either side recording the opposite side, so have a constant full duplex requirement. Am hoping for 150Mbps full duplex 'real world' transfers by using two 300Mbps links.

Got some new gear to play with:
2 x R433GL
4 x R52Hn
4 x 5GHz 11dbi dual polarised omni (current dishes are running with tx-power=0)

Plan was to setup OSPF on two dual chained links or nStream Dual but I can not get two links working concurrently. Equipment is not mounted yet so I can easily provide feedback and fiddle with things.

PS: Even tried dual single chained links where 1st card connected to H on omni 1 and 2nd card to V on omni 2. Tried with cards on opposite ends of the available spectrums but CCQ and link problems the moment both are enabled.

NB: Links work properly when run individually.

Any recommendations welcome.
300/300 mbits its data rate
aggregate throughput is different - but they corespondent eachother
in my experience almost I never use 300/300 I m lowering data rate to 180/180 - I have best results
 
InoX
Forum Guru
Forum Guru
Posts: 1966
Joined: Tue Jan 09, 2007 6:44 pm

Re: 300 Mbps Full Duplex 100m link - Quest

Mon Dec 03, 2012 2:02 pm

His problem is only download + upload. Only down or only up is working. This is what I've understand. It's a lot to read in those posts. Could you make a screenshot to the wireless status while band test? I'm interested in ccq rx and tx.
Anyway I haven't read any succes story about mikrotik fullduplex with N cards. Maybe you should try bonding instead. It worked very good for me, 330mbps UPS and 240 TCP but with very directional antennas.http://wiki.ubnt.com/Mikrotik_bonding
 
bbs2web
Member Candidate
Member Candidate
Topic Author
Posts: 202
Joined: Sun Apr 22, 2012 6:25 pm
Location: Johannesburg, South Africa
Contact:

Re: 300 Mbps Full Duplex 100m link - Quest

Tue Dec 04, 2012 9:39 am

That is correct. Herewith a summary as I understand the screenshots above might confuse the matter:

2 x RB433GL
4 x R52Hn
4 x 10dBi Ubiquiti AirMax MiMo 5GHz Omni (UBAM-O-5G10)

NB: I've disabled the second chain on the R52Hn cards to prevent the two links from running on the same polarisation, although there is 260MHz separation between the two frequencies.

Running 5GHz-N, nv2 protocol with routing separated over links using OSPF (routing table visible in bottom of screen shot):
AP1 wlan1 (ap bridge) - chain 0 - omni 1 vertical - omni 3 vertical - chain 0 - (station) wlan1 AP2
AP1 wlan2 (station) - chain 0 - omni 2 horizontal - omni 4 horizontal - chain 0 - (ap bridge) wlan2 AP2

Distance between omnis is about 1 metre, mounted above and below weather proof enclosure. Current distance between omni is about 8 meters (gear is in a lab environment, intended purpose is to jump a road between our offices; distance of about 100 meters). Using official Gigabit PoE injectors (http://routerboard.com/RBGPOE).

Results:
Uni directional UDP is excellent (either direction)
Uni direction TCP is excellent (either direction)
Full duplex UDP is excellent (screenshots earlier in this post, when links were chained)
Full duplex TCP is terrible (screenshots earlier in this post, when links were chained)

Screenshot of AP1 when transmitting TCP concurrently (single chain):
concurrent_tcp_ap1.gif
Screenshot of AP2 when transmitting TCP concurrently (single chain):
concurrent_tcp_ap2.gif
You do not have the required permissions to view the files attached to this post.
 
scampbell
Trainer
Trainer
Posts: 458
Joined: Thu Jun 22, 2006 5:20 am
Location: Wellington, NZ
Contact:

300 Mbps Full Duplex 100m link - Quest

Tue Dec 04, 2012 10:59 am

You may want to try shielding your radio cards to avoid crosstalk between them. Also try limiting output power if possible
MTCNA, MTCWE, MTCRE, MTCTCE, MTCSE, MTCINE, Trainer
___________________
Mikrotik Distributor - New Zealand
http://www.campbell.co.nz
 
bbs2web
Member Candidate
Member Candidate
Topic Author
Posts: 202
Joined: Sun Apr 22, 2012 6:25 pm
Location: Johannesburg, South Africa
Contact:

Re: 300 Mbps Full Duplex 100m link - Quest

Tue Dec 04, 2012 1:29 pm

Wouldn't the links then report the CCQ below 100%?
Full duplex UDP is working at an acceptable level.

Any recommendations on what I can try to do to shield the R52Hn cards from each other? A small sheet or tin placed in an anti-static packet between the two radio cards?

The transmit power is currently set as 'all-rates-fixed' with a 'tx-power' value of 11 on all 4 R52Hn cards. The link with the signal strength > 60 dB is actually the one that performs better though... I'll try increase the transmit power on the radios connected to the horizontal polarization to get them both to the recommended level (between 50-60 dB).
 
InoX
Forum Guru
Forum Guru
Posts: 1966
Joined: Tue Jan 09, 2007 6:44 pm

Re: 300 Mbps Full Duplex 100m link - Quest

Tue Dec 04, 2012 5:48 pm

If ccq is 100% there is no interference from my experience. I just haven't seen any success story about full duplex with N cards and NV2.
 
bbs2web
Member Candidate
Member Candidate
Topic Author
Posts: 202
Joined: Sun Apr 22, 2012 6:25 pm
Location: Johannesburg, South Africa
Contact:

Re: 300 Mbps Full Duplex 100m link - Quest

Wed Dec 05, 2012 11:28 am

For documentation purposes of this lab environment:
  • 2 x RB433GL connected to Linux systems using official Gigabit PoE adapters
    2 x RB433GL
    4 x R52Hn wireless cards (2 per enclosure)
    4 x 10dBi Ubiquiti AirMax MiMo 5GHz Omni (UBAM-O-5G10)
Layout:
ie: Linux 1Gbps - ether1 RB433GL R52Hn wlan1 ---------- wlan1 R52Hn RB433GL ether1 - 1Gbps Linux
                                 R52Hn wlan2 ---------- wlan2 R52Hn
Tests were performed by using NetCat (nc) to generating raw UDP traffic between the two Linux systems and TCP transfers used SCP. Routing separated over wireless links using OSPF. AP1 has wlan1 as AP and wlan2 as station, AP2 has wlan1 as station and wlan2 as AP (this is due to a previous observation that traffic transfer rates favoured AP to station).

Distance between transmitting and receiving omnis is about 1 metre, mounted above and below weather proof enclosure. Distance between gear is about 8 meters. Transmit power on the wireless cards was set as 'tx-power-mode=all-rates-fixed' with 'tx-power=11' to get the signal strength to the recommended level (between 50 and 60 dB). Frequency separation is 260MHz to reduce interference.

NB: Half duplex here refers to sending uni-directional test traffic in one direction at a time (ie: AP1 to AP2). OSPF was active during all 12 transfer tests, to split the routed traffic over the two links, so that data was only ever transmitted in a single direction over either link (ie: 2 x half duplex = 'proper' full duplex).

Speed test 1 - Chained (MiMo):
Antennae connections:
  • wlan1 - chain 0 - omni 1 vertical
    wlan1 - chain 1 - omni 1 horizontal
    wlan2 - chain 0 - omni 2 horizontal
    wlan2 - chain 1 - omni 2 vertical
  • UDP half duplex: 150Mbps/200+Mbps (Linux sending limited to 200Mbps)
    UDP full duplex: 97Mbps/120Mbps
    TCP half duplex: 135Mbps/200+Mbps (Linux sending limited to 200Mbps)
    TCP full duplex: 35Mbps/18Mbps
Speed test 2 - No chaining (SiSo):
Antennae connections:
  • wlan1 - chain 0 - omni 1 vertical
    wlan2 - chain 0 - omni 2 horizontal
  • UDP half duplex: 108Mbps/120Mbps
    UDP full duplex: 106Mbps/118Mbps
    TCP half duplex: 108Mbps/120Mbps
    TCP full duplex: 38Mbps/16Mbps

AP1 Configuration:
IP Address configuration:
/ip address
  add address=192.168.254.2/30 interface=ether1
  add address=192.168.254.5/30 interface=wlan1
  add address=192.168.254.9/30 interface=wlan2
OSPF routing configuration:
/routing ospf instance
  set [ find default=yes ] distribute-default=if-installed-as-type-1 metric-default=10 redistribute-connected=\
    as-type-1 redistribute-static=as-type-1 router-id=192.168.254.2
/routing ospf interface
  add interface=wlan1 network-type=nbma retransmit-interval=1s use-bfd=yes
  add interface=wlan2 network-type=nbma cost=100 priority=0 retransmit-interval=1s use-bfd=yes
/routing ospf network
  add area=backbone network=192.168.254.4/30
  add area=backbone network=192.168.254.8/30
/routing ospf nbma-neighbor
  add address=192.168.254.6 poll-interval=10s priority=1
  add address=192.168.254.10 poll-interval=10s
Wireless configuration:
/interface wireless
set 0 adaptive-noise-immunity=ap-and-client-mode band=5ghz-onlyn channel-width=20/40mhz-ht-above country="south africa" \
    disabled=no frequency=5240 ht-basic-mcs=mcs-7 ht-rxchains=0,1 ht-supported-mcs=mcs-7,mcs-15 ht-txchains=0,1 hw-retries=15 \
    l2mtu=2290 max-station-count=1 mode=ap-bridge nv2-cell-radius=10 nv2-preshared-key=nicetry123 nv2-security=enabled \
    radio-name=Syrex27_7-Tx scan-list=5240 ssid=Syrex-Tx tx-power=11 tx-power-mode=all-rates-fixed wireless-protocol=nv2
set 1 adaptive-noise-immunity=ap-and-client-mode band=5ghz-onlyn channel-width=20/40mhz-ht-above country="south africa" \
    disabled=no frequency=5500 ht-basic-mcs=mcs-7 ht-rxchains=0,1 ht-supported-mcs=mcs-7,mcs-15 ht-txchains=0,1 hw-retries=15 \
    l2mtu=2290 max-station-count=1 mode=station nv2-cell-radius=10 nv2-preshared-key=nicetry123 nv2-security=enabled \
    radio-name=Syrex27_7-Rx scan-list=5500 ssid=Syrex-Rx tx-power=11 tx-power-mode=all-rates-fixed wireless-protocol=nv2
AP2 Configuration:
IP Address configuration:
/ip address
  add address=192.168.254.6/30 interface=wlan1
  add address=192.168.254.10/30 interface=wlan2
  add address=192.168.254.13/30 interface=ether1
OSPF routing configuration:
/routing ospf instance
  set [ find default=yes ] distribute-default=if-installed-as-type-1 metric-default=10 redistribute-connected=\
    as-type-1 redistribute-static=as-type-1 router-id=192.168.254.13
/routing ospf interface
  add interface=wlan1 network-type=nbma cost=100 priority=0 retransmit-interval=1s use-bfd=yes
  add interface=wlan2 network-type=nbma retransmit-interval=1s use-bfd=yes
/routing ospf network
  add area=backbone network=192.168.254.4/30
  add area=backbone network=192.168.254.8/30
/routing ospf nbma-neighbor
  add address=192.168.254.5 poll-interval=10s
  add address=192.168.254.9 poll-interval=10s priority=1
Wireless configuration:
/interface wireless
set 0 adaptive-noise-immunity=ap-and-client-mode band=5ghz-onlyn channel-width=20/40mhz-ht-above country="south africa" \
    disabled=no frequency=5240 ht-basic-mcs=mcs-7 ht-rxchains=0,1 ht-supported-mcs=mcs-7,mcs-15 ht-txchains=0,1 hw-retries=15 \
    l2mtu=2290 max-station-count=1 mode=station nv2-cell-radius=10 nv2-preshared-key=nicetry123 nv2-security=enabled \
    radio-name=Syrex30_7-Rx scan-list=5240 ssid=Syrex-Tx tx-power=11 tx-power-mode=all-rates-fixed wireless-protocol=nv2
set 1 adaptive-noise-immunity=ap-and-client-mode band=5ghz-onlyn channel-width=20/40mhz-ht-above country="south africa" \
    disabled=no frequency=5500 ht-basic-mcs=mcs-7 ht-rxchains=0,1 ht-supported-mcs=mcs-7,mcs-15 ht-txchains=0,1 hw-retries=15 \
    l2mtu=2290 max-station-count=1 mode=ap-bridge nv2-cell-radius=10 nv2-preshared-key=nicetry123 nv2-security=enabled \
    radio-name=Syrex30_7-Tx scan-list=5500 ssid=Syrex-Rx tx-power=11 tx-power-mode=all-rates-fixed wireless-protocol=nv2
Last edited by bbs2web on Thu Dec 06, 2012 1:58 pm, edited 7 times in total.
 
bbs2web
Member Candidate
Member Candidate
Topic Author
Posts: 202
Joined: Sun Apr 22, 2012 6:25 pm
Location: Johannesburg, South Africa
Contact:

Re: 300 Mbps Full Duplex 100m link - Quest

Wed Dec 05, 2012 11:36 am

Screenshot collection 1 of 2.

MiMo - TCP transfer screenshots.zip
  • MiMo - TCP AP1 to AP2.gif
    MiMo - TCP AP2 to AP1.gif
    MiMo - TCP full duplex
MiMo - UDP transfer screenshots.zip
  • MiMo - UDP AP1 to AP2.gif
    MiMo - UDP AP2 to AP1.gif
    MiMo - UDP full duplex
You do not have the required permissions to view the files attached to this post.
 
bbs2web
Member Candidate
Member Candidate
Topic Author
Posts: 202
Joined: Sun Apr 22, 2012 6:25 pm
Location: Johannesburg, South Africa
Contact:

Re: 300 Mbps Full Duplex 100m link - Quest

Wed Dec 05, 2012 11:38 am

Screenshot collection 2 of 2.

SiSo - TCP transfer screenshots.zip
  • SiSo - TCP AP1 to AP2.gif
    SiSo - TCP AP2 to AP1.gif
    SiSo - TCP full duplex
SiSo - UDP transfer screenshots.zip
  • SiSo - UDP AP1 to AP2.gif
    SiSo - UDP AP2 to AP1.gif
    SiSo - UDP full duplex
You do not have the required permissions to view the files attached to this post.
 
InoX
Forum Guru
Forum Guru
Posts: 1966
Joined: Tue Jan 09, 2007 6:44 pm

Re: 300 Mbps Full Duplex 100m link - Quest

Wed Dec 05, 2012 5:14 pm

use bonding, you just need to make some settings in 2-3min is working.
http://forum.mikrotik.com/viewtopic.php?f=2&t=56028
http://wiki.ubnt.com/Mikrotik_bonding
 
bbs2web
Member Candidate
Member Candidate
Topic Author
Posts: 202
Joined: Sun Apr 22, 2012 6:25 pm
Location: Johannesburg, South Africa
Contact:

Re: 300 Mbps Full Duplex 100m link - Quest

Wed Dec 05, 2012 11:18 pm

The wiki article you refrenced specifically states that it's a half duplex solution. I don't want to bond the wireless links, I want the maximum possible speed in full duplex so that transmitting data in either direction doesn't affect the other.

I have the link running properly now after moving the 2nd pair of R52Hn cards to an additional pair of RB433GL boards. The OSPF routing essentially provides an automated safety net, should one of the wireless links go down, but separates traffic to ensure each link only transmits in a single direction (2 x half duplex = 'proper' full duplex).

I'll provide test results tomorrow...
 
bbs2web
Member Candidate
Member Candidate
Topic Author
Posts: 202
Joined: Sun Apr 22, 2012 6:25 pm
Location: Johannesburg, South Africa
Contact:

Re: 300 Mbps Full Duplex 100m link - Quest

Mon Dec 10, 2012 10:46 am

There appear to be major performance issues when asking the MikroTik RB433GL (probably not limited to this specific model) to route packets concurrently. The following setup completely separates the dual WiFi links but performance is terrible when asking the MikroTik to essentially apply routing rules to split traffic:
Linux
192.168.254.1/30
192.168.254.2/30 - ether1  RB433GL  wlan1 192.168.254.13/30 --- 192.168.254.14/30 wlan1 RB433GL ether1 - 192.168.254.25/30
                           ether2                                                       ether2
  Syrex27_7_tx        192.168.254.9/30                                             192.168.254.21/30        Syrex30_7_rx
                             |                                                            |
                             |                                                            |
  Syrex27_7_rx        192.168.254.10/30                                            192.168.254.22/30        Syrex30_7_tx
                           ether2                                                       ether2
192.168.254.6/30 - ether1 RB433GL   wlan1 192.168.254.17/30 --- 192.168.254.18/30 wlan1 RB433GL ether1 - 192.168.254.29/30
                                                                                                         192.168.254.30/30
                                                                                                                     Linux
Routing via OSPF, performance degradation when 30_7_rx has to forward traffic out of ether2 to reach the system connected to 307_7_tx's ether1 interface and performance degradation when 27_7_rx has to forward traffic for system on other side via 27_7_tx.

Moving system to ether1 on 30_7_rx results in spectacular speeds when sending from 192.168.254.1 towards 192.168.254.26 then but terrible (5Mbps!) performance when transmitting data from 192.168.254.26 to 192.168.254.1 (due to traffic being rerouted by 30_7_rx to 30_7_tx via ether2 and 27_7_rx rerouting via 27_t_tx to reach 192.168.254.1).

NB: There are no issues with ports and cables as everything works perfectly when we set ether1 and ether2 as switch ports and adjust the IP addresses accordingly (next post).
Last edited by bbs2web on Mon Dec 10, 2012 10:59 am, edited 1 time in total.
 
bbs2web
Member Candidate
Member Candidate
Topic Author
Posts: 202
Joined: Sun Apr 22, 2012 6:25 pm
Location: Johannesburg, South Africa
Contact:

Re: 300 Mbps Full Duplex 100m link - Quest - Solved (NB)

Mon Dec 10, 2012 10:59 am

We finally have a proper full duplex WiFi link working in a lab environment which will be mounted during the course of this week. We are now able to realise close on 200Mbps concurrently.

For documentation purposes of this lab environment:
4 x RB433GL (800MHz CPU, 3 Gigabit LAN, 3 mini PCI)
4 x RGPOE (Gigabit PoE injectors)
4 x R52Hn wireless cards (1 per weather proof enclosure)
4 x 10dBi Ubiquiti AirMax MiMo 5GHz Omni (UBAM-O-5G10)

Layout:
Linux
192.168.254.1/29
192.168.254.2/29 - ether1 RB433GL wlan1 192.168.254.9/30 --- 192.168.254.10/30 wlan1 RB433GL ether1 - 192.168.254.17/29
                          ether2 (master:ether1)                      (master:ether1) ether2
  Syrex27_7_tx              |                                                           |                 Syrex30_7_rx
                            |                                                           |
                            |                                                           |
  Syrex27_7_rx              |                                                           |                 Syrex30_7_tx
                          ether2 (master:ether1)                      (master:ether1) ether2
192.168.254.3/29 - ether1 RB433GL wlan1 192.168.254.13/30 --- 192.168.254.14/30 wlan1 RB433GL ether1 - 192.168.254.18/29
                                                                                                       192.168.254.19/29
                                                                                                                   Linux

Tests were performed by using NetCat (nc) to generating raw UDP traffic between the two Linux systems and TCP transfers used SCP. Routing separated over wireless links using OSPF. AP1 has wlan1 as AP and wlan2 as station, AP2 has wlan1 as station and wlan2 as AP (this is due to a previous observation that traffic transfer rates favoured AP to station).

Distance between transmitting and receiving omnis is about 1 metre, mounted above and below weather proof enclosure. Distance between gear is about 8 meters. Transmit power on the wireless cards was set as 'tx-power-mode=all-rates-fixed' with 'tx-power=11' to get the signal strength to the recommended level (between 50 and 60 dB). Frequency separation is 300MHz to reduce interference.

Working setup relies on ports switching traffic without using the CPU. Whilst we run two PoE points to each set to power the antenna we interlink the systems using a short cable on the ether2 ports. Only a single GPPoE cable is connected on either side and traffic is essentially switched by the physical ethernet ports.

NB: Half duplex here refers to sending uni-directional test traffic in one direction at a time (ie: AP1 to AP2). OSPF was active during all 12 transfer tests, to split the routed traffic over the two links, so that data was only ever transmitted in a single direction over either link (ie: 2 x half duplex = 'proper' full duplex).

Speed test - Chained (MiMo):
Antennae connections:
wlan1 - chain 0 - omni 1 vertical
wlan1 - chain 1 - omni 1 horizontal
wlan2 - chain 0 - omni 2 horizontal
wlan2 - chain 1 - omni 2 vertical
UDP half duplex: 205Mbps/222Mbps
UDP full duplex: 210Mbps/219Mbps
TCP half duplex: 210Mbps/218Mbps
TCP full duplex: 200Mbps/160Mbps

Syrex27_7_tx:
  /interface ethernet
    set 1 master-port=ether1
    set 2 disabled=yes
  /interface wireless
    set 0 adaptive-noise-immunity=ap-and-client-mode band=5ghz-onlyn channel-width=20/40mhz-ht-above country="south africa" \
      disabled=no frequency=5200 ht-basic-mcs=mcs-7 ht-rxchains=0,1 ht-supported-mcs=mcs-7,mcs-15 ht-txchains=0,1 hw-retries=15 \
      l2mtu=2290 max-station-count=1 mode=ap-bridge nv2-cell-radius=10 nv2-preshared-key=nicetry123 nv2-security=enabled \
      radio-name=Syrex27_7-Tx scan-list=5200 ssid=Syrex-Tx tx-power=11 tx-power-mode=all-rates-fixed wireless-protocol=nv2
  /ip firewall connection tracking
    set enabled=no
  /ip address
    add address=192.168.254.2/29 interface=ether1
    add address=192.168.254.9/30 interface=wlan1
  /ip route
    add distance=1 gateway=192.168.254.1
  /routing ospf interface
    add interface=ether1 network-type=broadcast retransmit-interval=1s use-bfd=yes cost=100
    add interface=wlan1 network-type=broadcast retransmit-interval=1s use-bfd=yes
  /routing ospf network
    add area=backbone network=192.168.254.0/29
    add area=backbone network=192.168.254.8/30
  /routing ospf instance
    set [ find default=yes ] distribute-default=if-installed-as-type-1 metric-default=10 redistribute-connected=\
      as-type-1 redistribute-static=as-type-1 router-id=192.168.254.2
  /system identity
    set name=Syrex27_7_tx
Syrex27_7_rx:
  /interface ethernet
    set 1 master-port=ether1
    set 2 disabled=yes
  /interface wireless
    set 0 adaptive-noise-immunity=ap-and-client-mode band=5ghz-onlyn channel-width=20/40mhz-ht-above country="south africa" \
      disabled=no frequency=5500 ht-basic-mcs=mcs-7 ht-rxchains=0,1 ht-supported-mcs=mcs-7,mcs-15 ht-txchains=0,1 hw-retries=15 \
      l2mtu=2290 max-station-count=1 mode=station nv2-cell-radius=10 nv2-preshared-key=nicetry123 nv2-security=enabled \
      radio-name=Syrex27_7-Rx scan-list=5500 ssid=Syrex-Rx tx-power=11 tx-power-mode=all-rates-fixed wireless-protocol=nv2
  /ip firewall connection tracking
    set enabled=no
  /ip address
    add address=192.168.254.3/29 interface=ether1
    add address=192.168.254.13/30 interface=wlan1
  /ip route
    add distance=1 gateway=192.168.254.1
  /routing ospf interface
    add interface=ether1 network-type=broadcast retransmit-interval=1s use-bfd=yes priority=0
    add interface=wlan1 network-type=broadcast retransmit-interval=1s use-bfd=yes cost=100 priority=0
  /routing ospf network
    add area=backbone network=192.168.254.0/29
    add area=backbone network=192.168.254.12/30
  /routing ospf instance
    set [ find default=yes ] distribute-default=if-installed-as-type-1 metric-default=10 redistribute-connected=\
      as-type-1 redistribute-static=as-type-1 router-id=192.168.254.3
  /system identity
    set name=Syrex27_7_rx
Syrex30_7_rx:
  /interface ethernet
    set 1 master-port=ether1
    set 2 disabled=yes
  /interface wireless
    set 0 adaptive-noise-immunity=ap-and-client-mode band=5ghz-onlyn channel-width=20/40mhz-ht-above country="south africa" \
      disabled=no frequency=5200 ht-basic-mcs=mcs-7 ht-rxchains=0,1 ht-supported-mcs=mcs-7,mcs-15 ht-txchains=0,1 hw-retries=15 \
      l2mtu=2290 max-station-count=1 mode=station nv2-cell-radius=10 nv2-preshared-key=nicetry123 nv2-security=enabled \
      radio-name=Syrex30_7-Rx scan-list=5200 ssid=Syrex-Tx tx-power=11 tx-power-mode=all-rates-fixed wireless-protocol=nv2
  /ip firewall connection tracking
    set enabled=no
  /ip address
    add address=192.168.254.17/29 interface=ether1
    add address=192.168.254.10/30 interface=wlan1
  /ip route
    add distance=1 dst-address=192.168.2.0/24 gateway=192.168.254.19
  /routing ospf interface
    add interface=ether1 network-type=broadcast retransmit-interval=1s use-bfd=yes
    add interface=wlan1 network-type=broadcast retransmit-interval=1s use-bfd=yes cost=100 priority=0
  /routing ospf network
    add area=backbone network=192.168.254.8/30
    add area=backbone network=192.168.254.16/29
  /routing ospf instance
    set [ find default=yes ] distribute-default=if-installed-as-type-1 metric-default=10 redistribute-connected=\
      as-type-1 redistribute-static=as-type-1 router-id=192.168.254.17
  /system identity
    set name=Syrex30_7_rx
Syrex30_7_tx:
  /interface ethernet
    set 1 master-port=ether1
    set 2 disabled=yes
  /interface wireless
    set 0 adaptive-noise-immunity=ap-and-client-mode band=5ghz-onlyn channel-width=20/40mhz-ht-above country="south africa" \
      disabled=no frequency=5500 ht-basic-mcs=mcs-7 ht-rxchains=0,1 ht-supported-mcs=mcs-7,mcs-15 ht-txchains=0,1 hw-retries=15 \
      l2mtu=2290 max-station-count=1 mode=ap-bridge nv2-cell-radius=10 nv2-preshared-key=nicetry123 nv2-security=enabled \
      radio-name=Syrex30_7-Tx scan-list=5500 ssid=Syrex-Rx tx-power=11 tx-power-mode=all-rates-fixed wireless-protocol=nv2
  /ip firewall connection tracking
    set enabled=no
  /ip address
    add address=192.168.254.18/29 interface=ether1
    add address=192.168.254.14/30 interface=wlan1
  /ip route
    add distance=1 dst-address=192.168.2.0/24 gateway=192.168.254.19
  /routing ospf interface
    add interface=ether1 network-type=broadcast retransmit-interval=1s use-bfd=yes cost=100 priority=0
    add interface=wlan1 network-type=broadcast retransmit-interval=1s use-bfd=yes
  /routing ospf network
    add area=backbone network=192.168.254.12/30
    add area=backbone network=192.168.254.16/29
  /routing ospf instance
    set [ find default=yes ] distribute-default=if-installed-as-type-1 metric-default=10 redistribute-connected=\
    as-type-1 redistribute-static=as-type-1 router-id=192.168.254.18
  /system identity
    set name=Syrex30_7_tx
 
bbs2web
Member Candidate
Member Candidate
Topic Author
Posts: 202
Joined: Sun Apr 22, 2012 6:25 pm
Location: Johannesburg, South Africa
Contact:

Re: 300 Mbps Full Duplex 100m link - Quest

Mon Dec 10, 2012 11:06 am

Screenshot collection.

MiMo - TCP transfer screenshots.zip
MiMo - TCP AP1 to AP2.gif
MiMo - TCP AP2 to AP1.gif
MiMo - TCP full duplex

MiMo - UDP transfer screenshots.zip
MiMo - UDP AP1 to AP2.gif
MiMo - UDP AP2 to AP1.gif
MiMo - UDP full duplex
You do not have the required permissions to view the files attached to this post.

Who is online

Users browsing this forum: dirman, romihg, ztx and 42 guests