I have setup two MTs to test the bonding. I followed the info here http://www.mikrotik.com/docs/ros/2.9/interface/bonding but it's not even trying to work. Need some help PLEASE!
--------------------------------------------------------------------------------------
[admin@ColoBonder] ip> address print
Flags: X - disabled, I - invalid, D - dynamic
# ADDRESS NETWORK BROADCAST INTERFACE
0 172.16.2.1/24 172.16.2.0 172.16.2.255 ether5
1 172.16.1.1/24 172.16.1.0 172.16.1.255 ether4
2 206.251.xxx.xxx/29 206.251.xxx.xxx 206.251.xxx.xxx ether2
3 10.10.10.1/24 10.10.10.0 10.10.10.255 ether2
4 3.3.3.1/24 3.3.3.0 3.3.3.255 bonding1
[admin@ColoBonder] interface> print
Flags: X - disabled, D - dynamic, R - running
# NAME TYPE RX-RATE TX-RATE MTU
0 X ether1 ether 0 0 1500
1 R ether2 ether 0 0 1500
2 R ether3 ether 0 0 1500
3 R ether4 ether 0 0 1500
4 R ether5 ether 0 0 1500
5 R eoip-tunnel2 eoip-tunnel 0 0 1500
6 R eoip-tunnel1 eoip-tunnel 0 0 1500
7 R bonding1 bonding 0 0 1500
[admin@ColoBonder] interface> bonding print
Flags: X - disabled, R - running
0 R name="bonding1" mtu=1500 mac-address=FE:FD:00:00:00:03 arp=enabled
slaves=eoip-tunnel1,eoip-tunnel2 mode=balance-rr primary=none
link-monitoring=none arp-interval=00:00:00.100 mii-interval=00:00:00.100
down-delay=00:00:00 up-delay=00:00:00 lacp-rate=30secs
[admin@ColoBonder] interface> eoip print
Flags: X - disabled, R - running
0 R name="eoip-tunnel2" mtu=1500 mac-address=FE:FD:00:00:00:04 arp=enabled
remote-address=172.16.2.2 tunnel-id=2
1 R name="eoip-tunnel1" mtu=1500 mac-address=FE:FD:00:00:00:03 arp=enabled
remote-address=172.16.1.2 tunnel-id=1
--------------------------------------------------------------------------------------
[admin@ClientBonder] > ip address print
Flags: X - disabled, I - invalid, D - dynamic
# ADDRESS NETWORK BROADCAST INTERFACE
0 172.16.2.2/24 172.16.2.0 172.16.2.255 ether3
1 172.16.1.2/24 172.16.1.0 172.16.1.255 ether2
2 10.10.10.2/24 10.10.10.0 10.10.10.255 ether1
3 3.3.3.2/24 3.3.3.0 3.3.3.255 bonding1
[admin@ClientBonder] > interface print
Flags: X - disabled, D - dynamic, R - running
# NAME TYPE RX-RATE TX-RATE MTU
0 R eoip-tunnel2 eoip-tunnel 0 0 1500
1 R eoip-tunnel1 eoip-tunnel 0 0 1500
2 R ether1 ether 0 0 1500
3 R ether2 ether 0 0 1500
4 R ether3 ether 0 0 1500
5 R bonding1 bonding 0 0 1500
[admin@ClientBonder] > interface bonding print
Flags: X - disabled, R - running
0 R name="bonding1" mtu=1500 mac-address=FE:FD:00:00:00:01 arp=enabled
slaves=eoip-tunnel1,eoip-tunnel2 mode=balance-rr primary=none
link-monitoring=none arp-interval=00:00:00.100
mii-interval=00:00:00.100 down-delay=00:00:00 up-delay=00:00:00
lacp-rate=30secs
[admin@ClientBonder] > interface eoip print
Flags: X - disabled, R - running
0 R name="eoip-tunnel2" mtu=1500 mac-address=FE:FD:00:00:00:02 arp=enabled
remote-address=172.16.2.1 tunnel-id=2
1 R name="eoip-tunnel1" mtu=1500 mac-address=FE:FD:00:00:00:01 arp=enabled
remote-address=172.16.1.1 tunnel-id=1
-------------------------------------------------------------------------------------
Has Anyone Been Working WithBonding *Feeling black balled*
Last edited by jober on Tue Apr 19, 2005 9:46 am, edited 1 time in total.
Upgraded to fix, I think
I just upgraded the two MTs from 2.9.16 to the 2.9.18 and it's working now. But, I upgraded via FTP and after the reboots it still reads as 2.9.16.
As long as it's working I don't care.
Sorry, I'm I dumb ass. I had the FTP pointing to the wrong files. LOL
Now I need to know if it will brake the bonding if I brige the Bonding1 interface to the ether1 interface?
Going to try it anyway. I'll let ya know!
As long as it's working I don't care.
Sorry, I'm I dumb ass. I had the FTP pointing to the wrong files. LOL
Now I need to know if it will brake the bonding if I brige the Bonding1 interface to the ether1 interface?
Going to try it anyway. I'll let ya know!
Last edited by jober on Tue Apr 19, 2005 9:42 am, edited 1 time in total.
It's All Good
Bridging is working!
Now if I can figure out the packing.
Now if I can figure out the packing.
??? you pasted it youself:
http://www.mikrotik.com/docs/ros/2.9/interface/bonding
so what is not working, please describe your setup. what you have set up on one end, and what on the other ?
http://www.mikrotik.com/docs/ros/2.9/interface/bonding
so what is not working, please describe your setup. what you have set up on one end, and what on the other ?
I am trying to change the Interface bonding link-monitor to arp. But in winbow the slaves show up as ether1 and ether1 where as the terminal show it right as eoip1 and eoip2.
So if i try to change the link-monitor seting or anything else it give me an error.
LOL, I guess what I need is for you to hold my hand and walk me thru the terminal commands on editing the bonding interface. LOL
So if i try to change the link-monitor seting or anything else it give me an error.
LOL, I guess what I need is for you to hold my hand and walk me thru the terminal commands on editing the bonding interface. LOL
I'm not the technical genius of our company, but I am the one that puts in the most time.nice to see such quick progress
Joe "the tech master" gave up on the MT bonding because of some routeing problems, but me, I beleive MT can do it.
I'm going to set it up as a bridge first, and after all the bumps are hammered out I'll try the routing.
Is there any way to prioritize the ACK Packet in MT?
On asymmetric Internet links like DSL and often Cable, a big upload that consumes all of the available upstream bandwidth can render the link almost unusable by producing a huge backlog in the DSL/Cable modem's buffer, thus increasing the delay to several seconds. Because ACK packets (TCP acknowledgments) for received data are delayed or even lost as well, download speed drops, too.
This problem can be solved by prioritizing these ACK packets, so they will be sent out before any other upload packets.
Here's how to do it with m0n0wall:
First of all, you need m0n0wall pb24 or later. Start by adding a new pipe to the traffic shaper. This is necessary because we need to move the upstream queue into m0n0wall (where the order in which packets are sent out can be changed while packets are in the queue) rather than the DSL/Cable modem. Once the packets are in the DSL/Cable modem's output queue, there's no way of having ACK packets sent out immediately anymore. Therefore, it is very important to set that pipe's bandwidth to a value that is slightly below the effective upstream bandwidth of your Internet link. Don't forget that e.g. 128 kbps ADSL line speed is only about 100 kbps effective. If you set this value too high, your modem buffer will still become full and prioritization will accomplish nothing.
When you have added that pipe, add two queues linked to that pipe with different weights, e.g. one queue with weight = 10 and one with weight = 1. The first queue becomes your high priority queue.
Now it's time to add rules that classify upstream traffic into one of these two queues. There are loads of possibilities, e.g. prioritizing by TCP/UDP port, but for now we'll focus on IP packet length and TCP flags. Add a new traffic shaper rule, link it to the first (high-priority) queue, interface = WAN, protocol = TCP, source = any, destination = any, direction = out, IP packet length 0-80, TCP flags: ACK = set, everything else = don't care. It is not sufficient to classify packets into the high-priority queue based on the ACK flag only, because (big) upstream TCP data packets can have the ACK flag set as well. 0-80 is just an example to get you started. Save the rule, and add another one below it, linked to the second (low priority) queue, interface = WAN, protocol = any, source = any, destination = any, direction = out. Enable the traffic shaper if necessary, apply the changes - that's it. Here are a few points to remember:
*
make sure no upstream Internet traffic can bypass the pipe
*
despite ACK prioritization, the delay will still go up, as it is not possible to stop sending a big packet mid-way. For example, a full-size (1500 bytes) packet at 100 kbps will take 120 ms
*
if you want to be able to surf the web while performing a large upload, you'll also have to prioritize HTTP upstream traffic (i.e. destination port = 80) - otherwise, TCP SYN packets (for connection establishment) to web servers will not get prioritized, and there will be a big initial delay until a connection is established. Prioritizing DNS packets is a good idea as well.
*
If you want to find out what prioritization does for you, add a rule to classify outgoing ICMP packets into the high-priority queue and try pinging some Internet host while you're uploading - once with the traffic shaper on, and once off. There should be a huge difference in response times.
On asymmetric Internet links like DSL and often Cable, a big upload that consumes all of the available upstream bandwidth can render the link almost unusable by producing a huge backlog in the DSL/Cable modem's buffer, thus increasing the delay to several seconds. Because ACK packets (TCP acknowledgments) for received data are delayed or even lost as well, download speed drops, too.
This problem can be solved by prioritizing these ACK packets, so they will be sent out before any other upload packets.
Here's how to do it with m0n0wall:
First of all, you need m0n0wall pb24 or later. Start by adding a new pipe to the traffic shaper. This is necessary because we need to move the upstream queue into m0n0wall (where the order in which packets are sent out can be changed while packets are in the queue) rather than the DSL/Cable modem. Once the packets are in the DSL/Cable modem's output queue, there's no way of having ACK packets sent out immediately anymore. Therefore, it is very important to set that pipe's bandwidth to a value that is slightly below the effective upstream bandwidth of your Internet link. Don't forget that e.g. 128 kbps ADSL line speed is only about 100 kbps effective. If you set this value too high, your modem buffer will still become full and prioritization will accomplish nothing.
When you have added that pipe, add two queues linked to that pipe with different weights, e.g. one queue with weight = 10 and one with weight = 1. The first queue becomes your high priority queue.
Now it's time to add rules that classify upstream traffic into one of these two queues. There are loads of possibilities, e.g. prioritizing by TCP/UDP port, but for now we'll focus on IP packet length and TCP flags. Add a new traffic shaper rule, link it to the first (high-priority) queue, interface = WAN, protocol = TCP, source = any, destination = any, direction = out, IP packet length 0-80, TCP flags: ACK = set, everything else = don't care. It is not sufficient to classify packets into the high-priority queue based on the ACK flag only, because (big) upstream TCP data packets can have the ACK flag set as well. 0-80 is just an example to get you started. Save the rule, and add another one below it, linked to the second (low priority) queue, interface = WAN, protocol = any, source = any, destination = any, direction = out. Enable the traffic shaper if necessary, apply the changes - that's it. Here are a few points to remember:
*
make sure no upstream Internet traffic can bypass the pipe
*
despite ACK prioritization, the delay will still go up, as it is not possible to stop sending a big packet mid-way. For example, a full-size (1500 bytes) packet at 100 kbps will take 120 ms
*
if you want to be able to surf the web while performing a large upload, you'll also have to prioritize HTTP upstream traffic (i.e. destination port = 80) - otherwise, TCP SYN packets (for connection establishment) to web servers will not get prioritized, and there will be a big initial delay until a connection is established. Prioritizing DNS packets is a good idea as well.
*
If you want to find out what prioritization does for you, add a rule to classify outgoing ICMP packets into the high-priority queue and try pinging some Internet host while you're uploading - once with the traffic shaper on, and once off. There should be a huge difference in response times.
me thinks you can already do it with routeros v 2.9, marking all ack packets in mangle, and then prioritizing them in queue tree. please note that no prioritization will happen if you will not mark and enqueue also everything else except the ack, so at least two mangles and at least two queues would be needed
it works !!!!!!!!!!!!!!!!!!!!
WELL WELL WELL, The fool got it to work. LOL HAHAHAHAHAHAHAHAHAHa
It took some time but it's working now!!!!
Should I post my configs?
Only bad part is the Joe (one of my partners) has already got a Linux solution going so it's going to be really hard to get my company to pay $45.00 for something thats free.
I'm not hoping in any way that his solution fails, but we have only used it for a week in three locations.
I the morning I'm going to move the setup from my house to a location that has had a 100% failure rate on all the bonding solutions we've tried.
Wish me luck!
It took some time but it's working now!!!!
Should I post my configs?
Only bad part is the Joe (one of my partners) has already got a Linux solution going so it's going to be really hard to get my company to pay $45.00 for something thats free.
I'm not hoping in any way that his solution fails, but we have only used it for a week in three locations.
I the morning I'm going to move the setup from my house to a location that has had a 100% failure rate on all the bonding solutions we've tried.
Wish me luck!
Re: it works !!!!!!!!!!!!!!!!!!!!
Yes, Sir! Please do post it!Should I post my configs?
This is the colo side. I will post the client side ASAP. The test this morning at the PC shop worked great, but the failover isnt working right.
I would be great if others would share their info with us as well.
Oh, And I know it's a bad idea to post IPs but they will be changed after the testing is done. So if any one wants to try setting up the client side they can connect to my server side. __________________________________________________________
[admin@ColoBonder] > ip address print
Flags: X - disabled, I - invalid, D - dynamic
# ADDRESS NETWORK BROADCAST INTERFACE
0 10.10.10.1/24 10.10.10.0 10.10.10.255 ether1
1 3.3.3.1/24 3.3.3.0 3.3.3.255 bonding1
2 206.251.180.26/24 206.251.180.0 206.251.180.255 ether3
3 10.10.200.100/24 10.10.200.0 10.10.200.255 eoip-tunnel2
4 10.10.100.100/24 10.10.100.0 10.10.100.255 eoip-tunnel1
[admin@ColoBonder] > ip route print
Flags: X - disabled, A - active, D - dynamic,
C - connect, S - static, r - rip, b - bgp, o - ospf
# DST-ADDRESS G GATEWAY DISTANCE INTERFACE
0 ADC 3.3.3.0/24 bridge1
1 ADC 10.10.10.0/24 bridge1
2 ADC 10.10.100.0/24 eoip-tunnel1
3 ADC 10.10.200.0/24 eoip-tunnel2
4 ADC 206.251.180.0/24 ether3
5 A S 0.0.0.0/0 r 206.251.180.1 ether3
[admin@ColoBonder] > interface print
Flags: X - disabled, D - dynamic, R - running
# NAME TYPE RX-RATE TX-RATE MTU
0 R eoip-tunnel2 eoip-tunnel 0 0 1500
1 R eoip-tunnel1 eoip-tunnel 0 0 1500
2 R ether1 ether 0 0 1500
3 R ether2 ether 0 0 1500
4 R ether3 ether 0 0 1500
5 R bonding1 bonding 0 0 1500
6 R bridge1 bridge 0 0 1500
[admin@ColoBonder] > interface bonding print
Flags: X - disabled, R - running
0 R name="bonding1" mtu=1500 mac-address=FE:FD:00:00:00:01 arp=enabled
slaves=eoip-tunnel1,eoip-tunnel2 mode=balance-rr primary=none
link-monitoring=arp arp-interval=100ms
arp-ip-targets=209.205.189.138,209.205.184.85 mii-interval=100ms
down-delay=0s up-delay=0s lacp-rate=30secs
[admin@ColoBonder] > interface eoip print
Flags: X - disabled, R - running
0 R name="eoip-tunnel2" mtu=1500 mac-address=FE:FD:00:00:00:02 arp=enabled
remote-address=209.205.189.138 tunnel-id=2
1 R name="eoip-tunnel1" mtu=1500 mac-address=FE:FD:00:00:00:01 arp=enabled
remote-address=209.205.184.85 tunnel-id=1
___________________________________________________________
I still have some work to do on the failover. I also need to try the other bonding modes.
I would be great if others would share their info with us as well.
Oh, And I know it's a bad idea to post IPs but they will be changed after the testing is done. So if any one wants to try setting up the client side they can connect to my server side. __________________________________________________________
[admin@ColoBonder] > ip address print
Flags: X - disabled, I - invalid, D - dynamic
# ADDRESS NETWORK BROADCAST INTERFACE
0 10.10.10.1/24 10.10.10.0 10.10.10.255 ether1
1 3.3.3.1/24 3.3.3.0 3.3.3.255 bonding1
2 206.251.180.26/24 206.251.180.0 206.251.180.255 ether3
3 10.10.200.100/24 10.10.200.0 10.10.200.255 eoip-tunnel2
4 10.10.100.100/24 10.10.100.0 10.10.100.255 eoip-tunnel1
[admin@ColoBonder] > ip route print
Flags: X - disabled, A - active, D - dynamic,
C - connect, S - static, r - rip, b - bgp, o - ospf
# DST-ADDRESS G GATEWAY DISTANCE INTERFACE
0 ADC 3.3.3.0/24 bridge1
1 ADC 10.10.10.0/24 bridge1
2 ADC 10.10.100.0/24 eoip-tunnel1
3 ADC 10.10.200.0/24 eoip-tunnel2
4 ADC 206.251.180.0/24 ether3
5 A S 0.0.0.0/0 r 206.251.180.1 ether3
[admin@ColoBonder] > interface print
Flags: X - disabled, D - dynamic, R - running
# NAME TYPE RX-RATE TX-RATE MTU
0 R eoip-tunnel2 eoip-tunnel 0 0 1500
1 R eoip-tunnel1 eoip-tunnel 0 0 1500
2 R ether1 ether 0 0 1500
3 R ether2 ether 0 0 1500
4 R ether3 ether 0 0 1500
5 R bonding1 bonding 0 0 1500
6 R bridge1 bridge 0 0 1500
[admin@ColoBonder] > interface bonding print
Flags: X - disabled, R - running
0 R name="bonding1" mtu=1500 mac-address=FE:FD:00:00:00:01 arp=enabled
slaves=eoip-tunnel1,eoip-tunnel2 mode=balance-rr primary=none
link-monitoring=arp arp-interval=100ms
arp-ip-targets=209.205.189.138,209.205.184.85 mii-interval=100ms
down-delay=0s up-delay=0s lacp-rate=30secs
[admin@ColoBonder] > interface eoip print
Flags: X - disabled, R - running
0 R name="eoip-tunnel2" mtu=1500 mac-address=FE:FD:00:00:00:02 arp=enabled
remote-address=209.205.189.138 tunnel-id=2
1 R name="eoip-tunnel1" mtu=1500 mac-address=FE:FD:00:00:00:01 arp=enabled
remote-address=209.205.184.85 tunnel-id=1
___________________________________________________________
I still have some work to do on the failover. I also need to try the other bonding modes.