MikroTik

Hello,

Please refer the this diagram for an overview of my setup: I need to setup a transmission unit that will be constantly moving to different locations to broadcast live HD video. I have 4 USB 3G modems (with theoretical upload speed up to 5.76Mbps each). Each USB modem will be connected to a different wireless provider. Each modem has a static public IP address associated to it.

A EoIP tunnel is set up for each 3G modem. The four EoIP tunnels are bound together.

I cannot specify on which interface I want the EoIP tunnel to be effective. The router uses its routing table to choose the right interface on which it should send the traffic. The result of this is the following: All the upload traffic from the remote site uses only ONE 3G connection.

I clearly see in the interfaces window that the traffic is following the bounding directives (separated across all 4 EoIP interfaces), but the combined upload traffic uses only ONE WAN interface.

Any idea how to specify on which PPP interface the EoIP tunnel should work ?

The only idea I had until now is to use 4 different IP addresses on the other MikroTik Router as well and to enter static routes on the remote site's router.

Thank you for your help,

Martin

Hello,

I "solved" my EoIP problem by using 4 different static IP addresses on the router at the data center.

I now have a much more problematic issue and I hope someone will be able to help me.

On each modem I can achieve 2.0-3.5 Mbps in download and 1.2-1.5 Mbps in upload speed.

Once again, my goal is to broadcast a HD video stream from a single device to a single server... so I need to receive packets in the right order and I cannot use a "per connection" load balancing mechanism.

However, when I use the bonding interface, I have lower transfer rates than with a single 3G interface!

Any idea? Your help is much appreciated...

Thank you,

Martin

I would suggest using the same USB modem and the same provider on all to try to get the closest connections as possible.

martinrochefort,

i do same , but do not have fix IP on 3G modems, then i used VPN (pptp) ...

bound interface work, both eoip, both pptp, ... but only one 3Gmodem

how you make traffic used more then one modems ?

do you used mangle, ddifferent routing tables ... ?

thanks

I have a similar setup with ADSL instead of 3G.

It is not possible to policy route each EoIP tunnel through each interface. I asked this to Mikrotik and the answer was : not possible.

I asked if it would be possible to add a mark according to Tunnel ID, but got no answer about this.

So the only solution actualy is to use multiple IP addresses on the destination router, or use a Linux box with a recent version of shorewall.


Are you using TCP or UDP traffic ?

Wich bonding method do you use ?

TCP will give bad results with bonding, excepts if you are using links of the same speed and 802.3ad. 802.3ad will garantee that packets keep their initial order.

But 802.3ad can be used only if the links have the same speed, typically between switches.

yes it si possible.

remove default route, and create fix route for peer IPs.

route for first peer setup trought first interface , same for second peer.
then create default route and setup trought bounding interface

its work.

but TCP are to slow , UDP are ok full bounding speed.

I think you misunderstand.


The problem is that the tunnels destination has only one IP. So you can't route according to destination IPs.

We need to policy route the tunnel himself through different providers to the same destination IP.


Can't be done because we can't policy route according to tunnel ID.

EoIP Source MAC is not usable neither for policy routing.

you must have 2 public IP on destination side (datacenter)

and make fix route for this IPs, each on different adsl

then crate 2 VPN tunnels, each must have different destination IP.

example for routes (public IP):
destionation x.x.x.50/30 on adsl-eth1
with this seting first VPN go trought adsl-eth1

do same for VPN 2, but on adsl-eth2
destionation x.x.x.100/30 on adsl-eth2
with this seting second VPN go trought adsl-eth2

default route 0.0.0.0/0
must be set on bounding interface

only with this setting will work.

2 separate eth for each ADSL are good,
becouse , this will work even you have DHCP and dinamic gateway on ADSL side.

we make this on mt v4.10

hi martinrochefort,

did you solve your problem on slow bonding link?

i have the same issu.

thanks

did you have problems, with modem rapidly disconect, when used 4 modems ?

and mikrotik stop working, then i need to reboot


edit:
btw, witch hub you used ?

I'm working in the same project.

With two 3G Modems, without using EoIP or VPN tunneling.

I'm marking the half of the packets and routing them through one modem 3g, and the other half through the other modem 3g.
To do this, I redirect all the traffic that comes with SRCAdd:XXX.XXX.XXX.XXX:554 to DSTADDRESS:YYY.YYY.YYY.YYY:8891 and YYY.YYY.YYY.YYY:8892

Then in the other side, I redirect all the traffic that comes from YYY.YYY.YYY.YYY:8891 and 8892 to ZZZ.ZZZ.ZZZ.ZZZ:554.

The problem is that one of the streamings is not beeing redirected through the port that should be.

Any Idea?

Did you make it work with EoIP? and VPN tunneling?

Thank you

hi martinrochefort,

did you solve your problem on slow bonding link?

i have the same issu.

thanks

Same here.

I've tried numerous types of configurations without any luck.
All types of VPN and tunnels that Mikrotik supports.
3 different routers in case there was a problem with the hardware.
7 different modems.
3 different ISPs
2 different datacenters on the other end.
different USB controllers for each modem in case the usb hub or the modems cannot simultaneously fully use the same usb channel.
All sorts of settings on the bonding interface.
I even used separate routers (Draytek Vigor) for the modems and bonded their ethernets on a 3rd Mikrotik router.

And I always get the same result:
When 2 or more modems are uploading at the same time, the upload speed suffers.
Each modem individually can upload up to 1mbit.
But together the aggregated speed cannot exceed 1mbit.
So with 2 modems their upload speed when uploading together is maximum ~500k
With 3 modems this goes down to ~330k
And so on...

I think it may be RF interference between the modems when they all transmit causing this behavior.
I haven't had the chance to confirm that, but after everything I tried I think it's the only thing left out.

But what's weird about that is that the total aggregated speed is always 1mbit.
If it was RF interference wouldn't the results vary?

I forgot to mention that the exact same setup with DSL instead of 3G works perfect.

I forgot to mention that the exact same setup with DSL instead of 3G works perfect.

What is your setup

2 EoIP and bonding?
2 VPN and bonding?


I'm doing it with mangle and packet mark and I can see how the packets are well marked and redirected to the specified default gateway, but just one stream get arrived to data center's router.

Do you have 2 or more IPs on the Datacenter router?
What type of VPNs are you using?

You don't need packet marking to do this.
You simply add a static route via the interface you want for each destination IP.

So each VPN/Tunnel goes via a specific interface.

I used EoIP over L2TP on the DSL setup I did.
I would have used directly EoIP but they didn't have static IPs on the 'dsl' end (and I was a bit bored to do some scripting to automatically update the IP on the EoIP interfaces on the datacenter router)

Do you have 2 or more IPs on the Datacenter router?
What type of VPNs are you using?

You don't need packet marking to do this.
You simply add a static route via the interface you want for each destination IP.

So each VPN/Tunnel goes via a specific interface.

I used EoIP over L2TP on the DSL setup I did.
I would have used directly EoIP but they didn't have static IPs on the 'dsl' end (and I was a bit bored to do some scripting to automatically update the IP on the EoIP interfaces on the datacenter router)

Cha0s,

I got just one public IP in my Datacenter router.
I'm not using VPN, I would like to test your method, but with one public IP address in the Datacenter I don't think it works.

In fact my idea is a little bit different.

I need to send HD video to a specific IP, and just one stream to that IP. I don't have many connections, just one connection, and I need to send some packets to one Dongle USB 3G, and the rest of the packets through the other 3G modem.

bonding with 802.3ad could make this, becouse i'ts a truly load balance, by packets not by connections. But I can't use bonding in a PPP interface.

So this is why I choosed to mark packets, and adding a routing mark.
Then I set a dst-nat rule to change the destination of all packets from one IP:PORT to one IP:Port1 and IP:port2, then in my Datacenter's router, I change the destination port again to just one port. IP:PORT.

But Mikrotik just redirect the packets to one port, not both port that i'm configuring.

I know that this is possible to do with Linux, and I allready do it and works fine. But I'm needing some board like Mikrotik has, and I think that this is possible to do with Mk.

What do you think?

any news about this topic?

bump

If you have to use TCP connections, forget it at all.

TCP requires all packets arrive orderer, and it isn't possible with most of WAN (3G, DSL, etc) connections. All packet that don't arrive in order, will be rejected.

If you had multiple identical connections (same ping, same exact jitter), it may work (for example, 3 ethernet wires)

Another option is to use ML-PPP protocol (Multi-Link PPP). Unfortunately, MK doesn't support ML-PPP server at this moment (you can use it just as client).

Regards

Just thinking, in terms of video, UDP should do the job right? as it should only result in dropped frames..