Ok, we run a small WISP and we have 1 8mb conection with 30 ip public adresses and 4 adsl4 mb conections, we balance across the five conections and everything is very fast and working ok but as the number of customers gets larger we are having the problem that they can’t download from megaupload or rapidshare because they receive the message that their ip is already downloading. This is because we are going out only from 5 ips one from each conection and don’t know how can we use the other 29 from the 8mb conection.
Is there a way to use the 30 ips from the 8mb conection?
Are you doing NAT then? You should probably start assigning each customer their own public IP; either statically, via PPPoE, or via DHCP, and let the customer router or CPE do the NAT.
Thanks for your reply, yes we are doing NAT, we can’t assign public IP adresses to the customers because we don’t have enough (we have 180 customers and only 30 ips) and also we are using 5 wans and the public ip adresses are only on 1.
Take a look at the PCC wiki. That example shows how to use PCC for load balancing, but you can just stop after marking the connection and instead of assigning a routing mark use the connection mark derived from PCC to src-nat. The both-addresses or even just the src-address qualifiers would work well.
Your upstream should be willing to give you as many IPs as you can justify, in this case you should be able to get a full /24.
But anyway, there might be a way to get NAT to use more than one IP in a round robin fashion, otherwise, depending on how your customers are connected, you could do something like create 30 vlans or 30 bridges and group the customers into those with each vlan or bridge having its own public IP and NAT rule. Just an idea, and there may or may not be better ways to do it (besides giving the customers their own IPs). Its hard to say without knowing more about how everything is set up; but maybe this will give you a creative nudge to help you come up with something (and there would still be the potential for problems with those download sites).
The basic idea would be something like this, shown for just three IPs, they are all balanced between and one of them is used for fallback:
# assign the addresses used for NAT to the WAN interface so that the router responds to ARP requests from the upstream
/ip address
add address=1.1.1.2/25 interface=WAN
add address=1.1.1.3/25 interface=WAN
add address=1.1.1.4/25 interface=WAN
# now mark connections with PCC to decide what to NAT to. Src-nat is very late in the packet flow, so below this is done in the postrouting mangle chain after it's been decided that the packet will go out to the WAN. Only packets going through the router are balanced for NAT, router generated packets are always NAT'd to the same IP. Adjust passthrough if you have later rules.
/ip firewall mangle
add chain=postrouting out-interface=WAN src-address-type=!local per-connection-classifier=both-addresses:3/0 action=mark-connection new-connection-mark=NAT_to_dot2 passthrough=no
add chain=postrouting out-interface=WAN src-address-type=!local per-connection-classifier=both-addresses:3/1 action=mark-connection new-connection-mark=NAT_to_dot3 passthrough=no
add chain=postrouting out-interface=WAN src-address-type=!local per-connection-classifier=both-addresses:3/2 action=mark-connection new-connection-mark=NAT_to_dot4 passthrough=no
# now NAT, fallback to .2, which is also used for router generated packets
/ip firewall nat
add chain=srcnat connection-mark=NAT_to_dot2 out-interface=WAN action=src-nat to-address=1.1.1.2
add chain=srcnat connection-mark=NAT_to_dot3 out-interface=WAN action=src-nat to-address=1.1.1.3
add chain=srcnat connection-mark=NAT_to_dot4 out-interface=WAN action=src-nat to-address=1.1.1.4
add chain=srcnat out-interface=WAN action=src-nat to-address=1.1.1.2
That should work and easily extend. If you need to have a client always NAT to the same IP just use the src-address classifier.
Thanks fot the example, it’s very helpful. I currently use pcc and can change it easily but after the pcc I have packet marks for the queue tree, this is going to make them stop working?
Depends on your current setup. The above doesn’t set packet marks so it wouldn’t affect those. It sets passthrough to no, but only in the postrouting chain, so unless you’re already using the postrouting chain for something else it won’t interfere. However, if you are already using PCC to make routing decisions it might interfere because you’re constantly changing the mark on the connection, which could make them unstable. I don’t think it would be advisable to mix the two. That would apply to any balancing scheme, the problem isn’t using PCC twice, the problem is marking connections twice.
You can use PCC straight in the NAT rules, though. Don’t use mangle at all to mark for NAT, just src-nat directly from the PCC decision:
# assign the addresses used for NAT to the WAN interface so that the router responds to ARP requests from the upstream
/ip address
add address=1.1.1.2/25 interface=WAN
add address=1.1.1.3/25 interface=WAN
add address=1.1.1.4/25 interface=WAN
# now NAT, fallback to .2, which is also used for router generated packets
/ip firewall nat
add chain=srcnat out-interface=WAN src-address-type=!local per-connection-classifier=both-addresses:3/0 action=src-nat to-address=1.1.1.2
add chain=srcnat out-interface=WAN src-address-type=!local per-connection-classifier=both-addresses:3/1 action=src-nat to-address=1.1.1.3
add chain=srcnat out-interface=WAN src-address-type=!local per-connection-classifier=both-addresses:3/2 action=src-nat to-address=1.1.1.4
add chain=srcnat out-interface=WAN action=src-nat to-address=1.1.1.2
I’ve never tried that but don’t see why it wouldn’t work.
I had a route that included all de adresses i had decalred from the public ip pool I have in ip/adress and all the traffic left throuh it using just the preferred adress (189.xx.xx.35) if I disabled that route the traffic stopped.
Also I dont know why but in firewall/nat the connection mark fibra4 y fibra 5 didn’t passed dat even though that the mangle rules that marcked that conections reported the expected data.
So the first trial was a desaster. Back to the sketch board.