Configuration for Proxy bandwith control

RouterOS General
1

I would like to ask you for some advice. I would like to make things simpler in my network.

Currently configuration:
I have 3 servers + ~50 workstations :

  1. Linux gateway, LAN eth1 - 192.168.1.1
    Linux is a gateway (Firewall, DNS, Radius, Log server), performing NAT for entire 192.168.1.0/24 network.

  2. MT (ver3.10 x86), two Ethernet cards bridged, bridge1 - 192.168.1.3
    MT is bandwidth controller for traffic from/to gateway.
    It has 2 Ethernet cards bridged, each cards has a queue tree assigned for download/upload shaping.

  3. Proxy, LAN eth1 - 192.168.1.2
    Squid with delay pools configured.

  4. Workstations, 192.168.1.5-192.168.1.50, IE/Firefox configured to use Proxy server (192.168.1.2).

The goal:
I would like to move bandwidth control from proxy server (Squid with delay pools) into MT, which already shapes traffic from/to main Linux router. I need more flexible way to assign bandwidth for every workstation. Delay pools are OK, but NOT when you need to assign 10 different speeds.

Proposals:
Proposal #1 is about moving Proxy behind MT.
Proposal #2 is about adding another Ethernet card to MT.
MT_schem1a.PNG
Currently I have queue tree assigned to each card, with allocated bandwidth for:

  • high priority traffic (D_HIGH, U_HIGH)
  • low priority traffic (D_LOW, U_LOW)
  • bandwidth per each workstation. Bandwidth assigned for each workstation divides into http traffic + other (with different priority).
    MT_schem2a.PNG
    What would be the best way to achieve my goal and integrate Proxy traffic control into my scheme?
Mikrotik + Squid
2

Why not use MT as your proxy and gateway? If you have got a PC with MT, would be no problem.

3

The reason is simple - one step at a time.

Squid is working great as Proxy. Hardware is configured for proxy (multiple SCSI U320 10k drives).
Linux gateway runs additional services, like apache, radius, firewall, port/address translations.
I am not sure Mikrotik is capable of taking all those issues, or rather I an not capable of doing it with MT :laughing:

Anyway, I need to weigh the pros and cons for two solutions I figured out and presented above.

4

Maybe I ask more specific question…

Let’s assume I pick up proposal #1, so I do not add another Ethernet card to MT and just move switch (with Proxy server) behind the WAN interface (from MT bandwidth manager perspective). So now all traffic from proxy goes through the MT bandwidth manager.

It creates following questions:

  1. How to emulate Squid’s delay pools at MT? Should I use PCQ, without defining maximum speed, but only limit per IP?
  2. Can I attach to physical Ethernet interface two different queues at the same?. I mean current I have queue tree, which I presented at first post and additionally another queue for Proxy traffic (PCQ) ?
5

Queues are processed in order.
You can assign a queue that clasifies traffic by dst-address, and put it before the others, so it catches the traffic to your proxy.

6

Jorj - Thanks for replying !

Since I do not want to limit total bandwidth for proxy (I can not even guess it due too cache hits) and I want limit it per-user basis - picking PCQ is a good idea, right? How can I assign different limits per-user with PCQ, so I could emulate different delay pools from Squid ?

7

Yes, you can use pcq in connection with mangle, in order for the queue to apply only for the proxy traffic.

8

I had no chance to test it, just wrote as I see it in theory.
No limit for total proxy bandwidth, only three different limit per-user basis (512k,1M,2M) simulating delay pools from Squid.

  1. Create address lists, for every delay pool from squid respectively.

Users with download limit 512k → address-list=Proxy_512k_IP
Users with download limit 1024k → address-list=Proxy_1024k_IP
Users with download limit 2048k → address-list=Proxy_2048k_IP

  1. Mark packets at MANGLE prerouting, using defined address lists (192.168.1.2 is IP of Proxy server).

/ip firewall mangle add chain=prerouting protocol=tcp src-addres=192.168.1.2
action=mark-packet new-packet-mark=Proxy_512k passthrough=no address-list=Proxy_512k_IP

/ip firewall mangle add chain=prerouting protocol=tcp src-addres=192.168.1.2
action=mark-packet new-packet-mark=Proxy_1024k passthrough=no address-list=Proxy_1024k_IP

/ip firewall mangle add chain=prerouting protocol=tcp src-addres=192.168.1.2
action=mark-packet new-packet-mark=Proxy_2048k passthrough=no address-list=Proxy_2048k_IP

  1. Create queues for every address list.

/queue type add name=“PCQ_download_512k” kind=pcq pcq-rate=512000 pcq-classifier=dst-address
/queue type add name=“PCQ_download_1024k” kind=pcq pcq-rate=1024000 pcq-classifier=dst-address
/queue type add name=“PCQ_download_2048k” kind=pcq pcq-rate=2048000 pcq-classifier=dst-address

  1. Attach queues to physical interface (LAN for download)

/queue tree add parent=ether2-LAN queue=PCQ_download_512k packet-mark=Proxy_512k
/queue tree add parent=ether2-LAN queue=PCQ_download_1024k packet-mark=Proxy_1024k
/queue tree add parent=ether2-LAN queue=PCQ_download_2048k packet-mark=Proxy_2048k

Is this the right approach ?

9

Glucobay… what is your cache hit ratio on squid?

10

Cache information for squid:
Hits as % of all requests: 5min: 45.4%, 60min: 37.6%
Hits as % of bytes sent: 5min: 8.3%, 60min: 5.1%
Memory hits as % of hit requests: 5min: 0.0%, 60min: 0.5%
Disk hits as % of hit requests: 5min: 27.6%, 60min: 32.0%

11

oooops ! poor cache hits dude !!!

12

I do not know if my hit ratio it poor or not… There are many domains in squid config under cache deny directive.

Let’s go back to the main subject - bandwidth control for Proxy server, regardless of their hit ratio :sunglasses:

13

You could specify simple queue for each user ip on MT board. With simple queue you could limit bandwidth in many possible ways with each user any configuration you want to. Very simple.

14

Thanks for the hint Egate - I know I could use simple queues.

However it would require one additional queue entry per each IP, plus one additional entry at MANGLE per IP.
With my PCQ proposal, total number of new entries is much lower.
Maybe I should now worry about MT resource for now, but my idea looks to me more elegant…

15

I need help too how to configure in my case how to redirect traffic to my squid box and not to have any limit for that traffic for content on cache bellow is my configuration.

INTERNET ----------------MIKROTIK-----------USERS
|
|
|
SQUID

WAN IP: 192.168.0.1/24
LAN IP: 10.0.0.1/24
SQUID: 192.168.1.2/24


on squid IP : 192.168.1.1/24

I can not ping from my clients 192.168.1.1, I can ping 192.168.1.2
Network.jpg

16

I want my users use squid as transparent proxy.

How to configure for cache content users to download with e.g. 512 kbps. But for traffic on the internet with their speed.

What to do?

17

I would go for the #1. I spent quite a time trying to make proxy work and get to the conclusion that only approach as your #1 proposal is simple and easy to maintain. You will have MT doing QoS and proxy doing proxy, and MT would not even have to know about existence of proxy sever.

I would do just one change. remove switch and connect proxy to the Linux router on separate interface so Linux router may decide if connection goes through proxy or not.

#2 approach would just make things far to complicated and very hard to control. Mikrotik does not have proper solution for having Proxy and QoS on the same MT.