Cache Server - Plan and Design

Hello There,

Anyone interested in a Cache Server? I believe its an important aspect for any network.

Question 1:
The wiki posted all reference a box to have 2 NIC cards so is it possible with a box of 1 NIC?

Scenario -
A Routerboard with 3 LANs
LAN A = ( out ) internet
LAN B = ( server ) cache server
LAN C = ( in ) users

Here a user connected to C will make a request say a video or image
Before getting to A Lan B will look at the cache server to see if the object is available
if yes, the object is delivered but if not the request will be elevated to A and out to internet.

can we run a cache server with one nic?

Thanks,

Yep, I use squid3. make sure that the cache server is on a different subnet, so you dont get issues like
“hairpin NAT”. make sure that squid is in transparent mode. and than you just need to add:

/ip firewall nat
add chain=dstnat action=dst-nat to-addresses=172.19.65.250 to-ports=3128 protocol=tcp dst-address=!172.16.0.0/12 in-interface=ether3 dst-port=80

So I direct port 80, everything that isnt to a local address from ether3 to 172.19.65.250:3128

The above rule is the simplest solution to redirect all HTTP request to External Proxy Server like Squid & it works fine.

But In my Opinion, Marking HTTP request and Route them to SQUID is much better. I always use packet marking to route HTTP request to squid, this way you will be able to log users ip’s for record purposes , usage reports , etc , & also it will prevent double NAT. Although double natting doesn’t prevent anything from being working, but routing is better then using double NAT :laughing:

If you want some ideas on this topology, you can read the Following . . .
http://aacable.wordpress.com/2011/07/21/mikrotik-howto-redirect-http-traffic-to-squid-with-original-source-client-ip/

Fighting to run it on win2k :smiley:

run what on w2k?

squid3

kind of off topic, but why would you be fighting to run squid on windows?

I always wanted Linux Ubuntu, but their last version is much graphic sometimes I don’t even know what I’m doing.

WIN2K is way out dated, but its still beloved to me. I’m gonna do my best to have worked else I will go with previous versions of Ubuntu.

:slight_smile:

You know getting a newer system makes drivers impossible to find for older OS. so yes, a full day just to make win2k fully worked.

Well, each to their own, but bad move in my opinion. I would use either Ubuntu 11.10 Server Edition or Debian Server, if i were you.

Windows 2000 is over 10 years old now and support from Microsoft ended on July 13, 2010 (http://support.microsoft.com/ph/1131). I suggest you move to something like Windows Server 2008 or as you said Ubuntu. Don’t forget Debian is also an option. Newer operating systems have a lot better hardware support for older computer. Squid was designed for Unix based systems, so it’s a bet easier to get up an running on Ubuntu and Debian.

With Debian, you can install it without the GUI and just do everything from the command line.

Lets start with Squid config file

In my network there is a master router - UM.
Lan 1 = Out - Internet
Lan 2 = (switch) connected to multiple routers across town and each router runs Hotspot.
Lan 3 = Server ( http and Squid)

Lan 2 = each user will get this range if IP (10.20.0.0/16)
Lan 3 = Server IP (10.10.10.1)

what line of squid.conf need to be changed?

thanks,

Come on guys, just little help making my transparent proxy work yo.

i just need the lines i must change in the squid config file according to the above setup so it can start the cache?

Well, I got it working… BUT little confused!!

/ip firewall nat
add chain=dstnat action=dst-nat to-addresses=172.19.65.250 to-ports=3128 protocol=tcp dst-address=!172.16.0.0/12 in-interface=ether3 dst-port=80

ip 172.19.65.250 is the proxy pc running transparent on 3128

my question here is on
1- ether3 ?
2- !172.16.0.0/12 ?

can you please clear these points.

example of my network:
ether1 = connected to internet (public)
ether2 = provides addresses to local users
ether3 = the proxy server

and !172.16.0.0/12 can you please explain if these are users ip or what?

i’m little lost here..

How can we know what 172.16.0.0/12 is on your network? It’s private IP space. Nothing in this thread mentions it before. The rule means, literally: take all traffic to tcp/80 that comes in via ether3 and isn’t going to 172.16.0.0/12, and send it to 172.19.65.250 on port tcp/3128 instead.

What that means on your network is up to you to determine.

I know its up to me to determine, but what i want is the idea of handling in and out.

can you make this rule simple to me to understand:

/ip firewall nat
add chain=dstnat
action=dst-nat 
to-addresses=10.1.1.1 //this is the ip of the cache server
to-ports=3128 //port of the cache server
protocol=tcp
dst-address=!172.16.0.0/12 //my user ip's are in the range of 10.10.*.* so should this ip be (dst-address=!10.10.0.0/16) ?
in-interface=ether3 //is this ether the public or cache or the user's ether ?
dst-port=80

Thank you

I guess I made it working.. just need to get a good php script that can also cache youtube videos.

Now what firewall rule i can use to bypass user’s simple queue if the transmitted file already existed in the cache and was downloaded from the cache server to the user?

thanks,

You can cache youtube like content via using 3rd party PAID Addon for SQUID name VIDEO CACHE .
I will not recommend you to Cache Youtube , it’s not worth it and you can’t actually cache even 1% of YOUTUBE, imagine how much space you will be needing , grrrrrrrr :p,
Also there are very slight chances that user will HIT the same video because of CDN.

To bypass rate limit for cached content, Using Squid with ZPH, and Mikotik Ver above 3+ , Use the following.

Add these Lines in SQUID.CONF

#==============
#ZPH 
#=======================
tcp_outgoing_tos 0x30 lanuser
zph_mode tos
zph_local 0x30
zph_parent 0
zph_option 136
#[lanuser is ACL for local network, modify at as per your network]

Make sure squid is marking cache content, you can do it via TCPDUMP command. i.e

# tcpdump -vni eth0 | grep ‘tos 0×30′
(eht0 = LAN connected interface)

At Mikrotik, create one mangle rule, which will marks cache contents coming from SQUID,
Second , QUEUE tree, which will enable cache content to deliver cache content to user at LAN speed.

/ip firewall mangle add action=mark-packet chain=prerouting disabled=no dscp=12 new-packet-mark=proxy-hit passthrough=no 

/queue tree add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=pmark packet-mark=proxy-hit parent=global-out priority=8 queue=default

Or Read more at:
http://aacable.wordpress.com/2011/07/21/mikrotik-with-squidzph-unlimited-speed-for-cache-content-traffic/

I must disagree on this. I have 25-30% of all trafiic coming from my proxies with VideoCache on squid ( 2TB disk space which is very cheap this days ). Peoples usually watch simillar videos. I put something on facebook, and all my friend want to see what it is and video gets viewed 20-30 times in few hours.

has anyone tried out this addon? if its working, can we get a free copy same or similar?
i followed the script posted on squid wiki (partial solution), but it didn’t work. although, the post did mention it may not work for everyone.