Community discussions

MikroTik App
 
Hellbound
Long time Member
Long time Member
Topic Author
Posts: 509
Joined: Tue Oct 26, 2004 11:21 am

cache all http downloads

Fri Jul 29, 2005 11:49 pm

hi ppl
how can I cache all EXE.ZIP.RAR etc etc file downloading so ppl dont have to redownload second time?

I have around 800gb hard disk (4x200gb) for doing so...

mikrotik can detect that? I haven't tried to put it together.
 
hzeid
Frequent Visitor
Frequent Visitor
Posts: 92
Joined: Tue Oct 12, 2004 11:57 am
Location: Lebanon

Sat Jul 30, 2005 12:53 pm

well i am also intersted in this topic. if it could be done, can i let the costumers use full speed for such downloads cause i limit the connection using pcq rules, if that works it would be great cause i will stop using microsoft isa for caching. hoping to see some answers
 
franco
Member Candidate
Member Candidate
Posts: 142
Joined: Fri Feb 25, 2005 6:26 pm
Location: Germany

Sat Jul 30, 2005 4:20 pm

That would be cool if it would function
 
User avatar
rickard
Member Candidate
Member Candidate
Posts: 145
Joined: Wed Jun 16, 2004 1:29 am
Location: Sweden
Contact:

Sun Jul 31, 2005 10:24 pm

Yes it works... we do that on our system but i dont think MT support 2 drives. i use one 120 Gb SATA disk , and that work.
from the Main router we use transparent proxy with parentproxy, to a second box with proxy enabled.
 
wildbill442
Forum Guru
Forum Guru
Posts: 1050
Joined: Wed Dec 08, 2004 7:29 am
Location: Sacramento, CA

Sun Jul 31, 2005 10:34 pm

Actually.. I think they do support up to two drives, I was just looking through the manual the other day and I remember seeing something about that.. however it may have been the 2.9 manual i was looking at..
 
Hellbound
Long time Member
Long time Member
Topic Author
Posts: 509
Joined: Tue Oct 26, 2004 11:21 am

Mon Aug 01, 2005 12:05 am

but how can I cache all EXE.ZIP.RAR etc etc file downloading?
 
hzeid
Frequent Visitor
Frequent Visitor
Posts: 92
Joined: Tue Oct 12, 2004 11:57 am
Location: Lebanon

Mon Aug 01, 2005 9:55 pm

Yes it works... we do that on our system but i dont think MT support 2 drives. i use one 120 Gb SATA disk , and that work.
from the Main router we use transparent proxy with parentproxy, to a second box with proxy enabled.

Pls Can u Tell how??????
 
hzeid
Frequent Visitor
Frequent Visitor
Posts: 92
Joined: Tue Oct 12, 2004 11:57 am
Location: Lebanon

Thu Aug 04, 2005 1:24 pm

no one interested in this topic or can provide detailed help????
 
goldclick
Frequent Visitor
Frequent Visitor
Posts: 51
Joined: Fri Sep 17, 2004 10:48 pm
Location: Nigeria
Contact:

Sun Aug 21, 2005 3:56 am

but how can I cache all EXE.ZIP.RAR etc etc file downloading?
All you need to cache EXE.ZIP.RAR should be all setup once you specify a cache drive, appropriate disk space to use and enable the web proxy. Objects/urls to be excluded from cache can be specified in the /ip web-proxy cache command. You need to decide and make a choice on maximum size of object (determines max size of EXE.ZIP.RAR) that the web-proxy keeps in cache. its a choice on speed vs bandwidth saving (more hits).
well i am also intersted in this topic. if it could be done, can i let the costumers use full speed for such downloads cause i limit the connection using pcq rules, if that works it would be great cause i will stop using microsoft isa for caching. hoping to see some answers
What I do is run a version of squid patched with TOS option marking for cached hits (see http://www.it-academy.bg/zph/) in another box. You can mark the cache hits with any TOS value of choice and then have a mangle rule in Mikrotik further mark packets with this TOS byte. The cache mangle rule stays above every other rule. You can then use Queue trees (2.8 and above) or simple Queues (2.9 and above) to decide how fast a user downloads the cached object. My config here has a Mikrotik as a bridge and bandwidth manager while the second box is Redhat 9 with transparent squid proxy patched with ZPH TOS marking. Been working great for me.

It will be nice to have mikrotik include this patch (as optional web-proxy-tos.NPK or so) and provide a configurable option to specify TOS of choice for cached hits. will be glad to throw away my second box and stick to Mikrotik's web-proxy.

Anybody doing this in another way?

Sonny
 
User avatar
rickard
Member Candidate
Member Candidate
Posts: 145
Joined: Wed Jun 16, 2004 1:29 am
Location: Sweden
Contact:

My settings...

Mon Aug 22, 2005 1:29 am

This is my settings. we are running Hotspot on 192.168.254.1 on port 80 and the proxy is on IP 192.168.254.4 and port 8080.
And this setting works for like Http downloads from Microsoft etc..

******On the Main GW MT Router*****

/ ip firewall dst-nat

add src-address=192.168.0.0/16 in-interface=Ether2 dst-address=!192.168.254.1/32:80 protocol=tcp \
connection=!hs-auth action=redirect to-dst-port=8080 comment="Transparent Proxy" disabled=no

/ ip web-proxy
set enabled=yes src-address=0.0.0.0 port=8080 hostname="proxy" transparent-proxy=yes \
parent-proxy=192.168.254.4:8080 cache-administrator="Email adress" max-object-size="0 kB" \
cache-drive=system max-cache-size=none
/ ip web-proxy access
add dst-port=!443 method=connect action=deny comment="allow CONNECT only to SSL ports 443 \
\[https\] and 563 \[snews\]" disabled=no
add src-address=192.168.0.0/16 action=allow comment="Our Customers" disabled=no
add dst-port=23-25 action=deny comment="Deny using us as telnet ans SNMP relay" disabled=no
add action=deny comment="Deny everything else" disabled=no
/ ip web-proxy cache
add url="cgi-bin \\?" action=deny comment="don't cache dynamic http pages" disabled=no



*****On the Proxy MT router****

/ ip web-proxy
set enabled=yes src-address=0.0.0.0 port=8080 hostname="192.168.254.4" transparent-proxy=no \
parent-proxy=0.0.0.0:0 cache-administrator="email adress" max-object-size="200000 kB" \
cache-drive=secondary-master max-cache-size=unlimited
/ ip web-proxy access
add dst-port=!443 method=connect action=deny comment="allow CONNECT only to SSL ports 443 \
\[https\] and 563 \[snews\]" disabled=no
add src-address=192.168.0.0/16 action=allow comment="" disabled=no
add action=deny comment="" disabled=no
/ ip web-proxy cache
add url="cgi-bin \\?" action=deny comment="" disabled=no
add url="\\.exe\$" action=allow comment="" disabled=no
add url="\\.zip\$" action=allow comment="" disabled=no
add url="\\.mpeg\$" action=allow comment="" disabled=no
add url="\\.mp3\$" action=allow comment="" disabled=no
add url="\\.avi\$" action=allow comment="" disabled=no
add url="\\.pdf\$" action=allow comment="" disabled=no
add url="\\.rar\$" action=allow comment="" disabled=no
add url="\\.mov\$" action=allow comment="" disabled=no
add url="\\.mpg\$" action=allow comment="" disabled=no
add url="\\.dat\$" action=allow comment="" disabled=no
add action=deny comment="" disabled=no
 
Hellbound
Long time Member
Long time Member
Topic Author
Posts: 509
Joined: Tue Oct 26, 2004 11:21 am

Mon Aug 22, 2005 5:38 am

I must have two mt box for doing this?
 
User avatar
rickard
Member Candidate
Member Candidate
Posts: 145
Joined: Wed Jun 16, 2004 1:29 am
Location: Sweden
Contact:

Mon Aug 22, 2005 10:09 am

Yes
 
Hellbound
Long time Member
Long time Member
Topic Author
Posts: 509
Joined: Tue Oct 26, 2004 11:21 am

Tue Aug 23, 2005 7:20 am

I tried that on one mt box and it was working,
thanks
 
User avatar
rickard
Member Candidate
Member Candidate
Posts: 145
Joined: Wed Jun 16, 2004 1:29 am
Location: Sweden
Contact:

Tue Aug 23, 2005 1:06 pm

Im sorry yes it will work on one MT box but i dont whant to have the main router and the proxy on the same box. We have 2 main routers with fail over and one proxy server.

//Rickard
 
hzeid
Frequent Visitor
Frequent Visitor
Posts: 92
Joined: Tue Oct 12, 2004 11:57 am
Location: Lebanon

Fri Sep 30, 2005 11:09 pm

hi Rickard

well pls can u post ur full configuration on how it worked with u and if u found it good. i am a newbie and liked the idea but i was un able to do it without the help u can provide
 
proxy
Frequent Visitor
Frequent Visitor
Posts: 82
Joined: Wed Dec 15, 2004 1:18 am

Fri Sep 30, 2005 11:11 pm

I tried that on one mt box and it was working,
thanks
which version are you using?
 
Hellbound
Long time Member
Long time Member
Topic Author
Posts: 509
Joined: Tue Oct 26, 2004 11:21 am

Sat Oct 01, 2005 1:29 am

2.8
 
hzeid
Frequent Visitor
Frequent Visitor
Posts: 92
Joined: Tue Oct 12, 2004 11:57 am
Location: Lebanon

Tue Oct 04, 2005 1:07 pm

I tried that on one mt box and it was working,
thanks

pls can u post ur setup?
 
Hellbound
Long time Member
Long time Member
Topic Author
Posts: 509
Joined: Tue Oct 26, 2004 11:21 am

Tue Oct 04, 2005 1:13 pm

setup is posted above, just remove transparent proxy IP and add the parent proxy setting to the same box, thats all

Who is online

Users browsing this forum: No registered users and 99 guests