The way 'm trying to intgr mktk & proxy. If
any request is made by client → mktk (it will first check if the
content is already in squid cache - if yes it will fetch the requested
content from cache; if not in the cache - the request will go direct
to remote server/site) → remote server/site → reply packet/content
to mktk → mktk will receive any packet/request from remote servers
send to → squid box → after receiving the packet/content squid will
throw it to ->client
When the topic comes to (web)proxy with squid cache integration - not only mikrotik@support but also our honored senior members and even moderators - seems to like keep silent.
I know there are loads of stuff about proxy & web proxy in this forum and in net - if one Google it. But there is not a single one with detail illustration.
What is this troll stuff? You want a plain ordinary Proxy service? Then just do it. Your post shows the default way of doing it? What do you want?
Nobody is “keeping silent” mate, people have stuff to do! Time is money! Important projects and clients are waiting for our expertise and here you are trolling in the forums with the default config ?!
I think the image 've posted has misguided you. Please read this - I think this will make you clear. I have 1500 pppoe user and around 600 pppoe users are always online. I can active transparent proxy for them. Then what happens is -
"My Mikrotik server will receive the entire request from my user and forward it to proxy, proxy will check its cache whether it’s in cache or not. If it isn’t cache then proxy will download it for the user. "
But I don’t want that. I want Mikrotik will receive the entire request from the user. It will check if it is in the cache or not & if it is - it wiil fetch the content from cache. If not mikrotik will fetch the content from remote servers not the squid/proxy.
If I try mikrotik web proxy - either the processor usage goes up around 70-80% and even at 100% and also I can’t see how much of the secondary HDD is being used for cache.
And if I use a squid/proxy it consumes most of the free bandwidth for even a single. But almost 2400 simultaneous request is made by my clients.
Even if I declare different delay poll - suppose 10 different delay pool, any 10 request among these 2400 requests - will be sufficient to consume all these pool bandwidth.
One more thing I want to do with mikrotik that is mikrotik will store what ever my users to squid/proxy and users gets their download from squid/proxy cache just like bluecoat.
The problems is the bandwidth management? You want to divide bandwidth equally per user, so that no one user can cause delays for the other users?
What does bluecoat do that you want MikroTik to do?
Do not run the proxy on MikroTik. If you already deployed Squid - use that.
Explain more about what end goal with the bandwidth you want to achieve
"My Mikrotik server will receive the entire request from my user and forward it to proxy, proxy will check its cache whether it’s in cache or not. If it isn’t cache then proxy will download it for the user. "
But I don’t want that. I want Mikrotik will receive the entire request from the user. It will check if it is in the cache or not & if it is - it wiil fetch the content from cache. If not mikrotik will fetch the content from remote servers not the squid/proxy.
MikroTik RouterOS has a built-in simple proxy that works well only in RAM for RouterBOARDS (slow I/O) or it can be used on fast x86 machines where it does a pretty decent job! But it causes login problems for some websites and exposes the internal IP addresses of customers to websites which is pretty bad IMO.
If its something crazy you need - RouterOS does not have such complexity in it. The design idea of RouterOS has been such that it should be a really light and simple and hopefully fast OS for a Router, that gives you the config freedom that you need for networking purposes.
For the crazy scenarios you have Open Source like Squid etc to make you insane and take away your sleep (just kidding I love Open Source, I see how good it is)
Maybe you can make the Squid box do the entire work - Linux can be configured as a router etc BUT its hard as hell OR if you are a fast learner - you could do it but for production environments where customers are waiting - I would suggest you hire an experienced guru that can make the Squid box do whatever you need it to.
Does that really make such a difference, in case the upstream squid is also located locally ?
IF YES then I would throw some HW at it, to use a standard PC instead of RouterBoard; and some memory:
To use the RoS caching proxy, but RAM only. And then a local, upstream (parent) squid-PC.
Or, how about a separate squid-PC just in front of the RoS-box ?
It looks like, the atual, custom implementation of the RoS-cache does not support ICP (Internet Caching Protocol) and config-options for that, like squid does.
Otherwise, your configuration should be possible.
I haven’t fully investigated the login problems yet, someone on the forums said he excluded cookies from caching for this kind of thing…
The exposure of internal IP address of the customers to the Public side is useful if it could be used to manage bandwidth per user with the Mangle content matcher or a Layer 7 rule.
I hope someone have looked into this and can share his opinions!
edit:
Yes, surely the string to put in the mangle ‘content’ matcher should be “X-Forwarded-For: 192.168.1.155” where 192.168.1.155 is the example client we want to match. Excellent excellent! I hope I can make it work for my bandwidth management…
