Hi,
I have problems with RB1000 and webproxy, I see that the I/O in the compact flash
produces 100% CPU. Anyone knows how to minimize this?
I have 150 users max, and the CPU goes 100% in about 30 minutes with 20 users.
Many thanks.
What occurs with slow CF cards, and small processors . IN some cases WebProxy will work fine on these, it just depends on the size of the CF etc. With the CF, as soon as you boot, it does a read on all of the files to rebuild the cache etc. SOOO. Thats why it takes a long time on reboot.
You might think about a LINUX-box with squid and HDD as an upstream (parent) cache for your RB, and use RAM-based caching on your RB only. Howver, still a pending bug with this one.
Refere to
http://forum.mikrotik.com/t/anybody-confirms-negates-bug-web-proxy-squid-as-parent/32402/1
Many thanks for the reply.
Actually, I have a LINUX-box with squid and HDD as an upstream (parent) cache for my RB, and use RAM-based caching on my RB only (sorry for copy and paste, but your english is better).
However when there are online a lot of clients (14-15Mbit of traffic), the CPU of RB1000 is over 80-90%. Is it normal? I think the it could be a bottleneck.
Many thanks.
P.S.: I don’t have problem with the website of the topic
Sorry, on my hotspot I do not have such a high load.
Regarding my reported problem: It is interesting, that you have no problems. I re-tested, and found same problem under various conditions always to appear, in case either hotspot is activated, or web-proxy. So it might be something regarding the “universal proxy” scheme, MT uses.
To be shure, that you really NOT have this problem, too, can you please do following test:
From hotspot-client,
(having web-proxy/upstream/transparent proxy enabled on RB), go to
http://messages.finance.yahoo.com/Stocks_(A_to_Z)/Stocks_N/threadview?m=tm&bn=12776&tid=230059&mid=230059&tof=1&frt=2
Click “Reply”
type a message, then click “Post Message”
On my system, error from squid shows up.
In case, you still do not encounter my problem, please post your version of squid, you are using. And version of RoS. That might be an issue,.
Thanx in advance.
Hello, I tried to post and my squid is working without any problem.
However, I don’t use proxy in hotspot, I use squid with my residential clients, and the CPU goes up when there are 15-16 Mbit of traffic on my network. I never had any type of problem with squid, however sometimes (with certain of website, like mail website) I exclude them to be proxed (I have an address list on mikrotik).
I think that the high CPU load is concerned to the number of the on-line clients (so the number of connections) and not to the traffic load.
My version of ros is 3.30 on the rb1000, and my parent proxy is Squid 2.7.STABLE6.
Some example:
4.8Mbps of traffic | CPU 37%
9.2Mbps of traffic | CPU 63%
17.2Mbps of traffic | CPU 90-100%
I have mangle rules, however if i disable the rules, the situation is the same.
Instead if I disable webproxy and activate the mangle rules the CPU usage is about 3-5%.
Thanks for the reply.
@welan: (I have an address list on mikrotik).
Can you give a link, please ?
I will test your prblem sites, too.
In worst case, I have an exception list to start with 
Will also try to use an older squid for my problem; RoS version should not be an issue, I guess.
Thanx for your work, anyway.
note that hotspot uses web-proxy, and uses settings set up in the proxy settings, except, it is enabled if you are running hotspot. You should configure proxy to reflect that no caching is required, or set up parent proxy feature.
Yes, I realized that.
You should configure proxy to reflect that no caching is required, or set up parent proxy feature<
No caching ??? Why ?
Actually I am using both: hotspot, web-proxy with RAM-cache only, parent proxy running squid. And usually it works. But again, one special problem site I have. So there is a bug, which I reported already some time ago. Still pending.
Ok, I think that it’s not clear my situation.
I’m running the RB1000 as a router in my network:
Clients 150+ —> |RB1000 - ROS3.30| —> INTERNET
*********************| /
*********************| /
*********************|SQUID2.6STABLE7| ------ /
The * is for align the picture, sorry.
In the RB1000 is running a web proxy server, with this settings:
enabled: yes
src-address: 0.0.0.0
port: 3128
parent-proxy: 172.16.28.100
parent-proxy-port: 800
cache-administrator: “pippo@pippo.com”
max-cache-size: none
cache-on-disk: no
max-client-connections: 4000
max-server-connections: 4000
max-fresh-time: 3d
serialize-connections: no
always-from-cache: no
cache-hit-dscp: 4
cache-drive: CF1
My parent proxy is running squid. The clients are surfing very well, but I think that the CPU usage on RB1000 is too high when traffic is heavy.
I attached a screenshot.
Many thanks.
You are still proxying traffic. Hence tracking it, etc. So, hence, you have the high cpu count. Think of this. What happens is the MT gets the traffic, and then sends it out, as a client to your parent proxy, and then returns that information to the client. So, yes, this will eat up CPU as it should.
What you SHOULD be doing, is the transparent proxy rule, but direct it stright to your parent proxy server. Then there is NO proxy that should be even running on your 1000. So, simple configuration issue!
Ok, I tried to do this, with a simple dst-nat like this:
chain=dstnat action=dst-nat to-addresses=172.16.28.49 to-ports=80
protocol=tcp src-address=172.16.28.0/24 in-interface=ether1 dst-port=800
but I cannot surf. What is wrong?
Thanks
PS: I bought your book, very interesting
I would have to look at it, but I am sure its a NAT rule. You would have to also accept traffic from your web proxy.
Tried, but not surfing.
chain=dstnat action=dst-nat to-addresses=172.16.28.49 to-ports=80 protocol=tcp src-address=172.16.28.0/24 in-interface=ether1 dst-port=800
You’re dst-nat’ing back into your own network. That’s problematic since the client is sending to the router, and is expecting an answer back from the router. But the router is forwarding the request to a host on the same subnet, which sends back directly to the client, which then discards the traffic as “what the hell is this”.
Try a parent proxy outside of the client network and a similar rule should work.
Edit: alternatively you could also src-nat the forwarded request to the router IP. At that point traffic would flow as expected (client → router, router → proxy, proxy → router, router → client) - but the proxy would only see the router IP address instead of the individual clients. All in all it’s cleaner to move the proxy to a different network.
Ok, thanks, but if I move my proxy over another subnet, this is the response of mikrotik when I try to surf:
ERROR: Gateway Timeout
While trying to retrieve the URL http://www.google.it/:
Connection refused
Your cache administrator is info@pippo.com.
Generated Tue, 01 Dec 2009 19:15:03 GMT by 172.16.28.251 (Mikrotik HttpProxy)
I have no idea.
If you’re proxying directly to the external proxy now moved to a different network, why is the Mikrotik proxy still active and intercepting the connection?
The message above is squid as parent proxy over another subnet, but I tried your solution too:
Assume that:
172.16.28.99 is my PC
172.16.27.251 is squid
chain=dstnat action=dst-nat to-addresses=172.16.27.251 to-ports=800
protocol=tcp src-address=172.16.28.99 in-interface=ether1 dst-port=80
172.16.27.251 is on ether4, this is the filter rule of firewall:
chain=input action=accept in-interface=ether4
What is wrong?
HI
I HAVE EXACTLLY the same problem with rb750g when downloading around 15mbs the cpu uses 100%. i use the proxy to redirect to parent squid but still is the same.
i tried to redirect in NAT Firewall but it doesent work, proxy err page in browser
Did u get this resolve?
I’m using 450G in our internet cafe, I’ve enable Hotspot and Webproxy.
first time I’ve enabled cache on disk, and always my CPU was %90-%100. Then I disable that and delete caches file from “Files”
now it’s CPU usage is better.
Seems MT webproxy cant handle too much traffic.