I have problems with a RB1000 and transparent webproxy cache in Compact Flash Extreme3. The LAN goes working slow when cache is filling up (4GB of 8GB total) and the CPU of the RB1000 goes to 100% too.
Uldis said to me that on every write on the CFlash, the CPU increases and it’s normal that the CPU stays always at 100% when using webproxy cache. He said that the system could work fine with the CPU at this very high % of use too.
My problem is that when I activate the webproxy and the CPU goeas to 100%, all the web browsing goes too slow. I use DNS cache too, but it doesn’t speed up my network.
Anyone of you has installed the RB1000 with webproxy and has a speed web browsing without speed problems? What can I do to make it faster? I need to use more than the 1Gb of RAM installed to make cache.
I usually have “only” 60 users at the same time, browsing internet.
Some websites do not work with the MikroTik Proxy in the first place. I think it is a cookies problem, someone reported google mail not working OK and I know some of our Bulgarian clients cant login into their dating sites for example http://aha.bg - through the MT Proxy.
Please keep in mind that Web Proxy needs time to rebuild the cache sometimes after it being started… you can see what the Proxy is up to in its settings window in WinBox on the bottom…
When dealing with Proxy storage - first Stop the Proxy - then make changes to the storage …
P.S. another thing that annoys me is that the MT Proxy does not use RAM to further speed up its process.
So soon I am starting to deploy fast ClarkConnect boxes with 1gig ethernet cards that have Squid running.
I have tried to upgrade to ROS 3.28 and the problem still exist , processor is on 100% almost all the time.
It works great about 15 minutes from turning on , and after that it goes to 100% CPU utilization.
I didn’t have this kind of problems with 2.9.xx version (squid version) on x86, so I thought that it shouldn’t have a problem running on RB1000 with CF on ROS 3.xx.
As NetworkPro suggested that cookie’s could make a problem I explicitly make not to cache cookies and still the problem persist.
I think that this is serious problem, I have not read comment from anyone who don’t have this kind or similar problem with new implementation of MT Proxy.
If the problem is with CF card I suppose that It would not work at all (anyway Dude is on the same CF and it works flawless).
Try turning off the option Cache on Disk - so the proxy will be tested in RAM. Just don’t power cycle it or anything - to keep the contents of the cache.
This should stop cookies from storing into web proxy.
I’ve tried turning off save on disk and the CPU utilization dropped to 70-80% but it is still unusable. Router response time is desperate.
Any other recommend ?
NetworkPro, I suppose that RB1000 should be more powerful than a standard PC (it’s PowerPC), that’s why I am using it.
I have about 80-90 online users regularly per RB1000, but the tests were done with about 20-30 online users.
Nevertheless I suppose it should fine with up to 200-300 users, about 1500-2000 connections.
BTW, I am now trying to simulate nearly same situation on my RB433 ,with system as a caching disk and results are disaster. It goes up to 40% on 2 clients.
For very active proxy caches, where many files are stored and retrieved at once, the RB1000 is not so optimized, as it doesn’t have native IDE controller on it (speed is limited). However, the CF slots work perfectly fine on regular load cache, for user manager, for logs, for Dude etc.
So Normis, what would be very active proxy , 1000-2000 connections ?
As I recall previous implementation (2.9.xx) had combination of ram proxy and disk proxy , so this kind of implementation was not very (disk speed) demanding and could satisfy our needs. Anyway I think that high-speed CF could get up to 18MB/s roughly about 140Mbit/s of communication, so this could not be a bottleneck of the system (if there is no some glitch in the file system).
Now, the question is will Mikrotik make some improvement in proxy, or you think that this is what you wanted to achieve with this part, maybe there is some trick to make it work with some configuration magic ?
Further to what noam_chom has said, might I make a suggestion?
It seems to me that the RB1000, and even the RB450G, are more than capable of keeping up with a caching proxy in CPU performance terms.
If the issue is that the CPU is being eaten alive by PIO calls, then how difficult would it be to issue a version of either system which has a SATA port built-in?
There are already SATA Mini-PCI cards available, but these don’t work because they require a 5V source. Again, how hard would it be to provide this 5V line to the card?
Then you could produce an oversized case that could take a 3.5" SATA drive (3.5" because they are more robust than the 2.5s). The 1U version of the RB1000 could be easily modified for this purpose.
This seems to be another one of the occasions when an otherwise excellent Mikrotik solution is spoiled by little silly things.
If you are going to include a cache, you should enable a more robust way of using it on your boards.
If you are going to include a CF shot, it should be IDE enabled – especially when you advertise TrueIDE on your site.
If you are going to include an OpenVPN implementation, give us enough options such that we can do things like push routes down.
A connected route that stays up when the link is down is bad practice, pure and simple.
All frustrating bugs that really shouldn’t be there! And I’ve only been using the product for a short time. I asked about the CF/SD performance issue a few weeks back, and Mikrotik assured that there wasn’t a problem…
I have done extensive testing of the WebProxy package and so far I found that the it works “as it should” (not too bad) just on x86. So my guess is that powerpc WebProxy package is not optimized for routerboard, so i tried to make a parent proxy on RB1000 and proxy on HP Proliant ML115. But when the number of users that use Proxy become over 20 users RB1000 started to jump to 90-100% cpu utilization, when redirecting http requests to x86 Proxy.
Conclusion: Something is really wrong with proxy package.
Please someone from Mikrotik confirm that this is noted, and either you are working on the solution (so we need to wait for solution) or should we work to find alternative non-mikrotik webproxy solution.
Just to lend a voice!
I am also having performance issues with mikrotik webproxy on compact flash+RB1000. I have had to dump it..
So for more than ‘regular load’, if we must use routeros it has to be x86?
I agree. The RB1000 is an otherwise excellent product - price, power, performance, features. If you can up it’s specs to handle heavy webproxy demands, you win..we all win!
as normis wrote before - no IDE = a lot of PIO to handle by CPU, all x86 boxes i ahve seen comes with some sorts of IDE controller, hence, decent operation, since all that PIO from disk writes/reads are handled by adapter not the CPU.
as a solution in larger network you can add 1 x86 box for proxy, and point routerboards to it as parent proxy.
if you have some free resources (RAM) on your RB1000 give some RAM for proxying and set up parent proxy. That way all the requests will be kept inside your network and proxy will work better as more requests and more proxed pages inside your network.
I am pretty concerned that when I used RB1000 just for parent proxy-ing (port 80 - to x86 box) made RB1000 processor too utilized (90-100%).
I have turned off cache on disk (roadrunner), and that didn’t helped.
So any solution including RouterBoard and Proxy right now is not possible for a larger network (except using x86 as a Router and other x86 as a Proxy).
As far as I know parent proxy-ing should not utilize RB1000 processor so much.
roadrunner, yes I’ve tried every CF 16gb that I could find on our market, and I can say MT has a great compatibility list concerning CF cards.
I can’t say that any of the CF I’ve tested worked better than other, they worked equally good.
Janis as you are saying there are too much requests (YES) , but i suppouse speed 18mb/s ~ 144 Mbit/s access to CF should be enough so the cache is usable.I think that if you have a cache filesystem table in the RB memory (with file physical address and file crc32) and access to CF read only on hit would make processor less utilized. So nevertheless multithreaded solution could be also helpful if the problem is access to CF because RB doesn’t need any CF access to manage the rest of the tasks it should accomplish.
It is not for me to judge but don’t you think that this is a problem worth of solving ?
Well well well what do you know MirkoTik proxy works perfectly with Facebook wheres Squid 2.6-3.1 fails miserably. Too bad MikroTik proxy can’t be anonymous. I mean stealth. Now it’s anonymous.
