Web proxy probs?

RouterOS General
1

Hi all
I’ve upgraded my MT-router to use 2.9RC5. Im my eyes it works fine but I have a few problems with the web proxy:

  • I know that http-traffic is going through the transparent proxy since all pages that users look at show up in the log.

  • I want to block access to a few pages via the proxy. That’s done with
    ip web-proxy access> print
    0 url=“ad.aftonbladet.se” action=deny
    1 url=“ad.adtoma” action=deny
    2 url=“snyggast.se” action=deny
    etc etc

This worked fine in 2.8, but not now. It’s free to surf anywhere :slight_smile:

  • I can not get my queue tree to work as it should. Mangling the http packages works fine, the data is increasing while users surf, but no way I can get the queues to react on that.
    ip firewall mangle> print
    0 chain=input in-interface=Internet protocol=tcp src-port=80
    action=mark-packet new-packet-mark=HTTP
    and
    queue tree> print
    0 name=“HTTP Surf” parent=global-in packet-mark=“HTTP”
    limit-at=2000000 queue=ethernet-default priority=4 max-limit=24000000
    burst-limit=0 burst-threshold=0 burst-time=0s

and some other tree entries…

I would be happy if anyone had any idéas about my problems.

2

Really no-one that uses “URL” to block sites???

If I put a rule in the webproxy like “deny all”, I get the Access denied page, but I can not use the URL field?

I just upgraded to RC6, but no difference regardning this issue.

3

also i m facing problems in 2.9 rc6 in web-proxy
when i try to access http://www.msn.com its shows me error 401 unathorized
all other sites working fine
number 2 slow profomace when i forward it to transparent proxy
number 3 error that ERROR The requested URL could not be retrieved
when i try to access such this url http://www.google.com/pagead/iclk?adurl=http://www.bambooweb.com/articles/l/a/LAN.html&sa=l&ai=B7zMZMZjCQom3IpO6RfaU-DSqtMoG0PWzlAGk37vFAdCGAxABGAE4AED6FEiQOVCMrd2yAaABpoTl_gPIAQE&num=1
i hope anyone using 2.9 rc6 tells us if he have same problems in rc5 it was worest

4

also i just noticed one more bug in web-proxy
it doesnt respond to my click on the link i need to click on it 3 or 4 times to responed (100% its not from the internet i have to pcs one throw the MT and one directly i can really see the problem i switched pcs cables its still the same)

5

:arrow_right: taloot
Hmm, your problems sounds quite weird I think. I tried the links in your post and they worked just fine here. I can also not see that links work “slower” when I forward the traffic through the transparent proxy. You havn’t managed to run your own proxy-traffic through the “tarpit” filter that’s new in v2.9? I happened to do so and things went reeealy slow. :unamused:

So again, my only problems with the proxy is that it doesn’t find the url.s I type in as filters, and the marked packages doesn’t show up in the queue tree…the rest seems to work just fine.

6

Maybe the problem is that the field now requires the complete string rather than just a substring, and it is not a wildcard rather than regexp. see the manual for more info on that.

7

hi rob,
mmm i m using hotspot and i forward the hotspot to my webproxy 10.5.50.1:3128 and the hotspot is transparent
when i disable the transpernt in the hotspot user profile and ues proxy 10.5.50.1:3128 in client pc it workkkkks great so i think its hotspot problem



btw whats is “tarpit” filter and how can i check it?

8

:arrow_right: taloot
Personally I don’t use the hotspot…and I really don’t know exactly what that is anyway :wink: I suppose it’s something wireless and my network is only wired…

The tarpit is something you can use to supress DoS (denial of service) attacs. It simply slows the connection down. It’s a filter you can use in ip/firewall. Take a look at MT’s demo2-router to see how they’ve done. I copied their example.

:arrow_right: lastguru

Maybe the problem is that the field now requires the complete string rather than just a substring, and it is not a wildcard rather than regexp. see the manual for more info on that.

Uhm? I’ve read the manual…the one I can find, which is
http://www.mikrotik.com/docs/ros/2.9/ip/webproxy
and I must say, I really don’t get it!

some examples:

if I use the URL:
ix.se
Now I can NOT access http://www.ix.se. Perfect!

lunarstorm
Now I can NOT access http://www.lunarstorm.se. Perfect!

snuttis
Now I still CAN access http://www.snuttis.se. Why??

ad.adtoma
…blocks ad.s from ad.adtoma.

ad.
…does NOT block ad.s from ad.whatever

HELP PLEASE :cry: Some examples or an external manual would be very great! (btw. the choises of url.s above is just something I picked by random to have something to fiddle with)

9

Theese are the complete settings for my web-proxy access. I guess there must be something wrong with my URL-syntax. See my previous posting above.

[admin@Hermes] ip web-proxy access> print
Flags: X - disabled, I - invalid 
 0   ;;; Deny using this proxy as telnet/smtp relay
     dst-port=23-25 action=deny 

 1   ;;; Deny access to some URL.s
     url="*ad\\.aftonbladet*" action=deny 

 2   url="*ad\\.adtoma*" action=deny 

 3   url="*snyggast\\.se*" action=deny 

 4   url="*ix\\.se*" action=deny 

 5   url="*lunarstorm*" action=deny 

 6   url="*snuttis*" action=deny 

 7   ;;; Allowed local users
     src-address=192.168.1.0/24 action=allow 

 8   ;;; Deny everything else
     action=deny 
[admin@Hermes] ip web-proxy access>
10

OK, more info will be put there shortly (i was sure there have been more examples earlier… weird). for now comments on your config:

  1. the URL is a wildcard by default, that is it only supports “*” for any number of any symbols, and “?” for any one symbol. it means that dots “.” do not need escaping with ""
  2. wildcard is a match on the whole string, so you are correct by putting a “*” at the beginning and at the end of it
  3. to match regular expression (which is not a complete string, but rather a substring), put “:” at the beginning of the value specified as URL
  4. the “\.” contruction put in wildcard means “\.” in the url. the same expression put in regexp meand "" appended by any symbol.
11

hi there
please any admin or offcial news from mikrotik regarding my error
as soon as possible please

12

OK, so to block any site I’m suppose to type something like this in the url-field:
theurl.com
If I got it right… Am I right or wrong?

but why on earth does
lunarstorm block http://www.lunarstorm.se
while
snuttis does NOT block http://www.snuttis.com

What am I doing wrong?? Or is it a…a…bug!!! :open_mouth: :wink:

13

well, work for us :slight_smile: maybe a typo somewhere…

14

well, work for us Smile maybe a typo somewhere…

Well, I wish it was that easy…mabye it is??? :confused: Here’s a long post…hi hi

This is now the complete access list for my web-proxy:

Terminal vt102 detected, using multiline input mode
[admin@Hermes] > ip 
[admin@Hermes] ip> web
[admin@Hermes] ip web-proxy> access 
[admin@Hermes] ip web-proxy access> print
Flags: X - disabled, I - invalid 
 0   ;;; Deny using this proxy as telnet/smtp relay
     dst-port=23-25 action=deny 

 1   ;;; Deny access to some URL.s
     url="*ad.aftonbladet.se*" action=deny 

 2   url="*ad.adtoma*" action=deny 

 3   url="*snyggast.se*" action=deny 

 4   url="*ix.se*" action=deny 

 5   url="*lunarstorm*" action=deny 

 6   url="*snuttis*" action=deny 

 7   url="*vilsharad.net*" action=deny 

 8   ;;; Allowed local users
     src-address=192.168.1.0/24 action=allow 

 9   ;;; Deny everything else
     action=deny 
[admin@Hermes] ip web-proxy access>

#0 - Not tested…
#1 - NOT working
#2 - Works!
#3 - Works!
#4 - Works!
#5 - Works!
#6 - NOT working
#7 - NOT working (my own domain btw. Just for testing)
#8 - Works!
#9 - Works!

Conclusion: some URL.s are possible to detect in the web-proxy, some are not. I’ve tried to clear the web-proxy-cache and the browser’s cache and reboot the MT. Neither helps.
There are no other rules under “cache” and “direct”

I tried to enter the snuttis’ IP-adress as follows, that didn’t work EITHER!!?

 6   dst-address=12.13.14.15/32 action=deny

also tried this, just in case

 6   src-address=12.13.14.15/32 action=deny

(However, I can drop the packages from that IP in the input chain “IP/firewall/filter”. That makes the site unreachable, but it doesn’t seem like the right way to deny access to an URL.)

Any tips were I shall look for typos. The rules are obviously runned through since some entries does work. If I change “trusted users” (i.e rule no.8 ) to something else, all access is denied through the proxy. And that rule is checked at the last.

This is starting to annoy me now, and I must therefore “fix” it so I can get some sleep at nights… :laughing:
Could it mabye be some kind of update-issue? I updated the router from 2.8 to 2.9RC5 and then to 2.9RC6. Could something we can not see be corrupt somehow??

15

I just made a backup file for my router.

I loaded it into a hex editor and found the settings for the proxy at the end of the file. It seems like the old settings (for v.2.8, that I no longer can access) is mixed up with the new settings (for v.2.9) and the strings looks quite corrupt.

My guess is that it is this that makes my proxy malfunction and that the solution is to reset all settings or re-install the whole router.

I’ll drop a note here with the results when I’ve tried to solve the case this hard way, since it seems to work for all you other guys. :sunglasses:

16

Just a little update of this issue…if anyone is interested :unamused:

After a complete reset of the router, the backup file looked more reasonable in the hex-editor. The web-proxy filtering worked better, but not good. Some url.s were still un-detected.

BUT!! The 2.9RC7 release seemed to help a lot of problems that I had with my proxy-settings! (I still don’t think it was a bad idéa to reset the system and set up the rules from scratch. It feels good.)

Thank’s! I’m happy again! :smiley:

17

i m using hotspot and i forward the hotspot to my webproxy 172.16.54.1:3128 and the hotspot is transparent
when i disable the transpernt in the hotspot user profile and ues proxy 172.16.54.1:3128 in client pc it works great . How do I make it work transparently?

Thanks to all who are thinking to reply!!!