RouterOS web proxy

Hi.

I have problems with my web proxy. When I download for second time any file less than max. cache size, so less than ~25MB, it downloads from internet again not from cache. Maybe the rule for caching all packets from proxy, DSCP=4 is not working properly,

23   ;;; HIT TRAFFIC FROM PROXY
     chain=output action=mark-packet new-packet-mark=proxyhit passthrough=no out-interface=Local dscp=4

or web proxy cache does not work at all.
I am using 3.30 RouterOS version on Intel processor based PC

Please, may someone help me to clear this thing.

thank you.

My config is:

[admin@MikroTik] > ip proxy pr
                 enabled: yes
             src-address: 0.0.0.0
                    port: 800
            parent-proxy: 0.0.0.0
       parent-proxy-port: 0
     cache-administrator: "webmaster"
          max-cache-size: 25200KiB
           cache-on-disk: yes
  max-client-connections: 500
  max-server-connections: 500
          max-fresh-time: 4w2d
   serialize-connections: yes
       always-from-cache: yes
          cache-hit-dscp: 4
             cache-drive: sata1
[admin@MikroTik] > ip firewall filter print 
Flags: X - disabled, I - invalid, D - dynamic 
 0   chain=input action=accept protocol=tcp src-address=192.168.0.0/16 in-interface=Local dst-port=800 

 1   chain=input action=accept protocol=tcp in-interface=WAN1 dst-port=500 

 2 X chain=input action=drop dst-address-list=facebook in-interface=Local 

 3 X chain=input action=accept src-address=192.168.12.100 in-interface=Local 

 4   chain=forward action=drop protocol=udp dst-address=209.249.222.80 dst-port=53 

 5   ;;; drop all traffic brute force attack sources
     chain=input action=drop src-address-list=sshblacklist 

 6   ;;; add new failed sshdarkgreylist to sshblacklist
     chain=input action=add-src-to-address-list connection-state=new protocol=tcp src-address-list=sshdarkgreylist address-list=sshblacklist 
     address-list-timeout=1h dst-port=21,22 

 7   ;;; add new failed sshgreylist to sshdarkgreylist
     chain=input action=add-src-to-address-list connection-state=new protocol=tcp src-address-list=sshgreylist address-list=sshdarkgreylist 
     address-list-timeout=1m dst-port=21,22 

 8   ;;; add new failed sshlightgreylist to sshgreylist
     chain=input action=add-src-to-address-list connection-state=new protocol=tcp src-address-list=sshlightgreylist address-list=sshgreylist 
     address-list-timeout=1m dst-port=21,22 

 9   ;;; new connections to sshlightgreylist
     chain=input action=add-src-to-address-list connection-state=new protocol=tcp address-list=sshlightgreylist address-list-timeout=1m dst-port=21,22 

10 X chain=input action=drop protocol=tcp src-address=211.21.47.50 dst-port=21 

11   ;;; Allow traffic between clients
     chain=forward action=accept in-interface=Local out-interface=Local 

12   chain=input action=accept src-address=xx.xx.xx..171 in-interface=!Local 

13   chain=input action=drop protocol=tcp dst-port=21 

14 X chain=input action=drop in-interface=!Local connection-type=ftp 

15 X chain=forward action=drop 

16 X chain=input action=drop 

17 X chain=input action=drop src-address=60.217.229.220 

18 X chain=forward action=drop dst-address=192.168.12.211 

19 X chain=input action=drop layer7-protocol=(unknown) 

20   ;;; Drop Telnet from Outside
     chain=input action=drop protocol=tcp in-interface=!Local dst-port=23 

21   ;;; WEB Proxy from outside
     chain=input action=drop protocol=tcp src-address=0.0.0.0/0 in-interface=WAN1 dst-port=800 

22   chain=input action=drop protocol=tcp src-address=0.0.0.0/0 in-interface=WAN2 dst-port=800 

23   chain=input action=drop protocol=tcp src-address=0.0.0.0/0 in-interface=WAN3 dst-port=800 

24 X chain=input action=accept src-address=192.168.12.0/24 

25 X chain=input action=accept dst-address=192.168.12.0/24 

26   chain=input action=accept src-address-list=Local-active-addresses 

27   chain=input action=accept dst-address-list=Local-active-addresses 

28   chain=forward action=accept src-address-list=Local-active-addresses 

29   chain=forward action=accept dst-address-list=Local-active-addresses 

30   chain=forward action=reject reject-with=icmp-network-unreachable protocol=tcp dst-port=25 

31   ;;; To WEB proxy
     chain=input action=reject reject-with=icmp-network-unreachable protocol=tcp src-address=!192.168.0.0/16 dst-port=800 

32   chain=output action=reject reject-with=icmp-network-unreachable protocol=tcp dst-address=xx.xx.xx.247 dst-port=800 

33   chain=output action=reject reject-with=icmp-network-unreachable protocol=tcp dst-address=73.106.3.43 dst-port=800 

34   chain=output action=reject reject-with=icmp-network-unreachable protocol=tcp dst-address=xx.xx.xx..207 dst-port=800 

35 X chain=input action=reject reject-with=icmp-network-unreachable protocol=tcp dst-port=445 

36   chain=input action=drop src-address=192.168.12.124 

37   chain=input action=drop dst-address=192.168.12.124 

38 X ;;; P2P
     chain=forward action=drop p2p=bit-torrent 

39 X chain=forward action=drop p2p=blubster 

40 X chain=forward action=drop p2p=direct-connect 

41 X chain=forward action=drop p2p=edonkey 

42 X chain=forward action=drop p2p=fasttrack 

43 X chain=forward action=drop p2p=gnutella 

44 X chain=forward action=drop p2p=soulseek 

45 X chain=forward action=drop p2p=warez 

46 X chain=forward action=drop p2p=winmx 

47 X chain=forward action=drop protocol=tcp src-port=3074 

48 X chain=forward action=drop protocol=udp src-port=3074 

49 X chain=forward action=drop src-address=192.168.0.0/16 

50 X chain=forward action=drop dst-address=192.168.0.0/16 

51   ;;; All Local Network Block
     chain=input action=drop src-address=192.168.0.0/16 

52   chain=input action=drop dst-address=192.168.0.0/16 

53 I ;;; Check if dest is an open customer
     chain=forward action=jump jump-target=open-customers dst-address-list=open-customers 

54 I ;;; Check Known Bad Hosts
     chain=forward action=jump jump-target=bad-hosts 

55   ;;; Reject if in the 24-hour-list
     chain=forward action=reject reject-with=icmp-network-unreachable src-address-list=24-hour-list 

56   ;;; Take no action on bogons
     chain=bad-host-detection action=return src-address-list=bogons 

57   ;;; Add to the 30 second list
     chain=bad-host-detection action=add-src-to-address-list address-list=30-seond-list address-list-timeout=30s 

58   chain=bad-host-detection action=return 

59   ;;; jump to the bad-host-detection chain
     chain=forward action=jump jump-target=bad-host-detection src-address-list=!local-addr 

60   ;;; jump to the bad-host-detection chain
     chain=forward action=jump jump-target=bad-host-detection src-address-list=!local-addr 

61 X ;;; log and reject the rest
     chain=forward action=log log-prefix="" 

62   chain=forward action=reject reject-with=icmp-network-unreachable 

[admin@MikroTik] > ip firewall mangle pr    
Flags: X - disabled, I - invalid, D - dynamic 
 0 X chain=postrouting action=mark-connection new-connection-mark=DNS_conn passthrough=yes protocol=udp out-interface=!Local dst-port=53 

 1 X chain=prerouting action=mark-routing new-routing-mark=DNS_rou passthrough=yes connection-mark=DNS_conn 

 2 X chain=input action=accept src-address=192.168.11.100 in-interface=Local connection-bytes=0-500000 

 3   chain=input action=mark-connection new-connection-mark=pub1_conn passthrough=yes in-interface=WAN1 

 4   chain=input action=mark-connection new-connection-mark=pub2_conn passthrough=yes in-interface=WAN2 

 5   chain=input action=mark-connection new-connection-mark=pub3_conn passthrough=yes in-interface=WAN3 

 6   chain=output action=mark-routing new-routing-mark=to_pub1 passthrough=yes connection-mark=pub1_conn 

 7   chain=output action=mark-routing new-routing-mark=to_pub2 passthrough=yes connection-mark=pub2_conn 

 8   chain=output action=mark-routing new-routing-mark=to_pub3 passthrough=yes connection-mark=pub3_conn 

 9   chain=prerouting action=accept dst-address=80.78.75.0/24 in-interface=Local 

10   chain=prerouting action=accept dst-address=79.106.3.0/24 in-interface=Local 

11   chain=prerouting action=mark-connection new-connection-mark=pub1_conn passthrough=yes dst-address-type=!local in-interface=Local 
     per-connection-classifier=both-addresses:3/0 

12   chain=prerouting action=mark-connection new-connection-mark=pub2_conn passthrough=yes dst-address-type=!local in-interface=Local 
     per-connection-classifier=both-addresses:3/1 

13   chain=prerouting action=mark-connection new-connection-mark=pub3_conn passthrough=yes dst-address-type=!local in-interface=Local 
     per-connection-classifier=both-addresses:3/2 

14   chain=prerouting action=mark-routing new-routing-mark=to_pub1 passthrough=yes in-interface=Local connection-mark=pub1_conn 

15   chain=prerouting action=mark-routing new-routing-mark=to_pub2 passthrough=yes in-interface=Local connection-mark=pub3_conn 

16   chain=prerouting action=mark-routing new-routing-mark=to_pub3 passthrough=yes in-interface=Local connection-mark=pub2_conn 

17   ;;; facebook upload
     chain=prerouting action=mark-routing new-routing-mark=facebook passthrough=yes dst-address-list=facebook in-interface=Local 

18   ;;; mark http connections
     chain=prerouting action=mark-connection new-connection-mark=http passthrough=yes protocol=tcp in-interface=Local dst-port=80 

19   ;;; Route mark messenger TCP ports
     chain=prerouting action=mark-routing new-routing-mark=messengers_routeup passthrough=yes protocol=tcp in-interface=Local 
     dst-port=1503,1863,5000,5001,5050,5100,5190-6901,54055 

20   ;;; Route mark messenger UDP ports
     chain=prerouting action=mark-connection new-connection-mark=messengers_routeup passthrough=yes protocol=udp in-interface=Local 
     dst-port=1503,5000-5010,5109,6901,54055 

21   ;;; Route mark Skype
     chain=prerouting action=mark-routing new-routing-mark=messengers_routeup passthrough=yes layer7-protocol=skypeout in-interface=Local 

22   chain=prerouting action=mark-routing new-routing-mark=messengers_routeup passthrough=yes layer7-protocol=skypetoskype in-interface=Local 

23   ;;; HIT TRAFFIC FROM PROXY
     chain=output action=mark-packet new-packet-mark=proxyhit passthrough=no out-interface=Local dscp=4
 

[admin@MikroTik] /ip firewall nat> pr
Flags: X - disabled, I - invalid, D - dynamic 
 0   chain=srcnat action=masquerade out-interface=WAN1 

 1   chain=srcnat action=masquerade out-interface=WAN2 

 2   chain=srcnat action=masquerade out-interface=WAN3 

 3   chain=dstnat action=dst-nat to-addresses=192.168.12.100 to-ports=3389 protocol=tcp dst-address=xx.xx.xx.247 dst-port=3389 

 4   chain=dstnat action=dst-nat to-addresses=192.168.12.20 to-ports=8080 protocol=tcp dst-address=xx.xx.xx.247 dst-port=8080 

 5   chain=dstnat action=dst-nat to-addresses=192.168.12.21 to-ports=8081 protocol=tcp dst-address=xx.xx.xx.247 dst-port=8081 

 6   ;;; Transparent DNS Cache
     chain=dstnat action=redirect in-interface=Local connection-mark=dns 

 7   ;;; Transparent Web Cache
     chain=dstnat action=redirect to-ports=800 protocol=tcp in-interface=Local connection-mark=http 

 8   ;;; Transparent proxy for NTP requests
     chain=dstnat action=redirect in-interface=Local connection-mark=ntp 


Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit 
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 A S  0.0.0.0/0                          xx.xx.xx.225       1       
 1 A S  0.0.0.0/0                          xx.xx.xx.1%WAN2    1       
 2 A S  0.0.0.0/0                          xx.xx.xx.1%WAN3    1       
 3 A S  0.0.0.0/0                          WAN1              1       
 4   S  0.0.0.0/0                          WAN3        2       
                                           WAN2       
 5 A S  0.0.0.0/0                          Local              1       
 6 X S  0.0.0.0/0                          WAN1              1       
 7 A S  0.0.0.0/0                          Local              1       
 8 A S  0.0.0.0/0                          WAN3        1       
 9  DS  0.0.0.0/0                          xx.xx.xx.1         1       
10  DS  0.0.0.0/0                          xx.xx.xx.1         1       
11   S  0.0.0.0/0                          xx.xx.xx.225       2       
12   S  0.0.0.0/0                          WAN2        3       
13 ADC  xx.xx.xx.1/32      xx.xx.xx.207    WAN3        0       
                                           WAN2       
14 ADC  xx.xx.xx.224/27    xx.xx.xx.247    WAN1              0       
15 ADC  192.168.0.0/16     192.168.0.0     Local              0       
16 A S  192.168.11.0/24                    Local              1       
17 ADC  192.168.11.0/32    192.168.11.1    Local              0       
18 ADC  192.168.12.0/24    192.168.12.1    Local              0

sorry even those may be helpful :slight_smile:

queue tree print

Flags: X - disabled, I - invalid, D - dynamic 
 0   chain=srcnat action=masquerade out-interface=ABCom 

 1   chain=srcnat action=masquerade out-interface=Albtelekom1 

 2   chain=srcnat action=masquerade out-interface=Albtelekom2 

 3   chain=dstnat action=dst-nat to-addresses=192.168.12.100 to-ports=3389 protocol=tcp dst-address=80.78.75.247 dst-port=3389 

 4   chain=dstnat action=dst-nat to-addresses=192.168.12.20 to-ports=8080 protocol=tcp dst-address=80.78.75.247 dst-port=8080 

 5   chain=dstnat action=dst-nat to-addresses=192.168.12.21 to-ports=8081 protocol=tcp dst-address=80.78.75.247 dst-port=8081 

 6   ;;; Transparent DNS Cache

17   name="Prio 8 up" parent=Upload All packet-mark=prio8_up limit-at=0 queue=PCQ_upload_prio7&8 priority=8 max-limit=2200k burst-limit=0 burst-threshold=0 
     burst-time=0s 

18 X name="All download" parent=global-out packet-mark=test-down limit-at=0 queue=PCQ_download priority=1 max-limit=0 burst-limit=0 burst-threshold=0 
     burst-time=0s 

19 X name="HTTP web down" parent=global-in packet-mark=prio3 limit-at=0 queue=PCQ_download_prio3 priority=1 max-limit=0 burst-limit=0 burst-threshold=0 
     burst-time=0s 

20   name="FwDownload_All" parent=global-out packet-mark=fw_all_dw limit-at=0 priority=1 max-limit=9M burst-limit=0 burst-threshold=0 burst-time=0s 

21 X name="FwUpload_All" parent=global-in packet-mark=Fw_prio1_up limit-at=0 priority=1 max-limit=2700k burst-limit=0 burst-threshold=0 burst-time=0s 

22   name="Forward Prio 1 dw" parent=FwDownload_All packet-mark=Fw_prio1 limit-at=200k queue=PCQ_download_prio1_2 priority=1 max-limit=510k burst-limit=0 
     burst-threshold=0 burst-time=0s 

23 X name="Forward Prio 1 up" parent=FwUpload_All packet-mark=Fw_prio1_up limit-at=200k queue=PCQ_download_prio1_2 priority=1 max-limit=660k burst-limit=0 
     burst-threshold=0 burst-time=0s 

24   name="Forward Prio 2 dw" parent=FwDownload_All packet-mark=Fw_prio2 limit-at=200k queue=PCQ_download_prio1_2 priority=2 max-limit=2M burst-limit=0 
     burst-threshold=0 burst-time=0s 

25 X name="Forward Prio 2 up" parent=FwUpload_All packet-mark=Fw_prio1_up limit-at=0 queue=PCQ_upload priority=2 max-limit=0 burst-limit=0 burst-threshold=0 
     burst-time=0s 

26   name="Forward Prio 3 dw" parent=FwDownload_All packet-mark=Fw_prio3 limit-at=2M queue=PCQ_download_prio3 priority=3 max-limit=5M burst-limit=0 
     burst-threshold=0 burst-time=0s 

27 X name="Forward Prio 3 up" parent=FwUpload_All packet-mark=Fw_prio3_up limit-at=600k queue=PCQ_upload priority=3 max-limit=2700k burst-limit=0 
     burst-threshold=0 burst-time=0s 

28   name="Forward Prio 4 dw" parent=FwDownload_All packet-mark=Fw_prio4 limit-at=1M queue=PCQ_download_prio4_5 priority=4 max-limit=3M burst-limit=0 
     burst-threshold=0 burst-time=0s 

29 X name="Forward Prio 4 up" parent=FwUpload_All limit-at=0 queue=PCQ_upload priority=4 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 

30   name="Forward Prio 5 dw" parent=FwDownload_All packet-mark=Fw_prio5 limit-at=300k queue=PCQ_download_prio4_5 priority=5 max-limit=3M burst-limit=0 
     burst-threshold=0 burst-time=0s 

31 X name="Forward Prio 5 up" parent=FwUpload_All packet-mark=Fw_prio5_up limit-at=0 queue=PCQ_upload priority=5 max-limit=0 burst-limit=0 burst-threshold=0 
     burst-time=0s 

32   name="Forward Prio 6 dw" parent=FwDownload_All packet-mark=Fw_prio6 limit-at=300k queue=PCQ_download_prio6 priority=6 max-limit=3200k burst-limit=0 
     burst-threshold=0 burst-time=0s 

33 X name="Forward Prio 6 up" parent=FwUpload_All limit-at=0 queue=PCQ_upload priority=6 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s 

34   name="Forward Prio 7 dw" parent=FwDownload_All packet-mark=Fw_prio7 limit-at=0 queue=PCQ_download_prio7_8 priority=7 max-limit=750k burst-limit=0 
     burst-threshold=0 burst-time=0s 

35 X name="Forward Prio 7 up" parent=FwUpload_All packet-mark=Fw_prio7_up limit-at=0 queue=PCQ_upload priority=7 max-limit=600k burst-limit=0 burst-threshold=>
     burst-time=0s 

36   name="Forward Prio 8 dw" parent=FwDownload_All packet-mark=Fw_prio8 limit-at=0 queue=PCQ_download_prio7_8 priority=8 max-limit=750k burst-limit=0 
     burst-threshold=0 burst-time=0s 

37 X name="Forward Prio 8 up" parent=FwUpload_All packet-mark=Fw_prio8_up limit-at=0 queue=PCQ_upload priority=8 max-limit=600k burst-limit=0 burst-threshold=>
     burst-time=0s 

queue type print

 0 name="default" kind=pfifo pfifo-limit=50 

 1 name="ethernet-default" kind=pfifo pfifo-limit=50 

 2 name="wireless-default" kind=sfq sfq-perturb=5 sfq-allot=1514 

 3 name="synchronous-default" kind=red red-limit=60 red-min-threshold=10 red-max-threshold=50 red-burst=20 red-avg-packet=1000 

 4 name="hotspot-default" kind=sfq sfq-perturb=5 sfq-allot=1514 

 5 name="PCQ_download" kind=pcq pcq-rate=0 pcq-limit=50 pcq-classifier=dst-address pcq-total-limit=3500 

 6 name="PCQ_upload" kind=pcq pcq-rate=0 pcq-limit=50 pcq-classifier=src-address pcq-total-limit=3500 

 7 name="PCQ_download_prio1_2" kind=pcq pcq-rate=510000 pcq-limit=50 pcq-classifier=dst-address pcq-total-limit=3500 

 8 name="PCQ_download_prio3" kind=pcq pcq-rate=5000000 pcq-limit=50 pcq-classifier=dst-address pcq-total-limit=3500 

 9 name="PCQ_download_prio4_5" kind=pcq pcq-rate=750000 pcq-limit=50 pcq-classifier=dst-address pcq-total-limit=3500 

10 name="PCQ_download_prio6" kind=pcq pcq-rate=2200000 pcq-limit=50 pcq-classifier=dst-address pcq-total-limit=3500 

11 name="PCQ_download_prio7_8" kind=pcq pcq-rate=250000 pcq-limit=50 pcq-classifier=dst-address pcq-total-limit=3500 

12 name="PCQ_upload_prio7&8" kind=pcq pcq-rate=300000 pcq-limit=50 pcq-classifier=src-address pcq-total-limit=3500 

13 name="PCQ_upload_prio3-6" kind=pcq pcq-rate=700000 pcq-limit=50 pcq-classifier=src-address pcq-total-limit=3500 

14 name="default-small" kind=pfifo pfifo-limit=10

enable ‘webproxy accounting’ logging and then you can check, is it ‘HIT’ or ‘MISS’

Hello.

I monitored traffic from router (web proxy cache) with:

chain=output action=log protocol=tcp out-interface=Local dscp=4 log-prefix="from_proxy="

When I downloaded for second time same files, they was miss not hit.
Hit was only a little bytes. For example refresh www.google.com has HIT.

What can I do for having all data cached? what is wrong :frowning:
Have someone packet flow with web proxy scheme?


Thanks.

Please, any help?

come on guys, nobody helps me.