mangle: forward chain problem

I’m experiencing weird problems when using mangle rules with forwarding chain.
I tried to setup a simple PCQ rule.
ROS 3.30, ADSL connection with NAT (masquerade)…
10.1.0.0/24 is the LAN network behind the NAT
pppoe-out1 is the ADSL connection


When mangle rules 10 and 11 are active as it’s the case here I have problems connecting to lots of web sites (http traffic timeouts) and SSH to remote sites breaks.
I get readings Byte/packet for both rules.

If I use prerouting chain it’s all fine.

I need forwarding chain since I want to use Out-interface HTB and evenly distribute traffic for my clients.


What could be the problem?

Here is my current setup:

[saiko@core] /ip firewall mangle> pr
Flags: X - disabled, I - invalid, D - dynamic 
10   chain=forward action=mark-connection new-connection-mark=users_conn 
     passthrough=yes src-address=10.1.0.0/24 

11   chain=forward action=mark-packet new-packet-mark=users passthrough=no 
     connection-mark=users_conn

(rules 1-9 are all disabled)

[saiko@core] /queue tree> pr
Flags: X - disabled, I - invalid 
 0   name="total_download" parent=LAN limit-at=0 priority=8 max-limit=10M burst-limit=0 burst-threshold=0 
     burst-time=0s 

 3   name="total_upload" parent=pppoe-out1 limit-at=0 priority=8 max-limit=500k burst-limit=0 burst-threshold=0 
     burst-time=0s 

12   name="equal_user_dl" parent=total_download packet-mark=users limit-at=0 queue=PCQ_down priority=8 max-limit=0 
     burst-limit=0 burst-threshold=0 burst-time=0s 

13   name="equal_user_ul" parent=total_upload packet-mark=users limit-at=0 queue=PCQ_up priority=8 max-limit=0 
     burst-limit=0 burst-threshold=0 burst-time=0s

filter:

[saiko@core] /ip firewall filter> pr
Flags: X - disabled, I - invalid, D - dynamic 
 0   ;;; Drop invalid connections
     chain=input action=drop connection-state=invalid 

 1   ;;; Allow established connections
     chain=input action=accept connection-state=established 

 2   ;;; Allow related connections
     chain=input action=accept connection-state=related 

 3   ;;; Allow UDP
     chain=input action=accept protocol=udp 

 4   ;;; Allow ICMP
     chain=input action=accept protocol=icmp 

 5   ;;; Allow connection to router from local network
     chain=input action=accept src-address=10.1.0.0/24 

 6   ;;; allow connecting to tcp port 80 from anywhere
     chain=input action=accept protocol=tcp dst-port=80 

 7   ;;; pptp from anywhere
     chain=input action=accept protocol=tcp dst-port=1723 

 8   ;;; accept prot 41 for 6to4 HE tunnel
     chain=input action=accept protocol=41 src-address=216.66.80.98 

 9   ;;; drop ssh brute forcers
     chain=input action=reject reject-with=tcp-reset protocol=tcp src-address-list=ssh_blacklist 
     dst-port=22 

10   chain=input action=add-src-to-address-list connection-state=new protocol=tcp 
     src-address-list=ssh_stage3 address-list=ssh_blacklist address-list-timeout=5d dst-port=22 

11   chain=input action=add-src-to-address-list connection-state=new protocol=tcp 
     src-address-list=ssh_stage2 address-list=ssh_stage3 address-list-timeout=1m dst-port=22 

12   chain=input action=add-src-to-address-list connection-state=new protocol=tcp 
     src-address-list=ssh_stage1 address-list=ssh_stage2 address-list-timeout=1m dst-port=22 

13   chain=input action=add-src-to-address-list connection-state=new protocol=tcp address-list=ssh_stage1 
     address-list-timeout=1m dst-port=22 

14   ;;; allow connecting to tcp port 22 from anywhere
     chain=input action=accept protocol=tcp dst-port=22 

15   ;;; Drop everything else
     chain=input action=drop

NAT:

[saiko@core] /ip firewall nat> pr
Flags: X - disabled, I - invalid, D - dynamic 
 0   ;;; allow connecting to WAN from LAN
     chain=srcnat action=masquerade out-interface=pppoe-out1 

 1   ;;; yamato ssh
     chain=dstnat action=dst-nat to-addresses=10.1.0.11 to-ports=22 protocol=tcp in-interface=pppoe-out1 
     dst-port=2211 

 2   ;;; yamato rtorrent
     chain=dstnat action=dst-nat to-addresses=10.1.0.11 to-ports=6890-6891 protocol=tcp in-interface=pppoe-out1 
     dst-port=6890-6891 

 3   ;;; dm7025 CS
     chain=dstnat action=dst-nat to-addresses=10.1.0.25 to-ports=2666 protocol=udp in-interface=pppoe-out1 
     dst-port=2666 

 4 X chain=dstnat action=dst-nat to-addresses=10.1.0.25 to-ports=22 protocol=tcp in-interface=pppoe-out1 dst-port=2225 

 5 X chain=dstnat action=dst-nat to-addresses=10.1.0.25 to-ports=21 protocol=tcp in-interface=pppoe-out1 dst-port=2125 

 6   ;;; souryuu utorrent port
     chain=dstnat action=dst-nat to-addresses=10.1.0.13 to-ports=3111 protocol=tcp in-interface=pppoe-out1 
     dst-port=3111 

 7 X chain=dstnat action=dst-nat to-addresses=10.1.0.1 to-ports=1723 protocol=tcp in-interface=pppoe-out1 dst-port=80 

 8   ;;; redirect udp DNS requests to router
     chain=dstnat action=dst-nat to-addresses=10.1.0.1 protocol=udp in-interface=LAN dst-port=53 

 9   ;;; redirect tcp DNS requests to router
     chain=dstnat action=dst-nat to-addresses=10.1.0.1 protocol=tcp in-interface=LAN dst-port=53 

10   ;;; yamato rdp
     chain=dstnat action=dst-nat to-addresses=10.1.0.11 protocol=tcp in-interface=pppoe-out1 dst-port=3389

Hmmm…

MIGHT have discovered the problem.
Since i’m connected to pppoe there’s an option to change MSS.
Indeed there are 2 rules for mangle (only shown with print all in console):

[vobelic@core] > ip firewall mangle print all
Flags: X - disabled, I - invalid, D - dynamic 
 0 D chain=forward action=change-mss new-mss=1452 tcp-flags=syn protocol=tcp in-interface=pppoe-out1 
     tcp-mss=1453-65535 

 1 D chain=forward action=change-mss new-mss=1452 tcp-flags=syn protocol=tcp out-interface=pppoe-out1 
     tcp-mss=1453-65535

Apparently the order of mangle rules is important since those two have to be before all other mangle forward chain rules.

if you’re use NAT and PCQ then you must use global-out interface in queue tree for upload traffic, because PCQ must know src-address of your lan-users