Packet Escaping in Mangle

RouterOS General
1

Hi, there.
I want my route-marked twitter traffic to go through a l2tp ipsec vpn(l2tp-out1), and confused by the packet mangling issue. The tcp handshake process is ok, when connection set up, the first “GET / HTTP/1.0” packet(PSH) always goes to the default gateway, and blocked by the “Great Firewall” made by our Big Big Brother. So I did an experiment using the following firewall configuration, try “wget http://www.twitter.com”, and found out that the PSH packet is NOT marked in prerouting phase every time. The mangle system just ignore it !!! Due to my poor network knowlege, I’ve read the packet document, still can’t figure out WHY. Sorry for my poor English. Please help, any advice would be appreciated.

/* experimental configuration in my hAP ac, ROS v6.37.1 */
/ip firewall address-list
add address=199.59.0.0/16 list=twitter
/ip firewall filter
add action=fasttrack-connection chain=forward comment=“defconf: fasttrack”
connection-state=established,related
add action=accept chain=forward comment=“defconf: accept established,related”
connection-state=established,related
add action=drop chain=forward comment=“defconf: drop invalid”
connection-state=invalid
add action=drop chain=forward comment=
“defconf: drop all from WAN not DSTNATed” connection-nat-state=!dstnat
connection-state=new in-interface=ether1
add action=accept chain=input protocol=icmp
add action=accept chain=input connection-state=established
add action=accept chain=input connection-state=related
add action=drop chain=input in-interface=pppoe-out1
add action=drop chain=input in-interface=l2tp-out1
/ip firewall mangle
add action=log chain=prerouting connection-mark=twitter disabled=no log=yes
log-prefix=ttt tcp-flags=“”
/ip firewall nat
add action=masquerade chain=srcnat comment=“defconf: masquerade”
out-interface=pppoe-out1
add action=masquerade chain=srcnat out-interface=l2tp-out1

/* packets captured by wireshark, using a static dst-address based routing policy to let it go the right way /
wireshark.png
/ hAP ac log, PSH packet missing */
22:42:16 firewall,info ttt prerouting: in:bridge out:(none), src-mac xx:xx:xx:xx:xx:xx, proto TCP (SYN), 192.168.1.100:46219->199.59.148.82:80, len 60
22:42:16 firewall,info ttt prerouting: in:bridge out:(none), src-mac xx:xx:xx:xx:xx:xx, proto TCP (SYN), 192.168.1.100:46219->199.59.148.82:80, len 60
22:42:16 firewall,info ttt prerouting: in:bridge out:(none), src-mac xx:xx:xx:xx:xx:xx, proto TCP (ACK), 192.168.1.100:46219->199.59.148.82:80, NAT (192.168.1.100:46219->10.0.0.2:46219)->199.59.148.82:80, len 52
22:42:16 firewall,info ttt prerouting: in:bridge out:(none), src-mac xx:xx:xx:xx:xx:xx, proto TCP (ACK), 192.168.1.100:46219->199.59.148.82:80, NAT (192.168.1.100:46219->10.0.0.2:46219)->199.59.148.82:80, len 52
22:42:16 firewall,info ttt prerouting: in:bridge out:(none), src-mac xx:xx:xx:xx:xx:xx, proto TCP (SYN), 192.168.1.100:38711->199.59.148.82:443, len 60
22:42:16 firewall,info ttt prerouting: in:bridge out:(none), src-mac xx:xx:xx:xx:xx:xx, proto TCP (SYN), 192.168.1.100:38711->199.59.148.82:443, len 60
22:42:17 firewall,info ttt prerouting: in:bridge out:(none), src-mac xx:xx:xx:xx:xx:xx, proto TCP (ACK), 192.168.1.100:38711->199.59.148.82:443, NAT (192.168.1.100:38711->10.0.0.2:38711)->199.59.148.82:443, len 52
22:42:17 firewall,info ttt prerouting: in:bridge out:(none), src-mac xx:xx:xx:xx:xx:xx, proto TCP (ACK), 192.168.1.100:38711->199.59.148.82:443, NAT (192.168.1.100:38711->10.0.0.2:38711)->199.59.148.82:443, len 52
22:42:18 firewall,info ttt prerouting: in:bridge out:(none), src-mac xx:xx:xx:xx:xx:xx, proto TCP (ACK,FIN), 192.168.1.100:46219->199.59.148.82:80, NAT (192.168.1.100:46219->10.0.0.2:46219)->199.59.148.82:80, len 52
22:42:18 firewall,info ttt prerouting: in:bridge out:(none), src-mac xx:xx:xx:xx:xx:xx, proto TCP (ACK,FIN), 192.168.1.100:46219->199.59.148.82:80, NAT (192.168.1.100:46219->10.0.0.2:46219)->199.59.148.82:80, len 52
22:42:19 firewall,info ttt prerouting: in:bridge out:(none), src-mac xx:xx:xx:xx:xx:xx, proto TCP (SYN), 192.168.1.100:62253->199.59.150.7:443, len 60
22:42:19 firewall,info ttt prerouting: in:bridge out:(none), src-mac xx:xx:xx:xx:xx:xx, proto TCP (SYN), 192.168.1.100:62253->199.59.150.7:443, len 60
22:42:19 firewall,info ttt prerouting: in:bridge out:(none), src-mac xx:xx:xx:xx:xx:xx, proto TCP (ACK), 192.168.1.100:46219->199.59.148.82:80, NAT (192.168.1.100:46219->10.0.0.2:46219)->199.59.148.82:80, len 52
22:42:19 firewall,info ttt prerouting: in:bridge out:(none), src-mac xx:xx:xx:xx:xx:xx, proto TCP (ACK), 192.168.1.100:46219->199.59.148.82:80, NAT (192.168.1.100:46219->10.0.0.2:46219)->199.59.148.82:80, len 52
22:42:19 firewall,info ttt prerouting: in:bridge out:(none), src-mac xx:xx:xx:xx:xx:xx, proto TCP (ACK), 192.168.1.100:62253->199.59.150.7:443, NAT (192.168.1.100:62253->10.0.0.2:62253)->199.59.150.7:443, len 52
22:42:19 firewall,info ttt prerouting: in:bridge out:(none), src-mac xx:xx:xx:xx:xx:xx, proto TCP (ACK), 192.168.1.100:62253->199.59.150.7:443, NAT (192.168.1.100:62253->10.0.0.2:62253)->199.59.150.7:443, len 52
22:42:20 firewall,info ttt prerouting: in:bridge out:(none), src-mac xx:xx:xx:xx:xx:xx, proto TCP (ACK,FIN), 192.168.1.100:38711->199.59.148.82:443, NAT (192.168.1.100:38711->10.0.0.2:38711)->199.59.148.82:443, len 52
22:42:20 firewall,info ttt prerouting: in:bridge out:(none), src-mac xx:xx:xx:xx:xx:xx, proto TCP (ACK,FIN), 192.168.1.100:38711->199.59.148.82:443, NAT (192.168.1.100:38711->10.0.0.2:38711)->199.59.148.82:443, len 52
22:42:20 firewall,info ttt prerouting: in:bridge out:(none), src-mac xx:xx:xx:xx:xx:xx, proto TCP (RST), 192.168.1.100:38711->199.59.148.82:443, NAT (192.168.1.100:38711->10.0.0.2:38711)->199.59.148.82:443, len 40
22:42:20 firewall,info ttt prerouting: in:bridge out:(none), src-mac xx:xx:xx:xx:xx:xx, proto TCP (RST), 192.168.1.100:38711->199.59.148.82:443, NAT (192.168.1.100:38711->10.0.0.2:38711)->199.59.148.82:443, len 40

Need help with policy based routing and VPN
2

I’ve obsessed by this issue a whole week, tried many methods to locate the real cause.
It’s definitely a bug, or a strange feature as we programmers always say. Orz…

During my tedious testing work, a SURPRISING FACT could be reproduced.

When packet sniffer tool starts, the ACK,PSH packets stop escaping from mangling, all packets go through the right gateway.
Almost any sniffer rule works, even it’s irrelevant to our src/dst address. OTL…
Like the following…
/tool sniffer
set filter-interface=wlan2 filter-ip-address=192.16.31.61/32 filter-ip-protocol=tcp filter-port=http

3

Your packets get fasttracked. In fasttrack, they circumvent all firewall rules, including mangle rules.
Disable fasttrack and it will work as expected.

During torching/sniffing, fasttrack gets automatically disabled, and that is the cause of your described behavior.
In ROS versions prior to 6.37, it all worked since PPPoE was not fasttrack enabled.

4

Thank you. When fasttrack rule disabled, It works perfect.
And…another stupid question, how can I just mask a particular address list from fasttrack?
“dst-address-list=![some list]” doesn’t work.


5

Now that’s a good question.
One thing working is using the input interface as a selection criteria.
I have 2 PPPoE interfaces, one of which needs mangling, the other doesn’t.
So I have 2 fasttrack rules in the forward chain:

  • first for interface PPPoE1 for established/related packets
  • second for “all ethernet” for established/related packets
    This will prevent fasttracking the packets for PPPoE2 which gives me the issues, and it works nicely.
    Never tried to go further…

In your case this would mean only to fasttrack packets with input_interface=pppoe-out1. No address list is actually needed.

6

Thank you very much!
I have one PPPoE connection linked to the Internet only, and I need to use dst-addr white list to separate outgoing traffic from default gateway.
It’s a good practice just to disable fasttrack and enjoy the view outside the Wall :smiley: