Forward Drop invalid | Broke my IP routing?

RouterOS General
1

I need some clarification on this because it’s been a head scratcher for me for days as to why I couldn’t reach ANY subnets beyond a certain point in my network
Here’s a rough diagram

The whole network was running OSPF/MPLS up to Router E
Network addresses were advertised just fine, everything had the correct routing table
I couldn’t reach any subnet on F from A/B/C, I also couldn’t reach any IP address on router E other than IP addresses I put on the interface facing D
Traceroute didn’t go even 1 hop from router A/B/C, but D worked fine.I think this is due to MPLS

On router D I have a firewall filter rule that just says “action=drop chain=forward comment=“Preset: Drop Invalid” connection-state=invalid” as soon as I disable that rule, everything works fine and I can ping end to end. Why is this?

2

The counters ticking on this rule? If yes, check firewall filters rules with source addresses, static arp records and mangle rules. Put config here. Or maybe you’ll find a mistake by yourself :slight_smile:

3

If the official firewall manual is not clear, try reading my summary for newbies.

4

Yes. It was at that point where I was just throwing my arms up in the air thinking “this HAS to be a bug/feature” and trying random things

Here’s part of the config - the entire thing is much larger as router D serves multiple purposes. I’ve pruned what shouldn’t be relevant but left entries in related to both interfaces.
ether1 is the uplink to C, ether2 is the downlink to E, D and E should only really talk to each other on VLAN52, the native interface and VLAN200 are part of a legacy bridged network

172.16.201.10 is the loopback address
172.16.52.1 is router D on the D->E segment
172.16.52.2 is router E on the D->E segment
The above addresses are reachable


/interface bridge
add comment="!! Need to remove later" name=TempBridge
add name="MPLS Loopback" protocol-mode=none
add name="Mgmt Bridge"
/interface ethernet
set [ find default-name=ether1 ] comment="Uplink to C"
set [ find default-name=ether2 ] comment="Downlink to E"
/interface pppoe-client
add disabled=no interface=PPPoE-Bridge max-mru=1500 max-mtu=1500 name=pppoe-WAN password=*** user=***
/interface vpls
add advertised-l2mtu=1516 disabled=no l2mtu=1516 mac-address=11:22:33:44:55:66 name="VPLS - Uplink" remote-peer=172.16.201.1 vpls-id=2:11
/interface vlan
add disabled=yes interface="VPLS - Uplink" name="VPLS.5 - Trusted PPPoE" vlan-id=5
add comment="Downlink Old Mgmt" interface=ether2 name="ether2.200 - Old Mgmt Network" vlan-id=200
add interface=ether2 name="ether2.52 - Downlink to E" vlan-id=52
/interface list
add name=InternetFacing
add name=Management
add name=CommsN_Out
add name=HS-PPPoE
/routing ospf instance
set [ find default=yes ] metric-default=5 router-id=172.16.201.10
/interface bridge port
add bridge=TempBridge interface=ether2
add bridge=TempBridge interface=ether4
/ip firewall connection tracking
set enabled=yes
/interface list member
add interface=ether1 list=InternetFacing
add interface="ether2.200 - Old Mgmt Network" list=InternetFacing
add interface=pppoe-WAN list=InternetFacing
/ip address
add address=172.16.201.10 interface="MPLS Loopback" network=172.16.201.10
add address=10.245.1.1/24 interface="Mgmt Bridge" network=10.245.1.0
add address=172.16.51.3/27 interface="ether2.200 - Old Mgmt Network" network=172.16.51.0
add address=172.16.52.1/27 interface="ether2.52 - Downlink to E" network=172.16.52.0
/ip dhcp-client
add default-route-distance=5 dhcp-options=hostname,clientid disabled=no interface=ether1
add add-default-route=no dhcp-options=hostname,clientid disabled=no interface=ether2
/ip firewall filter
add action=accept chain=input comment="Preset: Established/Related" connection-state=established,related
add action=accept chain=forward comment="Preset: Established/Related" connection-state=established,related
add action=drop chain=forward comment="Prevent Office routing to Private IP ranges" dst-address-list=PrivateIPs in-interface=OfficeBridge
add action=drop chain=forward comment="Prevent Hotspot routing to Private IP ranges" dst-address-list=PrivateIPs in-interface=hs-bridge
add action=accept chain=input comment="Preset: Accept ICMP" protocol=icmp
add action=accept chain=input comment="Preset: Accept WinBox (Trusted IP's)" dst-port=8291 protocol=tcp src-address-list=Trusted
add action=accept chain=input comment="Preset: Accept WinBox (Mgmt)" dst-port=8291 in-interface-list=Management protocol=tcp
add action=accept chain=input comment="Routing Preset: BFD" dst-port=3784,3785 protocol=udp
add action=accept chain=input comment="Routing Preset: OSPF" protocol=ospf
add action=accept chain=input comment="Routing Preset: MPLS / LDP" dst-port=646,711 protocol=tcp
add action=accept chain=input comment="Routing Preset: MPLS / LDP" dst-port=646 protocol=udp
add action=accept chain=input comment="Routing Preset: BGP" disabled=yes dst-port=179 protocol=tcp
add action=drop chain=input comment="Preset: Drop outside DNS requests" dst-port=53 in-interface-list=InternetFacing protocol=udp
add action=drop chain=input comment="Preset: Drop Invalid" connection-state=invalid
add action=drop chain=forward comment="Preset: Drop Invalid" connection-state=invalid disabled=yes
add action=drop chain=input comment="Preset: Drop all from InternetFacing" in-interface-list=InternetFacing
add action=passthrough chain=unused-hs-chain comment="place hotspot rules here" disabled=yes
add action=drop chain=HS-PPPoE comment="Prevent HS PPPoE Clients routing to Private IP ranges" dst-address-list=PrivateIPs
/ip firewall mangle
add action=change-ttl chain=prerouting comment="Increase TTL for hotspot to hide '2' routers" in-interface=hs-bridge new-ttl=increment:2 passthrough=yes
add action=change-ttl chain=prerouting comment="Increase TTL for office to hide '3' routers" in-interface=OfficeBridge new-ttl=increment:3 passthrough=yes src-address-list=!PrivateIPs
add action=mark-connection chain=input comment="Mark incoming on CommsN" in-interface="ether2.200 - SME Mgmt Network" new-connection-mark=CommsN_in_c passthrough=yes
add action=mark-connection chain=input comment="Mark incoming on CommsB" in-interface=pppoe-WAN new-connection-mark=CommsB_in_c passthrough=yes
add action=mark-connection chain=prerouting comment="Mark Hotspot -> CommsN" dst-address-list=!PrivateIPs in-interface=hs-bridge new-connection-mark=CommsN_c passthrough=yes
add action=mark-connection chain=prerouting comment="Mark PPP Clients -> CommsN" in-interface-list=CommsN_Out new-connection-mark=CommsN_c passthrough=yes
add action=mark-connection chain=prerouting comment="Mark Office -> CommsB" in-interface=OfficeBridge new-connection-mark=CommsB_c passthrough=yes src-address-list=!PrivateIPs
add action=mark-packet chain=prerouting comment="Mark CommsN packets" connection-mark=CommsN_c new-packet-mark=CommsN_p passthrough=yes
add action=mark-packet chain=prerouting comment="Mark CommsB packets" connection-mark=CommsB_c new-packet-mark=CommsB_p passthrough=yes
add action=mark-routing chain=prerouting comment="Send out GCOM" in-interface-list=!InternetFacing new-routing-mark=GCOM packet-mark=CommsN_p passthrough=no
add action=mark-routing chain=prerouting comment="Send out metered WAN (CommsB)" in-interface-list=!InternetFacing new-routing-mark=HotelWAN packet-mark=CommsB_p passthrough=no
/ip firewall nat
add action=masquerade chain=srcnat comment="masquerade hotspot network" src-address=10.0.0.0/21 to-addresses=172.16.51.3
add action=masquerade chain=srcnat comment="Masquerade to DSLAM" dst-address=10.11.104.2
add action=masquerade chain=srcnat comment="Preset: Masquerade Management" src-address=10.245.1.0/24
add action=masquerade chain=srcnat comment="Preset: NAT out internet facing interfaces (non private)" dst-address-list=!PrivateIPs out-interface-list=InternetFacing
add action=dst-nat chain=dstnat comment="DstNat to DSLAM (from trusted IP's only)" dst-port=23 protocol=tcp src-address-list=Trusted to-addresses=10.11.104.2
/mpls interface
set [ find default=yes ] mpls-mtu=1538
/mpls ldp
set enabled=yes loop-detect=yes lsr-id=172.16.201.10 transport-address=172.16.201.10
/mpls ldp interface
add interface=ether1 transport-address=172.16.201.10
add interface="ether2.52 - Downlink to E" transport-address=172.16.201.10
/routing ospf interface
add network-type=broadcast passive=yes
add authentication=md5 authentication-key=*** dead-interval=6s hello-interval=2s interface=ether1 network-type=nbma use-bfd=yes
add dead-interval=10s hello-interval=3s interface="ether2.52 - Downlink to E" network-type=point-to-point use-bfd=yes
/routing ospf nbma-neighbor
add address=172.16.23.33 poll-interval=5s priority=2
/routing ospf network
add area=backbone network=172.16.201.10/32
add area=backbone network=172.16.23.32/28
add area=backbone network=172.16.52.0/27

Edit: The packets are considered invalid coming ‘in’ from ether2.52. I can block invalid packets everywhere except that interface

5

Have you tried to set ****

log=yes log-prefix=weird-rule

at the drop rule for a couple of seconds and see what packets in particular (source, destination, other info depending on packet type) the rule actually drops? A packet is marked as

invalid

by connection tracking if it does not fit to the current state of a connection, such as a

SYN, ACK

packet when

SYN

packet was not seen before.

As your diagram shows a linear topology, I can only imagine things like each direction of a connection going through a different VLAN or MPLS tunnel, e.g. because packets are src-nat’ed in one direction and this along with routing in another element makes them come back via a different interface.