Packets with wrong src-addr leaking out of Public interface

Hi.

Packets with src-addr from the internal network are leaking out of the Public (ADSL1) interface. This was determined with the internal tool “Packet Sniffer” on the ADSL1 interface. RouterOS 3.19 mipsbe.

Routing table:

/ip route
add check-gateway=ping comment="Route All ToADSL2" distance=1 dst-address=0.0.0.0/0 gateway=ADSL2 routing-mark=ToADSL2
add check-gateway=ping comment="Route All else through ADSL1" distance=2 dst-address=0.0.0.0/0 gateway=ADSL1
add comment="failover to ADSL2" distance=3 dst-address=0.0.0.0/0 gateway=ADSL2

Nat:

/ip firewall nat
add action=masquerade chain=srcnat src-address=10.0.10.0/24

Mangle:

/ip firewall mangle
add action=mark-routing chain=prerouting dst-address-type=!local in-interface=Bridge-WDS new-routing-mark=ToADSL2 passthrough=yes src-address=10.0.10.9
add action=mark-connection chain=input in-interface=ADSL2 new-connection-mark=ADSL2Con2R passthrough=yes
add action=mark-routing chain=output connection-mark=ADSL2Con2R new-routing-mark=ToADSL2 passthrough=yes

http://www.mediafire.com/imageview.php?quickkey=ygtytwzywri&thumb=4

Two Screenshots - Click Next/Prev above the thumbnail to switch between them or click on either of the smaller thumbnails…

was this working pre 3.19?

my guess is you need more routes in your alternate routing table, and you need more mark-routings in other chains. I believe you should have the connected routes from the main table duplicated into your alternate tables. Use the routers IP as the gateway on those entries.

Do you have /ip route rules? My guess is the alternate routing table is queried, an entry doesnt match, so it falls back to the main table.

also, connection tracking seems to remember the source ip of a connection and use it even on alternate gateways. A new NAT association only happens on the first setup of the connection, so if you change gateways in the middle of the connection it will still use the original masq ip. Many people don’t get that part of it. There is also another thread right now about deleting connection entries, this is exactly why that is useful.

My guess is this will be the same with 3.18. But I consider this a serious software flaw. Why put more routing marks? It will become an unmanageable mess. It should work just fine like this, only it does not.

I believe you should have the connected routes from the main table duplicated into your alternate tables. Use the routers IP as the gateway on those entries.

Can you explain this in configuration, /ip route add … ? /ip route rules add ?

Currently there are no ip route rules, just ip route.

My guess is the alternate routing table is queried, an entry doesnt match, so it falls back to the main table.

I don’t think so. But you can contact me for login details to see for yourself.

A new NAT association only happens on the first setup of the connection, so if you change gateways in the middle of the connection it will still use the original masq ip. Many people don’t get that part of it. There is also another thread right now about deleting connection entries, this is exactly why that is useful.

I rebooted the RouterBoard. All connections must have been newly setup. And even if they have not, there is no reason for those packets being sent out that port.

Will write soon. Regards.

Why does not the NAT change the IP when the output route is changed? This is a flaw, no?

Where is this thread? This one ? Firewall Connection Remove seems broken again v3.15, v3.16

actually, no. change of IP will broke the connection, and what if you just want to use backup interface with the same NAT address?

but breaking all conections via interface in case of that interface disconnection would be a nice feature…

So what we are asking for here is for a “drop established connections on routes that are no longer valid” check-box? Let’s make it clear that now myself, the starter of the above noted thread, changeip and Chupaka now clearly point out that this is a missing feature, so MikroTik can not ignore this.

Its a feature. Any operating system using conntrack would so it the same way. At least with Mikrotik we have the option to script things : )

The router is just doing what you tell it. If you aren’t marking connections, marking packets, and then marking routing you’re going to miss things. A connection is only the beginning of a connection, not everything included within it.

Pretend your a packet, and you get thrown into the ToADSL2 table. You cannot see anything but whats in that table, which means you only have a way to talk to your pppoe modem. Okay, suppose you need to talk back to the network you came from for any number of reasons, including ICMP messages, etc. Oh, that’s not in your routing table, so you fall thru to the main table. Routing tables work just like firewall chains, if something isn’t in the routing table your in, you fall back to the main table. You use the /ip route rules chain to order these things and possibly prevent the fallback to main.

A side note, your nat masq rules should be specifying an output interface, one for each. You don’t want to masquerade all traffic no matter what if it has that source range, only when it leaves and interface.

Another side note: check-gateway=ping on a non-ip based route entry (interface routing) is probably not going to work, since it has no idea what to ping. It might work, but I’m guessing its not reliable.

Your mangle rules are only mark routing for input chain and output chain, not prerouting or forwarding chain. You have src-address=10.0.10.9 on the prerouting mangle chain.

In the end, I highly doubt the packets are leaking out because of some bug, it’s most likely improper setup. If you post more details we will help you fix it.

Not to a bunch of MT customers, including myself, trying to accomplish load balancing with fail-over. To us it’s a flaw. So at the end of the day, it must be changed from being a showstopping-flaw-feature to being a … useful feature. Or at least a way to compensate for it must be CLEAR and straightforward to customers.

The router is just doing what you tell it. If you aren’t marking connections, marking packets, and then marking routing you’re going to miss things. A connection is only the beginning of a connection, not everything included within it.

I thought the marking I am trying to use is according to MikroTik RouterOS manual. It will be just the beginning when the connection-state=new matcher is used. Or do you mean that it misses “related” connections?

Pretend your a packet, and you get thrown into the ToADSL2 table.

There is no such routing table in this entire topic, unless you are saying that the routing-mark === “a routing table” ?

You cannot see anything but whats in that table, which means you only have a way to talk to your pppoe modem. Okay, suppose you need to talk back to the network you came from for any number of reasons, including ICMP messages, etc.

ICMP messages are originating from the router itself. Packet do not talk to anything. Routers and hosts do, connections do. Maybe you mean that when an ICMP message is generated by the router, it must also be directed accordingly, taking into account, the route the packet that caused the generation of the ICMP message, is traveling? Can this be accomplished with connection-state=related? And how? (by the way, connection-state is found only in http://www.mikrotik.com/testdocs/ros/3.0/qos/mangle.php )

Oh, that’s not in your routing table, so you fall thru to the main table. Routing tables work just like firewall chains, if something isn’t in the routing table your in, you fall back to the main table. You use the /ip route rules chain to order these things and possibly prevent the fallback to main.

Can you please point me to an thorough example of /ip route rules ?

A side note, your nat masq rules should be specifying an output interface, one for each. You don’t want to masquerade all traffic no matter what if it has that source range, only when it leaves and interface.

src-nat is in postrouting, where the routing decision has already been made, so it should correctly assign addresses.

Another side note: check-gateway=ping on a non-ip based route entry (interface routing) is probably not going to work, since it has no idea what to ping. It might work, but I’m guessing its not reliable.

I’m using only arp, I think it’s faster.

Your mangle rules are only mark routing for input chain and output chain, not prerouting or forwarding chain. You have src-address=10.0.10.9 on the prerouting mangle chain.

That is correct. And it should be correct according to manual and facility order of RouterOS.

In the end, I highly doubt the packets are leaking out because of some bug, it’s most likely improper setup. If you post more details we will help you fix it.

It is not a bug. It is a major mistake in software design. And I am currently trying my best, with your help, to work around it.

Thank you.

Regards.

I find this

You can use policy routing even if you use masquerading on your private networks. The source address will be the same as it is in the local network. In previous versions of RouterOS the source address changed to 0.0.0.0

here http://www.mikrotik.com/testdocs/ros/3.0/ip/route.php And I find it extremely difficult to connect the pieces with so much missing. I (and other load balancing mikrotik victims) would be good if Anyone Could Help us out with all this.

Oh my god! This is completely unexplained! I am going to throw myself out the window.

OK enough with the comedy.

Can I create independent routing tables and direct one traffic (ToADSL1) to routing table “ToADSL1Routing” and the ToADSL2 traffic to “ToADSL2Routing” ? Where I would have different default gateways? If yes, how will I add fail-over? :slight_smile: And how can I solve all problems, like ICMP messages, related connections, connections coming from the internet to the router itself, packet leakage… etc. ?

This is a mistake, right? It should be 10.0.0.144/32, right?

So with this example, I can leak out my packets on purpose? :sunglasses: Cool.

Is policy routing with load-balancing possible with RouterOS?

Oh come on, this is taking way too much time and energy.

Wait, wait? Or do they mean, that I can use policy routing even when I use NAT, and that “the source IP would be the same as in the internal net” means that’s for the config only and the actual outgoing packets will be correctly NATed?

Answer Answer Answer.

Yes. Result determined by laboratory experimentation and analysis with simulation of an internet host, an internet gateway, a NAT host, and two internal-lan hosts. RouterOS x86 v4.0Beta1. No problems seen. Except in documentation (missing information).

So now the problem is that when one route fails, and the next possible route takes over, the router sends the packets to the new gateway with the old source ddress, until the connection is expired in connection tracking. I would rather have the connection dropped at that point, so that it would be re-established correctly. How to do this? :slight_smile:

Topic continues here: http://forum.mikrotik.com/viewtopic.php?f=2&t=29464 with possible solutions.