Hi!
I faced a problem I can not understand, so I need some help 
New router, mikrotik RB3011, BGP+OSPF, some vlans:
vlan_manage - 100.69.0.181/16
vlan_routing - 100.70.67.181/24 - OSPF backbone
vlan_bgp* - links to BGP PEERS (internal and confederation external)
Tried to add to mrtg system, invoke “cfgmaker 100.69.0.181” - no answer. Standard situation - UDP response is coming with different source IP than expected. Quick “tcpdump” approves assumption, but the source IP is not expected at all - it belongs to a router, which should be apart from the route. Some more investigation revealed the strange problem (take a look at the picture):
SNMP request path is X.Y.5.44 (mrtg host) → twp → tw4 (as expected)
SNMP answer is tw4 (source 100.69.0.181) → ras → mrtg host
Since ras (rb750) does NAT I was able to spot the strange reverse path - the answer came with source IP X.Y.5.49
All routing info seems OK:
[admin@tw4] > ip route print detail where X.Y.5.44 in dst-address
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit
0 ADb dst-address=0.0.0.0/0 gateway=100.70.61.178 gateway-status=100.70.61.178 reachable via vlan_bgp1 distance=200 scope=40 target-scope=30 bgp-local-pref=999 bgp-origin=igp received-from=twp
1 Db dst-address=0.0.0.0/0 gateway=100.70.63.108 gateway-status=100.70.63.108 reachable via vlan_bgp3_d2 distance=20 scope=40 target-scope=10 bgp-as-path="(65001)" bgp-local-pref=100 bgp-origin=igp received-from=gate1
2 ADo dst-address=X.Y.0.0/19 gateway=100.70.67.178,100.70.67.180 gateway-status=100.70.67.178 reachable via vlan_routing,100.70.67.180 reachable via vlan_routing distance=110 scope=20 target-scope=10 ospf-metric=10 ospf-type=external-type-2
3 ADb dst-address=X.Y.0.0/20 gateway=100.70.62.178 gateway-status=100.70.62.178 reachable via vlan_bgp2 distance=200 scope=40 target-scope=30 bgp-as-path="(65001)" bgp-local-pref=400 bgp-med=0 bgp-origin=igp bgp-communities=0:10,ASN:40,ASN:ASN received-from=twp
4 ADbU dst-address=X.Y.5.0/24 type=unreachable distance=20 bgp-origin=incomplete
5 Db dst-address=X.Y.5.0/24 gateway=100.70.62.178 gateway-status=100.70.62.178 reachable via vlan_bgp2 distance=200 scope=40 target-scope=30 bgp-local-pref=100 bgp-med=0 bgp-origin=igp bgp-communities=ASN:100,ASN:ASN received-from=twp
6 ADo dst-address=X.Y.5.40/29 gateway=100.70.67.180 gateway-status=100.70.67.180 reachable via vlan_routing distance=110 scope=20 target-scope=10 ospf-metric=10 ospf-type=external-type-2
7 ADo dst-address=X.Y.5.44/32 gateway=100.70.67.178 gateway-status=100.70.67.178 reachable via vlan_routing distance=110 scope=20 target-scope=10 ospf-metric=10 ospf-type=external-type-2
A “quick sniffer” approves packets come via twp but go out via ras:
3C:4A:92:7C:3E:D5 - twp
E4:8D:8C:79:DC:0E - tw4
00:0C:42:EF:DB:43 - ras
[admin@tw4] > tool sniffer quick interface=ether2 port=161
INTERFACE TIME NUM DIR SRC-MAC DST-MAC VLAN SRC-ADDRESS DST-ADDRESS PROTOCOL SIZE CPU FP
ether2 6.411 1 <- 3C:4A:92:7C:3E:D5 E4:8D:8C:79:DC:0E 3566 X.Y.5.44:46024 100.69.0.181:161 (snmp) ip:udp 87 1 no
ether2 6.411 2 -> E4:8D:8C:79:DC:0E 00:0C:42:EF:DB:43 3566 100.69.0.181:161 (snmp) X.Y.5.44:46024 ip:udp 109 0 no
The very strange is the fact EVERYTHING else follows the normal path:
ping to:
[admin@tw4] > tool sniffer quick interface=ether2 ip-protocol=icmp
INTERFACE TIME NUM DIR SRC-MAC DST-MAC VLAN SRC-ADDRESS DST-ADDRESS PROTOCOL SIZE CPU FP
ether2 18.478 1 <- 3C:4A:92:7C:3E:D5 E4:8D:8C:79:DC:0E 3566 X.Y.5.44 100.69.0.181 ip:icmp 102 1 no
ether2 18.478 2 -> E4:8D:8C:79:DC:0E 3C:4A:92:7C:3E:D5 3567 100.69.0.181 X.Y.5.44 ip:icmp 102 0 no
ping from:
[admin@tw4] > tool sniffer quick interface=ether2 ip-protocol=icmp
INTERFACE TIME NUM DIR SRC-MAC DST-MAC VLAN SRC-ADDRESS DST-ADDRESS PROTOCOL SIZE CPU FP
ether2 13.011 1 -> E4:8D:8C:79:DC:0E 3C:4A:92:7C:3E:D5 3567 100.70.67.181 X.Y.5.44 ip:icmp 74 0 no
ether2 13.011 2 <- 3C:4A:92:7C:3E:D5 E4:8D:8C:79:DC:0E 3567 X.Y.5.44 100.70.67.181 ip:icmp 74 0 no
traceroute:
[admin@tw4] > tool traceroute X.Y.5.44
# ADDRESS LOSS SENT LAST AVG BEST WORST STD-DEV STATUS
1 100.70.67.178 0% 10 0.2ms 0.1 0.1 0.2 0
2 X.Y.5.44 0% 10 0.4ms 0.4 0.3 0.5 0.1
The only packets going via ras are the snmp responses, nothing else.
There is no policy routing (/ip route rule)
I can not imagine - WHY these packets are being sent to the “wrong” gateway?
P.S. I was wrong, snmp responses are not the only packets, I enabled the remote DNS requests and they go the same path, seems all UDP responses use this path. But only responses. When connection is initiated by tw4 it goes via twp.
