Routes learned by BGP in VRF-Lite installed and active, but no routing beyond direcly connected subnets

dscp46 · Fri Jun 16, 2023 9:08 am

Hi there,

I've had a few issues with a RouterOS v7 setup (re-tested with v7.10 before posting), where I'm able to set up a BGP session within a VRF lite.

The session is established, both routers exchange prefixes, routes are installed and marked as active within the VRF, but I'm getting "22 (Invalid argument)" when trying to ping networks that are not directly connected in that VRF.

BGP session

[f4hof@e1-lyo69] > /routing/bgp/session/ print
Flags: E - established
0 E name="bgp_dc_69-1"
remote.address=44.168.37.207@hamnet .as=4220800022 .id=44.168.253.197 .capabilities=mp,rr,em,gr,as4,dyn,ap,err,llgr,fqdn .messages=1263
.bytes=193374 .gr-time=120 .eor=ip
local.role=ibgp .address=44.168.37.206@hamnet .as=4220800022 .id=44.168.37.206 .capabilities=mp,rr,gr,as4 .messages=10 .bytes=241 .eor=""
output.procid=20
input.procid=20 .filter=Hamnet_In ibgp
routing-table=hamnet multihop=yes hold-time=3m keepalive-time=1m uptime=8m38s120ms last-started=2023-06-16 07:41:22 prefix-count=2512

Routing table output

[f4hof@e1-lyo69] > /ip/route/print detail where routing-table=hamnet
Flags: D - dynamic; X - disabled, I - inactive, A - active;
c - connect, s - static, r - rip, b - bgp, o - ospf, d - dhcp, v - vpn, m - modem, y - bgp-mpls-vpn; H - hw-offloaded; + - ecmp
DAb dst-address=44.7.0.0/20 routing-table=hamnet gateway=44.168.37.207@hamnet immediate-gw=44.168.37.207%ipip-tunnel1 distance=201 scope=40
target-scope=30 suppress-hw-offload=yes

DAb dst-address=44.31.84.0/24 routing-table=hamnet gateway=44.168.37.207@hamnet immediate-gw=44.168.37.207%ipip-tunnel1 distance=201 scope=40
target-scope=30 suppress-hw-offload=yes

DAb dst-address=44.31.192.1/32 routing-table=hamnet gateway=44.168.37.207@hamnet immediate-gw=44.168.37.207%ipip-tunnel1 distance=201 scope=40
target-scope=30 suppress-hw-offload=yes

DAb dst-address=44.31.192.2/32 routing-table=hamnet gateway=44.168.37.207@hamnet immediate-gw=44.168.37.207%ipip-tunnel1 distance=201 scope=40
target-scope=30 suppress-hw-offload=yes

DAb dst-address=44.31.192.3/32 routing-table=hamnet gateway=44.168.37.207@hamnet immediate-gw=44.168.37.207%ipip-tunnel1 distance=201 scope=40
target-scope=30 suppress-hw-offload=yes

DAb dst-address=44.31.192.4/32 routing-table=hamnet gateway=44.168.37.207@hamnet immediate-gw=44.168.37.207%ipip-tunnel1 distance=201 scope=40
target-scope=30 suppress-hw-offload=yes

DAb dst-address=44.31.192.6/32 routing-table=hamnet gateway=44.168.37.207@hamnet immediate-gw=44.168.37.207%ipip-tunnel1 distance=201 scope=40
target-scope=30 suppress-hw-offload=yes
(output cut here, because there are over 2400 routes installed in that VRF)

Ping output

[f4hof@e1-lyo69] > ping 44.7.0.1 vrf=hamnet
SEQ HOST SIZE TTL TIME STATUS
0 22 (Invalid argument)
1 22 (Invalid argument)
2 22 (Invalid argument)
3 22 (Invalid argument)
4 22 (Invalid argument)
5 22 (Invalid argument)
sent=6 received=0 packet-loss=100%

You'll find attached the running configuration.

e1-lyo69_20230616_wo_senst.txt

As recommended by a friend who is more seasoned with RouterOS, I've tried to disable and enable back the IP adresses to no avail.
Do you have any idea on why I'm having this issue?

The equipment on the far side is a VyOS 1.4 appliance.

Details on the BGP session, from the other router:

> show ip bgp vrf hamnet neighbors 44.168.37.206
BGP neighbor is 44.168.37.206, remote AS 4220800022, local AS 4220800022, internal link
Local Role: undefined
Remote Role: undefined
Description: Radioclub via tunnel
BGP version 4, remote router ID 44.168.37.206, local router ID 44.168.253.197
BGP state = Established, up for 01:02:52
Last read 00:00:52, Last write 00:00:07
Hold time is 180 seconds, keepalive interval is 60 seconds
Configured hold time is 180 seconds, keepalive interval is 60 seconds
Configured conditional advertisements interval is 60 seconds
Neighbor capabilities:
4 Byte AS: advertised and received
Extended Message: advertised
AddPath:
IPv4 Unicast: RX advertised
Dynamic: advertised
Long-lived Graceful Restart: advertised
Route refresh: advertised and received(new)
Enhanced Route Refresh: advertised
Address Family IPv4 Unicast: advertised and received
Hostname Capability: advertised (name: vyos,domain name: n/a) not received
Graceful Restart Capability: advertised and received
Remote Restart timer is 0 seconds
Address families by peer:
none
Graceful restart information:
End-of-RIB send: IPv4 Unicast
End-of-RIB received:
Local GR Mode: Helper*
Remote GR Mode: Helper
R bit: False
N bit: False
Timers:
Configured Restart Time(sec): 120
Received Restart Time(sec): 0
IPv4 Unicast:
F bit: False
End-of-RIB sent: Yes
End-of-RIB sent after update: No
End-of-RIB received: No
Timers:
Configured Stale Path Time(sec): 360
Message statistics:
Inq depth is 0
Outq depth is 0
Sent Rcvd
Opens: 8 8
Notifications: 3 0
Updates: 8593129 26
Keepalives: 31599 31599
Route Refresh: 1 2
Capability: 0 0
Total: 8624740 31635
Minimum time between advertisement runs is 0 seconds
Update source is 44.168.37.207
For address family: IPv4 Unicast
Update group 13, subgroup 11
Packet Queue length 0
Inbound soft reconfiguration allowed
NEXT_HOP is always this router
Community attribute sent to this neighbor(all)
Inbound path policy configured
Outbound path policy configured
Incoming update prefix filter list is *Hamnet
Outgoing update prefix filter list is *Hamnet
4 accepted prefixes
Connections established 8; dropped 7
Last reset 01:05:26, Peer closed the session
Internal BGP neighbor may be up to 255 hops away.
Local host: 44.168.37.207, Local port: 37981
Foreign host: 44.168.37.206, Foreign port: 179
Nexthop: 44.168.37.207
Nexthop global: fe80::f1cf:60ff:fe80:b060
Nexthop local: fe80::f1cf:60ff:fe80:b060
BGP connection: shared network
BGP Connect Retry Timer in Seconds: 120
Estimated round trip time: 4 ms
Read thread: on Write thread: on FD used: 31

Received routes on the far side:

BGP table version is 27562501, local router ID is 44.168.253.197, vrf id 5
Default local pref 100, local AS 4220800022
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
*> 44.168.37.0/27 44.168.37.206 100 8192 i
*> 44.168.37.206/31 44.168.37.206 100 8192 i
*> 44.168.37.216/29 44.168.37.206 100 8192 i
*> 44.168.37.248/29 44.168.37.206 100 8192 i
*> 82.238.77.78/32 44.168.37.206 100 8192 i
Total number of prefixes 5 (1 filtered)

Don't pay attention to this route in 82.238.77.78/32, I've used it for a test, and haven't removed it yet.

(edit: add information from the remote end)

Kind regards,

wiseroute · Fri Jun 16, 2023 2:08 pm

hello,

[f4hof@e1-lyo69] > ping 44.7.0.1 vrf=hamnet
SEQ HOST SIZE TTL TIME STATUS
0 22 (Invalid argument)

there are not much info which source which interface did you ping 0.1 from? and is there really 0.1 host up?

are local and remote.role don't mean anything to your config?

++++ edit

and could you give us some clue about your topology - it is easier to interpret your diagram than reading those config. my eyes can't no longer see long config.

dscp46 · Sat Jun 24, 2023 7:17 pm

Hi,

Sorry for the late reply, currently, my two routers are interconnected by the interface ipip-tunnel1.
My local equipment (e1-lyo69) is set up with 44.168.37.206/31, the remote one (e1-dc69) is on 44.168.37.207.

You'll find below the overall topology of relevant equipments on the VRF of our interest.

hamnet-setup.png

e1-lyo69 and e1-dc69 are part of the same AS, hnetgw is part of a remote one. local.role has been specified because it was required to set up the bgp connection. Honestly, I'm not thinking this is part of the equation because routes are installed and active, as shown in my previous post.

My remote equipment has connectivity to 44.7.0.1

f4hof@e1-dc69:~$ ping 44.7.0.1 vrf Hamnet
PING 44.7.0.1 (44.7.0.1) 56(84) bytes of data.
64 bytes from 44.7.0.1: icmp_seq=1 ttl=62 time=27.4 ms
64 bytes from 44.7.0.1: icmp_seq=2 ttl=62 time=27.2 ms
64 bytes from 44.7.0.1: icmp_seq=3 ttl=62 time=27.9 ms

Given the routing table content, when I'm trying to ping 44.7.0.1, the source address should be 44.168.37.206.

Regardless on the source address, I'm appalled by the fact the ping returns "22 (Invalid argument)" instead of at least a timeout. When attempting to ping a directly connected network, pings behaves properly:

[f4hof@e1-lyo69] > ping 44.168.37.206 vrf=hamnet
SEQ HOST SIZE TTL TIME STATUS
0 44.168.37.206 56 64 479us
1 44.168.37.206 56 64 371us
2 44.168.37.206 56 64 373us
3 44.168.37.206 56 64 376us
sent=4 received=4 packet-loss=0% min-rtt=371us avg-rtt=399us max-rtt=479us

It it only was a ping issue, I wouldn't be bothered, and wouldn't have written the present thread. Despite having opened the appropriate firewall rules, my mikrotik router doesn't reply to pings from networks that are not directly connected, despite all routers having valid and active routes being installed.

I'm really feeling that I'm missing something stupid and obvious.

dscp46 · Sat Jun 24, 2023 9:42 pm

Problem solved.

Thanks to another thread, I've just found out on https://help.mikrotik.com/docs/display/ ... l+Overview that /31 routing isn't supported on ROS 7.x.

Routes learned by BGP in VRF-Lite installed and active, but no routing beyond direcly connected subnets [SOLVED]

Routes learned by BGP in VRF-Lite installed and active, but no routing beyond direcly connected subnets

Re: Routes learned by BGP in VRF-Lite installed and active, but no routing beyond direcly connected subnets

Re: Routes learned by BGP in VRF-Lite installed and active, but no routing beyond direcly connected subnets

Re: Routes learned by BGP in VRF-Lite installed and active, but no routing beyond direcly connected subnets [SOLVED]

Who is online