BGP v7.10.2 readvertisement of ebgp learned routes to ebgp peers stopped working in v7.10

netzwerghh · Thu Aug 03, 2023 2:07 pm

We have some CCR2216 with multiple external BGP feeds (eBGP). Our CCRs are exchanging learned routes via Bird route reflectors (iBGP). The CCRs are readvertising all learned routes to the route reflectors and are getting the routes from the other CCRs from the route reflectors. This is working fine. I can see all active routes from all our external BGP feeds an all CCRs.
We also have some external BGP peers which are downstream from us and want to get our full BGP table. This also worked fine until v7.9. After update to 7.10 the CCR to which our customer is connected only readvertises the active routes learned from the route reflectors and our own routes to our customer not routes learned from upstream on the same CCR. Did something change in the behavior in v7.10?

The examples below are advertisement to the RIPE route collectors at AMS-IX. So no "private" data one can't find in PeeringDB or bgp.he.net.

Example not being readvertised (learned from peer on same router):

[admin@XXX] > /routing/route/print detail where dst-address=2.255.248.0/21
Flags: X - disabled, F - filtered, U - unreachable, A - active; 
c - connect, s - static, r - rip, b - bgp, o - ospf, d - dhcp, v - vpn, m - modem, a - ldp-address, l - ldp-mapping, g - slaac, y - bgp-mpls-vpn; 
H - hw-offloaded; + - ecmp, B - blackhole 
 Ab   afi=ip4 contribution=active dst-address=2.255.248.0/21 routing-table=main gateway=62.115.XXX.140 immediate-gw=62.115.XXX.140%vlan-telia-iptransit 
       distance=20 scope=40 target-scope=10 belongs-to="bgp-IP-62.115.XXX.140" 
       bgp.peer-cache-id=*2C00001 .aggregator="1299:2.255.253.117" .as-path="1299" .communities=1299:30000,1299:37000 .local-pref=100 
       .atomic-aggregate=yes .origin=igp 
       debug.fwp-ptr=0x20311060
[admin@XXX] > /routing/bgp/advertisements/print where peer=NETZWERGE.RR01.v4-1  and dst=2.255.248.0/21    
 0 peer=NETZWERGE.RR01.v4-1 dst=2.255.248.0/21 afi=ip local-pref=100 nexthop=62.115.XXX.140 origin=0 as-path=sequence 1299 communities=1299:30000,1299:37000 
   atomic-aggregate=yes aggregator="1299:2.255.253.117" 
[admin@XXX] > /routing/bgp/advertisements/print where peer=ams-ix.ripe-rs.v4-1 and dst=2.255.248.0/21

[admin@XXX] >

Example being readvertised (active route learned from RR):

[admin@XXX] > /routing/route/print detail where dst-address=129.233.128.0/17
Flags: X - disabled, F - filtered, U - unreachable, A - active; 
c - connect, s - static, r - rip, b - bgp, o - ospf, d - dhcp, v - vpn, m - modem, a - ldp-address, l - ldp-mapping, g - slaac, y - bgp-mpls-vpn; 
H - hw-offloaded; + - ecmp, B - blackhole 
  b   afi=ip4 contribution=candidate dst-address=129.233.128.0/17 routing-table=main gateway=62.115.XXX.140 
       immediate-gw=62.115.XXX.140%vlan-telia-iptransit distance=20 scope=40 target-scope=10 belongs-to="bgp-IP-62.115.XXX.140" 
       bgp.peer-cache-id=*2C00001 .as-path="1299,553,5501" .communities=1299:30000 .local-pref=100 .atomic-aggregate=no .origin=igp 
       debug.fwp-ptr=0x20311060 

 Ab   afi=ip4 contribution=active dst-address=129.233.128.0/17 routing-table=main gateway=80.81.194.106 immediate-gw=194.39.187.XXX%bonding1 distance=200 
       scope=40 target-scope=30 belongs-to="bgp-IP-194.39.187.XXX" 
       bgp.peer-cache-id=*2C00006 .as-path="553,5501" 
       .communities=64800:42001,65103:276,64800:41002,64800:40001,64800:49999,553:111,65104:150,553:1200,65102:1000,65101:1006 
       .large-communities=6695:1000:1 .originator-id=194.39.187.XXX .local-pref=300 .med=225 .atomic-aggregate=yes .origin=egp 
       debug.fwp-ptr=0x2030A6C0 

  b   afi=ip4 contribution=candidate dst-address=129.233.128.0/17 routing-table=main gateway=80.81.194.106 immediate-gw=194.39.187.XXX%bonding1 
       distance=200 scope=40 target-scope=30 belongs-to="bgp-IP-194.39.187.XXX" 
       bgp.peer-cache-id=*2C00003 .as-path="553,5501" 
       .communities=64800:42001,65103:276,64800:41002,64800:40001,64800:49999,553:111,65104:150,553:1200,65102:1000,65101:1006 
       .large-communities=6695:1000:1 .originator-id=194.39.187.XXX .local-pref=300 .med=225 .atomic-aggregate=yes .origin=egp 
       debug.fwp-ptr=0x2030A6C0 
[admin@ICHAM-RTR02] > /routing/bgp/advertisements/print where peer=ams-ix.ripe-rs.v4-1 and dst=129.233.128.0/17
 0 peer=ams-ix.ripe-rs.v4-1 dst=129.233.128.0/17 afi=ip nexthop=80.249.211.237 origin=1 as-path=sequence 199938 553 5501 
   communities=64800:42001,65103:276,64800:41002,64800:40001,64800:49999,553:111,65104:150,553:1200,65102:1000,65101:1006 large-communities=6695:1000:1 
   atomic-aggregate=yes originator-id=194.39.187.2 cluster-list=194.39.187.6

BGP Peers:

[admin@XXX] > /routing/bgp/connection/print where name=ams-ix.ripe-rs.v4  
Flags: D - dynamic, X - disabled, I - inactive 
 0   name="ams-ix.ripe-rs.v4" 
     remote.address=80.249.208.69/32 .port=179 .as=12654 
     local.address=80.249.211.237 .role=ebgp 
     connect=no listen=yes routing-table=main router-id=194.39.187.3 templates=default as=199938 address-families=ip cisco-vpls-nlri-len-fmt=auto-bits 
     output.affinity=input .filter-chain=FULLVIEW-OUT .network=bgp-networks 
     input.affinity=alone .filter=AMS-IX-IN .limit-process-routes-ipv4=50
[admin@XXX] > /routing/bgp/connection/print where name=NETZWERGE.RR01.v4 
Flags: D - dynamic, X - disabled, I - inactive 
 1   name="NETZWERGE.RR01.v4" 
     remote.address=194.39.187.XXX/32 .port=179 .as=199938 
     local.address=194.39.187.3 .role=ibgp-rr-client 
     connect=yes listen=yes routing-table=main router-id=194.39.187.3 templates=default as=199938 address-families=ip cisco-vpls-nlri-len-fmt=auto-bits 
     output.affinity=input 
     input.affinity=alone
[admin@XXX] > /routing/bgp/connection/print where name=telia.v4 
Flags: D - dynamic, X - disabled, I - inactive 
 2   name="telia.v4" 
     remote.address=62.115.XXX.140/32 .port=179 .as=1299 
     local.address=62.115.XXX.141 .role=ebgp 
     tcp-md5-key="XXXXX" connect=yes listen=yes routing-table=main router-id=194.39.187.3 templates=default as=199938 address-families=ip 
     cisco-vpls-nlri-len-fmt=auto-bits 
     output.affinity=input .filter-chain=TELIA-OUT .network=bgp-networks 
     input.affinity=alone .filter=TELIA-IN

Route-Filters

[admin@XXX] > /routing/filter/rule/print where chain=FULLVIEW-OUT 
Flags: X - disabled, I - inactive 
 0   chain=FULLVIEW-OUT rule="jump rfc_5735_discard;" 

 1   chain=FULLVIEW-OUT rule="jump DISCARD-TOO-SPECIFIC;" 

 2   chain=FULLVIEW-OUT rule="if (dst in 2a01:55e0::/29 && dst-len in 31-128 && afi ipv6) { reject; }" 

 3   chain=FULLVIEW-OUT rule="accept;" 
[admin@XXX] > /routing/filter/rule/print where chain=rfc_5735_discard 
Flags: X - disabled, I - inactive 
 4   chain=rfc_5735_discard rule="if (dst in 0.0.0.0/8 && dst-len in 8-32 && afi ipv4) { reject; }\r\nif (dst in 127.0.0.0/8 && dst-len in 8-32 && afi
 
     ipv4) { reject; }\r\nif (dst in 169.254.0.0/16 && dst-len in 16-32 && afi ipv4) { reject; }\r\nif (dst in 192.0.0.0/24 && dst-len in 24-32 && 
     afi ipv4) { reject; }\r\nif (dst in 192.0.2.0/24 && dst-len in 24-32 && afi ipv4) { reject; }\r\nif (dst in 192.88.0.0/24 && dst-len in 24-32 
     && afi ipv4) { reject; }\r\nif (dst in 198.18.0.0/15 && dst-len in 15-32 && afi ipv4) { reject; }\r\nif (dst in 198.51.0.0/24 && dst-len in 24-
     32 && afi ipv4) { reject; }\r\nif (dst in 203.0.0.0/24 && dst-len in 24-32 && afi ipv4) { reject; }\r\nif (dst in 224.0.0.0/4 && dst-len in 4-
     32 && afi ipv4) { reject; }\r\nif (dst in 240.0.0.0/4 && dst-len in 4-32 && afi ipv4) { reject; }\r\nif (dst == 255.255.255.255 && afi ipv4) { 
     reject; }\r\njump rfc_1918_discard;\r\njump rfc_6598_discard;\r\nreturn;" 
[admin@XXX] > /routing/filter/rule/print where chain=DISCARD-TOO-SPECIFIC
Flags: X - disabled, I - inactive 
 5   chain=DISCARD-TOO-SPECIFIC rule="if (dst-len in 49-128 && afi ipv6) { reject; }\r\nif (dst-len in 25-32 && afi ipv4) { reject; }\r\nreturn;\r\n" 
[admin@XXX] > /routing/filter/rule/print where chain=TELIA-IN            
Flags: X - disabled, I - inactive 
 6   chain=TELIA-IN rule="jump COMMON_EXTERNAL_IN;" 

 7   chain=TELIA-IN rule="set bgp-local-pref 100;" 

 8   ;;; SET Local Pref to 50 for AS3320 (DTAG)
     chain=TELIA-IN rule="if (bgp-as-path 3320) { set bgp-local-pref 50; }" 

 9 X ;;; SET Local Pref to 50 for AS3356 (Level3)
     chain=TELIA-IN rule="if (bgp-as-path 3356) { set bgp-local-pref 50; }" 

10 X chain=TELIA-IN rule="set bgp-path-peer-prepend 2;" 

11   chain=TELIA-IN rule="accept;" 
[admin@XXX] > /routing/filter/rule/print where chain=COMMON_EXTERNAL_IN
Flags: X - disabled, I - inactive 
12   ;;; Jump to rfc_6598_discard
     chain=COMMON_EXTERNAL_IN rule="if (afi ipv4) { jump rfc_6598_discard; }" 

13   ;;; Jump to rfc_5735_discard
     chain=COMMON_EXTERNAL_IN rule="if (afi ipv4) { jump rfc_5735_discard; }" 

14   ;;; Jump to defaultroute_discard
     chain=COMMON_EXTERNAL_IN rule="jump defaultroute_discard;" 

15   ;;; Discard Self
     chain=COMMON_EXTERNAL_IN rule="jump DISCARD_SELF;" 

16   ;;; Filter prvate ASN
     chain=COMMON_EXTERNAL_IN rule="jump Filter_private_ASN;" 

17   ;;; Filter Too Specifics
     chain=COMMON_EXTERNAL_IN rule="jump DISCARD-TOO-SPECIFIC;" 

18   ;;; Filter ASN which behave bad (continous reannouncements etc.)
     chain=COMMON_EXTERNAL_IN rule="if (bgp-as-path 35782) { reject; }" 

19   chain=COMMON_EXTERNAL_IN rule="set bgp-origin igp;" 

20   chain=COMMON_EXTERNAL_IN rule="return;" 
[admin@XXX] >

DarkNate · Fri Aug 04, 2023 11:06 am

What are we supposed to debug with?

Share the config of the routing filters.

netzwerghh · Fri Aug 04, 2023 11:33 am

What are we supposed to debug with?

Share the config of the routing filters.

Good point. I updated the original post and added route filters that might affect the prefixes in question. I can not post our full filters. That is too much. The referenced "discard"-filters are doing exactly what their names say. Rejecting the corresponding prefixes und returning afterwards. So they wouldn't be activated/installed.
The example of the not readvertised prefix is just one prefix. There are hundretthousands of prefixes received from Telia which are not being readvertised. If I filter which prefixes received from Telia are readvertised to the RIPE-RS these are exactly 0. Although there are hundretthousands prefixes installed as active route and also readvertised to our internal route servers.

BGP v7.10.2 readvertisement of ebgp learned routes to ebgp peers stopped working in v7.10

BGP v7.10.2 readvertisement of ebgp learned routes to ebgp peers stopped working in v7.10

Re: BGP v7.10.2 readvertisement of ebgp learned routes to ebgp peers stopped working in v7.10

Re: BGP v7.10.2 readvertisement of ebgp learned routes to ebgp peers stopped working in v7.10

Who is online