BGP huge problem with second Mikrotik network announcement !

[johnvam]

Dear friends,

I have attached our network architecture. We splitted our upstreamers to both our BGP routers and when we enable the BGP networks announcement on second BGP router, the world is losing us… We have our own AS number and one /22 IPv4 pool that we announce from the first BGP router.

Our purpose is to :

1. add one more BGP router for failover purpose
2. to have failover also for outbound and inbound traffic

We get full FIRT from both Cogent and Level 3. Also both Mikrotik do OSPF between them and Switches.

Have you any idea why when we enable the announcement of /22 at the second BGP router we are dead??? We have one VPS outside our network and we lose ping when we enable the second announcement.

Thank you all!

[faisali]

post your config…

we are using a similar setup, we have two routers doing bgp to multiple peers each, and we are running ibgp & OSPF between the two. (OSPF is running on other routers as well)

[KillerOPS]

are you running ibgp between the two routers?

[johnvam]

BGP ROUTER 1 CONFIG
/routing filter
chain=BGP ROUTER 2-IN protocol=“” bgp-as-path=^XXX789_ invert-match=no action=accept set-distance=20 set-bgp-prepend-path=“”
chain=BGP ROUTER 2-OUT protocol=“” invert-match=no action=accept set-bgp-prepend-path=“”
chain=OUT-AS-IPV4 prefix=XXX.XX.4.0/22 invert-match=no action=accept set-bgp-prepend-path=“”
chain=OUT-AS-IPV4 invert-match=no action=discard set-bgp-prepend-path=“”
chain=IN-COGENT bgp-as-path=^174_ invert-match=no action=accept set-bgp-prepend-path=“”
chain=IN-COGENT invert-match=no action=discard set-bgp-prepend-path=“”

/routing bgp instance
name=“default” as=XXX789 router-id=XXX.XX.7.217 redistribute-connected=no redistribute-static=no redistribute-rip=no redistribute-ospf=no redistribute-other-bgp=no out-filter=“” client-to-client-reflection=yes ignore-as-path-len=no routing-table=“”

/routing bgp peer
name=“BGP ROUTER 2” instance=default remote-address=XXX.XX.7.218 remote-as=XXX789 tcp-md5-key=“” nexthop-choice=force-self multihop=no route-reflect=no hold-time=3m ttl=default in-filter=BGP ROUTER 2-IN out-filter=BGP ROUTER 2-OUT address-families=ip default-originate=never remove-private-as=no as-override=no passive=no use-bfd=no

name=“COGENT-IPV4” instance=default remote-address=XXX.XX.120.41 remote-as=174 tcp-md5-key=“” nexthop-choice=default multihop=no route-reflect=no hold-time=3m ttl=default in-filter=IN-COGENT out-filter=OUT-AS-IPV4 address-families=ip default-originate=never remove-private-as=no as-override=no passive=no use-bfd=no

/routing bgp network
network=XXX.XX.4.0/22 synchronize=no



BGP ROUTER 2 CONFIG
/routing filter
chain=BGP ROUTER 1-IN protocol=“” bgp-as-path=^XXX789_ invert-match=no action=accept set-distance=20 set-bgp-prepend-path=“”
chain=BGP ROUTER 1-OUT protocol=“” invert-match=no action=accept set-bgp-prepend-path=“”
chain=OUT-AS-IPV4 prefix=XXX.XX.4.0/22 invert-match=no action=accept set-bgp-prepend-path=“”
chain=OUT-AS-IPV4 invert-match=no action=discard set-bgp-prepend-path=“”
chain=IN-LEVEL3 bgp-as-path=^3356_ invert-match=no action=accept set-bgp-prepend-path=“”
chain=IN-LEVEL3 invert-match=no action=discard set-bgp-prepend-path=“”

/routing bgp instance
name=“default” as=XXX789 router-id=XXX.XX.7.218 redistribute-connected=no redistribute-static=no redistribute-rip=no redistribute-ospf=no redistribute-other-bgp=no out-filter=“” client-to-client-reflection=yes ignore-as-path-len=no routing-table=“”

/routing bgp peer
name=“BGP ROUTER 1” instance=default remote-address=XXX.XX.7.217 remote-as=XXX789 tcp-md5-key=“” nexthop-choice=force-self multihop=no route-reflect=no hold-time=3m ttl=default in-filter=BGP ROUTER 1-IN out-filter=BGP ROUTER 1-OUT address-families=ip default-originate=never remove-private-as=no as-override=no passive=no use-bfd=no

name=“LEVEL3-IPV4” instance=default remote-address=XXX.XX.220.71 remote-as=3356 tcp-md5-key=“” nexthop-choice=default multihop=no route-reflect=no hold-time=3m ttl=default in-filter=IN-LEVEL3 out-filter=OUT-AS-IPV4 address-families=ip default-originate=never remove-private-as=no as-override=no passive=no use-bfd=no

/routing bgp network
network=XXX.XX.4.0/22 synchronize=no

[johnvam]

I also read this example with Cisco routers and it’s easy to implement to Mikrotik.

http://www.cisco.com/c/en/us/support/docs/ip/border-gateway-protocol-bgp/13762-40.html#conf5

But my question is: if I add “set as-path prepend XXX789” to one router only then all the inbound traffic probably will go to the other router cause of smaller AS path.

Am I wrong?

[johnvam]

@faisali

post your config…

we are using a similar setup, we have two routers doing bgp to multiple peers each, and we are running ibgp & OSPF between the two. (OSPF is running on other routers as well)

Dear faisali.

could you please share with me your config in order to check it!

Thank you very much!

[faisali]

ok, looks like you are transitioning from a cisco system.. no worries, you will get the hang of it real quick.

You are doing something goofy with the filters…

May I suggest that you set them up modular,
See my message on this thread…
http://forum.mikrotik.com/t/bgp-routes-not-propagated-between-ibgp-and-ebgp/75040/1

The discussion is on a very similar configuration as yours.

1. Fix your filters, so that you are accepting full routes (no filtering by ASN).
   Your inbound filter should accept all routes
   Your outbound filter should advertise your prefix.

related to the sample config (other post, you have tinet-miami-out filter)…
my in-filter is as follows:-

 /routing filter print where chain=tinet-mia-in 
Flags: X - disabled 
 0   chain=tinet-mia-in match-chain=junk invert-match=no action=discard set-bgp-prepend-path="" 

 1   chain=tinet-mia-in match-chain=not-to-specific invert-match=no action=accept set-bgp-prepend-path="" 
     append-bgp-communities=11280:666

I am using communities, as such I am appending, you don’t have to.

Junk chain is:

/routing filter print where chain=junk         
Flags: X - disabled 
 0   chain=junk prefix=0.0.0.0/0 prefix-length=0 invert-match=no action=accept set-bgp-prepend-path="" 

 1   chain=junk prefix=10.0.0.0/8 prefix-length=8-32 invert-match=no action=accept set-bgp-prepend-path="" 

 2   chain=junk prefix=127.0.0.0/8 prefix-length=8-32 invert-match=no action=accept set-bgp-prepend-path="" 

 3   chain=junk prefix=172.16.0.0/12 prefix-length=12-32 invert-match=no action=accept set-bgp-prepend-path="" 

 4   chain=junk prefix=192.168.0.0/16 prefix-length=16-32 invert-match=no action=accept set-bgp-prepend-path="" 

 5   chain=junk prefix=223.255.255.0/24 prefix-length=24-32 invert-match=no action=accept set-bgp-prepend-path="" 

 6   chain=junk prefix=224.0.0.0/11 prefix-length=11-32 invert-match=no action=accept set-bgp-prepend-path=""

and no-so-specific chain is…

/routing filter print where chain=not-to-specific 
Flags: X - disabled 
 0   chain=not-to-specific prefix=0.0.0.0/0 prefix-length=0-24 invert-match=no action=accept set-bgp-prepend-path="" 
     append-bgp-communities=""

Additionally you will need to bring up an ibgp session between the two BGP routers.

Hope this helps in pointing you in the right direction.

[johnvam]

Dear faisali,

I found your information very useful but I would prefer if you could also send me your peer details.

Also do you have any idea why when we start advertisement of our networks from the second BGP router we loose also inbound and outbound traffic?

Thank you

[faisali]

we are peering with a couple of dozen networks, the basic setup is the same, and then each has it’s own specific setup.
i.e. external peers are same as listed in the config, only difference is filters.
the only one with different settings is the ibgp peer (it has route reflector setup)
(Peer detail is listed in the other thread I have referenced above)

As to what could be going wrong in your setup is a bit hard to guess… but not too difficult to troubleshoot.

1. Make sure you have loopback interface setup with IP address.

2. Get rid of the bgp-as-path filtering you are doing… e.g. bgp-as-path=^XXX789_

3. make sure that ospf is running between the two routers and they can see/reach each others loopback interface.

4. Bring up your bgp with one router, make sure you are receiving full tables appx 500k routes, and you are sending your prefix (using /routing bgp advertisements print PEERNAME)

5. bring up your IBGP session between the two routers, make sure you are getting all of your routes to the 2nd router. (remember to remove bgp-as-path=^XXX789_ filters, there no need for any and besides this filter is wrong for ibgp)

6. bring up the bgp session with the 2nd router, but make sure you have both content of filters inbound and outbound disabled, this way you are not receiving any routes or advertising any routes.
   once the bgp session is up…

7. modify your filter on at a time, you can do receive side, to make sure you get full routers… and do the send side to make sure you are advertising your prefix properly.

I suggest that you modify your OUT-AS-IPV4 to include larger prefix equivalent to le 24 command in cisco.. (see my code from the other thread).

[johnvam]

@faisali

1. Make sure you have loopback interface setup with IP address.

For eBGP or iBGP peers? Do you use public or private IPs for loopback?

@faisali

the only one with different settings is the ibgp peer (it has route reflector setup)

you have route reflector enabled on both routers for iBGP?

[faisali]

@johnvam

@faisali

1. Make sure you have loopback interface setup with IP address.

For eBGP or iBGP peers? Do you use public or private IPs for loopback?

@faisali

the only one with different settings is the ibgp peer (it has route reflector setup)

you have route reflector enabled on both routers for iBGP?

BGP Best Practices recommend Public IP for Loopback (one is sufficient, it will work for bgp/ibgp/ospf etc)

And in our case, since we are taking full ebgp feeds on each router, we have route reflector turned on both routers.