I have setup a bgp monitoring tool which sends me alerts by email.
I constantly receive alerts which I cant make any sense of.
The BGP we have in place was configured by our former net admin.
We have a /20 allotted to us by Arin however we have opted to use 8 /24’s at each of our edge routers(2 edge routers)
We have what we call the north network and south network.
we’ll say our /20 is 1.1.1.1/20
We have 2 upstream providers both are ATT but setup in a way to give us redundancy should one of them fail.
At each of of our edge routers we are peered with upstream on wan interface
On inside interfaces we have a /29 private range setup with both our edge routers using this range to communicate ibgp.
the edge router for north end of network has 1.1.1.1/24, 2.2.2.2/24, 3.3.3.3/24, 4.4.4.4/24, 5.5.5.5/24, 6.6.6.6/24, 7.7.7.7/24, 8.8.8.8/24 6.6.6.6/27 on the inside interface. I know the /27 is redundant but this is how he set it up to give a business customer a /27 for themselves.
We also have 172.16.16.3/29 on inside interface which is remote peer address for the south
the edge router for the south has 9.9.9.9/24, 10.10.10.10/24, 11.11.11.11/24, 12.12.12.12/24, 13.13.13.13/24, 14.14.14.14/24, 15.15.15.15/24, 16.16.16.16/24
We have 172.16.16.2/29 on inside interface which is remote peer address for mikrotik on north network.
Our configuration doesn’t seem to cause customer problems, however last week when our upstream provider on the south network went down instead of the south traffic going over our ibgp link to the north network upstream provider
it did nothing.
I have copied and pasted our configurations into notepad but will paste those in a reply post to avoid this post becoming a page long.
We also have some private ip ranges on both edge routers inside interfaces for our management subnets.
I read about something called route flapping and I have a feeling this may be going on or issues with our routing filters, which only have one entry apiece.
Please calling all BGP experts to advise me how to find my former net admin’s configuration mistakes.
0 A 1.1.1.0/24 yes
1 A 2.2.2.0/24 yes
2 A 3.3.3.0/24 yes
3 A 4.4.4.0/24 yes
4 A 5.5.5.0/24 yes
5 A 6.6.6.0/24 yes
6 A 7.7.7.0/24 yes
7 A 8.8.8.0/24 yes
8 X 9.9.9.0/24 no
9 X 10.10.10.0/24 no
10 X 11.11.11.0/24 no
11 X 12.12.12.0/24 no
12 X 13.13.13.0/24 no
13 X 14.14.14.0/24 no
14 X 15.15.15.0/24 no
15 X 16.16.16.0/24 no
LZ routes
0 X S ;;; Default OUT LZ
0.0.0.0/0 xxx.xxx.xxx.77 (upstream providers inside interface) 1
1 ADb 0.0.0.0/0 reachable xxx.xxx.xxx.77 20 eth1_WAN
2 Db 0.0.0.0/0 reachable 172.16.16.2 200 eth2_LAN
3 ADC 192.0.0.0/8 192.46.0.25 0 eth2_LAN
4 Db 192.0.0.0/8 reachable 172.16.16.2 200 eth2_LAN
5 ADb xxx.xxx.xxx.56/30 (south network gateway) reachable 172.16.16.2 200 eth2_LAN
6 ADC xxx.xxx.xxx.76/30 xxx.xxx.xxx.78 0 eth1_WAN
7 ADC 1.1.1.0/24 1.1.1.1 0 eth2_LAN
8 ADC 2.2.2.0/24 2.2.2.1 0 eth2_LAN
9 ADC 3.3.3.0/24 3.3.3.1 0 eth2_LAN
10 ADC 4.4.4.0/24 4.4.4.1 0 eth2_LAN
11 ADC 5.5.5.0/24 5.5.5.1 0 eth2_LAN
12 ADC 5.5.5.32/27 5.5.5.33 0 eth2_LAN
13 ADC 6.6.6.0/24 6.6.6.1 0 eth2_LAN
14 ADC 7.7.7.0/24 7.7.7.1 0 eth2_LAN
15 ADC 8.8.8.0/24 8.8.8.1 0 eth2_LAN
16 X S ;;; Failover route
9.9.9.0/21 xxx.xxx.xxx.77 1
17 X S ;;; force traffic out then back in
9.9.9.0/24 xxx.xxx.xxx.77 1
18 ADb 9.9.9.0/24 reachable 172.16.16.2 200 eth2_LAN
19 X S ;;; force traffic destined for south out and back in
10.10.10.0/24 xxx.xxx.xxx.77 1
20 ADb 10.10.10.0/24 reachable 172.16.16.2 200 eth2_LAN
21 ADb 10.10.10.52/32 reachable 172.16.16.2 200 eth2_LAN
22 X S ;;; force traffic destined for south out then back in
11.11.11.0/24 xxx.xxx.xxx.77 1
23 ADb 11.11.11.0/24 reachable 172.16.16.2 200 eth2_LAN
24 X S ;;; force traffic destined for south out and back in
12.12.12.0/24 xxx.xxx.xxx.77 1
25 ADb 12.12.12.0/24 reachable 172.16.16.2 200 eth2_LAN
26 X S ;;; force traffic destined for south out and back in
13.13.13.0/24 xxx.xxx.xxx.77 1
27 ADb 13.13.13.0/24 reachable 172.16.16.2 200 eth2_LAN
28 ADb 14.14.14.0/24 reachable 172.16.16.2 200 eth2_LAN
29 X S ;;; force traffic destined for south out and back in
14.14.14.0/24 xxx.xxx.xxx.77 1
30 X S ;;; force traffic destined for south out and back in
15.15.15.0/24 xxx.xxx.xxx.77 1
31 ADb 15.15.15.0/24 reachable 172.16.16.2 200 eth2_LAN
32 X S ;;; force traffic destined for south out and back in
16.16.16.0/24 xxx.xxx.xxx.77 1
33 ADb 16.16.16.0/24 reachable 172.16.16.2 200 eth2_LAN
34 ADC 172.16.16.0/29 172.16.16.3 0 eth2_LAN
35 Db 172.16.16.0/29 reachable 172.16.16.2 200 eth2_LAN
36 ADb 172.16.17.0/24 reachable 172.16.16.2 200 eth2_LAN
0 A 9.9.9.0/24 yes
1 A 17.17.17.0/24 yes
2 A 11.11.11.0/24 yes
3 A 12.12.12.0/24 yes
4 A 13.13.13.0/24 yes
5 A 14.14.14.0/24 yes
6 A 15.15.15.0/24 yes
7 A 16.16.16.0/24 yes
8 X 1.1.1.0/24 yes
9 X 2.2.2.0/24 yes
10 X 3.3.3.0/24 yes
11 X 4.4.4.0/24 yes
12 X 5.5.5.0/24 yes
13 X 6.6.6.0/24 yes
14 X 7.7.7.0/24 yes
15 X 8.8.8.0/24 yes
Jorie Routes
0 X S ;;; test
0.0.0.0/0 xxx.xxx.xxx.57 1
1 ADb 0.0.0.0/0 reachable xxx.xxx.xxx.57 20 ether1_WAN
2 Db 0.0.0.0/0 reachable 172.16.16.3 200 ether2_LAN
3 ADC 192.0.0.0/8 192.0.0.25 0 ether2_LAN <<<<management subnet
I’m curious, why are you running separate BGP instances for your iBGP and eBGP peers? You can terminate both on the same BGP instance.
What exactly did the traffic do when the South router’s eBGP peer dropped? I see you’re sending default routes between the two routers, so your traffic should have re-routed properly when that eBGP connection dropped.
This was all configured by our former net admin, I’m not sure as to why he did it this way, maybe only way he could figure out.
I personally don’t understand how we could run the ibgp and ebgp on same instance given it appears you can only have one remote id, but again I can be completely wrong here This is my first exposure really to bgp so I’m trying to learn from his config which apparently isn’t right;;
Peoples traffic was still to go through the down upstream gateway last week.
p.s. many BGP instances (i.e. one per peer) is a common mistake. might this be influenced by some strange vendors hardware that usually gets configured exactly that way (one instace=one peer)?
