All of our customer traffic goes through Site A. We have a /21 of Public IPs which are NAT’d to private IPs on the Edge Router.
My goal is to have our /21 advertised to both of our ISPs, so that if Site A goes down then our /21 will reroute to Site B.
The problem is: when the /21 is advertised to both ISPs, I cannot force all inbound traffic through ISP 1 instead of ISP 2. There is always some traffic that tries to come through ISP 2, which I don’t want. I only want ISP 2 to push traffic if our BGP Peer to ISP 1 goes down.
Prepending hasn’t worked. And I don’t think I can do MED since we’re using separate BGP Instances.
My current belief is that I’ll need to use BGP Communities. I don’t know much about Communities, so I’m hoping someone can point me in the right direction.
Thats likely because a single prepend to ISP B is not enough and some inbound traffic still has a shorter route through ISP B than they do ISP A. Try multiple prepends of ISP B
You can never truly control how traffic gets to you, if you provide multiple paths - but the more successful way is definitely via more specific prefixes.
You say you have a /21 - so advertise 2x /22’s via ISP A and just the /21 via ISP B - a more specific route in pretty much any route table takes precedence, and it’d take a provider manually kicking in to try and force it via the less specific route (or weird partial table imports also possible).
Its also advisable to still advertise the /21 via ISP A as well, so you’d be advertising /22, /22, /21 to A and /21 to B.
Last Friday I had a helpful conversation with an IT Solutions company that offices out of the same building where the Site B data center is. They’re still looking over our BGP config, but it sounds like they think ISP 2 just has too few options for manipulating incoming traffic. I will post an update once we find a solution.
