I am working on a design for a BGP multi-homed topology for a client. I have the basic design in mind, but would like to ask for your expert feedback on this.
ISP1 is the primary, with ISP2 as backup. There are two routers on the client side, sharing a VIP on the local interface. The idea is to have hardware and ISP redundancy. The two client routers share BGP information over a direct connection. When it comes to BGP I’m admittedly out of my depth, but I’ve managed to get this far.
Is it feasible or am I crazy?
Is there any information missing that would prevent you from answering?
faisali, thank you. I’m going to check out that solution and see if it fits.
CelticComms, the answer is yes and no. Yes, the second ISP will be only a backup for general traffic, but it will be the primary for voice traffic, which utilizes a VLAN and doesn’t rely on any routes.
If your firewall supports OSPF then I would get rid of the VRRP complexity.
Have R1 and R2 originate default-information-if-installed type=2 with R1 having metric 10 and R2 having metric 20.
Make sure both ISP1 and ISP2 are sending you a default route.
I would also make the firewall originate your master prefix into OSPF, and make R1 and R2 both originate this prefix with synchronization enabled. That way, if R1 loses its internal connectivity, but remains connected to ISP1 or ISP2, then R1 will stop advertising your network to the Internet. (I’m assuming that the 10.0.0.x network in your diagram is actually a subnet of your actual public prefix, and also assuming that it’s a SUBNET and NOT the entire prefix. (i.e. if you have a /24, the LAN interface of R1 is not x.x.x.1/24 but something like x.x.x.1/29)
Even RIPv2 could be used if the firewall supports RIP but not OSPF.
Dynamic routing protocols are better than first hop redundancy protocols like VRRP.
Since you’re planning to use iBGP between your two routers then you really don’t need the additional complexity of the two eBGP “cross links.” R1 talks to ISP1 and R2 talks to ISP2. This way you don’t need additional switches between your routers and the ISP routers (if it’s only one switch, then WHOOPS - despite your efforts, you have a single point of failure). Extra switches just burn extra power and are an additional point of failure that can just be eliminated. The BGP table will be smaller as well (if you switch to full routes).
For iBGP, remember to use a loopback bridge interface as the address on each router.
Another tip - make sure that in OSPF, the direct link from R1 to R2 is a lower cost than the direct connection to the “lan” network, so if R1 decides to hand an outbound packet over to R2 so it can be sent out ISP2, it will use the direct link and not the LAN link.
The firewall in this case is a Sonicwall NSA HA pair. It is currently set for “Simple RIP Advertisement” with an option for “Advanced Routing”. According to documentation OSPFv2 is supported.
Yes, the 10.0.0.x subnet is representing our direct assigned ARIN /24. So – and forgive my ignorance – you’re saying we should create a /29 subnet just for the three IPs between R1, R2, and FW?
Ok. I guess the reason I was looking at VRRP here is that we would get the hardware redundancy. If we have ISP1 paired with R1 and ISP2 paired with R2, then we lose ISP1 if R1 goes down for example. Maybe I’m overthinking it.
This makes sense. So, basically what you’re saying is that the design I presented is overkill. If a router goes down and that means ISP1 is down, that’s considered OK. Right?
As long as ISP2 has enough performance to carry your network’s full load on its own, then yes (at least in my mind). Besides, I’d imagine most ISPs are going to raise an eyebrow when you want to have two eBGP sessions across one link. They’re most likely going to either say ‘no’ or else charge you for two connections. Remember - the ISP is only going to have one router on their end of the circuit, so it’s a ‘single point of failure’ for that connection as well. (granted it’s likely to be a big-iron router)
After writing my long post, I was reflecting on your design and my design, and what made them different. Yours is exactly as you said - an active/standby pair, whereas my design is intended to be more of a “use both at the same time and make sure either ISP has enough bandwidth to carry the load alone” - It’s how I set up the routing for the telco where I worked previous to my current job. It just seems silly to me to have a whole internet connection just sitting there as a standby link. It would be like if there were two freeways leading into and out of your town, but you kept one closed in case the bridge on the main freeway went out.
As for the IP subnetting, yes, if you break up your /24 into subnets then you’re going to have much more flexibility. If you get a private circuit to another city, you can use another chunk of your address space over there. If you have all 256 addresses tied to a single network, then you’re kind of stuck there. On the firewall, if it lets you, create null routes for any NAT addresses that you will assign to that firewall, and then redistribute static into OSPF. Also null-route your master /24 there so it becomes the center of gravity of your entire space. The firewall shouldn’t need any NAT addresses to actually be part of any particular interface’s address range. All NAT does is erase the original source or destination IP and replace it with some other address. The /32 routes to the real addresses will cause your external routers to explicitly forward your NAT addresses to the firewall’s WAN interface. (as opposed to using ARP/proxy-arp)
One last tip - use two switches between the firewall pair and the router pair. Put FW-A into SW-A, and R1 into SW-A, then put FW-B into SW-B and R2 into SW-B. Finally, connect SW-A and SW-B directly. (or if the SonicWalls have switchports like Mikrotiks do, then you can just use a HW-switch on R1 to connect directly to both Sonicwalls, and do the same on R2. (This would be the best since it eliminates two pieces of gear) As long as SonicWall B remains truly passive and doesn’t forward frames on the switchports while it’s in standby mode, then you can use this topology. If it fowards as well, then they’re going to need to support STP because the Mikrotiks’ hardware switch ports don’t. (to my knowledge)