OSPF Redundancy and Summarization

LuizMeier · Wed May 14, 2014 5:06 pm

Good Morning,

My problem: Too many routes on the routing tables in all routers. They are adding one route for each tunnel for each branch office that I have OSPF in. I saw some articles about summarization of routes here and that by adding more areas.
My doubt is if I would have problems in summarization, once my destination network will be always the same, no matter the path.

I have the following scenario and I would like to know another opinion for the best setup.

1) 2 RB's 1100AH at my HQ. One for PPTP tunnels and another for the EoIP tunnels. They talk through vlan11.
2) The EoIP tunnel is made through my MPLs topology, delivered by my ISP. Without it, i couldn't make adjacencies with OSPF.
3) The PPTP tunnel is made throught internet link, delivered by another ISP, for redundancy.
4) RB 2011 at the branch office, with both tunnels running.

I have the following OSPF setup. This example is for one of the branch offices.

[R1]

ros code

/routing ospf network
add area=backbone network=172.20.1.26/32
add area=backbone network=10.254.254.0/28
/routing ospf interface
add authentication=md5 authentication-key=XXXXX authentication-key-id=10 cost=50 interface=pptp-tunnel-name network-type=point-to-point
add authentication=md5 authentication-key=XXXXX authentication-key-id=10 interface=vlan11 network-type=broadcast
/routing ospf instance
set [ find default=yes ] distribute-default=always-as-type-1 router-id=10.255.255.255

[R2]

ros code

/routing ospf network
add area=backbone network=10.1.35.0/30
add area=backbone network=10.254.254.0/28
/routing ospf interface
add authentication=md5 authentication-key=XXXXX authentication-key-id=10 interface=eoip-tunnel-name network-type=point-to-point
add authentication=md5 authentication-key=XXXXX authentication-key-id=10 interface=vlan11 network-type=broadcast
/routing ospf instance
set [ find default=yes ] redistribute-connected=as-type-1 router-id=10.255.255.250

[R3]

ros code

/routing ospf network
add area=backbone network=10.25.35.0/24
add area=backbone network=10.1.35.0/30
add area=backbone network=172.20.1.1/32
/routing ospf interface
add authentication=md5 authentication-key=XXXXX authentication-key-id=10 interface=eoip-tunnel-name network-type=point-to-point
add authentication=md5 authentication-key=XXXXX authentication-key-id=10 interface=ether1-lan network-type=broadcast passive=yes
add authentication=md5 authentication-key=XXXXX authentication-key-id=10 cost=50 interface=pptp-tunnel-name network-type=point-to-point
/routing ospf instance
set [ find default=yes ] redistribute-connected=as-type-1 router-id=10.255.255.250

jkarras · Thu May 15, 2014 4:49 am

So is this just a typical setup? As in are other branch offices just like the one mentioned here? The only way you can summarize things is if its a stub are or via some kind of choke point (area to area). So to do this it takes some IP address planning. Things like keeping all IP address ranges at each branch office setup in ranges that can be summarized. Unfortunately point to point links will always show up as individual links inside the same area. If you had a larger network where you could break it into two areas and your PtP links could be summarized where the two areas join then you could complete this.

You have duplicate router IDs on Router 2 and Router 3.

Curiosity question about your EoIP link via the MPLS. Is this MPLS service provided by a carrier/service provider? Is it a point to point type circuit or a L3 VPN service between your hub site and remote offices? If its a point to point VPLS style circuit there is no need to run EoIP on top to get OSPF to work.

LuizMeier · Thu May 15, 2014 3:10 pm

Thanks for your reply!

So is this just a typical setup? As in are other branch offices just like the one mentioned here?

Yes, there is another 4 branch offices running like that. And besides that, we are changing our topology to be this way on other ~50 branch offices.

The only way you can summarize things is if its a stub are or via some kind of choke point (area to area). So to do this it takes some IP address planning. Things like keeping all IP address ranges at each branch office setup in ranges that can be summarized. Unfortunately point to point links will always show up as individual links inside the same area. If you had a larger network where you could break it into two areas and your PtP links could be summarized where the two areas join then you could complete this.

I've been looking on the internet about some guys saying that, about having 2 areas, for example. My though is one area for my PPTP tunnels and another for my EoIP tunnels. So that I could summarize at least the middle networks, like 172.20.0.0 and 10.1.0.0.
I think if I would not have problems with the destination network (in the example, 10.25.35.0). I don't know where I would put the announce for that network. In both routers?

The ideal world would be the branch router with the default route (that my HQ router already gives) and a 10.25.0.0/16 route, that is the summarized network for all my branch offices plus HQ. We are a HQ with 10.25.0.0/23 and other offices with 10.25.x.0/24.
I could do that with a filter, but I think that is not the better form, because the routers would keep announcing the routes.

You have duplicate router IDs on Router 2 and Router 3.

Yes, typing mistake.

Curiosity question about your EoIP link via the MPLS. Is this MPLS service provided by a carrier/service provider? Is it a point to point type circuit or a L3 VPN service between your hub site and remote offices? If its a point to point VPLS style circuit there is no need to run EoIP on top to get OSPF to work.

My MPLS link is provided by a carrier/service provider. They call it a VPN, but it is not actually a PtP connection. In some branch offices, I pass over 3 hopes before reaching my router. Besides that, I wouldn't like to ask them for a OSPF instance through their router, because we would be depending on them for each change on topology.

StubArea51 · Thu May 15, 2014 8:12 pm

It sounds like you have an MPLS L3VPN from the carrier

What routing protocol is the carrier using to hand off the MPLS connection? If it is BGP, you might consider using BGP end to end for 50 sites and simplify your routing.

From a design standpoint, you don't want all OSPF routers in the same area for two reasons:

1 You need an area border router to summarize
2. The SPF algorithm runs within an area but not across - you don't want flaps at the edge to cause multiple SPF recalculations in Area 0 at the core.

LuizMeier · Thu May 15, 2014 8:55 pm

It sounds like you have an MPLS L3VPN from the carrier

What routing protocol is the carrier using to hand off the MPLS connection? If it is BGP, you might consider using BGP end to end for 50 sites and simplify your routing.

What I know, by peaking some configs during one maitenance, is that they are using only static routes on the branch offices. Just default route to the HQ router, that problably must have one static route for each branch office. Perhaps they have a dynamic routing protocol, but just for their use. And I don't believe that they'd give us a routing instance throug their routers.

From a design standpoint, you don't want all OSPF routers in the same area for two reasons:

1 You need an area border router to summarize
2. The SPF algorithm runs within an area but not across - you don't want flaps at the edge to cause multiple SPF recalculations in Area 0 at the core.

Your tip is to use one area for each tunnel router? R1 and PPTP tunnels on area0 and R2+EoIp tunnels in area 1?

CelticComms · Fri May 16, 2014 4:22 pm

Is the primary concern that there are too many routes on the branch RB2011s? I would have thought that the RB1100s can easily cope with the number of routes they would be seeing.

Stub areas may help to reduce the number of routes on the branch routers, but the scale of the reduction will depend on the IP numbering scheme and opportunities to summarize. To get the number of routes on the branch offices down you most likely want to *not* flood summary LSAs to the branch routers (totally stubby in Cisco parlance). Whether you use stub or NSSA depends on the routing arrangement at the branch offices.

Creating an area for every branch could become burdensome. If the branches have a simple network layout you could possibly consider running RIP2 at the branches and simply redistributing RIP into OSPF at the branch router. That would maintain the fast convergence of OSPF on the WAN links (in Area 0) while simplifying the workload on the branch routers.

Unfortunately EIGRP isn't an option - it would work well with fast convergence in this situation.

LuizMeier · Fri May 16, 2014 8:39 pm

Is the primary concern that there are too many routes on the branch RB2011s? I would have thought that the RB1100s can easily cope with the number of routes they would be seeing.

My primary concern is the best way to mantain an available infrastructure without the need of a manual intervention. That was our primary though when changing to a dynamic routing protocol, not changing static routes at any link down event.
For an example, we have today 6 branch offices in this new topology. Following is the routing table of one of the 2011 branch routers.

# DST-ADDRESS PREF-SRC GATEWAY DISTANCE
0 S 0.0.0.0/0 pppoe-XXXXX 1
1 A S 0.0.0.0/0 10.25.18.254 1
2 ADo 0.0.0.0/0 10.1.18.1 110
3 S ;;; update-route-vpn: DNS servers
8.8.4.4/32 pppoe-XXXXX 1
4 S ;;; update-route-vpn: DNS servers
8.8.8.8/32 pppoe-XXXXX 1
5 A S 10.0.0.0/8 10.25.18.254 1
6 ADo 10.1.10.0/30 10.1.18.1 110
7 ADo 10.1.12.0/30 10.1.18.1 110
8 ADC 10.1.18.0/30 10.1.18.2 eoip-YYYYY 0
9 ADo 10.1.24.0/30 10.1.18.1 110
10 ADo 10.1.35.0/30 10.1.18.1 110
11 ADo 10.1.54.0/30 10.1.18.1 110
12 ADo 10.2.10.0/29 10.1.18.1 110
13 ADo 10.25.0.0/23 10.1.18.1 110
14 ADo 10.25.10.0/24 10.1.18.1 110
15 ADo 10.25.12.0/24 10.1.18.1 110
16 ADC 10.25.18.0/24 10.25.18.20 ether1-lan 0
17 ADo 10.25.24.0/23 10.1.18.1 110
18 ADo 10.25.35.0/24 10.1.18.1 110
19 ADo 10.25.46.0/24 10.1.18.1 110
20 ADo 10.25.54.0/24 10.1.18.1 110
21 ADC 10.254.18.252/30 10.254.18.253 ether10-mpls 0
22 ADo 10.254.254.0/28 10.1.18.1 110
23 A S 10.254.254.5/32 10.254.18.254 1
24 ADo 10.254.254.16/30 10.1.18.1 110
25 ADC 10.255.18.255/32 10.255.18.255 loopback 0
26 ADC 172.20.1.1/32 172.20.1.21 pptp-YYYYY 0
27 ADo 172.20.1.2/32 10.1.18.1 110
28 ADo 172.20.1.9/32 10.1.18.1 110
29 ADo 172.20.1.10/32 10.1.18.1 110
30 ADo 172.20.1.21/32 10.1.18.1 110
31 ADo 172.20.1.22/32 10.1.18.1 110
32 ADo 172.20.1.26/32 10.1.18.1 110
33 ADC 172.30.1.0/24 172.30.1.1 vlan2-clientes 0
34 S ;;; update-route-vpn: pptp YYYYY.dyndns.org
177.220.177.57/32 pppoe-XXXXX 1

Can you imagine it with 50 branch offices? Will the 2011 handle it well?

Creating an area for every branch could become burdensome. If the branches have a simple network layout you could possibly consider running RIP2 at the branches and simply redistributing RIP into OSPF at the branch router. That would maintain the fast convergence of OSPF on the WAN links (in Area 0) while simplifying the workload on the branch routers.

I was thinking in make an area for R1 and its PPTP tunnels and another area for R2 and its EoIP tunnels. Could I make the branchs just see the summarized route to 10.25.0.0/16? They would see 1 throug R1 and in case of problem, for example, thry would switch to R2.

StubArea51 · Sat May 17, 2014 3:27 am

It sounds like you have an MPLS L3VPN from the carrier

What routing protocol is the carrier using to hand off the MPLS connection? If it is BGP, you might consider using BGP end to end for 50 sites and simplify your routing.
What I know, by peaking some configs during one maitenance, is that they are using only static routes on the branch offices. Just default route to the HQ router, that problably must have one static route for each branch office. Perhaps they have a dynamic routing protocol, but just for their use. And I don't believe that they'd give us a routing instance throug their routers.

From a design standpoint, you don't want all OSPF routers in the same area for two reasons:

1 You need an area border router to summarize
2. The SPF algorithm runs within an area but not across - you don't want flaps at the edge to cause multiple SPF recalculations in Area 0 at the core.
Your tip is to use one area for each tunnel router? R1 and PPTP tunnels on area0 and R2+EoIp tunnels in area 1?

The 1100AHx2 routers have been used to take two full BGP feeds, so you aren't going to come close to exhausting the routing table in them. We have used the 2011 routers as MPLS PE routers in large wireline carrier networks and put several hundred routes in them with no performance issues. In fact they could probably handle over a thousand with no problem.

I would probably segment this network into 3 areas and use OSPF as the routing protocol if you aren't familiar with BGP

Area 0 - Network Core - both of your 1100 routers
Area 1 - PPTP Tunnels
Area 2 - EOIP Tunnels

This will cut your routing tables significantly if you decide that you need it, and you can also enable stub areas to really reduce the workload. If there is only one path out of a network, then a default is really all you need.

The other benefit is that you will protect the core from SPF flaps a little better by segmenting into several areas.

There are many different ways you can go with this, but always try to keep the network as simple as you can and still accomplish the objective. we work in extremely large environments and simplicity really does contribute to stability.

Good Luck!

AlexS · Sat May 17, 2014 9:15 am

Hmm, i fi read right and understood correctly I would break up

any wan link as area 0, so ptp and eoip.
a new area for each office.

this is how I have mine setup, 1 office and 2 dc. but I have a broadcast domain shared by all sites.
I also use summarisable addressing
so say for office 1 I might allocate

10.1.0.0/16 and then break that down into /24's. bit of an over kill but, you know any 10.1 address belongs to office 1 and 10.2/16 to office 2 etc etc.
so office 1 would have
address from 10.1.0.0/16
10.1.0.0/24
10.1.1.0/24
it would have OSPF areaid 10.1.0.0, and a summary range of 10.1.0.0/16
so all you should outside of office 1 is 10.1.0.0/16

A

CelticComms · Sat May 17, 2014 7:07 pm

Are the branches given local internet access or does that traffic also flow via head office?

The 2011s could handle the routing rules but it looks as if there is no need to have all that routing traffic and potential route flapping - there are better alternative configs.

StubArea51 · Sat May 17, 2014 11:47 pm

I would probably avoid an area per site if you have 50 sites since you don't really need it....three areas should be plenty

CelticComms · Mon May 19, 2014 6:19 pm

The 2011s seem to be the devices which would benefit most from a reduction in routes/routing load. Creating separate areas for the EoIP links and PPTP links is all very well as regards summarisation being possible between the 1100s - but what about the 2011s? Are they going to now be in *both* areas? ... and acting as ASBRs into both too? I see no config or workload reduction there!

The config used by AlexS certainly allows full summarisation to the 2011s or even replacement of summary routes by defaults on the 2011s.

LuizMeier · Mon May 19, 2014 6:34 pm

Thanks for all replies!

The primary concern is the workload and organization. As you guys said, the 2011 is strong, and I don't need to have preoccupations about that.

What I understood from IPArchitect is if I do that setup, I wil receive in the 2011's just the summarized route to areas 1 and 2. It would be done problably by an area range configuration, right?
And the 2011 would stay in both areas, right?

About the tip from AlexS, I got concerned about the number of areas created. Won't it make the things more complicated?

I'll try both setups and return here for feedback.

StubArea51 · Mon May 19, 2014 7:22 pm

@ CelticComms

OSPF routers only become an ASBR when other routing protocols are in use. When only OSPF is in use, they would be only an ABR.

@ AlexS

You could put both of the tunnels in one area, but it might limit filtering an summarizing options should you need that later on.

It looks like you have more than one WAN link at each remote site via the tunnels, I would seriously consider BGP long term for a few reasons...

1. Much better than OSPF at handling multiple paths
2. Summarization and filtering is way better than OSPF
3. You could eliminate the need for EOIP tunnels for routing adjacency...BGP doesn't rely on multicast

CelticComms · Mon May 19, 2014 8:17 pm

@ CelticComms
OSPF routers only become an ASBR when other routing protocols are in use. When only OSPF is in use, they would be only an ABR.

Incorrect. An OSPF router is also an ASBR when it (for example) redistributes connected routes. The config shown earlier has both 2011s set to redistribute connected routes thus my comment. Of course the 2011 configs could be altered to include the connected networks directly via network statements in which case they would not be ASBRs, but in the config above they are acting as ASBRs.

StubArea51 · Tue May 20, 2014 2:45 am

@ CelticComms
OSPF routers only become an ASBR when other routing protocols are in use. When only OSPF is in use, they would be only an ABR.
Incorrect. An OSPF router is also an ASBR when it (for example) redistributes connected routes. The config shown earlier has both 2011s set to redistribute connected routes thus my comment. Of course the 2011 configs could be altered to include the connected networks directly via network statements in which case they would not be ASBRs, but in the config above they are acting as ASBRs.

Fair point..I missed the redistribution, but that would be routing that's not OPSF

Better to advertise than redistribute Type 1 external LSAs as those routes will cross area boundaries and defeat the purpose of summarizing at the ABR. True stub areas would help with this, but it wouldn't be a great idea to stub a network with multiple paths

The bigger question is whether or not OSPF is the right choice for a dual wan topology across 50 remote sites in the long term...my thoughts are that it isn't

The more I look at this, the I think BGP would solve most of the design challenges this network has more simply and with a greater flexibility down the road. Since you will be converting 50 something offices into this design, you really should consider something more flexible than OSPF - especially if the WAN link speeds are not equal. I didn't recommend that at first because I thought you had 50 sites already deployed with OSPF similar to the diagram you posted and the reconfig would be massive

My revised recommendations are:

1) Take a test site and connect it via BGP instead of OSPF (you shouldn't need an EOIP tunnel for BGP
1) It is very common in larger private networks now to use an autonomous system number per site so pick an AS for the 1100s and then use a new AS for each remote site.

At this point, you can summarize, filter and move traffic onto either WAN link much more easily without worrying about the quirks of OSPF hierarchy/areas and if you like the way this routing setup works, then you can make it cookie cutter across the board. If not, then you can still make some improvements to OSPF.

CelticComms · Tue May 20, 2014 6:52 am

Fair point..I missed the redistribution, but that would be routing that's not OPSF

People often assume that ASBRs imply another routing protocol as you originally said whereas it only implies another routing process/source such as connected, statics etc. Anyway I had a chuckle .............

I asked some questions earlier (which I don't think have been answered) regarding the nature of the branch office networks and internet access. It is quite possible that what the OP is trying to achieve could be achieved using weighted static routes and gateway checks!

I use BGP, OSPF, EIGRP and some other routing protocols on a daily basis. I have no favorites. You could carry out the OPs requirements with either OSPF or BGP - and maybe you could even carry out with static routes. My view is simply that the requirements can be met with less effort, less of a learning curve and less load on those branch routers by using OSPF which the OP has already started to work with.

In case there is any confusion........ OSPF does not need multicast to form adjacencies - that is simply a config issue.

StubArea51 · Tue May 20, 2014 7:58 am

In case there is any confusion........ OSPF does not need multicast to form adjacencies - that is simply a config issue.

No confusion, just didn't see the benefit of using OSPF as it won't be much simpler in this network than BGP since OSPF either needs static neighbor definitions or EOIP. Then add the prospect of jockeying cost and/or policy routing if some routes need to be prioritized on different WAN links.

These days in most greenfield designs, we tend to use OSPF to provide loopback and transit subnet connectivity and BGP for all subnets that will carry traffic - it is simple and scalable from a handful of routers to thousands.

If you haven't listened to this podcast, it may shed some light on what is happening in OSPF design in large environments these days:

http://packetpushers.net/show-134-ospf- ... area-myth/

They do a good job of changing the old mindset of limiting an area to 50ish routers...citing examples of thousands of routers in an area. Worth a listen

CelticComms · Tue May 20, 2014 8:38 am

I don't think that anybody with experience has paid any attention to the old Cisco OSPF design "constraints" for quite a long time.

However, when considering the number of areas or routers per area the actual OSPF implementation and router processing performance are still relevant considerations. I rarely see a modern Cisco or Juniper router tie itself up in knots running SPF but it can certainly happen on lower end hardware.

LuizMeier · Tue May 20, 2014 2:30 pm

My revised recommendations are:

1) Take a test site and connect it via BGP instead of OSPF (you shouldn't need an EOIP tunnel for BGP
1) It is very common in larger private networks now to use an autonomous system number per site so pick an AS for the 1100s and then use a new AS for each remote site

Point taken. We're doing the tests. I'm not familiar to BGP yetm, so I'm studying. For example, I didn't know that with BGP it wouldn' be necessary an adjacency like OSPF.

We're deploying OSPF, so it makes more easier to change the modus operandi now.

I asked some questions earlier (which I don't think have been answered) regarding the nature of the branch office networks and internet access. It is quite possible that what the OP is trying to achieve could be achieved using weighted static routes and gateway checks!

We have in each branch office one MPLS(EoIP) and one Internet(PPTP) link. In the past we used to use scripts to check the gateways and change the priority of routes. But with the enlargement of our network, it began to seem a little bit dumb to suffer with 50 scritps on HQ than make a dynamic topology.
In any case, i'm listenning all ideas.

CelticComms · Tue May 20, 2014 2:35 pm

We have in each branch office one MPLS(EoIP) and one Internet(PPTP) link. In the past we used to use scripts to check the gateways and change the priority of routes. But with the enlargement of our network, it began to seem a little bit dumb to suffer with 50 scritps on HQ than make a dynamic topology.
In any case, i'm listenning all ideas.

Do the branch offices internet sites such as Gmail, Yahoo etc. directly via the local Internet connection - or via head office?
What is the nature of the local branch networks? No of hosts? Typical number of subnets?
What network types did you try between the branch and head office for OSPF?

LuizMeier · Tue May 20, 2014 2:57 pm

We have in each branch office one MPLS(EoIP) and one Internet(PPTP) link. In the past we used to use scripts to check the gateways and change the priority of routes. But with the enlargement of our network, it began to seem a little bit dumb to suffer with 50 scritps on HQ than make a dynamic topology.
In any case, i'm listenning all ideas.

Do the branch offices internet sites such as Gmail, Yahoo etc. directly via the local Internet connection - or via head office?

The branch offices access the internet through HQ, so they will pass over our firewall to get the traffic filtered.

What is the nature of the local branch networks? No of hosts? Typical number of subnets?

They are /24 (we have just 4 or 5 with /23). The number of hosts depends from each to each site. The range would be 20-200 hosts from smaller to bigger site.

What network types did you try between the branch and head office for OSPF?

I didn't get it. You say types of tunneling? They were all MPLS (provided by ISP) since I'm here. Now that we finished the deploy of secondary links(and installed border 2011's) on each branch we started to play with dynamic protocols. EIGRP would be another great choice, but our routers are Mikrotik.
Once I'm not familiarized with BGP(now i'm studying it), started to test OSPF.

CelticComms · Tue May 20, 2014 9:23 pm

The branch offices access the internet through HQ, so they will pass over our firewall to get the traffic filtered.

Are you securing the path via the ISP's MPLS? If so how?

LuizMeier · Tue May 20, 2014 9:43 pm

The branch offices access the internet through HQ, so they will pass over our firewall to get the traffic filtered.
Are you securing the path via the ISP's MPLS? If so how?

No, I'm not. I assumed that It would not be necessary, once the traffic in this link is only ours(at least we pay ISP for that!). The traffic between branches over MPLS is not secured for anything but firewall. There is no crypto on this link.

CelticComms · Tue May 20, 2014 11:02 pm

No, I'm not. I assumed that It would not be necessary, once the traffic in this link is only ours(at least we pay ISP for that!). The traffic between branches over MPLS is not secured for anything but firewall. There is no crypto on this link.

Well it all depends how you analyse your risk, but L2 or L3 VPNs via your ISP can be intercepted in the same way that traffic flowing over the wider internet can. It is probably true that fewer people potentially have access, but what guarantees do you have as to who those people are? In other words who do you want to trust?

Given how some wide area MPLS traffic is handled an increasing number of customers are heading our advice and encrypting such links once they realise that they really have no more reason to trust their MPLS provider than they have to trust a tier 1 internet provider.

StubArea51 · Tue May 20, 2014 11:46 pm

No, I'm not. I assumed that It would not be necessary, once the traffic in this link is only ours(at least we pay ISP for that!). The traffic between branches over MPLS is not secured for anything but firewall. There is no crypto on this link.
Well it all depends how you analyse your risk, but L2 or L3 VPNs via your ISP can be intercepted in the same way that traffic flowing over the wider internet can. It is probably true that fewer people potentially have access, but what guarantees do you have as to who those people are? In other words who do you want to trust?

Given how some wide area MPLS traffic is handled an increasing number of customers are heading our advice and encrypting such links once they realise that they really have no more reason to trust their MPLS provider than they have to trust a tier 1 internet provider.

Any examples of a compromise in a an MPLS L2/L3 VPN you are aware of? I've been working on carrier MPLS networks for years and have yet to see a breach of an MPLS L2/L3 VPN. Furthermore, many providers employ CE to CE encryption and PE to PE encryption.

Not saying it couldn't happen, but what is the level of effort vs. the level of risk here? Breaking into a customer VRF is not an attack I would equate with the level of effort involved in an Internet based attack.

CelticComms · Wed May 21, 2014 3:11 am

Any examples of a compromise in a an MPLS L2/L3 VPN you are aware of? I've been working on carrier MPLS networks for years and have yet to see a breach of an MPLS L2/L3 VPN. Furthermore, many providers employ CE to CE encryption and PE to PE encryption.

To the first point - yes. As always it depends on you you want to "trust" and we see an increase in the number of organisations which are unwilling to trust carrier provided "secure" VPNs. Of course carriers really don't want to see a a trend in that direction. Those corporate "secure" VPNs are a nice cash cow!

Perhaps inevitably some organisations are re-examining the value of the supposed risk transfer element in such offerings.

StubArea51 · Wed May 21, 2014 3:48 am

To the first point - yes.

Would love to read about it

Links?

Security is hardly the only reason MPLS transit is so popular and that won't be going away anytime soon even if customers run IPSEC over the links. Guaranteed SLAs on latency and bandwidth as well as honoring DSCP markings in a priority queue for voice and video will continue to grow MPLS offerings. MPLS is extremely secure without encryption by using VRFs and inherent anti label-spoofing mechanisms. Add to that the inherent difficulty in getting logical/physical access to the infrastructure to carry out such an attack and the threat vector goes down considerably. Here is an example of technologies that can be used by Carriers to secure CE to CE communications over an MPLS network.

http://www.cisco.com/c/dam/en/us/produc ... ternal.pdf

It would be an interesting exercise to implement IPSEC on the transit links of a MikroTik based MPLS network and measure the performance.

To the OP, I would probably focus on solving your routing problems before worrying about the security of your data on a private MPLS network...the time, effort and added complexity of encrypting all of your private MPLS links is not nearly as important as building a clean routed network that can be scaled and will fail quickly and in a predictable manner.

CelticComms · Wed May 21, 2014 4:36 am

Would love to read about it Links?

Not something people generally want to publicize! It is an interesting topic of conversation at Cisco Live though. Of course MPLS is a great service offering and we use and implement it extensively. What is sometimes questioned is whether to "trust" the carrier or take direct responsibility for protecting confidentiality.

StubArea51 · Wed May 21, 2014 4:57 am

Not something people generally want to publicize! It is an interesting topic of conversation at Cisco Live though. Of course MPLS is a great service offering and we use and implement it extensively. What is sometimes questioned is whether to "trust" the carrier or take direct responsibility for protecting confidentiality.

Typically security vulnerabilities enjoy a great deal of transparency and attention these days. Compromises that affect customer data are disclosed fairly rapidly to limit corporate liability. Been following Live this week in bits and pieces, but hadn't seen anything on yet ...I'm assuming you're talking about Cisco Live 2014?

Just out of curiosity, at what scale are you encrypting MPLS transport links in customer networks? Link speeds?

LuizMeier · Wed May 21, 2014 3:59 pm

Hello!

I have some doubts about BGP. I was reading this article and I would appreciate some tips:

1) Will I use the same AS number for all routers, right?
2) Can I use a remote address of peer, even if it isn't directly connected(physical or through any tunnel)?
3) Should I use, in my case, the distribution of routes by BGP, right? I ask because of this quotation:

On the other hand, BGP networks are not installed in main routing table.

---------------------------
edited

Ok guys, I started with tests, and once the thing is gonna change a little, I think you need to see the topology on a macro view. This is a diagram of new topology.

Older ones with static routes have MPLS's router with 10.25.x.254 and 2011 with .20, with a bridge over ether1-lan and ether10-mpls. In case of MPLs problems, we brake the bridge, put .254 on lan interface and change de metric of route to come by pptp(and obviously change the route at R1). We are changing this as we are deploying OSPF.

1) FG, ISP Router(Cisco) and both RB1200 talk in Vlan11.
2) Vlan25 it's only for conversation between FG and R1.
3) FG has an instance of OSPF publishing the 10.25.0.0/23 route and learning other routes from R1. Today he has an static route to 10.25.0.0/16 with gateway = R1.
4) R2 has static routes to 2011's ether10-mpls addresses(10.254.x.253) to get EoIP established. In the same way, the 2011s has static routes to 10.254.254.5.
5) R1 is always 172.20.1.1 for tunnels, and each 2011 is 172.20.1.x/32.

For BGP to work, I understand that I will still have the static routes(on R2) to MPLS interface of 2011's. On R1, once I will configure the peer with PPTP address, the R1 already know how to get there.
Another question is: Can I advertise a route that is not directly connected? I did it in R1 and R2(for network 10.25.0.0/23) and it advertised. It worked because they already know by OSPF how to get 10.25.0.0/23?
This is a simple setup I've done:

R1

ros code

/routing bgp instance
set default router-id=10.255.255.255
/routing bgp network
add network=10.25.0.0/23 synchronize=no
/routing bgp peer
add name=espaco-parque remote-address=10.254.254.5 remote-as=65530 ttl=default
add name=bari-joinville remote-address=172.20.1.26 remote-as=65530 ttl=default

R2

ros code

/routing bgp instance
set default router-id=10.255.255.250
/routing bgp network
add network=10.25.0.0/23 synchronize=no
/routing bgp peer
add name=barigui-matriz remote-address=10.254.254.4 remote-as=65530 ttl=default
add name=bari-joinville remote-address=10.254.35.253 remote-as=65530 ttl=default

R3

ros code

/routing bgp network
add network=10.25.35.0/24 synchronize=no
/routing bgp peer
add name=barigui-matriz remote-address=172.20.1.1 remote-as=65530 ttl=default
add name=espaco-parque remote-address=10.254.254.5 remote-as=65530 ttl=default

[admin@RB2011-BARI-JOINVILLE] > ip route print where bgp
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme,
B - blackhole, U - unreachable, P - prohibit
# DST-ADDRESS PREF-SRC GATEWAY DISTANCE
32 Db 10.25.0.0/23 10.254.254.5 200
31 Db 10.25.0.0/23 172.20.1.1 200

Wich route would it choose? I will need some tips about tunning.

StubArea51 · Wed May 21, 2014 10:33 pm

Looks like you are making progress!

1) Use the same AS for both of your centralized 1100 routers and peer them to each other.
2) Use a different AS number for each remote site - this will simplify traffic management and routing.
3) As long as the BGP peer has a route to the other peer, they don't have to be directly connected...just be sure to turn on EBGP MultiHop. This can even be a /32 route so you don't have to worry about multiple default routes and let BGP advertise those.

I'll see if I can dig up a comparable network diagram of what you're trying to do so you can see an example.

LuizMeier · Wed May 21, 2014 10:46 pm

Looks like you are making progress!

1) Use the same AS for both of your centralized 1100 routers and peer them to each other.
2) Use a different AS number for each remote site - this will simplify traffic management and routing.
3) As long as the BGP peer has a route to the other peer, they don't have to be directly connected...just be sure to turn on EBGP MultiHop. This can even be a /32 route so you don't have to worry about multiple default routes and let BGP advertise those.

I'll see if I can dig up a comparable network diagram of what you're trying to do so you can see an example.

Should I maintain OSPF in HQ?

Didn't understand number 3. I must configure the static route to peers (it was already configured with /32) and let BGP advertise those /32 routes? Is it necessary?

Strange, after I configured multihop and AS numbers, the distance of the routes became 20. Before it was 200.

StubArea51 · Wed May 21, 2014 11:15 pm

If you only have the two 1100s and aren't going to expand much more than that, then I wouldn't worry about OSPF in HQ long term unless you have multiple paths between the two routers.

When you have multiple paths between routers and you need extremely rapid convergence, you can use OPSF and BGP together but this adds an element of complexity that probably isn't needed in your network - at least right now.

Here is a link to a presentation we did at USA MUM 2013 on that type of design.

http://mum.mikrotik.com/presentations/US13/kevin.pdf

As to your other questions, if you already have a /32 route to the far end peer, then you don't need to add one. I was just letting you know that you don't need a default route just for the BGP peering. The Administrative distance changed because you changed from iBGP to eBGP which is expected and normal.

CelticComms · Wed May 21, 2014 11:26 pm

Typically security vulnerabilities enjoy a great deal of transparency and attention these days.

On the contrary, actual security breach events that are are disclosed due to PII issues and jurisdictional requirements on disclosure represent the tip of the iceberg.

Just out of curiosity, at what scale are you encrypting MPLS transport links in customer networks? Link speeds?

Link speed in many cases but the pockets need to be deep in proportion to the link speed.

. In many cases there is a requirement to provide end-to-end security so trusting MPLS is not an option. If it is secure that is good - another layer. That topic came up again at a breakfast meeting today and burned up another 45 minutes. The equipment prices for high speed links are high. It would be nice if something like the CCR could bring the price point down one day.

The security subject is interesting but has become a distraction on this thread and I now realize that I never did point out a few easy steps for the OP to correct their OSPF installation and see the desired results, so I think the security discussion should move elsewhere.

LuizMeier · Wed May 21, 2014 11:48 pm

As to your other questions, if you already have a /32 route to the far end peer, then you don't need to add one. I was just letting you know that you don't need a default route just for the BGP peering.

You're saying that I wouldn't need at least one static route to 10.254.35.253/32 to get BGP established? I don't see how the router would get there, assuming that wasn't any OSPF running, by example.

StubArea51 · Thu May 22, 2014 1:00 am

As to your other questions, if you already have a /32 route to the far end peer, then you don't need to add one. I was just letting you know that you don't need a default route just for the BGP peering.
You're saying that I wouldn't need at least one static route to 10.254.35.253/32 to get BGP established? I don't see how the router would get there, assuming that wasn't any OSPF running, by example.

No just the opposite - a /32 route to reach the peer is great! Why don't you post your route table on the 2011 and 1100 to see what it looks like now.

LuizMeier · Thu May 22, 2014 3:51 pm

As to your other questions, if you already have a /32 route to the far end peer, then you don't need to add one. I was just letting you know that you don't need a default route just for the BGP peering.
You're saying that I wouldn't need at least one static route to 10.254.35.253/32 to get BGP established? I don't see how the router would get there, assuming that wasn't any OSPF running, by example.
No just the opposite - a /32 route to reach the peer is great! Why don't you post your route table on the 2011 and 1100 to see what it looks like now.

[admin@RB2011-BARI-JOINVILLE] > ip route print where bgp
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme,
B - blackhole, U - unreachable, P - prohibit
# DST-ADDRESS PREF-SRC GATEWAY DISTANCE
3 ADb 0.0.0.0/0 172.20.1.1 20
7 ADb 10.25.0.0/23 172.20.1.1 20

[admin@RB1100AHx2-MTZ] > ip route print where bgp
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme,
B - blackhole, U - unreachable, P - prohibit
# DST-ADDRESS PREF-SRC GATEWAY DISTANCE
0 ADb 10.25.35.0/24 172.20.1.26 20
1 Db 10.25.35.0/24 10.254.35.253 200

[admin@RB1200-MTZ-BKP] > ip route print where bgp
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme,
B - blackhole, U - unreachable, P - prohibit
# DST-ADDRESS PREF-SRC GATEWAY DISTANCE
0 Db 0.0.0.0/0 10.254.254.4 200
1 Db 10.25.0.0/23 10.254.254.4 200
2 ADb 10.25.35.0/24 10.254.35.253 20
3 Db 10.25.35.0/24 172.20.1.26 200

I'm having some trouble with the choice of best path. He is choosing the PPTP path, and I'd like it to choose MPLS path.

StubArea51 · Thu May 22, 2014 6:25 pm

A couple of things:

Please post all routes (not just filtered for BGP)

What is the source and destination of the traffic that isn't taking the path you would like?

LuizMeier · Thu May 22, 2014 7:02 pm

You see there is 2 routes for 10.25.35.0, one with 200 and other with 20 distance. Even with 20/200 on both he chooses PPTP. He turned 200 when I removed multihope on PPTP.

Can't I put by myself the weight for the preferred path?

[admin@RB2011-BARI-JOINVILLE] > ip route print
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme,
B - blackhole, U - unreachable, P - prohibit
# DST-ADDRESS PREF-SRC GATEWAY DISTANCE
0 A S 0.0.0.0/0 pppoe-gvt 1
1 A S 0.0.0.0/0 pptp-barigui 1
2 A S 0.0.0.0/0 10.254.35.254 1
3 ADb 0.0.0.0/0 172.20.1.1 20
4 A S ;;; update-route-vpn: DNS servers
8.8.4.4/32 pppoe-gvt 1
5 A S ;;; update-route-vpn: DNS servers
8.8.8.8/32 pppoe-gvt 1
6 ADC 10.1.35.0/30 10.1.35.2 eoip-barigui 0
7 ADb 10.25.0.0/23 172.20.1.1 20
8 ADC 10.25.35.0/24 10.25.35.254 bridge-lan 0
9 ADC 10.254.35.252/30 10.254.35.253 ether10-mpls-gvt 0
10 A S 10.254.254.5/32 10.254.35.254 1
11 ADC 10.255.35.255/32 10.255.35.255 loopback 0
12 ADC 172.20.1.1/32 172.20.1.26 pptp-barigui 0
13 ADC 172.30.1.0/24 172.30.1.1 vlan2-clientes 0
14 ADC 177.132.156.1/32 177.132.180.155 pppoe-gvt 0
15 A S ;;; update-route-vpn: pptp barigui.dyndns.org
177.220.176.252/32 pppoe-gvt 1
16 ADC 192.168.88.0/24 192.168.88.1 bridge-lan 0
[admin@RB2011-BARI-JOINVILLE] > tool trace 10.25.0.139
# ADDRESS LOSS SENT LAST AVG BEST WORST STD-DEV STATUS
1 172.20.1.1 0% 7 11.2ms 9.4 8.4 11.2 0.9
2 10.254.254.18 0% 7 24.2ms 12.9 9.4 24.2 4.7
3 10.25.0.139 0% 7 9.1ms 9.9 9.1 10.5 0.4

# may/22/2014 12:55:14 by RouterOS 6.11
# software id = S6VZ-27KC
#
Flags: X - disabled, A - active, D - dynamic,
C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme,
B - blackhole, U - unreachable, P - prohibit
# DST-ADDRESS PREF-SRC GATEWAY DISTANCE
0 A S 0.0.0.0/0 pppoe-copel 1
1 A S 0.0.0.0/0 200.150.102.113 1
2 A S 0.0.0.0/0 200.175.10.33 1
3 A S 0.0.0.0/0 10.254.254.18 1
4 A S 10.1.2.0/24 pptp-center-mar... 1
5 ADo 10.1.6.0/30 10.254.254.5 110
6 ADo 10.1.10.0/30 10.254.254.5 110
7 ADo 10.1.11.0/30 10.254.254.5 110
8 ADo 10.1.12.0/30 10.254.254.5 110
9 ADo 10.1.18.0/30 10.254.254.5 110
10 ADo 10.1.24.0/30 10.254.254.5 110
11 ADo 10.1.29.0/30 10.254.254.5 110
12 ADo 10.1.35.0/30 10.254.254.5 110
13 ADo 10.1.54.0/30 10.254.254.5 110
14 ADo 10.2.10.0/29 10.254.254.5 110
15 A S 10.24.0.0/14 10.254.254.18 1
16 ADo 10.25.0.0/23 10.254.254.18 110
17 A S ;;; Barigui Bettega
10.25.5.0/24 10.254.254.3 1
18 ADo 10.25.6.0/24 10.254.254.5 110
19 A S ;;; Center Sao Jose
10.25.7.0/24 10.254.254.3 1
20 A S ;;; Barigui Torres
10.25.8.0/23 10.254.254.3 1
21 ADo 10.25.10.0/24 10.254.254.5 110
22 ADo 10.25.11.0/24 10.254.254.5 110
23 ADo 10.25.12.0/24 10.254.254.5 110
24 A S ;;; Financeira
10.25.13.0/24 10.26.5.254 1
25 A S ;;; Aster SP
10.25.14.0/24 172.20.1.13 1
26 A S ;;; Barigui Caninana
10.25.15.0/24 172.20.1.4 1
27 A S ;;; Kia Unika
10.25.16.0/24 10.254.254.3 1
28 A S ;;; CIS
10.25.17.0/24 10.254.254.5 1
29 ADo 10.25.18.0/24 10.254.254.5 110
30 A S ;;; Center Maringa
10.25.19.0/24 10.254.254.3 1
31 A S ;;; Bari Balneario
10.25.20.0/24 172.20.1.8 1
32 A S ;;; Renault Minuto
10.25.21.0/24 172.20.1.11 1
33 A S ;;; DAF Sao José
10.25.22.0/24 172.20.1.15 1
34 A S ;;; Promotora
10.25.23.0/24 172.20.1.17 1
35 ADo 10.25.24.0/23 10.254.254.5 110
36 A S ;;; Formula Maringa
10.25.27.0/24 10.254.254.3 1
37 A S ;;; Carro Facil Joinville
10.25.28.0/24 172.20.1.18 1
38 ADo 10.25.29.0/24 10.254.254.5 110
39 A S ;;; Nix Blumenau
10.25.30.0/24 10.254.254.3 1
40 A S ;;; Formula Boa Vista
10.25.31.0/24 10.254.254.3 1
41 A S ;;; Vox/Nix Joinville
10.25.32.0/23 10.254.254.3 1
42 ADb 10.25.35.0/24 172.20.1.26 20
43 Db 10.25.35.0/24 10.254.35.253 200
44 A S ;;; Barigui Campo Largo
10.25.36.0/24 10.254.254.3 1
45 A S ;;; Center XV
10.25.37.0/24 10.254.254.3 1
46 A S ;;; Formula Tiradentes
10.25.39.0/24 10.254.254.3 1
47 A S ;;; Carro Facil Colombo
10.25.40.0/24 10.254.254.3 1
48 A S ;;; Maiori Portao
10.25.41.0/24 10.254.254.3 1
49 A S ;;; Financeira Maringa
10.25.42.0/24 10.254.254.3 1
50 A S ;;; Nix Jaragua
10.25.43.0/24 10.254.254.3 1
51 A S ;;; Vox Matriz
10.25.44.0/24 172.20.1.3 1
52 A S ;;; Maiori Balneario
10.25.45.0/24 10.254.254.3 1
53 ADo 10.25.46.0/24 10.254.254.5 110
54 A S ;;; Nix Brusque
10.25.47.0/24 10.254.254.3 1
55 A S ;;; Formula Apucarana
10.25.48.0/24 10.254.254.3 1
56 A S ;;; Espaço Torres
10.25.49.0/24 10.254.254.3 1
57 A S ;;; Vox Bingão
10.25.52.0/24 10.254.254.3 1
58 A S ;;; Vox Barracão
10.25.53.0/24 10.254.254.3 1
59 ADo 10.25.54.0/24 10.254.254.5 110
60 A S ;;; Nix Joinville - SN
10.25.55.0/24 172.20.1.5 1
61 A S ;;; Vlan-Servidores
10.25.250.0/23 10.254.254.18 1
62 A S ;;; Financeira Copel
10.26.4.254/32 10.26.5.254 1
63 ADC 10.26.5.0/24 10.26.5.250 vlan12-copel-fin 0
64 A S ;;; VW
10.112.0.0/16 pptp-vox-joinville 1
65 A S ;;; VW
10.135.0.0/16 pptp-vox-joinville 1
66 A S ;;; VW
10.244.130.0/23 pptp-vox-joinville 1
67 A S ;;; GVT
10.254.0.0/16 10.254.254.3 1
68 ADC 10.254.254.0/28 10.254.254.4 vlan11-mpls-gvt 0
69 ADC 10.254.254.16/30 10.254.254.17 vlan25-fortigat... 0
70 ADo 172.20.1.1/32 10.254.254.5 110
71 ADC 172.20.1.2/32 172.20.1.1 pptp-center-mar... 0
72 ADC 172.20.1.3/32 172.20.1.1 pptp-vox-matriz 0
73 ADC 172.20.1.4/32 172.20.1.1 pptp-brauto-itajai 0
74 ADC 172.20.1.5/32 172.20.1.1 pptp-nixsn-join... 0
75 ADC 172.20.1.6/32 172.20.1.1 pptp-center-xv 0
76 ADC 172.20.1.7/32 172.20.1.1 pptp-vox-joinville 0
77 ADC 172.20.1.8/32 172.20.1.1 pptp-bari-balne... 0
78 ADC 172.20.1.9/32 172.20.1.1 pptp-bari-blumenau 0
79 ADC 172.20.1.10/32 172.20.1.1 pptp-center-blu... 0
80 ADC 172.20.1.11/32 172.20.1.1 pptp-renault-mi... 0
81 ADC 172.20.1.12/32 172.20.1.1 pptp-nix-jaragua 0
82 ADC 172.20.1.13/32 172.20.1.1 pptp-aster-sp 0
83 ADC 172.20.1.14/32 172.20.1.1 pptp-formula-ma... 0
84 ADC 172.20.1.15/32 172.20.1.1 pptp-daf-sjp 0
85 ADC 172.20.1.16/32 172.20.1.1 pptp-nix-brusque 0
86 ADC 172.20.1.17/32 172.20.1.1 pptp-promotora 0
87 ADC 172.20.1.18/32 172.20.1.1 pptp-brauto-joi... 0
88 ADC 172.20.1.19/32 172.20.1.1 pptp-center-mar... 0
89 ADC 172.20.1.20/32 172.20.1.1 pptp-barigui-vale 0
90 ADC 172.20.1.21/32 172.20.1.1 pptp-bari-itajai 0
91 ADC 172.20.1.22/32 172.20.1.1 pptp-barigui-mi... 0
92 ADC 172.20.1.23/32 172.20.1.1 pptp-nix-blumenau 0
93 ADC 172.20.1.24/32 172.20.1.1 pptp-center-sjp 0
94 ADC 172.20.1.25/32 172.20.1.1 pptp-vox-barracao 0
95 ADC 172.20.1.26/32 172.20.1.1 pptp-bari-joinv... 0
96 ADC 172.20.1.27/32 172.20.1.1 pptp-kiaunika 0
97 ADC 172.20.1.29/32 172.20.1.1 pptp-formula-bo... 0
98 ADC 172.20.1.30/32 172.20.1.1 pptp-vox-bingao 0
99 ADC 172.20.1.31/32 172.20.1.1 pptp-barigui-xv 0
100 ADC 172.20.1.32/32 172.20.1.1 pptp-formula-ti... 0
101 ADC 172.20.1.33/32 172.20.1.1 pptp-barigui-it... 0
102 ADC 172.20.1.34/32 172.20.1.1 pptp-espaco-torres 0
103 ADC 172.20.1.35/32 172.20.1.1 pptp-maiori-portao 0
104 ADC 172.26.0.0/24 172.26.0.250 vlan22-dmz 0
105 ADC 172.30.1.0/24 172.30.1.1 ether1-vlan50 0
106 ADC 172.30.2.0/24 172.30.2.1 vlan51-clientes... 0
107 ADC 172.30.3.0/24 172.30.3.1 vlan52-clientes... 0
108 ADC 172.30.4.0/24 172.30.4.1 vlan53-clientes... 0
109 ADC 192.168.1.0/24 192.168.1.1 bridge-navegacao 0
110 ADC 200.150.95.28/32 177.220.176.252 pppoe-copel 0
111 ADC 200.150.102.112/29 200.150.102.116 vlan16-internet... 0
112 ADC 200.175.10.32/28 200.175.10.41 vlan2-internet-... 0

[admin@RB1200-MTZ-BKP] > ip route print
Flags: X - disabled, A - active, D - dynamic,
C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme,
B - blackhole, U - unreachable, P - prohibit
# DST-ADDRESS PREF-SRC GATEWAY DISTANCE
0 A S 0.0.0.0/0 pppoe-gvt 1
1 ADo 0.0.0.0/0 10.254.254.4 110
2 Db 0.0.0.0/0 10.254.254.4 200
3 ADC 10.1.6.0/30 10.1.6.1 eoip-barigui-xv 0
4 ADC 10.1.10.0/30 10.1.10.1 eoip-barigui-bl... 0
5 ADC 10.1.11.0/30 10.1.11.1 eoip-barigui-it... 0
6 ADC 10.1.12.0/30 10.1.12.1 eoip-center-blu... 0
7 ADC 10.1.17.0/29 10.1.17.1 vlan23-cis 0
8 ADC 10.1.18.0/30 10.1.18.1 eoip-bari-itajai 0
9 ADC 10.1.24.0/30 10.1.24.1 eoip-center-mar... 0
10 ADC 10.1.29.0/30 10.1.29.1 eoip-barigui-vale 0
11 ADC 10.1.35.0/30 10.1.35.1 eoip-bari-joinv... 0
12 ADC 10.1.54.0/30 10.1.54.1 eoip-bari-blumenau 0
13 ADo 10.2.10.0/29 10.1.10.2 110
14 ADo 10.25.0.0/23 10.254.254.4 110
15 Db 10.25.0.0/23 10.254.254.4 200
16 ADo 10.25.6.0/24 10.1.6.2 110
17 ADo 10.25.10.0/24 10.1.10.2 110
18 ADo 10.25.11.0/24 10.1.11.2 110
19 ADo 10.25.12.0/24 10.1.12.2 110
20 A S 10.25.17.0/24 10.1.17.2 1
21 ADo 10.25.18.0/24 10.1.18.2 110
22 ADo 10.25.24.0/23 10.1.24.2 110
23 ADo 10.25.29.0/24 10.1.29.2 110
24 ADb 10.25.35.0/24 10.254.35.253 20
25 Db 10.25.35.0/24 172.20.1.26 200
26 ADo 10.25.46.0/24 10.1.24.2 110
27 ADo 10.25.54.0/24 10.1.54.2 110
28 A S 10.254.6.253/32 10.254.254.3 1
29 A S 10.254.10.253/32 10.254.254.3 1
30 A S 10.254.11.253/32 10.254.254.3 1
31 A S 10.254.12.252/32 10.254.254.3 1
32 A S 10.254.18.253/32 10.254.254.3 1
33 A S 10.254.24.253/32 10.254.254.3 1
34 A S 10.254.29.253/32 10.254.254.3 1
35 A S 10.254.35.253/32 10.254.254.3 1
36 A S 10.254.54.253/32 10.254.254.3 1
37 ADC 10.254.254.0/28 10.254.254.5 vlan11-mpls-gvt 0
38 ADo 10.254.254.16/30 10.254.254.4 110
39 ADC 10.255.255.250/32 10.255.255.250 loopback 0
40 ADo 172.20.1.1/32 10.1.29.2 110
10.1.54.2
10.1.6.2
10.1.10.2
10.1.12.2
10.1.18.2
10.1.24.2
10.1.11.2
41 ADo 172.20.1.2/32 10.254.254.4 110
42 ADo 172.20.1.9/32 10.254.254.4 110
43 ADo 172.20.1.10/32 10.254.254.4 110
44 ADo 172.20.1.20/32 10.254.254.4 110
45 ADo 172.20.1.21/32 10.254.254.4 110
46 ADo 172.20.1.22/32 10.254.254.4 110
47 ADo 172.20.1.26/32 10.254.254.4 110
48 ADo 172.20.1.31/32 10.254.254.4 110
49 ADo 172.20.1.33/32 10.254.254.4 110
50 ADC 172.30.2.0/24 172.30.2.100 vlan51-navegacao 0
51 ADC 201.22.40.3/32 179.186.217.132 pppoe-gvt 0

StubArea51 · Fri May 23, 2014 4:34 am

You absolutely can chose which path you would like to select. Weight or Local Preference can be used to control outbound traffic and Prepending or MED can be used to control inbound traffic - there are many ways to get things done in BGP but those are the basic mechanisms.

Can you post the output of

ros code

routing bgp advertisements print

NetNotGross · Fri May 23, 2014 6:55 pm

To fix your original OSPF config you can either resditribute static routes from the hub routers (assuming a hierarchical IP layout) thus avoiding ABR/ASBR summarisation restrictions - or use multiple areas.

To use one area per branch:

1) Create a new area for each branch on both the branch router and hub routers. Place the branch connected routes and branch's WAN links in that area. Set the area to stub and no summary LSAs.

2) Manually create interfaces for the WAN links (don't use dynamic) and set the OSPF costs so that the primary is preferred. You can also set the link type to point to point to avoid the need for multicast. If you do the latter then also define the neighbors manually in NBMA neighbors.

This is a pretty ideal application for OSPF - convergence should be good in the event of link failure. You could also create summaries (area range) for certain routes depending on your IP numbering but since the branches will only see their own routes plus defaults there isn't much advantage.

LuizMeier · Mon May 26, 2014 4:56 pm

You absolutely can chose which path you would like to select. Weight or Local Preference can be used to control outbound traffic and Prepending or MED can be used to control inbound traffic - there are many ways to get things done in BGP but those are the basic mechanisms.

Can you post the output of

ros code
routing bgp advertisements print

[admin@RB1100AHx2-MTZ] > routing bgp advertisements print
PEER PREFIX NEXTHOP AS ORIGIN LOCAL-PREF
espac... 10.25.0.0/23 10.254.254.4 igp 100
espac... 0.0.0.0/0 10.254.254.4 igp 100
espac... 10.25.35.0/24 172.20.1.26 35 igp 100
bari-... 10.25.0.0/23 172.20.1.1 igp
bari-... 0.0.0.0/0 172.20.1.1 igp

[admin@RB1200-MTZ-BKP] > routing bgp advertisements print
PEER PREFIX NEXTHOP AS-PATH ORIGIN LOCAL-PREF
barig... 10.25.35.0/24 10.254.35.253 35 igp 100

[admin@RB2011-BARI-JOINVILLE] > routing bgp advertisements print
PEER PREFIX NEXTHOP AS-PATH ORIGIN LOCAL-PREF
barig... 10.25.35.0/24 172.20.1.26 igp
espac... 10.25.0.0/23 10.254.35.253 65530 igp
espac... 10.25.35.0/24 10.254.35.253 igp

To fix your original OSPF config you can either resditribute static routes from the hub routers (assuming a hierarchical IP layout) thus avoiding ABR/ASBR summarisation restrictions - or use multiple areas.

To use one area per branch:

1) Create a new area for each branch on both the branch router and hub routers. Place the branch connected routes and branch's WAN links in that area. Set the area to stub and no summary LSAs.

2) Manually create interfaces for the WAN links (don't use dynamic) and set the OSPF costs so that the primary is preferred. You can also set the link type to point to point to avoid the need for multicast. If you do the latter then also define the neighbors manually in NBMA neighbors.

This is a pretty ideal application for OSPF - convergence should be good in the event of link failure. You could also create summaries (area range) for certain routes depending on your IP numbering but since the branches will only see their own routes plus defaults there isn't much advantage.

I'm not using dynamic interfaces, I'm using them fixed and with password.
There is a concept on Cisco's routers that I think I didn't understand on RoouterOS. On Cisco, we can declare the networks that will be advertised on OSPF without the need of interface being declared, like we do in RouterOS. And after that, if needed, we configure passive interfaces. Can I advertise a route without declaring the interface?
The ideal would be the possibility of just advertise, in hub routers to the branchs, the route 10.25.0.0/16(and the default 0.0.0.0/0, that is already advertised in our actual topology,), that is the all the branchs need to work. They don't need to know all the middle-networks of all other tunnels.
Do I need to use NBMA neighbors even using different eoip and pptp tunnels for each connection?

NetNotGross · Mon May 26, 2014 6:34 pm

You can advertise networks simply by declaring them in OSPF networks. The system creates dynamic interfaces for any interfaces included within the network definitions. If you want more control over things like cost, network-type, authenticaton etc. then it is normal to create interfaces manually.

To have the branch area receive only a default route from the hub set the branch area to "inject-summary-lsas=no". That will make the ABR behave as a Cisco ABR would on a "totally" stub/nssa area. i.e. no summaries from other areas - just a default.

Whether you use stub or nssa for the branch area depends on whether you want to only advertise networks declared under OSPF networks (in which case stub) or want to include other items such as static routes (in which case nssa).

LuizMeier · Mon May 26, 2014 6:43 pm

You can advertise networks simply by declaring them in OSPF networks. The system creates dynamic interfaces for any interfaces included within the network definitions. If you want more control over things like cost, network-type, authenticaton etc. then it is normal to create interfaces manually.

To have the branch area receive only a default route from the hub set the branch area to "inject-summary-lsas=no". That will make the ABR behave as a Cisco ABR would on a "totally" stub/nssa area. i.e. no summaries from other areas - just a default.

Whether you use stub or nssa for the branch area depends on whether you want to only advertise networks declared under OSPF networks (in which case stub) or want to include other items such as static routes (in which case nssa).

When I declare a network on RB's OSPF and it creates a dynamic interface, it means that interface will start to advertise(considering no passive configurations), right? Can I declare an interface without declaring the network that is binded to it? Will RouterOs accept it and start to advertise all other networks declared?
I won't include static routes from HQ routers. There are many static routes that the branchs do not need to know.

NetNotGross · Mon May 26, 2014 7:03 pm

When I declare a network on RB's OSPF and it creates a dynamic interface, it means that interface will start to advertise(considering no passive configurations), right?

Correct.

Can I declare an interface without declaring the network that is binded to it? Will RouterOs accept it and start to advertise all other networks declared?

Remember - OSPF uses a nodal model and the links between the nodes need to be declared to be in an area, so the above wouldn't really make sense. You would not need to do this to achieve your goals so perhaps there is a misunderstanding?

LuizMeier · Tue May 27, 2014 2:36 pm

I must be doing something wrong.

Even making another area for the branch, like NetNotGross said, I am still receiving the routes from the backbone area. It is about 50 routes.

All the routes related to that branch were declared on new area.

ros code

/routing ospf network
add area=bari-joinville network=10.25.35.0/24
add area=bari-joinville network=10.1.35.0/30
add area=bari-joinville network=172.20.1.1/3

/routing ospf area
add area-id=1.1.1.35 default-cost=1 inject-summary-lsas=no name=bari-joinville type=stub

ros code

/routing ospf network
add area=backbone network=10.254.254.0/28
add area=backbone network=10.1.12.0/30
add area=backbone network=10.1.24.0/30
add area=bari-joinville network=10.1.35.0/30
add area=backbone network=10.1.54.0/30
add area=backbone network=10.1.18.0/30
add area=backbone network=10.1.10.0/30
add area=backbone network=10.1.29.0/30
add area=backbone network=10.1.11.0/30
add area=backbone network=10.1.6.0/30
add area=backbone network=10.1.8.0/30
add area=backbone network=10.1.19.0/30
add area=backbone network=10.1.7.0/30
add area=backbone network=10.1.17.0/29

/routing ospf area
add area-id=1.1.1.35 default-cost=1 inject-summary-lsas=yes name=bari-joinville type=stub

ros code

/routing ospf network
add area=backbone network=10.254.254.16/30
add area=backbone network=10.254.254.0/28
add area=backbone network=172.20.1.10/32
add area=backbone network=172.20.1.2/32
add area=bari-joinville network=172.20.1.26/32
add area=backbone network=172.20.1.22/32
add area=backbone network=172.20.1.9/32
add area=backbone network=172.20.1.21/32
add area=backbone network=172.20.1.20/32
add area=backbone network=172.20.1.33/32
add area=backbone network=172.20.1.31/32
add area=backbone network=172.20.1.36/32
add area=backbone network=172.20.1.24/32
add area=backbone network=172.20.1.19/32
add area=backbone network=172.20.1.37/32

/routing ospf area
add area-id=1.1.1.35 default-cost=1 inject-summary-lsas=no name=bari-joinville type=stub

NetNotGross · Tue May 27, 2014 8:23 pm

R2 seems to have inject-summary-lsas=yes on the stub area.

Check that the branch routers and their links from head office only have stub area settings on them now. The head office router should be carrying out the ABR function.

If above doesn't show a difference after correcting perhaps more config info with an indication of which router is which.

LuizMeier · Tue May 27, 2014 11:52 pm

R2 seems to have inject-summary-lsas=yes on the stub area.

Check that the branch routers and their links from head office only have stub area settings on them now. The head office router should be carrying out the ABR function.

If above doesn't show a difference after correcting perhaps more config info with an indication of which router is which.

You're right!

I think we got it for the summarization! Following is the routing table of one of branches:

[admin@RB2011-BARI-JOINVILLE] > ip route print where ospf
Flags: X - disabled, A - active, D - dynamic,
C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme,
B - blackhole, U - unreachable, P - prohibit
# DST-ADDRESS PREF-SRC GATEWAY DISTANCE
0 ADo 0.0.0.0/0 10.1.35.1 110
1 ADo 172.20.1.26/32 172.20.1.1 110

One doubt: The 0.0.0.0/0 route includes 10.1.35.0/30, that is binded to one of OSPF's interface? Is that the reason for not appearing in this output?

Although, the redundancy didn't work. when I closed the EoIP tunnel, the connection didn't migrate to PPTP.

NetNotGross · Wed May 28, 2014 1:43 pm

Check that both links are configured for OSPF and have their networks included in the branch area. On the branch router you should show 2 neighbor entries - one at end of each link. You can also check the body of the LSA and see if it shows both paths. Make sure that interface costs are assigned correctly too.

The routing table is moving in the right direction!

LuizMeier · Wed May 28, 2014 6:15 pm

Check that both links are configured for OSPF and have their networks included in the branch area. On the branch router you should show 2 neighbor entries - one at end of each link. You can also check the body of the LSA and see if it shows both paths. Make sure that interface costs are assigned correctly too.

The routing table is moving in the right direction!

Both links configured and neighboors seen on all routers, but still not redundancy. In this scenario (with an area for each branch), both hub routers know how to reach the branch, but only through its own tunnel with branch. R1 know by PPTP and R2 by EoIP.

But they are not speaking with each other about the best path, for example. PPTP should be the secondary link, so the R1 should pass the packet to R2, so R2 would send the packet through EoIP.
Once R1 is the firewall's GW, the traffic is going only by PPTP tunnel.

Tried with redistribute OSPF, but got no results with that.

NetNotGross · Wed May 28, 2014 6:34 pm

What costs do you have on the two links to the branch and on the link between the 1100 routers? The total cost of the path to the branch from either router must be lower on the EoIP path. e.g. if the cost of the path between the routers is 10 and the EoIP branch path is 20 then you would make the branch PPTP path something like 50 to ensure it is always least attractive from either router.

Very crude numbers to demonstrate the idea - always good to pick a standard cost basis and use it throughout the network.

Are both 1100s in area 0?

LuizMeier · Wed May 28, 2014 8:47 pm

What costs do you have on the two links to the branch and on the link between the 1100 routers? The total cost of the path to the branch from either router must be lower on the EoIP path. e.g. if the cost of the path between the routers is 10 and the EoIP branch path is 20 then you would make the branch PPTP path something like 50 to ensure it is always least attractive from either router.

Very crude numbers to demonstrate the idea - always good to pick a standard cost basis and use it throughout the network.

Are both 1100s in area 0?

Yes, the cost of PPTP is great than EoIP. I have 50 for PPTP and 10 for EoIP.

Both hub routers are in area 0 through vlan 11, as shown in my diagram. That's why I tried to distribute routes, so that they could distribute branchs area's routes to area 0.
R1 AND R2 need to know both ways, once R1 always participate of the traffic. It must route to R2 in case of EoIP preference.

NetNotGross · Wed May 28, 2014 8:58 pm

Both hub routers should end up knowing both routes within OSPF but only the currently active route (as determiend by OSPF) is inserted in the main routing table.

You can see which routes are being advertised to either of the hub routers by looking at LSAs.

I suggest that you upload the OSPF config elements from the three routers - sounds like it is close but something is not quite right.

LuizMeier · Thu May 29, 2014 9:36 pm

Thanks for your reply!

Both hub routers should end up knowing both routes within OSPF but only the currently active route (as determiend by OSPF) is inserted in the main routing table.

You can see which routes are being advertised to either of the hub routers by looking at LSAs.

I suggest that you upload the OSPF config elements from the three routers - sounds like it is close but something is not quite right.

Following is the configuration:

ros code

/routing ospf instance
set [ find default=yes ] distribute-default=always-as-type-1 router-id=10.255.255.255

/routing ospf area
add area-id=1.1.1.35 default-cost=1 inject-summary-lsas=no name=bari-joinville type=stub

/routing ospf network
add area=backbone network=10.254.254.16/30
add area=backbone network=10.254.254.0/28
add area=bari-joinville network=172.20.1.26/32

/routing ospf interface
add authentication=md5 authentication-key=XXXXX authentication-key-id=10 interface=vlan11-mpls-gvt network-type=broadcast
add authentication=md5 authentication-key=XXXXX authentication-key-id=10 interface=vlan25-fortigate-mikrotik network-type=broadcast
add authentication=md5 authentication-key=XXXXX authentication-key-id=10 cost=50 interface=pptp-bari-joinville network-type=point-to-point

/ip address
add address=10.254.254.4/28 comment="MPLS GVT" interface=vlan11-mpls-gvt network=10.254.254.0
add address=10.254.254.17/30 interface=vlan25-fortigate-mikrotik network=10.254.254.16

R1 is always 172.20.1.1/32 for all PPTP tunnels. All others PPTP tunnels are on backbone area. This one is just for test.

ros code

/interface eoip
add local-address=10.254.254.5 mac-address=02:0E:80:6B:D8:BD name=eoip-bari-joinville remote-address=10.254.35.253 tunnel-id=35

/ip address
add address=10.255.255.250/32 interface=loopback network=10.255.255.250
add address=10.254.254.5/28 interface=vlan11-mpls-gvt network=10.254.254.0
add address=10.1.35.1/30 interface=eoip-bari-joinville network=10.1.35.0

/ip route
add distance=1 dst-address=10.254.35.253/32 gateway=10.254.254.3

/routing ospf instance
set [ find default=yes ] router-id=10.255.255.250

/routing ospf area
add area-id=1.1.1.35 default-cost=1 inject-summary-lsas=no name=bari-joinville type=stub

/routing ospf network
add area=backbone network=10.254.254.0/28
add area=bari-joinville network=10.1.35.0/30

/routing ospf interface
add authentication=md5 authentication-key=XXX authentication-key-id=10 interface=eoip-bari-joinville network-type=point-to-point

All other EoIP tunnels are on backbone area.

ros code

/interface eoip
add local-address=10.254.35.253 mac-address=02:0E:E2:EF:88:EB name=eoip-barigui remote-address=10.254.254.5 tunnel-id=35

/ip address
add address=10.25.35.254/24 interface=ether1-lan network=10.25.35.0
add address=10.254.35.253/30 interface=ether10-mpls-gvt network=10.254.35.252
add address=10.255.35.255/32 interface=loopback network=10.255.35.255
add address=10.1.35.2/30 interface=eoip-barigui network=10.1.35.0
add address=172.30.1.1/24 interface=vlan2-clientes network=172.30.1.0

/ip route
add distance=1 dst-address=10.254.254.5/32 gateway=10.254.35.254

/routing ospf instance
set [ find default=yes ] router-id=10.255.35.255

/routing ospf area
add area-id=1.1.1.35 default-cost=1 inject-summary-lsas=no name=bari-joinville type=stub

/routing ospf network
add area=bari-joinville network=10.25.35.0/24
add area=bari-joinville network=10.1.35.0/30
add area=bari-joinville network=172.20.1.1/32

/routing ospf interface
add authentication=md5 authentication-key=XXXXX authentication-key-id=10 interface=eoip-barigui network-type=point-to-point
add authentication=md5 authentication-key=XXXXX authentication-key-id=10 interface=ether1-lan network-type=broadcast passive=yes
add authentication=md5 authentication-key=XXXXX authentication-key-id=10 cost=50 interface=pptp-barigui network-type=point-to-point
add interface=bridge-lan network-type=broadcast passive=yes

R1 is the default gateway of our firewall and contains the PPTP's tunnels. R2 is the EoIP's router and R2 is the branch.
Please look at my diagram on Wed May 21, 2014 9:59 am. That should help you with the topology.

NetNotGross · Fri May 30, 2014 6:54 pm

Can you upload the print output from /router ospf lsa on the branch router so that we can see its view. That should let us see the LSAs it is receiving at the moment.

LuizMeier · Fri May 30, 2014 8:42 pm

Can you upload the print output from /router ospf lsa on the branch router so that we can see its view. That should let us see the LSAs it is receiving at the moment.

[admin@RB2011-BARI-JOINVILLE] > routing ospf lsa print detail
instance=default area=bari-joinville type=router id=10.255.35.255 originator=10.255.35.255 sequence-number=0x80000003 age=11 checksum=0x5991
options="" body=
flags=
link-type=Point-To-Point id=10.255.255.255 data=172.20.1.26 metric=50
link-type=Stub id=172.20.1.1 data=255.255.255.255 metric=50
link-type=Stub id=10.25.35.0 data=255.255.255.0 metric=10
link-type=Point-To-Point id=10.255.255.250 data=10.1.35.2 metric=10
link-type=Stub id=10.1.35.0 data=255.255.255.252 metric=10

instance=default area=bari-joinville type=router id=10.255.255.250 originator=10.255.255.250 sequence-number=0x80000002 age=21 checksum=0xCAD4
options="" body=
flags=BORDER
link-type=Point-To-Point id=10.255.35.255 data=10.1.35.1 metric=10
link-type=Stub id=10.1.35.0 data=255.255.255.252 metric=10

instance=default area=bari-joinville type=router id=10.255.255.255 originator=10.255.255.255 sequence-number=0x80000002 age=15 checksum=0x7789
options="" body=
flags=BORDER
link-type=Point-To-Point id=10.255.35.255 data=172.20.1.1 metric=50
link-type=Stub id=172.20.1.26 data=255.255.255.255 metric=50

instance=default area=bari-joinville type=summary-network id=0.0.0.0 originator=10.255.255.250 sequence-number=0x80000001 age=29 checksum=0x5702
options="" body=
netmask=0.0.0.0
metric=1

instance=default area=bari-joinville type=summary-network id=0.0.0.0 originator=10.255.255.255 sequence-number=0x80000001 age=35 checksum=0x391B
options="" body=
netmask=0.0.0.0
metric=1

instance=default area=external type=as-external id=0.0.0.0 originator=10.255.255.255 sequence-number=0x800008A3 age=548 checksum=0xB1EB options="E"
body=
netmask=0.0.0.0
forwarding-address=0.0.0.0
metric=1
route-tag=0x0
type1

CelticComms · Tue Jun 17, 2014 7:21 pm

Did you resolve the OSPF branch office stub config?

LuizMeier · Tue Aug 05, 2014 4:04 pm

Did you resolve the OSPF branch office stub config?

Celtic,

Sorry by the delay on my response. I was on vacation, so no Mikrotiks for me on that period.

We "solve" the problem by filtering the routes on border's routers. I accepted only the routes I wanted there making sure that it would not brake the redundancy.
With BGP I had no success with redundancy, and I've tried both setups for OSPF, but none of them let me happy with the results.

I appreciate all the effort of all of you on this problem.

ros code

ros code

ros code

ros code

ros code

ros code

ros code

ros code

ros code

ros code

ros code

ros code

ros code

ros code

Who is online