MikroTik

Posted: **Fri Jan 16, 2015 8:57 pm**

Hello Guys!

After dealing with this problem, no the BGP have come back to me. My manager now wants to end with the EoIP tunnels, so we can make the things a little easier, and want me to implement BGP.

I am new at BGP and just have made some tests when I've started with the OSPF implementation. Now i'm making some new tests on a test lab. Attached to this post is my test topology, and below I put some background information and the results of my tests untill now, with problems.

Background
1) QEMU devices are my RB's;
2) The path through R2 and R2 supposes to ilustrate my ISP's MPLS topology, which I have no access. Just made an OSPF instance to get the routes between them working;
3) The QEMU1-QEMU3 path supposes to ilustrate an PPTP session through internet as a redundant link;
4) The SW2 and SW3 ilustrates my lan on HQ and branch office;
5) Today, I use all 3 RB's to get this topology working. QEMU2 is my EoIP tunnels and QEMU1 is my PPTP and core router.

QEMU1

[admin@MikroTik] > routing bgp export
# jan/16/2015 18:48:15 by RouterOS 5.20
# software id = W5EY-LHT9
#
/routing bgp instance
set default as=65530 client-to-client-reflection=yes disabled=no ignore-as-path-len=no name=default out-filter="" redistribute-connected=no \
    redistribute-ospf=no redistribute-other-bgp=no redistribute-rip=no redistribute-static=no router-id=0.0.0.0 routing-table=""
/routing bgp network
add disabled=no network=10.25.0.0/23 synchronize=yes
/routing bgp peer
add address-families=ip as-override=no default-originate=never disabled=no hold-time=3m in-filter="" instance=default multihop=no name=BKP \
    nexthop-choice=default out-filter="" passive=no remote-address=10.254.254.5 remote-as=65530 remove-private-as=no route-reflect=no tcp-md5-key=\
    "" ttl=255 use-bfd=no
add address-families=ip as-override=no default-originate=never disabled=no hold-time=3m in-filter="" instance=default multihop=no name=PPTP \
    nexthop-choice=default out-filter="" passive=no remote-address=172.20.1.2 remote-as=65535 remove-private-as=no route-reflect=no tcp-md5-key="" \
    ttl=255 use-bfd=no
[admin@MikroTik] > ip route export
# jan/16/2015 18:48:25 by RouterOS 5.20
# software id = W5EY-LHT9
#
/ip route
add disabled=no distance=1 dst-address=10.254.35.253/32 gateway=10.254.254.1 scope=30 target-scope=10
[admin@MikroTik] >
[admin@MikroTik] > ip route print
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme,
B - blackhole, U - unreachable, P - prohibit
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 ADC  10.25.0.0/23       10.25.1.254     ether2-lan                0
 1 ADb  10.25.35.0/24                      172.20.1.2               20
 2  Db  10.25.35.0/24                      10.254.35.253           200
 3 A S  10.254.35.253/32                   10.254.254.1              1
 4 ADC  10.254.254.0/29    10.254.254.4    ether1-wan                0
 5 ADC  172.20.1.0/30      172.20.1.1      ether3-pptp               0
[admin@MikroTik] > routing bgp peer print
Flags: X - disabled, E - established
 #   INSTANCE                                          REMOTE-ADDRESS                                                                    REMOTE-AS
 0 E default                                           10.254.254.5                                                                      65530
 1 E default                                           172.20.1.2                                                                        65535

QEMU2

[admin@MikroTik] > routing bgp export
# jan/16/2015 18:51:11 by RouterOS 5.20
# software id = W5EY-LHT9
#
/routing bgp instance
set default as=65530 client-to-client-reflection=yes disabled=no ignore-as-path-len=no name=default out-filter="" redistribute-connected=no \
    redistribute-ospf=no redistribute-other-bgp=no redistribute-rip=no redistribute-static=no router-id=0.0.0.0 routing-table=""
/routing bgp peer
add address-families=ip as-override=no default-originate=never disabled=no hold-time=3m in-filter="" instance=default multihop=no name=peer1 \
    nexthop-choice=default out-filter="" passive=no remote-address=10.254.254.4 remote-as=65530 remove-private-as=no route-reflect=no tcp-md5-key=\
    "" ttl=255 use-bfd=no
add address-families=ip as-override=no default-originate=never disabled=no hold-time=3m in-filter="" instance=default multihop=yes name=peer2 \
    nexthop-choice=default out-filter="" passive=no remote-address=10.254.35.253 remote-as=65535 remove-private-as=no route-reflect=no \
    tcp-md5-key="" ttl=255 use-bfd=no
[admin@MikroTik] > ip route export
# jan/16/2015 18:51:22 by RouterOS 5.20
# software id = W5EY-LHT9
#
/ip route
add disabled=no distance=1 dst-address=10.254.35.253/32 gateway=10.254.254.1 scope=30 target-scope=10
[admin@MikroTik] > ip route print
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme,
B - blackhole, U - unreachable, P - prohibit
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 ADb  10.25.0.0/23                       10.254.254.4            200
 1 ADb  10.25.35.0/24                      10.254.35.253            20
 2  Db  10.25.35.0/24                      172.20.1.2              200
 3 A S  10.254.35.253/32                   10.254.254.1              1
 4 ADC  10.254.254.0/29    10.254.254.5    ether1                    0
[admin@MikroTik] > routing bgp peer print
Flags: X - disabled, E - established
 #   INSTANCE                                          REMOTE-ADDRESS                                                                    REMOTE-AS
 0 E default                                           10.254.254.4                                                                      65530
 1 E default                                           10.254.35.253                                                                     65535

QEMU3

[admin@MikroTik] > routing bgp export
# jan/16/2015 18:52:03 by RouterOS 5.20
# software id = W5EY-LHT9
#
/routing bgp instance
set default as=65535 client-to-client-reflection=yes disabled=no ignore-as-path-len=no name=default out-filter="" redistribute-connected=no \
    redistribute-ospf=no redistribute-other-bgp=no redistribute-rip=no redistribute-static=no router-id=0.0.0.0 routing-table=""
/routing bgp network
add disabled=no network=10.25.35.0/24 synchronize=yes
/routing bgp peer
add address-families=ip as-override=no default-originate=never disabled=no hold-time=3m in-filter=entrada-mpls instance=default multihop=yes name=\
    MPLS nexthop-choice=default out-filter="" passive=no remote-address=10.254.254.5 remote-as=65530 remove-private-as=no route-reflect=no \
    tcp-md5-key="" ttl=255 use-bfd=no
add address-families=ip as-override=no default-originate=never disabled=no hold-time=3m in-filter=entrada-pptp instance=default multihop=no name=\
    PPTP nexthop-choice=default out-filter="" passive=no remote-address=172.20.1.1 remote-as=65530 remove-private-as=no route-reflect=no \
    tcp-md5-key="" ttl=255 use-bfd=no
[admin@MikroTik] > ip route export
# jan/16/2015 18:52:08 by RouterOS 5.20
# software id = W5EY-LHT9
#
/ip route
add disabled=no distance=1 dst-address=10.254.254.4/32 gateway=10.254.35.254 scope=30 target-scope=10
add disabled=no distance=1 dst-address=10.254.254.5/32 gateway=10.254.35.254 scope=30 target-scope=10
[admin@MikroTik] > ip route print
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme,
B - blackhole, U - unreachable, P - prohibit
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 ADb  10.25.0.0/23                       172.20.1.1               20
 1  Db  10.25.0.0/23                       10.254.254.4             20
 2 ADC  10.25.35.0/24      10.25.35.254    ether2-lan                0
 3 ADC  10.254.35.252/30   10.254.35.253   ether1-mpls               0
 4 A S  10.254.254.4/32                    10.254.35.254             1
 5 A S  10.254.254.5/32                    10.254.35.254             1
 6 ADC  172.20.1.0/30      172.20.1.2      ether3-pptp               0
[admin@MikroTik] > routing bgp peer print
Flags: X - disabled, E - established
 #   INSTANCE                                          REMOTE-ADDRESS                                                                    REMOTE-AS
 0 E default                                           10.254.254.5                                                                      65530
 1 E default                                           172.20.1.1                                                                        65530

My results:
1) I get the routes learned on all routers;
2) I feel some delay on BGP to see the link down;
3) When I shut the PPTP link down and the QEMU1 needs to go through MPLS, the ISP router says me that do not know the address that I am looking for. I though that BGP would delivery the package to the remote peer(which it knows how to reach by static route) and then would go to the destination network.
4) Testing using just one router, I so have problems with metrics, because os iBGP and eBGP.

Posted: **Mon Jan 19, 2015 1:12 am**

BGP will indeed typically be slower to respond than OSPF using default settings. I lost track of your original thread but it seemed at that time that your application was very well suited to OSPF - given that it is a link-state protocol and your application essentially came down to link states.

What are your goals at this point? Is there something specific that you want to achieve using BGP that couldn't be achieved in OSPF?

Posted: **Mon Jan 19, 2015 5:37 am**

If you must implement BGP you could look at turning on BFD to help with the timing issue. Won't solve everything but will help.

Posted: **Mon Jan 19, 2015 2:48 pm**

BGP will indeed typically be slower to respond than OSPF using default settings. I lost track of your original thread but it seemed at that time that your application was very well suited to OSPF - given that it is a link-state protocol and your application essentially came down to link states.

What are your goals at this point? Is there something specific that you want to achieve using BGP that couldn't be achieved in OSPF?

Celtic, my main point is to end with the EoIP tunnels. They add an aditional complexity that I would not like to have.

Is there any way to avoid these EoIp tunnels?

I've attached a diagram at this post.

Posted: **Mon Jan 19, 2015 3:20 pm**

To avoid any confusion, could you clarify how the diagram above relates to the branch office / head office scenario in the original thread?

From the original thread it seems that EoIP was only being used to address OSPF adjacency. Do you still have PPTP and MPLS links to each branch office? Which network types did you try when attempting OSPF directly over the provided MPLS network?

Posted: **Mon Jan 19, 2015 3:27 pm**

Celtic,

Sorry for the missing information.

1) HQ is the area from SW1 to SW2, includins QEMU2 and QEMU1;
2) Branch is the area with QEMU3;
3) The network 172.20.0.0/30 is to ilustrate PPTP;
4) EoIP runs over R1 and R2, which ilustrate the MPLS circuit provided by my ISP.

Yes, I still have MPLS and PPTP on mos of my Branch Offices. I have a few with just 2 PPTP's, but no problems with them.

Yes, EoIP is just to make adjacencies on OSPF. To make some tests, I've tried to configure it from point-to-point to nbma.

Posted: **Mon Jan 19, 2015 3:42 pm**

OK - so in reality how does the MPLS network present to you? In the diagram it shows as a /30 network to the branch. Did you ever confirm that the reason why adjacency could not be established was understood?

Posted: **Mon Jan 19, 2015 3:57 pm**

Celtic,

I presented it that way to make the topology easier to understand, but actually I have over 4 ISP's routers between my HQ and branch.

And yes, I understood (correct me if i'm wring) that I cannot make adjacencies on OSPF through that way, once OSPF only talks with neighbors that can receive the multicast conversation. On unicast, I could only make this to work if I had an nbma network (like fram-relay, X.25) or something like that.

Once I got questioned about the option of ending the EoIP's tunnel, my first though was BGP. If it could be made by OSPF, would be more easier.

Posted: **Mon Jan 19, 2015 4:46 pm**

If the MPLS network was providing Layer 2 connectivity then OSPF adjacencies would simply work. Assuming that what you have is a Layer 3 network via MPLS are you exchanging routing information with the provider's PE routers? If not can you and which protocols can they support?

Posted: **Mon Jan 19, 2015 6:32 pm**

They don't give this option to clients. Besides that, I don't trust them 100% to depend in case of problems.

I would like to leave the minimum of work on their hands.

Posted: **Mon Jan 19, 2015 6:40 pm**

OK - then can you describe exactly what the layer 3 situation is for Branch Office<>Head Office traffic - typical IP address arrangement and routing arrangement.

Posted: **Mon Jan 19, 2015 7:30 pm**

How we work today:

1) Traffic comes from WAN, passes through my firewall and then reachs QEMU1 Mikrotik, which is the PPTP manager. Each PPTP tunnel has 172.20.1.X/32 network.
2) QEMU1 is on the same subnet as QEMU2 (EoIP manager with 10.1.X.0/30 network for tunnel) and Cisco's ISP router.
3) The addressing scheme is the same as in the image I uploaded on my earlier post.
4) To connect EoIP, I have a static route from QEMU2 to QEMU3 through 10.254.254.1. Same thing in the other way.

So, as you can see, QEMU1 is the core router. It decides the best path to te branch offices. We manage the paths by setting the cost at the routers, once I have places where PPTP is better than MPLS and the other way at other places.

Below is a output of a trace through MPLS. You can see that there is at least 3 routers between me and my branches.

[admin@MTZ] > tool trace 10.254.35.253
 # ADDRESS                          LOSS SENT    LAST     AVG    BEST   WORST STD-DEV STATUS                              
 1 10.254.254.1                       0%    4   0.3ms     0.5     0.3     1.1     0.3                                     
 2 10.2.1.5                           0%    4   6.3ms     5.5     3.2     6.9     1.4                                     
 3                                  100%    4 timeout                                                                     
 4                                  100%    3 timeout                                                                     
 5 201.86.63.177                      0%    3  10.5ms    29.7    10.5    45.1    14.4 <MPLS:L=1024,E=0 L=704423,E=0,T=255>
 6 10.254.35.253                      0%    3   7.2ms    12.7     7.2    23.7     7.8

Posted: **Mon Jan 19, 2015 10:18 pm**

How do the branch/head officxes determine which traffic is sent to the MPLS network. Are there static routes set?

Posted: **Mon Jan 19, 2015 10:25 pm**

The only traffic that is sent directly to MPLS is the one to accomplish the EoIP tunnel.

There is static routes at the branch and the hq to make this tunnel working.

The core router decides the path between EoIP or PPTP by OSPF costs.

Posted: **Mon Jan 19, 2015 11:25 pm**

OK - it so it sounds as if they have provided an L3 MPLS cloud but with no dynamic routing. Often on a multi-site L3 MPLS cloud the PE routers will exchange routing information with a customer owned router thus the MPLS cloud knows which sites are up and down and signals the same to the other sites.

Is there any chance that your provider can provide Layer 2 functionality instead? That would get rid of the need for EoIP and remove the OSPF adjacency problems. If you are stuck with the MPLS functionality as it currently is then you may have to resort to BGP but you would also have to apply some further tricks to reduce the convergence times. That may not seem like "simplifying" the current EoIP arrangement. Is the MPLS provider a mainstream telco?

Posted: **Tue Jan 20, 2015 7:17 pm**

Yeah, i though about that, but will need to test anyway.

Actually I'm already doing it. My first problems were those on my first post.

Posted: **Mon Jan 26, 2015 9:15 pm**

Still the same thing here.

Tried using bridges instead of interface's addresses, but I still get the same problem. When I need to go through the multihop BGP path, the package don't get the destination because the ISP don't know how to reach it.

I though that BGP would send the package to the peer, and then to the endpoint.

Posted: **Tue Jan 27, 2015 3:06 pm**

I though that BGP would send the package to the peer, and then to the endpoint.

No - routes determine the next hop not the full path. Each router in between makes its own decision. This is why providers typically provide a VRF that you can interact with via a routing process on the PE router for a layer 3 private network. Without that you need to use some sort of tunneling which is effectively what EoIP is currently providing.

Is the current service the only type of service that the upstream can supply?

MikroTik

BGP Implementation

BGP Implementation

Re: BGP Implementation

Re: BGP Implementation

Re: BGP Implementation

Re: BGP Implementation

Re: BGP Implementation

Re: BGP Implementation

Re: BGP Implementation

Re: BGP Implementation

Re: BGP Implementation

Re: BGP Implementation

Re: BGP Implementation

Re: BGP Implementation

Re: BGP Implementation

Re: BGP Implementation

Re: BGP Implementation

Re: BGP Implementation

Re: BGP Implementation