❓ MPLS + VPLS + PPPOE Central (DataTransfer Problem on 3rd hops)

RouterOS General
1

Hi friends.
I’m now use a central pppoe server with EoIP tunnels and now want change my tunnels to VPLS .
This is my sample network and configurations :
Untitled Diagram.png
R1 Configs :

LDP interface : ETH2 + ETH3
LSRID = 172.16.1.1
TRANSPORT ADDR : 172.16.1.1

R2 Configs :

LDP interface : ETH1 + ETH2
LSRID = 172.16.1.2
TRANSPORT ADDR : 172.16.1.2

R3 Configs :

LDP interface : ETH1
LSRID = 172.16.2.2
TRANSPORT ADDR : 172.16.2.2

I can setup VPLS on R2 and R3 , both VPLS tunnels ready but just the R2 works and pppoe server discover .
on R3 VPLS ready but PPPOE discovery not work .

anyone know what’s the problem ? or have a sample configuration ?

Thank you :wink:

2

Best practice is setting up an empty bridge (loopback interface), and assigning the LSRID IP there, so that it’s not bound to any physical interface.

Are both ends of the R3 VPLS tunnel running?

3

Thanks pukkita , Yes both VPLS tunnels are running .

I have two vpls tunnel and both are running but just first vpls (R1toR2) work and data transfer.
Problem is no any data can transfer over R1toR3 even ICMP , PPPOE discovery or any other .
Diagram :
vpls_diagram.jpg
My Configurations :
R1

/mpls interface
set [ find default=yes ] mpls-mtu=1550
/mpls ldp
set enabled=yes lsr-id=172.16.0.1 transport-address=172.16.0.1
/mpls ldp interface
add interface=ether2

/ip address
add address=172.16.1.1/24 interface=ether2 network=172.16.1.0
add address=172.16.0.1 interface=loopback network=172.16.0.1

/interface ethernet
set [ find default-name=ether2 ] l2mtu=1598 mtu=1584

/ip route
add distance=1 dst-address=172.16.0.2/32 gateway=172.16.1.2
add distance=1 dst-address=172.16.0.3/32 gateway=172.16.1.2

/interface vpls
add disabled=no l2mtu=1500 name=R1toR3 remote-peer=172.16.0.3 vpls-id=9:9
add disabled=no l2mtu=1500 name=R1toR2 remote-peer=172.16.0.2 vpls-id=8:8

R2

/mpls interface
set [ find default=yes ] mpls-mtu=1550
/mpls ldp
set enabled=yes lsr-id=172.16.0.2 transport-address=172.16.0.2
/mpls ldp interface
add interface=ether1
add interface=ether2

/ip address
add address=172.16.1.2/24 interface=ether1 network=172.16.1.0
add address=172.16.2.1/24 interface=ether2 network=172.16.2.0
add address=172.16.0.2 interface=loopback network=172.16.0.2

/interface ethernet
set [ find default-name=ether1 ] l2mtu=1598 mtu=1584
set [ find default-name=ether2 ] l2mtu=1598 mtu=1584
/interface vpls
add disabled=no l2mtu=1500 name=R2toR1 remote-peer=172.16.0.1 vpls-id=8:8

/ip route
add distance=1 gateway=172.16.1.1
add distance=1 dst-address=172.16.0.3/32 gateway=172.16.2.2

R3

/mpls interface
set [ find default=yes ] mpls-mtu=1550
/mpls ldp
set enabled=yes lsr-id=172.16.0.3 transport-address=172.16.0.3
/mpls ldp interface
add interface=ether1

/ip address
add address=172.16.2.2/24 interface=ether1 network=172.16.2.0
add address=172.16.0.3 interface=loopback network=172.16.0.3

/interface ethernet
set [ find default-name=ether1 ] l2mtu=1598 mtu=1584

/ip route
add distance=1 gateway=172.16.2.1

/interface vpls
add disabled=no l2mtu=1500 name=R3toR1 remote-peer=172.16.0.1 vpls-id=9:9

Screenshots :
Routers.jpg

4

Which routerboard model is used on R2?.

Why are you modifying L3MTU?

Can you post an /interface bridge print detail on R1, R2 and R3?

5

Which routerboard model is used on R2?.

@ pukkita , Thanks , My devices that was used at first are with below details :

(R1)CCR1036 ----Cat6--> (R2)CCR1009 ---Wireless LigowaveLink---> (R3)RB2011UiAS

VPLS works well on R2 but on R3 just running and does not work , at first I think it maybe from Wireless Link (between R2 & R3) because of this I test with another devices and get same problem :

(R1)CCR1036 ----Cat6--> (R2)HexLite ---Cat6---> (R3)HexLite

Same as first , VPLS for R1toR3 get running but does not work .

Why are you modifying L3MTU?

I’ve set it after got problem and think maybe solve it , I see it at some example configs at forum.
for second test with HexLite routers I get back MPLS MTU to 1508 and set default all MTU of ethernets .

Can you post an /interface bridge print detail on R1, R2 and R3?

R1

[admin@R1] > /in brid print deta
Flags: X - disabled, R - running 
 0  R name="Bridge_EoIP" mtu=1500 actual-mtu=1500 l2mtu=65535 arp=enabled arp-timeout=auto mac-address=02:24:9E:17:69:75 
      protocol-mode=none priority=0x8000 auto-mac=yes admin-mac=00:00:00:00:00:00 max-message-age=20s forward-delay=15s 
      transmit-hold-count=6 ageing-time=5m 

 1  R name="bridge1_local" mtu=auto actual-mtu=1500 l2mtu=1580 arp=enabled arp-timeout=auto mac-address=E4:8D:8C:3F:B4:BA 
      protocol-mode=rstp priority=0x8000 auto-mac=yes admin-mac=00:00:00:00:00:00 max-message-age=20s forward-delay=15s 
      transmit-hold-count=6 ageing-time=5m 

 2  R name="bridge2_vpls" mtu=auto actual-mtu=1500 l2mtu=1500 arp=enabled arp-timeout=auto mac-address=02:C2:FF:D6:47:C1 
      protocol-mode=rstp priority=0x8000 auto-mac=yes admin-mac=00:00:00:00:00:00 max-message-age=20s forward-delay=15s 
      transmit-hold-count=6 ageing-time=5m 

 3  R name="loopback" mtu=auto actual-mtu=1500 l2mtu=65535 arp=enabled arp-timeout=auto mac-address=00:00:00:00:00:00 
      protocol-mode=rstp priority=0x8000 auto-mac=yes admin-mac=00:00:00:00:00:00 max-message-age=20s forward-delay=15s 
      transmit-hold-count=6 ageing-time=5m

R2

[admin@R2] > /in bri print det
Flags: X - disabled, R - running 
 0  R name="bridge1" mtu=auto actual-mtu=1500 l2mtu=1500 arp=enabled arp-timeout=auto 
      mac-address=02:31:FB:06:42:34 protocol-mode=rstp priority=0x8000 auto-mac=yes 
      admin-mac=00:00:00:00:00:00 max-message-age=20s forward-delay=15s transmit-hold-count=6 ageing-time=5m 

 1  R name="loopback" mtu=auto actual-mtu=1500 l2mtu=65535 arp=enabled arp-timeout=auto 
      mac-address=00:00:00:00:00:00 protocol-mode=rstp priority=0x8000 auto-mac=yes 
      admin-mac=00:00:00:00:00:00 max-message-age=20s forward-delay=15s transmit-hold-count=6 ageing-time=5m

R3

[admin@R3] > /in bri prin det
Flags: X - disabled, R - running 
 0  R name="bridge1" mtu=auto actual-mtu=1500 l2mtu=65535 arp=enabled arp-timeout=auto mac-address=00:00:00:00:00:00 
      protocol-mode=rstp priority=0x8000 auto-mac=yes admin-mac=00:00:00:00:00:00 max-message-age=20s forward-delay=15s 
      transmit-hold-count=6 ageing-time=5m 

 1  R name="loopback" mtu=auto actual-mtu=1500 l2mtu=1500 arp=enabled arp-timeout=auto mac-address=02:C3:45:97:10:FA 
      protocol-mode=rstp priority=0x8000 auto-mac=yes admin-mac=00:00:00:00:00:00 max-message-age=20s forward-delay=15s 
      transmit-hold-count=6 ageing-time=5m
6

Please post

/interface bridge port print detail
7

Now I using one PPPoE service on bridge2_vpls at R1 , I tried with direct PPPoE server on R1toL2 and another on R1toL3 too . but I got same problem !!

R1:

[admin@R1] > /interface vpls print deta
Flags: X - disabled, R - running, D - dynamic, B - bgp-signaled, C - cisco-bgp-signaled 
 0 R   name="R1toL2" mtu=1500 l2mtu=1500 mac-address=02:C2:FF:D6:47:C1 arp=enabled arp-timeout=auto disable-running-check=no remote-peer=172.16.0.2 vpls-id=12:12 
       cisco-style=no cisco-style-id=0 advertised-l2mtu=1500 pw-type=raw-ethernet use-control-word=default 

 1 R   name="R1toL3" mtu=1500 l2mtu=1500 mac-address=02:60:85:0F:04:78 arp=enabled arp-timeout=auto disable-running-check=no remote-peer=172.16.0.3 vpls-id=18:18 
       cisco-style=no cisco-style-id=0 advertised-l2mtu=1500 pw-type=raw-ethernet use-control-word=default 

[admin@R1] > /interface bridge port print detail
Flags: X - disabled, I - inactive, D - dynamic 

 9    interface=R1toL2 bridge=bridge2_vpls priority=0x80 path-cost=10 edge=auto point-to-point=auto external-fdb=auto horizon=none auto-isolate=no 

10    interface=R1toL3 bridge=bridge2_vpls priority=0x80 path-cost=10 edge=auto point-to-point=auto external-fdb=auto horizon=none auto-isolate=no 

 0    interface=eoip-Basetotape bridge=Bridge_EoIP priority=0x80 path-cost=10 edge=auto point-to-point=auto external-fdb=auto horizon=none auto-isolate=no 

 1    interface=eoip-shohada2 bridge=Bridge_EoIP priority=0x80 path-cost=10 edge=auto point-to-point=auto external-fdb=auto horizon=none auto-isolate=no 

 2    interface=ether6 bridge=bridge1_local priority=0x80 path-cost=10 edge=auto point-to-point=auto external-fdb=auto horizon=none auto-isolate=no 

 3    interface=ether4 bridge=bridge1_local priority=0x80 path-cost=10 edge=auto point-to-point=auto external-fdb=auto horizon=none auto-isolate=no 

 4    interface=ether5 bridge=bridge1_local priority=0x80 path-cost=10 edge=auto point-to-point=auto external-fdb=auto horizon=none auto-isolate=no 

 5 XI  interface=ether1 bridge=bridge1_local priority=0x80 path-cost=10 edge=auto point-to-point=auto external-fdb=auto horizon=none auto-isolate=no 

 6    interface=eoip-tape-2011 bridge=Bridge_EoIP priority=0x80 path-cost=10 edge=auto point-to-point=auto external-fdb=auto horizon=none auto-isolate=no 

 7    interface=eoip-tape-4th bridge=Bridge_EoIP priority=0x80 path-cost=10 edge=auto point-to-point=auto external-fdb=auto horizon=none auto-isolate=no 

 8    interface=ether11 bridge=bridge1_local priority=0x80 path-cost=10 edge=auto point-to-point=auto external-fdb=auto horizon=none auto-isolate=no

R2:

[admin@R2] > /interface bridge port print detail
Flags: X - disabled, I - inactive, D - dynamic 
 0    interface=vpls1 bridge=bridge1 priority=0x80 path-cost=10 edge=auto point-to-point=auto external-fdb=auto horizon=none 
      auto-isolate=no

R3:

[admin@R3] > /interface bridge port print detail
Flags: X - disabled, I - inactive, D - dynamic 
 0    interface=vpls1 bridge=bridge1 priority=0x80 path-cost=10 edge=auto point-to-point=auto external-fdb=auto horizon=none 
      auto-isolate=no
8

I upgrade three routers to 6.38.7 and now I see a problem on direct pppoe-server on R1toL3 (R1toR3) , PPPoE Server is invalid !

Note : this time VPLS tunnels at R1 are not member of any bridge and running and have PPPoE service for each one .
R1 :

[admin@R1] /interface pppoe-server server print detail
 53   service-name="service1" interface=R1toL2 max-mtu=auto max-mru=auto mrru=disabled authentication=pap,chap keepalive-timeout=10 one-session-per-host=no 
     max-sessions=unlimited pado-delay=0 default-profile=default 

 54 I service-name="service2" interface=R1toL3 max-mtu=auto max-mru=auto mrru=disabled authentication=pap,chap keepalive-timeout=10 one-session-per-host=no 
     max-sessions=unlimited pado-delay=0 default-profile=default 

[admin@R1] /interface vpls print detail
Flags: X - disabled, R - running, D - dynamic, B - bgp-signaled, C - cisco-bgp-signaled 
 0 R   name="R1toL2" mtu=1500 l2mtu=1500 mac-address=02:C2:FF:D6:47:C1 arp=enabled arp-timeout=auto disable-running-check=no remote-peer=172.16.0.2 vpls-id=12:12 
       cisco-style=no cisco-style-id=0 advertised-l2mtu=1500 pw-type=raw-ethernet use-control-word=default 

 1 R   name="R1toL3" mtu=1500 l2mtu=1500 mac-address=02:60:85:0F:04:78 arp=enabled arp-timeout=auto disable-running-check=no remote-peer=172.16.0.3 vpls-id=18:18 
       cisco-style=no cisco-style-id=0 advertised-l2mtu=1500 pw-type=raw-ethernet use-control-word=default

VPLS2.png

9

post

/interface vpls monitor [find]
10

I upgrade three routers to 6.38.7 and now I see a problem on direct pppoe-server on R1toL3 (R1toR3) , PPPoE Server is invalid !

That’s odd… something else is going on when the pppoe server alerts (red). Post a full R1 export.

Make sure firmware is up to date.

Try this:

  • Set all ether ports to L2MTU=2024 (R3: ether1, R2: ether2, ether1, R1: ether2)
  • Set MPLS > MPLS Interface MPLS MTU to 1508 on R1,2,3
  • Set VPLS tunnels Advertised L2MTU to 1508
11

@pukkita , Thanks again
Before I set this MTU changes , I reboot once again the R1 and that invalid PPPoE server on R1toL3 (R1toR3) was fixed but still have the First problem .

Now I change the MTU settings on MPLS interface , Ethernets and VPLS adverties with following details but still have problem and the 3rd hop (R3) VPLS running and don’t transfer anything even pppoe discovery (scan)
Here’s the VPLS monitors
R1:

[admin@R1] > /interface vpls monitor [find]
         remote-label: 19            18              
          local-label: 96            97              
        remote-status:                               
            transport: 172.16.0.2/32 172.16.0.3/32  
    transport-nexthop: 172.16.1.2    172.16.1.2      
       imposed-labels: 19            16              
                                     18              
-- [Q quit|D dump|C-z pause]

R2:

[admin@R2] >  /interface vpls monitor [find]
       remote-label: 96
        local-label: 19
      remote-status: 
          transport: 0.0.0.0/0
  transport-nexthop: 172.16.1.1
     imposed-labels: 96
-- [Q quit|D dump|C-z pause]

R3:

[admin@R3] > /interface vpls monitor [find]
       remote-label: 97
        local-label: 18
      remote-status: 
          transport: 0.0.0.0/0
  transport-nexthop: 172.16.2.1
     imposed-labels: 97
-- [Q quit|D dump|C-z pause]
12

What about firmware? post

/system routerboard print

did you test using the ligowaves, or the Hex Poe Lites?

13

because of probable odd problem on my Core router , I simulate it with 3 new different router .

Now I’m using three new hAp Lite with the following items .
Reset Default , rOS v6.38.7 and routerboard firmware 3.39 , but still have same problem :open_mouth: :confused:
R1

/interface bridge
add name=loopback
/interface ethernet
set [ find default-name=ether4 ] name=laptop
/interface vpls
add advertised-l2mtu=1508 disabled=no l2mtu=1508 mac-address=\
    02:28:05:50:8B:C8 name=vpls2 remote-peer=172.16.0.2 vpls-id=2:2
add advertised-l2mtu=1508 disabled=no l2mtu=1508 mac-address=\
    02:A2:D7:20:CF:D0 name=vpls3 remote-peer=172.16.0.3 vpls-id=3:3
/interface pppoe-server server
add authentication=pap,chap disabled=no interface=vpls3 service-name=service1
add authentication=pap,chap disabled=no interface=vpls2 service-name=service2
/ip address
add address=172.16.1.1/24 interface=ether1 network=172.16.1.0
add address=172.16.10.1/24 interface=laptop network=172.16.10.0
add address=172.16.0.1 interface=loopback network=172.16.0.1
/ip route
add distance=1 dst-address=172.16.0.2/32 gateway=172.16.1.2
add distance=1 dst-address=172.16.0.3/32 gateway=172.16.1.2
add distance=1 dst-address=172.16.2.0/24 gateway=172.16.1.2
/mpls ldp
set enabled=yes lsr-id=172.16.0.1 transport-address=172.16.0.1
/mpls ldp interface
add interface=ether1
/system identity
set name=R1

R2

/interface bridge
add name=bridge1
add name=loopback
/interface vpls
add advertised-l2mtu=1508 disabled=no l2mtu=1508 mac-address=\
    02:C6:A6:06:15:4A name=vpls1 remote-peer=172.16.0.1 vpls-id=2:2
/interface bridge port
add bridge=bridge1 interface=vpls1
/ip address
add address=172.16.1.2/24 interface=ether1 network=172.16.1.0
add address=172.16.2.1/24 interface=ether2 network=172.16.2.0
add address=172.16.0.2 interface=loopback network=172.16.0.2
/ip route
add distance=1 gateway=172.16.1.1
add distance=1 dst-address=172.16.0.3/32 gateway=172.16.2.2
/mpls ldp
set enabled=yes lsr-id=172.16.0.2 transport-address=172.16.0.2
/mpls ldp interface
add interface=ether1
add interface=ether2
/system identity
set name=R2

R3

/interface bridge
add name=bridge1
add name=loopback
/interface vpls
add advertised-l2mtu=1508 disabled=no l2mtu=1508 mac-address=\
    02:E8:93:3C:14:1F name=vpls1 remote-peer=172.16.0.1 vpls-id=3:3
/interface bridge port
add bridge=bridge1 interface=vpls1
/ip address
add address=172.16.2.2/24 interface=ether1 network=172.16.2.0
add address=172.16.0.3 interface=loopback network=172.16.0.3
/ip route
add distance=1 gateway=172.16.2.1
/mpls ldp
set enabled=yes lsr-id=172.16.0.3 transport-address=172.16.0.3
/mpls ldp interface
add interface=ether1
14

Set MPLS L2MTU back to defaults.

Look at both end of the tunnels, are all in running state?

What happens, a pppoe scan on R3’s VPLS tunnel (happens also on bridge1?) doesn’t “see” the pppoe server on R1?

15

…

16

Pukkita’s recommendation of 1508 MPLS MTU (the default) is incorrect, if you are running VPLS tunnels that is too low to pass anything but a few labels. Your 1550 that you had is what we use and that is fine. L2 MTU for all ports that will have tunnels has to be at least 1550 but usually the default on Routerboards is around 1598 so this is good enough. IP MTU should be left at 1500.

Advertised L2 MTU can be set to 1508 on your VPLS tunnels that will carry PPPoE traffic only if you want to allow RFC4638, which allows customers to get full 1500 IP MTU with oversize PPPoE frames of 1508 bytes. Offering this would also entail the last mile to the customer supporting 1508 MTU. IP MTU can be 1500 on these tunnels.

I highly doubt that your limitations were the result of misconfiguration on the MikroTik side but most likely an intervening device, like a switch or wireless radio between the routers that is giving you an MTU limitation.

Before running PPPoE live customers over this tunnel, you should test it with a ping, by assigning an IP on both sides of the tunnel (on the routers themselves). You will probably find that you cannot ping the other side with the MikroTik ping tool with packet size set to 1500, but it will work with smaller packets, which indicates an MTU limitation on an intermediate device.

17

Also, you do not need to set different VPLS ID for each tunnel. You can have them all as 1:1 for instance. It is not like EoIP where each tunnel requires a unique ID on a given router. VPLS tunnel IDs only need to be unique where you have more than one between the same pair of routers (i.e. if there were two VPLS tunnels between R1 and R2, then you would use different IDs for each tunnel).

18

I try again with 3 new hAp Lite router to test it :
RouterOS v6.38.7 , routerboard firmare 6.36 , default configuration after upgrade , and config manually all .

MPLS MTU        --> 1550
VPLS L2MTU Adv  --> 1508
interface MTU   --> 1500 (by default)
interface L2MTU --> 1598 (by default)

Still have the first problem! , VPLS of R1toR2 is OK and works well , but VPLS of R1toR3 is running but no any data transfer .

After MTU settings of mducharme suggestion is fragmentation between three routers are :
loopback ICMP ping between routers are 1546 because of MPLS MTU interface is 1550 (Ping of 172.16.0.2 or 172.16.0.3)
normal ICMP ping to interface address is 1500 (Ping of 172.16.1.2 or 172.16.2.2)

This is the diagram :
diagram.png
really connected routers :
MPLS_Routers.jpg
This is the configurations that was exported :
R1

/interface bridge
add name=loopback
/interface vpls
add advertised-l2mtu=1508 disabled=no l2mtu=1508 mac-address=\
    02:C6:2D:B1:C4:CC name=toR2 remote-peer=172.16.0.2 vpls-id=1:1
add advertised-l2mtu=1508 disabled=no l2mtu=1508 mac-address=\
    02:22:C5:9C:5C:4A name=toR3 remote-peer=172.16.0.3 vpls-id=2:2
/interface pppoe-server server
add disabled=no interface=toR3 service-name=service1
/ip address
add address=172.16.1.1/24 interface=ether1 network=172.16.1.0
add address=172.16.0.1 interface=loopback network=172.16.0.1
add address=192.168.16.1/24 interface=toR3 network=192.168.16.0
add address=172.16.10.1/24 interface=ether4 network=172.16.10.0
/ip firewall nat
add action=masquerade chain=srcnat src-address=172.16.10.2
/ip route
add distance=1 gateway=192.168.3.1
add distance=1 dst-address=172.16.0.2/32 gateway=172.16.1.2
add distance=1 dst-address=172.16.0.3/32 gateway=172.16.1.2
add distance=1 dst-address=172.16.2.0/24 gateway=172.16.1.2
/mpls interface
set [ find default=yes ] mpls-mtu=1550
/mpls ldp
set enabled=yes lsr-id=172.16.0.1 transport-address=172.16.0.1
/mpls ldp interface
add interface=ether1
/system identity
set name=R1

R2

/interface bridge
add name=loopback
/interface vpls
add advertised-l2mtu=1508 disabled=no l2mtu=1508 mac-address=\
    02:B6:FB:A1:48:2B name=vpls1 remote-peer=172.16.0.1 vpls-id=1:1
/ip address
add address=172.16.1.2/24 interface=ether1 network=172.16.1.0
add address=172.16.2.1/24 interface=ether2 network=172.16.2.0
add address=172.16.0.2 interface=loopback network=172.16.0.2
/ip route
add distance=1 gateway=172.16.1.1
add distance=1 dst-address=172.16.0.3/32 gateway=172.16.2.2
/mpls interface
set [ find default=yes ] mpls-mtu=1550
/mpls ldp
set enabled=yes lsr-id=172.16.0.2 transport-address=172.16.0.2
/mpls ldp interface
add interface=ether1
add interface=ether2
/system identity
set name=R2

R3

/interface bridge
add name=bridge1
add name=loopback
/interface vpls
add advertised-l2mtu=1508 disabled=no l2mtu=1508 mac-address=
    02:EE:45:29:60:B1 name=vpls1 remote-peer=172.16.0.1 vpls-id=2:2
/interface bridge port
add bridge=bridge1 disabled=yes interface=vpls1
/ip address
add address=172.16.2.2/24 interface=ether1 network=172.16.2.0
add address=172.16.0.3 interface=loopback network=172.16.0.3
add address=192.168.16.2/24 interface=vpls1 network=192.168.16.0
/ip route
add distance=1 gateway=172.16.2.1
/mpls interface
set [ find default=yes ] mpls-mtu=1550
/mpls ldp
set enabled=yes lsr-id=172.16.0.3 transport-address=172.16.0.3
/mpls ldp interface
add interface=ether1
/system identity
set name=R3
19

The routers all need the loopback interfaces for all other routers in their routing tables as /32 routes. These static routes you have set up are insufficient because they are summarizing the loopbacks. Please set up OSPF and ensure that all of your routers can ping each others loopback from the OSPF advertisement (get rid of these static routes to make sure you haven’t forgotten to advertise something). Switch off masquerade for the time being, your rule is a bit strange and may interfere. Make sure LDP neighbor is formed properly with the other routers. Then test again.

20

@mducharme, Thanks .
With my static route configurations , all of routers can ping each other with loopback ip address.
I’m not using the OSPF because of my core router (1036) is a central pppoe server with + 4K pppoe session , each session make a route , OSPF will clone all of CoreRouter routes to each other routers , more than 1000+ routes will freeze other than CCR series .

What’s your suggestion ? What’s the best way ?