PPPoE + CPU Usage and High Latency

natanielklug · March 18, 2013, 6:06pm

Hello all,

I am trying to run a PPPoE Concentrator over Mikrotik and I am running over several problems. I am using Mikrotik as PPPoE Concentrator for 8 years, at least, and now as my network grow it is going to a leak. I will explain the scenario and all things I already done:

First Scenario: Mikrotik RoS v5.24 over Dell R410 (x86)
Hardware specs:

[admin@csc.cpd.rt.brd.g.01] > system resource print 
                   uptime: 1d59m30s
                  version: 5.24
              free-memory: 1326448KiB
             total-memory: 1943140KiB
                      cpu: Intel(R)
                cpu-count: 8
            cpu-frequency: 2393MHz
                 cpu-load: 20%
           free-hdd-space: 480443840KiB
          total-hdd-space: 480705076KiB
  write-sect-since-reboot: 239381554
         write-sect-total: 239381554
        architecture-name: x86
               board-name: x86
                 platform: MikroTik

The problem:
LATENCY when conections go through the concentrator are going really high and CPU usage is hitting 100% on some cores (normaly over cpu0 and cpu1). Please see
CSC-auth-01.zip (112 KB)
for the images. I can’t accept this would be a normal situation over Mikrotik PPPoE. It can not be normal.

Second Scenario: RB1100AH direct attached to the network (no switch, direct connection to the router in the network, like a point of presence)
As I saw the problem I configured a new RB1100AH and direct attached my POP into this and I saw the latency goes down. and it was possible to assure something about 400 pppoe sessions over this routerboard.
The problem:
When it got something between 450 and 550 pppoe sessions CPU load gone crazy and got to 100% taking down the network.

Third Scenario: New Dell R410
So we tought the problem was the CPU over this RB and we setup a new Dell R410 with 4 Broadcom Giga NICs. The result was disastrous. Latency gone to sky and jitter get crazy. No solutions found and we made a new RB1100AH cluster to act as a big PPPoE Concentrator. Now its getting crazy again while the CPU and Memory goes up and the latency too without any visible impact on number of PPPoE sessions or network traffic load.

I would love to hear what can I do to solve the problem. I need 3.5k PPPoE sessions with to run over my network but I need something I can count on. Bellow I will send the configuration in this RB1100AH (now it’s running 6.0RC11 but I tried 5.24 with no luck).

[admin@CSC-auth-11] > interface pppoe-server server export
/interface pppoe-server server
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-fmto-11an-pmp-01.C-Formato-a01 max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-fmto-11an-pmp-02.C-Formato-a02 max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-pqsp-ge-rt-01.C-PqSaoPaulo01 max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-pqsp-ge-rt-01.C-PqSaoPaulo02 max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-pqsp-ge-rt-01.C-PqSaoPaulo03 max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-pqsp-11bv-pmp-01.Certto-PqSaoPaulo-01 \
    max-mru=1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-pqsp-11bv-pmp-01.Certto-PqSaoPaulo-02 \
    max-mru=1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-pqsp-11bv-pmp-01.Certto-PqSaoPaulo-03 \
    max-mru=1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-neva-11bv-pmp-01.Certto-Neva-01 max-mru=\
    1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-neva-11bv-pmp-01.Certto-Neva-02 max-mru=\
    1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-neva-11bv-pmp-01.Certto-Neva-03 max-mru=\
    1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-neva-ge-rt-01.C-Neva-a01 max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-neva-ge-rt-01.C-Neva-a02 max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-neva-ge-rt-01.C-Neva-a03 max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-neva-ge-rt-01.C-Neva-a04 max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-algr-11bv-pmp-01.C-AltoAlegre-01 max-mru=\
    1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-algr-11bv-pmp-01.C-AltoAlegre-02 max-mru=\
    1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-algr-11bv-pmp-01.C-AltoAlegre-03 max-mru=\
    1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-algr-ge-rt-01.C-AltoAlegre-a03 max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-algr-ge-rt-01.C-AltoAlegre-a04 max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-algr-ge-rt-01.C-AltoAlegre-a01 max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-algr-ge-rt-01.C-AltoAlegre-a02 max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-pqvd-11bv-pmp-01.C-PARQUEVERDE-01 max-mru=\
    1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-pqvd-11bv-pmp-01.C-PARQUEVERDE-02 max-mru=\
    1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-pqvd-11bv-pmp-01.C-PARQUEVERDE-03 max-mru=\
    1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-pqvd-11bv-pmp-01.C-PARQUEVERDE-04 max-mru=\
    1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-athe-ge-rt-03.C-Tropical-a01 max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-athe-ge-rt-03.C-Tropical-a02 max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-athe-ge-rt-03.C-Tropical-a03 max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-athe-ge-rt-03.C-Tropical-a04 max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-cldt-11bv-pmp-01.Certto-Claudete-01 \
    max-mru=1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-cldt-11bv-pmp-01.Certto-Claudete-02 \
    max-mru=1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-cldt-11bv-pmp-01.Certto-Claudete-03 \
    max-mru=1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-cldt-ge-rt-01.C-Claudete-a01 max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-cldt-ge-rt-01.C-Claudete-a02 max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-cldt-ge-rt-01.C-Claudete-a03 max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-cldt-ge-rt-01.C-Claudete-a04 max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-pqvd-ge-rt-01.C-ParqueVerde-a01 max-mru=\
    1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-pqvd-ge-rt-01.C-ParqueVerde-a02 max-mru=\
    1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-pqvd-ge-rt-01.C-ParqueVerde-a03 max-mru=\
    1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-pqvd-ge-rt-01.C-ParqueVerde-a04 max-mru=\
    1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-athe-ge-rt-02.PTP-Wealth max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-athe-11bh-pmp-01.Certto-AltoAlegre max-mru=\
    1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-athe-11bh-pmp-01.Certto-Coqueiral max-mru=\
    1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-athe-11bh-pmp-01.C-Coqueiral max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-athe-11bh-pmp-01.C-jl max-mru=1452 max-mtu=\
    1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-coql-11bv-pmp-01.C-Coqueiral-01 max-mru=\
    1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-coql-11bv-pmp-01.C-Coqueiral-02 max-mru=\
    1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-coql-11bv-pmp-01.C-Coqueiral-03 max-mru=\
    1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-coql-ge-rt-01.C-Coqueiral-a01 max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-coql-ge-rt-01.C-Coqueiral-a02 max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-coql-ge-rt-01.C-Coqueiral-a03 max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-coql-ge-rt-01.C-Coqueiral-a04 max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-coql-ge-rt-02.PTP-TerraNovaI max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-coql-ge-rt-02.PTP-Dickson max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-athe-ge-rt-02.PauloGodoyGERAL max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-inda-11bv-pmp-01.Certto-Industrial-01 \
    max-mru=1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-inda-11bv-pmp-01.Certto-Industrial-02 \
    max-mru=1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-inda-11bv-pmp-01.Certto-Industrial-03 \
    max-mru=1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-pgdy-11bh-pmp-01.C-PauloGodoyH-01 max-mru=\
    1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-pgdy-11bh-pmp-01.C-PauloGodoyH-02 max-mru=\
    1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-pgdy-11bh-pmp-02.C-PauloGodoyH-03 max-mru=\
    1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-pgdy-11bh-pmp-02.C-PauloGodoyH-04 max-mru=\
    1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-pgdy-11bv-pmp-01.C-PauloGodoy-01 max-mru=\
    1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-pgdy-11bv-pmp-01.C-PauloGodoy-02 max-mru=\
    1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-pgdy-11bv-pmp-01.C-PauloGodoy-03 max-mru=\
    1452 max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-pgdy-ge-rt-02.C-PauloGodoy-a01 max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-pgdy-ge-rt-02.C-PauloGodoy-a02 max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-pgdy-ge-rt-02.C-PauloGodoy-a03 max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-pgdy-ge-rt-02.C-PauloGodoy-a04 max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
add authentication=pap default-profile=PPPoE-Certto disabled=no interface=CSC-inda-11bv-pmp-01.PTP-FozTintas max-mru=1452 \
    max-mtu=1452 service-name=CSC-auth-11
[admin@CSC-auth-11] > interface eoip export
/interface eoip
add local-address=172.16.240.27 mac-address=FE:7A:8C:AE:51:82 name=CSC-algr-11bv-pmp-01.C-AltoAlegre-01 remote-address=\
    172.16.255.20 tunnel-id=3323
add local-address=172.16.240.27 mac-address=FE:F4:85:60:55:54 name=CSC-algr-11bv-pmp-01.C-AltoAlegre-02 remote-address=\
    172.16.255.20 tunnel-id=3324
add local-address=172.16.240.27 mac-address=FE:65:7C:56:88:95 name=CSC-algr-11bv-pmp-01.C-AltoAlegre-03 remote-address=\
    172.16.255.20 tunnel-id=3325
add local-address=172.16.240.27 mac-address=FE:69:40:DE:E9:B0 name=CSC-algr-ge-rt-01.C-AltoAlegre-a01 remote-address=\
    172.16.255.21 tunnel-id=3328
add local-address=172.16.240.27 mac-address=FE:0E:B7:CF:7D:3D name=CSC-algr-ge-rt-01.C-AltoAlegre-a02 remote-address=\
    172.16.255.21 tunnel-id=3329
add local-address=172.16.240.27 mac-address=FE:64:09:86:04:D2 name=CSC-algr-ge-rt-01.C-AltoAlegre-a03 remote-address=\
    172.16.255.21 tunnel-id=3326
add local-address=172.16.240.27 mac-address=FE:67:4B:BA:21:8C name=CSC-algr-ge-rt-01.C-AltoAlegre-a04 remote-address=\
    172.16.255.21 tunnel-id=3327
add local-address=172.16.240.27 mac-address=FE:31:1A:27:05:48 name=CSC-athe-11bh-pmp-01.C-Coqueiral remote-address=\
    172.16.255.25 tunnel-id=3352
add local-address=172.16.240.27 mac-address=FE:B8:F7:48:D3:79 name=CSC-athe-11bh-pmp-01.C-jl remote-address=172.16.255.25 \
    tunnel-id=3353
add local-address=172.16.240.27 mac-address=FE:6B:18:0E:E9:CC name=CSC-athe-11bh-pmp-01.Certto-AltoAlegre remote-address=\
    172.16.255.25 tunnel-id=3350
add local-address=172.16.240.27 mac-address=FE:C7:A8:2D:6E:D5 name=CSC-athe-11bh-pmp-01.Certto-Coqueiral remote-address=\
    172.16.255.25 tunnel-id=3351
add local-address=172.16.240.27 mac-address=FE:33:BC:59:38:31 name=CSC-athe-ge-rt-02.PTP-Wealth remote-address=\
    172.16.255.14 tunnel-id=3349
add local-address=172.16.240.27 mac-address=FE:75:9D:09:A8:E8 name=CSC-athe-ge-rt-02.PauloGodoyGERAL remote-address=\
    172.16.255.30 tunnel-id=3363
add local-address=172.16.240.27 mac-address=FE:2D:D3:8D:D1:C1 name=CSC-athe-ge-rt-03.C-Tropical-a01 remote-address=\
    172.16.255.26 tunnel-id=3334
add local-address=172.16.240.27 mac-address=FE:70:5E:DD:44:99 name=CSC-athe-ge-rt-03.C-Tropical-a02 remote-address=\
    172.16.255.26 tunnel-id=3335
add local-address=172.16.240.27 mac-address=FE:DB:8D:B6:4D:3D name=CSC-athe-ge-rt-03.C-Tropical-a03 remote-address=\
    172.16.255.26 tunnel-id=3336
add local-address=172.16.240.27 mac-address=FE:C0:B7:49:2B:51 name=CSC-athe-ge-rt-03.C-Tropical-a04 remote-address=\
    172.16.255.26 tunnel-id=3337
add local-address=172.16.240.27 mac-address=FE:65:77:A6:FB:A2 name=CSC-cldt-11bv-pmp-01.Certto-Claudete-01 \
    remote-address=172.16.255.19 tunnel-id=3338
add local-address=172.16.240.27 mac-address=FE:A3:00:8D:03:18 name=CSC-cldt-11bv-pmp-01.Certto-Claudete-02 \
    remote-address=172.16.255.19 tunnel-id=3339
add local-address=172.16.240.27 mac-address=FE:FE:63:53:E3:FD name=CSC-cldt-11bv-pmp-01.Certto-Claudete-03 \
    remote-address=172.16.255.19 tunnel-id=3340
add local-address=172.16.240.27 mac-address=FE:1D:01:D9:1E:BF name=CSC-cldt-ge-rt-01.C-Claudete-a01 remote-address=\
    172.16.255.24 tunnel-id=3341
add local-address=172.16.240.27 mac-address=FE:50:06:15:A0:A9 name=CSC-cldt-ge-rt-01.C-Claudete-a02 remote-address=\
    172.16.255.24 tunnel-id=3342
add local-address=172.16.240.27 mac-address=FE:B1:86:D8:42:A2 name=CSC-cldt-ge-rt-01.C-Claudete-a03 remote-address=\
    172.16.255.24 tunnel-id=3343
add local-address=172.16.240.27 mac-address=FE:50:38:CF:B4:0B name=CSC-cldt-ge-rt-01.C-Claudete-a04 remote-address=\
    172.16.255.24 tunnel-id=3344
add local-address=172.16.240.27 mac-address=FE:91:27:63:DF:10 name=CSC-coql-11bv-pmp-01.C-Coqueiral-01 remote-address=\
    172.16.255.29 tunnel-id=3354
add local-address=172.16.240.27 mac-address=FE:07:8E:82:8C:FF name=CSC-coql-11bv-pmp-01.C-Coqueiral-02 remote-address=\
    172.16.255.29 tunnel-id=3355
add local-address=172.16.240.27 mac-address=FE:18:24:60:D7:B0 name=CSC-coql-11bv-pmp-01.C-Coqueiral-03 remote-address=\
    172.16.255.29 tunnel-id=3356
add local-address=172.16.240.27 mac-address=FE:EB:37:11:3C:A1 name=CSC-coql-ge-rt-01.C-Coqueiral-a01 remote-address=\
    172.16.255.27 tunnel-id=3357
add local-address=172.16.240.27 mac-address=FE:8C:37:04:B5:9F name=CSC-coql-ge-rt-01.C-Coqueiral-a02 remote-address=\
    172.16.255.27 tunnel-id=3358
add local-address=172.16.240.27 mac-address=FE:03:98:23:42:AB name=CSC-coql-ge-rt-01.C-Coqueiral-a03 remote-address=\
    172.16.255.27 tunnel-id=3359
add local-address=172.16.240.27 mac-address=FE:A4:FE:0D:9E:97 name=CSC-coql-ge-rt-01.C-Coqueiral-a04 remote-address=\
    172.16.255.27 tunnel-id=3360
add local-address=172.16.240.27 mac-address=FE:CA:2C:E1:CC:80 name=CSC-coql-ge-rt-02.PTP-Dickson remote-address=\
    172.16.255.28 tunnel-id=3362
add local-address=172.16.240.27 mac-address=FE:09:BB:58:36:9B name=CSC-coql-ge-rt-02.PTP-TerraNovaI remote-address=\
    172.16.255.28 tunnel-id=3361
add local-address=172.16.240.27 mac-address=FE:58:FF:2C:7E:DC name=CSC-fmto-11an-pmp-01.C-Formato-a01 remote-address=\
    172.16.255.11 tunnel-id=3308
add local-address=172.16.240.27 mac-address=FE:AB:05:31:34:37 name=CSC-fmto-11an-pmp-02.C-Formato-a02 remote-address=\
    172.16.255.11 tunnel-id=3309
add local-address=172.16.240.27 mac-address=FE:70:9B:C3:09:86 name=CSC-inda-11bv-pmp-01.Certto-Industrial-01 \
    remote-address=172.16.255.18 tunnel-id=3367
add local-address=172.16.240.27 mac-address=FE:91:90:C9:57:C0 name=CSC-inda-11bv-pmp-01.Certto-Industrial-02 \
    remote-address=172.16.255.18 tunnel-id=3368
add local-address=172.16.240.27 mac-address=FE:B0:67:61:A9:E0 name=CSC-inda-11bv-pmp-01.Certto-Industrial-03 \
    remote-address=172.16.255.18 tunnel-id=3369
add local-address=172.16.240.27 mac-address=FE:B7:A0:8A:2C:41 name=CSC-inda-11bv-pmp-01.PTP-FozTintas remote-address=\
    172.16.255.18 tunnel-id=3371
add local-address=172.16.240.27 mac-address=FE:96:82:83:4A:99 name=CSC-neva-11bv-pmp-01.Certto-Neva-01 remote-address=\
    172.16.255.15 tunnel-id=3316
add local-address=172.16.240.27 mac-address=FE:39:BA:C3:9A:85 name=CSC-neva-11bv-pmp-01.Certto-Neva-02 remote-address=\
    172.16.255.15 tunnel-id=3317
add local-address=172.16.240.27 mac-address=FE:81:7A:5E:08:8E name=CSC-neva-11bv-pmp-01.Certto-Neva-03 remote-address=\
    172.16.255.15 tunnel-id=3318
add local-address=172.16.240.27 mac-address=FE:2C:F0:86:86:E7 name=CSC-neva-ge-rt-01.C-Neva-a01 remote-address=\
    172.16.255.16 tunnel-id=3319
add local-address=172.16.240.27 mac-address=FE:09:70:9F:8F:7D name=CSC-neva-ge-rt-01.C-Neva-a02 remote-address=\
    172.16.255.16 tunnel-id=3320
add local-address=172.16.240.27 mac-address=FE:B8:E9:4D:F5:AF name=CSC-neva-ge-rt-01.C-Neva-a03 remote-address=\
    172.16.255.16 tunnel-id=3321
add local-address=172.16.240.27 mac-address=FE:35:C6:4A:4E:38 name=CSC-neva-ge-rt-01.C-Neva-a04 remote-address=\
    172.16.255.16 tunnel-id=3322
add local-address=172.16.240.27 mac-address=FE:D7:E0:3C:CB:96 name=CSC-pgdy-11bh-pmp-01.C-PauloGodoyH-01 remote-address=\
    172.16.255.33 tunnel-id=3372
add local-address=172.16.240.27 mac-address=FE:9F:CF:4B:58:CA name=CSC-pgdy-11bh-pmp-01.C-PauloGodoyH-02 remote-address=\
    172.16.255.33 tunnel-id=3373
add local-address=172.16.240.27 mac-address=FE:08:64:4F:ED:73 name=CSC-pgdy-11bh-pmp-02.C-PauloGodoyH-03 remote-address=\
    172.16.255.34 tunnel-id=3374
add local-address=172.16.240.27 mac-address=FE:B4:9D:5B:B0:88 name=CSC-pgdy-11bh-pmp-02.C-PauloGodoyH-04 remote-address=\
    172.16.255.34 tunnel-id=3375
add local-address=172.16.240.27 mac-address=FE:55:BF:D8:C3:58 name=CSC-pgdy-11bv-pmp-01.C-PauloGodoy-01 remote-address=\
    172.16.255.35 tunnel-id=3376
add local-address=172.16.240.27 mac-address=FE:11:A3:FE:0A:6B name=CSC-pgdy-11bv-pmp-01.C-PauloGodoy-02 remote-address=\
    172.16.255.35 tunnel-id=3377
add local-address=172.16.240.27 mac-address=FE:64:98:FA:7E:96 name=CSC-pgdy-11bv-pmp-01.C-PauloGodoy-03 remote-address=\
    172.16.255.35 tunnel-id=3378
add local-address=172.16.240.27 mac-address=FE:62:CF:D6:FC:73 name=CSC-pgdy-ge-rt-02.C-PauloGodoy-a01 remote-address=\
    172.16.255.36 tunnel-id=3379
add local-address=172.16.240.27 mac-address=FE:86:97:AF:20:BD name=CSC-pgdy-ge-rt-02.C-PauloGodoy-a02 remote-address=\
    172.16.255.36 tunnel-id=3380
add local-address=172.16.240.27 mac-address=FE:11:4C:78:1B:98 name=CSC-pgdy-ge-rt-02.C-PauloGodoy-a03 remote-address=\
    172.16.255.36 tunnel-id=3381
add local-address=172.16.240.27 mac-address=FE:D7:F0:83:B6:3F name=CSC-pgdy-ge-rt-02.C-PauloGodoy-a04 remote-address=\
    172.16.255.36 tunnel-id=3382
add local-address=172.16.240.27 mac-address=FE:1D:48:F3:64:89 name=CSC-pqsp-11bv-pmp-01.Certto-PqSaoPaulo-01 \
    remote-address=172.16.255.13 tunnel-id=3313
add local-address=172.16.240.27 mac-address=FE:24:D9:D2:78:DC name=CSC-pqsp-11bv-pmp-01.Certto-PqSaoPaulo-02 \
    remote-address=172.16.255.13 tunnel-id=3314
add local-address=172.16.240.27 mac-address=FE:53:E4:54:FF:72 name=CSC-pqsp-11bv-pmp-01.Certto-PqSaoPaulo-03 \
    remote-address=172.16.255.13 tunnel-id=3315
add local-address=172.16.240.27 mac-address=FE:C4:76:3E:1A:FE name=CSC-pqsp-ge-rt-01.C-PqSaoPaulo01 remote-address=\
    172.16.255.12 tunnel-id=3310
add local-address=172.16.240.27 mac-address=FE:A5:A8:30:F0:3A name=CSC-pqsp-ge-rt-01.C-PqSaoPaulo02 remote-address=\
    172.16.255.12 tunnel-id=3311
add local-address=172.16.240.27 mac-address=FE:E2:B7:73:22:A7 name=CSC-pqsp-ge-rt-01.C-PqSaoPaulo03 remote-address=\
    172.16.255.12 tunnel-id=3312
add local-address=172.16.240.27 mac-address=FE:FE:BE:7E:C9:FF name=CSC-pqvd-11bv-pmp-01.C-PARQUEVERDE-01 remote-address=\
    172.16.255.22 tunnel-id=3330
add local-address=172.16.240.27 mac-address=FE:94:76:CC:A4:29 name=CSC-pqvd-11bv-pmp-01.C-PARQUEVERDE-02 remote-address=\
    172.16.255.22 tunnel-id=3331
add local-address=172.16.240.27 mac-address=FE:96:69:14:35:93 name=CSC-pqvd-11bv-pmp-01.C-PARQUEVERDE-03 remote-address=\
    172.16.255.22 tunnel-id=3332
add local-address=172.16.240.27 mac-address=FE:22:25:87:B1:03 name=CSC-pqvd-11bv-pmp-01.C-PARQUEVERDE-04 remote-address=\
    172.16.255.22 tunnel-id=3333
add local-address=172.16.240.27 mac-address=FE:87:99:77:0D:F8 name=CSC-pqvd-ge-rt-01.C-ParqueVerde-a01 remote-address=\
    172.16.255.23 tunnel-id=3345
add local-address=172.16.240.27 mac-address=FE:9D:AC:34:D7:53 name=CSC-pqvd-ge-rt-01.C-ParqueVerde-a02 remote-address=\
    172.16.255.23 tunnel-id=3346
add local-address=172.16.240.27 mac-address=FE:24:FB:82:D9:36 name=CSC-pqvd-ge-rt-01.C-ParqueVerde-a03 remote-address=\
    172.16.255.23 tunnel-id=3347
add local-address=172.16.240.27 mac-address=FE:0D:5C:A6:24:A9 name=CSC-pqvd-ge-rt-01.C-ParqueVerde-a04 remote-address=\
    172.16.255.23 tunnel-id=3348
[admin@CSC-auth-11] > queue export
/queue type
set 0 pfifo-limit=100

I am attaching some images of CSC-auth-11, one of our RB1100AH in the cluster. It’s now getting to 100% CPU usage and I am about to restart it.
Pictures.zip (186 KB)

joshaven · March 18, 2013, 6:37pm

Here is a MUM presentation that sounds like it is very similar to your situation.

http://mum.mikrotik.com/presentations/AE12/charbel.pdf

natanielklug · March 18, 2013, 7:17pm

joshaven,

It’s really amazing the trick configuration for this setup but we already have a balance using EoIP and the bridge is only in the remote equipment, where our subscribers are connected. Thank you for your help. I will try to understand better this option and see if this is useful here.

joshaven · March 18, 2013, 9:39pm

Your quite welcome, I hope the document gives you some good ideas.

As usual, there are multiple ways address this issue and even more of pros and cons that come along with the options.

We terminate our PPPoE sessions at every tower location so I don’t have any individual routers with more then 500 connections. My model is indignantly scalable and not complicated except that we have pain points in the dynamic routing protocol area.

natanielklug · March 19, 2013, 12:24am

joshaven,

In Brazil it’s not legal to have your authentication over the last mile. It has to be in your border. That’s a strange law but we have to be in the law… For the routing and IP distribution we use FreeRadius + PostgreSQL + Ippool… It’s awsome and distribute the IP over the network very well.

joshaven · March 19, 2013, 3:32am

That is a really strange law! Do they tell you what color of cable to use a well? We have our fair share of odd laws but that is overly intrusive and i cannot imagine the purpose.

natanielklug · March 19, 2013, 11:03am

The color of ethernet cable should be black in the towers and in outside instalations (like subscribers over UTP)… So the laws.

savage · September 23, 2014, 10:37pm

Sorry to bring up a old post, but how did you manage to install ROS on a R410? We’re seriously battling. Not detecting disks during boot, tried all the different SATA modes available in the BIOS, with/without RAID controller, etc…

Are you running something special like a Areca in that box?

natanielklug · September 24, 2014, 12:41am

Hello Chris,

Not a problem to bring back an old post. We had the same problems you said. To install we put this HD on another machine and installed Mikrotik. Then we put it back into R410 and start that up. I actually don’t remember anything more specific about this.

savage · September 24, 2014, 12:45am

Cewl - that and USB is the one thing I haven’t tried yet.

Updating all firmware and what not now (just to make sure), but I’ll be sure to try this out tomorrow when I have remote-hands available to do the drive swaps.