RouterOS 6.38.3's LLDP Craches Vulnerable Cisco Routers

RouterOS General
1

Hello,

I’m just trying to give heads up for those who may be affected.

Today one of our X86 machines have been updated to version 6.38.3 and after that both uplink Cisco routers started a crash loop.
After investigation noticed LLDP triggers a bug in some of the IOS versions.
The version that we were on was 12.2(33)SXI3
The only solution for us was to disable LLDP on the routers.

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCtj22354/
https://bst.cloudapps.cisco.com/bugsearch/bug/CSCun63132/
https://tools.cisco.com/security/center/content/CiscoSecurityAdvisory/cisco-sa-20160616-ios

Following is the debug output from crash dump:

(gdb) print *entry
$18 = {
  next = 0x0,
  prev = 0x0,
  idb = 0x0,
  rxInfoTTL = {
    mt_next = 0x0,
    mt_prev = 0x0,
    mt_head = 0x0,
    mt_union = {
      mt_down = 0xaaaaaaaa,
      mt_context = 0xaaaaaaaa
    },
    mt_exptime = {
      u = {
        value = 0,
        p = {
          high = 0,
          low = 0
        }
      }
    },
    mt_type = 0,
    mt_initialized = 0 '\000',
    mt_fence = 0 '\000',
    mt_leaf = 0 '\000',
    mt_istimer = 0 '\000',
    mt_sched_linked = 0 '\000',
    mt_proc_notify = 0 '\000',
    mt_intrpt_env = 0 '\000',
    mt_additional_context = 0x462abf90
  },
  chassis_id = {
    basic_tlv = {
      value = 0x49911edc "",
      length = 7,
      type = 1 '\001'
    },
    subtype = 4 '\004'
  },
  port_id = {
    basic_tlv = {
      value = 0xaaaaaaaa "\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252"...,
      length = 7,
      type = 2 '\002'
    },
    subtype = 5 '\005'
  },
  mgmt_addrs = 0x52638a00,
  remote_med_annex = 0x52638b40,
  port_descr = 0x47fa0be4 "",
  sys_name = 0x48d4ae74 "\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252\252"...,
  sys_descr = 0x52637e8c "MikroTik RouterOS 6.38.3 (st\252\252\252\252) x86",
  ttl = 43690,
  capabilities = 0,
  somethingChangedRemote = 0,
  port_vlan_id = 0,
  num_ma = 10 '\n',
  mau_type = 0,
  pmd_auto_neg = 0,
  auto_neg = 0 '\000',
  hash = 0 '\000',
  known_entry = 0 '\000'
}

Hopefully this helps someone

v6.38.3 [current]
2

“MikroTik, the Cisco killer” - sounds sort of nice, although one would expect it to be just a little exaggerated praise of MikroTik qualities, not a literal thing. :slight_smile:

3

Kinda old Vulnerable shure there must be upgrade from cisco out if your router is still supported

Getting cisco ios is easy even whitout an support deal

4

Cool, that’s not the first time MikroTik kills Cisco. Previous one was with BGP AS Path > 254 ASes, AFAIR :slight_smile:

5

Could this cause other devices to crash? I’m not a router expert and all my stuff is in my home.

I use Windows Media Center and last night I was watching TV and it just stopped… After 30 seconds or so the Xbox said it lost it’s network connection. I logged into the IMPI of my server (connected directly to my 3011 router) and the OS was locked up solid. The server payload traffic is connected to my 226 switch via 10Gbps fiber.

So it appears that something happened in the switch to cause a halt in all packets.

I just noticed .4 was available so I upgraded hoping this issue goes away.

Sent from my XT1650 using Tapatalk

6

.4 didn’t resolve this issue. So I rolled back to the Bug Fix only 6.37 build and everything has been up and trouble free for over 3 days now.

I ran into a similar issue last year when I was using a Netgear switch. All was good until they “fixed” something in a firmware update. Rolling it back cleared it up too.

Sent from my XT1650 using Tapatalk

7

And how is this a Mikrotik bug? You should upgrade your Cisco router instead.

8

I’ve been steady for two weeks after rolling back to 6.37.5 on my switch so… Good job Mikrotik for screwing up something that was working fine. Looks like my CRS-226 won’t be getting anymore updates.

Sent from my XT1650 using Tapatalk

9

So, you have a server, it receives LLDP packet from RouterOS and locks up. And it’s MikroTik’s fault (and Netgear’s too, lets not forget about them) for sending out packets that server does not like. Definitely not server’s fault for failing to correctly process input.

Maybe it’s just me, but locking up at the mere sight of some packet - no matter how much the server doesn’t like it, for any reason - sounds extreme to me, and I would probably tend to blame the server. :wink:

10

I have no idea what you just said… All I know is something in recent firmware upgrade doesn’t get along with Windows Media Center.

Sent from my XT1650 using Tapatalk