Community discussions

 
User avatar
NathanA
Long time Member
Long time Member
Topic Author
Posts: 695
Joined: Tue Aug 03, 2004 9:01 am

RouterOS making unaccounted outbound winbox connections

Sun Mar 25, 2018 10:03 am

I haven't seen any chatter about this on these forums or elsewhere...

Just tonight we discovered a multitude of RouterOS devices on our network -- mostly customer devices, so far only observed on MIPS architecture -- that appear to be infected with something. The routers themselves are generating hundreds of outbound connections every second to random IP addresses, targeting telnet (TCP port 23), TR-069 (TCP port 7547), and WINBOX!!!! (TCP 8291). I have confirmed that this traffic is not originating from a device on the customers' LANs and then getting NATted...it is coming from the router itself and is successfully blocked using firewall rules in the "output" chain.

Rebooting the device does not stop it, so whatever it is has managed to write itself to flash, as well as insert itself into the OS startup/boot process. So it knows its way around RouterOS internals, it isn't phased by the CPU architecture, it seems to know of some RouterOS vulnerability that it is exploiting somehow, and it targets Winbox.

The two devices I have examined remotely in detail are both running 6.34.2. (I know this is not current, but it is a customer device and we have not been forcing customers to update the software on their routers. This might have to change.)

I am hoping to collect a hardware sample soon, which will allow me to investigate the "worm" more in-depth and figure out where it has managed to hook itself into the OS. But maybe whatever this "worm" is is already known (and whatever security vulnerability still existed in 6.34.2 has since been patched), and I am just not up-to-date on the news. If anyone has any information about this I would really appreciate any leads.

-- Nathan
 
MatthiasMerkel
just joined
Posts: 3
Joined: Sat Mar 24, 2018 9:28 pm

Re: Aggressive RouterOS worm infecting our network??

Sun Mar 25, 2018 10:19 am

I haven't heard about that yet but it's certainly worth investigating. Do you happen to have a Packet Sniffer output file that contains the outbound connection you could share?
 
zikachu
newbie
Posts: 29
Joined: Sat May 26, 2007 4:08 am

Re: Aggressive RouterOS worm infecting our network??

Sun Mar 25, 2018 12:17 pm

use netinstall for fresh start and keep update your version.
 
User avatar
NathanA
Long time Member
Long time Member
Topic Author
Posts: 695
Joined: Tue Aug 03, 2004 9:01 am

Re: Aggressive RouterOS worm infecting our network??

Sun Mar 25, 2018 12:39 pm

use netinstall for fresh start and keep update your version.
:roll: Thanks. It's not like I don't know how to do a Netinstall. But we are potentially talking about a few hundred devices here. Anyway, the real question is not how to recover from it, but what this worm is precisely, and how to protect ourselves from it. For all I know, it could be exploiting a zero-day that hasn't been fixed in a current version of RouterOS. In that case, Netinstalling a few hundred devices is going to be a big fat waste of time, don't you think? They will all just become infected again.

I'm also surprised that I haven't heard anything from anyone else being hit by something similar...

-- Nathan
 
User avatar
soulflyhigh
Member Candidate
Member Candidate
Posts: 165
Joined: Wed Sep 08, 2010 11:20 am

Re: Aggressive RouterOS worm infecting our network??

Sun Mar 25, 2018 1:11 pm

Hello Nathan,
we have seen it last night too, mostly on 6.34.x version, SXTs/LHGs - mipsbe.
I managed first to block the telnet traffic to and from them and then I updated them to latest current version 6.41.3.

All affected Mikrotik routers had public IPs and exposed common service ports (80,21,22,23, etc) to the internet but whatever it was it didn't or couldn't change their login data.
I was very scared until I discovered what caused this - a year ago one of our partners forgot to apply the common firewall rules to a batch of CPEs and nobody noticed it until now.

The attack had dramatic effect on our radius server but AFAIK it caused no other damage.
We are following logs and so far it looks like that this is over.
So solution is just common sense - always use regular firewall rules and updated ROS.

Regards,
M.
MTCRE, MTCTCE
 
pe1chl
Forum Guru
Forum Guru
Posts: 4079
Joined: Mon Jun 08, 2015 12:09 pm

Re: Aggressive RouterOS worm infecting our network??

Sun Mar 25, 2018 1:28 pm

yes, there clearly is a problem!
we are not affected because we do not have management ports open to the internet, but scanning the internet link shows a lot of
incoming connects to port 8291 from wildly varying addresses. this clearly points to a worm.
(the usual scans from the self-appointed "security scanners" and "researches" have always been there but this is different)
 
jarda
Forum Guru
Forum Guru
Posts: 7139
Joined: Mon Oct 22, 2012 4:46 pm

Re: Aggressive RouterOS worm infecting our network??

Sun Mar 25, 2018 1:41 pm

Just a side note. Even I have never saw such behaviour I always populate the output firewall chain by similar rules like in input with final drop rule.

As it was mentioned long time ago, the consistency check or downgrade / upgrade exercise should delete everything else than installed packages.

Mikrotik always stated that there is no way to run custom code in ros. Hope this was not proved as a lie by this.

I hope also that the bad code is not part of standard ros packages distribution...
 
pe1chl
Forum Guru
Forum Guru
Posts: 4079
Joined: Mon Jun 08, 2015 12:09 pm

Re: Aggressive RouterOS worm infecting our network??

Sun Mar 25, 2018 2:19 pm

Mikrotik always stated that there is no way to run custom code in ros. Hope this was not proved as a lie by this.
Of course we have always known this is just a naive claim. There is no way they can back that up, because in all software there are bugs and those could
be used to circumvent security measures. That it did not happen yet does not mean it is impossible, maybe it has not been tried or they did not notice it.

Also, there is little transparency. At the minimum, there should be some list of vulnerabilities and version where they have been fixed, so that users can
quickly judge if existing installations are at risk. "always install the latest version" does not cut it, because there are developments as well, and newer
versions do not always have the functionality that is required or they require further work before they can be deployed (e.g. version 6.41)

At the moment I see a large flow of incoming connects from residential addresses, which clearly points to a botnet. It is different from the always-present
scans from virtual servers done by the wellknown losers. It can no longer be denied, and all previous attempts at getting things rectified have mostly
gone to waste due to the denial of there being a problem. Let's see what happens now.
 
hapi
Member Candidate
Member Candidate
Posts: 187
Joined: Fri Mar 11, 2011 11:21 am
Location: Czech Republic

Re: Aggressive RouterOS worm infecting our network??

Sun Mar 25, 2018 2:55 pm

Vault 7?
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 23015
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: Aggressive RouterOS worm infecting our network??

Sun Mar 25, 2018 5:26 pm

Nathan, RIF and if possible remote access is possible? RC on these or not?
No answer to your question? How to write posts
 
User avatar
soulflyhigh
Member Candidate
Member Candidate
Posts: 165
Joined: Wed Sep 08, 2010 11:20 am

Re: Aggressive RouterOS worm infecting our network??

Sun Mar 25, 2018 7:51 pm

Nathan, RIF and if possible remote access is possible? RC on these or not?
I just sent you a supout file from one of remaining infected SXTs to Mikrotik support email.
Let me know if I could help with this problem.

Regards,
M.
MTCRE, MTCTCE
 
dst
just joined
Posts: 2
Joined: Sun Mar 25, 2018 8:05 pm

Re: Aggressive RouterOS worm infecting our network??

Sun Mar 25, 2018 8:22 pm

Yeah, it has begun... I have 3 routers on remote locations infected. I cant access the router physically so i needed to fix it the time being.
After some attemps i got it under control, adding some filter rules to my firewall and blocking everyone except my ip(s).

Here are the rules, hope it helps someone. Replace YOURIP. Better turn on safe mode before typing.
Forwarding is not affected by that rules.

/ip firewall filter
add chain=input dst-address-type=local src-address=YOURIP
add action=drop chain=input dst-address-type=local protocol=tcp
add action=drop chain=input dst-address-type=local protocol=udp
add chain=output dst-address=YOURIP src-address-type=local
add action=drop chain=output protocol=tcp src-address-type=local
add action=drop chain=output protocol=udp src-address-type=local


Best regards, John
Last edited by dst on Sun Mar 25, 2018 11:00 pm, edited 1 time in total.
 
srosen
just joined
Posts: 4
Joined: Fri Mar 17, 2006 5:25 am

Re: Aggressive RouterOS worm infecting our network??

Sun Mar 25, 2018 9:14 pm

We were seeing what sounds like it may be the same exploit yesterday. I noticed telnet login attempts from some of our Mikrotiks to other Mikrotiks on our network. When I went to the source box, I could see in the logs something to the effect of "fetch: downloaded .i " (I apologize, I didn't note the exact verbiage, but it did reference a file named ".i"). So, my guess is the .i file was some kind of script that was able to run on these routers and then start attempting telnet attempts. We believe that it was trying logins with common user names and password combinations (i.e., admin, root, ubnt, Administrator, etc.), as we were seeing these login attempts to other Mikrotiks on our network. It looked like the script was probably trying addresses close to it's own IP address and then expanding the addresses outward from there.

We had another unrelated network (one of our admins has another network he runs) where they were seeing the same issue. On our network, we had about a half dozen machines that appeared to be exploited. All of them had http service enabled (a bad oversight on my part) and almost all of them were running older routerOS (typically 6.27 to 6.35.4). On the other network they had at least a dozen machines and they all had older RouterOS and http enabled) However, one of the compromised machines on my network was running 6.39.3. We had believed that this was most likely from a known security bug in RouterOs prior to 6.38.5 (What's new in 6.38.5 (2017-Mar-09 11:32): !) www - fixed http server vulnerability;), although the 6.39.3 exploited machine has us concerned that it may be something different. We did not have any machines exploited that were not running the http service, so we're mostly convinced that was the entry point.

We had one machine that did appear to continue running the script after disabling http and rebooting. So, it does seem like it was something that was embedded in the boot process. We've since disabled http on all machines and upgraded all pre 6.38.5 routers to 6.41.3 (we still need to upgrade some of our 6.39.3 machines, but we needed to fix the most vulnerable first). We also installed input and output chain filters to all machines to block connections from outside our network, so that will hopefully mitigate some of these issues.

Hope these data points might be useful.

Scott
 
User avatar
manuzoli
Frequent Visitor
Frequent Visitor
Posts: 59
Joined: Mon Oct 03, 2016 6:47 pm

Re: Aggressive RouterOS worm infecting our network??

Sun Mar 25, 2018 9:29 pm

hey, thx for the thread. our network had a major attack today as well. it seems like they opened some devices via the http port (quite a old firmware - damn) and they tried to spread or access by brute forcing mikrotik neighbours. quite a strange behaviour, because they could access passwords and we have the same password for all devices.
i did a portscan and one of the devices has port 61267 open. i am trying to figure out a solution at the moment.
 
User avatar
NathanA
Long time Member
Long time Member
Topic Author
Posts: 695
Joined: Tue Aug 03, 2004 9:01 am

Re: Aggressive RouterOS worm infecting our network??

Sun Mar 25, 2018 9:38 pm

Normis,

I'm in the middle of some other things at the moment but me see what I can do about remote access for you in a bit here. Do you want me to just e-mail support@ or message you some other way?

srosen,

Interesting. I don't think HTTP is the vector (though I could be wrong), mostly because I don't see infected devices making any attempts to contact other IPs via HTTP...just telnet, TR-069, and Winbox. In our case we haven't seen any kind of visible script, either on the (exposed) filesystem or in '/system script', nor are there any active scripts/jobs running nor logged in users/sessions that are unaccounted for.

Interestingly, I just ran across a device running 6.37.4 that also showed signs of infection, but this time they disappeared after a reboot.

-- Nathan
 
pe1chl
Forum Guru
Forum Guru
Posts: 4079
Joined: Mon Jun 08, 2015 12:09 pm

Re: Aggressive RouterOS worm infecting our network??

Sun Mar 25, 2018 10:21 pm

I have a /16 subnet on internet that I can trace, and I see about 100 TCP SYN requests per second to port 8291 to this network.
(from many different IP addresses on internet to random addresses in this network)
These are blocked in the firewall of the router towards that network.
I have tried to telnet back to a couple of those addresses and several times a RouterOS logon came back.
Looking up reverse for those addresses shows them to be residential.
 
srosen
just joined
Posts: 4
Joined: Fri Mar 17, 2006 5:25 am

Re: Aggressive RouterOS worm infecting our network??

Sun Mar 25, 2018 10:28 pm

srosen,

Interesting. I don't think HTTP is the vector (though I could be wrong), mostly because I don't see infected devices making any attempts to contact other IPs via HTTP...just telnet, TR-069, and Winbox. In our case we haven't seen any kind of visible script, either on the (exposed) filesystem or in '/system script', nor are there any active scripts/jobs running nor logged in users/sessions that are unaccounted for.
Hi Nathan -

For clarity, we were seeing telnets from the compromised machines (failed login attempts). What I was thinking was that the initial exploit that allowed the code to run was an exploit of an http security hole. All of the machines that were compromised had http (i.e., the www service on the Mikrotik) enabled - no machines that had this service disabled were compromised.

As far as the "script" - that may not have been good terminology. What I know is that the logs on the compromised machines showed the ".i " file having been downloaded. My assumption was that it was running some kind of "process" (be it a script process or some code kept in memory only), but clearly the ".i" file had something to do with the exploit. I just checked the scripts on one of the machines that had previously been compromised, and there's no script by that name on it now, and there weren't any files on it by that name at the time of the exploit, either.

If you find another compromised machine (hopefully you don't), you may want to check the logs to see if you see anything like the "fetch download .i file" messages that I had seen.

Scott
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 23015
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: Aggressive RouterOS worm infecting our network??

Sun Mar 25, 2018 10:37 pm

Can others please also check if upgrade to 6.40+ (or even latest RC) fixes this? We did implement additional internal checks and checksums in later versions to prevent any 3rd party files from being copied into any router, even if it has no password and firewall
No answer to your question? How to write posts
 
jarda
Forum Guru
Forum Guru
Posts: 7139
Joined: Mon Oct 22, 2012 4:46 pm

Re: RouterOS making unaccounted outbound winbox connections

Sun Mar 25, 2018 11:04 pm

I checked my networks several times during the time we talk here about this and still I can not find any signs of the infection. At least so far. The oldest devices are around 6.25, the newest around 6.40 and all do as usual without any strange traffic. Just time to time some login attempts to admin user without success, because there is no such user at all...
 
User avatar
maznu
Member Candidate
Member Candidate
Posts: 124
Joined: Tue May 05, 2015 11:12 am
Location: Manchester, UK
Contact:

Re: RouterOS making unaccounted outbound winbox connections

Sun Mar 25, 2018 11:19 pm

360 Netlab are tweeting: "So the old Hajime botnet is coming back with a new exploit which was published only about 13 days ago ( https://www.exploit-db.com/exploits/44284/ ), it also looks for some old exploits like tr-064 but nothing exciting there." — https://twitter.com/360Netlab/status/977932206835462146

They spotted increased scanning earlier in the day — https://twitter.com/360Netlab/status/977732944273068032
Marek
 
dst
just joined
Posts: 2
Joined: Sun Mar 25, 2018 8:05 pm

Re: RouterOS making unaccounted outbound winbox connections

Sun Mar 25, 2018 11:23 pm

I am counting the hosts trying to compromise my network, in the last one hour i counted 15500 unique hosts attacking 768 IP's, every hour the count is incrementing by, maybe, 400-500 hosts atm.

The attacks started yesterday 15:00 GMT

Regards, John
You do not have the required permissions to view the files attached to this post.
 
R1CH
Member
Member
Posts: 447
Joined: Sun Oct 01, 2006 11:44 pm

Re: RouterOS making unaccounted outbound winbox connections

Mon Mar 26, 2018 4:30 am

If the worm is targeting winbox ports something is seriously broken, even with admin access to winbox you shouldn't be able to get arbitrary code execution. At this point I'm wondering if any Mikrotik service is safe!
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 23015
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: RouterOS making unaccounted outbound winbox connections

Mon Mar 26, 2018 9:08 am

Just pointing out the obvious:

What's new in 6.38.5 (2017-Mar-09 11:32):
!) www - fixed http server vulnerability;)

Above mentioned devices were running older software. The vulnerability exploited open WWW port on the RouterOS devices.
No answer to your question? How to write posts
 
User avatar
manuzoli
Frequent Visitor
Frequent Visitor
Posts: 59
Joined: Mon Oct 03, 2016 6:47 pm

Re: RouterOS making unaccounted outbound winbox connections

Mon Mar 26, 2018 9:21 am

@normis
Yeah we were aware of the bugfix release, we just had some devices haven't had the license to upgrade to 6.x. Being a software developer myself I know it's not Mikrotiks fault - updates were available and should have been deployed by admins. But for those people who have been infected nonetheless this thread might be helpful.
Do you know if updating already infected devices will solve the problem or do we need to flash the devices?

@all
- Disabling www services and restarting the devices seemed to help.
- Our devices have no strange open ports anymore.
- Most, if not all, infected devices showed a red www service in the Winbox.
- We coded a Powershell-Script to change passwords on multiple mikrotiks at once - if anyone needs smthg like this, i can upload it to github
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 23015
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: RouterOS making unaccounted outbound winbox connections

Mon Mar 26, 2018 9:32 am

some devices haven't had the license to upgrade to 6.x
Just note that license upgrade is free of charge, if the device doesn't allow you to upgrade by upload+reboot, we will give you a free license upgrade.
Do you know if updating already infected devices will solve the problem
YES, RouterOS has been hardened multiple times since that version, it will not only double check every file, it will also remove all unknown files.
No answer to your question? How to write posts
 
User avatar
manuzoli
Frequent Visitor
Frequent Visitor
Posts: 59
Joined: Mon Oct 03, 2016 6:47 pm

Re: RouterOS making unaccounted outbound winbox connections

Mon Mar 26, 2018 9:41 am

some devices haven't had the license to upgrade to 6.x
Just note that license upgrade is free of charge, if the device doesn't allow you to upgrade by upload+reboot, we will give you a free license upgrade.
wow thats great - i think i had some old information there.
keep your great work up! thx a lot
Do you know if updating already infected devices will solve the problem
YES, RouterOS has been hardened multiple times since that version, it will not only double check every file, it will also remove all unknown files.
thats very good news! thank you very much
 
User avatar
petrb
Frequent Visitor
Frequent Visitor
Posts: 65
Joined: Thu Jan 26, 2017 4:17 pm

Re: RouterOS making unaccounted outbound winbox connections

Mon Mar 26, 2018 10:02 am

2 normis: thanks for our quick reaction (I have 6.39.3 at the about 500 devices) and I can sleep well now. Just old bug with new wave.
Did you read manual? .... What? .... Read the manual.
 
Kindis
Member Candidate
Member Candidate
Posts: 170
Joined: Tue Nov 01, 2011 6:54 pm

Re: RouterOS making unaccounted outbound winbox connections

Mon Mar 26, 2018 10:15 am

Would it be possible for MT staff to create a pinned thread about this. With info about what attack vector was used, from what version of ROS are you safe and how to mitigate if you are effected.
I know you can read all here, if HTTP vulnerability is confirmed to be used, but it would be great to have everything in one post for all to read.
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 23015
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: RouterOS making unaccounted outbound winbox connections

Mon Mar 26, 2018 10:24 am

Would it be possible for MT staff to create a pinned thread about this. With info about what attack vector was used, from what version of ROS are you safe and how to mitigate if you are effected.
I know you can read all here, if HTTP vulnerability is confirmed to be used, but it would be great to have everything in one post for all to read.
We will do that once we have more confirmation that upgrade to v6.38.5 or above does indeed fix this. Please report your results.
No answer to your question? How to write posts
 
Kindis
Member Candidate
Member Candidate
Posts: 170
Joined: Tue Nov 01, 2011 6:54 pm

Re: RouterOS making unaccounted outbound winbox connections

Mon Mar 26, 2018 10:29 am

Would it be possible for MT staff to create a pinned thread about this. With info about what attack vector was used, from what version of ROS are you safe and how to mitigate if you are effected.
I know you can read all here, if HTTP vulnerability is confirmed to be used, but it would be great to have everything in one post for all to read.
We will do that once we have more confirmation that upgrade to v6.38.5 or above does indeed fix this. Please report your results.
Great. Personally not affected on any device but I'm on 6.41.3 on every device but I still follow this to know what the attack vector was.
 
pe1chl
Forum Guru
Forum Guru
Posts: 4079
Joined: Mon Jun 08, 2015 12:09 pm

Re: RouterOS making unaccounted outbound winbox connections

Mon Mar 26, 2018 10:43 am

I did a trace on the /16 network and captured the IP addresses connecting to port 8291 counting the number of attempts from each.
After removing the top ones (that obviously are scanning servers that are always on the hunt to find and classify systems like this)
I have over 345.000 unique IP addresses of possibly infected routers. Connecting random addresses from this list on the HTTP
port indeed shows RouterOS in some cases, in other cases an inoperative webserver (accepts the connection but does not reply
to a GET / HTTP/1.0)
 
chadd
Member
Member
Posts: 347
Joined: Fri Dec 31, 2004 2:40 am

Re: RouterOS making unaccounted outbound winbox connections

Mon Mar 26, 2018 10:56 am

I had several devices effected by this, it seemed like they were all version 6.35 to 6.37.5 ish and only devices that were accessible from the internet. I upgraded to the latest current version of 6.41.3 and it took care of the problem. As someone else had mentioned the devices all had an entry in the log file about a file being downloaded from/to the device.

I am just happy that an upgrade fixed the issue without having to go through the nightmare that I had with UBNT devices when they had the worm problem.

MT is there any definite word on how these devices were vulnerable? Was it for sure through the web interface? Also is it known if this gave them access to device configs, passwords etc?
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 23015
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: RouterOS making unaccounted outbound winbox connections  [SOLVED]

Mon Mar 26, 2018 11:19 am

Upgrade to v6.38.5 or above should fix the issue and clean your system. To be 100% safe, upgrade to Current v6.41.3, as there are more double-checks included.

It seems the system scans for winbox port 8291 to quickly identify MikroTik devices (since this is a unique port), then checks if port 80 is available, then uses the vulnerability that was fixed in 6.38.5 to copy itself into that system and starts the same process again.

To be absolutely sure about the above, we would love to see remote access or RIF files from affected devices. Also the file it tries to fetch.
No answer to your question? How to write posts
 
pe1chl
Forum Guru
Forum Guru
Posts: 4079
Joined: Mon Jun 08, 2015 12:09 pm

Re: RouterOS making unaccounted outbound winbox connections

Mon Mar 26, 2018 11:34 am

checks if port 80 is available, then uses the vulnerability that was fixed in 6.38.4 to copy itself into that system
Does it require valid username and password to do that, or is it sufficient to have access to the webserver?
 
sid5632
Member Candidate
Member Candidate
Posts: 169
Joined: Fri Feb 17, 2017 6:05 pm

Re: RouterOS making unaccounted outbound winbox connections

Mon Mar 26, 2018 11:59 am

It seems the system scans for winbox port 8192 to quickly identify MikroTik devices (since this is a unique port)...
I think you mean 8291 :shock:
 
User avatar
maznu
Member Candidate
Member Candidate
Posts: 124
Joined: Tue May 05, 2015 11:12 am
Location: Manchester, UK
Contact:

Re: RouterOS making unaccounted outbound winbox connections

Mon Mar 26, 2018 12:01 pm

checks if port 80 is available, then uses the vulnerability that was fixed in 6.38.5 to copy itself into that system
Does it require valid username and password to do that, or is it sufficient to have access to the webserver?
If it's a botnet using ChimayRed, as suggested earlier in the thread, then it's exploiting a security vulnerability — it doesn't need valid credentials to gain access, just an open HTTP server :(
Marek
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 23015
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: RouterOS making unaccounted outbound winbox connections

Mon Mar 26, 2018 12:22 pm

If it's a botnet using ChimayRed, as suggested earlier in the thread, then it's exploiting a security vulnerability — it doesn't need valid credentials to gain access, just an open HTTP server :(
Yes. Webfig specifically. Hotspot NOT affected.
No answer to your question? How to write posts
 
User avatar
NathanA
Long time Member
Long time Member
Topic Author
Posts: 695
Joined: Tue Aug 03, 2004 9:01 am

Re: RouterOS making unaccounted outbound winbox connections

Mon Mar 26, 2018 12:48 pm

I finally got my hands on an infected device, spent some time with it, and can confirm that this appears to be Hajime, as maznu mentioned earlier.

I haven't been able to catch the infection happening live yet, but I am now pretty confident that this is exploiting the old web server vuln that was already patched before 6.39. I can confirm all infected devices I've found so far had port 80 open on them. :( (All infected routers were customers, and thankfully not our network core, which were properly protected. Our customers can use whatever router they want, and we offer to sell MikroTiks to them if they want to buy a router from us, but as mentioned earlier we don't actively maintain them. So many of them run with various configs that I'm sure are unsafe, and the software is not going to get updated unless the customer does it. We have resisted "managed router service" until now but this may no longer be tenable.)

Also, I was wrong about the 6.37.4 device earlier not showing symptoms after a reboot. I just didn't wait long enough.

In case anyone is curious:

Through the vulnerable web server, commands are issued to download a file called ".telnetd" to /flash/bin. Of course, it is not actually a Telnet daemon. :? When executed, ".telnetd" creates a named pipe at /flash/bin/fifo and a directory at /flash/bin/.p and then connects to the Hajime P2P network to grab the latest version of the loader (.telnetd) and the .i and atk modules which it puts in the .p directory. At that point it starts its outward attacks.

A startup script is also placed at /flash/etc/rc.d/run.d/S99telnetd which causes /flash/bin/.telnetd to launch at boot every time, so once the device is infected it will remain infected until you take steps to remove it. Interestingly, the Hajime loader does not create this script; it must be that whatever external agent exploited the vulnerability on your router put it there along with the Hajime loader binary.

I can confirm that upgrading to 6.40.6 removes the /flash/etc/rc.d directory tree, which of course deletes the startup script and thus renders the Hajime binaries in /flash/bin inert. So an infected router can in fact be "cured" just by updating it.

-- Nathan

EDIT: I forgot to mention that some devices don't respond well to Hajime. I ran into a 450G that I think was running 6.19 when it got infected, and the Hajime loader actually causes it to reboot itself when it is started! So because of the startup script, the 450G was stuck in a boot loop.
 
User avatar
maznu
Member Candidate
Member Candidate
Posts: 124
Joined: Tue May 05, 2015 11:12 am
Location: Manchester, UK
Contact:

Re: RouterOS making unaccounted outbound winbox connections

Mon Mar 26, 2018 1:38 pm

I finally got my hands on an infected device, spent some time with it, and can confirm that this appears to be Hajime

I can confirm that upgrading to 6.40.6 removes the /flash/etc/rc.d directory tree, which of course deletes the startup script and thus renders the Hajime binaries in /flash/bin inert. So an infected router can in fact be "cured" just by updating it.
Excellent work! Many thanks for this, Nathan!
Marek
 
pe1chl
Forum Guru
Forum Guru
Posts: 4079
Joined: Mon Jun 08, 2015 12:09 pm

Re: RouterOS making unaccounted outbound winbox connections

Mon Mar 26, 2018 3:36 pm

checks if port 80 is available, then uses the vulnerability that was fixed in 6.38.4 to copy itself into that system
Does it require valid username and password to do that, or is it sufficient to have access to the webserver?
If it's a botnet using ChimayRed, as suggested earlier in the thread, then it's exploiting a security vulnerability — it doesn't need valid credentials to gain access, just an open HTTP server :(
I think it is a bit imprudent to run the webserver under a user-id that has write access to the filesystem. No Linux system admin would do that; all wellknown Linux distributions run the webserver under a user like "wwwrun", "httpd" or so, which has no write permissions on the persistent filesystem.
Even when the server has a vulnerability, it would be more difficult to exploit it when you cannot write files and any code you execute does not run as root.
Action routines that need root privilege could be implemented as a setuid or sudo program that is easier to guard against abuse.
 
pe1chl
Forum Guru
Forum Guru
Posts: 4079
Joined: Mon Jun 08, 2015 12:09 pm

Re: RouterOS making unaccounted outbound winbox connections

Mon Mar 26, 2018 3:45 pm

All infected routers were customers, and thankfully not our network core, which were properly protected. Our customers can use whatever router they want, and we offer to sell MikroTiks to them if they want to buy a router from us, but as mentioned earlier we don't actively maintain them. So many of them run with various configs that I'm sure are unsafe, and the software is not going to get updated unless the customer does it. We have resisted "managed router service" until now but this may no longer be tenable.
This is going to be tough...
I did a quick scan of our HAMNET network and mailed a couple of users with old versions.
Of course precisely the people who have not kept their routers uptodate are also the ones that feel unsure how to do that
and how much effort it will be.
I already got a reply that says "at the moment I am too busy but I hope to find time for this in the upcoming days".
Anyone who did an update before will know that it will take less time to do this than to compose an e-mail reply like that,
so I mailed him a description how to do it. Fortunately he has done the update now.
You'll probably need to send a newsletter to your clients with enough detail to make them do the update without encountering
issues that make them postpone it. And that is not really easy because in this version range the web interface has been
changed a number of times, so screenshots included in the mailing will likely look different from what they have.
 
User avatar
macgaiver
Forum Guru
Forum Guru
Posts: 1634
Joined: Wed May 18, 2005 5:57 pm
Location: Sol III, Sol system, Sector 001, Alpha Quadrant

Re: RouterOS making unaccounted outbound winbox connections

Mon Mar 26, 2018 3:54 pm

Don't you have a point in the user agreement, if they use Internet for illegal activities you have the right to disconnect them?

Allow only mikrotik.com domain for them until they have upgraded, and just redirect them to warning page that explains what is the problem.
Works like a charm here.
With great knowledge comes great responsibility, because of ability to recognize id... incompetent people much faster.
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 23015
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: RouterOS making unaccounted outbound winbox connections

Mon Mar 26, 2018 3:55 pm

I know and understand, but unfortunately, those people also don't read our newsletters and emails.
No answer to your question? How to write posts
 
User avatar
BartoszP
Forum Guru
Forum Guru
Posts: 1424
Joined: Mon Jun 16, 2014 1:13 pm
Location: Poland

Re: RouterOS making unaccounted outbound winbox connections

Mon Mar 26, 2018 4:40 pm

@macgaiver
Isn't your proposal going hand-in-hand with the new sex trafficking law :-) ? https://www.wired.com/story/how-a-contr ... e-the-web/
Real admins use real keyboards.
 
CsXen
Frequent Visitor
Frequent Visitor
Posts: 56
Joined: Wed Sep 10, 2014 8:31 pm
Location: Budapest - Hungary

Re: RouterOS making unaccounted outbound winbox connections

Mon Mar 26, 2018 5:32 pm

Hi Normis!
You wrote:
Upgrade to v6.38.5 or above should fix the issue and clean your system.
Can you tell me please, how can I upgrade my RB532A (mipsle) to this level ? :)
Please make a security release for those old, but perfectly working boards on mipsle!

Thanks and best regards: Xen
 
GamacchioAssiderato
just joined
Posts: 3
Joined: Fri Dec 29, 2017 2:38 pm

Re: RouterOS making unaccounted outbound winbox connections

Mon Mar 26, 2018 8:49 pm

RouterOS 6.38.4 is still vulnerable to chimay red!!
You have to update at least to 6.38.5
But please pay attention that if you have services like samba enabled, you have to update to 6.41.3 because of this https://github.com/BigNerd95/Chimay-Blue
 
CsXen
Frequent Visitor
Frequent Visitor
Posts: 56
Joined: Wed Sep 10, 2014 8:31 pm
Location: Budapest - Hungary

Re: RouterOS making unaccounted outbound winbox connections

Tue Mar 27, 2018 1:16 am

Hi.
I have an infected RB951 with firmware 6.32.4.
I detected strange traffic on UDP/33549 port. In the packets some readable parts:
d2:ip6:T.e.. .1:ad2:id20:..6i sQ.J.)......F|.g e1:t4:....1:q4:p ing1:y1:qe
E..}.n@. 6.....O.T.e..... .i..d1:ad2:id20: .f.;6.......O.Q. S..N9:info_hash2 0:/......[.5.e.. ;.&.,.e1:q9:get_ peers1:t4:gp/.1: y1:qe
You can read: ip6, ad2:id20, ping1, infohash20, get_peers

So... not just copying themself from one router to another, but there is some hidden activity I think!

Best regards: Xen
 
User avatar
NathanA
Long time Member
Long time Member
Topic Author
Posts: 695
Joined: Tue Aug 03, 2004 9:01 am

Re: RouterOS making unaccounted outbound winbox connections

Tue Mar 27, 2018 1:42 am

Please make a security release for those old, but perfectly working boards on mipsle!
In my experience, the last version of RouterOS to work *well* on RB532 was 5.x. :( When I upgrade a 532 to 6.x it starts acting like a RB100-series board that has just been upgraded from 2.9 to 3.x or anything newer...slow, laggy console, CPU spiking up all the time, etc. even when it really isn't busy doing anything. I'm not sure why this is or why a similar thing happened in both cases (100 and 500) and MT devs couldn't fix it, but there you go. I would suggest the time has come to retire the venerable RB500.
I detected strange traffic on UDP/33549 port. In the packets some readable parts:
[snip]
So... not just copying themself from one router to another, but there is some hidden activity I think!
Read up on Hajime. Rather than phoning home to a centralized command & control server, it constructs a BitTorrent-like P2P mesh with other infected devices, and they all pass along information to each other. It's primarily used to transmit the latest config and to distribute software updates. From my experiments yesterday, it appears that every time Hajime starts up on an infected host, it picks a random UDP port to listen on and make the P2P exchange with, so if you reboot your infected router, then you will see the same traffic coming/going to a different UDP port #.

-- Nathan
 
pe1chl
Forum Guru
Forum Guru
Posts: 4079
Joined: Mon Jun 08, 2015 12:09 pm

Re: RouterOS making unaccounted outbound winbox connections

Tue Mar 27, 2018 11:11 am

I did another night of scanning and this time I have found a little over 370.000 infected devices.
Compared to yesterday's 345.000 addresses, about 190.000 addresses have vanished and 215.000 new ones have appeared.
One can only hope that it was that many that have already been updated...

I did not yet see any activity inside our local network (added logging of TCP port 8291 SYN packets passing through some of the routers).
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 23015
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: RouterOS making unaccounted outbound winbox connections

Tue Mar 27, 2018 11:19 am

Can you tell me please, how can I upgrade my RB532A (mipsle) to this level ? :)
Please make a security release for those old, but perfectly working boards on mipsle!
Firewall protects against this issue. You should only worry if your MIPSLE device has no firewall on the internet side. You can also simply Netinstall to clear away any malicious files.
No answer to your question? How to write posts

Who is online

Users browsing this forum: pe1chl, Steveocee and 42 guests