Why isn’t this NPK available for everyone? If winbox, webfig, etc can’t show proper indicators of compromise then it’s important that we can use CLI tools to verify there are no rogue binaries, especially as the screenshots seem to demonstrate that the updated RouterOS doesn’t actually remove persistent malware.
It isn’t available because MikroTik has decided that they do not want it available. As for verification, there are NUMEROUS examples in this thread of people saying how to tell you are infected and how to remove the infection.
If you have outbound traffic from the router on telnet (and maybe more) with no other explanation, then that router is infected.
To fix it, upgrade the router to the latest routeros (this fixes all known attacks)
To keep it from happening again, you should do what SHOULD be done BEFORE it is put on the internet and protect these ports.
Kind of .. if you have enabled filesystem for r/w operations, if any (if there are some) preventive locks are released with enabling “internals”, if worm has access to bigger part of system files … who knows what could happen. Even it happens then your example is not representative as car crash during WRC race is not representative example of common car crash for “average” driver.
BTW … I’m waiting when you publish this “100% official” access to Mikrotiks internals.
That is right, this is why I raise this. I would not like the entire network to reboot whenever a new release is made that may not even be important (e.g. it just introduces new features outside the scope of our network), but conversely I would like to offer the service to keep those that do not pay attention uptodate to a major version that is not known to be vulnerable.
So the version on the local server would be updated when a security advisory occurs and whenever a newer version is stable and desirable to have.
I would be happy to let it retrieve it from the MikroTik servers if only we got a capability to have some “custom channel” where a local server determines what is the target version.
(e.g. define a custom channel by specifying a server or router that has a file or variable holding the target version, and only upgrade, not downgrade, to that version)
We have done precisely what you describe. It’s a rather complicated script, because the MikroTik-supplied facility breaks down when the connection fails for any reason in mid transfer, etc., but the scripts we have written take care of all that and do run nightly. If you contact me off list, I’ll send you a copy that you can work from.
Ah, and breaking connections is of course a common occurrence when all routers in the network decide at 03:00 to fetch and upgrade their RouterOS…
I already thought of that problem. There should be some randomness in the schedule time, probably. And indeed a recovery when a transfer fails.
Off-list contact is disabled on this board. I would appreciate it when you mail me your script at my username @ amsat.org
There is a Posix (Linux/freebsd/etc) mactelnet tool available on the internet and with the “expect” tool it should be possible to make it connect to a list of routers and issue the required commands:
Of course it could be considered to offer some read-only commands like ps and ls via RouterOS commands. I have seen other embedded-linux deviced that have this.
Or, they could be made part of the supout (they probably already are…) and displayed in the supout viewer tool.
Let me tell you how I see if from the customer perspective.
Imagine that something (like mail notification) or someone (a colleague of mine) told me about “Urgent security advisory” that Mikrotik have published on their website/forum.
And for some reasons, I have a version that it lower than v6.38.5 on my device. After reading your first post in this thread I am installing “v6.38.5” and this let me think that I am safe.
But I am not. Because you did not mention that there was an even worse vulnerability (SMB or some other) that was discovered later and I have to install 6.41.3 to fix this latter issue as well.
Seems that that was exactly our case. We did install 6.40.5 when we discovered that our mikrotiks made ssh connections to our internal network and also there were established connections from mikrotiks to some other machines on the internet (may be it was a tunnel through mikrotik, I don’t know).
You are not publishing details about discovered vulnerabilities, even brief description and what your customers told you and what your support engineers discovered during their investigations of vulnerable devices.
You just telling us - upgrade to some version and you are safe.
For me, it seems that your very first fix was not complete and did not fully fix the issue.
That’s why we could see malicious processes even when we upgraded to 6.40.5.
And you solved this issue only when you released a subsequent fix (in the next version of firmware).
Since you have shell access, I can’t say how you got those files in your system.
This sounds funny: you blame me on everything just because I have this “shell access” on my device.
The scripts we have already randomize by the last nibble of the MAC address snd it works well. Link radios, CPEs, and core routers all have different time blocks. Mostly the interruptions are caused by storm activity or power outages.
Also, the MikroTik standard process doesn’t load NPKs in a safe order. Nothing is as much fun as when a CPE loads the system package first, then gets interrupted before downloading wireless. When it next comes up, it’s running the new system, but nobody can talk to It and you have to send out a truck.
I will ship you the script sometime before Monday. I have a full schedule for the next two days.
Under no circumstances should your device have unfirewalled and open SMB shares on the internet. If you don’t, this issue should not worry you.
We will put the fix into Bugfix once it is tested enough, the Bugfix versions must be highly stable before they are released, we don’t rush them.
Ah I thought the procedure was to first download all the packages, then install them all upon reboot… maybe this is different for that method.
Usually our devices run the bundle package, plus sometimes an optional package like ntp, So this should not be an issue.
And our network is sufficiently meshed to allow access to a device from wireless and ethernet in most cases.
Upon reviewing my code, I remembered what the problem was. There is a bug in /system upgrade where if you choose download-and-reboot, it will begin downloading packages – then if for some reason it doesn’t complete (like a network disconnection), it will timeout the download, but then will reboot anyway! This means that you end up with a machine that doesn’t have all the packages it needs, and if it’s a CPE and one of the missing packages is wireless, it’s off the network until you can drive out to it. Like you, we maintain redundant paths to almost all of our transport units, but CPEs are dead-ended by their nature.
We don’t use the consolidated NPK – to optimize resources, we load only the six or seven individual packages that we need.
Ironic wouldn’t be the word I would choose… Not having shell access in routeros has always been a serious flaw in a great product. And that this access is there without Mikrotik for the black hats now, pretty much invalidates any reason that we shouldn’t. Strange files were the easiest way of determining if UBNT products were hacked. Just telling us Upgrade and pray is completely the wrong tact to take. “Trust us we are on this” did not work for UBNT..