MikroTik

Is it possible to have Dude to store its database on external storages, like SD card or USB stick connected to USB port? The device is CCR1009, it has no dedicated SSD storage but I also don't want to wear out built-in flash with Dude data, so which is the best approach?

Another question is how to make SD/USB storage robust enough? The SD won't stay long under decent constant write load, neither most USB sticks as well, so which is the best way to overcome that? The only thing I can see is to put SSD into external disk box and connect it via USB but that's not en elegant approach, isn't it?

You can put dude storage onto USB or sd card without any problems. The durability depends on the model you choose... There are reliable professional flashes that you can choose if you are afraid of the lifetime.

Is it possible to have Dude to store its database on external storages, like SD card or USB stick connected to USB port? The device is CCR1009, it has no dedicated SSD storage but I also don't want to wear out built-in flash with Dude data, so which is the best approach?

Another question is how to make SD/USB storage robust enough? The SD won't stay long under decent constant write load, neither most USB sticks as well, so which is the best way to overcome that? The only thing I can see is to put SSD into external disk box and connect it via USB but that's not en elegant approach, isn't it?

yes it is possible (i used such setup on 3011).
as for speed, there was enough usn 3.0 for monitoring about 200 hosts.
but! There were problems after rebooting (i don't know about other, but in 3011) - RB loses disk and only low level format from PC help. This problem was with usb flash, hdd in 3.0 box, CF card in card reader.
as a result now i use CHR with dude for monitoring.

Отправлено с моего Redmi 3S через Tapatalk

There were problems after rebooting (i don't know about other, but in 3011) - RB loses disk and only low level format from PC help.

Am I right then, that 3011 loses disk despite the Dude uses it or not? That is, after the reboot the 3011 won't be able to use USB disk for any purpose? Then looks like it is a bug, the just a Dude problem.
I really interesting if CCR treats the same way the SD card, really! It should be able to find it before completely init finished, since SD is supposed to keep keys on it (yet I've never tried to do that). Moreover if you look at https://i.mt.lv/routerboard/files/CCR10 ... 141507.pdf you'll find CCR1009-7G-1C-1S+ have, in addition to USB port, as much as two SD slots (one named "smart card" on the front and another named "micro sd" on the rear panel), and also the block diagram https://i.mt.lv/routerboard/files/CCR10 ... 154504.png features three independent blocks: USB, micro-sd and smart-card. Hope one of these is able to be ready right at startup, isn't it?

P.S. By the way I can find no information on "Smart Card" slot that is there on several MT devices. No Wiki or whatever info, so looks like all reviews only uses slot name as a feature description! The only I was able to find is viewtopic.php?t=109531 this post which says "Card must support GlobalPlatform JavaCard" so this is not generic storage card to keep Dude database, then.

Yes, after rebooting 3011 even doesn't see this disk in usb devices. There are no problems with plugin other devices. 3011 start to work with this particular usb disk only after low level format of the disk (usual format doesn't help). As for SD card (if i don't make a mistake it is microsd) i think it will be rather difficult to find industry card that can be written a lot of times. As for testing: you can try on ccr to setup with few devices, export database and try to reboot (both soft reboot from menu and hard with unplugging power)

Отправлено с моего Redmi 3S через Tapatalk

you can try ... try to reboot

Will definitely do that but right now I have no spare CCR and I need this info to plan The Dude deployment. Simple to say, do I need to setup separate Dude server or I'll be able to go with CCR.

Considering your USB stick, did you tried to use different sticks? If other devices in the same USB port are OK after reboot, then the problem may be in USB stick itself or maybe the software write won't close the file and filesystem so the reboot breaks the very filesystem so unless you have the disk formated it is not possible to operate the filesystem anymore. I don't know now if it is possible to flush filesystem often or maybe disable cache so write operation will be done synchronously. And, what's ROS and firware version you're running your device on, may the update fix the issues?

you can try ... try to reboot

Will definitely do that but right now I have no spare CCR and I need this info to plan The Dude deployment. Simple to say, do I need to setup separate Dude server or I'll be able to go with CCR.

Considering your USB stick, did you tried to use different sticks? If other devices in the same USB port are OK after reboot, then the problem may be in USB stick itself or maybe the software write won't close the file and filesystem so the reboot breaks the very filesystem so unless you have the disk formated it is not possible to operate the filesystem anymore. I don't know now if it is possible to flush filesystem often or maybe disable cache so write operation will be done synchronously. And, what's ROS and firware version you're running your device on, may the update fix the issues?

I test it on different usb disk (about 5 or 6 both 2.0 and 3.0, one hdd disk in usb case, one CF card). All this disks were tested and works good (even when 3011 doesn't see them). I didn't test on the latest ROS, but on last 3 or 4 versions it was so

Отправлено с моего Redmi 3S через Tapatalk

on last 3 or 4 versions it was so

This is only in connection with The Dude? I mean, may this be due to way The Dude uses disk? if it is, then it is useful to fix The Dude, which is not the case considering the efforts MT put into The Dude during last time. "AHx4 with The Dude" edition features SSD storage which hopefully won't ruined down after the reboot, but who will care for 3011 or CCRs, then?

I tried to check ROS changelogs for USB fixes, but I find no huge changelog for The Dude package.

on last 3 or 4 versions it was so

This is only in connection with The Dude? I mean, may this be due to way The Dude uses disk? if it is, then it is useful to fix The Dude, which is not the case considering the efforts MT put into The Dude during last time. "AHx4 with The Dude" edition features SSD storage which hopefully won't ruined down after the reboot, but who will care for 3011 or CCRs, then?

I tried to check ROS changelogs for USB fixes, but I find no huge changelog for The Dude package.

Yes, only with dude - i have lots of setup with usb disk (with log, cache and so on) and other usb devices and all works good. It seems to be a problem of dude on 3011. I can't test it because i have only several 3011 other my devices don't support dude.

Отправлено с моего Redmi 3S через Tapatalk

Maybe it could be problematic with some older ros versions on 3011. Should be tried with 6.39.1 to be sure. My experience with crs125s, rb2011s, omnitiks, and others is that they do not loose USB sticks at all for years. I have moved the dude from CHRs to 750Gr3 with cheap but relatively big (to spread the wear) consumer grade micro sd and it works reliably without any outages or flash loosing. And I am doing backup, of course...

Oh, thank you!

But then, how big the database become over a time? What size of card should be used/planned?

Depending on how much data you need to store. Keeping additional 1 day of raw data makes the database to be about 40% bigger than with standard settings. Also other averages I am keeping for longer time. How many devices with how many values are you going to store? How often you will be checking them... All of it matters.

Oh, thank you!

But then, how big the database become over a time? What size of card should be used/planned?

All depends on how many devices and services you want to monitor, will you save logs in dude or in other log server and so on. For example in my setup about 200 (mostly servers), 3-15 probes on each device, i use separate log server, so the database is about 9-10 months and it is only about 40mb. But if you will save all logs in dude it will be much more bigger

Отправлено с моего Redmi 3S через Tapatalk

I have around 70 devices with around 5-10 services each, default polling intervals, 2days raw, 30days of 10min avg, 1year of 2h avg, 10 years of 1d avg. 3 years of recorded data. 230MB is dB file, 90MB is backup file. Response times in client are quite long - and few (10) seconds - when opening the graphs, even when looking at 48h viewframe...
Running on 16GB Kingston sdhc uhs-i card.

Looks like old good SNMP poller on PC will do better this way!

Unles some probes do not rely on ros commands...

Looks like old good SNMP poller on PC will do better this way!

It depends on what do you want to have. I use it for "fast" look on network and services. It sends me telegram notifications on critical events. It is easy to setup and use. for working with logs it is better (at least for me) to use other tools

Отправлено с моего Redmi 3S через Tapatalk

Unles some probes do not rely on ros commands...

I used to use API for this purpose, which also gives me more precious replies. For example, SNMP replay for temperature sometimes gives me pretty big number but via API it is within reasonable limits all the times.
I'd prefer to get stats to be send by ROS to my stat server (via trap for example) but this is not the case.