Community discussions

MUM Europe 2020
 
jarda
Forum Guru
Forum Guru
Topic Author
Posts: 7602
Joined: Mon Oct 22, 2012 4:46 pm

Cheap multi-NIC devices to run CHR in some supervisor

Thu Sep 01, 2016 1:11 pm

Dears,
have you ever thought or tried to run CHR in some supervisor on anything like this?
https://www.aliexpress.com/item/Eglobal ... .70.cT23SM
or this:
https://www.aliexpress.com/item/HCiPC-B ... .76.cT23SM

Some manufacturers webpage:
http://www.hcipctech.com/Home/ProductLi ... &english=2
Image
Image
Image

Or, if not virtualized, which is my goal, to run Ros x86 on bare metal?

If you have it running, what are your achievements and experience with such around 100-150 usd devices to run ROS under vSphere or XEN?

This is not any promotion, links and pictures are just for descriptive purposes only, if you have experience with anything similar, please, share.
Thank you.
 
jarda
Forum Guru
Forum Guru
Topic Author
Posts: 7602
Joined: Mon Oct 22, 2012 4:46 pm

Re: Cheap multi-NIC devices to run CHR in some supervisor

Sat Sep 03, 2016 11:39 am

Two days are gone and none has experience with this? I do not believe it...
 
pe1chl
Forum Guru
Forum Guru
Posts: 6130
Joined: Mon Jun 08, 2015 12:09 pm

Re: Cheap multi-NIC devices to run CHR in some supervisor

Sat Sep 03, 2016 1:51 pm

I don't think you will be able to run any modern hypervisor on such boxes that have Intel Atom processor,
but maybe there are ways I don't know about.
 
jarda
Forum Guru
Forum Guru
Topic Author
Posts: 7602
Joined: Mon Oct 22, 2012 4:46 pm

Re: Cheap multi-NIC devices to run CHR in some supervisor

Tue Sep 06, 2016 7:18 pm

Well,
I have got just info that this one:
https://www.aliexpress.com/item/Mini-pc ... 9fa8a5b0fe

is able to run vsphere and host 64bit systems in it so the CHR should not be problem... But do believe chinese salesmen... :shock: They are even promising to preinstall vsphere.

Image
Image

Thinking about to buy and test. But I would appreciate any info from anyone who have whatever experience with it.
 
User avatar
smurphy
Member Candidate
Member Candidate
Posts: 103
Joined: Wed Feb 06, 2008 6:48 pm
Location: Clermont / France
Contact:

Re: Cheap multi-NIC devices to run CHR in some supervisor

Thu Sep 08, 2016 4:13 pm

I actually just got that one today and tried to install routerOS on it.
My plan is to replace the RB493G hareware with that one, as I am getting fiber, and with the current filters and configuration the RB493G is able to forward around 190Mbps.
This beast however should be able to max the 4 ethernet ports without sweating. What I thought.

Result - Unable to install routeros on it.
I even burned it to a CD and put a USB CD/DVD drive on it to boot from. In fact, once the installer kernel has booted, the system is unable to actually find the CD Drive and everything stops there.

So - having had a little time, I noticed that I only managed to install Ubuntu and/or Debian in UEFI mode. Old mode won't work.
I will try to see if I can update the BIOS/UEFI - but I did not even get a manual/nor CD as shown in the pictures at Amazon.

With Debian 8.5 jessie - testing the interface speed brings me 920 to 955Mbps using iperf. So it seems to be Ok.


Anyone has a hint on how to get routerOS on it? I wouldn't want to get back to Debian and shoreline firewall... I so got used to routerOS :)
 
jarda
Forum Guru
Forum Guru
Topic Author
Posts: 7602
Joined: Mon Oct 22, 2012 4:46 pm

Re: Cheap multi-NIC devices to run CHR in some supervisor

Thu Sep 08, 2016 7:13 pm

Can you try to run vmware type 1 hypervisor and CHR in it virtually?
 
User avatar
smurphy
Member Candidate
Member Candidate
Posts: 103
Joined: Wed Feb 06, 2008 6:48 pm
Location: Clermont / France
Contact:

Re: Cheap multi-NIC devices to run CHR in some supervisor

Thu Sep 08, 2016 9:07 pm

Can you try to run vmware type 1 hypervisor and CHR in it virtually?
Good question. As I don't know what you want to do exactly, you have a link somewhere explaining it ?
I have Debian 8.5 running on it at the moment.
 
jarda
Forum Guru
Forum Guru
Topic Author
Posts: 7602
Joined: Mon Oct 22, 2012 4:46 pm

Re: Cheap multi-NIC devices to run CHR in some supervisor

Thu Sep 08, 2016 10:24 pm

I mean to use this:
https://my.vmware.com/en/web/vmware/eva ... free-esxi6
on bare metal, without any common operating system under it. Then run CHR virtually in it. Or any other system side by side to CHR at once.

Which device exactly do you have?
 
User avatar
smurphy
Member Candidate
Member Candidate
Posts: 103
Joined: Wed Feb 06, 2008 6:48 pm
Location: Clermont / France
Contact:

Re: Cheap multi-NIC devices to run CHR in some supervisor

Fri Sep 09, 2016 10:36 am

I have exactly this device in it: http://hamsing.com/product/html/?39.html

FYI - I have virtualbox headless running on the device.
I'll have to see if I can run vsphere 6 on it. Well, if it installs.
First I got to fo through registration. Will take a while as Greylisting etc. is in play on my mail-server.
 
jarda
Forum Guru
Forum Guru
Topic Author
Posts: 7602
Joined: Mon Oct 22, 2012 4:46 pm

Re: Cheap multi-NIC devices to run CHR in some supervisor

Fri Sep 09, 2016 10:44 am

Looks like it is the same like I would like to buy.
I was repeatedly convinced by the seller, that it is able run vmware vsphere type1 on bare metal and 64bit operating systems in it. This is my favourite combination, as I could run multiple operating systems doing different things together with CHR at once.
Hope you succeed.
Keeping fingers crossed.
 
User avatar
smurphy
Member Candidate
Member Candidate
Posts: 103
Joined: Wed Feb 06, 2008 6:48 pm
Location: Clermont / France
Contact:

Re: Cheap multi-NIC devices to run CHR in some supervisor

Fri Sep 09, 2016 11:03 am

I think the problem will not be to run it. The problem will be to install it.
Below the CPU type (4 cores - I think only 2 and HT, didn't check). But the device remains really cool even though it is only passively cooled.
I have a 32GB m-stat drive in it, and 2GB Ram (Though I don't need more to run RouterOS on it). For my other systems, or what you intend to do with it, I have bought myself a HP Microserver gen8 (Got it for 180 euro incl. 2GB Ram and Celeron CPU) that I upgraded to Xeon CPU and 16GB Ram. So I need that device really only as high performance firewall. And I would love to stay with routerOS.
processor       : 3
vendor_id       : GenuineIntel
cpu family      : 6
model           : 55
model name      : Intel(R) Celeron(R) CPU  J1900  @ 1.99GHz
stepping        : 8
microcode       : 0x829
cpu MHz         : 1332.357
cache size      : 1024 KB
physical id     : 0
siblings        : 4
core id         : 3
cpu cores       : 4
apicid          : 6
initial apicid  : 6
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 movbe popcnt tsc_deadline_timer rdrand lahf_lm 3dnowprefetch ida arat epb dtherm tpr_shadow vnmi flexpriority ept vpid tsc_adjust smep erms
bogomips        : 3993.60
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
 
pe1chl
Forum Guru
Forum Guru
Posts: 6130
Joined: Mon Jun 08, 2015 12:09 pm

Re: Cheap multi-NIC devices to run CHR in some supervisor

Fri Sep 09, 2016 11:17 am

When you have that HP server with VMware, why don't you run RouterOS under VMware on that same server?
Then you don't need the small box at all.
 
jarda
Forum Guru
Forum Guru
Topic Author
Posts: 7602
Joined: Mon Oct 22, 2012 4:46 pm

Re: Cheap multi-NIC devices to run CHR in some supervisor

Fri Sep 09, 2016 11:46 am

Thats right. But.

I have HP microserver Gen7 too, mainly as storage server, dude server and running some virtuals for testing purposes mainly. 8GB RAM is enough for it.

This small PC would be capable to run ROS as CHR in virtual nicely and allow to test and freely switch any other versions if necessary. It would be running maybe much more with very small consumption. I expect the performance comparable to CCR1009, maybe better but for fraction of costs and with wider variety of possibilites.

The best thing is it can be powered by 12v backuped power supply with some quite small battery that keeps it running in power outage for long time, it can be put everywhere on the roof in a box, on the towers, just everywhere you need. Unlike big case with fully sized PC. Plus it has multiple intel NICs, unlike big not-so-cheap servers, so according to your advice to run CHR in vmware on the virutalization server it would involve necessity to have a manageable switch to trunk the wans and lans into vlans for the chr. What is more, the question is, whether the vlans will be passed correctly if the vmware is running in bridge moce as type 2 under some operating system that manages its NIC directly.

Maybe mikrotik can get inspired and start to produce similar products, cheaper and more powerful than ccr with much lower consumption. Sure not with such old cpu, like i J1900 is... :-)
 
pe1chl
Forum Guru
Forum Guru
Posts: 6130
Joined: Mon Jun 08, 2015 12:09 pm

Re: Cheap multi-NIC devices to run CHR in some supervisor

Fri Sep 09, 2016 12:09 pm

Well, I think the CCR1009 is already good value for money in the segment where I use it (at work).
It is cheaper and more powerful than Cisco routers in that segment.

For home and hobby I use RB series routers, and they are unbelievably cheap.
(I consider that MikroTik not only makes hardware, but also develops software for it and offers
updates to every customer that bought hardware, contrary to Cisco that requires a separate support
contract to be able to download software updates)

So I don't want to compare the price of a cheap Chinese bare box with a MikroTik router.
 
jarda
Forum Guru
Forum Guru
Topic Author
Posts: 7602
Joined: Mon Oct 22, 2012 4:46 pm

Re: Cheap multi-NIC devices to run CHR in some supervisor

Fri Sep 09, 2016 12:12 pm

Yep, it is also a way of life. Someone likes to play and to achieve some new things. If I was a network admin, then I would have different approach, the same like you, but my profession has nothing to do with networks. I am just playing... :-)
 
User avatar
smurphy
Member Candidate
Member Candidate
Posts: 103
Joined: Wed Feb 06, 2008 6:48 pm
Location: Clermont / France
Contact:

Re: Cheap multi-NIC devices to run CHR in some supervisor

Fri Sep 09, 2016 12:15 pm

When you have that HP server with VMware, why don't you run RouterOS under VMware on that same server?
Then you don't need the small box at all.
I don't like to mix things up. The Microserver is our NAS. No Internet access, internal cloud etc.
Web/Mail Server is again one other dedicated machine, in the Service network.
I then have one device which handles the firewall and separates networks etc.

That's how I think ;)
 
User avatar
smurphy
Member Candidate
Member Candidate
Posts: 103
Joined: Wed Feb 06, 2008 6:48 pm
Location: Clermont / France
Contact:

Re: Cheap multi-NIC devices to run CHR in some supervisor

Fri Sep 09, 2016 12:20 pm

So - I got virtualbox running on that little beast (Headless), and through a remote rdp connection, I got into routeros.
smurphy   2613  0.0  0.2  22996  4896 pts/1    Ss   11:08   0:00 -bash
smurphy   2681  0.0  0.5 116176 10864 ?        S    11:11   0:00 /usr/lib/virtualbox/VBoxXPCOMIPCD
smurphy   2686  0.1  0.9 538468 18108 ?        Sl   11:11   0:00 /usr/lib/virtualbox/VBoxSVC --auto-shutdown
smurphy   2706  3.6  4.1 1621944 82184 pts/0   Sl+  11:11   0:20 /usr/lib/virtualbox/VBoxHeadless --startvm Router OS
smurphy   2751  0.0  0.1  19100  2528 pts/1    R+   11:21   0:00 ps aux
chr_j1900.jpg
IMHO, virtualization works.

Still - is there are possibility, as I tend to do for many other things here - to side-boot a live Distribution and perform the installation manually?
In fact, the issues I have with native routerOS are as follow:
1. No UEFI boot. Means I have to configure the device to run legacy mode
2. It does not find the CD/DVD after boot. Hence it can't access the installation files.

Solution would be to start from a different linux, initiate the installer from there (Eventually mound CD/DVD drive manually if I have to), and once that is done, configure the boot-loader (I don't know which one is used).

Alternatively - as the routerOS code is very small. Havn't you folks thought about putting it into a EFI image?
Would make life of everyone way easier.
You do not have the required permissions to view the files attached to this post.
 
User avatar
smurphy
Member Candidate
Member Candidate
Posts: 103
Joined: Wed Feb 06, 2008 6:48 pm
Location: Clermont / France
Contact:

Re: Cheap multi-NIC devices to run CHR in some supervisor

Fri Sep 09, 2016 12:57 pm

Ok. So I managed to install RouterOS using the NetInstall method.
In fact - what I suspect happens, is that the USB Drivers (This motherboard has USB 3.0 chips inside) cut off the connected devices as soon as the kernel drivers are loaded.
Even when using the netinstall method, as soon as the kernel is loaded I loose access to all devices connected through USB. This probably also explains why we can't install through CD/USB image - because these are connected through USB.

Any idea when the drivers will be updated? I mean, the kernel routerOS us using should actually get updated. Still running 2.6.x if I remember what I did read everywhere.
chr_j1900_2.jpg
You do not have the required permissions to view the files attached to this post.
 
pe1chl
Forum Guru
Forum Guru
Posts: 6130
Joined: Mon Jun 08, 2015 12:09 pm

Re: Cheap multi-NIC devices to run CHR in some supervisor

Fri Sep 09, 2016 4:24 pm

When you have that HP server with VMware, why don't you run RouterOS under VMware on that same server?
Then you don't need the small box at all.
I don't like to mix things up. The Microserver is our NAS. No Internet access, internal cloud etc.
Web/Mail Server is again one other dedicated machine, in the Service network.
I then have one device which handles the firewall and separates networks etc.

That's how I think ;)
In my opinion the advantage of VMware is that you can draw and construct such a network without limitations
on things like network cards, cables, etc. You can define serveral virtual switches for all your
internal networks, connect all your services to it as separate virtual machines having access to one
or more of those networks, and run the router on it as well (having access to the external world and the networks).

For me, it is the ideal solution. For you, apparently not.
 
User avatar
smurphy
Member Candidate
Member Candidate
Posts: 103
Joined: Wed Feb 06, 2008 6:48 pm
Location: Clermont / France
Contact:

Re: Cheap multi-NIC devices to run CHR in some supervisor

Fri Sep 09, 2016 5:26 pm

In my opinion the advantage of VMware is that you can draw and construct such a network without limitations
on things like network cards, cables, etc. You can define serveral virtual switches for all your
internal networks, connect all your services to it as separate virtual machines having access to one
or more of those networks, and run the router on it as well (having access to the external world and the networks).

For me, it is the ideal solution. For you, apparently not.
I can do the same thing with the right hardware too ... :)
Oh - I use virtualization a lot for development, testing and prototyping, but I'll never run production on it.
The thing with Virtualization is that it costs dearly in resources, and for me as a purist, it is not a "clean" setup.
The virtualization hype came before the cloud hype, and with it unfortunately the IT techs dealing with these technologies are less and less qualified to actually troubleshoot or fix something. As soon as there is no point&mouse interface, they are lost. They lack the base understanding.

In the end however I get smarter and understand how things work (The fact that I am using linux since kernel 0.7p11 also helps.) ;)
 
pe1chl
Forum Guru
Forum Guru
Posts: 6130
Joined: Mon Jun 08, 2015 12:09 pm

Re: Cheap multi-NIC devices to run CHR in some supervisor

Fri Sep 09, 2016 5:38 pm

Ok that is maybe different for me. I knew about virtualization long before it entered the PC scene,
I very welcomed it when VMware was first released (and I have a VMware license dated Jul 6 1999
for the first version running under Linux), and today I run everything under ESXi when at all possible.
(especially production systems, because it makes it so easy to move them around, clone them to
setup a test environment, etc)
 
jarda
Forum Guru
Forum Guru
Topic Author
Posts: 7602
Joined: Mon Oct 22, 2012 4:46 pm

Re: Cheap multi-NIC devices to run CHR in some supervisor

Fri Sep 09, 2016 8:46 pm

smurphy, nice report. Thank you.

Do you think it could be possible you try to install the hypervisor directly on the hardware and test chr in it? It could be better option for you too, even you do not like virtualization for production, you might change your mind...

:-)
 
User avatar
smurphy
Member Candidate
Member Candidate
Posts: 103
Joined: Wed Feb 06, 2008 6:48 pm
Location: Clermont / France
Contact:

Re: Cheap multi-NIC devices to run CHR in some supervisor

Fri Sep 09, 2016 11:14 pm

smurphy, nice report. Thank you.

Do you think it could be possible you try to install the hypervisor directly on the hardware and test chr in it? It could be better option for you too, even you do not like virtualization for production, you might change your mind...
I tried. It won't install. As I said before - I suspect not having enough memory in my device (2GB). It loads all modules then tells it will relocate the kernel and everything stops there. Won't continue.
As the USB is disabled as soon as an old kernel is loaded, I can't change console to alt-F1 to check what is happening. I bet that is a BIOS issue.
ESXi.jpg
I tried to change some settings in the USB/BIOS - but I have to re-open the box to clear the CMOS ....

I welcome any hints.
You do not have the required permissions to view the files attached to this post.
Last edited by smurphy on Fri Sep 09, 2016 11:33 pm, edited 1 time in total.
 
jarda
Forum Guru
Forum Guru
Topic Author
Posts: 7602
Joined: Mon Oct 22, 2012 4:46 pm

Re: Cheap multi-NIC devices to run CHR in some supervisor

Fri Sep 09, 2016 11:30 pm

I am sorry, I have no experience with this. I have only experience with vmware workstation running under windows. But anyway, 2GB of ram is the official minimum, so it should not be a problem to run installer with this amout of ram. Starting to be afraid whether more memory would help.
 
User avatar
smurphy
Member Candidate
Member Candidate
Posts: 103
Joined: Wed Feb 06, 2008 6:48 pm
Location: Clermont / France
Contact:

Re: Cheap multi-NIC devices to run CHR in some supervisor

Fri Sep 09, 2016 11:36 pm

From the Paper I got from the contact - that box supports max 8GB Ram.
Even though I had Virtualbox running on it very nicely - and the cloud routerOS version fine, I am unable to install anything using old kernels.
 
jarda
Forum Guru
Forum Guru
Topic Author
Posts: 7602
Joined: Mon Oct 22, 2012 4:46 pm

Re: Cheap multi-NIC devices to run CHR in some supervisor

Sat Sep 10, 2016 12:19 am

That is what I wanted to put there: 8GB RAM module. But without prooven ability to run CHR in vsphere hypervisor I will not maybe even try it.
 
jarda
Forum Guru
Forum Guru
Topic Author
Posts: 7602
Joined: Mon Oct 22, 2012 4:46 pm

Re: Cheap multi-NIC devices to run CHR in some supervisor

Sat Sep 10, 2016 12:32 am

Oh, according to this paper:
http://pubs.vmware.com/vsphere-60/topic ... -guide.pdf
esxi requires 4GB as minimal RAM. I do not know from where I do remember the 2GB that I wrote before...
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 1749
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: Cheap multi-NIC devices to run CHR in some supervisor

Sat Sep 10, 2016 12:37 am

I don't think you will be able to run any modern hypervisor on such boxes that have Intel Atom processor,
but maybe there are ways I don't know about.
celeron j1900 is a new generation of atom processor architecture called Silvermont, supports intel VT-x hardware virtualization, another improvement is the evolution form in order execution to out of order improving performance a lot in contrast to anemic previous atom
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 1749
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: Cheap multi-NIC devices to run CHR in some supervisor

Sat Sep 10, 2016 12:45 am

When you have that HP server with VMware, why don't you run RouterOS under VMware on that same server?
Then you don't need the small box at all.
I don't like to mix things up. The Microserver is our NAS. No Internet access, internal cloud etc.
Web/Mail Server is again one other dedicated machine, in the Service network.
I then have one device which handles the firewall and separates networks etc.

That's how I think ;)
In my opinion the advantage of VMware is that you can draw and construct such a network without limitations
on things like network cards, cables, etc. You can define serveral virtual switches for all your
internal networks, connect all your services to it as separate virtual machines having access to one
or more of those networks, and run the router on it as well (having access to the external world and the networks).

For me, it is the ideal solution. For you, apparently not.
its ideal but not optimal,virtualization puts big latency overhead, because that telecom applications still not migrate completely to virtualization
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 1749
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: Cheap multi-NIC devices to run CHR in some supervisor

Sat Sep 10, 2016 12:50 am

i thik celeron j1900 and the new j3160 and n3160 are very interesting to use it with routeros because the low power consumption and relative good cpu performance vs mipsbe,powerpc e500v2, an tilera cpus

will be interesting to see what performance can be achieved with this silvermont 4 core CPU on routerOS
 
User avatar
smurphy
Member Candidate
Member Candidate
Posts: 103
Joined: Wed Feb 06, 2008 6:48 pm
Location: Clermont / France
Contact:

Re: Cheap multi-NIC devices to run CHR in some supervisor

Sat Sep 10, 2016 12:07 pm

i thik celeron j1900 and the new j3160 and n3160 are very interesting to use it with routeros because the low power consumption and relative good cpu performance vs mipsbe,powerpc e500v2, an tilera cpus

will be interesting to see what performance can be achieved with this silvermont 4 core CPU on routerOS
If you guys explain me how to test this, I'll let you know.

Does a regular Test-scheme exist, or a regular configuration for routerOS to use for the speed-tests? Reason I ask - is that on my RB493G's I have around 190 active firewall rules, NAT, Mangle, ~200 Blacklist entries (can go up to 2000 in periods of high script kiddy activities) and some other configurations active. With that they achieve, tested with iperf from one i7 system to one other i7 system 192Mbps max speed if no other activity happens on the network.

I could send you the comparative to my setup, but not to the regular setup's Mikrotik uses.
Last edited by smurphy on Sat Sep 10, 2016 12:15 pm, edited 1 time in total.
 
jarda
Forum Guru
Forum Guru
Topic Author
Posts: 7602
Joined: Mon Oct 22, 2012 4:46 pm

Re: Cheap multi-NIC devices to run CHR in some supervisor

Sat Sep 10, 2016 12:12 pm

well, just use the same config that you can compare with something that is generally known.
 
User avatar
smurphy
Member Candidate
Member Candidate
Posts: 103
Joined: Wed Feb 06, 2008 6:48 pm
Location: Clermont / France
Contact:

Re: Cheap multi-NIC devices to run CHR in some supervisor

Sat Sep 10, 2016 12:18 pm

well, just use the same config that you can compare with something that is generally known.
That's the point. I don't have a general config. I only have my specialized config... I can compare it, but I don't know if you will be able to make something out of it.

IMHO, Mikrotik should have some default test router configurations they could provide.
Anyone from Mikrotik reading who could confirm???
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 1749
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: Cheap multi-NIC devices to run CHR in some supervisor

Sat Sep 10, 2016 1:32 pm

i thik celeron j1900 and the new j3160 and n3160 are very interesting to use it with routeros because the low power consumption and relative good cpu performance vs mipsbe,powerpc e500v2, an tilera cpus

will be interesting to see what performance can be achieved with this silvermont 4 core CPU on routerOS
If you guys explain me how to test this, I'll let you know.

Does a regular Test-scheme exist, or a regular configuration for routerOS to use for the speed-tests? Reason I ask - is that on my RB493G's I have around 190 active firewall rules, NAT, Mangle, ~200 Blacklist entries (can go up to 2000 in periods of high script kiddy activities) and some other configurations active. With that they achieve, tested with iperf from one i7 system to one other i7 system 192Mbps max speed if no other activity happens on the network.

I could send you the comparative to my setup, but not to the regular setup's Mikrotik uses.
thats an issue plaguing the industry, on one side are the marketing folks who love to publish pretty numbers about equipment performance, far from real scenario performance, on the other side we have us, the people who has to deal with surprises about performance in real scenarios.

two group of people with different objectives, very different, thats the reason of discrepancy on testing results, vendors do know the real performance of equipment but do not publish it, or at least not in a way to make easy to people to find it, for example i have some document of cisco showing performacne of routers under some scenarios close to real:

for example for isrg2 3945 router:

max performance rfc 2544 = best performance scenario: 8 gbps and 1mppps
ipsec best case scenario = 850mbps
firewall max performance = 3 gpbs

that was the pretty numbers, now go with the interesting ones:

hqos performance with mixed size packet at 75% cpu = 600mbps
pat performance with mixed size packet at 75% cpu = 500mbps
recommended ipsec positioning of product = 180mbps
nat + qos + acl performance with mixed size packet at 75% cpu = 230mbps
recommended WAN circuit for platform = 150mbps around 100kpps

as we can see a 10x performance difference between max firewall performance and the real world scenario performance (3gbps 300mbps (150mbps symetric wan circuit)

is something similar to what mikrtik intend to do with numbers published on routerboard.com product page, for example with CCR1036-8G-2S+EM:

max performance rfc 2544: 27gbps/2.2mpps with 1500byte packets, 27gbps/6.5mpps with 500byte packets and 21gbps/41mpps with 64byte packets
25 ip filter rules: 22gbps/1.8mpps with 1500byte packets, 12gbps/3mpps with 500byte packets and 1.5gbps/3mpps with 64byte packets

64byte packet reveals the same downgrade of 10x+ between this two tests, but this tests are far from real scenarios who pullulate here at the forum where we face the surprise when the performance obtained is far from expected, and the worst thing is that's very difficult to predict, mostly because is hard to find the situation when some guy with the skills, the time, and the equipment to test properly a device capable of multi gigabit speeds, and kind to share the result of this hard work to the community.

Is not easy and vendors take advantage of this to confuse the guy buying the equipment.

will be nice to lay the foundations of more serious real world approach to performance testing on mikrotik/routeros devices to the benefit of all.

I think the first step is to establish some base configurations and testing procedures taken from real world to start gathering results and build an information base useful to all
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 1749
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: Cheap multi-NIC devices to run CHR in some supervisor

Sat Sep 10, 2016 1:40 pm

well, just use the same config that you can compare with something that is generally known.
That's the point. I don't have a general config. I only have my specialized config... I can compare it, but I don't know if you will be able to make something out of it.

IMHO, Mikrotik should have some default test router configurations they could provide.
Anyone from Mikrotik reading who could confirm???
mikrotik published this page with info

http://wiki.mikrotik.com/wiki/Manual:Pe ... _Generator
 
jarda
Forum Guru
Forum Guru
Topic Author
Posts: 7602
Joined: Mon Oct 22, 2012 4:46 pm

Re: Cheap multi-NIC devices to run CHR in some supervisor

Sat Sep 10, 2016 2:16 pm

well, just use the same config that you can compare with something that is generally known.
That's the point. I don't have a general config. I only have my specialized config... I can compare it, but I don't know if you will be able to make something out of it.

IMHO, Mikrotik should have some default test router configurations they could provide.
Anyone from Mikrotik reading who could confirm???
Just use the same config you use now and you told that rb493g does 192Mbits. I wonder what will do the J1900 with the same config (maybe will be capped by interfaces, so then what the cpu utilisation will be under the test...

It is nothing to compare "standard test scenario" because you will not use the device with such config, neither I, so the "real config" comparison is much more usable both for you and me. And probably for others.
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 1749
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: Cheap multi-NIC devices to run CHR in some supervisor

Sun Sep 11, 2016 4:24 am

can be useful to known your config to test it on another equipment
 
User avatar
smurphy
Member Candidate
Member Candidate
Posts: 103
Joined: Wed Feb 06, 2008 6:48 pm
Location: Clermont / France
Contact:

Re: Cheap multi-NIC devices to run CHR in some supervisor

Sun Sep 11, 2016 11:39 am

can be useful to known your config to test it on another equipment
Thx, and to all of you others for the advice (and also the link published by Mikrotik).
To use Mikrotiks method,I'd need a second device (same one) or a faster device, to measure the capabilities of the J1900 one - reason I will skip it. It is a very interesting read though.

So I intend to make the test by connecting 2 i7 + 16GB Ram + SSD systems to that device running KUbuntu 16.04.1 64bit, one connected at the World Interface, one at the LAN Interface, and will replicate the configuration I have on my RB493G device. I'll then use iperf to make a transfer-test with different payload sizes (1518 byte, 512 byte and 64 byte).
I'll report when I did the tests ...
 
jarda
Forum Guru
Forum Guru
Topic Author
Posts: 7602
Joined: Mon Oct 22, 2012 4:46 pm

Re: Cheap multi-NIC devices to run CHR in some supervisor

Sun Sep 11, 2016 11:43 am

Thanks in advance, will be waiting for it.
 
User avatar
smurphy
Member Candidate
Member Candidate
Posts: 103
Joined: Wed Feb 06, 2008 6:48 pm
Location: Clermont / France
Contact:

Re: Cheap multi-NIC devices to run CHR in some supervisor

Sun Sep 11, 2016 1:39 pm

Well - the results are quite disappointing at first sight with iperf.
Note that on the J1900 device, the filters seem to have very low influence, if it at all.

Test setup done on:
- NJ1900FW, J1900 CPU @ 2GHz, 2GB Ram, 32GB SSD/M-SATA, 4xIntel WG82583, ROS License level 1 (test license only - still don't know if I'll run RouterOS on it)
- Server: i7-4600U @ 2.10GHz, 16GB Ram, SSD Disk, Intel I218-LM Gigabit Ethernet, KUbuntu 16.04.01 - nothing else running
- Client: i7-3720QM @ 2.6GHz, 16GB Ram, SSD Disk, NetXtreme BCM57766 Gigabit Ethernet, KUbuntu 16.04.1 - nothing else running

Cabling - Cat6 connected directly to the device. No intermediates.

For the tests, I ran "iperf -s" on the server, on the client iperf -c [Server-IP] -l <PacketSize>

Below the results.
Device	64Bytes		CPU Load	512Bytes		CPU Load	1518Bytes	CPU Load	Comment
RB493G	35.3MBits/s	4.00%		283Mbits/s	3.00%		555MBits/s	3.00%		Lan 2 Lan / FastPath active
RB493G	35.4MBits/s	34.00%		216Mbits/s	100.00%		233Mbits/s	100.00%		Lan 2 Service / FastPath active

Device			64Bytes		CPU Load	512Bytes		CPU Load	1518Bytes	CPU Load	Comment							
HS-NJ1900FW	35.4MBits/s	0.00%		284Mbits/s	3.00%		302Mbits/s	10.00%		Lan 2 Service
HS-NJ1900FW	35.4MBits/s	0.00%		284Mbits/s	0.00%		302Mbits/s	10.00%		Lan 2 Service / No filters
Lan 2 Lan does not make sense on the j1900, as all interfaces are ethernet to ethernet and pass through the CPU/OS.

Note that I also transferred some large files (scp) to see if this reflects right:
20150304_MIN-CentOS6u5-cd_b133_GA-x86_64.iso			100%  699MB 116.6MB/s   00:06 
The CPU speed on clilent/server is enough to handle the encryption without really influencing the speed.
These are some Kickstart Images I do for work. Transfer speed is here 116MBytes/s which is equivalent to 932.8Mbits/s - I would say this is not far theoretical max. for a Gigabit interface and contradicts the iperf speed results. CPU Load on the device was at 13%.

Same scp copy operation on my RB493G has the following effect - speed of ~180Mbits/s
20150304_MIN-CentOS6u5-cd_b133_GA-x86_64.iso				100%  699MB  22.6MB/s   00:31
On the other hand, as I have quite lots to do with Monitoring networking, my network interfaces in house (Work Laptop and Private laptop) are all configured with max buffers to use etc., and usually are quite good at what they are doing.

Any hints on what to fine-tune ?

I may check the test-setup as explained in http://wiki.mikrotik.com/wiki/Manual:Pe ... _Generator - question is, will the RB493G be the limiting factor here?
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 1749
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: Cheap multi-NIC devices to run CHR in some supervisor

Sun Sep 11, 2016 7:58 pm

maybe trying with multiple connections for the test for example 10 simultaneous connections
 
User avatar
smurphy
Member Candidate
Member Candidate
Posts: 103
Joined: Wed Feb 06, 2008 6:48 pm
Location: Clermont / France
Contact:

Re: Cheap multi-NIC devices to run CHR in some supervisor

Mon Sep 12, 2016 11:09 am

I'll have to check what other methods can be used to try the transfer speeds. At this time, I can't do more tests as I need my network for work again :}
 
User avatar
janisk
MikroTik Support
MikroTik Support
Posts: 6283
Joined: Tue Feb 14, 2006 9:46 am
Location: Riga, Latvia

Re: Cheap multi-NIC devices to run CHR in some supervisor

Mon Sep 12, 2016 12:14 pm

little note on how to use rfc2544 published results from routerboard.com:

when you want to know how the router will perform, you can compare these numbers between the routers. As RFC is quite strict on what numbers can be reported relative performance should be as stated.

next, when you make test setup - you have reference value that you should get with your own test system. When you have that - add your configuration on the router and with the same method test for the performance. It should scale with the original test result.

Such loging for "general router test" is completely wrong, as there is no defined term to what that "general performance" is and everyone can just paint whatever numbers they wish.
 
User avatar
smurphy
Member Candidate
Member Candidate
Posts: 103
Joined: Wed Feb 06, 2008 6:48 pm
Location: Clermont / France
Contact:

Re: Cheap multi-NIC devices to run CHR in some supervisor

Mon Sep 12, 2016 12:31 pm

little note on how to use rfc2544 published results from routerboard.com:

when you want to know how the router will perform, you can compare these numbers between the routers. As RFC is quite strict on what numbers can be reported relative performance should be as stated.

next, when you make test setup - you have reference value that you should get with your own test system. When you have that - add your configuration on the router and with the same method test for the performance. It should scale with the original test result.

Such loging for "general router test" is completely wrong, as there is no defined term to what that "general performance" is and everyone can just paint whatever numbers they wish.
I admit that you are right, but that is also the reason I have added data on the RB493G, because I don't have the same test environment you have, nor do I know how you perform the measurements, and because you won't test this setup...
And adding my router configuration (which provides quite some internals of my production system) is a no-go to be made public. Sorry on that.

The information I have provided are a comparison between a RB493G, and the same configuration (interface bound to bridge names -> then firewall/Nat setup/Queues etc. ported over), and tests done in the same interface direction - to have a common ground for comparison.
And, what is most important here is not the raw "numbers", but how the device performs in a real live environment. This is also why I took the data from the RB493G to have a comparison.

Who is online

Users browsing this forum: No registered users and 3 guests