Community discussions

 
mpreissner
Member
Member
Topic Author
Posts: 356
Joined: Tue Mar 11, 2014 11:16 pm
Location: Columbia, MD

Beast x86 hardware

Fri Feb 19, 2016 2:32 pm

Recently announced by SuperMicro, the 5018D-FN8T looks to be a beast of a 1U machine to run ROS on. New Xeon-D 1518 CPU (4 core, 8 thread) at 2.2 GHZ, only 35W, and loaded with 30 Gbps worth of interfaces! I think this could give the CCR line a run for its money. If it performs as expected, it ought to give the CCR1036-8G-2S+ some competition for about the same price, with similarly low power draw. Would probably make a good platform for hosting multiple virtual routers too. Thoughts?
Michael Preissner
CISSP, CCSP, CEH, PMP
 
SystemErrorMessage
Member
Member
Posts: 378
Joined: Sat Dec 22, 2012 9:04 pm

Re: Beast x86 hardware

Sat Feb 20, 2016 11:19 am

if it is priced about $1000 than yes it would.
35W power draw is much lower than the CCR1036 60W power draw and the CCR consumes more power and loses stability as the temperatures increase. Above 80C the CCR becomes unstable and i find a lot of flaws in the cooling, something i only noticed in the summer. The usb port of the CCR also doesnt supply power properly (i powered a raspberry pi from it and it froze but powering from another normal usb port (of another device or even the edgerouter pro) and it hasnt frozen yet and has been months).

I even modded the cooling on the CCR to make it quieter than ubiquiti's edgerouter pro and it involved some tape and some fans.

You should measure the supermicro 1U from the wall and see how much power it really uses. Its not only the CPU that consumes power but a lot of other things.

I think the CCR can be improved without requiring a major redesign, all it needs is a PSU that is 10W-20W higher, a change of fans, heatsink and tunnel cover and it can be quiet and cool at the same time.

I wonder how quiet supermicro's server is. At least you get PCIe which mikrotik doesnt implement in their CCR line, SATA and other things.

I wouldnt say supermicro's server is a beast because it will only handle up to 20Gb/s with that CPU but it will be much faster than the CCR in VPN and other things. I remember a test that was done before during the 1st gen iseries in which dual quad xeons were used that achieved up to 20Gb/s of routing whereas using GPU routing(using dual GTX 480) theoratically achieved 100Gb/s or more. The only problem with using GPUs is that they are quite power hungry.
 
mpreissner
Member
Member
Topic Author
Posts: 356
Joined: Tue Mar 11, 2014 11:16 pm
Location: Columbia, MD

Re: Beast x86 hardware

Sat Feb 20, 2016 3:35 pm

Yeah, I can't wait for these to become available. I have to imagine that it will crank out 30 Gbps...the Xeon-D 1518 CPU is optimized for network performance (denoted by the xxx8 CPU code), and they gave it 2x SFP+ and 10x 1Gb RJ-45's...I have to imagine they wouldn't have configured it that way if the CPU couldn't push it. It only has a 200W PSU in it, and with the CPU maxing at 35W, I can't imagine the board eating up too much more.
Michael Preissner
CISSP, CCSP, CEH, PMP
 
djdrastic
Member
Member
Posts: 305
Joined: Wed Aug 01, 2012 2:14 pm

Re: Beast x86 hardware

Sat Feb 20, 2016 6:14 pm

Yeah I'm mega interested in this too for exactly the same reason as you.
I would have liked to have a couple more SFP+ ports on there , but you can't always have everything in life :lol:

Been bitten hard by the CCR's before so another option would be welcomed.
 
SystemErrorMessage
Member
Member
Posts: 378
Joined: Sat Dec 22, 2012 9:04 pm

Re: Beast x86 hardware

Sat Feb 20, 2016 9:03 pm

If there are free PCIe 2.0 x4 slots you can use SFP+ cards on those. Each PCIe 2.0 x4 slot supports a single SFP+ slot.
 
brwainer
newbie
Posts: 47
Joined: Tue Feb 02, 2016 2:55 am

Re: Beast x86 hardware

Sun Feb 21, 2016 2:49 am

If there are free PCIe 2.0 x4 slots you can use SFP+ cards on those. Each PCIe 2.0 x4 slot supports a single SFP+ slot.
"1x PCIe 3.0 x8 slot" - it's an mITX motherboard. But I'm sure there is 2x or 4x SFP+ PCIe cards out there. I have no doubt it has the power to route it's 2x SFP+ ports and 6x 1GbE ports, but adding more SFP+ ports sounds like asking for disappointment.
 
mpreissner
Member
Member
Topic Author
Posts: 356
Joined: Tue Mar 11, 2014 11:16 pm
Location: Columbia, MD

Re: Beast x86 hardware

Mon Feb 22, 2016 2:20 pm

FYI, I have seen just the motherboard listed online for about $515 US. Throw in some RAM, an M.2 SSD, 1U chassis of your choice...
Michael Preissner
CISSP, CCSP, CEH, PMP
 
hedele
Member
Member
Posts: 338
Joined: Tue Feb 24, 2009 11:23 pm

Re: Beast x86 hardware

Mon Feb 22, 2016 4:29 pm

I don't hold high hopes that this is going to work without a hitch on ROS 6 with the normal x86 distribution :( Mikrotik seems hellbent on keeping the x86 distribution of ROS as barely usable as possible (no 64 bit support, missing support for lots of newer NICs) to push people towards using CHR, but CHR needs a hypervisor to work, which itself generates a non-trivial amount of overhead in regards to networking :(
 
mpreissner
Member
Member
Topic Author
Posts: 356
Joined: Tue Mar 11, 2014 11:16 pm
Location: Columbia, MD

Re: Beast x86 hardware

Mon Feb 22, 2016 4:59 pm

Has anyone tried taking an image of a CHR's storage and deploying it on bare metal? The CHR is essentially an x86_64 OS, so if one could image the virtual hard disk to bare metal, theoretically, we could have ourselves a real 64-bit ROS for x86_64 hardware...
Michael Preissner
CISSP, CCSP, CEH, PMP
 
hedele
Member
Member
Posts: 338
Joined: Tue Feb 24, 2009 11:23 pm

Re: Beast x86 hardware

Tue Feb 23, 2016 4:06 pm

I already mentioned that idea, but it was suggested by Mikrotik staff that essential kernel modules/drivers that are required to successfully run CHR on physical hardware are disabled/removed in the x86_64 kernel so this can't happen. I did not bother to verify that yet.
 
meshnet
Frequent Visitor
Frequent Visitor
Posts: 55
Joined: Tue Jun 01, 2004 6:57 pm

Re: Beast x86 hardware

Tue Feb 23, 2016 8:37 pm

I have one of the supermicro Atom version of this unit..
http://www.supermicro.com/products/syst ... A-FTN4.cfm

Mikrotik doesn't see the i354 ( the onboard nics )
I have a dual 10gb card in it, and that works, but the onboard nics don't.
Had to load it with netinstall, with a intel NIC, then swap to the 10gb card.

Richard
 
mpreissner
Member
Member
Topic Author
Posts: 356
Joined: Tue Mar 11, 2014 11:16 pm
Location: Columbia, MD

Re: Beast x86 hardware

Wed Feb 24, 2016 1:48 am

I have one of the supermicro Atom version of this unit..
http://www.supermicro.com/products/syst ... A-FTN4.cfm

Mikrotik doesn't see the i354 ( the onboard nics )
I have a dual 10gb card in it, and that works, but the onboard nics don't.
Had to load it with netinstall, with a intel NIC, then swap to the 10gb card.

Richard
Good to know. I was looking at those too. Have you thought about loading a hypervisor on it like ESXi or KVM and running ROS as a virtual appliance? The hardware is optimized for virtualization, and that layer of abstraction between the physical NIC and supported virtual NICs may allow you to make use of the onboard 1 Gbps ports. Plus, since you'd be running the CHR as a virtual machine, you'd actually get a true 64-bit ROS as opposed to the 32-bit x86 ROS. I'd be interested to see what kind of performance penalty you incur by virtualizing ROS versus running on bare metal.
Michael Preissner
CISSP, CCSP, CEH, PMP
 
meshnet
Frequent Visitor
Frequent Visitor
Posts: 55
Joined: Tue Jun 01, 2004 6:57 pm

Re: Beast x86 hardware

Wed Feb 24, 2016 4:41 pm

No, I have not. We were testing multi-gigabit interfaces, and I was not ready to spend for the CHR license that supports over 1gb, and I might not use.

I ended up running different software on it, and it can really push the traffic.

Richard
 
mpreissner
Member
Member
Topic Author
Posts: 356
Joined: Tue Mar 11, 2014 11:16 pm
Location: Columbia, MD

Re: Beast x86 hardware

Wed Feb 24, 2016 4:55 pm

I was not ready to spend for the CHR license that supports over 1gb
You can get trial licenses for all license levels for free (good for 60 days). I'd just request a trial license at the "Unlimited" license level and see how much you can push using CHR. If you can get an idea as to routing performance dependent upon different virtual hardware configs, you might find you can run two virtual CHR instances that each meet your needs. As long as the physical hardware doesn't fail, you could do VRRP between the virtual instances for redundancy in case one of the virtual machines has a problem.
Michael Preissner
CISSP, CCSP, CEH, PMP
 
meshnet
Frequent Visitor
Frequent Visitor
Posts: 55
Joined: Tue Jun 01, 2004 6:57 pm

Re: Beast x86 hardware

Wed Feb 24, 2016 5:25 pm

I had no more time to test, and the unit is already put it into production.

Richard
 
SystemErrorMessage
Member
Member
Posts: 378
Joined: Sat Dec 22, 2012 9:04 pm

Re: Beast x86 hardware

Wed Feb 24, 2016 5:29 pm

You could build your own x86 with as many xeons and as much ram and PCIe as you like and get CCR1036 performance for much less.

A single PCIe 2.0 link supports 2 1Gb/s link (2.5Gb/s per direction)
A 4 lane PCIe 2.0 link supports a single SFP+ (10Gb/s per direction)

The only issue with installing routerOS on x86 is the drivers, a lot of things may not work so you will be limited to mikrotik NICs. I hope theres a list of NICs supported by routerOS.

Supermicro solutions tend to be pricey but also tend to be reliable.
 
panteraboy
just joined
Posts: 18
Joined: Wed Jan 27, 2010 11:25 pm
Location: Romania

Re: Beast x86 hardware

Fri Mar 04, 2016 12:35 pm

I want to buy this NIC: HP NC364T https://computernetworkingproduct007.wo ... ver-adptr/ , Anyone using this network card?
RB750, RB433AH, RB2011RM
 
ofca
Member Candidate
Member Candidate
Posts: 190
Joined: Fri Aug 20, 2004 7:18 pm

Re: Beast x86 hardware

Thu Mar 17, 2016 7:52 pm

Just tried to install CHR on baremetal today, and it boots fine, until a moment when some check fails and it decides that my x86_64 machine is not a x86_64 machine, then shutdowns.
 
scubyx
just joined
Posts: 6
Joined: Wed May 11, 2016 1:20 pm

Re: Beast x86 hardware

Fri Aug 12, 2016 1:23 pm

Hii,

just t make sure guys, i want to buy this SuperServer 5018D-FN8T and installed mikrotik on it, is this device support mikrotik ROS?
are the built in 10G NICs working fine?
 
kristaps
Member Candidate
Member Candidate
Posts: 272
Joined: Mon Jan 27, 2014 1:37 pm

Re: Beast x86 hardware

Fri Aug 12, 2016 4:21 pm

I suggest to instill some VM host and use CHR on newer hardware, when RouterOS v7 comes out there will be batter support for newer hardware.
 
User avatar
BartoszP
Forum Guru
Forum Guru
Posts: 1717
Joined: Mon Jun 16, 2014 1:13 pm
Location: Poland

Re: Beast x86 hardware

Fri Aug 12, 2016 4:31 pm

when RouterOS v7 comes out there will be batter support for newer hardware.
Isn't is easier to spot UFO on the sky ? :D :D :D
Real admins use real keyboards.
 
mpreissner
Member
Member
Topic Author
Posts: 356
Joined: Tue Mar 11, 2014 11:16 pm
Location: Columbia, MD

Re: Beast x86 hardware

Fri Aug 12, 2016 5:20 pm

Hii,

just t make sure guys, i want to buy this SuperServer 5018D-FN8T and installed mikrotik on it, is this device support mikrotik ROS?
are the built in 10G NICs working fine?
You can try to install the x86 ROS on it, but it may not work with all the NICs. Best bet is to run a hypervisor like VMware, KVM, etc., and then run the CHR as a virtual machine. Any modern hypervisor should have pretty good support for the onboard NICs, including the SFP+'s, and the CHR should easily be able to use them once they're abstracted as supported virtual NICs. The CHR will also give you access to more than 2GB RAM (which is a limitation of the x86 ROS), and any latency you would get from virtualizing ROS is negligible (we're talking microseconds here).
Michael Preissner
CISSP, CCSP, CEH, PMP
 
User avatar
nz_monkey
Forum Guru
Forum Guru
Posts: 1821
Joined: Mon Jan 14, 2008 1:53 pm
Location: Straya
Contact:

Re: Beast x86 hardware

Sun Aug 14, 2016 12:20 pm

I suggest to instill some VM host and use CHR on newer hardware, when RouterOS v7 comes out there will be batter support for newer hardware.
Key word "when".

When will RouterOS v7 come out ? :)
http://thebrotherswisp.com/ | Mikrotik MTCNA, MTCRE, MTCINE | Fortinet FTCNA, FCNSP, FCT | Extreme Networks ENA
 
jarda
Forum Guru
Forum Guru
Posts: 7604
Joined: Mon Oct 22, 2012 4:46 pm

Re: Beast x86 hardware

Sun Aug 14, 2016 12:24 pm

And usual answer like always: when it will be then it will be.
 
scubyx
just joined
Posts: 6
Joined: Wed May 11, 2016 1:20 pm

Re: Beast x86 hardware

Tue Oct 11, 2016 5:58 am

Hii,

just t make sure guys, i want to buy this SuperServer 5018D-FN8T and installed mikrotik on it, is this device support mikrotik ROS?
are the built in 10G NICs working fine?
You can try to install the x86 ROS on it, but it may not work with all the NICs. Best bet is to run a hypervisor like VMware, KVM, etc., and then run the CHR as a virtual machine. Any modern hypervisor should have pretty good support for the onboard NICs, including the SFP+'s, and the CHR should easily be able to use them once they're abstracted as supported virtual NICs. The CHR will also give you access to more than 2GB RAM (which is a limitation of the x86 ROS), and any latency you would get from virtualizing ROS is negligible (we're talking microseconds here).
I see, so CHR can access more than 2 GB RAM, that is awesome.. :D

thank you
 
User avatar
docmarius
Forum Guru
Forum Guru
Posts: 1219
Joined: Sat Nov 06, 2010 12:04 pm
Location: Timisoara, Romania
Contact:

Re: Beast x86 hardware

Tue Oct 11, 2016 12:23 pm

Don't be that optimistic about that 35W power consumption. A RAM module can draw up to 15W... And there are other parts inside it, too. They did not put that 200W power supply in there for nothing.
Torturing CCR1009-7G-1C-1S+, RB450G, RB750GL, RB951G-2HnD, RB960PGS, RB260GSP, OmniTIK 5HnD and NetMetal 922UAGS-5HPacD + R11e-5HnD in my home network.
 
mpreissner
Member
Member
Topic Author
Posts: 356
Joined: Tue Mar 11, 2014 11:16 pm
Location: Columbia, MD

Re: Beast x86 hardware

Tue Oct 11, 2016 5:23 pm

Don't be that optimistic about that 35W power consumption. A RAM module can draw up to 15W... And there are other parts inside it, too. They did not put that 200W power supply in there for nothing.
You're right, but the CPU is typically the most power-hungry component in any computer, short of a high-end GPU that needs it's own PSU. The performance that you get out of this system considering the small PSU is pretty awesome, and definitely something to keep in mind if you're running your own home lab, where being power-efficient is a significant concern.
Michael Preissner
CISSP, CCSP, CEH, PMP
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 1743
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: Beast x86 hardware

Tue Oct 11, 2016 7:18 pm

Don't be that optimistic about that 35W power consumption. A RAM module can draw up to 15W... And there are other parts inside it, too. They did not put that 200W power supply in there for nothing.

agree

keep in mind 35w is the tdp for the cpu only it runs at 2.2ghz and is a x86 quad core.

i think we can expect 30watt idle and 70 watt at full load

in terms of performance can be an interesting device because:
* supports virtualization hw acceleration/optimization features of xeon broadwell
* provides 2 x 10g ethernet
* provides 16x pci express lanes directly connected to cpu to expand with more 10g cards
* dual channel of ddr4 ram to provide bandwidth to networking tasks
* 6mbyte of Level 3 cache to improve performance
 
mpreissner
Member
Member
Topic Author
Posts: 356
Joined: Tue Mar 11, 2014 11:16 pm
Location: Columbia, MD

Re: Beast x86 hardware

Sat Oct 15, 2016 1:30 am

Not only that, but the CPU is also optimized specifically for networking functions. The latest numbering scheme with the Xeon CPU's uses the last digit in the product code to identify different CPU optimizations. For instance, the Xeon D-1520 is a general purpose CPU, good for running any OS, virtualization, etc., however, the new xxx7 and xxx8 coded CPUs (such as the 1518 in the product we're currently discussing) are optimized for communications and networking. Some new xxx1 CPUs will be coming out soon that are geared toward storage...so lots of exciting things coming along from Intel!
Michael Preissner
CISSP, CCSP, CEH, PMP
 
snd1988
just joined
Posts: 5
Joined: Sun Feb 05, 2017 4:38 pm

Re: Beast x86 hardware

Sun Apr 09, 2017 4:56 pm

Does anybody tested 5018D-FN8T this??

Who is online

Users browsing this forum: No registered users and 24 guests