Community discussions

MikroTik App
 
mtmx80
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 87
Joined: Wed Mar 16, 2011 11:12 pm

Request for real Cloud Core Router HW (10Gbps)

Thu Apr 10, 2014 6:36 pm

I'm using your products for several years. They are not bug less, but in general they are quite stable once they are configured properly. Your new CCR and CRS are very nice products as well. I would like to start to deploy them in enterprise/private cloud environments, but they are still not capable for doing that.

We are looking to connect primarily HP (optionally IBM and SuperMicro) blade enclosures with 10G uplinks. 2 SFP+ connections are simply not enough for that. They would need to act more like L3 switches than a serious router or firewall, but switching on 80Gbps+ will require high CPU performance or ASIC made for this. Unfortunately system mentioned below is not tested, so I can't state how good or bad it performs with ROS.

Here is what would be nice to have: Can you create something with much more 10G ports? Even oversubscribed system. Here is an example:
With a cheaper HP server it's possible to create a multi port router with 8SFP+ ports and 4x 1Gbps + console / remote management and optionally redundant PSU.

HP ProLiant DL380e Gen8 1 2,479.95 €

669255-B21#B19 HP ProLiant DL380e Gen8 8 LFF Configure-to-order Europe-Multilingual Localization Server [1]
726113-B21#0D1 HP 8GB SD Enterprise Mainstream Flash Media Kit [1]
663476-B21 HP 2U FIO Friction Rail Kit [1]
661404-B21#0D1 HP DL380e Gen8 CPU1 Riser Kit [1]
708497-L21 HP DL380e Gen8 Intel Xeon E5-2407v2 (2.4GHz/4-core/10MB/80W) FIO Processor Kit [1]
713977-B21#0D1 HP 4GB (1x4GB) Dual Rank x8 PC3L-12800E (DDR3-1600) Unbuffered CAS-11 Low Voltage Memory Kit [1]
614203-B21#0D1 HP NC552SFP 10Gb 2-port Ethernet Server Adapter [4]
503296-B21#0D1 HP 460W HE 12V Hot Plug AC Power Supply Kit [1]


A basic Cisco 4900M costs 3 times more. Ok it's different category and it can be expanded even more, but from other side RouterOS can do more things for less.
NetGear has 10G switches which costs about $100 per port so I would expect similar price range.

Any configuration would be welcome for example: 8x SFP+ and/or 8x 10Gbase-T, 8x 1Gbps, redundant PSU and cooling + option to stack it with another MT Switch product... Am I asking too much? :)
 
eflanery
Member
Member
Posts: 382
Joined: Fri May 28, 2004 10:11 pm
Location: Moscow, ID
Contact:

Re: Request for real Cloud Core Router HW (10Gbps)

Thu Apr 10, 2014 7:05 pm

They have announced the CCR-1072-8S+

72 cores, 8 10G ports, 1 1G port, redundant PSU.

Should be nice, and close to what you are looking for.

Hopefully they will be out sometime this year.
 
mtmx80
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 87
Joined: Wed Mar 16, 2011 11:12 pm

Re: Request for real Cloud Core Router HW (10Gbps)

Thu Apr 10, 2014 10:29 pm

That's it. I hope it will be under $2000, work stable and it will be released very soon :D
They have announced the CCR-1072-8S+

72 cores, 8 10G ports, 1 1G port, redundant PSU.

Should be nice, and close to what you are looking for.

Hopefully they will be out sometime this year.
 
Zorro
Long time Member
Long time Member
Posts: 676
Joined: Wed Apr 16, 2014 2:43 pm

Re: Request for real Cloud Core Router HW (10Gbps)

Wed Apr 16, 2014 3:38 pm

presently each Tile core can be saturated by 1Gb, right ?
so multiply it to 10 and number of desired ports and you will get expected horsepower and price of required Tilera chip(they had Any kind of them, from 4-core to hungreds :P).
MikroTik developers announced that they soon - will able to spread even fragmented TCP traffic across cores(maybe adopted Haskel or Erlang ? :)in future versions, so there would be very warm expectations among customers about CCR performance, achievable, thus.
 
AlexS
Member Candidate
Member Candidate
Posts: 266
Joined: Thu Oct 10, 2013 7:21 am

Re: Request for real Cloud Core Router HW (10Gbps)

Thu Apr 24, 2014 6:24 am

presently each Tile core can be saturated by 1Gb, right ?
so multiply it to 10 and number of desired ports and you will get expected horsepower and price of required Tilera chip(they had Any kind of them, from 4-core to hungreds :P).
MikroTik developers announced that they soon - will able to spread even fragmented TCP traffic across cores(maybe adopted Haskel or Erlang ? :)in future versions, so there would be very warm expectations among customers about CCR performance, achievable, thus.
where did you get the 1G per core limit from.

Strangely i have been running into 1Gb/s limit on TCP streams in my testing of CCR1036-8G-2S+EM

This would explain a lot ... I have logged a support request.. been silent for a while, long while..
 
roadracer96
Forum Veteran
Forum Veteran
Posts: 714
Joined: Tue Aug 25, 2009 12:01 am

Re: Request for real Cloud Core Router HW (10Gbps)

Mon May 05, 2014 6:24 am

You gotta be shitting me. I've been waiting on a reply about something similar. 1gb per core , single stream? Throw gre an ipsec in the mix and you are. Down to. 50 mbit. Rb1100 works better.
 
User avatar
nz_monkey
Forum Guru
Forum Guru
Posts: 1870
Joined: Mon Jan 14, 2008 1:53 pm
Location: Straya
Contact:

Re: Request for real Cloud Core Router HW (10Gbps)

Mon May 05, 2014 12:10 pm

We are seeing max 1Gbps per "connection" on CCR1036-8G-2S+ as well.

We can obviously achieve much higher throughput using multiple connections, but any individual connection can only reach 1Gbps. I believe this is due to the current queuing implementation used on Tilera, and can possibly be improved.

Maybe Janis from Mikrotik can clear this up ?
http://thebrotherswisp.com/ | Mikrotik MTCNA, MTCRE, MTCINE | Fortinet FTCNA, FCNSP, FCT | Extreme Networks ENA
 
joegoldman
Long time Member
Long time Member
Posts: 533
Joined: Mon May 27, 2013 2:05 am

Re: Request for real Cloud Core Router HW (10Gbps)

Mon May 05, 2014 1:04 pm

For ISP aggregate traffic, which is really what MikroTik seems to be playing for, 1gbps single stream isn't that bad a limitation. We are a mid-sized wholesaler in AU and still will only do about 500-600mbit aggregate traffic.

Big single streams are more suited to switching rather than routing.
 
User avatar
doneware
Trainer
Trainer
Posts: 565
Joined: Mon Oct 08, 2012 8:39 pm
Location: Hungary

Re: Request for real Cloud Core Router HW (10Gbps)

Mon May 05, 2014 5:15 pm

They have announced the CCR-1072-8S+

72 cores, 8 10G ports, 1 1G port, redundant PSU.
spotted some nice pix of a prototype on the web:

https://pbs.twimg.com/media/Bg_zYx7IIAEIndN.jpg:large

if you look closely on the left side there is something strange. it has a mini-pcie/m-sata
connector, as i figure the mounting screws facing towards the back of the device. and there
is something what seems to be a sim card socket. e.g. it could have integrated 3g/4g connectivity
for like OoB access.

http://lanmarket.ua/upload-files/foto/p ... 2553-b.jpg
and a connector is named as "smart card" on this image.
also note the slide-in connectors at the back of the PSUs, which - i suspect - gives us hot swappable power supplies.
#TR0359
 
HaQs
Member Candidate
Member Candidate
Posts: 153
Joined: Sat Oct 20, 2007 3:26 pm
Location: POLAND

Re: Request for real Cloud Core Router HW (10Gbps)

Mon May 05, 2014 7:04 pm

+1 for hotswap PSU

+1 for PSU module (AC , DC-48V, DC+24V) or DC wide band 20-60V (with isolation +/-)
 
User avatar
doneware
Trainer
Trainer
Posts: 565
Joined: Mon Oct 08, 2012 8:39 pm
Location: Hungary

Re: Request for real Cloud Core Router HW (10Gbps)

Tue May 06, 2014 1:52 am

+1 for hotswap PSU

+1 for PSU module (AC , DC-48V, DC+24V) or DC wide band 20-60V (with isolation +/-)
telco dc (-48VDC) is usually 36-72V tolerant, and most of the time i measured voltages just above 50V.
i agree, a real -48vDC power supply in the ccr1072 would be a win for the telco market.
i easily added 48V->24V open frame power supplies to all mikrotik devices, because there was
plenty of free space available inside the rack mount case.

but looking at the 1072 prototypes my "retrofit" approach won't cut it anymore. either mtik delivers dc option
or we must design a compatible dc-dc power module because the stuff we normally use has a different form
factor if we go over 20w. and looking at the fans i suspect the 1072 will need around 80-100w at least.
or we need to use external 48VDC-230V inverters to get this box running in telco environments.
#TR0359
 
roadracer96
Forum Veteran
Forum Veteran
Posts: 714
Joined: Tue Aug 25, 2009 12:01 am

Re: Request for real Cloud Core Router HW (10Gbps)

Tue May 06, 2014 3:54 am

For ISP aggregate traffic, which is really what MikroTik seems to be playing for, 1gbps single stream isn't that bad a limitation. We are a mid-sized wholesaler in AU and still will only do about 500-600mbit aggregate traffic.

Big single streams are more suited to switching rather than routing.

Switching and routing are all the same thing these days. Can't see how you would think 1gbit throughput on a 10gb router is acceptable. Heck. A single Iscsi connection could max it.
 
joegoldman
Long time Member
Long time Member
Posts: 533
Joined: Mon May 27, 2013 2:05 am

Re: Request for real Cloud Core Router HW (10Gbps)

Tue May 06, 2014 12:42 pm

Switching and routing are all the same thing these days. Can't see how you would think 1gbit throughput on a 10gb router is acceptable. Heck. A single Iscsi connection could max it.
In no way are they the same 'these days'. Backplanes on good switches are massive, and if you have high transfer networks, you specifically design your network so as much data as possible goes via switching rather than through a router port. Even the big boys like Cisco, there is a big difference in routing and switching still.

Granted some of the Mikrotik ranges have switch chips for good performance, the high performance routers do not, and the specifically designed switches in Mikrotik range have much lower power CPU's where you could 'route' no where near the same amount of traffic as their proper routers.

Again - large single streams are typically not 'routed', and usually more local, hence switching offers a better solution. I agree there is a performance limitation that should be dealt with, but the fact of the matter is these tools are crafted for a specific job, in my opinion...aggregate traffic, which they will do well even with this limitation. Right tool for the right job applies and if Mikrotik is not achieving what you are after, then it is not the right tool for the job.
 
roadracer96
Forum Veteran
Forum Veteran
Posts: 714
Joined: Tue Aug 25, 2009 12:01 am

Re: Request for real Cloud Core Router HW (10Gbps)

Tue May 06, 2014 3:19 pm

Old switch ASICS only knew how to process VLAN tags, MAC Addresses, and maybe DSCP/TOS. Modern switch ASICS know how to process data at L4. This encompasses routing.

The $4k SWITCHES we use can ROUTE or SWITCH at line-rate across all ports simultaneously, 50 gbe ports and 2-4 10gbe ports. The core SWITCHES we use as ROUTERS can ROUTE or SWITCH 10gb x48 ports at line rate across all ports simultaneously. So, in the second case, that is 480gb full duplex = 960gb TCP, UDP, GRE, whatever, switched, routed, MPLS, running OSPF, BGP, PIM, etc, etc. The port-to-port latency is the same wether the packet was switched or routed. The handling of the packet is done in the same way wether it was switched or routed.

IE: These days, a switch and a router are the same thing. The software frontend is just that. A frontend that tells the ASICS/PGAs how to do their work, in hardware.

That differs when you are talking about a SOFT router that incorporates a switch-chip. But still, the assumption that nobody needs to ROUTE a single connection at more than 1gbit is just that, an assumption. It is a bad limitation. You don't advertise a device that can do xyz gbit fast path forwarding in bridging or routing when it can't really do that. They should have a little asterisk at the bottom of the claim *The following only applies to single/multiple stream lossy UDP/GRE/Non TCP/out of order processed/super fragmented packets. Otherwise, the single stream TCP limitation is 1gbit/second because we haven't finished making everything work right.

Mikrotiks are awesome.. Ive used them for years in the case we DONT need 960gbit of throughput with a $20k price tag.. But seriously... the snake-oil has to stop. You buy a 36 core soft-router that is advertised at being able to do ~3gb of ipsec throughput and upwards of 10gbit of slow-path forwarding and you end up with a device that can only manage a GRE or IPIP tunnel over IPSEC at 50mbit aggregate duplex using almost no CPU power, then get no response on forums or from support after multiple attempts to make contact.

Now, would a single stream limitation of 1gbit affect my desired use of these devices as a VPN concentrator? No... The max connection speed to the remote sites would be 1gbit.. BUT.. I need way more than 50mbit...
 
joegoldman
Long time Member
Long time Member
Posts: 533
Joined: Mon May 27, 2013 2:05 am

Re: Request for real Cloud Core Router HW (10Gbps)

Wed May 07, 2014 3:25 am

The $4k SWITCHES we use can ROUTE or SWITCH at line-rate across all ports simultaneously, 50 gbe ports and 2-4 10gbe ports. The core SWITCHES we use as ROUTERS can ROUTE or SWITCH 10gb x48 ports at line rate across all ports simultaneously. So, in the second case, that is 480gb full duplex = 960gb TCP, UDP, GRE, whatever, switched, routed, MPLS, running OSPF, BGP, PIM, etc, etc. The port-to-port latency is the same wether the packet was switched or routed. The handling of the packet is done in the same way wether it was switched or routed.

IE: These days, a switch and a router are the same thing. The software frontend is just that. A frontend that tells the ASICS/PGAs how to do their work, in hardware.
They are capable of talking routing protocols, but are not built for it. You wouldn't load a L3 switch up with multiple full routing tables plus IGP. The CPU/RAM wouldn't hold it. Same with technically you can route with the Mikrotik CRS range but you wouldn't put it at the core or edge (unless you are really small).
That differs when you are talking about a SOFT router that incorporates a switch-chip. But still, the assumption that nobody needs to ROUTE a single connection at more than 1gbit is just that, an assumption. It is a bad limitation. You don't advertise a device that can do xyz gbit fast path forwarding in bridging or routing when it can't really do that. They should have a little asterisk at the bottom of the claim *The following only applies to single/multiple stream lossy UDP/GRE/Non TCP/out of order processed/super fragmented packets. Otherwise, the single stream TCP limitation is 1gbit/second because we haven't finished making everything work right.
All their numbers and test results indicate exactly how the tests were done, including frame-size, pps and even the technology used.
Mikrotiks are awesome.. Ive used them for years in the case we DONT need 960gbit of throughput with a $20k price tag.. But seriously... the snake-oil has to stop. You buy a 36 core soft-router that is advertised at being able to do ~3gb of ipsec throughput and upwards of 10gbit of slow-path forwarding and you end up with a device that can only manage a GRE or IPIP tunnel over IPSEC at 50mbit aggregate duplex using almost no CPU power, then get no response on forums or from support after multiple attempts to make contact.

Now, would a single stream limitation of 1gbit affect my desired use of these devices as a VPN concentrator? No... The max connection speed to the remote sites would be 1gbit.. BUT.. I need way more than 50mbit...
Can't comment on the IPsec limitations I dont use the technology enough to have run into this problem, but fair enough I could see how THAT problem would be annoying.
 
roadracer96
Forum Veteran
Forum Veteran
Posts: 714
Joined: Tue Aug 25, 2009 12:01 am

Re: Request for real Cloud Core Router HW (10Gbps)

Wed May 07, 2014 3:41 am

Use l3 switches for our bgp peerings. 3 full v6 and v4 tables. All in hardware. Works perfect. College campus.

Yes. The tests show that. Which makes it even worse. They designed the tests to cover up the flaws.
 
AlexS
Member Candidate
Member Candidate
Posts: 266
Joined: Thu Oct 10, 2013 7:21 am

Re: Request for real Cloud Core Router HW (10Gbps)

Thu May 08, 2014 1:49 am

Old switch ASICS only knew how to process VLAN tags, MAC Addresses, and maybe DSCP/TOS. Modern switch ASICS know how to process data at L4. This encompasses routing.

Mikrotiks are awesome.. Ive used them for years in the case we DONT need 960gbit of throughput with a $20k price tag.. But seriously... the snake-oil has to stop. You buy a 36 core soft-router that is advertised at being able to do ~3gb of ipsec throughput and upwards of 10gbit of slow-path forwarding and you end up with a device that can only manage a GRE or IPIP tunnel over IPSEC at 50mbit aggregate duplex using almost no CPU power, then get no response on forums or from support after multiple attempts to make contact.

Now, would a single stream limitation of 1gbit affect my desired use of these devices as a VPN concentrator? No... The max connection speed to the remote sites would be 1gbit.. BUT.. I need way more than 50mbit...
I have to agree, i like the brand, but when you advertise a 28Gb/s router, that is actually rate limited to max 1Gb/s tcp streams, thats more than just snake-oil.....
 
snons
just joined
Posts: 8
Joined: Sun Feb 17, 2013 9:11 am

Re: Request for real Cloud Core Router HW (10Gbps)

Thu May 29, 2014 2:50 pm

Use l3 switches for our bgp peerings. 3 full v6 and v4 tables. All in hardware. Works perfect. College campus.
.
Which switches do you use?
 
computercolin
just joined
Posts: 4
Joined: Mon Dec 09, 2013 11:57 am

Re: Request for real Cloud Core Router HW (10Gbps)

Tue Mar 10, 2015 6:08 pm

Yes, I am curious as well.
roadracer96, what switches are you using for BGP?

Who is online

Users browsing this forum: Bing [Bot], npero and 25 guests