Did you think about making a CCR that has empty slots for ports?

i.e.
CCR1009-MOD with 3 slots
CCR1016-MOD with 4 slots
CCR1036-MOD with 4 slots (I think that's max possible here?)
CCR1072-MOD with 8 slots (maybe 2U if it can't fit in 1U)

and daughter-boards for these slots would be:
MOD-1S+ - 1x SFP+
MOD-4S - 4x SFP
MOD-4G - 4x Copper Gigabit

We'd love CCR1036-8S-2S+, others would probably want some other kind of setup. This would solve everyone's needs and wouldn't require you to support (and market) a zillion of configurations.

Just make sure there will be enough lines to cpu cores to feed the additional ethernet / sfp / sfp+ ports...

that will be nice but keep in mind modular equipment is expensive

The slots need to be keyed similarly to M.2, because it is not always possible to replace 4x 1Gb with 1x 10GbE.

A 4x 1Gb module from Cisco costs as much as a CCR1009!

Modular equipment is expensive because why not. There isn't any technical reason for it to be expensive.

As to technical aspects, Mikrotik guys already shown that they're more than capable of making good devices -- I'm sure then can come up with a good solution, especially with past daughterboard experience. They came a long way and learned a lot since RB333 times, when for every 10 RBs we bought, 11 were broken

@vortex, I wasn't suggesting any interoperability with other vendors modules. These would only work with mikrotik's CCRs to allow more flexibility to customers.

I'm just saying, if that is Cisco's price, I would not be surprised if a Mikrotik module cost $100.

Well, to find out about pricing, someone from mikrotik would need to take an interest with the subject...

i am totally ignorant about building networking devices, but looking the schematics of Tile Gx chips CCR line is based on looks like that platform are designed for modularity, will be a good idea

looking schematics tile gx 36 and tile gx 72 can exchange 4 gigabit ethernet by each 10g ethernet interface, to build maybe a 32 gigabit ethernet ccr1072 or 4 10gigabit ethernet + 16 gigabit ethernet.

It costs nothing to dream

Yes, it should all be possible. Now we only need grab some traction with mikrotik guys.
I'm sure more flexibility means more customers.

i am totally ignorant about building networking devices, but looking the schematics of Tile Gx chips CCR line is based on looks like that platform are designed for modularity, will be a good idea

looking schematics tile gx 36 and tile gx 72 can exchange 4 gigabit ethernet by each 10g ethernet interface, to build maybe a 32 gigabit ethernet ccr1072 or 4 10gigabit ethernet + 16 gigabit ethernet.

It costs nothing to dream

I wrote several times here product names for configurations that would be useful for different reasons.

It would be better to make them fully modular even if for some slots you can only choose between 4x 1GbaseT and 4x SFP. If the price for a module is not too high.

If modular is no go, the first thing that there should be is 2S+ versions for all CPU models, as I mentioned recently.

modularity would be nice for the CCR. Based on Tilera's specs some of the stuff on it can be made modular while some cant.

A few things that the chip itself has
- extra ethernet ports through an interface.
- 4 fixed interface handlers (you cant make these modular, essentially its 4 ethernet ports/1 10gbe per handler)
- Multiple PCIe allowing up to 3 or 4 devices (not lanes)

What disappointed me about mikrotik's CCR was that it lacked many things especially SATA and PCIe. It had 4 fan connectors but only 2 worked, there was no cutout or a way you could add an additional PSU yourself. The cooling could've used a much better design so it would not only be silent but also have better airflow.

Read the TileGX documentation on Tilera's site. Even their dev devices (rackmount not PCIE) have modularity such as PCIe,

Mikrotik i know a lot of things can make the device cost more but im not asking for dual PSU CCRs, im asking for the option to be able to add fans and PSU and other things so you can keep the base device cheap and it would be up to the customer to expand on these options if they want to.

I still want a fanless desktop unit with 1 provided internal PSU, so modularity is a problem.

If CCR1009-4G-2S+ cannot handle routing a single connection at 5Gbps through NAT firewall, I'm not interested.

that will be nice but keep in mind modular equipment is expensive

not necessarily. until you can solve everything in the same SOC, it's sort of plug and play.
doing distributed stuff, now that's where the purse opens.

there are obvious tradeoffs when you play around with traces on a pcb and try to replace them
with connectors. as of now, the TILE chips do the stuff all alone. you may expose the XAUI or
sgmii interfaces to daugtherboards, and you can keep the same features as you do have right now.
you may also use pcie lanes, but right now all networking ports are on the cpu, just some "mgmt" gige
is connected to pcie on the 1072.
but i am not a PCB design expert, and bitrates like this have their tricky parts, so it may not
will be that easy as i explained.

on the other hand i can do "modular" CCR. get the 1072, and start adding on crs226s as "port expanders"
to the 10GE ports if you keep two 10GEs for redundant uplinks, you can fan out the rest 6 into 72
almost wirespeed routed gige ports, where the crs just does plain wirespeed forwarding between the
10GE ports and the ethernets. all routing and acrobatics is done on the 1072.

what this concept does miss is the all-sfp switches. you can have the crs210 on a 10ge port with 10xSFPs,
which is nice, but it's not connected redundantly to the core. and you can't mount two of them SBS in
a rack. we really need a 24ports SFP switch with 2xSFP+ uplink and _proper_ cooling.

I still want a fanless desktop unit with 1 provided internal PSU, so modularity is a problem.

If CCR1009-4G-2S+ cannot handle routing a single connection at 5Gbps through NAT firewall, I'm not interested.

Someone used the CCR1009 with NAT and PPPOE with some firewall and QoS, it used one core to handle 500Mb/s of throughput. So it could handle 5Gb/s with NAT if you didnt need PPP otherwise it would just barely handle it.

I still want a fanless desktop unit with 1 provided internal PSU, so modularity is a problem.

If CCR1009-4G-2S+ cannot handle routing a single connection at 5Gbps through NAT firewall, I'm not interested.

Someone used the CCR1009 with NAT and PPPOE with some firewall and QoS, it used one core to handle 500Mb/s of throughput. So it could handle 5Gb/s with NAT if you didnt need PPP otherwise it would just barely handle it.

The keyword is single connection.

on the other hand i can do "modular" CCR. get the 1072, and start adding on crs226s as "port expanders"
to the 10GE ports if you keep two 10GEs for redundant uplinks, you can fan out the rest 6 into 72
almost wirespeed routed gige ports, where the crs just does plain wirespeed forwarding between the
10GE ports and the ethernets. all routing and acrobatics is done on the 1072.

very good idea

I still want a fanless desktop unit with 1 provided internal PSU, so modularity is a problem.

If CCR1009-4G-2S+ cannot handle routing a single connection at 5Gbps through NAT firewall, I'm not interested.

Someone used the CCR1009 with NAT and PPPOE with some firewall and QoS, it used one core to handle 500Mb/s of throughput. So it could handle 5Gb/s with NAT if you didnt need PPP otherwise it would just barely handle it.

The keyword is single connection.

If it is a single WAN than yes.

I still want a fanless desktop unit with 1 provided internal PSU, so modularity is a problem.

If CCR1009-4G-2S+ cannot handle routing a single connection at 5Gbps through NAT firewall, I'm not interested.

If it is a single WAN than yes.

i think he refers to one tcp/ip connection not 1 isp connection

in contrast in 1 isp connection you can have hundreds if not thousands of tcp/ip connections

Yes.

Yes.

i think a good performance with NAT can be achieved with Fast track

the real problem is queues, is the most taxing and throttling thing.

what throughput you achieve with fasttrack when doing NAT?? which equipment?cpu sage?? cpu usage on most occupied core??

on single tcp/ip connection off course

I meant Yes, the problem is CCR's limitation to 1Gbps per TCP connection.

I meant Yes, the problem is CCR's limitation to 1Gbps per TCP connection.

Have you tried FastPath/FastTrack ?

A modular CCR would indeed be possible.
There are two ways I can see it being done:

1. Use XAUI / SGMII from mainboard to "Interface Cards"
This would be easyish to engineer, but would likely require the router to be off when swapping line cards.

This is similar to how the Juniper SRX's and Cisco ISR's work. Pretty easy to engineer but it is basic.

2. Have a chassis based solution. Each linecard has a Tilera Gx16 or other suitable processor (EZCHIP NPx series), and is running cutdown OS and it's own FIB, then use PLX controllers on the PCIe bus to talk to a PCIe switch and transport traffic between linecards over the PCIe based bus, or some other proprietary bus.
There would be then be "Route Processors" that run RouterOS, these do very little packet processing, they are just for running the routing protocols and RIB, LACPBDU processing, STPBDU processing and so on. All the forwarding is done on the linecards.

This is similar to what AlcatelLucent do with their SR7750's, Cisco with their ASR's and Juniper with their MX's. Not a small amount of engineering.

I think Mikrotik are better concentrating on getting RouterOS v7 out the door, feature complete and stable before taking on a massive project like making a modular CCR.

I meant Yes, the problem is CCR's limitation to 1Gbps per TCP connection.

Have you tried FastPath/FastTrack ?

I don't have a CCR. It is always quoted here that there's a core limitation to this speed.

I meant Yes, the problem is CCR's limitation to 1Gbps per TCP connection.

There no such limitations.

Only real life situation where you possibly have 1gbps+ single TCP connection is in your local network, where you can increase MTU to 9000 and it can get all the way up to 10Gpbs.

This myth was generated by some noobs, that tried to use TCP Bandwidth test on CCR, and as we all know Bandwidth tool is limited to single core for traffic generation.

This is similar to what AlcatelLucent do with their SR7750's, Cisco with their ASR's and Juniper with their MX's. Not a small amount of engineering.

apparently cisco uses ethernet in asr9k to establish the control plane between the linecard NPUs and the RP CPU. this concept is nothing new, they had similar in 6500/7600, it is referred to as EOBC (ethernet over backplane channel).
even their multi-chassis 9k setup uses external 10GE ports to interconnect nodes.
the smaller asr1k shares the same EOBC concept internally.

so there's a hidden switch inside the router just for controlling the beast

the data plane is done with fabric where there's an arbiter which takes care that speakers and listeners use the fabric paths in a controlled manner.
this is described very easily, but it really takes huge efforts to make it work that way. and there are special cases like multicast which make this setup even more complicated.

Great info!

A lot of equipment uses PCIe as the "fabric" between the nodes/line cards. There are off-the-shelf chipsets and toolkits from PLX/Avago for this exact purpose.

We expressed an interest in having MikroTik develop a multi slot router by releasing the super secret photos of the CCR-6506-96G-48S-16S+ on April 1st

http://www.stubarea51.net/2016/04/01/ph ... g-48s-16s/

Only real life situation where you possibly have 1gbps+ single TCP connection is in your local network, where you can increase MTU to 9000 and it can get all the way up to 10Gpbs.

This myth was generated by some noobs, that tried to use TCP Bandwidth test on CCR, and as we all know Bandwidth tool is limited to single core for traffic generation.

i agree with you

i have tested NAT with fastrack on a CCR1009 and easily moves 700mbit/s (64kpps) with a total CPU usage of 2%, Idle cpu of minimum 94%, and the most busy core gets a maximum of 8% of load on a single tcp connection

unfortunately i don't have equipment to test beyond 1gbit, but i think its clear that the CCR is not the limitation on my test

testing beyond 1gbit interface is not an easy task, even more harder using 1500 mtu

CCR-6506-96G-48S-16S+/

First of all there should be no dash after CCR!!! you need to be strict about those things (lol)
Second first number is too unrealistic - 6th series with 506 cpu cores?

CCR-6506-96G-48S-16S+/

First of all there should be no dash after CCR!!! you need to be strict about those things (lol)
Second first number is too unrealistic - 6th series with 506 cpu cores?

Multi-chassis CCRs have new naming conventions (And using part of the Cisco 6506 designation was part of the joke )

Only real life situation where you possibly have 1gbps+ single TCP connection is in your local network, where you can increase MTU to 9000 and it can get all the way up to 10Gpbs.

This myth was generated by some noobs, that tried to use TCP Bandwidth test on CCR, and as we all know Bandwidth tool is limited to single core for traffic generation.

i agree with you

i have tested NAT with fastrack on a CCR1009 and easily moves 700mbit/s (64kpps) with a total CPU usage of 2%, Idle cpu of minimum 94%, and the most busy core gets a maximum of 8% of load on a single tcp connection

unfortunately i don't have equipment to test beyond 1gbit, but i think its clear that the CCR is not the limitation on my test

testing beyond 1gbit interface is not an easy task, even more harder using 1500 mtu

It does take a lot of resources to do testing at any kind of volume. We currently have up to 80 Gbps of capacity for throughput testing in our lab and have taken the CCR1009 to 10 Gbps @ 9000 MTU

Only real life situation where you possibly have 1gbps+ single TCP connection is in your local network, where you can increase MTU to 9000 and it can get all the way up to 10Gpbs.

Start of rollout of DOCSIS 3.1 @ 5Gbps is planned for next year in Europe.

It is easy to max out your 500Mbps today downloading from hyperscale data centers.

This gives Mikrotik a year to come up with a proper 2S+ router in the CCR1009 spirit.

Only real life situation where you possibly have 1gbps+ single TCP connection is in your local network, where you can increase MTU to 9000 and it can get all the way up to 10Gpbs.

Start of rollout of DOCSIS 3.1 @ 5Gbps is planned for next year in Europe.

It is easy to max out your 500Mbps today downloading from hyperscale data centers.

This gives Mikrotik a year to come up with a proper 2S+ router in the CCR1009 spirit.

Not only that, Data Center interconnects use 9000 MTU typically and will easily run more than 1 Gig on single TCP streams for storage replication and many other high volume tasks.

In other words, the cloud kinda needs it to function

What should be MTU if i use 10G sfp+ ? Does having a mtu of 1500 cause any problems on 10Gb interface ?

What should be MTU if i use 10G sfp+ ? Does having a mtu of 1500 cause any problems on 10Gb interface ?

the answer is : THE BIGGER YOU CAN !!

the fact is with 1500mtu you need almost 1mpps (1 million packet per second) to achieve 10gbps

changing to 9000mtu you cut that by 6x, that is almost 150kpps (150 thousand packet per second)

6X less computational resources to achieve the same result, that's a lot of difference!!

you have to be sure all your infrastructure support the MTU size you intend to use

9000 byte MTU does not bring you much for an internet router, as the de-facto MTU on internet is still 1500 bytes so the router would either need to fragment or advertise the MTU bottleneck to your endsystems resulting in 1500 byte MTU on your local network.
It could be useful internal to a company network, especially in storage area networks and similar.

With the dropping prices of SFP(+) to copper modules, a modular CCR is probably further away than ever. Just get a model with SFP(+) ports and plugin the SFP to copper modules you require.

I write same answer at other topic also sory for this.
I hope this will be understandable for my problem.
<<Provider>> ===802.3ad (3x 1Gb interface) === <<CCR1072>>> ===10Gb interface=== <<CCR1036>> after this all other networks 1Gb interface. so for this 10Gb connection do I need increase mtu size ?

Thanks