Today made necessary change for collabsing hardware (Xeon), but will return it immediately.
CCR1016-12G latency jitter is just terrible from constant ,3ms to following on picture
1880mangles, 350firewalls, 200Mbit traffic. Unbalanced load of CPUs touching by one of those all the time about 85percent, but average 20percent
This is ping thru router on local network gbit network, before it was constant ,3ms
— 10.10.9.253 ping statistics —
947 packets transmitted, 947 received, 0% packet loss, time 948371ms
rtt min/avg/max/mdev = 0.280/1.756/23.996/2.478 ms
gosh.. keep on subject, what it has to do with CCR?
XEON 2 cores 2,9GHz, 2GB RAM with built in 4 Gbit ports What exactly is that I dont know.. I can take a look once returning it instead of CCR
I haven’t purchased a CCR yet and the question I was asking what hardware would CCR replace or put another way should I be considering other hardware than CCR - a difficult question and very much depending on application but sometimes it’s better to get a older product with a proven track record of reliability and average performance than a new product with high performance but has low reliability record – this comment applies to all new products and not just the CCR?
We have one CCR for testing. It died and we get it repaired by MT.
Since then it runs flawless running in our OSPF core holding one minor connection.
We put much hope in this beast at it has the hw to hold 4 licensed links per site
and should be fast enough to replace our gigabit switches by simply bridging ports.
At the moment we proceed slowly as we are not sure ROS6 is stable enough and
how the MTBF of this HW is.
And now try to guess, which part of graph is CCR 16core running 25 percent top load and what is Xeon working in on 75percent top load..
Guys do the CPU balance better way, then I believe it could be really bulls eye, until that I spent 3K USD not the right way
I repeat it is approximately 400firewall rules, 250 NAT rules and 1890mangles for queue tree
I mean: Why is your CCR routes so cute and fun… NO. I mean what is the problem, that your platform brings me such heavy latency jitter and huge latency at all. I presume, that the problem is bonded directly with a instant load of 1 of cores on CCR. The load is pretty unbalanced,where average load is 25 percent, but 1 or 2 cores are running 70-90percent of load randomly, while rest other 2 cores are having load 1-5 percent.
Basicly say yes.
If I will compare it with x86 platform there is no latency jitter at all - under all circumstances thruput thru router takes about 100us up to 95 percent load under same setting and 300-350Mbit load.
Which is normall and for example RB1200 which we are having on one location where we need it to change as soon as possible because of 100CPU time to time load, but latency grows fluently approaching 100percent of CPU time, but the jitter is still like 1-2ms (long term average 2 ms, but jitter makes it 2,3 -4 ms randomly, and under heavy load 7-9 ms randomly).
By CCR with a same setting as X86 it behaves strange way.
As a regular user I cant see differences, but once I have latency statistics i got miserable latency jitter like 10-30ms. So instead of fluent 1ms ping I am getting results as in ms is:
1
4
3
2
1
4
13
2
15
25
2
4
3
1
4
3
4
etc..
The average load of all CPU cores is 25 percent, but once you look on resources and CPU use, and you will allign CPUs in row, you will see, that 70 percent of load are splitted between 3-4 cores and rest is idling about 5-30percent. An as you watch the single largest core load, every time it jumps toward 80 percent on single core, latency jitter appears. So it has to do with load balance between cores i believe. Once you will sack off mangles, it stabilises, but only because lower overal load.. By other words, CCR has good enough result up to 15 percent average load of all CPU..
It is no secret, and it has been discussed in other topics, that some few features are not yet optimized for multicore. Some specific firewall matchers etc. Upcoming versions will solve this.
If one packet needs to be processed by another core, this will happen as you see it. Adding full multicore optimisation for the remaining few things will fix your problem.
Hello,
I think this discussion is exactly about the problem I’m dealing with.
My CCR started to be unstable and after come crash and reboot, there was only “Starting services” on the LCD, so i had to replace it for a while with my backup device UBNT EdgeMAX PRO router.
Can you please explain the difference between pings? It is just a direct 1Gbps line. One meter of cable fron the server to the router. Same server, same cable.
Now I have the router back, but I’m about to leave the UBNT router as a main router and the CCR as a dissaster replacement.
As I can see, I’m not the only one, who can see the problem with delays.
The CCR performs poorly with virtual networks but performs very fast on physical networks. Ping times significantly increase using virtual networks (internal bridge routing, VPN, etc) while physical routing remains really fast. Even VPN is unstable at full loads and disconnects every few minutes when at full load. Its almost as is if the mesh network of the TILE CPU has high latency when the latencies on the specsheets are in nanoseconds.
