You can either run wireshark/tcpdump on firewall - the interface towards router. Or add a simpke linux host without any firewall into same subnet with firewall and router (you’ll have to make it larger than /30 though) and test connectivity between servers and test host. Router will behave the same way for both test host and firewall. You’ll have to configure router’s address as default gateway on test host to make sure firewall is not in the way.
Hi mate you are the man of the day i found why not working now. It was the OPNSense Firewall after an update this block on Floating rules you can not move the rules down weird devolopment and they wanna make with such things Enterprise Firewall. But now the weirdest problem i forget my password to this device and idk how to come inside tryed over Console connection but there is password too. Gives Such thing to recover the password without loosing config? I have not a backup.
Good moment to erase everything and start from scratch!
Yes i thing the same but after so much work and now the routing is there this will be an pain lol if i can not fins the password. What can i expect for routing performance for this device? When i make m a bond with 4 sfp+ ports? But the question is then which mode rr or lacp. Underhood will be the CRS326-24S-2Q.
I try to to make this so the CCR2004 is main core router for max routing performance.
downlink is an CRS326-24G-2S+ i am configuring this right now wanna 2 sfp+ ports to CCR with bond but mode lacp or rr for max performance idk at the moment
downlink is an CRS326-24S-2Q minimum with 2 sfp+ but preffer with 4 sfp+ for 40 gb links routing but again which mode made the bond will i don’t know. But i thing for max performance rr mode good right? And sry for my bad english. By the many thx for any help
Official performance numbers are here: https://mikrotik.com/product/ccr2004_1g_12s_2xs#fndtn-testresults
If you go without any firewall rule, then I guess the relevant line will be Routing, none (fast path) configuration. And performance will probably be somewhere between the first and second column (1518 and 512 byte packets), probably nearer the second due to VLAN decapsulation and encapsulation, so something like 25G bps to 30 Gbps. Unless vast majority of traffic will indeed use jumbo frames, in which case routing speed might be nearer 40 Gbps.
And I agree with @pe1chl, this is ideal opportunity to configure stuff again from scratch. For simple router, setup will be pretty simple: only bridge with appropriate interfaces as ports, VLAN interfaces, IP addresses and default route. None of firewall configuration.
Hi mate many thx for tips i get my password back,
i have all firewall rules deleted so no rule is present at the moment now, just only this nat is enabled even not any port is seledted this working. Till this firewall not blocks will be so if not more block will be delete this too but i am searching an option mybe to change another firewall who this works. For bonding is the right way rr mode or lacp?
For bonding make sure you select bonding mode well supported by both link partners. CRS3xx series supports LACP (803.2ad) and RR modes in hardware (others include switch CPU meaning miserable throughputs). With 803.2ad, you have possibility to choose between different transmit-hash-policy settings which affect bond performance depending on traffic pattern. The RR mode has one advantage which is not available with other modes: it will distribute traffic between member links also for single L4 connection (TCP or UDP) whereas other modes will keep single L4 connection on single link member.
Very nice thx all this devices support rr mode so i will make between them rr bond 2 links from 1 gb switch and 4 links from 10 gb switch this was my idea to make this so Proxmox servers will have each 40 gb links this way i can make max throughput out of them i think.
hi mate a question on rr mode it show link as connected but when i go to port no dhcp and with manual ip adress nothing can ping. on lacp ad mode it works both was with Layer 2 need by rr layer 3 and 4?
I don’t have any good ideas about RR not working, I’ve been using RR between linux hosts in the past. One gotcha I already mentioned: out-of-order delivery. TCP in theory should be able to deal with out-of-order packets (some TCP implementations are not exactly happy about it, reducing throughput and increasing retransmission counts), but for other protocols (UDP, ICMP, ..,) that’s definitely an issue. In your particular case (jumbo frames on LAN segments, normal frames on ethernet) it’s even harder on router (and consequently on the rest) because if MTU change … either router has to perform fragmentation or it has to drop packets if those are marked with DF flag. Fragmentation means more delay and delay jitter for each packet in router and greater chance to end up with out-of-order delivery.
Personally I’d test to see if using jumbo frames actually improves LAN performance considerably enough to warrant bothering with different MTU sizes.
Hi mate many thx for answer i had to to change Switch to CCR2004 uplinks to LACP to in order it works. One of my proxmox Server didn’t play the rr game and not worked but the other 2 Working don’t know what to say half working half not lol. Btw i changed my Firewall with a cheap EdgeRouer and i is working like a charm without nat now.
On qsfp ports i see 4 10 gb links you need to bond them? to get the full 40 gb speed?
@mkx,
Only 802.3ad and balance-xor mode are hardware offloaded on CRS3xx devices…
https://wiki.mikrotik.com/wiki/Manual:CRS3xx_series_switches#Bonding
I 've not used balance-rr on a CRS3xx device only 802.3ad, so i can’t tell if that is true or not, but according to the manual rr is not hardware offloaded.. Unless i understand something wrong …
I stand corrected.
ok, any reference on that?
That rr is hardware offloaded on CRS3xx devices ? I can’t find any reference on the wiki my self…
Also http://forum.mikrotik.com/t/high-cpu-plus-latency-plus-packet-drops-when-bonding-with-balance-rr/127016/1
Can someone help me on the below post ? I have posted on Aug 22nd but nobody helped.
HELP NEEDED RB750Gr3- Load balancing and Failover configuration
http://forum.mikrotik.com/t/help-needed-rb750gr3-load-balancing-and-failover-configuration/151226/1