Community discussions

MikroTik App
 
syadnom
Forum Veteran
Forum Veteran
Topic Author
Posts: 824
Joined: Thu Jan 27, 2011 7:29 am

vxlan performance?

Sat Feb 15, 2020 1:09 am

Will vxlan have a fast-path option for hardware accelleration? I'm seeing some overhead in a VM with bandwidth test over vxlan right now. I know it's early, but this is a very interesting subject matter for me.
 
mbovenka
Member
Member
Posts: 372
Joined: Mon Oct 14, 2019 10:14 am

Re: vxlan performance?

Sat Feb 15, 2020 1:32 am

The Marvell PresteraDX chips in the CRS3xx series switches can do VXLAN in hardware, so offload is technically possible. It's up to MT to actually implement it in RouterOS.
 
syadnom
Forum Veteran
Forum Veteran
Topic Author
Posts: 824
Joined: Thu Jan 27, 2011 7:29 am

Re: vxlan performance?

Sat Feb 15, 2020 1:39 am

yeah, CRS3xx and Routeros7 with vxlan is a game changer... The prospect of this really puts some other hardware purchases in jeopardy.
 
User avatar
StubArea51
Trainer
Trainer
Posts: 1758
Joined: Fri Aug 10, 2012 6:46 am
Location: stubarea51.net
Contact:

Re: vxlan performance?

Sat Feb 15, 2020 4:01 am

Agreed. If we see EVPN added to BGP to support VxLAN in the future, MikroTik will see a massive increase in sales from people who need inexpensive devices to act as a VTEP.
 
syadnom
Forum Veteran
Forum Veteran
Topic Author
Posts: 824
Joined: Thu Jan 27, 2011 7:29 am

Re: vxlan performance?

Sat Feb 15, 2020 4:14 am

Evan just vxlan and Bridges is awesome if the crs3xxx line can bridge the vxlan to a port in hardware, but full EVPN kit would be stellar.

There is nothing in this price range by a mile.
 
User avatar
nz_monkey
Forum Guru
Forum Guru
Posts: 2208
Joined: Mon Jan 14, 2008 1:53 pm
Location: Over the Rainbow

Re: vxlan performance?

Wed Feb 19, 2020 2:08 am

Not all CRS3xx hardware has VXLAN support in the ASIC.

CRS309 and CRS317 do.

CRS326 and CRS354 do not.
 
mbovenka
Member
Member
Posts: 372
Joined: Mon Oct 14, 2019 10:14 am

Re: vxlan performance?

Wed Feb 19, 2020 2:05 pm

Not all CRS3xx hardware has VXLAN support in the ASIC.
CRS309 and CRS317 do.
CRS326 and CRS354 do not.

Interesting. So the 98DX82xx ASICs do VXLAN, but the 98DX32xx & 98DX83xx don't? I can't find that anywhere, and on the main PresteraDX page Marvell seems to claim VXLAN support for the entire PresteraDX family. Where did you find that it's not that simple?
 
vFastChrisHumphreys
just joined
Posts: 13
Joined: Wed Feb 25, 2015 8:24 pm

Re: vxlan performance?

Sun Feb 23, 2020 3:10 am

Hardware Office VXLAN on CRS326-24S+2Q+RM would be amazing.
 
Barlennan
just joined
Posts: 7
Joined: Sun Jun 05, 2011 9:45 am

Re: vxlan performance?

Tue Feb 25, 2020 12:32 am

Hardware Office VXLAN on CRS326-24S+2Q+RM would be amazing.
It's a different switch chip for the CRS326-24S+2Q+RM (98DX8332) than for the other CRS326-24g-2S+RM (98DX3236)
Even on their webpage the PDF for the 98dx83xx series says that it suports NVRGE, VXLAN-GPE, GENEVE, SPB, and 802.1BR port extender. https://www.marvell.com/content/dam/mar ... 016-12.pdf

I might be missing something but doesn't that mean that the CRS326-24S+2Q+RM would support VXLAN with the 7.0beta5 release as well?
 
User avatar
nz_monkey
Forum Guru
Forum Guru
Posts: 2208
Joined: Mon Jan 14, 2008 1:53 pm
Location: Over the Rainbow

Re: vxlan performance?

Tue Feb 25, 2020 1:22 am

Not all CRS3xx hardware has VXLAN support in the ASIC.
CRS309 and CRS317 do.
CRS326 and CRS354 do not.

Interesting. So the 98DX82xx ASICs do VXLAN, but the 98DX32xx & 98DX83xx don't? I can't find that anywhere, and on the main PresteraDX page Marvell seems to claim VXLAN support for the entire PresteraDX family. Where did you find that it's not that simple?
It is buried in the datasheets. The 98DX32xx have no VXLAN support in hardware, but the 98DX82xx and 98DX83xx do have it. So it looks like the newer CRS326-24S+2Q+RM model may be able to support it as well, however the CRS354 will not.
 
User avatar
StubArea51
Trainer
Trainer
Posts: 1758
Joined: Fri Aug 10, 2012 6:46 am
Location: stubarea51.net
Contact:

Re: vxlan performance?

Tue Feb 25, 2020 5:39 pm

Having VxLAN support in HW on the CRS326-24S+2Q+RM would be just amazing.
 
Barlennan
just joined
Posts: 7
Joined: Sun Jun 05, 2011 9:45 am

Re: vxlan performance?

Tue Feb 25, 2020 5:39 pm

Agreed. If we see EVPN added to BGP to support VxLAN in the future, MikroTik will see a massive increase in sales from people who need inexpensive devices to act as a VTEP.
+1,000 (CRS326-24S+2Q+RM) If these really do VxLAN and we get EVPN so we can make them VTEP's
Instead of +1. We get to do it for the number of units we would potentially buy right?
While I'm putting together a wish list could we get a...
CCR1288-1G-8Q+RM :shock:
1U rackmount, 1x Gigabit Ethernet, 8xQSFP+ cages, LCD, 288 cores x 1GHz CPU, 64GB RAM, up to 480 million packets per second, 320Gbps throughput, RouterOS L7
???
 
Barlennan
just joined
Posts: 7
Joined: Sun Jun 05, 2011 9:45 am

Re: vxlan performance?

Tue Feb 25, 2020 5:44 pm

Having VxLAN support in HW on the CRS326-24S+2Q+RM would be just amazing.
Low cost Municipal Fiber Open Network here we come!
 
User avatar
nz_monkey
Forum Guru
Forum Guru
Posts: 2208
Joined: Mon Jan 14, 2008 1:53 pm
Location: Over the Rainbow

Re: vxlan performance?

Wed Feb 26, 2020 4:56 am

The key thing to allow scalability of VXLAN in Metro Ethernet and Datacentre/Cloud is eVPN support.

Hopefully Mikrotik Product Manager's understand this.
 
anubisg1
just joined
Posts: 11
Joined: Sat Feb 23, 2019 1:29 pm
Contact:

Re: vxlan performance?

Sat Feb 29, 2020 4:08 pm

i understand that we only have VXLAN itself... is it correct to assume it's working in flood and learn? is EVPN already supported?
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 23406
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: vxlan performance?

Sat Feb 29, 2020 8:04 pm

Hey, dont forget us with current MT routers like the RB450Gx4 with the AR8327 switch chip. Can Router Os7 please have hw offload in vlan bridges extra pretty please.
 
Barlennan
just joined
Posts: 7
Joined: Sun Jun 05, 2011 9:45 am

Re: vxlan performance?

Wed Mar 04, 2020 2:28 am

I used the VxLAN lab from IPANetEngineer's example configuration post. VxLAN destroys my 951ui2hnd's 100% CPU at around 60Mb to 75Mb TCP.
Same lab with HAPAC2 is better I average 220Mb but it's not balancing across the CPU's. With my computer plugged into R1 when I'm downloading it uses 100% of one of the CPU's and almost nothing of the other three. On upload it is using 100% of the CPU on R3(To Internet) and almost nothing on the other CPU's.
Am I missing something or is there something I should be able to tweak?
 
syadnom
Forum Veteran
Forum Veteran
Topic Author
Posts: 824
Joined: Thu Jan 27, 2011 7:29 am

Re: vxlan performance?

Wed Mar 04, 2020 3:09 am

I used the VxLAN lab from IPANetEngineer's example configuration post. VxLAN destroys my 951ui2hnd's 100% CPU at around 60Mb to 75Mb TCP.
Same lab with HAPAC2 is better I average 220Mb but it's not balancing across the CPU's. With my computer plugged into R1 when I'm downloading it uses 100% of one of the CPU's and almost nothing of the other three. On upload it is using 100% of the CPU on R3(To Internet) and almost nothing on the other CPU's.
Am I missing something or is there something I should be able to tweak?
I think it's just too early on to really expect much more than a software path on this. I would really like to see more down the pipe for sure, like some hardware decapsulation for switches and proper evpn support but it's a very cool toy to play with right now.
 
Barlennan
just joined
Posts: 7
Joined: Sun Jun 05, 2011 9:45 am

Re: vxlan performance?

Wed Mar 04, 2020 5:55 pm

I think it's just too early on to really expect much more than a software path on this. I would really like to see more down the pipe for sure, like some hardware decapsulation for switches and proper evpn support but it's a very cool toy to play with right now.
It is a cool toy to play with right now. Of course I want the hardware path in the future. I don't even mind the software only side for now...
I was just wondering if anyone had got it to work across more than one CPU?
 
RcRaCk2k
Member Candidate
Member Candidate
Posts: 151
Joined: Mon May 07, 2012 10:40 pm

Re: vxlan performance?

Sun Jul 16, 2023 4:29 pm

CRS317 do.
Is there any prove that CRS317 do VxLAN in Hardware on the SwitchChips ASIC?

Because the documentation of ROS 6 and 7 do not tell anything about that. There is VxLAN flagged to be done on CPU only.

https://help.mikrotik.com/docs/display/ ... ng-Monitor
vxlan.jpg
You do not have the required permissions to view the files attached to this post.
 
User avatar
tangent
Forum Guru
Forum Guru
Posts: 1691
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: vxlan performance?

Sun Jul 16, 2023 5:12 pm

The detailed programming manuals for these ASICs are intended to be available only to those who have signed NDAs. Either someone is talking out of school, or they've gotten a copy of the reference books another way and are reporting what they've found.

The thing is, it doesn't matter which is the case. Even if the claim up-thread is correct, all he's claiming is that the chip at the core of the CRS317 can do VXLAN in hardware. This does not tell you that RouterOS currently programs the chip to do what you're asking, or if it does, that the programmed functionality meets your requirements.

You're free to ask MikroTik to support this feature, and they will decide what resources to put into it. The existence of ASIC-level support guarantees you nothing about how they will respond, how soon, or how well.
 
syadnom
Forum Veteran
Forum Veteran
Topic Author
Posts: 824
Joined: Thu Jan 27, 2011 7:29 am

Re: vxlan performance?

Sun Jul 16, 2023 5:15 pm

It’s not actually buried deep in reference material, it’s in the bullet point features on Marvell’s website for the chipset.
 
User avatar
tangent
Forum Guru
Forum Guru
Posts: 1691
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: vxlan performance?

Sun Jul 16, 2023 5:19 pm

Yes, I was indeed ignoring the possibility that the public marketing slicks — the things they dare to call "datasheets" — had useful information on them. It's rare, but it does happen. :)
 
DarkNate
Forum Guru
Forum Guru
Posts: 1065
Joined: Fri Jun 26, 2020 4:37 pm

Re: vxlan performance?

Sun Jul 16, 2023 8:39 pm

VXLAN on Tik is useless for professional use, as it doesn't support EVPN. Hardware/CPU or not.
 
User avatar
BartoszP
Forum Guru
Forum Guru
Posts: 3323
Joined: Mon Jun 16, 2014 1:13 pm
Location: Poland

Re: vxlan performance?

Sun Jul 16, 2023 8:53 pm

DarkNate ... seems that Mikrotik is a guilty pleasure for you .... you hate it but you cannot live without it. If you are so disgusted you can just not read, not comment and leave the forum.

Please change your attitude ... please do look on the bright side of administrator life.
 
DarkNate
Forum Guru
Forum Guru
Posts: 1065
Joined: Fri Jun 26, 2020 4:37 pm

Re: vxlan performance?

Sun Jul 16, 2023 8:57 pm

DarkNate ... seems that Mikrotik is a guilty pleasure for you .... you hate it but you cannot live without it. If you are so disgusted you can just not read, not comment and leave the forum.

Please change your attitude ... please do look on the bright side of administrator life.
viewtopic.php?t=197658#p1013392
 
syadnom
Forum Veteran
Forum Veteran
Topic Author
Posts: 824
Joined: Thu Jan 27, 2011 7:29 am

Re: vxlan performance?

Sun Jul 16, 2023 9:13 pm

It looks like mikrotik is moving towards EVPN, it just has a number of components.

My only criticism here is that for most, it's not super useful without hardware accelleration.
 
RcRaCk2k
Member Candidate
Member Candidate
Posts: 151
Joined: Mon May 07, 2012 10:40 pm

Re: vxlan performance?

Tue Jul 18, 2023 5:30 pm

Seeing the new CRS504-4XQ-IN that will be really cool.
Swtich-Chip: MARVELL® Prestera® 98DX4310 Multi-Layer Ethernet Switch

Also this switch supports
  • Layer-2 Wire-Speed Switching engine with large MAC table
  • Layer-3 Wire-Speed Routing engine with large LPM table
  • CAPWAP overlay support
  • Virtualization – IEEE 802.1Qbg EVB, 802.1BR Port Extender
  • Virtual overlay networking – NVGRE, VXLAN-GPE, GENEVE, SPB, TRILL, GRE
  • NFV Service Function Chaining – Network Service Header (NSH)
  • SW compatibility to 98DX42xx, 98DX83xx, 98DX84xx, 98DX85xx
https://www.marvell.com/content/dam/mar ... 018-07.pdf

I hope we will see that HW-Offloading Implementation supported by ROS in the next new versions.
 
dkayza
just joined
Posts: 5
Joined: Tue May 16, 2023 11:21 am

Re: vxlan performance?

Mon Nov 27, 2023 10:06 am

has there been any news on this from the devs?

would really love to see HW offloading support available on CRS326 for vxlan
 
RcRaCk2k
Member Candidate
Member Candidate
Posts: 151
Joined: Mon May 07, 2012 10:40 pm

Re: vxlan performance?

Fri Dec 22, 2023 10:30 pm

So as there is no VxLAN with Hardware Offload available for CRS317, because of leak of software side support...

What is the correct way with CRS317 to tunnel sfpplus10 on SW#1 (CRS317) to sfpplus10 on SW#2 (CRS317) on Layer3 with wirespeed?
 
RcRaCk2k
Member Candidate
Member Candidate
Posts: 151
Joined: Mon May 07, 2012 10:40 pm

Re: vxlan performance?

Tue Mar 04, 2025 10:33 pm

HOLD ON GUYS... BIG NEWS ON THAT !!!!

Seeing VXLAN on Switch Layer-3 HW Offload Monitor on CRS326-24S+2Q+ !

After 5 years, the Mikrotik guys implemented a feature, where the hardware was able to provide VxLAN but only the software part was missing. Don't know if i should be happy or not ^^

Available since 7.18
Source: https://help.mikrotik.com/docs/spaces/R ... ta%20plane

Screenshot 2025-03-04 214053.jpg
Screenshot 2025-03-04 213818.jpg
You do not have the required permissions to view the files attached to this post.
 
syadnom
Forum Veteran
Forum Veteran
Topic Author
Posts: 824
Joined: Thu Jan 27, 2011 7:29 am

Re: vxlan performance?

Wed Mar 05, 2025 12:00 am

except most of the stuff I need isn't there haha. IPv6 VTEPs not supported :/
 
RcRaCk2k
Member Candidate
Member Candidate
Posts: 151
Joined: Mon May 07, 2012 10:40 pm

Re: vxlan performance?

Wed Mar 05, 2025 9:03 am

Yeah, that hits for me also. VTEP on VLAN not supported. My network is 100% vlan-only :-)

I hope that this is supported by the switch-chip. If not, that will be the worst issue i ever have heard.

How hard could it be to analyze the ether-type for 0x8100 [Tag protocol identifier (TPID)] and than skip the next few bytes to get to the UDP-Header.
 
syadnom
Forum Veteran
Forum Veteran
Topic Author
Posts: 824
Joined: Thu Jan 27, 2011 7:29 am

Re: vxlan performance?

Wed Mar 05, 2025 4:53 pm

it'd be heartbreaking.

I need specifically, VTEP on IPv6, and VLAN on the interface.
 
zpa0
just joined
Posts: 7
Joined: Tue Jan 17, 2012 11:02 pm
Location: Sofia, Bulgaria

Re: vxlan performance?

Tue Mar 25, 2025 10:34 am

I've tried offloading on CRS326-24S+2Q+, but Hw: Offloaded status is not active on VxLAN interface.

Probably I'm missing some requirements for offloading or it is still not fully implemented?
 
biki73
just joined
Posts: 6
Joined: Fri Jun 23, 2023 11:06 pm

Re: vxlan performance?

Tue Mar 25, 2025 11:10 am

I've tried offloading on CRS326-24S+2Q+, but Hw: Offloaded status is not active on VxLAN interface.

Probably I'm missing some requirements for offloading or it is still not fully implemented?
try to enable 'don't fragment'