Community discussions

MikroTik App
 
User avatar
herger
newbie
Topic Author
Posts: 49
Joined: Tue Aug 18, 2020 2:48 pm

CCR2004-1G-2XS-PCIe and FreeBSD

Wed Jul 27, 2022 5:38 pm

Hi,

I have a really easy question: Has anyone be able to get the CCR2004-1G-2XS-PCIe running on FreeBSD? If so, how?

I'll give a bit of background here: I've had various problems with the card and recent ROS versions. I'm always using the default config of the ROS version I'm testing and do a full config reset after every update (to be sure I've not messed up the config). Running FreeBSD 13.1 the card is detected and 4 alc[0-3] interfaces are created.
On 7.2.3 setting one of the interfaces UP, didn't have any effect and made the ifconfig command hang indefinitely.
On 7.3.1 setting the interface UP does not hang anymore, I see the interface get into state RUNNING on the ROS side but i only see traffic originating from the host being forwarded to the SFP+ interface, i don't see any returning traffic (neither on ROS, nor FreeBSD)
On 7.4 I don't see a difference to 7.3.1.

It makes no difference if I configure the pass through feature or not; no traffic flowing towards the host. Also the interface speed is wrongly displayed on FreeBSD (1G instead of 10G).
Generally I'd agree on this is a problem with me being stupid and unable to configure the card properly, BUT when i boot a recent Linux (5.15.0) instead of the FreeBSD while keeping the ROS config untouched, things start to work without any problem.

I've been into discussion with Mikrotik Support about this, but they stooped replying about a month ago (SUP-82886).

As I'm seeing this a the moment, there is a problem with the alc(4) drivers, more precisely with this addition from last year.

So if *YOU* have this running, please lend me a hand and point me to why I'm stupid.
If some Mikrotik staff is reading, it would be nice to at least acknowledge there is a problem (or at least someone is reading the ticket :) so I can stop running in circles.

Thanks!
 
User avatar
herger
newbie
Topic Author
Posts: 49
Joined: Tue Aug 18, 2020 2:48 pm

Re: CCR2004-1G-2XS-PCIe and FreeBSD

Sun Jul 31, 2022 11:58 am

Really, no one? Not even the cheeky cat or the jester donkey?
Show some love for the world's best server OS (;
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11967
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: CCR2004-1G-2XS-PCIe and FreeBSD

Sun Jul 31, 2022 3:55 pm

Not having CCR2004-1G-2XS-PCIe nor runing FreeBSD, I can't help...
 
User avatar
herger
newbie
Topic Author
Posts: 49
Joined: Tue Aug 18, 2020 2:48 pm

Re: CCR2004-1G-2XS-PCIe and FreeBSD

Thu Aug 04, 2022 1:02 pm

So it seams I'm the only one trying this setup?
Why would Mikrotik advertise something that's clearly not working?

I still have hope, but I'm running out of patience *sigh*
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11967
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: CCR2004-1G-2XS-PCIe and FreeBSD

Thu Aug 04, 2022 2:28 pm

 
User avatar
woland
Member Candidate
Member Candidate
Posts: 258
Joined: Mon Aug 16, 2021 4:49 pm

Re: CCR2004-1G-2XS-PCIe and FreeBSD

Thu Aug 04, 2022 2:52 pm

Hi!

I have got mine delivered last week and I´m planning to put it into my Opnsense box, probably this week. This has FreeBSD 13.1, I will report about my progress, when I get so far.
So no you are not alone. :)

If it won´t work with BSD, I will use it with Proxmox and virtualize the Opnsense running on it.

I did not find much info on BSD support, but there is a howto for Proxmox in German:
https://blog.mikrotik-store.eu/2022/05/ ... ox-system/

Regards
W
 
User avatar
herger
newbie
Topic Author
Posts: 49
Joined: Tue Aug 18, 2020 2:48 pm

Re: CCR2004-1G-2XS-PCIe and FreeBSD

Thu Aug 04, 2022 5:00 pm

Where do you see any instructions regarding FreeBSD in there?
 
User avatar
woland
Member Candidate
Member Candidate
Posts: 258
Joined: Mon Aug 16, 2021 4:49 pm

Re: CCR2004-1G-2XS-PCIe and FreeBSD

Thu Aug 04, 2022 5:03 pm

Where do you see any instructions regarding FreeBSD in there?
Nowhere, but Proxmox will be my solution to still run BSD if it can´t run on bare metal. Did you read my post?
I did not find much info on BSD support, but there is a howto for Proxmox in German:
W
 
joegoldman
Forum Veteran
Forum Veteran
Posts: 766
Joined: Mon May 27, 2013 2:05 am

Re: CCR2004-1G-2XS-PCIe and FreeBSD

Fri Aug 05, 2022 4:46 am

unfortunately the BSD kernel is often forgotten / lagged behind the linux kernel so it wouldn't surprise me if its close but not working.

Perhaps email support directly to try get more specific help. I used to use FreeBSD everywhere and got over the management overhead of generally older binaries or different in just enough way to be annoying compared to 90% of other servers out there, and with dockers etc BSD jail's aren't as attractive anymore. That's just my opinion of course.
 
User avatar
woland
Member Candidate
Member Candidate
Posts: 258
Joined: Mon Aug 16, 2021 4:49 pm

Re: CCR2004-1G-2XS-PCIe and FreeBSD

Fri Aug 05, 2022 9:44 am

Largely agree, however if you want to have an open source firewall or NAS distribution, the Linux variants are nowhere near the BSD ones (the Linux ones are getting better and better over time).
I have both my NAS and my Firewalls on BSD and the getting compatible new (old) hardware part was always a pain in the a... But once you get them working, I always had rock solid and highly performant systems.
My last chance to get the MT card running is to go virtual, it has some benefits.
Anyway my chance to try it this week is gone, probably next week then.
 
User avatar
herger
newbie
Topic Author
Posts: 49
Joined: Tue Aug 18, 2020 2:48 pm

Re: CCR2004-1G-2XS-PCIe and FreeBSD

Mon Aug 08, 2022 1:56 pm

Tested with 7.4.1, same story.
  • link is detected correctly
  • interface speed is incorrectly reported (1000baseT instead of 10000baseSR)
  • outgoing traffic passes the interface and is correctly passed through to sfp28-1
  • incoming traffic not visible, neither on sfp28-1, ether-pcie1 nor alc0 on host
 
r00t
Long time Member
Long time Member
Posts: 672
Joined: Tue Nov 28, 2017 2:14 am

Re: CCR2004-1G-2XS-PCIe and FreeBSD

Tue Aug 09, 2022 6:24 pm

I considered getting one for SSD NAS build, but no FreeBSD/TrueNAS support just kills that use case.
It's a shame because it would be interesting hardware to test and play with...
 
User avatar
herger
newbie
Topic Author
Posts: 49
Joined: Tue Aug 18, 2020 2:48 pm

Re: CCR2004-1G-2XS-PCIe and FreeBSD

Tue Aug 09, 2022 7:14 pm

Well, Mikrotik advertises FreeBSD support.... whatever that means ¯\_(ツ)_/¯
I don't see this working as of now.
 
User avatar
woland
Member Candidate
Member Candidate
Posts: 258
Joined: Mon Aug 16, 2021 4:49 pm

Re: CCR2004-1G-2XS-PCIe and FreeBSD

Tue Aug 09, 2022 7:30 pm

Hi!
I managed to do a short test yesterday, and not surprisingly I can confirm: on Opnsense 22.7 (OpenBSD 13.1) the 4 alcX interfaces show up, link is detected at 1G. I was using an SFP+ 10G direct attach cable. Had no time for more...
I did not check which ROS did it have. I will check next time with ROS 7.5beta, but I have not much hope.
I will probably open a ticket with MT as well.
Too bad, but I hope they will fix this, or maybe provide us with some advice?

W
 
mada3k
Long time Member
Long time Member
Posts: 682
Joined: Mon Jul 13, 2015 10:53 am
Location: Sweden

Re: CCR2004-1G-2XS-PCIe and FreeBSD

Tue Aug 09, 2022 8:04 pm

FreeBSD is indeed a wonderfull OS, but it's most likely up to the kernel developers to implement the Mikrotik-specific stuff in the alc driver, that have been done to the Linux kernel one.
 
User avatar
herger
newbie
Topic Author
Posts: 49
Joined: Tue Aug 18, 2020 2:48 pm

Re: CCR2004-1G-2XS-PCIe and FreeBSD

Tue Aug 09, 2022 8:36 pm

Do not quote preceding post - use "Post Reply" instead.
There is a change to the FreeBSD alc(4) driver from Gatis Peisenieks <gatis@mikrotik.com> here https://cgit.freebsd.org/src/commit/sys ... ac283962f4. Its the same Gatis implementing all the Linux driver side here https://git.kernel.org/pub/scm/linux/ke ... l1c_main.c, only that there are _a lot_ more commits to the linux driver as to the FreeBSD one. Also both sources show the last commit in May last year, so Mikrotik might believe development is finished.

Also, how should someone from the FreeBSD team know all the quirks of the Mikrotik NIC without the ROS side being open-sourced?
 
User avatar
woland
Member Candidate
Member Candidate
Posts: 258
Joined: Mon Aug 16, 2021 4:49 pm

Re: CCR2004-1G-2XS-PCIe and FreeBSD

Wed Aug 10, 2022 10:48 am

I have searched for the changes regarding the alc driver, but there have been none since the commit of Konstantin Belousov. Does not seem, like too much going on.
The commit was already introduced into FreeBSD 12.3.

https://www.bsdforen.de/threads/freebsd ... nen.36360/

Also that guy Konstantin does not seem to work for MT at least according to his LinkedIn profile.
 
User avatar
herger
newbie
Topic Author
Posts: 49
Joined: Tue Aug 18, 2020 2:48 pm

Re: CCR2004-1G-2XS-PCIe and FreeBSD

Wed Aug 10, 2022 11:33 am

Also that guy Konstantin does not seem to work for MT at least according to his LinkedIn profile.
Yea, he is the one that handled this bug https://bugs.freebsd.org/bugzilla/show_ ... ?id=256000. Has nothing to do with Mikrotik.
 
User avatar
woland
Member Candidate
Member Candidate
Posts: 258
Joined: Mon Aug 16, 2021 4:49 pm

Re: CCR2004-1G-2XS-PCIe and FreeBSD

Mon Aug 15, 2022 8:20 pm

Hi, I got to test my card again, albeit for a short period of time only. ROS 7.5b8, FreeBSD 13.1.
SFP+ 10G Direct attach between an CSS610 and sfp1 on the card.
It sets the state of alc0 correctly up and down. If I send something into the port while set into the passthrough mode I see the traffic via torch from RoS.
The same is true if I don´t use passthrough, but just bridge the real interface and the alc0 together.
However I did not get a lease on the BSD side. (I checked, I can get a lease on the ROS bridge.)
I did not get to do more, but it again does not look too well. Packets seem to arrive from BSD into the card, but nothing comes back into the alc0
I will do more testing.

W
 
User avatar
woland
Member Candidate
Member Candidate
Posts: 258
Joined: Mon Aug 16, 2021 4:49 pm

Re: CCR2004-1G-2XS-PCIe and FreeBSD

Sun Aug 21, 2022 10:49 pm

Hi,

I opened a ticket at MT with some additional infos, I hope to get some help or at least some statement about BSD support. Let's see.
W
 
User avatar
herger
newbie
Topic Author
Posts: 49
Joined: Tue Aug 18, 2020 2:48 pm

Re: CCR2004-1G-2XS-PCIe and FreeBSD

Tue Aug 23, 2022 10:59 am

Another discovery i made (still on 7.4.1): when configuring one of the pcie-ether interfaces as non-passthrough, non-bridged, trying to ping the hosts ipv6 link-local address from a ros console gives me '105 no buffer space available'. this changes when i disable auto negotiation on the port, after which i simply get timeouts and address unreachables.
By now i highly suspect the FreeBSD driver is missing all the card specific implementations, like 'atl1c: show correct link speed on Mikrotik 10/25G NIC https://git.kernel.org/pub/scm/linux/ke ... 5de922d9ee' and the likes. Also, when having a ping running on the host and then disabling the interface on the ros side, the freebsd host hang in an unresponsive state until rebooted ):

very disappointed with the FreeBSD 'driver' work from Mikrotik.
 
User avatar
herger
newbie
Topic Author
Posts: 49
Joined: Tue Aug 18, 2020 2:48 pm

Re: CCR2004-1G-2XS-PCIe and FreeBSD

Fri Sep 02, 2022 5:54 pm

Finally got a reply on SUP-82886:
We confirm that there is a software problem with packet transmit from the CCR2004-PCIe to the FreeBSD host, it works only in direction from the FreeBSD host to the CCR2004-PCIe.
And we are looking forward to fix it in upcoming RouterOS releases.
So it seems to be a problem in ROS, not the FreeBSD driver and they are going to fix it \o/
 
User avatar
woland
Member Candidate
Member Candidate
Posts: 258
Joined: Mon Aug 16, 2021 4:49 pm

Re: CCR2004-1G-2XS-PCIe and FreeBSD

Mon Sep 05, 2022 11:24 am

Do not quote preceding post - use "Post Reply" instead.
Hi! Thanks for the update! So at least there is _some_ hope that in the future something might change.
My ticket SUP-90362 (with Supout and detailed Infos) was successfully ignored. :(

W
 
User avatar
woland
Member Candidate
Member Candidate
Posts: 258
Joined: Mon Aug 16, 2021 4:49 pm

Re: CCR2004-1G-2XS-PCIe and FreeBSD

Sun Oct 23, 2022 11:21 pm

Hi,
if someone is still interested: after a month I again tried to get this strange card running on Opnsense.
This time in another machine with a fresh install of Opnsense 22.7.6 and ROS 7.6 on the card itself.
No success, but one new piece of the puzzle: the install procedure of the Opnsense hung a lot's of times, until I found out, that I had to disable all ether-pciX ports on the ROS side.
Than I was able to get the link up, but still no packets are transmitted.
I went to a Proxmox based solution instead, which works perfectly.

It'd be great to have some info from MT about BSD support!

W
 
User avatar
herger
newbie
Topic Author
Posts: 49
Joined: Tue Aug 18, 2020 2:48 pm

Re: CCR2004-1G-2XS-PCIe and FreeBSD

Mon Oct 24, 2022 9:59 am

It took 3 months for them to confirm the problem in my ticket, I don't expect anything close to a fix this year /:
The ticket hasn't received any updates, neither has the driver https://cgit.freebsd.org/src/log/sys/dev/alc?h=main in CURRENT.
I try to keep my sarcasm out of this forum (there is enough from lots of people already) but by now I'm quite used to have to wait a year plus for the software to catch up with what was promised for a given hardware. Fingers crossed this will not be the case here.

best
 
User avatar
woland
Member Candidate
Member Candidate
Posts: 258
Joined: Mon Aug 16, 2021 4:49 pm

Re: CCR2004-1G-2XS-PCIe and FreeBSD

Mon Oct 24, 2022 10:19 am

Hi herger!
Thanks for the info! MT did not respond to my ticket either.
Let´s just say: there is room for some improvement ont the SW side of MT. :?
W
 
User avatar
herger
newbie
Topic Author
Posts: 49
Joined: Tue Aug 18, 2020 2:48 pm

Re: CCR2004-1G-2XS-PCIe and FreeBSD

Thu Jan 12, 2023 4:53 pm

Did a quick test with the 7.7 stable, release today, exactly the same behavior.

I would really appreciate if i could use the advertised features of this card a year after buying it ):
 
User avatar
woland
Member Candidate
Member Candidate
Posts: 258
Joined: Mon Aug 16, 2021 4:49 pm

Re: CCR2004-1G-2XS-PCIe and FreeBSD

Fri Jan 13, 2023 10:15 am

Same situation here as well. Please don´t ignore BSD compatibility, Mikrotik!
Thanks for the update @herger!
 
rdmweb
just joined
Posts: 3
Joined: Sun Sep 21, 2014 8:34 pm

Re: CCR2004-1G-2XS-PCIe and FreeBSD

Tue Feb 14, 2023 9:28 pm

Hi there,

I was thinking of buying one of these CCR2004-1G-2XS-PCIe to use in my home FreeBSD server. On paper the setup fits wonderfully with my environment but the FreeBSD issues described here are a big let down.
Is there any updates from Mikrotik about when they should be fixed?
 
User avatar
woland
Member Candidate
Member Candidate
Posts: 258
Joined: Mon Aug 16, 2021 4:49 pm

Re: CCR2004-1G-2XS-PCIe and FreeBSD

Wed Feb 15, 2023 10:28 am

I am not aware of any update/plans. This card is not compatible with BSD unforunately.
BR

Who is online

Users browsing this forum: Google [Bot], rplant and 78 guests