For a main link I have the following setup: Main router with internet connection is a CCR-1036, from this one I run fiber as VLAN trunk carrying numerous VLANS to a CRS-125 on the rooftop.
CRS: The switch chip in the CRS is programmed to connect VLAN 109 for the connection in question directly to ether9, i.e no CPU involved in CRS. A pair of Airfiber AF-24 is making a 1 km link to far side, where a CCR1016 does further routing/distribution to sublinks.
AF-24 runs at highest modulation, theoretical speed of 775Mbps in each direction.
Ip address 192.168.5.1/29 is assigned to the VLAN109 in the CCR1036, and 192.168.5.2/29 to ether11 on the CCR1016. Full transparency in both directions, latency 0-1ms.
CCR1016: When I do a speedtest, I get a consistent 680Mbps TCP on “uplink” from the far side CCR1016 to the main router CCR1036, this is far more than I dared to hope for.
However, in the other direction (“downlink”), which is where I need the full speed, I only get a disappointing 200Mbps..
These results are the same no matter which one of the CCRs are BW test server and client.
Monitoring the Interfaces page on each side, I can see that for traffic from the CCR1016, to CCR1036, Fastpath (FP Tx/Rx) is employed on both sides, so far so good.

Upload speed:
Testing from the CCR1036 to the CCR1016, Fastpath does not appear to be employed for traffic leaving the CCR1036, however upon arrival at the CCR1036 it is..

Download speed:
That means that traffic is travelling on Slowpath from the CCR1036, and somewhere in between Fastpath is employed.
I assume this is the reason for the poor performance?

How can I possibly put Fastpath to work also from the CCR1036?
It is running a lot of firewalling and queuing stuff, so disabling the Conntracker is no alternative.

I hope someone can help me out on this one

Okay a few things here:

The first thing to consider is that the BTest program chews into CPU cycles on each device - I don't know if it's multi-threaded or not, but if it's not, then it's only going to be able to do what one core's max capacity is.

The next thing to consider is that since traffic is coming to/from the CPU, so if you don't have fasttrack-connection rules in your input/output chains, then such traffic won't get fasttracked.

In general, it's better to speed test with attached dedicated test point devices (PCs running IPerf are better than using BTest IMO) because this eliminates the router's own CPU capacity issues from the speed testing equation. When testing to/from the router directly with BTest, you might get slower results because some of the CPU's muscle is being spent in he effort of generating / verifying the packets themselves as opposed to just purely forwarding them.

Thank you Zerobyte,

Well I guess I cannot disagree with you on your views on BW test methods.
But I believe CPU consumption should not be the real issue here, as when traffic is generated by the less powerful CCR1016, I get a speed close to the theoretical maximum of the AFs.

I guess I need some enlightenment about fasttrack...how would such rules look like?
I have inserted none of these on the CCR1016 and it still uses Fastpath.

I'm still a bit fuzzy on the difference between fastpath and fasttrack myself.

Fasttrack is when a rule in the filter table moves a flow of packets into a fastpath-forwarding state.
Fastpath happens automatically when certain conditions are true (consult the Wiki)

It could actually be the receive side slowing things down as the CCR1016 is verifying the checksums / etc....

I tried to add a rule to the output chain, marking traffic on the actual vlan for Fasttracking.
Counters are updated, but still no joy.

CPU still at a few % on CCR1016, not even one single core running full.

So anyone?

Just trying to think of silly things - have you verified that the modulation rate in the upstream direction on your AF link is as good as the modulation rate of the downstream direction?
Since AF uses two channels for full duplex operation, there could be more interference/noise/etc making that direction's link be worse than the other.

Both directions stick at highest modulation, 6X (64QAM MIMO) with estimated throughput of 775 Mbps.
Downlink (towards my CCR1016) is even a few dBs better than the opposite, 63/61 vs 65/66 dBm.

So I still suspect this to be about Fastpath vs Slowpath.

For info, I see the same thing on a second licensed 18GHz link sitting in parallell, 420 vs 200 Mbps. But I did not want to involve this in my initial post, in order not to confuse.

Thinking loud.... ..Am I right when I assume that the link can be considered as one homogenous, transparent L2 connection...straight from the vlan in the CCR1036 to the Ether11 on the CCR1016? A kind of "virtual cable"?
And then, what can possibly slow it down in one direction, but not in the other?

Is the answer related to the fact that on the CCR1016 the data are inserted into a physical ethernet interface, whilst on the CCR1036 it is inserted into a vlan?

In order to locate the problem I'd start with doing speed test to/from the CRS125 from both sides. Either by configuring the CRS125 so that it can be used for speedtest, or (better) bring another box up on the roof and connect it to the CRS125 for performing the speed test trough the CRS125.
Another test could be to bring the CCR1016 to the roof and connect to CRS125 to eliminate air fiber issues.

That means that traffic is travelling on Slowpath from the CCR1036, and somewhere in between Fastpath is employed.
I assume this is the reason for the poor performance?

No. both CCR's CPUs won't miss a sweat for this task. Neither fatspath nor Fastrack are an issue for them for the speeds AFs are going to pass.

In order to locate the problem I'd start with doing speed test to/from the CRS125 from both sides. Either by configuring the CRS125 so that it can be used for speedtest, or (better) bring another box up on the roof and connect it to the CRS125 for performing the speed test trough the CRS125.
Another test could be to bring the CCR1016 to the roof and connect to CRS125 to eliminate air fiber issues.

Don't... CRS is a switch, it has a CPU for your convenience, but not suitable for these tests.

Both directions stick at highest modulation, 6X (64QAM MIMO) with estimated throughput of 775 Mbps.
Downlink (towards my CCR1016) is even a few dBs better than the opposite, 63/61 vs 65/66 dBm.

I wouldn't trust AirFibers main too much. Have you tested directly with PCs and iperf directly connected to the AFs?

If this proves the link throughput is symmetrical: AirFibers are known for causing ethernet problems, noise or signal reflection gets injected back on the ethernet port causing FCS and/or Duplex problems.

Can you post
For the ports connected to the AFs?

Look for (blue) FCS errors on the logs, are there any?

I gave the AFs a shot and found that the link was easily able to handle 400+ Mbps. So maybe I was fooled by something regarding the BW test in Winbox? Any idea?
Either way I am now running AFs for my main link

ifperf is nothing that I know of, haven't tested.

Here is the output of the interface stats:

CRS (Near end): CCR1016 (Far end): A little cut and paste was necessary to get a sane format out of it..haven't found a way to list one specific interface.
No colors in the Winbox terminal...

Interface stats look fine.

regarding iperf: Download proper version for each end at https://iperf.fr/iperf-download.php.

start in on remote side with -s switch (server) leave it open.

Open an instance on your local end, with -d -c ip.of.remote.side

toy with the options a --help outputs, is really easy to use.

Check https://code.google.com/archive/p/xjperf/downloads too. (GUI for jperf)