Pcc with pppoe behind brigde really slow

RouterOS General
1

I’m having an issue with implementing pcc in my mikrotik RB4011iGS+5HacQ2HnD i have followed multiple tutorials and forums,in my setup i have 2 pppoe links for the internet one 1gb link and one 300mb link,all my clients are behind a bridge trough pppoe,so for the mangle rules too work i have to go in /bridge/setting and enable ip firewall for pppoe,the problem is that when i enable it and test the connection on the clients they get instead of the default 10mb/s connection some 50kb/s.Ive tried to change mangle and firewall rules and checked the cpu load,but it doesn’t exceed 10% worst case,so i don’t really know what to do,it appears to be worse on windows machines(i use them for testing in the network) than on linux(my personal pc),but they are all behind the same modem that uses pppoe in the bridge.If someone could shed some light in what could i do to get a decent troughput i would be glad.

I have this mangle rules rigth now:

add action=mark-connection chain=prerouting comment=\
    "marca conn 300mb_in no modem" connection-mark=no-mark in-interface=\
    Link_300mb_Alg new-connection-mark=Link_300mb passthrough=yes
add action=mark-connection chain=prerouting comment=\
    "Marca conn 1gb_in no modem" connection-mark=no-mark in-interface=\
    Link_1Gb_Alg new-connection-mark=Link_1gb passthrough=yes
add action=mark-connection chain=prerouting comment=\
    "marca conn 1gb da bridge pra fora com pcc" connection-mark=no-mark \
    connection-state=new dst-address-type=!local in-bridge-port-list=Lan2 \
    in-interface=bridge new-connection-mark=Link_1gb passthrough=yes \
    per-connection-classifier=src-address:2/0 routing-mark=!link_1gb
add action=mark-connection chain=prerouting comment=\
    "marca conn 300mb da bridge pra fora com pcc" connection-mark=no-mark \
    connection-state=new dst-address-type=!local in-bridge-port-list=Lan2 \
    in-interface=bridge new-connection-mark=Link_300mb passthrough=yes \
    per-connection-classifier=src-address:2/1 routing-mark=!link_300mb
add action=mark-connection chain=prerouting comment=\
    "marca conn 1gb da bridge pra fora com pcc" connection-mark=no-mark \
    connection-state=new disabled=yes dst-address-type=!local \
    in-bridge-port-list=Lan2 in-interface=bridge new-connection-mark=Link_1gb \
    passthrough=yes per-connection-classifier=src-address:5/3
add action=mark-connection chain=prerouting comment=\
    "marca conn 1gb da bridge pra fora com pcc" connection-mark=no-mark \
    connection-state=new disabled=yes dst-address-type=!local \
    in-bridge-port-list=Lan2 in-interface=bridge new-connection-mark=Link_1gb \
    passthrough=yes per-connection-classifier=src-address:5/4
add action=mark-connection chain=prerouting comment=\
    "marca conn 1gb da bridge pra fora com pcc" connection-mark=no-mark \
    connection-state=new disabled=yes dst-address-type=!local \
    in-bridge-port-list=Lan2 in-interface=bridge new-connection-mark=Link_1gb \
    passthrough=yes per-connection-classifier=src-address:5/2
add action=mark-routing chain=prerouting comment=\
    "roteia conn 1gb_in marcada da bridge para o modem" connection-mark=\
    Link_1gb connection-state=new disabled=yes in-bridge-port-list=Lan2 \
    in-interface=bridge new-routing-mark=link_1gb passthrough=no
add action=mark-routing chain=prerouting comment=\
    "roteia conn 300mb_in marcada da bridge pro modem" connection-mark=\
    Link_300mb connection-state=new disabled=yes in-bridge-port-list=Lan2 \
    in-interface=bridge new-routing-mark=link_300mb passthrough=no \
    routing-mark=!link_300mb
add action=mark-routing chain=prerouting comment=\
    "marca conn 1gb saindo do modem" connection-mark=Link_1gb \
    connection-state=new new-routing-mark=link_1gb passthrough=no \
    routing-mark=!link_1gb
add action=mark-routing chain=prerouting comment=\
    "marca con 300mb saindo do modem" connection-mark=Link_300mb \
    connection-state=new new-routing-mark=link_300mb passthrough=no \
    routing-mark=!link_300mb
2

I’ve looked more into it but still don’t know,if i am behind pppoe pcc gets really slows from 100mb to 10mbs at best,but if i am using the management lan via vpn or cable connected it works fine,so i think that there is a problem in balancing connection trough the bridge.Since no one has said anything so far and its been almost 400 views maybe its just that this isn’t a viable way to load balance,if theres another way to load balance pppoe connection in a bridge to maximize even link usage it would be great to hear.

3

I would like to comment but as soon as you said filter rules on the bridge, I tend to lose interest.
If you meant IP firewall rules ( the normal ones) then I can look at the config…

Just looking pcc, however there are some things I dont understand.
You use 2.0, 2.1 and then some weird 5.3 setting out of nowhere???
Why are using the ! symbol for route-marks???
Why you use same name for connection mark and route-mark DONT! too confusing.

Try something a bit more conventional/normal.
First four rules for WAN1 and WAN2 ( traffic that comes in via WANx, will go out WANx - mostly for router service purposes or incoming lan server traffic )

add action=mark-connection chain=prerouting connection-mark=no-mark
in-interface=Link_300mb_Alg new-connection-mark=Link_300mb passthrough=yes
add action=mark-connection chain=prerouting connection-mark=no-mark
in-interface=Link_1Gb_Alg new-connection-mark=Link_1gb passthrough=yes
add action=mark-routing chain=output connection-mark= Link_300mb
new-routing-mark=use_300mb passthrough=no
add action=mark-routing chain=output connection-mark= Ling_1gb
new-routing-mark=use_1gb passthrough=no

The next 3 RULES, attempt to PCC network by realizing throughput difference 300 vs 1000 so emulating a 3:1 ratio ( 33%).

add action=mark-connection chain=prerouting connection-mark=no-mark in-interface=bridge
new dst-address-type=!local new-connection-mark=Link-1g-conn passthrough=yes
per-connection-classifier=src-address:3/2
add action=mark-connection chain=prerouting connection-mark=no-mark in-interface=bridge
new dst-address-type=!local new-connection-mark**=Link-300-conn** passthrough=yes
per-connection-classifier=src-address:3/1
add action=mark-connection chain=prerouting connection-mark=no-mark in-interface=bridge
new dst-address-type=!local new-connection-mark=Link-1g-conn passthrough=yes
per-connection-classifier=src-address:3/0

The last 2 provide the route marks for the LAN traffic.
add action=mark-connection chain=prerouting connection-mark=Link-300-conn
new-routing-mark=use_300mb passthrough=yes
add action=mark-connection chain=prerouting connection-mark=Link-1g-conn
new-routing-mark=use_1g passthrough=yes

Note1: you will need tables to match
add fib name=use_300mb
add fib name=use_1g

Note2: ensure in your fastrack rule you add mark=no-mark

4

Let me leave this here.

PCC 33%/66% connection distribution

add action=mark-connection chain=prerouting connection-mark=no-mark in-interface=bridge connection-state=new dst-address-type=!local new-connection-mark=Link-300-conn passthrough=yes per-connection-classifier=src-address:3/0

add action=mark-connection chain=prerouting connection-mark=no-mark in-interface=bridge connection-state=new dst-address-type=!local new-connection-mark=Link-1g-conn passthrough=yes

You are using only the source addres and can lead to a uneven distribution if you don’t have not many clients on your network.

5

Not sure what you are saying other than, the Op should use source and destination???

6

Yes i meant IP firewall rules,they’re pretty much the default ones with fast track disabled and some block to special servers on lan.
The pcc part was also my bad,i was using 2.0 & 2.1 to test(makes it easier to see distribution is a small network),but the distribution i will apply will be a 4/1 hence the 5.0 and so on,with 4 to the 1gb link and 1 to the 300mb,the reason i used the ! in routing marks was to make sure that the rule wouldn’t mark it multiple times(pretty useless since i match connection state but i was desperate).
The problems isn’t with mangling in itself,because when i use mangle rules the traffic gets distributed but from around 95mb it drops to 10mbs(using speedtest),some sites cant even load.
And that just happens if i enable bridge to use ip firewall and enable mangle rules.
Will try enabling fast-track and enabling mangle,maybe it works.
Also changed the names for clarity sake,thanks for the suggestions.

7

These are my firewall rules,if i enable for bridge to use ip firewall rules the download speed gets bumped from 95mbs to around 10mbs.
From what ive seen if you’re bellow 25 firewall rules it shouldn’t be that bad of a drop.

add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment=\
    "defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=accept chain=input comment="VPN Ipsec rule" protocol=ipsec-esp
add action=accept chain=input comment="VPN Ports" port=1701,500,4500 protocol=\
    udp
# VPN_router_access not ready
add action=accept chain=input comment=VPN_router_access_rule in-interface=\
    VPN_router_access
# VPN_Maori_router_access not ready
add action=accept chain=input comment=VPN_router_access_rule in-interface=\
    VPN_Maori_router_access
add action=drop chain=input comment="defconf: drop all not coming from LAN" \
    in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" \
    ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" \
    ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" \
    connection-mark="" connection-state=established,related
add action=accept chain=forward comment=\
    "defconf: accept established,related, untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=\
    invalid
add action=drop chain=forward comment="defconf: drop all from WAN not DSTNATed" \
    connection-nat-state=!dstnat connection-state=new in-interface-list=WAN
add action=drop chain=forward comment="block access olt" dst-address=\
    172.27.27.7 src-address=100.64.0.0/24
add action=drop chain=forward comment="block access nas" dst-address=\
    172.27.27.17 src-address=100.64.0.0/24
add action=drop chain=forward comment="block access mkauth" dst-address=\
    172.27.27.27 src-address=100.64.0.0/24
add action=drop chain=forward comment="block access genieacs-ui" dst-address=\
    172.27.27.37 dst-port=3000 protocol=tcp src-address=100.64.0.0/2
8

I should fix your rules, they sucketh!! also on the fastrack no need to disable just add connection-mark=no-mark

You go to trouble to allow all full access to router on input chain, not recommended… should only be the admin.
All users only need access typically to DNS services and sometimes NTP for the smart switches, APs.

_/ip firewall address-list { using static dhcp leases or provided vpn addresses }
add address=AdminIP-1 list=TRUSTED comment=“admin desktop”
add address=AdminIP-2 list=TRUSTED comment=“admin laptop”
add address=AdminIP-3 list=TRUSTED comment=“admin smartphone”
add address=AdminIP-4 list=TRUSTED comment=“admin remote wireguard”
add address=AdminIP-5 list=TRUSTED comment=“admin other vpn”
/ip firewall filter
{Input Chain}
add action=accept chain=input comment=“defconf: accept established,related,untracked” connection-state=established,related,untracked
add action=drop chain=input comment=“defconf: drop invalid” connection-state=invalid
add action=accept chain=input comment=“defconf: accept ICMP” protocol=icmp
add action=accept chain=input comment=“defconf: accept to local loopback (for CAPsMAN)” dst-address=127.0.0.1
add action=accept chain=input comment=“VPN Ipsec rule” protocol=ipsec-esp
add action=accept chain=input comment=“VPN” port=1701,500,4500 protocol=udp
add action=accept chain=input in-interface-list=TRUSTED
add action=accept chain=input comment=“Allow LAN DNS queries-UDP”
dst-port=53 in-interface-list=LAN protocol=udp
add action=accept chain=input comment=“Allow LAN DNS queries - TCP”
dst-port=53 in-interface-list=LAN protocol=tcp
add action=drop chain=input comment=“drop all else”
{forward chain}
add action=accept chain=forward comment=“defconf: accept in ipsec policy” ipsec-policy=in,ipsec
add action=accept chain=forward comment=“defconf: accept out ipsec policy” ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment=“defconf: fasttrack” connection-state=established,related connection-mark=no-mark
add action=accept chain=forward comment=“defconf: accept established,related, untracked” connection-state=established,related,untracked
add action=drop chain=forward comment=“defconf: drop invalid” connection-state=invalid
add action=accept chain=forward comment=“allow internet traffic” in-interface-list=LAN out-interface-list=WAN
add action=accept chain=forward comment=“port forwarding” connection-nat-state=dstnat

add action=drop chain=forward comment=“drop all else”_

NOTE: If you have additional traffic you need to pass, add it where the ****** are located.
Common needs are a shared printer or the admin needs access to the entire LAN.

9

Thanks for the new rules,but since clients use pppoe they cant really access mikrotik router anyway,implemented some parts of it because better security is always a good thing.But it didn’t really affect my problem with slow download speed when using bridge and ip firewall?,i didn’t think it would because the rules don’t do anything to freaky,but i really wanted to make this work so i could load balance my clients since i have one link just as a fail over and the waste doesn’t really make me happy,as i thought the problems don’t seem to be in the mangle rules.But this set of firewall rules didn’t do it either?maybe i should make the connection coming from the bridge(pppoe) avoid all the other firewall rules or something?