Peculiar PCC setup/issue and more

Pandinvs · Sun May 19, 2024 11:15 pm

Hello Guys,

I recently acquired a new Hap Ax^3 (C53UiG+5HPaxD2HPaxD) router. By this occasion I tried to set it up PCC LB+FO config, but the settings for one of my ISPs is quite strange (further referred as O2). In this case, the connection is ADSL2: the phone lines are connected to a bridge and the bridge to my router where I have to set a VLAN and PPPoE. How this setup works, I don't understand at all. The DHCP client is set, but it seems never get any configuration. I suspect that may get somehow via the PPPoE interface, but that cannot be configured as a DHCP client. It is visible in the IP address list as a dynamic address.
When I try to set the static routes, ping test is always failing, so my PCC connection not gonna work. As usual, my provider's helpdesk is useless, they cannot provide me the valid GW address.
Other problem is the NS resolution. There are 2 NS servers for each provider, added in DNS and DHCP DNS server list too. From the terminal the DNS resolution works, but not from any DHCP client computer.
Here is the router config:

# 2024-05-19 21:32:21 by RouterOS 7.14.3
# software id = 5QSS-81G1
#
# model = C53UiG+5HPaxD2HPaxD
/interface bridge
add name=bridge1
/interface ethernet
set [ find default-name=ether4 ] name=ether4-WAN-O2
set [ find default-name=ether5 ] mac-address=04:8D:38:62:E9:A8 name=ether5-WAN-NDC
/interface vlan
add interface=ether4-WAN-O2 name=vlan1-O2 vlan-id=848
/interface pppoe-client
add add-default-route=yes disabled=no interface=vlan1-O2 max-mru=1492 max-mtu=1492 name=pppoe-O2 user=**
/ip pool
add name=dhcp_pool0 ranges=192.168.88.10-192.168.88.50
/ip dhcp-server
add address-pool=dhcp_pool0 interface=bridge1 name=dhcp1
/ip smb users
set [ find default=yes ] disabled=yes
/routing table
add disabled=no fib name=To-O2
add disabled=no fib name=To-NDC
/interface bridge port
add bridge=bridge1 interface=ether1
add bridge=bridge1 interface=ether2
add bridge=bridge1 interface=ether3
add bridge=bridge1 interface=wifi2
add bridge=bridge1 interface=wifi1
/ip firewall connection tracking
set udp-timeout=10s
/ip neighbor discovery-settings
set discover-interface-list=!dynamic
/ip address
add address=192.168.88.0/24 interface=bridge1 network=192.168.88.0
/ip dhcp-client
add add-default-route=no interface=ether5-WAN-NDC
add add-default-route=no interface=ether4-WAN-O2
/ip dhcp-server network
add address=192.168.88.0/24 dns-server=\
    160.218.161.54,194.228.41.65,176.74.128.10,176.74.128.11 gateway=192.168.88.1
/ip dns
set servers=160.218.161.54,194.228.41.65
/ip firewall mangle
add action=accept chain=prerouting dst-address-list=10.74.200.34/28 in-interface=bridge1
add action=accept chain=prerouting dst-address-list=10.233.23.0/24 in-interface=bridge1
add action=mark-connection chain=prerouting connection-mark=no-mark in-interface=ether5-WAN-NDC new-connection-mark=NDC-Mark passthrough=yes
add action=mark-connection chain=prerouting connection-mark=no-mark in-interface=ether4-WAN-O2 new-connection-mark=O2-Mark passthrough=yes
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-type=!local in-interface=bridge1 new-connection-mark=O2-Mark \
    passthrough=yes per-connection-classifier=both-addresses:2/0
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-type=!local in-interface=bridge1 new-connection-mark=NDC-Mark \
    passthrough=yes per-connection-classifier=both-addresses:2/1
add action=mark-routing chain=prerouting connection-mark=NDC-Mark in-interface=bridge1 new-routing-mark=To-NDC passthrough=yes
add action=mark-routing chain=prerouting connection-mark=O2-Mark in-interface=bridge1 new-routing-mark=To-O2 passthrough=yes
add action=mark-routing chain=output connection-mark=NDC-Mark new-routing-mark=To-NDC passthrough=yes
add action=mark-routing chain=output connection-mark=O2-Mark new-routing-mark=To-O2 passthrough=yes
/ip firewall nat
add action=masquerade chain=srcnat out-interface=ether5-WAN-NDC
add action=masquerade chain=srcnat out-interface=pppoe-O2
/ip route
#ping OK
add check-gateway=ping disabled=no distance=2 dst-address=0.0.0.0/32 gateway=10.74.200.33 pref-src="" routing-table=To-NDC scope=30 suppress-hw-offload=no target-scope=10
#Ping KO
add check-gateway=ping disabled=no distance=1 dst-address=0.0.0.0/32 gateway=10.233.23.92 pref-src="" routing-table=To-O2 scope=30 suppress-hw-offload=no target-scope=10
#Ping OK
add check-gateway=ping disabled=no distance=1 dst-address=0.0.0.0/32 gateway=10.74.200.33 pref-src="" routing-table=main scope=30 suppress-hw-offload=no target-scope=10
#Ping KO
add check-gateway=ping disabled=no distance=1 dst-address=0.0.0.0/32 gateway=10.233.23.92 pref-src="" routing-table=main scope=30 suppress-hw-offload=no target-scope=10

Thank you in advance!

sindy · Wed May 22, 2024 10:28 am

First, let's sort out some things.

just to avoid confusing other folks reading your post - the phone cable is not connected to a "bridge" but to a modem in bridge mode.
the reason why the PPPoE client cannot be attached directly to the Ethernet interface and a VLAN is required is that other services (namely, IPTV and telephony) may be optionally provided on the same physical path and VLANs are used to logically partition the physical media. Big companies have to use "one size fits all" solutions wherever possible so even if you only subscribe to internet, the protocol hierarchy on the physical media stays the same.
PPPoE, like all the other protocols based on PPP, does not use DHCP to assign an address to the client; instead, PPP uses its own procedures for that. So you cannot use any methods of obtaining the gateway address that rely on operation of a DHCP client.

Next, the routing.

/ip/address/print shows you three items - address, network, and interface:
Code: Select all
```
[me@myTik] > ip address print
Flags: X - disabled, I - invalid, D - dynamic
 #   ADDRESS            NETWORK         INTERFACE
 0   192.168.88.1/24    192.168.88.0    bridge
 1 D 10.213.126.51/32   10.10.10.1      pppoe-o2
```
In case of PPP, the network one is what you should set as a gateway of a route to let the routing look the out-interface up by an address. According to your configuration, it seems you have used your own address assigned by O2 as the gateway of the routes, which is not correct. What you haven't done right either is the configuration of the dst-address of those routes: 0.0.0.0/32 means literally only a single exact address 0.0.0.0, as the /32 specifies that each of the "first 32" bits of the address (which actually means all of them) must have a value of 0. To make those routes work, you have to change that to 0.0.0.0/0, meaning "the whole IPv4 address range", as each of the 32 bits of the address except the "0th one" (so actually all of them) can have either value.
it's very likely that you cannot ping the network address associated to the PPPoE address (10.10.10.1 in the example above) manually, so set check-gateway on the routes using that gateway to none. Most likely no equipment actually listens at that address, because the sole purpose of that address is to serve as a kind of alias for the interface name, as mentioned above. If you want to check the transparency of the "oh too" uplink all the way to the internet because you suspect the PPPoE part could be up but the data would not get through (which can indeed happen), you have to use the "recursive routing" approach as described multiple times here on the forum, the primary resource being viewtopic.php?t=157048 .

DNS server addresses are yet another issue - those listed in your configuration seem to be "internal" ones of the respective ISPs, which typically means that although their addresses are "public" in the sense of address type, they do not respond to requests coming from outside the network of the respective ISP. So if you send a request via a "wrong" uplink by chance, you'll never get a response. To deal with this, you have two options, either manually set the DNS addresses to some publicly accessible ones (like Google, Cloudflare, Quad9 to name the most well-known ones), or add dedicated /32 routes towards the addresses provided by the ISPs via their respective uplinks, but such a solution is a bit fragile as the ISPs may decide to change those addresses next year or next minute. If you decide to use the publicly accessible ones and at the same time to use the recursive next hop search to check the fitness of the uplink, make sure that you do not use the same address for DNS and as a canary address for the next hop search.

Peculiar PCC setup/issue and more

Peculiar PCC setup/issue and more

Re: Peculiar PCC setup/issue and more

Who is online