Weird routing problem

RouterOS General
1

Can someone explain this to me>

I have multiple openwrt + mikrotik devices.. they are all on the same network /24 .. first one is lets say 192.168.1.1 and that is gateway to other network which leads to another.. all over to Internet.
that device is openwrt and it’s working perfectly and has internet access with or without nat (routing / ospf).

Now.. If I add mikrotik device that will connect to that openwrt and put any device after mikrotik (lan/wlan) it has internet access with packet loss… that is if I use 192.168.1.1 as direct gateway.. if I put 192.168.1.2 (mikrotik device) than it works relatively ok ..

I don’t know if you can get me.. they are all in bridge

router ↔ MKT ↔ PC

router has DHCP .. so PC gets it’s as gateway and dns… but only if I change it to MKT device than it works… is it MTU problem or what?

2

From your description it is not completely clear, what exactly you want achieve:

  1. for mikrotik to route between it’s own network and 192.168.1.0/24
  2. for mikrotik to NAT from it’s own network to outside world
  3. for mikrotik to act as a switch (bridge)
3

router…router..router ← MikrotikOS ← client

let’s say like this.. .. mikrotik os is wlan/lan device in bridge mode.. all firewall rules deleted.. no nat , no anything .. routerOS has default gw of router and it works.. but if client has default gw of router then it doesent work.. it works if set manualy to MikrotikOS router .. why is that? isnt all in bridge just transparent?

4

Bridge is L2 entity.
WAN ↔ LAN routing is L3.

I guess some mixture of both entities is producing unpredictable result.

Post your mikrotik configuration.

5

Lets get back to this.

so .. I have right now bridge with two LHG 5 dishes doing about 3km link. Current protocol is nstreme.

so .. router <> LHG ↔ LHG <> router

problem is.. some pages wont load.. specifically console.thethingsnetwork.org
I just don’t know how to debug this issue.
First LHG is on bridge mode and second is on station bridge.
All traffic should be “untouched” by bridge, but this seems not to be so.

I have 192.168.20.1 and 192.168.20.2 .. between them is this bridge .. I want to do transparent bridge (like LAN cable) and do default gateway on 20.1 from 20.2 and reverse. So mikrotik devices are like bridge, they are and should not be gateway to anything.

Any help debugging this is welcome. I downgraded to stable but still there is issue.

6

If the radios are in bridge mode they pass traffic transparently. The IP address is only required for management access to the radio, plus a default gateway if they need to communicate with something outside of their subnet e.g. a PC on another subnet or the internet to download updates - the gateway certainly shouldn’t be set to the address of the other radio.

7

I don’t see it as transparent! I had to do macvlan tunnel to get it fully working! Anyone knows why is that?

8

Without seeing the actual configurations rather than what you believe you have configured it is impossible to say.

9

pretty simple config.. one in bridge mode, other in station bridge mode.
how can I export important info?

10

All info is important (because the issue is typically caused by something you assume not to be important).
See my automatic signature right below for a hint how to properly exclude those few bits of information which need to stay private.

11

device 1

[admin@dev1] > /export hide-sensitive
# sep/20/2020 16:49:28 by RouterOS 6.47.3
# software id = DBKU-GQIG
#
# model = RouterBOARD LHG G-5acD
# serial number = 80B9077C1887
/interface bridge
add name=bridge1
/interface ethernet
set [ find default-name=ether1 ] speed=100Mbps
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
add exclude=dynamic name=discover
add name=mactel
add name=mac-winbox
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
add authentication-types=wpa-psk,wpa2-psk eap-methods="" group-ciphers=tkip,aes-ccm management-protection=allowed mode=dynamic-keys name=xxx \
    supplicant-identity="" unicast-ciphers=tkip,aes-ccm
/interface wireless
set [ find default-name=wlan1 ] ampdu-priorities=0,1,2 antenna-gain=0 band=5ghz-a/n/ac basic-rates-a/g=12Mbps,18Mbps,24Mbps,36Mbps,48Mbps,54Mbps \
    channel-width=20/40/80mhz-Ceee country=no_country_set disabled=no distance=3 frequency=5360 frequency-mode=superchannel ht-supported-mcs=\
    mcs-0,mcs-1,mcs-2,mcs-3,mcs-4,mcs-5,mcs-6,mcs-7 mode=station-bridge scan-list=5360 security-profile=xxx ssid=NOD10-AC_FORCE station-roaming=\
    enabled supported-rates-a/g=12Mbps,18Mbps,24Mbps,36Mbps,48Mbps,54Mbps vht-supported-mcs=mcs0-7,mcs0-7,mcs0-7 wmm-support=enabled wps-mode=disabled
/interface wireless nstreme
set wlan1 enable-nstreme=yes
/ip hotspot profile
set [ find default=yes ] html-directory=flash/hotspot
/ip pool
add name=dhcp ranges=192.168.1.10-192.168.1.250
/port
set 0 name=serial0
/routing bgp instance
set default disabled=yes
/snmp community
set [ find default=yes ] addresses=0.0.0.0/0
/user group
set full policy=local,telnet,ssh,ftp,reboot,read,write,policy,test,winbox,password,web,sniff,sensitive,api,romon,dude,tikapp
/interface bridge port
add bridge=bridge1 hw=no interface=ether1
add bridge=bridge1 interface=wlan1
/ip neighbor discovery-settings
set discover-interface-list=discover
/ip settings
set accept-redirects=yes accept-source-route=yes allow-fast-path=no route-cache=no
/interface list member
add interface=ether1 list=discover
add interface=bridge1 list=discover
add interface=ether1 list=mactel
add interface=ether1 list=mac-winbox
add interface=wlan1 list=WAN
add interface=ether1 list=LAN
/ip address
add address=192.168.20.254/24 interface=ether1 network=192.168.20.0
/ip dns
set allow-remote-requests=yes servers=8.8.8.8,1.1.1.1
/ip dns static
add address=192.168.20.254 name=router.lan
/ip route
add distance=1 gateway=192.168.20.1
/ip service
set telnet disabled=yes
set ftp disabled=yes
/routing bfd interface
set [ find default=yes ] disabled=yes
/system clock
set time-zone-name=Europe/Zagreb
/system identity
set name=dev1
/system logging
set 0 disabled=yes
set 1 disabled=yes
set 2 disabled=yes
set 3 disabled=yes
/system watchdog
set automatic-supout=no ping-start-after-boot=0ms ping-timeout=10s
/tool mac-server
set allowed-interface-list=mactel
/tool mac-server mac-winbox
set allowed-interface-list=mac-winbox

dev2

[admin@dev2] >  /export hide-sensitive
# sep/20/2020 16:50:59 by RouterOS 6.47.3
# software id = PBMM-K38Y
#
# model = RouterBOARD LHG G-5acD
# serial number = 8108072D6445
/interface bridge
add name=bridge1
/interface ethernet
set [ find default-name=ether1 ] speed=100Mbps
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
add exclude=dynamic name=discover
add name=mactel
add name=mac-winbox
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
add authentication-types=wpa-psk,wpa2-psk management-protection=allowed mode=dynamic-keys name=xxx supplicant-identity=MikroTik
/interface wireless
set [ find default-name=wlan1 ] ampdu-priorities=0,1,2 antenna-gain=0 band=5ghz-a/n/ac channel-width=20/40/80mhz-Ceee country=no_country_set disabled=no \
    distance=3 frequency=5360 frequency-mode=superchannel mode=bridge scan-list=5360 security-profile=xxx ssid=NOD10-AC_FORCE station-roaming=\
    enabled wireless-protocol=nstreme wmm-support=enabled wps-mode=disabled
/interface wireless nstreme
set wlan1 enable-nstreme=yes
/ip hotspot profile
set [ find default=yes ] html-directory=flash/hotspot
/port
set 0 name=serial0
/snmp community
set [ find default=yes ] addresses=0.0.0.0/0
/user group
set full policy=local,telnet,ssh,ftp,reboot,read,write,policy,test,winbox,password,web,sniff,sensitive,api,romon,dude,tikapp
/interface bridge port
add bridge=bridge1 interface=wlan1
add bridge=bridge1 interface=ether1
/ip neighbor discovery-settings
set discover-interface-list=all
/ip settings
set accept-redirects=yes accept-source-route=yes allow-fast-path=no route-cache=no
/interface list member
add interface=ether1 list=discover
add interface=ether1 list=mactel
add interface=ether1 list=mac-winbox
add interface=ether1 list=WAN
add interface=wlan1 list=LAN
/interface wireless align
set receive-all=yes
/ip address
add address=192.168.20.253/24 comment=NOD10_LAN interface=wlan1 network=192.168.20.0
/ip dns
set allow-remote-requests=yes servers=8.8.8.8
/ip dns static
add address=192.168.20.253 name=router.lan
/ip route
add distance=1 gateway=192.168.20.2
/ip service
set telnet disabled=yes
set ftp disabled=yes
/system clock
set time-zone-name=Europe/Zagreb
/system identity
set name=dev2
/system logging
set 0 disabled=yes
set 1 disabled=yes
set 2 disabled=yes
set 3 disabled=yes
/tool mac-server
set allowed-interface-list=mactel
/tool mac-server mac-winbox
set allowed-interface-list=mac-winbox
12

The IP addresses should be applied to the bridge, not any of the member ports as this breaks things in odd ways.

13

Except no-country-set in the wireless settings (so you’re on your own regarding conformance to national regulations), the only minor issue I can see is that your IP address configuration is wrongly attached to a member port of the bridge (ether1 in one case, wlan1 in the other one) rather than to the bridge port itself, but that doesn’t affect how the bridging A.ether1<=>A.wlan1<=>B.wlan1<=>B.ether1 works. So the 192.168.20.1 connected externally to ether1 of one LHG should be able to ping 192.168.20.2 connected externally to ether1 of the other LHG as far as these two Mikrotik configurations are concerned. Is that not the case?

14

Yup. It’s not the case. Actually – not full case.. ping works as expected, but some sites dont load.. I suspect MTU problem, but tried changing it without success. I’ll try to do IP on bridge, but this is how it’s default set from Quick settings.

15

I think this might have fixed the problem. I’ll check more tommorow as it could be some caching thing. How could IP on network interface and not bridge even cause this? isn’t that IP only used for device itself and should not affect traffic going through?

16

Pojma nemam - exactly as you write, it should be related only to local IP traffic, not to L2 forwarding. Quickset is a separate story, what is even worse is that even the configuration upgrade from pre-6.41 migrates the IP configuration in a wrong way (it stays attached to the previous master-port rather then being moved to the bridge).

17

Pojma nemam :smiley:
It certainly bugs me.. I need to be on that network and use it to see things that might have happend and I have a large network.
For example, first time I noticed this was in ssh to openwrt node… I did dmesg and half result came back and ssh session stalled. I guessed MTU problem so I did “-j TCPMSS --clamp-mss-to-pmtu” and thing worked. That’s why I rely on openwrt in mikrotik boards and not on mikrotikOS, but some devices are not yet supported. If someone from MKT could investigate this, that’ll be great.

18

In L2-transparent connection the PMTU doesn’t work as there is no equivalent of the ICMP “fragmentation needed”, so I’m not sure how clamp-mss-to-pmtu fixes the issue? As for the actual MTU of the bridged link, pinging through it with large packets and DF bit set should reveal the truth - there should be a range of packet sizes which already don’t get through although the declared MTU of the interface is not exceeded yet, but I can see nothing in your configuration that would indicate an L2 MTU restriction.

But I’m no expert at nstreme, so there is a small chance it has some hidden features related to MTU.

19

o I was talking about routed connection, not bridged. sry.
As for nstreme, its with or without nstreme.. I also tried other modes, like pseudobridge and such.. same thing.