Community discussions

MikroTik App
 
User avatar
Watts
just joined
Topic Author
Posts: 10
Joined: Mon Aug 31, 2020 9:07 pm
Location: Dartmouth, NS

Intermittent ping timeout between OVPN clients

Tue Mar 22, 2022 4:21 pm

I have CHR installed on an AWS instance (used the AMI from the Marketplace), set up for 4000 remote units to connect, and security on the instance has it open to the world. The two units I have for testing connect to the VPN with RBM33G, via USB connected R11e-LTE-US, without issue and remain connected. They can ping the CHR, it can ping them. The end goal is to have all the units isolated from one another, then have a script automatically add several units to an address list so they can communicate when deployed to the field. For the time being the script is not in place. I manually added the test units to the address list "test" and added a filter rule that any communication from test to test is accepted. When I attempt to ping from one unit to another I get intermittent timeouts. I've done several tests of various lengths (between 6 and 230 pings) and I get between 15% and 40% drop rate. I am unable to determine why some pings timeout. Below is the export of the CHR and an example of one of the clients. The CHR was edited to remove the majority of the 4000 clients as 92 lines is easier to parse than 12000.

CHR
# mar/22/2022 13:11:19 by RouterOS 6.44.3
# software id = 
#
#
#
/interface bridge
add arp=proxy-arp fast-forward=no name=clients priority=0x8192 \
    transmit-hold-count=1
/interface ethernet
set [ find default-name=ether1 ] disable-running-check=no
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip pool
add name=clientpool ranges=10.8.0.1-10.8.127.255
/ppp profile
add bridge=clients local-address=10.8.0.1 name=clientProfiles remote-address=\
    clientpool use-encryption=yes
/interface bridge port
add bridge=clients interface=ether1
add bridge=clients interface=*F005C9
add bridge=clients interface=*F004E9
add bridge=clients interface=dynamic
/interface ovpn-server server
set auth=sha1 certificate=ServerCert cipher=aes256 default-profile=\
    clientProfiles enabled=yes keepalive-timeout=30 netmask=17
/ip dhcp-client
# DHCP client can not run on slave interface!
add dhcp-options=hostname,clientid disabled=no interface=ether1
/ip firewall address-list
# 4000 units have been added to the list, but removed here for brevity
add address=10.8.40.1 list=undeployed
add address=10.8.80.23 list=test
add address=10.8.80.24 list=test
add address=10.8.87.207 list=undeployed
/ip firewall filter
add action=accept chain=forward comment=\
    "Allows members of the test address list to communicate." \
    dst-address-list=test src-address-list=test
add action=accept chain=forward comment=\
    "Allows all traffic from Internal Trusted Servers to units." \
    dst-address-list=!InternalTrustedServers src-address=0.0.0.0 \
    src-address-list=InternalTrustedServers
add action=accept chain=forward comment=\
    "Allows all traffic from units to Internal Trusted Servers." \
    dst-address-list=InternalTrustedServers
add action=accept chain=forward comment="Test of unit to unit communication" \
    disabled=yes dst-address-list=test src-address-list=test
add action=accept chain=forward comment=\
    "Accept Forward for Established and Related Connections" \
    connection-state=established,related,untracked
add action=accept chain=forward comment="Allow Forwarding by OVPN Clients" \
    src-address=192.168.22.128/25
add action=accept chain=input comment=\
    "Accept Input for Established and Related Connections" connection-state=\
    established,related,untracked
add action=accept chain=input comment="Allow OpenVPN Connection" dst-port=\
    1194 protocol=tcp
add action=accept chain=input comment="Allow Input by OVPN Clients" \
    in-interface=all-ppp
add action=accept chain=input comment="Allow Winbox Input" dst-port=8291 \
    protocol=tcp
add action=accept chain=input comment="Allow HTTPS Input" dst-port=443 \
    protocol=tcp
add action=drop chain=input comment="Input drop for all other connection" \
    disabled=yes
add action=drop chain=forward comment="Forward drop for all other connection" \
    disabled=yes
add action=drop chain=forward comment="Invalid drop for all other connection" \
    connection-state=invalid disabled=yes
add action=drop chain=forward comment="PREVENT ALL TALK BETWEEN UNITS." \
    disabled=yes src-address=!10.8.0.5
/ip firewall nat
add action=masquerade chain=srcnat out-interface=all-ppp
/ip service
set telnet disabled=yes
set ftp disabled=yes
set www disabled=yes
set api-ssl disabled=yes
/ppp secret
# 4000 units have been added to the list, but removed here for brevity
add name=UNIT1 profile=clientProfile remote-address=10.8.80.1 service=ovpn
add name=UNIT23 profile=clientProfile remote-address=10.8.80.23 service=\
    ovpn
add name=UNIT24 profile=clientProfile remote-address=10.8.80.24 service=\
    ovpn
add name=UNIT4000 profile=clientProfile remote-address=10.8.47.207 service=\
    ovpn
/system identity
set name=CHRserver
/system logging
add topics=ovpn
add topics=debug

Client
# feb/18/2022 15:56:20 by RouterOS 6.48.3
# software id = QR61-PILT
#
# model = RBM33G
# serial number = A2FD0E5B89AE
/interface bridge
add arp=proxy-arp mtu=1350 name="S2LAN Bridge"
add fast-forward=no name=bridge1
/interface ethernet
set [ find default-name=ether1 ] arp=proxy-arp l2mtu=1500
set [ find default-name=ether2 ] arp=proxy-arp l2mtu=1500
set [ find default-name=ether3 ] arp=proxy-arp l2mtu=1500
/interface ovpn-client
add add-default-route=yes certificate=client.crt cipher=aes256 \
    connect-to=[PUBLIC IP] mac-address=FE:AB:ED:98:94:4A name=ovpn-out2 \
    user=UNIT24
/interface lte apn
set [ find default=yes ] add apn=[APN NAME] name=APN
/interface lte
set [ find ] apn-profiles=APN name=lte1
/interface wireless security-profiles
set [ find default=yes ] authentication-types=wpa2-psk mode=dynamic-keys \
    supplicant-identity=MikroTik
add authentication-types=wpa2-psk eap-methods="" management-protection=\
    allowed mode=dynamic-keys name=LoRa supplicant-identity=MikroTik
/interface wireless
set [ find default-name=wlan1 ] arp=proxy-arp band=5ghz-a/n/ac country=\
    "united states" disabled=no frequency=auto mode=ap-bridge radio-name=\
    UNIT24 scan-list=5180,5200,5220,5240,5745,5765,5785,5805,5825 \
    security-profile=LoRa ssid=UNIT24 station-roaming=\
    enabled wds-default-bridge="S2LAN Bridge" wds-default-cost=50 \
    wireless-protocol=802.11 wps-mode=disabled
set [ find default-name=wlan2 ] arp=proxy-arp band=2ghz-onlyg \
    basic-rates-a/g=6Mbps,12Mbps,18Mbps,24Mbps basic-rates-b="" country=\
    "united states" disabled=no disconnect-timeout=6s frequency=2437 \
    hw-protection-mode=rts-cts hw-retries=3 max-station-count=64 mode=\
    ap-bridge multicast-helper=full nv2-cell-radius=10 nv2-downlink-ratio=20 \
    nv2-security=enabled radio-name=UNIT24 scan-list=2412,2437,2462 \
    security-profile=LoRa ssid=Wi-Fi_LR station-roaming=\
    enabled supported-rates-a/g=6Mbps,12Mbps,18Mbps,24Mbps supported-rates-b=\
    "" wds-default-bridge="S2LAN Bridge" wds-mode=dynamic-mesh \
    wireless-protocol=802.11 wps-mode=disabled
/ip pool
add name=pool1 ranges=192.168.2.101-192.168.2.102
add name=dhcp_pool1 ranges=10.8.0.101-10.8.0.200
/ip dhcp-server
add address-pool=dhcp_pool1 disabled=no interface="S2LAN Bridge" name=dhcp1
/ppp profile
set *0 bridge-path-cost=1500
add bridge="S2LAN Bridge" bridge-path-cost=1500 name=ppp_bridge \
    use-encryption=yes
/queue type
add kind=pcq name=custom pcq-classifier=\
    src-address,dst-address,src-port,dst-port pcq-dst-address6-mask=64 \
    pcq-limit=10KiB pcq-src-address6-mask=64
/queue interface
set wlan1 queue=custom
/user group
set full policy="local,telnet,ssh,ftp,reboot,read,write,policy,test,winbox,pas\
    sword,web,sniff,sensitive,api,romon,dude,tikapp"
/interface bridge port
add bridge=bridge1 interface=ether1
add bridge=bridge1 interface=ether2
add bridge=bridge1 interface=ether3
add bridge="S2LAN Bridge" interface=wlan1 multicast-router=disabled
add bridge="S2LAN Bridge" interface=wlan2 multicast-router=disabled
/ip firewall connection tracking
set enabled=yes loose-tcp-tracking=no tcp-syn-received-timeout=15s \
    tcp-syn-sent-timeout=15s
/ip neighbor discovery-settings
set discover-interface-list=all
/ip address
add address=192.168.2.1/24 interface=bridge1 network=192.168.2.0
add address=10.8.80.24/17 interface="S2LAN Bridge" network=10.8.0.0
/ip dhcp-server network
add address=10.8.0.0/17 gateway=10.8.0.1 netmask=17
add address=192.168.2.0/24 dns-server=1.1.1.1 gateway=192.168.2.1
/ip firewall nat
add action=masquerade chain=srcnat comment="NAT masquerade" \
    out-interface="S2LAN Bridge"
add action=dst-nat chain=dstnat comment="BCP 4002 - redirect to pi DST-nat" \
    dst-address=10.8.80.24 dst-port=4002 in-interface="S2LAN Bridge" \
    protocol=tcp to-addresses=192.168.2.2
add action=dst-nat chain=dstnat comment="Access Front Cam RTSP DST-NAT" \
    dst-address=10.8.80.24 dst-port=8500 protocol=tcp to-addresses=\
    192.168.2.8 to-ports=8500
add action=dst-nat chain=dstnat comment="Access Front Cam RTSP DST-NAT" \
    dst-address=10.8.80.24 dst-port=9500 protocol=tcp to-addresses=\
    192.168.2.9 to-ports=9500
add action=dst-nat chain=dstnat comment="Access Front Cam HTTP DST-NAT" \
    dst-address=10.8.80.24 dst-port=8001 protocol=tcp to-addresses=\
    192.168.2.8 to-ports=80
add action=dst-nat chain=dstnat comment="Access Front Cam HTTP DST-NAT" \
    dst-address=10.8.80.24 dst-port=9001 protocol=tcp to-addresses=\
    192.168.2.9 to-ports=80
add action=dst-nat chain=dstnat dst-address=10.8.80.24 dst-port=6000-6999 \
    in-interface="S2LAN Bridge" protocol=udp to-addresses=192.168.2.2
add action=dst-nat chain=dstnat dst-address=10.8.80.24 dst-port=33 protocol=\
    tcp to-addresses=192.168.2.2 to-ports=22
add action=src-nat chain=srcnat out-interface=lte1 src-address=192.168.2.2 \
    to-addresses=10.80.47.99
/ip ssh
set always-allow-password-login=yes
/system clock
set time-zone-autodetect=no time-zone-name=Etc/UTC
/system identity
set name=UNIT24
/system leds
add interface=wlan1 leds="wlan1_signal1-led,wlan1_signal2-led,wlan1_signal3-le\
    d,wlan1_signal4-led,wlan1_signal5-led" type=wireless-signal-strength
add interface=wlan1 leds=wlan1_tx-led type=interface-transmit
add interface=wlan1 leds=wlan1_rx-led type=interface-receive
/system logging
add topics=ovpn,debug
/system routerboard settings
set boot-delay=5s boot-device=nand-only
/system scheduler
add interval=30s name=modemScript on-event=NatLTEIPFix policy=\
    ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon \
    start-date=jan/01/1970 start-time=00:00:00
/system script
add dont-require-permissions=no name=NatLTEIPFix owner=admin policy=\
    ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon source=":\
    global currentIP;\
    \n\
    \n:local newIP [/ip address get [find interface=\"lte1\"] address];\
    \n\
    \n:if (\$newIP != \$currentIP) do={\
    \n    :put \"ip address \$currentIP changed to \$newIP\";\
    \n    :set currentIP \$newIP;\
    \n    /ip firewall nat set [find action=src-nat] to-address=\$currentIP\
    \n    /ip firewall connection remove [find]\
    \n}"
/tool romon
set enabled=yes
 
User avatar
Watts
just joined
Topic Author
Posts: 10
Joined: Mon Aug 31, 2020 9:07 pm
Location: Dartmouth, NS

Re: Intermittent ping timeout between OVPN clients  [SOLVED]

Tue Mar 22, 2022 7:20 pm

This can be disregarded. We have discovered that the antennas on the units needed to be replaced.

Who is online

Users browsing this forum: No registered users and 12 guests