Community discussions

MikroTik App
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Unable to update CCR

Wed May 13, 2020 10:52 pm

I am unable to update RouterOS, the setup is complex but there are no issues with regard to traffic. Bizarrely a similar CCR which is at the other end of a L2TP connection is able to Update normally.

Happy to post logs but would need instructions.

Current Version is 6.45.8
ping to mikrotik.com is successful
ping to upgrade.mikrotik.com is successful

Thanks
 
User avatar
mutluit
Forum Veteran
Forum Veteran
Posts: 821
Joined: Wed Mar 25, 2020 4:04 am

Re: Unable to update CCR

Wed May 13, 2020 11:31 pm

What happens if you attempt to do the update? Do you get any error message? How does the failing manifest itself?

Not sure, just a guess: there could be a setting that prevents any updates; would need first to change that setting.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Wed May 13, 2020 11:36 pm

What happens if you attempt to do the update? Do you get any error message? How does the failing manifest itself?
In the Check for Updates Window:

ERROR: connection timed out


Thanks,
Andrew.
 
User avatar
mutluit
Forum Veteran
Forum Veteran
Posts: 821
Joined: Wed Mar 25, 2020 4:04 am

Re: Unable to update CCR

Wed May 13, 2020 11:49 pm

Can you issue the following in the terminal (CLI) and post the output:
/system package update check-for-updates

At this very moment the update server is operational from my location in Central Europe. It says in my case the following:
[admin2@CRS326] > /system package update check-for-updates
channel: development
installed-version: 7.0beta5
latest-version: 7.0beta5
status: System is already up to date
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Thu May 14, 2020 12:03 am

At this very moment the update server is operational from my location in Central Europe. It says in my case the following:
[admin2@CRS326] > /system package update check-for-updates
channel: development
installed-version: 7.0beta5
latest-version: 7.0beta5
status: System is already up to date
I get:

[admin@Malford1016] >> /system package update check-for-updates
channel: long-term
installed-version: 6.45.8
status: ERROR: connection timed out
 
User avatar
mutluit
Forum Veteran
Forum Veteran
Posts: 821
Joined: Wed Mar 25, 2020 4:04 am

Re: Unable to update CCR

Thu May 14, 2020 12:14 am

Try this too:
/tool fetch mode=https url="https://update.mikrotik.com/dummy.html"

Do you get something different than 404 ?

It could be a firewall issue at your side. ping uses a different protocol (ICMP) than https (TCP),
and the IP of update.mikrotik.com seems to have changed in the DNS.
A few days ago it had these 2 IPs:
159.148.147.204
159.148.172.226

but today it has just this (new?) IP:
159.148.147.205

So, it could also be a DNS server issue, ie. maybe it could also be possible that it's not yet replicated to your DNS server.

Try via IP too:
/tool fetch mode=https url="https://159.148.147.205/dummy.html"

Should give error "404 Page Not Found", but this indicates that the connection at least was ok.


UPDATE: they are 2 different domains, as follows:

1)
Name: update.mikrotik.com
Address: 159.148.147.205

2)
upgrade.mikrotik.com canonical name = download.mikrotik.com.
Name: download.mikrotik.com
Address: 159.148.147.204
Name: download.mikrotik.com
Address: 159.148.172.226
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Thu May 14, 2020 2:12 pm

/tool fetch mode=https url="https://update.mikrotik.com/dummy.html"
/tool fetch mode=https url="https://159.148.147.205/dummy.html"
I've tried those as well as 172.226 and all give me:

[admin@Malford1016] > /tool fetch mode=https url="https://159.148.172.226/dummy.html"
status: failed
failure: connection timeout

I have also changed the DNS setting to 1.1.1.1 to isolate the P-Hole based White/Black list and get the same result. I've tried with and without "Allow remote requests"

I have another CCR in a remote location whose DNS is set by RouterOS to 1.1.1.1 and it updates fine. That is connected to the home location via L2TP.

Thanks.
 
User avatar
mutluit
Forum Veteran
Forum Veteran
Posts: 821
Joined: Wed Mar 25, 2020 4:04 am

Re: Unable to update CCR

Thu May 14, 2020 2:56 pm

/tool fetch mode=https url="https://update.mikrotik.com/dummy.html"
/tool fetch mode=https url="https://159.148.147.205/dummy.html"
I've tried those as well as 172.226 and all give me:

[admin@Malford1016] > /tool fetch mode=https url="https://159.148.172.226/dummy.html"
status: failed
failure: connection timeout

I have also changed the DNS setting to 1.1.1.1 to isolate the P-Hole based White/Black list and get the same result. I've tried with and without "Allow remote requests"

I have another CCR in a remote location whose DNS is set by RouterOS to 1.1.1.1 and it updates fine. That is connected to the home location via L2TP.

Thanks.
It very much looks like that in your firewall rules on this device all traffic originating from that device, except ICMP (ping), gets blocked. Maybe a security setting by you or your co-admins?
If there is any other firewall on the way between this device and the WAN, then you should check it too for the cause.

So, you should first ensure that your device can make TCP connections to the WAN (ie. try to get the index.html of other web sites with the above said fetch tool).

You could also export and post here your config with this CLI command:
/export hide-sensitive file=myconfig.rsc
It will be put into the Files area on the device. Via Winbox/Webfig/ssh/telnet etc. you can download it to your PC.
It is a text file. You can inspect it and remove/edit any sensitive private data, and then post it here for analysis by others.

You can of course also compare that config with the config of your other working device that you mentioned.

Another possibility is to download the correct file from here https://www.mikrotik.com/download and install it manually as described here:
https://wiki.mikrotik.com/wiki/Manual:U ... de_methods

As can be seen, you have many alternatives to chose from :-)
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Thu May 14, 2020 3:26 pm

/export hide-sensitive file=myconfig.rsc
The time won't update either, seems to be about a minute out.

This is the config of the Problematic CCR (below that is the remote CCR which has no issues):
# may/14/2020 13:15:01 by RouterOS 6.45.8
# software id = PZJX-KJ04
#
# model = CCR1016-12G
# serial number = 42D402B557C9
/interface bridge
add fast-forward=no name=bridge-lan
add name=bridgeDRGuest
add name=bridgeGuest
/interface ethernet
set [ find default-name=ether2 ] name=BT
set [ find default-name=ether1 ] comment=\
"Virgin Modem Mode connection to DHCP Client" name=ISP-Virgin
/interface pppoe-client
add add-default-route=yes comment="BT PPPoE Client Modem Mode" \
default-route-distance=3 disabled=no interface=BT name=ISP-BT user=\
bthomehub@btbroadband.com
/ip pool
add name=pool-malford ranges=172.16.105.90-172.16.105.129
add name=pGuest ranges=10.10.10.10-10.10.10.100
add name=pDRGuest ranges=20.20.20.20-20.20.20.100
/ip dhcp-server
add address-pool=pool-malford interface=bridge-lan lease-time=1d name=\
dhcp-lan
add address-pool=pGuest disabled=no interface=bridgeGuest name=DHCPGuest
add address-pool=pDRGuest disabled=no interface=bridgeDRGuest name=\
DHCPDRGUEST
/system logging action
add email-to=andrew.gebhardt@finexlondon.com name=sendemail target=email
/interface bridge port
add bridge=bridge-lan interface=ether3
add bridge=bridge-lan interface=ether4
add bridge=bridge-lan interface=ether5
add bridge=bridge-lan interface=ether6
add bridge=bridge-lan interface=ether7
add bridge=bridge-lan interface=ether8
add bridge=bridge-lan interface=ether9
add bridge=bridge-lan interface=ether10
add bridge=bridgeDRGuest interface=ether11
add bridge=bridgeGuest interface=ether12
/ip settings
set tcp-syncookies=yes
/interface l2tp-server server
set enabled=yes
/interface pptp-server server
set enabled=yes
/interface sstp-server server
set certificate=Server default-profile=default-encryption port=8443 \
tls-version=only-1.2
/ip address
add address=172.16.105.1/24 interface=bridge-lan network=172.16.105.0
add address=192.168.1.2/24 interface=BT network=192.168.1.0
add address=192.168.2.2/24 disabled=yes interface=BT network=192.168.2.0
add address=10.10.10.254/24 interface=bridgeGuest network=10.10.10.0
add address=20.20.20.254/24 interface=bridgeDRGuest network=20.20.20.0
/ip cloud
set ddns-update-interval=2m
/ip dhcp-client
add add-default-route=no comment=\
"DHCP Client for Virgin Router in Modem Mode" dhcp-options=\
hostname,clientid disabled=no interface=ISP-Virgin use-peer-dns=no
/ip dhcp-server network
add address=10.10.10.0/24 dns-server=1.1.1.1,1.0.0.1 gateway=10.10.10.254 \
netmask=24
add address=20.20.20.0/24 dns-server=8.8.8.8,8.8.4.4 gateway=20.20.20.254 \
netmask=24
add address=172.16.105.0/24
/ip dns
set servers=172.16.105.2
/ip firewall address-list
add address=172.16.105.0/24 list=allowed
add address=172.16.110.0/24 list=allowed
add address=2.87.174.170 disabled=yes list=allowed
add address=2.87.174.170 list=TinosIP
/ip firewall filter
add action=drop chain=forward dst-address=172.16.105.0/24 src-address=\
10.10.10.0/24
add action=drop chain=forward dst-address=172.16.105.0/24 src-address=\
20.20.20.0/24
add action=drop chain=forward dst-address=10.10.10.0/24 src-address=\
172.16.105.0/24
add action=accept chain=input src-address-list=allowed
add action=accept chain=input protocol=icmp
add action=accept chain=forward connection-state=established,related,new
add action=accept chain=input dst-port=1701 protocol=udp
add action=accept chain=input dst-port=500 protocol=udp
add action=accept chain=input dst-port=4500 protocol=udp
add action=accept chain=input disabled=yes protocol=ipsec-ah
add action=accept chain=input disabled=yes protocol=ipsec-esp
add action=drop chain=input dst-port=53 in-interface=ISP-Virgin protocol=tcp
add action=drop chain=input dst-port=53 in-interface=ISP-Virgin protocol=udp
add action=drop chain=input dst-port=53 in-interface=ISP-BT protocol=tcp
add action=drop chain=input dst-port=53 in-interface=ISP-BT protocol=udp
add action=drop chain=input connection-state=invalid in-interface=ISP-Virgin
add action=drop chain=input connection-state=invalid in-interface=ISP-BT
add action=drop chain=input in-interface=ISP-Virgin
add action=drop chain=input in-interface=ISP-BT
/ip firewall mangle
add action=mark-routing chain=output disabled=yes log=yes new-routing-mark=\
virgin passthrough=no src-address=10.10.10.0/24
add action=change-mss chain=forward dst-address=172.16.110.0/24 new-mss=1442 \
passthrough=yes protocol=tcp tcp-flags=syn
add action=change-mss chain=forward new-mss=1470 passthrough=yes protocol=tcp \
tcp-flags=syn
/ip firewall nat
add action=masquerade chain=srcnat comment="NAT to Internet"
add action=dst-nat chain=dstnat disabled=yes dst-port=53 protocol=udp \
src-address-list=allowed to-addresses=172.16.105.2
add action=src-nat chain=srcnat disabled=yes dst-port=53 protocol=tcp \
to-addresses=172.16.105.2 to-ports=53
add action=src-nat chain=srcnat disabled=yes dst-port=53 protocol=udp \
to-addresses=172.16.105.2 to-ports=53
/ip firewall service-port
set ftp disabled=yes
set tftp disabled=yes
set irc disabled=yes
set h323 disabled=yes
set sip disabled=yes
set udplite disabled=yes
set dccp disabled=yes
set sctp disabled=yes
/ip route
add distance=1 gateway=82.17.140.1
add comment="Tinos via SSTP" distance=1 dst-address=172.16.110.0/24 gateway=\
10.10.20.10
/ip service
set telnet disabled=yes
set ftp disabled=yes
set www disabled=yes
set ssh address=172.16.105.0/24,172.16.110.0/24
set api disabled=yes
set winbox address=172.16.105.0/24,172.16.110.0/24
set api-ssl disabled=yes
/ip ssh
set allow-none-crypto=yes forwarding-enabled=remote
/ip upnp
set enabled=yes show-dummy-rule=no
/ppp secret
add local-address=172.16.105.1 name=tinos-l2tp remote-address=10.10.20.10 \
service=l2tp
/queue simple
add disabled=yes max-limit=10M/10M name=queue-tinos target=*F
/system clock
set time-zone-name=Europe/London
/system identity
set name=Malford1016
/system logging
add action=sendemail topics=script
/system ntp client
set enabled=yes primary-ntp=130.159.196.118 secondary-ntp=178.79.152.182
/system package update
set channel=long-term
/system scheduler
add interval=5s name=RunFailover on-event=FailoverScript policy=\
ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon \
start-date=jun/10/2019 start-time=14:09:11
/system script
add dont-require-permissions=no name=FailoverScript owner=admin policy=\
ftp,reboot,read,write,policy,test,password,sniff,sensitive source="# -----\
-------------- header -------------------\r\
\n# Script by Tomas Kirnak, version 1.0.7\r\
\n# If you use this script, or edit and\r\
\n# re-use it, please keep the header intact.\r\
\n#\r\
\n# For more information and details about\r\
\n# this script please visit the wiki page at\r\
\n# http://wiki.mikrotik.com/wiki/Failover_Scripting\r\
\n# ------------------- header -------------------\r\
\n\r\
\n\r\
\n\r\
\n# ------------- start editing here -------------\r\
\n# Edit the variables below to suit your needs\r\
\n\r\
\n# Please fill the WAN interface names\r\
\n:local InterfaceISP1 ISP-Virgin\r\
\n:local InterfaceISP2 ISP-BT\r\
\n\r\
\n# Please fill the gateway IPs (or interface names in case of PPP)\r\
\n:global GatewayISP1 82.17.140.1\r\
\n:global GatewayISP2 ISP-BT\r\
\n\r\
\n# Please fill the ping check host - currently: resolver1.opendns.com\r\
\n:local PingTarget 82.17.140.1\r\
\n\r\
\n\r\
\n# Please fill how many ping failures are allowed before fail-over happen\
ds\r\
\n:local FailTreshold 3\r\
\n\r\
\n# Define the distance increase of a route when it fails\r\
\n:local DistanceIncrease 5\r\
\n\r\
\n# Editing the script after this point may break it\r\
\n# -------------- stop editing here --------------\r\
\n\r\
\n\r\
\n\r\
\n# Declare the global variables\r\
\n:global PingFailCountISP1\r\
\n:global PingFailCountISP2\r\
\n\r\
\n#set PingFailCountISP1 0\r\
\n#set PingFailCountISP2 0\r\
\n\r\
\n# This inicializes the PingFailCount variables, in case this is the 1st \
time the script has ran\r\
\n:if ([:typeof \$PingFailCountISP1] = \"nothing\") do={:set PingFailCount\
ISP1 0}\r\
\n:if ([:typeof \$PingFailCountISP2] = \"nothing\") do={:set PingFailCount\
ISP2 0}\r\
\n\r\
\n# This variable will be used to keep results of individual ping attempts\
\r\
\n:local PingResult\r\
\n\r\
\n\r\
\n\r\
\n# Check ISP1\r\
\n:set PingResult [ping \$PingTarget count=1]\r\
\n:put \$PingResult\r\
\n\r\
\n:if (\$PingResult = 0) do={\r\
\n\t:if (\$PingFailCountISP1 < (\$FailTreshold+2)) do={\r\
\n\t\t:set PingFailCountISP1 (\$PingFailCountISP1 + 1)\r\
\n\t\t\r\
\n\t\t:if (\$PingFailCountISP1 = \$FailTreshold) do={\r\
\n\t\t\t:log warning \"ISP1 has a problem en route to \$PingTarget - incre\
asing distance of routes.\"\r\
\n\t\t\t:foreach i in=[/ip route find gateway=\$GatewayISP1 && static] do=\
\\\r\
\n\t\t\t\t{/ip route set \$i distance=\$DistanceIncrease}\r\
\n\r\
\n\t\t\t{ :local cons [/ip firewall connection find protocol=\"udp\"]; :if\
\_([:len \$cons] > 0) do={ /ip firewall connection remove numbers=\$cons;}\
}\r\
\n\t\t\t{ :local cons [/ip firewall connection find protocol=\"tcp\"]; :if\
\_([:len \$cons] > 0) do={ /ip firewall connection remove numbers=\$cons;}\
}\r\
\n\t\t\t{ :local cons [/ip firewall connection find protocol=\"rtp\"]; :if\
\_([:len \$cons] > 0) do={ /ip firewall connection remove numbers=\$cons;}\
}\r\
\n\t\t\t{ :local cons [/ip firewall connection find protocol=\"icmp\"]; :i\
f ([:len \$cons] > 0) do={ /ip firewall connection remove numbers=\$cons;}\
}\r\
\n\r\
\n\t\t\t:log warning \"Route distance increase finished.\"\r\
\n\t\t}\r\
\n\t}\r\
\n}\r\
\n:if (\$PingResult = 1) do={\r\
\n\t:if (\$PingFailCountISP1 > 0) do={\r\
\n\t\t:set PingFailCountISP1 (\$PingFailCountISP1 - 1)\r\
\n\t\t\r\
\n\t\t:if (\$PingFailCountISP1 = (\$FailTreshold -1)) do={\r\
\n\t\t\t:log warning \"ISP1 can reach \$PingTarget again - bringing back o\
riginal distance of routes.\"\r\
\n\t\t\t:foreach i in=[/ip route find gateway=\$GatewayISP1 && static] do=\
\\\r\
\n\t\t\t\t{/ip route set \$i distance=1}\r\
\n\r\
\n\t\t\t{ :local cons [/ip firewall connection find protocol=\"udp\"]; :if\
\_([:len \$cons] > 0) do={ /ip firewall connection remove numbers=\$cons;}\
}\r\
\n\t\t\t{ :local cons [/ip firewall connection find protocol=\"tcp\"]; :if\
\_([:len \$cons] > 0) do={ /ip firewall connection remove numbers=\$cons;}\
}\r\
\n\t\t\t{ :local cons [/ip firewall connection find protocol=\"rtp\"]; :if\
\_([:len \$cons] > 0) do={ /ip firewall connection remove numbers=\$cons;}\
}\r\
\n\t\t\t{ :local cons [/ip firewall connection find protocol=\"icmp\"]; :i\
f ([:len \$cons] > 0) do={ /ip firewall connection remove numbers=\$cons;}\
}\r\
\n\r\
\n\t\t\t:log warning \"Route distance decrease finished.\"\r\
\n\t\t}\r\
\n\t}\r\
\n}\r\
\n"
add dont-require-permissions=no name=ResetFailover owner=admin policy=\
ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon source="#\
\_Declare the global variables\r\
\n:global PingFailCountISP1\r\
\n:global PingFailCountISP2\r\
\n:global GatewayISP1\r\
\n:global GatewayISP2\r\
\n\r\
\n# Reset Variables\r\
\nset PingFailCountISP1 0\r\
\nset PingFailCountISP2 0\r\
\n\r\
\n# Reset Routes\r\
\n:foreach i in=[/ip route find gateway=\$GatewayISP1 && static] do=\\\r\
\n {/ip route set \$i distance=1}\r\
\n\r\
\n# Reset Connections\r\
\n{ :local cons [/ip firewall connection find protocol=\"udp\"]; :if ([:le\
n \$cons] > 0) do={ /ip firewall connection remove numbers=\$cons;}}\r\
\n{ :local cons [/ip firewall connection find protocol=\"tcp\"]; :if ([:le\
n \$cons] > 0) do={ /ip firewall connection remove numbers=\$cons;}}\r\
\n{ :local cons [/ip firewall connection find protocol=\"rtp\"]; :if ([:le\
n \$cons] > 0) do={ /ip firewall connection remove numbers=\$cons;}}\r\
\n{ :local cons [/ip firewall connection find protocol=\"icmp\"]; :if ([:l\
en \$cons] > 0) do={ /ip firewall connection remove numbers=\$cons;}}\r\
\n\r\
\n:log warning \"Route Failover Reset.\""
/tool e-mail
set address=smtp.office365.com from=andrew@finexlondon.com port=587 \
start-tls=yes user=andrew@finexlondon.com
/tool netwatch
add down-script="tool e-mail send to=andrew@finexlondon.com subject=\
\"Virgin DOWN\"" host=8.8.8.8 up-script="tool e-mail send to=andrew@finexlondon.com subject=\"Virgin UP\""

Below is the remote CCR where everything works:

# may/14/2020 13:16:03 by RouterOS 6.46.4
# software id = 2Y8R-F05B
#
# model = CCR1016-12G
# serial number = 42D402FA6367
/interface bridge
add fast-forward=no mtu=1500 name=bridge-lan protocol-mode=none
/interface ethernet
set [ find default-name=ether1 ] comment="OTE Router - NOT Modem Mode" l2mtu=\
1590 name=ether1-isp1 speed=100Mbps
set [ find default-name=ether2 ] speed=100Mbps
set [ find default-name=ether3 ] speed=100Mbps
set [ find default-name=ether4 ] speed=100Mbps
set [ find default-name=ether5 ] speed=100Mbps
set [ find default-name=ether6 ] speed=100Mbps
set [ find default-name=ether7 ] speed=100Mbps
set [ find default-name=ether8 ] speed=100Mbps
set [ find default-name=ether9 ] speed=100Mbps
set [ find default-name=ether10 ] speed=100Mbps
set [ find default-name=ether11 ] speed=100Mbps
set [ find default-name=ether12 ] speed=100Mbps
/interface l2tp-client
add add-default-route=yes comment=\
"Virgin VPN Connection - change only Virgin external IP" connect-to=\
82.17.140.24 disabled=no name=l2tp-out1 user=tinos-l2tp
add add-default-route=yes comment=\
"BT VPN Connection - change only BT external IP" connect-to=\
109.145.13.198 default-route-distance=2 name=l2tp-out2 user=tinos-l2tp
/ip pool
add name=pool-tinos ranges=172.16.110.10-172.16.110.200
/ip dhcp-server
add address-pool=pool-tinos authoritative=after-2sec-delay disabled=no \
interface=bridge-lan lease-time=1d name=dhcp-tinos
/interface bridge port
add bridge=bridge-lan hw=no interface=ether2
add bridge=bridge-lan hw=no interface=ether3
add bridge=bridge-lan hw=no interface=ether4
add bridge=bridge-lan hw=no interface=ether5
add bridge=bridge-lan hw=no interface=ether6
add bridge=bridge-lan hw=no interface=ether7
add bridge=bridge-lan hw=no interface=ether8
add bridge=bridge-lan hw=no interface=ether9
add bridge=bridge-lan hw=no interface=ether10
/ip firewall connection tracking
set udp-stream-timeout=30s
/ipv6 settings
set max-neighbor-entries=1024
/ip address
add address=172.16.110.1/24 interface=bridge-lan network=172.16.110.0
add address=192.168.1.2/24 interface=ether1-isp1 network=192.168.1.0
/ip dhcp-server network
add address=172.16.110.0/24 dns-server=1.1.1.1 gateway=172.16.110.1
/ip dns
set servers=1.1.1.1
/ip firewall filter
add action=accept chain=input comment=\
"Change to Malford/Virgin/BT external IP address" src-address=\
82.17.140.24
add action=accept chain=forward
add action=accept chain=input src-address=172.16.110.0/24
add action=drop chain=input connection-state=invalid
add action=accept chain=forward dst-address=172.16.110.0/24 src-address=\
172.16.105.0/24
add action=accept chain=forward dst-address=172.16.105.0/24 src-address=\
172.16.110.0/24
/ip firewall mangle
add action=change-mss chain=forward comment=\
"Rule to change MTU size to 105.x try not to use" disabled=yes \
dst-address=172.16.105.0/24 new-mss=1442 passthrough=yes protocol=tcp \
tcp-flags=syn
add action=mark-routing chain=prerouting new-routing-mark=VOIP passthrough=\
yes src-address=172.16.110.125
/ip firewall nat
add action=accept chain=srcnat dst-address=172.16.105.0/24 src-address=\
172.16.110.0/24
add action=accept chain=srcnat dst-address=172.16.100.0/24 src-address=\
172.16.110.0/24
add action=masquerade chain=srcnat src-address=172.16.110.0/24
add action=dst-nat chain=dstnat disabled=yes dst-port=8443 protocol=tcp \
src-address=82.17.140.24 to-addresses=172.16.110.125 to-ports=443
/ip firewall service-port
set ftp disabled=yes
set tftp disabled=yes
set irc disabled=yes
set h323 disabled=yes
set sip disabled=yes
set udplite disabled=yes
set dccp disabled=yes
set sctp disabled=yes
/ip route
add distance=1 gateway=192.168.1.1 routing-mark=VOIP
add comment="Main connection in case of L2TP IP Address Change" distance=3 \
gateway=192.168.1.1
/ip service
set telnet disabled=yes
set ftp disabled=yes
set www disabled=yes
set ssh disabled=yes
set api disabled=yes
set winbox address=172.16.0.0/16
set api-ssl disabled=yes
/ip ssh
set allow-none-crypto=yes forwarding-enabled=remote
/ipv6 nd
set [ find default=yes ] advertise-dns=no
/lcd
set default-screen=stats time-interval=hour
/lcd interface
set ether2 disabled=yes
set ether3 disabled=yes
set ether4 disabled=yes
set ether5 disabled=yes
set ether6 disabled=yes
set ether7 disabled=yes
set ether8 disabled=yes
set ether9 disabled=yes
set ether10 disabled=yes
set ether11 disabled=yes
set ether12 disabled=yes
/lcd screen
set 1 disabled=yes
set 2 disabled=yes
set 3 disabled=yes
set 4 disabled=yes
set 5 disabled=yes
/system clock
set time-zone-name=Europe/London
/system health
set fan-mode=manual
/system identity
set name=Tinos1016
/system leds
add disabled=yes leds=user-led type=on
Much obliged.

Andrew.
 
User avatar
mutluit
Forum Veteran
Forum Veteran
Posts: 821
Joined: Wed Mar 25, 2020 4:04 am

Re: Unable to update CCR

Thu May 14, 2020 5:20 pm

Hi, can you please also do this in the CLI of the two devices and post the output:
/tool traceroute duration=15 download.mikrotik.com
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Thu May 14, 2020 6:07 pm

Hi, can you please also do this in the CLI of the two devices and post the output:
/tool traceroute duration=15 download.mikrotik.com
No Update CCR:
[admin@Malford1016] > /tool traceroute duration=15 download.mikrotik.com
# ADDRESS LOSS SENT LAST AVG BEST WORST STD-DEV STATUS
1 100% 2 timeout
2 62.255.109.1 0% 2 287.6ms 238.2 188.7 287.6 49.5
3 100% 2 timeout
4 100% 2 timeout
5 62.254.42.174 0% 2 23.3ms 77 23.3 130.6 53.7
6 100% 2 timeout
7 100% 1 timeout
8 213.248.84.25 0% 1 69ms 69 69 69 0
9 100% 1 timeout
10 62.115.122.160 0% 1 68.3ms 68.3 68.3 68.3 0 <MPLS:L=682,E=0>
11 62.115.115.59 0% 1 126.1ms 126.1 126.1 126.1 0
12 62.115.136.79 0% 1 189.9ms 189.9 189.9 189.9 0
13 213.248.84.33 0% 1 120.9ms 120.9 120.9 120.9 0
14 100% 1 timeout
15 100% 1 timeout
16 100% 1 timeout
17 159.148.147.204 0% 1 183.7ms 183.7 183.7 183.7 0

[admin@Malford1016] >

Working remote CCR (notice first hop is into the CCR that doesn't update!!!!)

[admin@Tinos1016] > /tool traceroute duration=15 download.mikrotik.com
# ADDRESS LOSS SENT LAST AVG BEST WORST STD-DEV STATUS
1 172.16.105.1 0% 2 213.5ms 154.2 94.9 213.5 59.3
2 100% 2 timeout
3 62.255.109.1 0% 2 263.3ms 364.8 263.3 466.2 101.5
4 100% 2 timeout
5 100% 2 timeout
6 62.254.42.174 0% 1 310ms 310 310 310 0
7 100% 1 timeout
8 84.116.136.102 0% 1 254.2ms 254.2 254.2 254.2 0
9 213.248.84.25 0% 1 488.9ms 488.9 488.9 488.9 0
10 62.115.114.234 0% 1 494.5ms 494.5 494.5 494.5 0 <MPLS:L=1722,E=0>
11 80.91.249.11 0% 1 342.9ms 342.9 342.9 342.9 0
12 80.91.249.9 0% 1 337.3ms 337.3 337.3 337.3 0
13 62.115.136.77 0% 1 501.8ms 501.8 501.8 501.8 0
14 213.248.84.33 0% 1 733.1ms 733.1 733.1 733.1 0
15 100% 1 timeout
16 100% 1 timeout
17 100% 1 timeout
18 159.148.172.226 0% 1 223.7ms 223.7 223.7 223.7 0

[admin@Tinos1016] >
Cheers,

Andrew.
 
User avatar
mutluit
Forum Veteran
Forum Veteran
Posts: 821
Joined: Wed Mar 25, 2020 4:04 am

Re: Unable to update CCR

Thu May 14, 2020 7:43 pm

Andrew, you have an advanced, not to say complicated :-), network environment.
I'm not sure but do you think it could have to do with the following:
/ppp secret
add local-address=172.16.105.1 name=tinos-l2tp remote-address=10.10.20.10 \
service=l2tp
According to this thread viewtopic.php?t=131615 :
you can set any address to local address, no problem. you can set local address same as remote address"

Same with the 2nd entry at this location (btw, the distance of the 2nd should be set to a higher value, IMO):
/ip route
add distance=1 gateway=82.17.140.1
add comment="Tinos via SSTP" distance=1 dst-address=172.16.110.0/24 gateway=\
10.10.20.10

Can you in the GUI temporarily disable the 2 entries and see what happens?
I'll continue looking in the cfg.

Maybe some more people with more experience than me could take a look in to this case.

Btw, can you confirm that 82.17.140.1 is indeed your gateway? Or is this perhaps a misconfiguration? B/c the traceroute did not list that IP...
 
User avatar
mutluit
Forum Veteran
Forum Veteran
Posts: 821
Joined: Wed Mar 25, 2020 4:04 am

Re: Unable to update CCR

Thu May 14, 2020 8:44 pm

Andrew, you seem to have multiple ISPs, ie. multiple WAN connections (ISP-Virgin, ISP-BT). Are both or just one active?
I think best would be if you could make a simple drawing of your network structure and attach it to your posting.
 
User avatar
mutluit
Forum Veteran
Forum Veteran
Posts: 821
Joined: Wed Mar 25, 2020 4:04 am

Re: Unable to update CCR

Thu May 14, 2020 9:18 pm

You have this one:
add action=accept chain=forward connection-state=established,related,new

I would add also the following before or after the above one:
add action=accept chain=input connection-state=established,related,new
Maybe without the ",new" at the end.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Thu May 14, 2020 9:40 pm

Andrew, you seem to have multiple ISPs, ie. multiple WAN connections (ISP-Virgin, ISP-BT). Are both or just one active?
I think best would be if you could make a simple drawing of your network structure and attach it to your posting.
Indeed I have two ISPs in failover. One L2TP connection to a remote location. There are two guest networks 10.x and 20.x which are physically isolated.

I have tried either ISP what baffles me is that the remote CCR connects first to the faulty CCR and then to Mikrotik with no issues, yet the faulty one doesn't:

Remote CCR is 172.16.110.1 Local CCR is 172.16.105.1

On the traceroute the remote CCR connect to x.105.1 then to exactly the same hops as the traceroute initiated from 105.1 itself.

How can this possible be???

I am very confused.

With regard to the last rule you spoke about I saw no change but also little or no activity.

Thanks.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Fri May 15, 2020 1:10 am

On the traceroute the remote CCR connect to x.105.1 then to exactly the same hops as the traceroute initiated from 105.1 itself.

How can this possible be???

I am very confused.
The reason of that remarkable difference is that your firewall on the problematic CCR is very unusual, to put it really softly.

You have no /ip firewall filter rules whatsoever for chain=forward of the problematic CCR, which means that it sets no limit on what the other CCR, or whatever else connected via this problematic CCR, can send and/or receive.

But the rules in chain=input of your /ip firewall filter, which handle incoming traffic to the problematic CCR itself, drop almost everything (with a few exceptions like icmp) that arrives via WANs, so no surprise that any download fails.

The way your chain=input is currently configured, you do not use the benefits of the stateful firewall concept, where packets belonging to already existing connections are handled by a single rule action=accept connection-state=established,related placed as the very first one in the chain, and the decision whether to permit or deny a given connection as a whole is taken when handling its initial packet. So even if you router itself initiates a new connection by sending a packet to some remote server, which it is free to do because there are no rules in chain=output of /ip firewall filter, the response packet from the remote server, which makes the state of that connection change from new to established, is later dropped because it matches none of the rules in chain=input of your /ip firewall filter except one of the last two ones ("drop whatever has arrived via WAN x").

Do you have any special reason not to use the stateful firewall approach? If not, just insert chain=input connection-state=established,related action=accept as the first rule in chain=input, and you'll be good.
 
User avatar
mutluit
Forum Veteran
Forum Veteran
Posts: 821
Joined: Wed Mar 25, 2020 4:04 am

Re: Unable to update CCR

Fri May 15, 2020 3:19 am

Andrew, as @sindy's analysis shows, you should insert these 3 rules at the right locations
in their respective chains, ie. after the specific drops, but before the final drop in each of the 3 chains, if present:

add action=accept chain=input connection-state=established,related
add action=accept chain=output connection-state=established,related,new
add action=accept chain=forward connection-state=established,related,new

If it still does not work, then try also with ".new" at end also in the input chain.

See also this very useful posting of @sindy on how to organize the firewall rules:
viewtopic.php?f=2&t=130419&start=50#p642323

And: see also your log file (via GUI) for any errors/warnings.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Fri May 15, 2020 9:25 am

you should insert these 3 rules at the right locations
in their respective chains, ie. after the specific drops, but before the final drop in each of the 3 chains, if present:

add action=accept chain=input connection-state=established,related
add action=accept chain=output connection-state=established,related,new
add action=accept chain=forward connection-state=established,related,new

If it still does not work, then try also with ".new" at end also in the input chain.
No.
accept (established or related or new) before those drop anything from WANs ones would effectively permit any connection in the respective chain, so it would invalidate the firewall functionality.

On the other hand, as long as there are currently no rules in chains forward and output, there is no need to add accept (established or related or new) to them, because the default handling of the chains is accept anyway, so adding such a rule would only slow down packet processing but would change nothing about the result.
 
User avatar
mutluit
Forum Veteran
Forum Veteran
Posts: 821
Joined: Wed Mar 25, 2020 4:04 am

Re: Unable to update CCR

Fri May 15, 2020 9:47 am

you should insert these 3 rules at the right locations
in their respective chains, ie. after the specific drops, but before the final drop in each of the 3 chains, if present:

add action=accept chain=input connection-state=established,related
add action=accept chain=output connection-state=established,related,new
add action=accept chain=forward connection-state=established,related,new

If it still does not work, then try also with ".new" at end also in the input chain.
No.
accept (established or related or new) before those drop anything from WANs ones would effectively permit any connection in the respective chain, so it would invalidate the firewall functionality.

Yes, I hink I understand what you mean: the connection-state=established,related,new would be wrong in following scheme, so it has to be w/o ",new":
add action=drop chain=forward dst-address=172.16.105.0/24 src-address=10.10.10.0/24
add action=drop chain=forward dst-address=172.16.105.0/24 src-address=20.20.20.0/24
add action=drop chain=forward dst-address=10.10.10.0/24 src-address=172.16.105.0/24
add action=accept chain=forward connection-state=established,related
... (add all those to accept)
add action=drop chain=forward comment="drop all other"
And "add action=accept chain=forward connection-state=established,related" can also safely be moved to the top of the chain.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Fri May 15, 2020 10:33 am

Do you have any special reason not to use the stateful firewall approach? If not, just insert chain=input connection-state=established,related action=accept as the first rule in chain=input, and you'll be good.
Thanks Sindy, no particular reason. I tried a couple of people to help build the config on the CCRs but both "experts" left me with big bills and big problems. Then I found a kind soul who helped me a lot but as he only did it to help I couldn't ask him for too much and he got the failover working really well, email doesn't always work but failover does really well.

Just checking the following rules block ANY traffic between 10.10.x.x and 172.16.x.x, and 20.20.x.x and 172.16.x.x. These are my rules #0 to #2
add action=drop chain=forward dst-address=172.16.105.0/24 src-address=10.10.10.0/24
add action=drop chain=forward dst-address=172.16.105.0/24 src-address=20.20.20.0/24
add action=drop chain=forward dst-address=10.10.10.0/24 src-address=172.16.105.0/24
If so shouldn't there be a 4th rule? ie #3
add action=drop chain=forward dst-address=172.16.105.0/24 src-address=10.10.10.0/24
add action=drop chain=forward dst-address=172.16.105.0/24 src-address=20.20.20.0/24
add action=drop chain=forward dst-address=10.10.10.0/24 src-address=172.16.105.0/24
add action=drop chain=forward dst-address=20.20.20.0/24 src-address=172.16.105.0/24
If the above is correct I should add as Rule #4 and #5. confusingly the 5th and 6th rule
add action=accept chain=forward connection-state=established,related,new
add action=drop chain=forward comment="drop all other"
However starting to read the thread Mutluit pointed me to my Rules look very different to what is suggested.

Thanks,

Andrew.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Fri May 15, 2020 10:53 am

See also this very useful posting of @sindy on how to organize the firewall rules:
viewtopic.php?f=2&t=130419&start=50#p642323
I'm reading the post you suggested, and I am starting to see. So I can happily deal with those 4 rules at the highest order, the 4 that stop traffic between segregated LANs. Then normal service resumes:
add chain=input connection-state=established,related action=accept
add chain=input connection-state=invalid action=drop
Does this make sense?

Thanks.
 
User avatar
mutluit
Forum Veteran
Forum Veteran
Posts: 821
Joined: Wed Mar 25, 2020 4:04 am

Re: Unable to update CCR

Fri May 15, 2020 11:21 am

See also this very useful posting of @sindy on how to organize the firewall rules:
viewtopic.php?f=2&t=130419&start=50#p642323
I'm reading the post you suggested, and I am starting to see. So I can happily deal with those 4 rules at the highest order, the 4 that stop traffic between segregated LANs. Then normal service resumes:
add chain=input connection-state=established,related action=accept
add chain=input connection-state=invalid action=drop
Does this make sense?
Yes.
But see also the beforementioned general scheme for each chain.
Read also this info on the 3 built-in chains (input, output, forward) https://wiki.mikrotik.com/wiki/Manual:I ... ter#Chains

Btw, on my CRS device I'm primarily using the "Hardware Firewall" (ACL on ASIC chip) b/c of performance reasons. It does not have any connection-states as it's a stateless firewall. But it is possible to redirect a packet to the normal stateful firewall(s) (dubbed "CPU firewall") like that under "/ip firewall filter".
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Fri May 15, 2020 11:55 am

So I can happily deal with those 4 rules at the highest order, the 4 that stop traffic between segregated LANs.
No. The accept established or related rule should be the very first one in the chain (except very specific cases), because it won't allow any packet not belonging to a connection (flow) whose first ever packet was not accepted by one of the subsequent rules (which it has reached because the accept established or related didn't match it). So all mid-connection packets will be handled by that single rule, and only the initial packet of each connection will be handled by one of the subsequent rules. If the first one is dropped, the subsequent ones will never come (some UDP flows may continue coming anyway but will be dropped too, by the same rule which has dropped the initial one).

It is critical for the overall routing performance to force most packets through as few firewall rules as possible.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Fri May 15, 2020 1:13 pm

No. The accept established or related rule should be the very first one in the chain (except very specific cases),
Understood, and I have created and moved that rule to position #0 and low and behold, Updates and SNTP work!!
It is critical for the overall routing performance to force most packets through as few firewall rules as possible.
Is it normal to have this many firewall rules?
/ip firewall filter
add action=accept chain=input connection-state=established,related
add action=drop chain=forward dst-address=172.16.105.0/24 src-address=\
10.10.10.0/24
add action=drop chain=forward dst-address=172.16.105.0/24 src-address=\
20.20.20.0/24
add action=drop chain=forward dst-address=10.10.10.0/24 src-address=\
172.16.105.0/24
add action=drop chain=forward dst-address=20.20.20.0/24 src-address=\
172.16.105.0/24
add action=accept chain=input src-address-list=allowed
add action=accept chain=input protocol=icmp
add action=accept chain=forward connection-state=established,related,new
add action=accept chain=input dst-port=1701 protocol=udp
add action=accept chain=input dst-port=500 protocol=udp
add action=accept chain=input dst-port=4500 protocol=udp
add action=accept chain=input disabled=yes protocol=ipsec-ah
add action=accept chain=input disabled=yes protocol=ipsec-esp
add action=drop chain=input dst-port=53 in-interface=ISP-Virgin protocol=tcp
add action=drop chain=input dst-port=53 in-interface=ISP-Virgin protocol=udp
add action=drop chain=input dst-port=53 in-interface=ISP-BT protocol=tcp
add action=drop chain=input dst-port=53 in-interface=ISP-BT protocol=udp
add action=drop chain=input connection-state=invalid in-interface=ISP-Virgin
add action=drop chain=input connection-state=invalid in-interface=ISP-BT
add action=drop chain=input in-interface=ISP-Virgin
add action=drop chain=input in-interface=ISP-BT
Thanks.
 
User avatar
mutluit
Forum Veteran
Forum Veteran
Posts: 821
Joined: Wed Mar 25, 2020 4:04 am

Re: Unable to update CCR

Fri May 15, 2020 2:03 pm

Then the reason was the following:
Your previous firewall rules in the input chain did allow input traffic from those addresses in the "allowed" address list
OR icmp traffic OR traffic to some select ports (1701, 500, 4500), else it dropped all other incoming packets.
Ie. this is about traffic to the device itself only (ie. the input chain), not traffic through the device (ie. the forward chain).

Hmm. IMO there's still a mystery in this setup. Maybe @sindy can tell more.

But, glad to see it now works.

Btw, you should group your firewall rules by the chain.

You can delete the following rules as they are redundant:
add action=drop chain=input dst-port=53 in-interface=ISP-Virgin protocol=tcp
add action=drop chain=input dst-port=53 in-interface=ISP-Virgin protocol=udp
add action=drop chain=input dst-port=53 in-interface=ISP-BT protocol=tcp
add action=drop chain=input dst-port=53 in-interface=ISP-BT protocol=udp
add action=drop chain=input connection-state=invalid in-interface=ISP-Virgin
add action=drop chain=input connection-state=invalid in-interface=ISP-BT

And check whether you need these two currently disabled rules:
add action=accept chain=input disabled=yes protocol=ipsec-ah
add action=accept chain=input disabled=yes protocol=ipsec-esp
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Fri May 15, 2020 2:26 pm

Is it normal to have this many firewall rules?
It is normal to have as many rules as you really need. In absolute figures it may mean five or five thousand depending on the role of your router in the network.

As it looked yesterday, the router could be managed from any device in one of its LAN subnets, and itself it was basically unable to do anything but ping via any of the WANs. It wasn't possible to manage it from WAN (definitely good). It didn't limit any transit connections ( (V)LAN <-> (V)LAN or (V)LAN <-> WAN) because chain forward in firewall table filter was totally empty.

To get a tailored advice, you have to provide sufficient information about the network topology (a photo of a handmade drawing is sufficient) and specify the required firewall rules in plain English - at best by specifying what needs to be allowed, so that the rest could be dropped.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Fri May 15, 2020 3:38 pm

To get a tailored advice, you have to provide sufficient information about the network topology (a photo of a handmade drawing is sufficient) and specify the required firewall rules in plain English - at best by specifying what needs to be allowed, so that the rest could be dropped.
I do not have any specifics other than security, we do not host any services nor broadcast. Only different devices are our SIP Phones. The connection to the remote CCR has been a problem, tried a number of protocols and then chose L2TP, although throughput is very low compared to its theoretical limit. (Upload of Sender - Download of Receiver)

Does this picture help:
Network.png
You do not have the required permissions to view the files attached to this post.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Fri May 15, 2020 3:57 pm

So if I get you right, the sole purpose of the two CCRs is to interconnect the two sites using a VPN, and no firewall functionality between internet and the LANs is required (if really not but devices in the LANs should have internet access, how else is protection of these networks provided, do you rely on NAT alone)?
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Fri May 15, 2020 4:12 pm

So if I get you right, the sole purpose of the two CCRs is to interconnect the two sites using a VPN, and no firewall functionality between internet and the LANs is required (if really not but devices in the LANs should have internet access, how else is protection of these networks provided, do you rely on NAT alone)?
Definitely Internet access, remote CCR accesses Internet via the local CCR, if L2TP fails it accesses Internet directly

Local CCR accesses Internet directly.

Both would need full Internet firewalls.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Mon May 18, 2020 10:11 pm

So if I get you right, the sole purpose of the two CCRs is to interconnect the two sites using a VPN, and no firewall functionality between internet and the LANs is required (if really not but devices in the LANs should have internet access, how else is protection of these networks provided, do you rely on NAT alone)?
Sindy? Are you able to help me please.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Mon May 18, 2020 10:35 pm

Likely yes, but it won't be within hours.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Mon May 18, 2020 10:41 pm

Likely yes, but it won't be within hours.
Brilliant, I can wait. One thing the addition of the first rule:
add action=accept chain=input connection-state=established,related
Doesn't expose the network?

Thanks.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Mon May 18, 2020 10:45 pm

You can delete the following rules as they are redundant:
And check whether you need these two currently disabled rules:
add action=accept chain=input disabled=yes protocol=ipsec-ah
add action=accept chain=input disabled=yes protocol=ipsec-esp
I'm not left with much!!! How can I check that nothing untoword can get through?

Thanks,

Andrew.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Mon May 18, 2020 10:58 pm

How can I check that nothing untoword can get through?
You can't check. You can only set up firewall rules which will prevent that, but bear in mind that L3 firewall rules only prevent a certain kind of malware from actively connecting to your network; other kind of malware, which relies on downloads from infected web servers as people click on phishing e-mails or people opening e-mail attachments, cannot be stopped by a firewall which doesn't analyze the payload. And analysing payload requires deciphering HTTPS communication using MITM techniques. Most anti-virus software does that, which brings other kind of security concerns (passwords are often transported in plaintext if HTTPS is used, assuming that the whole channel is encrypted by TLS, whereas this security software can see the TLS payload in plaintext). Also cross-platform malware exists, which gets downloaded to your PCs but attacks your router (or SIP phones) from LAN side. So you shouldn't enable plaintext management services, such as telnet or http, even for access from LAN. And even if you don't, there are keyloggers which record the passwords you fill in to secure clients...

Post the export of the current configurations of both routers following the hint in my automatic signature below so that I could give you some lines of configuration scripts to add or modify the firewall rules.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Tue May 19, 2020 12:03 am

So you shouldn't enable plaintext management services, such as telnet or http, even for access from LAN.
Does PuTTY use the same principals as Telnet? I use that quite often to manage a few headless Raspberry Pi. Can HTTP simply not be silenced at the network level? My greatest fear is quite honestly loosing family pictures, almost everything else is backed up to a cloud service.

Local CCR
# may/18/2020 21:54:29 by RouterOS 6.46.6
# software id = PZJX-KJ04
#
# model = CCR1016-12G
# serial number = 42D402B557C9
/interface bridge
add fast-forward=no name=bridge-lan
add name=bridgeDRGuest
add name=bridgeGuest
/interface ethernet
set [ find default-name=ether2 ] name=BT
set [ find default-name=ether1 ] comment=\
"Virgin Modem Mode connection to DHCP Client" name=ISP-Virgin
/interface pppoe-client
add add-default-route=yes comment="BT PPPoE Client Modem Mode" \
default-route-distance=3 disabled=no interface=BT name=ISP-BT user=\
bthomehub@btbroadband.com
/ip pool
add name=pool-malford ranges=172.16.105.90-172.16.105.129
add name=pGuest ranges=10.10.10.10-10.10.10.100
add name=pDRGuest ranges=20.20.20.20-20.20.20.100
/ip dhcp-server
add address-pool=pool-malford interface=bridge-lan lease-time=1d name=\
dhcp-lan
add address-pool=pGuest disabled=no interface=bridgeGuest name=DHCPGuest
add address-pool=pDRGuest disabled=no interface=bridgeDRGuest name=\
DHCPDRGUEST
/system logging action
set 1 disk-file-name=flash/log
add email-to=andrew.gebhardt@finexlondon.com name=sendemail target=email
/user group
set full policy="local,telnet,ssh,ftp,reboot,read,write,policy,test,winbox,pas\
sword,web,sniff,sensitive,api,romon,dude,tikapp"
/interface bridge port
add bridge=bridge-lan interface=ether3
add bridge=bridge-lan interface=ether4
add bridge=bridge-lan interface=ether5
add bridge=bridge-lan interface=ether6
add bridge=bridge-lan interface=ether7
add bridge=bridge-lan interface=ether8
add bridge=bridge-lan interface=ether9
add bridge=bridge-lan interface=ether10
add bridge=bridgeDRGuest interface=ether11
add bridge=bridgeGuest interface=ether12
/ip settings
set tcp-syncookies=yes
/interface l2tp-server server
set enabled=yes
/interface pptp-server server
set enabled=yes
/interface sstp-server server
set certificate=Server default-profile=default-encryption port=8443 \
tls-version=only-1.2
/ip address
add address=172.16.105.1/24 interface=bridge-lan network=172.16.105.0
add address=192.168.1.2/24 interface=BT network=192.168.1.0
add address=192.168.2.2/24 disabled=yes interface=BT network=192.168.2.0
add address=10.10.10.254/24 interface=bridgeGuest network=10.10.10.0
add address=20.20.20.254/24 interface=bridgeDRGuest network=20.20.20.0
/ip cloud
set ddns-update-interval=2m
/ip dhcp-client
add add-default-route=no comment=\
"DHCP Client for Virgin Router in Modem Mode" disabled=no interface=\
ISP-Virgin use-peer-dns=no
/ip dhcp-server network
add address=10.10.10.0/24 dns-server=1.1.1.1,1.0.0.1 gateway=10.10.10.254 \
netmask=24
add address=20.20.20.0/24 dns-server=8.8.8.8,8.8.4.4 gateway=20.20.20.254 \
netmask=24
add address=172.16.105.0/24
/ip dns
set servers=172.16.105.2
/ip firewall address-list
add address=172.16.105.0/24 list=allowed
add address=172.16.110.0/24 list=allowed
add address=2.87.174.170 disabled=yes list=allowed
add address=2.87.174.170 list=TinosIP
/ip firewall filter
add action=accept chain=input connection-state=established,related
add action=drop chain=forward dst-address=172.16.105.0/24 src-address=\
10.10.10.0/24
add action=drop chain=forward dst-address=172.16.105.0/24 src-address=\
20.20.20.0/24
add action=drop chain=forward dst-address=10.10.10.0/24 src-address=\
172.16.105.0/24
add action=drop chain=forward dst-address=20.20.20.0/24 src-address=\
172.16.105.0/24
add action=accept chain=input src-address-list=allowed
add action=accept chain=input protocol=icmp
add action=accept chain=forward connection-state=established,related,new
add action=accept chain=input dst-port=1701 protocol=udp
add action=accept chain=input dst-port=500 protocol=udp
add action=accept chain=input dst-port=4500 protocol=udp
add action=accept chain=input disabled=yes protocol=ipsec-ah
add action=accept chain=input disabled=yes protocol=ipsec-esp
add action=drop chain=input disabled=yes dst-port=53 in-interface=ISP-Virgin \
protocol=tcp
add action=drop chain=input disabled=yes dst-port=53 in-interface=ISP-Virgin \
protocol=udp
add action=drop chain=input disabled=yes dst-port=53 in-interface=ISP-BT \
protocol=tcp
add action=drop chain=input disabled=yes dst-port=53 in-interface=ISP-BT \
protocol=udp
add action=drop chain=input connection-state=invalid disabled=yes \
in-interface=ISP-Virgin
add action=drop chain=input connection-state=invalid disabled=yes \
in-interface=ISP-BT
add action=drop chain=input disabled=yes in-interface=ISP-Virgin
add action=drop chain=input disabled=yes in-interface=ISP-BT
/ip firewall mangle
add action=mark-routing chain=output disabled=yes log=yes new-routing-mark=\
virgin passthrough=no src-address=10.10.10.0/24
add action=change-mss chain=forward dst-address=172.16.110.0/24 new-mss=1442 \
passthrough=yes protocol=tcp tcp-flags=syn
add action=change-mss chain=forward new-mss=1470 passthrough=yes protocol=tcp \
tcp-flags=syn
/ip firewall nat
add action=masquerade chain=srcnat comment="NAT to Internet"
add action=dst-nat chain=dstnat disabled=yes dst-port=53 protocol=udp \
src-address-list=allowed to-addresses=172.16.105.2
add action=src-nat chain=srcnat disabled=yes dst-port=53 protocol=tcp \
to-addresses=172.16.105.2 to-ports=53
add action=src-nat chain=srcnat disabled=yes dst-port=53 protocol=udp \
to-addresses=172.16.105.2 to-ports=53
/ip firewall service-port
set ftp disabled=yes
set tftp disabled=yes
set irc disabled=yes
set h323 disabled=yes
set sip disabled=yes
set udplite disabled=yes
set dccp disabled=yes
set sctp disabled=yes
/ip route
add distance=1 gateway=TINOSGATEWAY
add comment="Tinos via SSTP" distance=1 dst-address=172.16.110.0/24 gateway=\
10.10.20.10
/ip service
set telnet disabled=yes
set ftp disabled=yes
set www disabled=yes
set ssh address=172.16.105.0/24,172.16.110.0/24
set api disabled=yes
set winbox address=172.16.105.0/24,172.16.110.0/24
set api-ssl disabled=yes
/ip ssh
set allow-none-crypto=yes forwarding-enabled=remote
/ip upnp
set enabled=yes show-dummy-rule=no
/ipv6 nd
set [ find default=yes ] advertise-dns=no
/ppp secret
add local-address=172.16.105.1 name=tinos-l2tp remote-address=10.10.20.10 \
service=l2tp
/queue simple
add disabled=yes max-limit=10M/10M name=queue-tinos target=*F
/system clock
set time-zone-name=Europe/London
/system identity
set name=Malford1016
/system logging
add action=sendemail topics=script
/system ntp client
set enabled=yes primary-ntp=130.159.196.118 secondary-ntp=178.79.152.182
/system scheduler
add interval=5s name=RunFailover on-event=FailoverScript policy=\
ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon \
start-date=jun/10/2019 start-time=14:09:11
/system script
add dont-require-permissions=no name=FailoverScript owner=admin policy=\
ftp,reboot,read,write,policy,test,password,sniff,sensitive source="# -----\
-------------- header -------------------\r\
\n# Script by Tomas Kirnak, version 1.0.7\r\
\n# If you use this script, or edit and\r\
\n# re-use it, please keep the header intact.\r\
\n#\r\
\n# For more information and details about\r\
\n# this script please visit the wiki page at\r\
\n# http://wiki.mikrotik.com/wiki/Failover_Scripting\r\
\n# ------------------- header -------------------\r\
\n\r\
\n\r\
\n\r\
\n# ------------- start editing here -------------\r\
\n# Edit the variables below to suit your needs\r\
\n\r\
\n# Please fill the WAN interface names\r\
\n:local InterfaceISP1 ISP-Virgin\r\
\n:local InterfaceISP2 ISP-BT\r\
\n\r\
\n# Please fill the gateway IPs (or interface names in case of PPP)\r\
\n:global GatewayISP1 82.17.140.1\r\
\n:global GatewayISP2 ISP-BT\r\
\n\r\
\n# Please fill the ping check host - currently: resolver1.opendns.com\r\
\n:local PingTarget 82.17.140.1\r\
\n\r\
\n\r\
\n# Please fill how many ping failures are allowed before fail-over happen\
ds\r\
\n:local FailTreshold 3\r\
\n\r\
\n# Define the distance increase of a route when it fails\r\
\n:local DistanceIncrease 5\r\
\n\r\
\n# Editing the script after this point may break it\r\
\n# -------------- stop editing here --------------\r\
\n\r\
\n\r\
\n\r\
\n# Declare the global variables\r\
\n:global PingFailCountISP1\r\
\n:global PingFailCountISP2\r\
\n\r\
\n#set PingFailCountISP1 0\r\
\n#set PingFailCountISP2 0\r\
\n\r\
\n# This inicializes the PingFailCount variables, in case this is the 1st \
time the script has ran\r\
\n:if ([:typeof \$PingFailCountISP1] = \"nothing\") do={:set PingFailCount\
ISP1 0}\r\
\n:if ([:typeof \$PingFailCountISP2] = \"nothing\") do={:set PingFailCount\
ISP2 0}\r\
\n\r\
\n# This variable will be used to keep results of individual ping attempts\
\r\
\n:local PingResult\r\
\n\r\
\n\r\
\n\r\
\n# Check ISP1\r\
\n:set PingResult [ping \$PingTarget count=1]\r\
\n:put \$PingResult\r\
\n\r\
\n:if (\$PingResult = 0) do={\r\
\n\t:if (\$PingFailCountISP1 < (\$FailTreshold+2)) do={\r\
\n\t\t:set PingFailCountISP1 (\$PingFailCountISP1 + 1)\r\
\n\t\t\r\
\n\t\t:if (\$PingFailCountISP1 = \$FailTreshold) do={\r\
\n\t\t\t:log warning \"ISP1 has a problem en route to \$PingTarget - incre\
asing distance of routes.\"\r\
\n\t\t\t:foreach i in=[/ip route find gateway=\$GatewayISP1 && static] do=\
\\\r\
\n\t\t\t\t{/ip route set \$i distance=\$DistanceIncrease}\r\
\n\r\
\n\t\t\t{ :local cons [/ip firewall connection find protocol=\"udp\"]; :if\
\_([:len \$cons] > 0) do={ /ip firewall connection remove numbers=\$cons;}\
}\r\
\n\t\t\t{ :local cons [/ip firewall connection find protocol=\"tcp\"]; :if\
\_([:len \$cons] > 0) do={ /ip firewall connection remove numbers=\$cons;}\
}\r\
\n\t\t\t{ :local cons [/ip firewall connection find protocol=\"rtp\"]; :if\
\_([:len \$cons] > 0) do={ /ip firewall connection remove numbers=\$cons;}\
}\r\
\n\t\t\t{ :local cons [/ip firewall connection find protocol=\"icmp\"]; :i\
f ([:len \$cons] > 0) do={ /ip firewall connection remove numbers=\$cons;}\
}\r\
\n\r\
\n\t\t\t:log warning \"Route distance increase finished.\"\r\
\n\t\t}\r\
\n\t}\r\
\n}\r\
\n:if (\$PingResult = 1) do={\r\
\n\t:if (\$PingFailCountISP1 > 0) do={\r\
\n\t\t:set PingFailCountISP1 (\$PingFailCountISP1 - 1)\r\
\n\t\t\r\
\n\t\t:if (\$PingFailCountISP1 = (\$FailTreshold -1)) do={\r\
\n\t\t\t:log warning \"ISP1 can reach \$PingTarget again - bringing back o\
riginal distance of routes.\"\r\
\n\t\t\t:foreach i in=[/ip route find gateway=\$GatewayISP1 && static] do=\
\\\r\
\n\t\t\t\t{/ip route set \$i distance=1}\r\
\n\r\
\n\t\t\t{ :local cons [/ip firewall connection find protocol=\"udp\"]; :if\
\_([:len \$cons] > 0) do={ /ip firewall connection remove numbers=\$cons;}\
}\r\
\n\t\t\t{ :local cons [/ip firewall connection find protocol=\"tcp\"]; :if\
\_([:len \$cons] > 0) do={ /ip firewall connection remove numbers=\$cons;}\
}\r\
\n\t\t\t{ :local cons [/ip firewall connection find protocol=\"rtp\"]; :if\
\_([:len \$cons] > 0) do={ /ip firewall connection remove numbers=\$cons;}\
}\r\
\n\t\t\t{ :local cons [/ip firewall connection find protocol=\"icmp\"]; :i\
f ([:len \$cons] > 0) do={ /ip firewall connection remove numbers=\$cons;}\
}\r\
\n\r\
\n\t\t\t:log warning \"Route distance decrease finished.\"\r\
\n\t\t}\r\
\n\t}\r\
\n}\r\
\n"
add dont-require-permissions=no name=ResetFailover owner=admin policy=\
ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon source="#\
\_Declare the global variables\r\
\n:global PingFailCountISP1\r\
\n:global PingFailCountISP2\r\
\n:global GatewayISP1\r\
\n:global GatewayISP2\r\
\n\r\
\n# Reset Variables\r\
\nset PingFailCountISP1 0\r\
\nset PingFailCountISP2 0\r\
\n\r\
\n# Reset Routes\r\
\n:foreach i in=[/ip route find gateway=\$GatewayISP1 && static] do=\\\r\
\n {/ip route set \$i distance=1}\r\
\n\r\
\n# Reset Connections\r\
\n{ :local cons [/ip firewall connection find protocol=\"udp\"]; :if ([:le\
n \$cons] > 0) do={ /ip firewall connection remove numbers=\$cons;}}\r\
\n{ :local cons [/ip firewall connection find protocol=\"tcp\"]; :if ([:le\
n \$cons] > 0) do={ /ip firewall connection remove numbers=\$cons;}}\r\
\n{ :local cons [/ip firewall connection find protocol=\"rtp\"]; :if ([:le\
n \$cons] > 0) do={ /ip firewall connection remove numbers=\$cons;}}\r\
\n{ :local cons [/ip firewall connection find protocol=\"icmp\"]; :if ([:l\
en \$cons] > 0) do={ /ip firewall connection remove numbers=\$cons;}}\r\
\n\r\
\n:log warning \"Route Failover Reset.\""
/tool e-mail
set address=smtp.office365.com from=andrew.gebhardt@finexlondon.com port=587 \
start-tls=yes user=andrew.gebhardt@finexlondon.com
/tool netwatch
add down-script="tool e-mail send to=andrew.gebhardt@finexlondon.com subject=\
\"Virgin DOWN\"" host=8.8.8.8 up-script="tool e-mail send to=andrew.gebhar\
dt@finexlondon.com subject=\"Virgin UP\""
Remote CCR
# may/18/2020 21:54:29 by RouterOS 6.46.4
# software id = 2Y8R-F05B
#
# model = CCR1016-12G
# serial number = 42D402FA6367
/interface bridge
add fast-forward=no mtu=1500 name=bridge-lan protocol-mode=none
/interface ethernet
set [ find default-name=ether1 ] comment="OTE Router - NOT Modem Mode" l2mtu=\
1590 name=ether1-isp1 speed=100Mbps
set [ find default-name=ether2 ] speed=100Mbps
set [ find default-name=ether3 ] speed=100Mbps
set [ find default-name=ether4 ] speed=100Mbps
set [ find default-name=ether5 ] speed=100Mbps
set [ find default-name=ether6 ] speed=100Mbps
set [ find default-name=ether7 ] speed=100Mbps
set [ find default-name=ether8 ] speed=100Mbps
set [ find default-name=ether9 ] speed=100Mbps
set [ find default-name=ether10 ] speed=100Mbps
set [ find default-name=ether11 ] speed=100Mbps
set [ find default-name=ether12 ] speed=100Mbps
/interface l2tp-client
add add-default-route=yes comment=\
"Virgin VPN Connection - change only Virgin external IP" connect-to=\
GATEWAY84 disabled=no name=l2tp-out1 user=tinos-l2tp
add add-default-route=yes comment=\
"BT VPN Connection - change only BT external IP" connect-to=\
GATEWAY198 default-route-distance=2 name=l2tp-out2 user=tinos-l2tp
/ip pool
add name=pool-tinos ranges=172.16.110.10-172.16.110.200
/ip dhcp-server
add address-pool=pool-tinos authoritative=after-2sec-delay disabled=no \
interface=bridge-lan lease-time=1d name=dhcp-tinos
/interface bridge port
add bridge=bridge-lan hw=no interface=ether2
add bridge=bridge-lan hw=no interface=ether3
add bridge=bridge-lan hw=no interface=ether4
add bridge=bridge-lan hw=no interface=ether5
add bridge=bridge-lan hw=no interface=ether6
add bridge=bridge-lan hw=no interface=ether7
add bridge=bridge-lan hw=no interface=ether8
add bridge=bridge-lan hw=no interface=ether9
add bridge=bridge-lan hw=no interface=ether10
/ip firewall connection tracking
set udp-stream-timeout=30s
/ipv6 settings
set max-neighbor-entries=1024
/ip address
add address=172.16.110.1/24 interface=bridge-lan network=172.16.110.0
add address=192.168.1.2/24 interface=ether1-isp1 network=192.168.1.0
/ip dhcp-server network
add address=172.16.110.0/24 dns-server=1.1.1.1 gateway=172.16.110.1
/ip dns
set servers=1.1.1.1
/ip firewall filter
add action=accept chain=input comment=\
"Change to Malford/Virgin/BT external IP address" src-address=\
GATEWAY84
add action=accept chain=forward
add action=accept chain=input src-address=172.16.110.0/24
add action=drop chain=input connection-state=invalid
add action=accept chain=forward dst-address=172.16.110.0/24 src-address=\
172.16.105.0/24
add action=accept chain=forward dst-address=172.16.105.0/24 src-address=\
172.16.110.0/24
/ip firewall mangle
add action=change-mss chain=forward comment=\
"Rule to change MTU size to 105.x try not to use" disabled=yes \
dst-address=172.16.105.0/24 new-mss=1442 passthrough=yes protocol=tcp \
tcp-flags=syn
add action=mark-routing chain=prerouting new-routing-mark=VOIP passthrough=\
yes src-address=172.16.110.125
/ip firewall nat
add action=accept chain=srcnat dst-address=172.16.105.0/24 src-address=\
172.16.110.0/24
add action=accept chain=srcnat dst-address=172.16.100.0/24 src-address=\
172.16.110.0/24
add action=masquerade chain=srcnat src-address=172.16.110.0/24
add action=dst-nat chain=dstnat disabled=yes dst-port=8443 protocol=tcp \
src-address=GATEWAY84 to-addresses=172.16.110.125 to-ports=443
/ip firewall service-port
set ftp disabled=yes
set tftp disabled=yes
set irc disabled=yes
set h323 disabled=yes
set sip disabled=yes
set udplite disabled=yes
set dccp disabled=yes
set sctp disabled=yes
/ip route
add distance=1 gateway=192.168.1.1 routing-mark=VOIP
add comment="Main connection in case of L2TP IP Address Change" distance=3 \
gateway=192.168.1.1
/ip service
set telnet disabled=yes
set ftp disabled=yes
set www disabled=yes
set ssh disabled=yes
set api disabled=yes
set winbox address=172.16.0.0/16
set api-ssl disabled=yes
/ip ssh
set allow-none-crypto=yes forwarding-enabled=remote
/ipv6 nd
set [ find default=yes ] advertise-dns=no
/lcd
set default-screen=stats time-interval=hour
/lcd interface
set ether2 disabled=yes
set ether3 disabled=yes
set ether4 disabled=yes
set ether5 disabled=yes
set ether6 disabled=yes
set ether7 disabled=yes
set ether8 disabled=yes
set ether9 disabled=yes
set ether10 disabled=yes
set ether11 disabled=yes
set ether12 disabled=yes
/lcd screen
set 1 disabled=yes
set 2 disabled=yes
set 3 disabled=yes
set 4 disabled=yes
set 5 disabled=yes
/system clock
set time-zone-name=Europe/London
/system health
set fan-mode=manual
/system identity
set name=Tinos1016
/system leds
add disabled=yes leds=user-led type=on
Thanks.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Tue May 19, 2020 7:29 pm

OK, so for the "local CCR" (Malford), the script below should make the firewall rules simple and efficient:
  • devices in all LAN subnets will be able to set up connections to any servers in the internet (including the Tinos subnet accessing internet via Malford via the VPN link),
  • devices in the primary LAN subnet of Malford will be able to set up connections to devices in the primary subnet of Tinos, and vice versa, over the VPN link
Any other communication passing through the router will be forbidden.
  • devices in all LAN subnets will be free to use the Malford CCR as a DNS server (if themselves configured to use it)
  • from devices in the primary LAN subnets of both Malford and Tinos, it will be possible to manage the Malford router using SSH, HTTPS (currently not configured) or Winbox
All other incoming connections to the router itself will be forbidden.

The router itself will stay free to initiate connections towards anywhere.

If the above is what you want, copy-paste the script below. If it is not, suggest modifications.

The next step will be to do a similar thing at the Tinos device. And then I'd suggest to either switch from L2TP to bare IPsec, or at least to add IPSec encryption of the L2TP, because currently anyone who can sniff the internet traffic betwen the sites can read the data flowing between Malford and Tinos LAN subnets in plaintext.

/system backup save name=before-change

/ip firewall address-list
add list=primary-subnets address=172.16.105.0/24
add list=primary-subnets address=172.16.110.0/24

/interface list
add name=WAN

/interface list member
add list=WAN interface=ISP-Virgin
add list=WAN interface=ISP-BT

/ip firewall filter
set [find] comment=the-old-rules

add action=accept chain=input connection-state=established,related,untracked
add action=drop connection-state=invalid
add action=accept protocol=udp dst-port=53 in-interface-list=!WAN
add action=accept protocol=icmp
add action=accept protocol=udp dst-port=500,1701,4500 in-interface-list=WAN
add action=accept protocol=tcp dst-port=22,443,8291 src-address-list=allowed
add action=drop

add action=accept chain=forward connection-state=established,related,untracked
add action=drop connection-state=invalid
add action=accept in-interface-list=!WAN out-interface-list=WAN
add action=accept src-address-list=primary-subnets dst-address-list=primary-subnets
add action=drop

/ip firewall filter remove [find comment~"the-old-rules"]
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Tue May 19, 2020 10:08 pm

OK, so for the "local CCR" (Malford), the script below should make the firewall rules simple and efficient:
To run the script I just create a new Script with all policies checked, permissions uncheked, and then just save and run script.
it will be possible to manage the Malford router using SSH, HTTPS (currently not configured) or Winbox
I'd rather only access internally via Winbox, last night I already a number of login attempts.
The next step will be to do a similar thing at the Tinos device. And then I'd suggest to either switch from L2TP to bare IPsec, or at least to add IPSec encryption of the L2TP
That would make sense.

Thanks,

Andrew.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Wed May 20, 2020 12:12 am

To run the script I just create a new Script with all policies checked, permissions uncheked, and then just save and run script.
No. Log in using SSH (PuTTY) or connect using Winbox and open a terminal in it. Then, press Ctrl-X to get to Safe Mode in the text window, and then copy-paste the script there. Then keep the session running and connect using the same application again, to confirm that the rules allowed you in. If yes, go to the original window and press Ctrl-X again to leave the Safe Mode and thus make the changes permanent.

You can also do it the way you've suggested, but before running the script, you have to activate the Safe Mode by the corresponding button in the left hand menu of Winbox, and then do the same test before leaving Safe Mode.

I'd rather only access internally via Winbox, last night I already a number of login attempts.
Login attempts from LAN? In your export the firewall was broken again, as the "drop the rest" rules for input traffic coming in via WANs were disabled. And having only Winbox open doesn't save you from attacks from outside.

When you say "access internally via Winbox", do you want to connect to an IP address or to a MAC address? I'm not sure from where mac-winbox is permitted by default, so you'd have to set up an interface list first.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Wed May 20, 2020 10:18 am

Then keep the session running and connect using the same application again, to confirm that the rules allowed you in.
All good thanks, now I have a rules list that looks like this:
add action=accept chain=input comment=the-old-rules connection-state=established,related disabled=yes
add action=drop chain=forward comment=the-old-rules dst-address=172.16.105.0/24 src-address=10.10.10.0/24
add action=drop chain=forward comment=the-old-rules dst-address=172.16.105.0/24 src-address=20.20.20.0/24
add action=drop chain=forward comment=the-old-rules dst-address=10.10.10.0/24 src-address=172.16.105.0/24
add action=drop chain=forward comment=the-old-rules dst-address=20.20.20.0/24 src-address=172.16.105.0/24
add action=accept chain=input comment=the-old-rules src-address-list=allowed
add action=accept chain=input comment=the-old-rules protocol=icmp
add action=accept chain=forward comment=the-old-rules connection-state=established,related,new
add action=accept chain=input comment=the-old-rules dst-port=1701 protocol=udp
add action=accept chain=input comment=the-old-rules dst-port=500 protocol=udp
add action=accept chain=input comment=the-old-rules dst-port=4500 protocol=udp
add action=accept chain=input comment=the-old-rules disabled=yes protocol=ipsec-ah
add action=accept chain=input comment=the-old-rules disabled=yes protocol=ipsec-esp
add action=drop chain=input comment=the-old-rules dst-port=53 in-interface=ISP-Virgin protocol=tcp
add action=drop chain=input comment=the-old-rules dst-port=53 in-interface=ISP-Virgin protocol=udp
add action=drop chain=input comment=the-old-rules dst-port=53 in-interface=ISP-BT protocol=tcp
add action=drop chain=input comment=the-old-rules dst-port=53 in-interface=ISP-BT protocol=udp
add action=drop chain=input comment=the-old-rules connection-state=invalid in-interface=ISP-Virgin
add action=drop chain=input comment=the-old-rules connection-state=invalid in-interface=ISP-BT
add action=drop chain=input comment=the-old-rules in-interface=ISP-Virgin
add action=drop chain=input comment=the-old-rules in-interface=ISP-BT
add action=accept chain=input connection-state=established,related,untracked
add action=drop chain="add action=accept protocol=udp dst-port=53 in-interface-list=!WAN" connection-state=invalid
add action=accept chain="add action=accept protocol=udp dst-port=500,1701,4500 in-interface-list=WAN" protocol=\
icmp
add action=accept chain="add action=drop" dst-port=22,443,8291 protocol=tcp src-address-list=allowed
add action=accept chain=forward connection-state=established,related,untracked
add action=drop chain="add action=accept in-interface-list=!WAN out-interface-list=WAN" connection-state=invalid
add action=accept chain="add action=drop" dst-address-list=primary-subnets src-address-list=primary-subnets
Don't the new rules make most of the old rules redundant? At the very least I would need to shift them higher up in the rankings? The rules you kindly wrote are now numbers 20 to 27.

Login attempts from LAN? In your export the firewall was broken again, as the "drop the rest" rules for input traffic coming in via WANs were disabled.
From the WAN, I lost the logs as had to restore all settings and reboot if not I would have posted. With regard to the rule I had disabled I needed to do that because it was interfering with the SIP phones.

And having only Winbox open doesn't save you from attacks from outside.
I'm happy with services on being available from the LAN (in IP Service List). Thank you!
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Wed May 20, 2020 11:44 am

Don't the new rules make most of the old rules redundant? At the very least I would need to shift them higher up in the rankings? The rules you kindly wrote are now numbers 20 to 27.
The last line of my script,
/ip firewall filter remove [find comment~"the-old-rules"]
should have removed those old rules - that's why it has marked the existing ones with a specific comment before adding the new ones. Have you executed that last line? It is a common issue that when copy-pasting, one doesn't copy-paste the "enter" on the last line.

If this was the case, you can now just execute that line, but again, please do that while in Safe Mode, then test that you can still establish a new connection, and only if you can, exit the Safe Mode in the original connection.

With regard to the rule I had disabled I needed to do that because it was interfering with the SIP phones.
I can't see how a rule in chain=input dealing with connections to the router itself (whose disabling has caused failed ssh attempts to appear in the log) could interfere with operation of SIP phones which are handled by chain=forward.

Regardless that, are the SIP phones registering to a server in the internet or you have some own PBX at one of the sites?
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Wed May 20, 2020 1:20 pm

If this was the case, you can now just execute that line, but again, please do that while in Safe Mode, then test that you can still establish a new connection, and only if you can, exit the Safe Mode in the original connection.
Did them manually in the end as I needed the isolation rules so, these are the active ones:
/ip firewall filter
add action=accept chain=input connection-state=established,related,untracked
add action=drop chain="add action=accept protocol=udp dst-port=53 in-interface-list=!WAN" connection-state=invalid
add action=accept chain="add action=accept protocol=udp dst-port=500,1701,4500 in-interface-list=WAN" protocol=icmp
add action=accept chain="add action=drop" dst-port=22,443,8291 protocol=tcp src-address-list=allowed
add action=accept chain=forward connection-state=established,related,untracked
add action=drop chain="add action=accept in-interface-list=!WAN out-interface-list=WAN" connection-state=invalid
add action=accept chain="add action=drop" dst-address-list=primary-subnets src-address-list=primary-subnets
add action=drop chain=forward dst-address=172.16.105.0/24 src-address=10.10.10.0/24
add action=drop chain=forward dst-address=172.16.105.0/24 src-address=20.20.20.0/24
add action=drop chain=forward dst-address=10.10.10.0/24 src-address=172.16.105.0/24
add action=drop chain=forward dst-address=20.20.20.0/24 src-address=172.16.105.0/24
and these are all the disabled ones:
add action=accept chain=input comment=the-old-rules disabled=yes src-address-list=allowed
add action=accept chain=input comment=the-old-rules disabled=yes protocol=icmp
add action=accept chain=forward comment=the-old-rules connection-state=established,related,new disabled=yes
add action=accept chain=input comment=the-old-rules disabled=yes dst-port=1701 protocol=udp
add action=accept chain=input comment=the-old-rules disabled=yes dst-port=500 protocol=udp
add action=accept chain=input comment=the-old-rules disabled=yes dst-port=4500 protocol=udp
add action=accept chain=input comment=the-old-rules disabled=yes protocol=ipsec-ah
add action=accept chain=input comment=the-old-rules disabled=yes protocol=ipsec-esp
add action=drop chain=input comment=the-old-rules disabled=yes dst-port=53 in-interface=ISP-Virgin protocol=tcp
add action=drop chain=input comment=the-old-rules disabled=yes dst-port=53 in-interface=ISP-Virgin protocol=udp
add action=drop chain=input comment=the-old-rules disabled=yes dst-port=53 in-interface=ISP-BT protocol=tcp
add action=drop chain=input comment=the-old-rules disabled=yes dst-port=53 in-interface=ISP-BT protocol=udp
add action=drop chain=input comment=the-old-rules connection-state=invalid disabled=yes in-interface=ISP-Virgin
add action=drop chain=input comment=the-old-rules connection-state=invalid disabled=yes in-interface=ISP-BT
add action=drop chain=input comment=the-old-rules disabled=yes in-interface=ISP-Virgin
add action=drop chain=input comment=the-old-rules disabled=yes in-interface=ISP-BT
[/quote]

Regardless that, are the SIP phones registering to a server in the internet or you have some own PBX at one of the sites?
I have absolutely no idea why, but it did the job. Currently SIP is working fine with the above configuration.

Are the following ports ok to be visible from the outside:
ports.JPG
You do not have the required permissions to view the files attached to this post.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Wed May 20, 2020 2:34 pm

Login attempts from LAN?
Today's attempts:
11:26:59 warning denied winbox/dude connect from 113.190.197.22
11:26:59 warning denied winbox/dude connect from 113.190.197.22
11:27:00 warning denied winbox/dude connect from 113.190.197.22
11:49:34 warning denied winbox/dude connect from 41.90.249.200
11:56:55 warning denied winbox/dude connect from 90.143.205.249
11:56:56 warning denied winbox/dude connect from 90.143.205.249
11:56:56 warning denied winbox/dude connect from 90.143.205.249
12:11:44 warning denied winbox/dude connect from 82.200.40.29
12:11:44 warning denied winbox/dude connect from 82.200.40.29
12:11:44 warning denied winbox/dude connect from 82.200.40.29
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Wed May 20, 2020 3:38 pm

Did them manually in the end as I needed the isolation rules so, these are the active ones:
No, you didn't need any extra isolation rules. I gave you firewall rules which drop everything except what I've explicitly stated will be permitted.

You have actually not implemented my rules - according to what you've posted, what you ended up with is a nonsense, sometimes a fraction of my original rule ended as a chain name which is a complete nonsense and cannot work.

So until you manage to run the script as I gave it, there is nothing more to talk about. I'm not going to debug the configuration remotely if you cannot do properly what I ask you to do.

Every proper run of my script will replace any previously existing firewall rules with the new ones, so you can simply run it again, preferably after changing the name of tha backup file.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Wed May 20, 2020 5:23 pm

No, you didn't need any extra isolation rules. I gave you firewall rules which drop everything except what I've explicitly stated will be permitted.
If I switch off these 4 rules:
add action=drop chain=forward dst-address=172.16.105.0/24 src-address=10.10.10.0/24
add action=drop chain=forward dst-address=172.16.105.0/24 src-address=20.20.20.0/24
add action=drop chain=forward dst-address=10.10.10.0/24 src-address=172.16.105.0/24
add action=drop chain=forward dst-address=20.20.20.0/24 src-address=172.16.105.0/24
Then machines in the segregated "guest" networks, 10.10.10.x and 20.20.20.x can ping the home networks 172.16.105.x and 172.16.110.x.

I have checked as I have a Ubuntu box in network 20.20.20.x that can ping past the gateway 172.16.105.1 to devices across the network.
You have actually not implemented my rules
The rules as you have applied then in the script are all there, just those extra 4 at the end. As you suggested I have reapplied the Script, and uncommented my 4 Rules and the result is this:
/ip firewall filter
add action=accept chain=input connection-state=established,related,untracked
add action=drop chain="add action=accept protocol=udp dst-port=53 in-interface-list=!WAN" connection-state=invalid
add action=accept chain="add action=accept protocol=udp dst-port=500,1701,4500 in-interface-list=WAN" protocol=icmp
add action=accept chain="add action=drop" dst-port=22,443,8291 protocol=tcp src-address-list=allowed
add action=accept chain=forward connection-state=established,related,untracked
add action=drop chain="add action=accept in-interface-list=!WAN out-interface-list=WAN" connection-state=invalid
add action=accept chain="add action=drop" dst-address-list=primary-subnets src-address-list=primary-subnets
add action=drop chain=forward dst-address=172.16.105.0/24 src-address=10.10.10.0/24
add action=drop chain=forward dst-address=172.16.105.0/24 src-address=20.20.20.0/24
add action=drop chain=forward dst-address=10.10.10.0/24 src-address=172.16.105.0/24
add action=drop chain=forward dst-address=20.20.20.0/24 src-address=172.16.105.0/24
In fact it is clear that I have not isolated properly the 110 Tinos network so would need to add:
add action=drop chain=forward dst-address=172.16.110.0/24 src-address=10.10.10.0/24
add action=drop chain=forward dst-address=172.16.110.0/24 src-address=20.20.20.0/24
add action=drop chain=forward dst-address=10.10.10.0/24 src-address=172.16.110.0/24
add action=drop chain=forward dst-address=20.20.20.0/24 src-address=172.16.110.0/24
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Wed May 20, 2020 5:37 pm

Can you copy-paste my script here exactly the same way how you have copy-pasted it to the Mikrotik? Because the resulting rules are not those in my script, something went totally wrong:

add action=drop chain="add action=accept protocol=udp dst-port=53 in-interface-list=!WAN" connection-state=invalid

whereas in my script, the above was two rules:
add action=drop chain=input connection-state=invalid
add action=accept chain=input protocol=udp dst-port=53 in-interface-list=!WAN


This corruption is also the reason why you need your selective drop rules, because my wide-span drop one which normally does their job hasn't survived the copy-paste.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Wed May 20, 2020 5:46 pm

Can you copy-paste my script here exactly the same way how you have copy-pasted it to the Mikrotik?
Sure, I first copy/paste it into notepad then onto Terminal via a Winbox connection.
/system backup save name=before-change

/ip firewall address-list
add list=primary-subnets address=172.16.105.0/24
add list=primary-subnets address=172.16.110.0/24

/interface list
add name=WAN

/interface list member
add list=WAN interface=ISP-Virgin
add list=WAN interface=ISP-BT

/ip firewall filter
set [find] comment=the-old-rules

add action=accept chain=input connection-state=established,related,untracked
add action=drop connection-state=invalid
add action=accept protocol=udp dst-port=53 in-interface-list=!WAN
add action=accept protocol=icmp
add action=accept protocol=udp dst-port=500,1701,4500 in-interface-list=WAN
add action=accept protocol=tcp dst-port=22,443,8291 src-address-list=allowed
add action=drop

add action=accept chain=forward connection-state=established,related,untracked
add action=drop connection-state=invalid
add action=accept in-interface-list=!WAN out-interface-list=WAN
add action=accept src-address-list=primary-subnets dst-address-list=primary-subnets
add action=drop
This corruption is also the reason why you need your selective drop rules, because my wide-span drop one which normally does their job hasn't survived the copy-paste.
Understood, thanks. It could have been a forum issue, or browser, for example "Code: Select All" doesn't work on Edge browser.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Wed May 20, 2020 6:09 pm

I have just copied the rules from here (Firefox used - mark, copy, paste, no "select all") and pasted them into a PuTTY window, no corruption. Please do it that way. It seems that pasting to Winbox window has some negative effect.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Wed May 20, 2020 7:35 pm

I have just copied the rules from here (Firefox used - mark, copy, paste, no "select all") and pasted them into a PuTTY window, no corruption. Please do it that way. It seems that pasting to Winbox window has some negative effect.
I feel like I am doing something really dumb here. So I put them in manually.

Is this what we should be seeing:
add action=accept chain=input connection-state=established,related,untracked
add action=drop chain=input connection-state=invalid
add action=accept chain=input dst-port=53 in-interface-list=!WAN protocol=udp
add action=accept chain=input protocol=icmp
add action=accept chain=input dst-port=500,1701,4500 in-interface-list=WAN protocol=udp
add action=accept chain=input dst-port=22,443,8291 protocol=tcp src-address-list=allowed
add action=drop chain=input
add action=accept chain=forward connection-state=established,related,untracked
add action=drop chain=forward connection-state=invalid
add action=accept chain=forward in-interface-list=!WAN out-interface-list=WAN
add action=accept chain=forward dst-address-list=primary-subnets src-address-list=primary-subnets
add action=drop chain=forward
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Wed May 20, 2020 7:47 pm

Yes. In the /export, you should see the rules in exactly the same form and order in which they appear in the script.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Wed May 20, 2020 7:58 pm

Yes. In the /export, you should see the rules in exactly the same form in which they appear in the script.
The "chain" component of the script wasn't working on my Winbox or Putty. When I did it line by line manually it asked me every time which chain this rule belongs to, I assumed "input" for the first 6 then "forward" for the last 4.

Unfortunately I seem to be able to ping across segregated networks now, so from 20.20.20.xx to a local machine on 172.16.105.xxx.

There must be something really simple I am doing wrong. If I run it as a script, it fails after the creation of Rule 1, basically at the same place it expects a "Chain" input.
script1.JPG
You do not have the required permissions to view the files attached to this post.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Wed May 20, 2020 8:17 pm

OK, so the stupidity was at my end, sorry for that. I should not do serious things in the evening.

I've always placed the chain=something only to the first rule in each chain.
And as you've added "input" to all rules, it could not work properly for forwarding.

So a corrected version is here:
/system backup save name=before-change

/ip firewall address-list
add list=primary-subnets address=172.16.105.0/24
add list=primary-subnets address=172.16.110.0/24

/interface list
add name=WAN

/interface list member
add list=WAN interface=ISP-Virgin
add list=WAN interface=ISP-BT

/ip firewall filter
set [find] comment=the-old-rules

add action=accept chain=input connection-state=established,related,untracked
add action=drop chain=input connection-state=invalid
add action=accept chain=input protocol=udp dst-port=53 in-interface-list=!WAN
add action=accept chain=input protocol=icmp
add action=accept chain=input protocol=udp dst-port=500,1701,4500 in-interface-list=WAN
add action=accept chain=input protocol=tcp dst-port=22,443,8291 src-address-list=allowed
add action=drop chain=input

add action=accept chain=forward connection-state=established,related,untracked
add action=drop chain=forward connection-state=invalid
add action=accept chain=forward in-interface-list=!WAN out-interface-list=WAN
add action=accept chain=forward src-address-list=primary-subnets dst-address-list=primary-subnets
add action=drop chain=forward

/ip firewall filter remove [find comment~"the-old-rules"]
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Wed May 20, 2020 9:03 pm

I should not do serious things in the evening.
Please! You've been an enormous help!
add action=accept chain=input connection-state=established,related,untracked
add action=drop chain=input connection-state=invalid
add action=accept chain=input dst-port=53 in-interface-list=!WAN protocol=udp
add action=accept chain=input protocol=icmp
add action=accept chain=input dst-port=500,1701,4500 in-interface-list=WAN protocol=udp
add action=accept chain=input dst-port=22,443,8291 protocol=tcp src-address-list=allowed
add action=drop chain=input
add action=accept chain=forward connection-state=established,related,untracked
add action=drop chain=forward connection-state=invalid
add action=accept chain=forward in-interface-list=!WAN out-interface-list=WAN
add action=accept chain=forward dst-address-list=primary-subnets src-address-list=primary-subnets
add action=drop chain=forward
This is what I have now and can confirm that you cannot ping across networks. Brilliant! Now all I need to try and sort out is the encryption on the L2TP connection to Tinos.

Thanks,

Andrew.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Wed May 20, 2020 11:53 pm

I can confirm that you cannot ping across networks.
...
Now all I need to try and sort out is the encryption on the L2TP connection to Tinos.
First I'd like to know that the SIP devices are doing fine with these rules at Malford.

Then, I need to have a look at Tinos firewall rules. To do that, I'd like to understand why you prefer that the Tinos site devices access internet via the VPN tunnel through Malford but you admit direct internet access if both tunnels eventually fail.

Regarding the VPN, are both WAN addresses at Malford stable or does the contract allow the ISPs to change them freely?
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Fri Jun 12, 2020 12:10 am

Then, I need to have a look at Tinos firewall rules. To do that, I'd like to understand why you prefer that the Tinos site devices access internet via the VPN tunnel through Malford but you admit direct internet access if both tunnels eventually fail.

Since I changed the rules on the Local CCR I have been unable to ping the remote network "Tinos". I have tried applying the same rule to the Tinos CCR in Safe mode, but still cannot ping across. In either case the L2TP connection engages MPPE128 stateless.

All the Tinos CCR needs to do is connect to the Malford one for all traffic. If the connection fails Tinos can access the internet locally instead. It was set up so I could manually select each connection on the Malford side in case of failover. Note that the Tinos modem is NOT in "modem mode".

Here is the Tinos log:
MMM MMM KKK TTTTTTTTTTT KKK
MMMM MMMM KKK TTTTTTTTTTT KKK
MMM MMMM MMM III KKK KKK RRRRRR OOOOOO TTT III KKK KKK
MMM MM MMM III KKKKK RRR RRR OOO OOO TTT III KKKKK
MMM MMM III KKK KKK RRRRRR OOO OOO TTT III KKK KKK
MMM MMM III KKK KKK RRR RRR OOOOOO TTT III KKK KKK

MikroTik RouterOS 6.46.4 (c) 1999-2020 http://www.mikrotik.com/

[?] Gives the list of available commands
command [?] Gives help on the command and list of arguments

[Tab] Completes the command/word. If the input is ambiguous,
a second [Tab] gives possible options

/ Move up to base level
.. Move up one level
/command Use command at the base level
[admin@Tinos1016] > /export hide sensitive
expected end of command (line 1 column 14)
[admin@Tinos1016] > /export hide-sensitive
# jun/11/2020 22:00:40 by RouterOS 6.46.4
# software id = 2Y8R-F05B
#
# model = CCR1016-12G
# serial number = 42D402FA6367
/interface bridge
add fast-forward=no mtu=1500 name=bridge-lan protocol-mode=none
/interface ethernet
set [ find default-name=ether1 ] comment="OTE Router - NOT Modem Mode" l2mtu=\
1590 name=ether1-isp1 speed=100Mbps
set [ find default-name=ether2 ] speed=100Mbps
set [ find default-name=ether3 ] speed=100Mbps
set [ find default-name=ether4 ] speed=100Mbps
set [ find default-name=ether5 ] speed=100Mbps
set [ find default-name=ether6 ] speed=100Mbps
set [ find default-name=ether7 ] speed=100Mbps
set [ find default-name=ether8 ] speed=100Mbps
set [ find default-name=ether9 ] speed=100Mbps
set [ find default-name=ether10 ] speed=100Mbps
set [ find default-name=ether11 ] speed=100Mbps
set [ find default-name=ether12 ] speed=100Mbps
/interface l2tp-client
add add-default-route=yes comment=\
"Virgin VPN Connection - change only Virgin external IP" connect-to=\
82.xx.xxx.24 name=l2tp-out1 user=tinos-l2tp
add add-default-route=yes comment=\
"BT VPN Connection - change only BT external IP" connect-to=109.xxx.xx.198 \
default-route-distance=2 name=l2tp-out2 user=tinos-l2tp
/interface list
add name=WAN
/ip pool
add name=pool-tinos ranges=172.16.110.10-172.16.110.100
/ip dhcp-server
add address-pool=pool-tinos authoritative=after-2sec-delay disabled=no \
interface=bridge-lan lease-time=1d name=dhcp-tinos
/interface bridge port
add bridge=bridge-lan hw=no interface=ether2
add bridge=bridge-lan hw=no interface=ether3
add bridge=bridge-lan hw=no interface=ether4
add bridge=bridge-lan hw=no interface=ether5
add bridge=bridge-lan hw=no interface=ether6
add bridge=bridge-lan hw=no interface=ether7
add bridge=bridge-lan hw=no interface=ether8
add bridge=bridge-lan hw=no interface=ether9
add bridge=bridge-lan hw=no interface=ether10
/ip firewall connection tracking
set udp-stream-timeout=30s
/ipv6 settings
set max-neighbor-entries=1024
/ip address
add address=172.16.110.1/24 interface=bridge-lan network=172.16.110.0
add address=192.168.1.2/24 interface=ether1-isp1 network=192.168.1.0
/ip dhcp-server network
add address=172.16.110.0/24 dns-server=1.1.1.1 gateway=172.16.110.1
/ip dns
set servers=1.1.1.1,1.0.0.1
/ip firewall address-list
add address=172.16.105.0/24 list=primary-subnets
add address=172.16.110.0/24 list=primary-subnets
/ip firewall filter
add action=accept chain=input comment=the-old-rules connection-state=\
established,related,untracked
add action=drop chain=\
"add action=accept protocol=udp dst-port=53 in-interface-list=!WAN" \
comment=the-old-rules connection-state=invalid
add action=accept chain="add action=accept protocol=udp dst-port=500,1701,4500 i\
n-interface-list=WAN" comment=the-old-rules protocol=icmp
add action=accept chain="add action=drop" comment=the-old-rules dst-port=\
22,443,8291 protocol=tcp src-address-list=allowed
add action=accept chain=forward comment=the-old-rules connection-state=\
established,related,untracked
add action=drop chain=\
"add action=accept in-interface-list=!WAN out-interface-list=WAN" comment=\
the-old-rules connection-state=invalid
add action=accept chain="add action=drop" comment=the-old-rules \
dst-address-list=primary-subnets src-address-list=primary-subnets
add action=accept chain=input comment=the-old-rules connection-state=\
established,related,untracked disabled=yes
add action=drop chain=\
"add action=accept protocol=udp dst-port=53 in-interface-list=!WAN" \
comment=the-old-rules connection-state=invalid disabled=yes
add action=accept chain="add action=accept protocol=udp dst-port=500,1701,4500 i\
n-interface-list=WAN" comment=the-old-rules disabled=yes protocol=icmp
add action=accept chain="add action=drop" comment=the-old-rules disabled=yes \
dst-port=22,443,8291 protocol=tcp src-address-list=allowed
add action=accept chain=forward comment=the-old-rules connection-state=\
established,related,untracked disabled=yes
add action=drop chain=\
"add action=accept in-interface-list=!WAN out-interface-list=WAN" comment=\
the-old-rules connection-state=invalid disabled=yes
add action=accept chain="add action=drop" comment=the-old-rules disabled=yes \
dst-address-list=primary-subnets src-address-list=primary-subnets
add action=accept chain=forward comment=the-old-rules disabled=yes dst-address=\
172.16.110.0/24 src-address=172.16.105.0/24
add action=accept chain=forward comment=the-old-rules disabled=yes dst-address=\
172.16.105.0/24 src-address=172.16.110.0/24
add action=accept chain=input comment=the-old-rules disabled=yes src-address=\
82.17.140.24
/ip firewall mangle
add action=change-mss chain=forward comment=\
"Rule to change MTU size to 105.x try not to use" disabled=yes dst-address=\
172.16.105.0/24 new-mss=1442 passthrough=yes protocol=tcp tcp-flags=syn
add action=mark-routing chain=prerouting new-routing-mark=VOIP passthrough=yes \
src-address=172.16.110.125
/ip firewall nat
add action=accept chain=srcnat dst-address=172.16.105.0/24 src-address=\
172.16.110.0/24
add action=accept chain=srcnat dst-address=172.16.100.0/24 src-address=\
172.16.110.0/24
add action=masquerade chain=srcnat src-address=172.16.110.0/24
add action=dst-nat chain=dstnat disabled=yes dst-port=8443 protocol=tcp \
src-address=82.xx.xxx.24 to-addresses=172.16.110.125 to-ports=443
/ip firewall service-port
set ftp disabled=yes
set tftp disabled=yes
set irc disabled=yes
set h323 disabled=yes
set sip disabled=yes
set udplite disabled=yes
set dccp disabled=yes
set sctp disabled=yes
/ip route
add distance=1 gateway=192.168.1.1 routing-mark=VOIP
add comment="Main connection in case of L2TP IP Address Change" distance=3 \
gateway=192.168.1.1
/ip service
set telnet disabled=yes
set ftp disabled=yes
set www disabled=yes
set ssh disabled=yes
set api disabled=yes
set winbox address=172.16.0.0/16
set api-ssl disabled=yes
/ip ssh
set allow-none-crypto=yes forwarding-enabled=remote
/ipv6 nd
set [ find default=yes ] advertise-dns=no
/lcd
set default-screen=stats time-interval=hour
/lcd interface
set ether2 disabled=yes
set ether3 disabled=yes
set ether4 disabled=yes
set ether5 disabled=yes
set ether6 disabled=yes
set ether7 disabled=yes
set ether8 disabled=yes
set ether9 disabled=yes
set ether10 disabled=yes
set ether11 disabled=yes
set ether12 disabled=yes
/lcd screen
set 1 disabled=yes
set 2 disabled=yes
set 3 disabled=yes
set 4 disabled=yes
set 5 disabled=yes
/system clock
set time-zone-name=Europe/London
/system health
set fan-mode=manual
/system identity
set name=Tinos1016
/system leds
add disabled=yes leds=user-led type=on
[admin@Tinos1016] >
Much obliged
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Fri Jun 12, 2020 12:12 am

Regarding the VPN, are both WAN addresses at Malford stable or does the contract allow the ISPs to change them freely?
They are dynamic, but it seemed easier at the time to simply set that manually as Tinos would default to normal internet and I could log in to a local PC via Teamviewer.

Thanks,

Andrew.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Fri Jun 12, 2020 9:52 pm

It seems you've tried to use the broken version of my script for Malford at Tinos, good you had the backdoor via Teamviewer. The firewall at Tinos is far less critical than at Malford, as the ISP's router has a NAT on it and likely some firewall too (unless you've intentionally set up a 1:1 dst-nat on it, effectively bypassing the firewall on the modem - in that case the one at this CCR would be as important as on the Malford one).

Slightly worse than that, I don't get how the L2TP connections can work stable, as both have add-default-route set to yes and there are no exceptional routes to Malford's public addresses. The thing is that once the L2TP interface comes up, the default route with distance=3 via the ISP router as gateway gets superseded by the one provided by the L2TP, and as soon as the routing cache expires (which should happen in 15 minutes or so), the transport packets of the VPN tunnel start being routed via the VPN tunnel itself, so the VPN fails and re-establishes.

There are two ways out, one is to route anything from local UDP port 1701 to remote UDP port 1701 via the ISP router, so when the public IP of Malford changes, it is enough to change the connect-to address on the respective /interface l2tp-client row, but you don't need to change any routes. The other way out is to configure the usual static routes, but that requires to change them along with the corresponding connect-to addresses, so I'd recommend the first approach, as you've already got it partially implemented anyway.

So tell me your choice regarding this.

As for MPPE 128, I am surprised it works for you - I recently did some tests and the RouterOS acting as L2TP client was rejecting the CCP offer for MPPE encryption coming from RouterOS acting as L2TP server. So if it works between two CCRs, it is an interesting information, but that's a minor at the moment.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Fri Jun 12, 2020 11:04 pm

There are two ways out, one is to route anything from local UDP port 1701 to remote UDP port 1701 via the ISP router, so when the public IP of Malford changes, it is enough to change the connect-to address on the respective /interface l2tp-client row, but you don't need to change any routes. The other way out is to configure the usual static routes, but that requires to change them along with the corresponding connect-to addresses, so I'd recommend the first approach, as you've already got it partially implemented anyway.
I do not mind changing the IP in the Tinos CCR connection manually in case of Malford ISPs external IP change.

I need the Tinos CCR's Firewall set as if it connected directly to the internet, for when the VPN fails. The local ISP (OTE) provides the router. I have set the CCR's IP in the DMZ list (192.168.1.2) of the ISP's Router. The router is reachable internally.

My primary concern is performance of the VPN, as we work remotely and need access to files. Other than that I am totally open to suggestions.

Many thanks.

Andrew.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Sat Jun 13, 2020 10:16 am

First, add some interface lists and member interfaces:
/interface list {
  add name=LAN
  add name=VPN
}
/interface list member {
  add list=WAN interface=ether1-isp1
  add list=LAN interface=bridge-lan
  add list=VPN interface=l2tp-out1
  add list=VPN interface=l2tp-out2
}

Then, add the routes towards the two public IPs of Malford (as said, this could be done without need to update the IP manually at two places each time it changes, but you've chosen this approach):
/ip route {
  add dst-address=current.malford.virgin.ip gateway=192.168.1.1
  add dst-address=current.malford.bt.ip gateway=192.168.1.1
}

Lastly, replace the current firewall filter rules by slightly modified default ones:
/ip firewall filter set [find] comment=rule-to-remove
/ip firewall filter {
  add chain=input action=accept connection-state=established,related,untracked comment="defconf: accept established,related,untracked"
  add chain=input action=drop connection-state=invalid comment="defconf: drop invalid"
  add chain=input action=accept protocol=icmp comment="defconf: accept ICMP"
  add chain=input action=accept dst-address=127.0.0.1 comment="defconf: accept to local loopback"
  add chain=input action=accept in-interface-list=VPN comment="allow access to router itself via VPN tunnels"
  add chain=input action=drop in-interface-list=!LAN comment="defconf: drop all not coming from LAN"
  add chain=forward action=accept connection-state=established,related,untracked comment="defconf: accept established,related, untracked"
  add chain=forward action=drop connection-state=invalid comment="defconf: drop invalid"
  add chain=forward action=drop connection-state=new connection-nat-state=!dstnat in-interface-list=WAN comment="defconf: drop all from WAN not DSTNATed"
}
/ip firewall filter remove [find comment=rule-to-remove]
That should be it.

I still wonder whether the MPPE is actually used on the L2TP connections, does /interface l2tp-client monitor [find] show that on the encoding row? If it is not (there is a bug on mipsbe architecture, but it may not exist on the tile architecture (CCR), as it doesn't exist on the arm one), there is no encryption at all; if it is, it is done in software, so it is slower (or at least more CPU-hungry) than a stronger encryption which would be provided in hardware if IPsec was used. So once you confirm that the changes above went fine, we may migrate from bare L2TP to an IPsec encrypted one.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Sat Jun 13, 2020 11:17 am

Then, add the routes towards the two public IPs of Malford (as said, this could be done without need to update the IP manually at two places each time it changes, but you've chosen this approach):
How could we modify those IP addresses automatically? Thought about DynDNS and even tried Mikrotik's own but couldn't get it to work with the failover on the Malford side.

Here is the current output after the suggested changes:
/interface bridge
add fast-forward=no mtu=1500 name=bridge-lan protocol-mode=none
/interface ethernet
set [ find default-name=ether1 ] comment="OTE Router - NOT Modem Mode" l2mtu=\
1590 name=ether1-isp1 speed=100Mbps
set [ find default-name=ether2 ] speed=100Mbps
set [ find default-name=ether3 ] speed=100Mbps
set [ find default-name=ether4 ] speed=100Mbps
set [ find default-name=ether5 ] speed=100Mbps
set [ find default-name=ether6 ] speed=100Mbps
set [ find default-name=ether7 ] speed=100Mbps
set [ find default-name=ether8 ] speed=100Mbps
set [ find default-name=ether9 ] speed=100Mbps
set [ find default-name=ether10 ] speed=100Mbps
set [ find default-name=ether11 ] speed=100Mbps
set [ find default-name=ether12 ] speed=100Mbps
/interface l2tp-client
add add-default-route=yes comment=\
"Virgin VPN Connection - change only Virgin external IP" connect-to=\
82.xx.xxx.24 name=l2tp-out1 user=tinos-l2tp
add add-default-route=yes comment=\
"BT VPN Connection - change only BT external IP" connect-to=109.xxx.xx.198 \
default-route-distance=2 name=l2tp-out2 user=tinos-l2tp
/interface list
add name=WAN
add name=LAN
add name=VPN
/ip pool
add name=pool-tinos ranges=172.16.110.10-172.16.110.100
/ip dhcp-server
add address-pool=pool-tinos authoritative=after-2sec-delay disabled=no \
interface=bridge-lan lease-time=1d name=dhcp-tinos
/interface bridge port
add bridge=bridge-lan hw=no interface=ether2
add bridge=bridge-lan hw=no interface=ether3
add bridge=bridge-lan hw=no interface=ether4
add bridge=bridge-lan hw=no interface=ether5
add bridge=bridge-lan hw=no interface=ether6
add bridge=bridge-lan hw=no interface=ether7
add bridge=bridge-lan hw=no interface=ether8
add bridge=bridge-lan hw=no interface=ether9
add bridge=bridge-lan hw=no interface=ether10
/ip firewall connection tracking
set udp-stream-timeout=30s
/ipv6 settings
set max-neighbor-entries=1024
/interface list member
add interface=ether1-isp1 list=WAN
add interface=bridge-lan list=LAN
add interface=l2tp-out1 list=VPN
add interface=l2tp-out2 list=VPN
/ip address
add address=172.16.110.1/24 interface=bridge-lan network=172.16.110.0
add address=192.168.1.2/24 interface=ether1-isp1 network=192.168.1.0
/ip dhcp-server network
add address=172.16.110.0/24 dns-server=1.1.1.1 gateway=172.16.110.1
/ip dns
set servers=1.1.1.1,1.0.0.1
/ip firewall address-list
add address=172.16.105.0/24 list=primary-subnets
add address=172.16.110.0/24 list=primary-subnets
/ip firewall filter
add action=accept chain=input comment=\
"defconf: accept established,related,untracked" connection-state=\
established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment="defconf: accept to local loopback" \
dst-address=127.0.0.1
add action=accept chain=input comment=\
"allow access to router itself via VPN tunnels" in-interface-list=VPN
add action=drop chain=input comment="defconf: drop all not coming from LAN" \
in-interface-list=!LAN
add action=accept chain=forward comment=\
"defconf: accept established,related, untracked" connection-state=\
established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=\
invalid
add action=drop chain=forward comment="defconf: drop all from WAN not DSTNATed" \
connection-nat-state=!dstnat connection-state=new in-interface-list=WAN
/ip firewall mangle
add action=change-mss chain=forward comment=\
"Rule to change MTU size to 105.x try not to use" disabled=yes dst-address=\
172.16.105.0/24 new-mss=1442 passthrough=yes protocol=tcp tcp-flags=syn
add action=mark-routing chain=prerouting new-routing-mark=VOIP passthrough=yes \
src-address=172.16.110.125
/ip firewall nat
add action=accept chain=srcnat dst-address=172.16.105.0/24 src-address=\
172.16.110.0/24
add action=accept chain=srcnat dst-address=172.16.100.0/24 src-address=\
172.16.110.0/24
add action=masquerade chain=srcnat src-address=172.16.110.0/24
add action=dst-nat chain=dstnat disabled=yes dst-port=8443 protocol=tcp \
src-address=82.17.140.24 to-addresses=172.16.110.125 to-ports=443
/ip firewall service-port
set ftp disabled=yes
set tftp disabled=yes
set irc disabled=yes
set h323 disabled=yes
set sip disabled=yes
set udplite disabled=yes
set dccp disabled=yes
set sctp disabled=yes
/ip route
add distance=1 gateway=192.168.1.1 routing-mark=VOIP
add comment="Main connection in case of L2TP IP Address Change" distance=3 \
gateway=192.168.1.1
add distance=1 dst-address=82.17.140.24/32 gateway=192.168.1.1
add distance=1 dst-address=109.145.13.198/32 gateway=192.168.1.1
/ip service
set telnet disabled=yes
set ftp disabled=yes
set www disabled=yes
set ssh disabled=yes
set api disabled=yes
set winbox address=172.16.0.0/16
set api-ssl disabled=yes
/ip ssh
set allow-none-crypto=yes forwarding-enabled=remote
/ipv6 nd
set [ find default=yes ] advertise-dns=no
/lcd
set default-screen=stats time-interval=hour
/lcd interface
set ether2 disabled=yes
set ether3 disabled=yes
set ether4 disabled=yes
set ether5 disabled=yes
set ether6 disabled=yes
set ether7 disabled=yes
set ether8 disabled=yes
set ether9 disabled=yes
set ether10 disabled=yes
set ether11 disabled=yes
set ether12 disabled=yes
/lcd screen
set 1 disabled=yes
set 2 disabled=yes
set 3 disabled=yes
set 4 disabled=yes
set 5 disabled=yes
/system clock
set time-zone-name=Europe/London
/system health
set fan-mode=manual
/system identity
set name=Tinos1016
/system leds
add disabled=yes leds=user-led type=on
[admin@Tinos1016] >
.
When I enable the old Virgin VPN Connection in Interface List I see connection on both sides but cannot ping across, only to each others routers xxx.xxx.1. This is what I see:
.
[admin@Malford1016] > /interface l2tp-server monitor
numbers: 0
status: connected
uptime: 5m19s
user: tinos-l2tp
caller-id: 45.xxx.xxx.183
encoding: MPPE128 stateless
mtu: 1450
mru: 1450
local-address: 172.16.105.1
remote-address: 10.10.20.10
-- [Q quit|D dump|C-z pause]
.
.
[admin@Tinos1016] > /interface l2tp-client monitor
numbers: 0
;;; Virgin VPN Connection - change only
;;; Virgin VPN Connection - change only
;;; Virgin VPN Connection - change only
;;; Virgin VPN Connection - change only Virgin external IP
status: connected
uptime: 5m52s
encoding: MPPE128 stateless
mtu: 1450
mru: 1450
local-address: 10.10.20.10
remote-address: 172.16.105.1
-- [Q quit|D dump|C-z pause]
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Sat Jun 13, 2020 1:33 pm

How could we modify those IP addresses automatically? Thought about DynDNS and even tried Mikrotik's own but couldn't get it to work with the failover on the Malford side.
I actually didn't have in mind changing the routes' dst-address automatically, but using a routing-mark to make sure that the ISP's router is used as a gateway for the VPN transport packets, like you do for the VoIP case.

If you want to track the changes of Malford's IP addresses so that you wouldn't need to change them manually in the L2TP configuration, a dynamic DNS is the only reliable way to go. It would not be 100% safe to rely on only one of the addresses to change at a time and use the running tunnel to deliver the information about the change directly. However, the DDNS fucntionality of the cloud provides only a single fqdn, so you can track only one of the addresses that way. For the other one, you'd need a 3rd party DDNS service and a corresponding update script.

When I enable the old Virgin VPN Connection in Interface List I see connection on both sides but cannot ping across, only to each others routers xxx.xxx.1.
Explain "ping across". If you have in mind that you cannot ping between hosts in Malford's LAN subnet and Tinos' LAN subnets, the reason is that there is no route to Tinos LAN subnet (172.16.110.0/24) in Malford's configuration, and in Tinos' configuration, connections from LAN subnet towards 172.16.100.0/24 and 172.16.105.0/24 are excluded from NAT, so ping requests and responses sent from Malford to 172.16.110.0/24 take the default route via WAN so they get nowhere. To change that, you need to add the route to Tinos' LAN subnet into the /ppp secret configuration at Malford: /ppp secret set tinos-l2tp routes=172.16.110.0/24

To make the routes get actually added, you have to disable and re-enable the /interface l2tp-client at Tinos, or disable and re-enable the /interface l2tp-server server at Malford. There is no way to specify the distance of the routes added this way, so the traffic towards the Tinos LAN subnet will use one of the tunnels randomly. If you wanted to change this, you'd have to create two /ppp secret rows with different user names, one for each tunnel, link static /interface l2tp-server rows to them, and manually configure two routes to 172.16.110.0/24 with different distance values, each using one of these static interfaces as gateway.

But this routing issue was always there, it has not changed due to the changes of the firewall, so I guess you actually have something else in mind?
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Sat Jun 13, 2020 1:47 pm

Explain "ping across". If you have in mind that you cannot ping between hosts in Malford's LAN subnet and Tinos' LAN subnets, the reason is that there is no route to Tinos LAN subnet (172.16.110.0/24) in Malford's configuration, and in Tinos' configuration, connections from LAN subnet towards 172.16.100.0/24 and 172.16.105.0/24 are excluded from NAT, so ping requests and responses sent from Malford to 172.16.110.0/24 take the default route via WAN so they get nowhere. To change that, you need to add the route to Tinos' LAN subnet into the /ppp secret configuration at Malford: /ppp secret set tinos-l2tp routes=172.16.110.0/24
.
This is what I get:
.
[admin@Tinos1016] > /ppp secret set tinos-l2tp routes=172.16.110.0/24
no such item
.
To make the routes get actually added, you have to disable and re-enable the /interface l2tp-client at Tinos, or disable and re-enable the /interface l2tp-server server at Malford. There is no way to specify the distance of the routes added this way, so the traffic towards the Tinos LAN subnet will use one of the tunnels randomly. If you wanted to change this, you'd have to create two /ppp secret rows with different user names, one for each tunnel, link static /interface l2tp-server rows to them, and manually configure two routes to 172.16.110.0/24 with different distance values, each using one of these static interfaces as gateway.
But this routing issue was always there, it has not changed due to the changes of the firewall, so I guess you actually have something else in mind?
.
Before we changed the rules on the Malford firewall the two networks could see each other, ie devices on Tinos could ping/access devices on Malford and vice versa. Tinos devices would access the internet via Malford.

On the Tinos CCR I simply enable the relevant connection l2tp-out1 or l2tpout2

Currently only the routers can ping each other no other traffic gets through from 110.x to 105.x and vice versa.

I have had this issue before that the scripts don't run properly. Have you checked that the rules you gave me were applied correctly?

Thanks,

Andrew.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Sat Jun 13, 2020 2:04 pm

This is what I get:
[admin@Tinos1016] > /ppp secret set tinos-l2tp routes=172.16.110.0/24
no such item
Post the current export of Malford, it seems I don't have the complete picture after the changes.

Before we changed the rules on the Malford firewall the two networks could see each other, ie devices on Tinos could ping/access devices on Malford and vice versa. Tinos devices would access the internet via Malford.
Tinos devices should be able to access internet via Malford when the tunnel is up because the exclusion from NAT at Tinos doesn't apply to packets to other destination than Malford's LAN subnets.

On the Tinos CCR I simply enable the relevant connection l2tp-out1 or l2tp-out2
Are you saying that you manually activate only one tunnel at a time?

I have had this issue before that the scripts don't run properly. Have you checked that the rules you gave me were applied correctly?
I have, yes, the result is the one I've planned.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Sat Jun 13, 2020 2:28 pm

Post the current export of Malford, it seems I don't have the complete picture after the changes.
.
This is Tinos current:
.
[admin@Tinos1016] > /export hide-sensitive
# jun/13/2020 12:19:07 by RouterOS 6.46.4
# software id = 2Y8R-F05B
#
# model = CCR1016-12G
# serial number = 42D402FA6367
/interface bridge
add fast-forward=no mtu=1500 name=bridge-lan protocol-mode=none
/interface ethernet
set [ find default-name=ether1 ] comment="OTE Router - NOT Modem Mode" l2mtu=1590 name=\
ether1-isp1 speed=100Mbps
set [ find default-name=ether2 ] speed=100Mbps
set [ find default-name=ether3 ] speed=100Mbps
set [ find default-name=ether4 ] speed=100Mbps
set [ find default-name=ether5 ] speed=100Mbps
set [ find default-name=ether6 ] speed=100Mbps
set [ find default-name=ether7 ] speed=100Mbps
set [ find default-name=ether8 ] speed=100Mbps
set [ find default-name=ether9 ] speed=100Mbps
set [ find default-name=ether10 ] speed=100Mbps
set [ find default-name=ether11 ] speed=100Mbps
set [ find default-name=ether12 ] speed=100Mbps
/interface l2tp-client
add add-default-route=yes comment=\
"Virgin VPN Connection - change only Virgin external IP" connect-to=82.17.140.24 \
disabled=no name=l2tp-out1 user=tinos-l2tp
add add-default-route=yes comment="BT VPN Connection - change only BT external IP" \
connect-to=109.145.13.198 default-route-distance=2 name=l2tp-out2 user=tinos-l2tp
/interface list
add name=WAN
add name=LAN
add name=VPN
/ip pool
add name=pool-tinos ranges=172.16.110.10-172.16.110.100
/ip dhcp-server
add address-pool=pool-tinos authoritative=after-2sec-delay disabled=no interface=\
bridge-lan lease-time=1d name=dhcp-tinos
/interface bridge port
add bridge=bridge-lan hw=no interface=ether2
add bridge=bridge-lan hw=no interface=ether3
add bridge=bridge-lan hw=no interface=ether4
add bridge=bridge-lan hw=no interface=ether5
add bridge=bridge-lan hw=no interface=ether6
add bridge=bridge-lan hw=no interface=ether7
add bridge=bridge-lan hw=no interface=ether8
add bridge=bridge-lan hw=no interface=ether9
add bridge=bridge-lan hw=no interface=ether10
/ip firewall connection tracking
set udp-stream-timeout=30s
/ipv6 settings
set max-neighbor-entries=1024
/interface list member
add interface=ether1-isp1 list=WAN
add interface=bridge-lan list=LAN
add interface=l2tp-out1 list=VPN
add interface=l2tp-out2 list=VPN
/ip address
add address=172.16.110.1/24 interface=bridge-lan network=172.16.110.0
add address=192.168.1.2/24 interface=ether1-isp1 network=192.168.1.0
/ip dhcp-server network
add address=172.16.110.0/24 dns-server=1.1.1.1 gateway=172.16.110.1
/ip dns
set servers=1.1.1.1,1.0.0.1
/ip firewall address-list
add address=172.16.105.0/24 list=primary-subnets
add address=172.16.110.0/24 list=primary-subnets
/ip firewall filter
add action=accept chain=input comment="defconf: accept established,related,untracked" \
connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment="defconf: accept to local loopback" dst-address=\
127.0.0.1
add action=accept chain=input comment="allow access to router itself via VPN tunnels" \
in-interface-list=VPN
add action=drop chain=input comment="defconf: drop all not coming from LAN" \
in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept established,related, untracked" \
connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf: drop all from WAN not DSTNATed" \
connection-nat-state=!dstnat connection-state=new in-interface-list=WAN
/ip firewall mangle
add action=change-mss chain=forward comment=\
"Rule to change MTU size to 105.x try not to use" disabled=yes dst-address=\
172.16.105.0/24 new-mss=1442 passthrough=yes protocol=tcp tcp-flags=syn
add action=mark-routing chain=prerouting new-routing-mark=VOIP passthrough=yes \
src-address=172.16.110.125
/ip firewall nat
add action=accept chain=srcnat dst-address=172.16.105.0/24 src-address=172.16.110.0/24
add action=accept chain=srcnat dst-address=172.16.100.0/24 src-address=172.16.110.0/24
add action=masquerade chain=srcnat src-address=172.16.110.0/24
add action=dst-nat chain=dstnat disabled=yes dst-port=8443 protocol=tcp src-address=\
82.17.140.24 to-addresses=172.16.110.125 to-ports=443
/ip firewall service-port
set ftp disabled=yes
set tftp disabled=yes
set irc disabled=yes
set h323 disabled=yes
set sip disabled=yes
set udplite disabled=yes
set dccp disabled=yes
set sctp disabled=yes
/ip route
add distance=1 gateway=192.168.1.1 routing-mark=VOIP
add comment="Main connection in case of L2TP IP Address Change" distance=3 gateway=\
192.168.1.1
add distance=1 dst-address=82.xx.xx.24/32 gateway=192.168.1.1
add distance=1 dst-address=109.xx.xxx.198/32 gateway=192.168.1.1
/ip service
set telnet disabled=yes
set ftp disabled=yes
set www disabled=yes
set ssh disabled=yes
set api disabled=yes
set winbox address=172.16.0.0/16
set api-ssl disabled=yes
/ip ssh
set allow-none-crypto=yes forwarding-enabled=remote
/ipv6 nd
set [ find default=yes ] advertise-dns=no
/lcd
set default-screen=stats time-interval=hour
/lcd interface
set ether2 disabled=yes
set ether3 disabled=yes
set ether4 disabled=yes
set ether5 disabled=yes
set ether6 disabled=yes
set ether7 disabled=yes
set ether8 disabled=yes
set ether9 disabled=yes
set ether10 disabled=yes
set ether11 disabled=yes
set ether12 disabled=yes
/lcd screen
set 1 disabled=yes
set 2 disabled=yes
set 3 disabled=yes
set 4 disabled=yes
set 5 disabled=yes
/system clock
set time-zone-name=Europe/London
/system health
set fan-mode=manual
/system identity
set name=Tinos1016
/system leds
add disabled=yes leds=user-led type=on
[admin@Tinos1016] >
.
This is Malford current
.
[admin@Malford1016] > /export hide-sensitive
# jun/13/2020 12:22:02 by RouterOS 6.46.6
# software id = PZJX-KJ04
#
# model = CCR1016-12G
# serial number = 42D402B557C9
/interface bridge
add fast-forward=no name=bridge-lan
add name=bridgeDRGuest
add name=bridgeGuest
/interface ethernet
set [ find default-name=ether2 ] name=BT
set [ find default-name=ether1 ] comment="Virgin Modem Mode connection to DHCP Client" name=ISP-Virgin
/interface pppoe-client
add add-default-route=yes comment="BT PPPoE Client Modem Mode" default-route-distance=3 disabled=no interface=BT name=ISP-BT user=\
bthomehub@btbroadband.com
/interface list
add name=WAN
/ip pool
add name=pool-malford ranges=172.16.105.90-172.16.105.129
add name=pGuest ranges=10.10.10.10-10.10.10.100
add name=pDRGuest ranges=20.20.20.20-20.20.20.100
/ip dhcp-server
add address-pool=pool-malford interface=bridge-lan lease-time=1d name=dhcp-lan
add address-pool=pGuest disabled=no interface=bridgeGuest name=DHCPGuest
add address-pool=pDRGuest disabled=no interface=bridgeDRGuest name=DHCPDRGUEST
/system logging action
set 1 disk-file-name=flash/log
add email-to=andrew.gebhardt@finexlondon.com name=sendemail target=email
/user group
set full policy=local,telnet,ssh,ftp,reboot,read,write,policy,test,winbox,password,web,sniff,sensitive,api,romon,dude,tikapp
/interface bridge port
add bridge=bridge-lan interface=ether3
add bridge=bridge-lan interface=ether4
add bridge=bridge-lan interface=ether5
add bridge=bridge-lan interface=ether6
add bridge=bridge-lan interface=ether7
add bridge=bridge-lan interface=ether8
add bridge=bridge-lan interface=ether9
add bridge=bridge-lan interface=ether10
add bridge=bridgeDRGuest interface=ether11
add bridge=bridgeGuest interface=ether12
/ip settings
set tcp-syncookies=yes
/interface l2tp-server server
set enabled=yes
/interface list member
add interface=ISP-Virgin list=WAN
add interface=ISP-BT list=WAN
/interface pptp-server server
set enabled=yes
/interface sstp-server server
set certificate=Server default-profile=default-encryption port=8443 tls-version=only-1.2
/ip address
add address=172.16.105.1/24 interface=bridge-lan network=172.16.105.0
add address=192.168.1.2/24 interface=BT network=192.168.1.0
add address=192.168.2.2/24 disabled=yes interface=BT network=192.168.2.0
add address=10.10.10.254/24 interface=bridgeGuest network=10.10.10.0
add address=20.20.20.254/24 interface=bridgeDRGuest network=20.20.20.0
/ip cloud
set ddns-update-interval=2m
/ip dhcp-client
add add-default-route=no comment="DHCP Client for Virgin Router in Modem Mode" disabled=no interface=ISP-Virgin use-peer-dns=no
/ip dhcp-server network
add address=10.10.10.0/24 dns-server=1.1.1.1,1.0.0.1 gateway=10.10.10.254 netmask=24
add address=20.20.20.0/24 dns-server=8.8.8.8,8.8.4.4 gateway=20.20.20.254 netmask=24
add address=172.16.105.0/24
/ip dns
set servers=172.16.105.2
/ip firewall address-list
add address=172.16.105.0/24 list=allowed
add address=172.16.110.0/24 list=allowed
add address=2.xxx.xxx.170 disabled=yes list=allowed
add address=2.xxx.xxx.170 list=TinosIP
add address=172.16.105.0/24 list=primary-subnets
add address=172.16.110.0/24 list=primary-subnets
/ip firewall filter
add action=accept chain=input connection-state=established,related,untracked
add action=drop chain=input connection-state=invalid
add action=accept chain=input dst-port=53 in-interface-list=!WAN protocol=udp
add action=accept chain=input protocol=icmp
add action=accept chain=input dst-port=500,1701,4500 in-interface-list=WAN protocol=udp
add action=accept chain=input dst-port=22,443,8291 protocol=tcp src-address-list=allowed
add action=drop chain=input
add action=accept chain=forward connection-state=established,related,untracked
add action=drop chain=forward connection-state=invalid
add action=accept chain=forward in-interface-list=!WAN out-interface-list=WAN
add action=accept chain=forward dst-address-list=primary-subnets src-address-list=primary-subnets
add action=drop chain=forward
add action=drop chain=forward comment=the-old-rules disabled=yes dst-address=172.16.105.0/24 src-address=10.10.10.0/24
add action=drop chain=forward comment=the-old-rules disabled=yes dst-address=172.16.110.0/24 src-address=10.10.10.0/24
add action=drop chain=forward comment=the-old-rules disabled=yes dst-address=172.16.105.0/24 src-address=20.20.20.0/24
add action=drop chain=forward comment=the-old-rules disabled=yes dst-address=172.16.110.0/24 src-address=20.20.20.0/24
add action=drop chain=forward comment=the-old-rules disabled=yes dst-address=10.10.10.0/24 src-address=172.16.105.0/24
add action=drop chain=forward comment=the-old-rules disabled=yes dst-address=10.10.10.0/24 src-address=172.16.110.0/24
add action=drop chain=forward comment=the-old-rules disabled=yes dst-address=20.20.20.0/24 src-address=172.16.105.0/24
add action=drop chain=forward comment=the-old-rules disabled=yes dst-address=20.20.20.0/24 src-address=172.16.110.0/24
/ip firewall mangle
add action=mark-routing chain=output disabled=yes log=yes new-routing-mark=virgin passthrough=no src-address=10.10.10.0/24
add action=change-mss chain=forward dst-address=172.16.110.0/24 new-mss=1442 passthrough=yes protocol=tcp tcp-flags=syn
add action=change-mss chain=forward new-mss=1470 passthrough=yes protocol=tcp tcp-flags=syn
/ip firewall nat
add action=masquerade chain=srcnat comment="NAT to Internet"
add action=dst-nat chain=dstnat disabled=yes dst-port=53 protocol=udp src-address-list=allowed to-addresses=172.16.105.2
add action=src-nat chain=srcnat disabled=yes dst-port=53 protocol=tcp to-addresses=172.16.105.2 to-ports=53
add action=src-nat chain=srcnat disabled=yes dst-port=53 protocol=udp to-addresses=172.16.105.2 to-ports=53
/ip firewall service-port
set ftp disabled=yes
set tftp disabled=yes
set irc disabled=yes
set h323 disabled=yes
set sip disabled=yes
set udplite disabled=yes
set dccp disabled=yes
set sctp disabled=yes
/ip route
add distance=1 gateway=82.xx.xxx.1
add comment="Tinos via SSTP" distance=1 dst-address=172.16.110.0/24 gateway=10.10.20.10
/ip service
set telnet disabled=yes
set ftp disabled=yes
set www disabled=yes
set ssh address=172.16.105.0/24,172.16.110.0/24
set api disabled=yes
set winbox address=172.16.105.0/24,172.16.110.0/24
set api-ssl disabled=yes
/ip ssh
set allow-none-crypto=yes forwarding-enabled=remote
/ip upnp
set enabled=yes show-dummy-rule=no
/ipv6 nd
set [ find default=yes ] advertise-dns=no
/ppp secret
add local-address=172.16.105.1 name=tinos-l2tp remote-address=10.10.20.10 service=l2tp
/queue simple
add disabled=yes max-limit=10M/10M name=queue-tinos target=*F
/system clock
set time-zone-name=Europe/London
/system identity
set name=Malford1016
/system logging
add action=sendemail topics=script
/system ntp client
set enabled=yes primary-ntp=130.159.196.118 secondary-ntp=178.79.152.182
/system scheduler
add interval=5s name=RunFailover on-event=FailoverScript policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon start-date=\
jun/10/2019 start-time=14:09:11
/system script
add dont-require-permissions=no name=FailoverScript owner=admin policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive source="# -\
------------------ header -------------------\r\
\n# Script by Tomas Kirnak, version 1.0.7\r\
\n# If you use this script, or edit and\r\
\n# re-use it, please keep the header intact.\r\
\n#\r\
\n# For more information and details about\r\
\n# this script please visit the wiki page at\r\
\n# http://wiki.mikrotik.com/wiki/Failover_Scripting\r\
\n# ------------------- header -------------------\r\
\n\r\
\n\r\
\n\r\
\n# ------------- start editing here -------------\r\
\n# Edit the variables below to suit your needs\r\
\n\r\
\n# Please fill the WAN interface names\r\
\n:local InterfaceISP1 ISP-Virgin\r\
\n:local InterfaceISP2 ISP-BT\r\
\n\r\
\n# Please fill the gateway IPs (or interface names in case of PPP)\r\
\n:global GatewayISP1 82.17.140.1\r\
\n:global GatewayISP2 ISP-BT\r\
\n\r\
\n# Please fill the ping check host - currently: resolver1.opendns.com\r\
\n:local PingTarget 82.17.140.1\r\
\n\r\
\n\r\
\n# Please fill how many ping failures are allowed before fail-over happends\r\
\n:local FailTreshold 3\r\
\n\r\
\n# Define the distance increase of a route when it fails\r\
\n:local DistanceIncrease 5\r\
\n\r\
\n# Editing the script after this point may break it\r\
\n# -------------- stop editing here --------------\r\
\n\r\
\n\r\
\n\r\
\n# Declare the global variables\r\
\n:global PingFailCountISP1\r\
\n:global PingFailCountISP2\r\
\n\r\
\n#set PingFailCountISP1 0\r\
\n#set PingFailCountISP2 0\r\
\n\r\
\n# This inicializes the PingFailCount variables, in case this is the 1st time the script has ran\r\
\n:if ([:typeof \$PingFailCountISP1] = \"nothing\") do={:set PingFailCountISP1 0}\r\
\n:if ([:typeof \$PingFailCountISP2] = \"nothing\") do={:set PingFailCountISP2 0}\r\
\n\r\
\n# This variable will be used to keep results of individual ping attempts\r\
\n:local PingResult\r\
\n\r\
\n\r\
\n\r\
\n# Check ISP1\r\
\n:set PingResult [ping \$PingTarget count=1]\r\
\n:put \$PingResult\r\
\n\r\
\n:if (\$PingResult = 0) do={\r\
\n\t:if (\$PingFailCountISP1 < (\$FailTreshold+2)) do={\r\
\n\t\t:set PingFailCountISP1 (\$PingFailCountISP1 + 1)\r\
\n\t\t\r\
\n\t\t:if (\$PingFailCountISP1 = \$FailTreshold) do={\r\
\n\t\t\t:log warning \"ISP1 has a problem en route to \$PingTarget - increasing distance of routes.\"\r\
\n\t\t\t:foreach i in=[/ip route find gateway=\$GatewayISP1 && static] do=\\\r\
\n\t\t\t\t{/ip route set \$i distance=\$DistanceIncrease}\r\
\n\r\
\n\t\t\t{ :local cons [/ip firewall connection find protocol=\"udp\"]; :if ([:len \$cons] > 0) do={ /ip firewall connection remove number\
s=\$cons;}}\r\
\n\t\t\t{ :local cons [/ip firewall connection find protocol=\"tcp\"]; :if ([:len \$cons] > 0) do={ /ip firewall connection remove number\
s=\$cons;}}\r\
\n\t\t\t{ :local cons [/ip firewall connection find protocol=\"rtp\"]; :if ([:len \$cons] > 0) do={ /ip firewall connection remove number\
s=\$cons;}}\r\
\n\t\t\t{ :local cons [/ip firewall connection find protocol=\"icmp\"]; :if ([:len \$cons] > 0) do={ /ip firewall connection remove numbe\
rs=\$cons;}}\r\
\n\r\
\n\t\t\t:log warning \"Route distance increase finished.\"\r\
\n\t\t}\r\
\n\t}\r\
\n}\r\
\n:if (\$PingResult = 1) do={\r\
\n\t:if (\$PingFailCountISP1 > 0) do={\r\
\n\t\t:set PingFailCountISP1 (\$PingFailCountISP1 - 1)\r\
\n\t\t\r\
\n\t\t:if (\$PingFailCountISP1 = (\$FailTreshold -1)) do={\r\
\n\t\t\t:log warning \"ISP1 can reach \$PingTarget again - bringing back original distance of routes.\"\r\
\n\t\t\t:foreach i in=[/ip route find gateway=\$GatewayISP1 && static] do=\\\r\
\n\t\t\t\t{/ip route set \$i distance=1}\r\
\n\r\
\n\t\t\t{ :local cons [/ip firewall connection find protocol=\"udp\"]; :if ([:len \$cons] > 0) do={ /ip firewall connection remove number\
s=\$cons;}}\r\
\n\t\t\t{ :local cons [/ip firewall connection find protocol=\"tcp\"]; :if ([:len \$cons] > 0) do={ /ip firewall connection remove number\
s=\$cons;}}\r\
\n\t\t\t{ :local cons [/ip firewall connection find protocol=\"rtp\"]; :if ([:len \$cons] > 0) do={ /ip firewall connection remove number\
s=\$cons;}}\r\
\n\t\t\t{ :local cons [/ip firewall connection find protocol=\"icmp\"]; :if ([:len \$cons] > 0) do={ /ip firewall connection remove numbe\
rs=\$cons;}}\r\
\n\r\
\n\t\t\t:log warning \"Route distance decrease finished.\"\r\
\n\t\t}\r\
\n\t}\r\
\n}\r\
\n"
add dont-require-permissions=no name=ResetFailover owner=admin policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon \
source="# Declare the global variables\r\
\n:global PingFailCountISP1\r\
\n:global PingFailCountISP2\r\
\n:global GatewayISP1\r\
\n:global GatewayISP2\r\
\n\r\
\n# Reset Variables\r\
\nset PingFailCountISP1 0\r\
\nset PingFailCountISP2 0\r\
\n\r\
\n# Reset Routes\r\
\n:foreach i in=[/ip route find gateway=\$GatewayISP1 && static] do=\\\r\
\n {/ip route set \$i distance=1}\r\
\n\r\
\n# Reset Connections\r\
\n{ :local cons [/ip firewall connection find protocol=\"udp\"]; :if ([:len \$cons] > 0) do={ /ip firewall connection remove numbers=\$co\
ns;}}\r\
\n{ :local cons [/ip firewall connection find protocol=\"tcp\"]; :if ([:len \$cons] > 0) do={ /ip firewall connection remove numbers=\$co\
ns;}}\r\
\n{ :local cons [/ip firewall connection find protocol=\"rtp\"]; :if ([:len \$cons] > 0) do={ /ip firewall connection remove numbers=\$co\
ns;}}\r\
\n{ :local cons [/ip firewall connection find protocol=\"icmp\"]; :if ([:len \$cons] > 0) do={ /ip firewall connection remove numbers=\$c\
ons;}}\r\
\n\r\
\n:log warning \"Route Failover Reset.\""
add dont-require-permissions=yes name=script1 owner=admin policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon source="/sy\
stem backup save name=before-change\r\
\n\r\
\n/ip firewall filter\r\
\nset [find] comment=the-old-rules\r\
\n\r\
\nadd action=accept chain=input connection-state=established,related,untracked\r\
\nadd action=drop connection-state=invalid\r\
\nadd action=accept protocol=udp dst-port=53 in-interface-list=!WAN\r\
\nadd action=accept protocol=icmp\r\
\nadd action=accept protocol=udp dst-port=500,1701,4500 in-interface-list=WAN\r\
\nadd action=accept protocol=tcp dst-port=22,443,8291 src-address-list=allowed\r\
\nadd action=drop\r\
\n\r\
\nadd action=accept chain=forward connection-state=established,related,untracked\r\
\nadd action=drop connection-state=invalid\r\
\nadd action=accept in-interface-list=!WAN out-interface-list=WAN\r\
\nadd action=accept src-address-list=primary-subnets dst-address-list=primary-subnets\r\
\nadd action=drop\r\
\n\r\
\n"
/tool e-mail
set address=smtp.office365.com from=andrew.gebhardt@finexlondon.com port=587 start-tls=yes user=andrew.gebhardt@finexlondon.com
/tool netwatch
add down-script="tool e-mail send to=andrew.gebhardt@finexlondon.com subject=\"Virgin DOWN\"" host=8.8.8.8 up-script=\
"tool e-mail send to=andrew.gebhardt@finexlondon.com subject=\"Virgin UP\""
[admin@Malford1016] >
.
Are you saying that you manually activate only one tunnel at a time?
.
Correct, depending on which ISP Malford is using I manually select the respective L2TP rule on the Tinos CCR. I don't mind this.
.
Much obliged
.
Andrew
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Sat Jun 13, 2020 3:44 pm

This is what I get:
[admin@Tinos1016] > /ppp secret set tinos-l2tp routes=172.16.110.0/24
no such item
I've missed that you've missed that this command needs to be used at Malford, not at Tinos :)
At Tinos, the "better" (lower distance value) default route is added once the L2TP tunnel goes up.
But at Malford, there is no route to Tinos LAN subnets (and never was so I really cannot see how it worked before).

Once you run the command above at Malford, disable and re-enable the L2TP server there or disable and re-enable the L2TP client at Tinos, whichever you can do without locking yourself out.

After that, /ip route print where dynamic at Malford should show you a route to 172.16.110.0/24.

There is one more minor mistake at Malford, fix it using
/ip firewall nat set [find action=masquerade] out-interface-list=WAN

If at this point the pinging from something in 172.16.105.0/24 to something in 172.16.110.0/24 and/or vice versa still doesn't work, you'll have to sniff at both CCRs while pinging, to see where the issue is. But if you ping a Windows machine, you have to know that by default the Windows firewall drops ping requests coming from outside the interface's own subnet.

The sniffing command to run at both CCRs would be
/tool sniffer quick ip-protocol=icmp ip-address=172.16.0.0/16

Before running it, make the command line window as wide as your screen allows, so that the interface names would be visible in the output.


In the firewalls, I can see that at Malford, the connections between Tinos LAN subnet and Malford LAN subnet are permitted thanks to the following bits of configuration:

/ip firewall address-list
...
add address=172.16.105.0/24 list=primary-subnets
add address=172.16.110.0/24 list=primary-subnets

/ip firewall filter
... permit some other traffic ...
add action=accept chain=forward dst-address-list=primary-subnets src-address-list=primary-subnets
... drop the rest ...


The traffic coming from Tinos via the L2TP tunnel to Malford is permitted to reach the internet by the other rule in /ip firewall filter, action=accept chain=forward in-interface-list=!WAN out-interface-list=WAN. This rule allows also hosts in all the Malford LAN subnets to reach the internet.


At Tinos, all forwarded connections except those coming in via WAN are currently permitted:
/ip firewall filter
...
add action=drop chain=forward comment="defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new in-interface-list=WAN


So the firewall at Tinos doesn't actively drop even connection attempts from the guest subnet at Malford, but Malford's firewall doesn't allow them to even leave towards Tinos.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Sat Jun 13, 2020 5:18 pm

After that, /ip route print where dynamic at Malford should show you a route to 172.16.110.0/24.
.
This is what I see:
.
[admin@Malford1016] > /ip route print where dynamic
# DST-ADDRESS PREF-SRC GATEWAY DISTANCE
0 DS 0.0.0.0/0 ISP-BT 3
1 ADC 10.10.10.0/24 10.10.10.254 bridgeGuest 0
2 ADC 10.10.20.10/32 172.16.105.1 <l2tp-tinos-l2tp> 0
3 ADC 20.20.20.0/24 20.20.20.254 bridgeDRGuest 0
4 ADC 82.17.140.0/24 82.xx.xxx.24 ISP-Virgin 0
5 ADC 172.16.19.220/32 86.xxx.xx.174 ISP-BT 0
6 ADC 172.16.105.0/24 172.16.105.1 bridge-lan 0
7 DS 172.16.110.0/24 <l2tp-tinos-l2tp> 1
8 ADC 192.168.1.0/24 192.168.1.2 BT 0
.
and
.
[admin@Tinos1016] > /ip route print where dynamic
# DST-ADDRESS PREF-SRC GATEWAY
0 ADS 0.0.0.0/0 l2tp-out1
1 ADS 82.xx.xxx.24/32 192.168.1.1
2 ADC 172.16.105.1/32 10.10.20.10 l2tp-out1
3 ADC 172.16.110.0/24 172.16.110.1 bridge-lan
4 ADC 192.168.1.0/24 192.168.1.2 ether1-isp1
.
Rules adjusted
.
If at this point the pinging from something in 172.16.105.0/24 to something in 172.16.110.0/24 and/or vice versa still doesn't work, you'll have to sniff at both CCRs while pinging, to see where the issue is. But if you ping a Windows machine, you have to know that by default the Windows firewall drops ping requests coming from outside the interface's own subnet.
The sniffing command to run at both CCRs would be
The traffic coming from Tinos via the L2TP tunnel to Malford is permitted to reach the internet by the other rule in /ip firewall filter, action=accept chain=forward in-interface-list=!WAN out-interface-list=WAN. This rule allows also hosts in all the Malford LAN subnets to reach the internet.
.
I am running the command line at both CCRs and have checked that I can ping my Windows machines locally and they respond. Cannot ping past the router in either direction, results are:
.
Malford to Tinos - 172.16.105.1 pinging out
.
<l2tp-tinos-l2tp> 2.756 1 -> 172.16.105.1 172.16.110.1 ip:icmp 56 13 no
<l2tp-tinos-l2tp> 2.862 2 <- 172.16.110.1 172.16.105.1 ip:icmp 56 8 no
<l2tp-tinos-l2tp> 10.441 5 -> 172.16.105.1 172.16.110.17 ip:icmp 56 13 no
<l2tp-tinos-l2tp> 11.443 6 -> 172.16.105.1 172.16.110.17 ip:icmp 56 13 no
.
Tinos to Malford - 172.16.110.1 pinging out
.
l2tp-out1 34.663 1 -> 10.10.20.10 172.16.105.1 ip:icmp 56 3 no
l2tp-out1 34.764 2 <- 172.16.105.1 10.10.20.10 ip:icmp 56 12 no
l2tp-out1 2.44 1 -> 10.10.20.10 172.16.105.104 ip:icmp 56 3 no
l2tp-out1 3.442 2 -> 10.10.20.10 172.16.105.104 ip:icmp 56 3 no
.
Can it be something to do with the 10.10.20.10 component of the Tinos network?
.
Thanks,
.
Andrew.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Sat Jun 13, 2020 6:58 pm

After that, /ip route print where dynamic at Malford should show you a route to 172.16.110.0/24.
This is what I see:
[admin@Malford1016] > /ip route print where dynamic
# DST-ADDRESS PREF-SRC GATEWAY DISTANCE
...
7 DS 172.16.110.0/24 <l2tp-tinos-l2tp> 1
OK, so the whole idea of a missing route was wrong - the missing A at this route means that it is not active. So I've searched through the configuration, and found that there is a static route to 172.16.110.0/24 via gateway=10.10.20.10; I've ignored it as it is commented as "Tinos via SSTP", but in fact the IP address used as gateway here is the one you currently assign to the L2TP tunnel, not the SSTP one. So good, it explains why it worked before, and shifts the focus to something else. You can remove the routes parameter from the /ppp secret, or remove the static route, or keep both in place.

I am running the command line at both CCRs and have checked that I can ping my Windows machines locally and they respond. Cannot ping past the router in either direction, ...
Can it be something to do with the 10.10.20.10 component of the Tinos network?
There is a difference when you ping from the router itself and when you ping from a device on its LAN. When the router itself sends a packet (ping), it chooses the route first, and depending on the gateway interface chosen, it sets the source address of the packet. So when you ping the Malford router itself from Tinos router, the rule in Malford's input chain "accept icmp" accepts the ping request; when you ping something else in 172.16.105.0/24, the forward chain in Malford's firewall handles the ping request, and there is no rule which would accept packets from 10.10.20.10. But if you ping from some other device in Tinos' subnet, the source address will be 172.16.110.x (the exceptions from NAT in Tinos' /ip firewall nat will prevent it from being changed), so the firewall at Malford will let it through. You can also add src-address=172.16.110.1 to the ping command when pinging 172.16.105.104 from the Tinos router, it should make the firewall at Malford let the request through.

But in the opposite direction I still cannot see the reason why it doesn't work except something on the 172.16.110.17 itself - Windows firewall, missing default route, ...

That's the reason why I wanted you to sniff at both CCRs simultaneously - it will show how far the request and the response will get. If everything worked properly, at Tinos, you would see the request from Malford's CCR to 172.16.110.17 to come in via L2TP and leave towards the LAN device via bridge-lan and then the physical Ethernet interface; the response should pass through the same interfaces in the opposite direction. If you can see the request to leave via Ethernet, but no response at all, the issue is on the 172.16.110.17 itself. If you can see the request or the response to come in but not leave, the issue is in the CCR. In the opposite direction (Tinos to Malford), observe the same.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Sat Jun 13, 2020 7:30 pm

You can remove the routes parameter from the /ppp secret, or remove the static route, or keep both in place.
.
Sorry am confused, on the Malford router, in PPP Secrets I should remove which? There is no PPP Secrets entry in the Tinos router, presumably because that is the client
.
fgfgfg.JPG
.
But if you ping from some other device in Tinos' subnet, the source address will be 172.16.110.x (the exceptions from NAT in Tinos' /ip firewall nat will prevent it from being changed), so the firewall at Malford will let it through. You can also add src-address=172.16.110.1 to the ping command when pinging 172.16.105.104 from the Tinos router, it should make the firewall at Malford let the request through.
.
Its strange because pretty much all devices will respond to ping. I've got 2 Raspberry Pi which report humidity, a couple of APs as well in the Tinos network.
.
But in the opposite direction I still cannot see the reason why it doesn't work except something on the 172.16.110.17 itself - Windows firewall, missing default route, ...
.
Want me to post the original Malford config from before we change the Firewall rules, might have a clue on how the 10.10.20.10 network was handled.

Regards,

Andrew.
You do not have the required permissions to view the files attached to this post.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Sat Jun 13, 2020 7:56 pm

Sorry am confused, on the Malford router, in PPP Secrets I should remove which? There is no PPP Secrets entry in the Tinos router, presumably because that is the client
You could remove the contents of the Routes field in this window, but it is not necessary as I wrote. The two routes (the static one and the one added by L2TP server) are effectively identical so you may keep both in place.

Its strange because pretty much all devices will respond to ping. I've got 2 Raspberry Pi which report humidity, a couple of APs as well in the Tinos network.
Respond to ping from LAN or respond to ping via the tunnel? To respond to ping from LAN, the device needs no route at all.

While the action=masquerade rule at Malford was not restricted, whatever was leaving towards the LAN subnet was NATed to the Malford CCR's own address, so along with no restrictions in the previous version of the Malford firewall it explains why pings from Tinos CCR to devices in Malford LAN subnet worked before.

Want me to post the original Malford config from before we change the Firewall rules, might have a clue on how the 10.10.20.10 network was handled.
As I've discovered the route, we already know how it was handled. What I want you to do is to sniff the way I've described in the previous post. The current firewall rules at Tinos really do not restrict pings towards its LAN subnet coming from anywhere but WAN.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Sat Jun 13, 2020 8:34 pm

I have not changed the PPP Secrets in the Malford router.
.
While the action=masquerade rule at Malford was not restricted, whatever was leaving towards the LAN subnet was NATed to the Malford CCR's own address, so along with no restrictions in the previous version of the Malford firewall it explains why pings from Tinos CCR to devices in Malford LAN subnet worked before.
.
I presume that is not a good thing?
.
I am Pinging from Tinos CCR in this way
.
ping src-address=172.16.110.1 172.16.105.104
.
The result from the sniffer is
.
INTERFACE TIME NUM DIR SRC-MAC DST-MAC VLAN SRC-ADDRESS DST-ADDRESS PROTOCOL SIZE CPU FP
l2tp-out1 2.108 1 -> 172.16.110.1 172.16.105.104 ip:icmp 56 3 no
.
Worth noting that I get this when I try the same command in a DOS box in Windows:
.
C:\Users\ag> ping src-address=172.16.110.1 172.16.105.104
Ping request could not find host src-address=172.16.110.1. Please check the name and try again.
.
As I've discovered the route, we already know how it was handled. What I want you to do is to sniff the way I've described in the previous post. The current firewall rules at Tinos really do not restrict pings towards its LAN subnet coming from anywhere but WAN.
.
Below I have added the Sniffer result from Pinging from the Windows PC in the Tinos network to the Malford Router
.
INTERFACE TIME NUM DIR SRC-MAC DST-MAC VLAN SRC-ADDRESS DST-ADDRESS PROTOCOL SIZE CPU FP
ether9 5.059 7 <- F8:B1:56:A7:E4:9A D4:CA:6D:AE:5F:A7 172.16.110.17 172.16.105.1 ip:icmp 74 7 no
bridge-lan 5.059 8 <- F8:B1:56:A7:E4:9A D4:CA:6D:AE:5F:A7 172.16.110.17 172.16.105.1 ip:icmp 74 7 no
l2tp-out1 5.059 9 -> 172.16.110.17 172.16.105.1 ip:icmp 60 7 no
l2tp-out1 5.154 10 <- 172.16.105.1 172.16.110.17 ip:icmp 60 12 no
.
Thanks,
.
Andrew.
.
EDIT
.
I cannot ping from a Malford PC (172.16.105.104) the Tinos router 172.16.110.1
but I can ping from a Tinos PC (172.16.110.17) the Malford router 172.16.105.1
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Sat Jun 13, 2020 9:03 pm

I presume that is not a good thing?
Normally you should use NAT only where absolutely necessary. So while it would probably resolve the current ping issue, you might have issues in future.

I am Pinging from Tinos CCR in this way
ping src-address=172.16.110.1 172.16.105.104
The result from the sniffer is
INTERFACE TIME NUM DIR SRC-MAC DST-MAC VLAN SRC-ADDRESS DST-ADDRESS PROTOCOL SIZE CPU FP
l2tp-out1 2.108 1 -> 172.16.110.1 172.16.105.104 ip:icmp 56 3 no
Good, but what's the result from the sniffer at Malford at that time? Sniffing at Tinos shows that the packet leaves with the necessary source address, but nevertheless no response arrives. What is going on at that time at Malford is what we need to know.

Worth noting that I get this when I try the same command in a DOS box in Windows:
C:\Users\ag> ping src-address=172.16.110.1 172.16.105.104
Ping request could not find host src-address=172.16.110.1. Please check the name and try again.
Of course, because MS-DOS syntax of ping parameters is different from the RouterOS one. The particular IP address to ping from is specified using -S on Windows. But you do not need to do that on a Windows machine connected to Tinos LAN, as its own address already is 172.16.110.x, isn't it? Plus you cannot choose a random source address, you only choose from the ones existing on the pinging device.

Below I have added the Sniffer result from Pinging from the Windows PC in the Tinos network to the Malford Router
INTERFACE TIME NUM DIR SRC-MAC DST-MAC VLAN SRC-ADDRESS DST-ADDRESS PROTOCOL SIZE CPU FP
ether9 5.059 7 <- F8:B1:56:A7:E4:9A D4:CA:6D:AE:5F:A7 172.16.110.17 172.16.105.1 ip:icmp 74 7 no
bridge-lan 5.059 8 <- F8:B1:56:A7:E4:9A D4:CA:6D:AE:5F:A7 172.16.110.17 172.16.105.1 ip:icmp 74 7 no
l2tp-out1 5.059 9 -> 172.16.110.17 172.16.105.1 ip:icmp 60 7 no
l2tp-out1 5.154 10 <- 172.16.105.1 172.16.110.17 ip:icmp 60 12 no

Are you telling me that the pings time out on the PC because the ping responses from the Malford router come back to the Tinos CCR via l2tp-out1 but the Tinos CCR doesn't forward them to the PC via bridge-lan and ether9? Or you just haven't pasted the following lines after the last one?
bridge-lan 5.154 11 <- D4:CA:6D:AE:5F:A7 F8:B1:56:A7:E4:9A 172.16.105.1 172.16.110.17 ip:icmp 74 12 no
ether9 5.154 12 <- D4:CA:6D:AE:5F:A7 F8:B1:56:A7:E4:9A 172.16.105.1 172.16.110.17 ip:icmp 74 12 no
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Sat Jun 13, 2020 9:17 pm

Good, but what's the result from the sniffer at Malford at that time? Sniffing at Tinos shows that the packet leaves with the necessary source address, but nevertheless no response arrives. What is going on at that time at Malford is what we need to know.
.
Apologies, I completely misunderstood the use of the Sniffer. Here goes.
.
Ping from Tinos CCR "ping src-address=172.16.110.1 172.16.105.104"
.
Tinos Sniffer
.
INTERFACE TIME NUM DIR SRC-MAC DST-MAC VLAN SRC-ADDRESS DST-ADDRESS PROTOCOL SIZE CPU FP
l2tp-out1 32.086 1 -> 172.16.110.1 172.16.105.104 ip:icmp 56 3 no
l2tp-out1 33.089 2 -> 172.16.110.1 172.16.105.104 ip:icmp 56 3 no
l2tp-out1 34.091 3 -> 172.16.110.1 172.16.105.104 ip:icmp 56 3 no
.
Malford Sniffer
.
INTERFACE TIME NUM DIR SRC-MAC DST-MAC VLAN SRC-ADDRESS DST-ADDRESS PROTOCOL SIZE CPU FP
bridge-lan 10.715 32 -> D4:CA:6D:AE:39:AE BC:5F:F4:B8:DA:81 172.16.110.1 172.16.105.104 ip:icmp 70 8 no
ether3 10.715 33 -> D4:CA:6D:AE:39:AE BC:5F:F4:B8:DA:81 172.16.110.1 172.16.105.104 ip:icmp 70 8 no
<l2tp-tinos-l2tp> 11.716 34 <- 172.16.110.1 172.16.105.104 ip:icmp 56 8 no
bridge-lan 11.716 35 -> D4:CA:6D:AE:39:AE BC:5F:F4:B8:DA:81 172.16.110.1 172.16.105.104 ip:icmp 70 8 no
ether3 11.716 36 -> D4:CA:6D:AE:39:AE BC:5F:F4:B8:DA:81 172.16.110.1 172.16.105.104 ip:icmp 70 8 no
<l2tp-tinos-l2tp> 12.717 37 <- 172.16.110.1 172.16.105.104 ip:icmp 56 8 no
bridge-lan 12.717 38 -> D4:CA:6D:AE:39:AE BC:5F:F4:B8:DA:81 172.16.110.1 172.16.105.104 ip:icmp 70 8 no
ether3 12.717 39 -> D4:CA:6D:AE:39:AE BC:5F:F4:B8:DA:81 172.16.110.1 172.16.105.104 ip:icmp 70 8 no
.
Thanks,
.
Andrew.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Sat Jun 13, 2020 10:32 pm

OK, so as you can see by the timestamps and order numbers of the packets, the request arrives from Tinos via the tunnel and is delivered to the destination via bridge-lan and ether3, but no response arrives. So whatever 172.16.105.104 is, it does not respond to a ping request coming from 172.16.110.1, or possibly sends the response via some other gateway.

You could add a targeted masquerade rule, /ip firewall nat add src-address=172.16.110.0/24 out-interface=bridge-lan action=masquerade, to confirm that the current issue is really the routing/firewall on the 172.16.105.104 itself. A similar rule, /ip firewall nat add src-address=172.16.105.0/24 out-interface=bridge-lan action=masquerade can be used at Tinos to verify that it "fixes" pinging from Malford CCR to 172.16.110.17. But both are workarounds and may cause surprises if left in place. If it is just the firewall on the target devices which prevents pings from being responded, it's not a big deal to live without ping; if it is a routing issue, other things will fail - after all, I suppose you wanted transparency between the two sites' LANs on purpose?

Also, you might want to ping 1.1.1.1, 8.8.8.8, or 9.9.9.9 from the Tinos CCR while sniffing at both CCRs, to see whether the internet is reachable from Tinos via the L2TP tunnel.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Sat Jun 13, 2020 11:53 pm

You could add a targeted masquerade rule, /ip firewall nat add src-address=172.16.110.0/24 out-interface=bridge-lan action=masquerade, to confirm that the current issue is really the routing/firewall on the 172.16.105.104 itself. A similar rule, /ip firewall nat add src-address=172.16.105.0/24 out-interface=bridge-lan action=masquerade
.
What chain is it? srcnat?
.
Also, you might want to ping 1.1.1.1, 8.8.8.8, or 9.9.9.9 from the Tinos CCR while sniffing at both CCRs, to see whether the internet is reachable from Tinos via the L2TP tunnel.
.
Ping from Tinos router to 1.1.1.1 shows nothing on both Sniffers. Tinos PC ping to 1.1.1.1 shows the following on the Tinos side and nothing on the Malford side.
.
Tinos
INTERFACE TIME NUM DIR SRC-MAC DST-MAC VLAN SRC-ADDRESS DST-ADDRESS PROTOCOL SIZE CPU FP
bridge-lan 1.35 1 -> D4:CA:6D:AE:5F:A7 F8:B1:56:A7:E4:9A 192.168.1.2 172.16.110.17 ip:icmp 94 4 no
ether9 1.35 2 -> D4:CA:6D:AE:5F:A7 F8:B1:56:A7:E4:9A 192.168.1.2 172.16.110.17 ip:icmp 94 4 no
ether9 3.359 3 <- F8:B1:56:A7:E4:9A D4:CA:6D:AE:5F:A7 172.16.110.17 1.1.1.1 ip:icmp 74 6 no
bridge-lan 3.359 4 <- F8:B1:56:A7:E4:9A D4:CA:6D:AE:5F:A7 172.16.110.17 1.1.1.1 ip:icmp 74 6 no
bridge-lan 3.469 5 -> D4:CA:6D:AE:5F:A7 F8:B1:56:A7:E4:9A 1.1.1.1 172.16.110.17 ip:icmp 74 12 no
ether9 3.469 6 -> D4:CA:6D:AE:5F:A7 F8:B1:56:A7:E4:9A 1.1.1.1 172.16.110.17 ip:icmp 74 12 no
.
Running tracert 1.1.1.1 from the Tinos PC showed traffic on both sides however I cannot grab the sniffer output. I have tried "set file-name tinos.txt" the perform Dump then quit but the content of the file is not plain text.
.
Thanks,
.
Andrew.
Last edited by AJSG on Sun Jun 14, 2020 12:11 am, edited 1 time in total.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Sun Jun 14, 2020 12:07 am

What chain is it? forward or input?
action=masquerade can only be placed in table nat, chain srcnat. Sorry, forgot the chain again.

Ping from Tinos router to 1.1.1.1 shows nothing on both Sniffers. Tinos PC ping to 1.1.1.1 shows the following on the Tinos side and nothing on the Malford side.

Tinos
INTERFACE TIME NUM DIR SRC-MAC DST-MAC VLAN SRC-ADDRESS DST-ADDRESS PROTOCOL SIZE CPU FP
bridge-lan 1.35 1 -> D4:CA:6D:AE:5F:A7 F8:B1:56:A7:E4:9A 192.168.1.2 172.16.110.17 ip:icmp 94 4 no
ether9 1.35 2 -> D4:CA:6D:AE:5F:A7 F8:B1:56:A7:E4:9A 192.168.1.2 172.16.110.17 ip:icmp 94 4 no
ether9 3.359 3 <- F8:B1:56:A7:E4:9A D4:CA:6D:AE:5F:A7 172.16.110.17 1.1.1.1 ip:icmp 74 6 no
bridge-lan 3.359 4 <- F8:B1:56:A7:E4:9A D4:CA:6D:AE:5F:A7 172.16.110.17 1.1.1.1 ip:icmp 74 6 no
bridge-lan 3.469 5 -> D4:CA:6D:AE:5F:A7 F8:B1:56:A7:E4:9A 1.1.1.1 172.16.110.17 ip:icmp 74 12 no
ether9 3.469 6 -> D4:CA:6D:AE:5F:A7 F8:B1:56:A7:E4:9A 1.1.1.1 172.16.110.17 ip:icmp 74 12 no
.
Running tracert 1.1.1.1 from the Tinos PC showed traffic on both sides however I cannot grab the sniffer output. I have tried "set file-name tinos.txt" the perform Dump then quit but the content of the file is not plain text.
The masquerade rule at Tinos is also too wide, so it rewrites the source address to the one assigned to the L2TP interface when the packets leave through that interface if their destination address is not 172.16.100.0/24 or 172.16.105.0/24. So please fix it too: /ip firewall nat set [find action=masquerade !out-interface-list] out-interface-list=WAN. I suppose you were still sniffing with ip-address=172.16.110.0/24 - sniffing with ip-address=1.1.1.1 instead would have revealed this. On Malford, the src-nat (masquerade) takes place as the packets leave through WAN, so unless you change the sniffing filter there, you'll see the requests and responses only on L2TP, not on WAN.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Sun Jun 14, 2020 12:18 am

action=masquerade can only be placed in table nat, chain srcnat. Sorry, forgot the chain again.
.
I have added to Malford:
/ip firewall nat add src-address=172.16.110.0/24 out-interface=bridge-lan action=masquerade
chain: srcnat
.
I have added to Tinos:
/ip firewall nat add src-address=172.16.105.0/24 out-interface=bridge-lan action=masquerade
chain: srcnat
.
I can confirm that Tinos PC can now see "shared folders" of Malford devices, and that Tinos external IP is that of Malford. What is the danger with this kind of rule structure?
.
Much obliged
.
Andrew.
.
EDIT
.
Worth noting I can ping from Tinos to Malford, but not the other way round. Also Tinos can see Malford shared folder, but not the other way round.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Sun Jun 14, 2020 12:49 am

So please fix it too: /ip firewall nat set [find action=masquerade !out-interface-list] out-interface-list=WAN
.
I changed the out-interface-list to WAN for the Malford router but it disabled access to the network from the Tinos side. I did disable and re-enable to Interface just to make sure. Worth noting that both NATs have several rules in them:
.
Tinos NAT
.
/ip firewall nat
add action=accept chain=srcnat dst-address=172.16.105.0/24 src-address=172.16.110.0/24
add action=accept chain=srcnat dst-address=172.16.100.0/24 src-address=172.16.110.0/24
add action=masquerade chain=srcnat src-address=172.16.110.0/24
add action=dst-nat chain=dstnat disabled=yes dst-port=8443 protocol=tcp src-address=82.17.140.24 to-addresses=172.16.110.125 to-ports=443
add action=masquerade chain=srcnat out-interface=bridge-lan src-address=172.16.105.0/24
.
Malford NAT
.
/ip firewall nat
add action=masquerade chain=srcnat comment="NAT to Internet" out-interface-list=WAN
add action=dst-nat chain=dstnat disabled=yes dst-port=53 protocol=udp src-address-list=allowed to-addresses=172.16.105.2
add action=src-nat chain=srcnat disabled=yes dst-port=53 protocol=tcp to-addresses=172.16.105.2 to-ports=53
add action=src-nat chain=srcnat disabled=yes dst-port=53 protocol=udp to-addresses=172.16.105.2 to-ports=53
add action=masquerade chain=srcnat out-interface=bridge-lan src-address=172.16.110.0/24
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Sun Jun 14, 2020 11:41 am

I don't understand why adding out-interface-list=WAN to the masquerade rule at Malford should prevent Tinos LAN devices from reaching the internet via the L2TP tunnel. So again sniffing needs to come at hand to visualize what went wrong.

Same case with Malford CCR not being able to ping Tinos LAN, and Malford LAN devices not seeing Tinos LAN devices' shared folders. Only pinging PC to PC and sniffing at both CCRs can give a clue.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Sun Jun 14, 2020 12:04 pm

Only pinging PC to PC and sniffing at both CCRs can give a clue.
.
Agreed this is very weird
.
  • Malford router can ping from 172.16.105.1 to 172.16.110.17
    PC on Malford network cannot ping from 172.16.105.104 to 172.16.110.17
    .
    Tinos router cannot ping from 172.16.110.1 to 172.16.105.104
    PC on Tinos network can ping from 172.16.110.17 to 172.16.105.104
.
Could you please give me some hints on "sniffing" usage. There is so much data in the Malford network that it all goes off screen and I cannot copy/paste it.
.
Thanks.
.
Andrew
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Sun Jun 14, 2020 12:35 pm

The key is the capture filter - you can use a list of addresses in the ip-address field, and you can use ip-protocol=icmp to sniff only ICMP packets. The condition between different filter fields is "and", the conditions between list items for a single field are "or". So /tool sniffer quick ip-protocol=icmp ip-address=1.1.1.1,172.16.110.17 will capture all ICMP packets from or to 1.1.1.1 as well as all ICMP packets to or from 172.16.110.17. You can also use interface as a filtering criterion, but it would be counter-productive for this particular type of analysis.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Sun Jun 14, 2020 2:01 pm

You can also use interface as a filtering criterion, but it would be counter-productive for this particular type of analysis.
.
My problem is that when I run
.
/tool sniffer quick ip-protocol=icmp ip-address=172.16.0.0/16
.
permissible keys are only [Q quit I D dump I C-z pause]
.
On the Tinos network not much happens so its ok, but on the Malford network it captures a ton of data which goes off screen so I cannot see data. I am trying my best but sometimes the Malford capture just creates too much data. How can I display all results in a Terminal.
.
Results ping from Tinos PC (172.16.110.17 ) to Malford PC (172.16.105.104). Ping is a success.
.
Tinos router sniffer.
.
INTERFACE TIME NUM DIR SRC-MAC DST-MAC VLAN SRC-ADDRESS DST-ADDRESS PROTOCOL SIZE CPU FP
bridge-lan 5.897 5 -> D4:CA:6D:AE:5F:A7 F8:B1:56:A7:E4:9A 172.16.105.104 172.16.110.17 ip:icmp 74 12 no
ether9 5.897 6 -> D4:CA:6D:AE:5F:A7 F8:B1:56:A7:E4:9A 172.16.105.104 172.16.110.17 ip:icmp 74 12 no
ether9 6.807 7 <- F8:B1:56:A7:E4:9A D4:CA:6D:AE:5F:A7 172.16.110.17 172.16.105.104 ip:icmp 74 14 no
bridge-lan 6.807 8 <- F8:B1:56:A7:E4:9A D4:CA:6D:AE:5F:A7 172.16.110.17 172.16.105.104 ip:icmp 74 14 no
l2tp-out1 6.807 9 -> 172.16.110.17 172.16.105.104 ip:icmp 60 14 no
l2tp-out1 6.903 10 <- 172.16.105.104 172.16.110.17 ip:icmp 60 12 no
bridge-lan 6.903 11 -> D4:CA:6D:AE:5F:A7 F8:B1:56:A7:E4:9A 172.16.105.104 172.16.110.17 ip:icmp 74 12 no
ether9 6.903 12 -> D4:CA:6D:AE:5F:A7 F8:B1:56:A7:E4:9A 172.16.105.104 172.16.110.17 ip:icmp 74 12 no
ether9 7.816 13 <- F8:B1:56:A7:E4:9A D4:CA:6D:AE:5F:A7 172.16.110.17 172.16.105.104 ip:icmp 74 14 no
bridge-lan 7.816 14 <- F8:B1:56:A7:E4:9A D4:CA:6D:AE:5F:A7 172.16.110.17 172.16.105.104 ip:icmp 74 14 no
l2tp-out1 7.816 15 -> 172.16.110.17 172.16.105.104 ip:icmp 60 14 no
l2tp-out1 7.918 16 <- 172.16.105.104 172.16.110.17 ip:icmp 60 12 no
bridge-lan 7.918 17 -> D4:CA:6D:AE:5F:A7 F8:B1:56:A7:E4:9A 172.16.105.104 172.16.110.17 ip:icmp 74 12 no
ether9 7.918 18 -> D4:CA:6D:AE:5F:A7 F8:B1:56:A7:E4:9A 172.16.105.104 172.16.110.17 ip:icmp 74 12 no
ether9 8.821 19 <- F8:B1:56:A7:E4:9A D4:CA:6D:AE:5F:A7 172.16.110.17 172.16.105.104 ip:icmp 74 14 no
bridge-lan 8.821 20 <- F8:B1:56:A7:E4:9A D4:CA:6D:AE:5F:A7 172.16.110.17 172.16.105.104 ip:icmp 74 14 no
l2tp-out1 8.821 21 -> 172.16.110.17 172.16.105.104 ip:icmp 60 14 no
l2tp-out1 8.92 22 <- 172.16.105.104 172.16.110.17 ip:icmp 60 12 no
bridge-lan 8.92 23 -> D4:CA:6D:AE:5F:A7 F8:B1:56:A7:E4:9A 172.16.105.104 172.16.110.17 ip:icmp 74 12 no
ether9 8.92 24 -> D4:CA:6D:AE:5F:A7 F8:B1:56:A7:E4:9A 172.16.105.104 172.16.110.17 ip:icmp 74 12 no
.
Malford router sniffer
.
INTERFACE TIME NUM DIR SRC-MAC DST-MAC VLAN SRC-ADDRESS DST-ADDRESS PROTOCOL SIZE CPU FP
bridge-lan 3.834 5 <- BC:5F:F4:B8:DA:81 D4:CA:6D:AE:39:AE 172.16.105.104 172.16.105.1 ip:icmp 74 15 no
<l2tp-tinos-l2tp> 3.834 6 -> 172.16.105.104 172.16.110.17 ip:icmp 60 15 no
<l2tp-tinos-l2tp> 4.836 7 <- 172.16.110.17 172.16.105.104 ip:icmp 60 2 no
bridge-lan 4.836 8 -> D4:CA:6D:AE:39:AE BC:5F:F4:B8:DA:81 172.16.105.1 172.16.105.104 ip:icmp 74 2 no
ether3 4.836 9 -> D4:CA:6D:AE:39:AE BC:5F:F4:B8:DA:81 172.16.105.1 172.16.105.104 ip:icmp 74 2 no
ether3 4.837 10 <- BC:5F:F4:B8:DA:81 D4:CA:6D:AE:39:AE 172.16.105.104 172.16.105.1 ip:icmp 74 15 no
bridge-lan 4.837 11 <- BC:5F:F4:B8:DA:81 D4:CA:6D:AE:39:AE 172.16.105.104 172.16.105.1 ip:icmp 74 15 no
<l2tp-tinos-l2tp> 4.837 12 -> 172.16.105.104 172.16.110.17 ip:icmp 60 15 no
<l2tp-tinos-l2tp> 5.844 13 <- 172.16.110.17 172.16.105.104 ip:icmp 60 2 no
bridge-lan 5.844 14 -> D4:CA:6D:AE:39:AE BC:5F:F4:B8:DA:81 172.16.105.1 172.16.105.104 ip:icmp 74 2 no
ether3 5.844 15 -> D4:CA:6D:AE:39:AE BC:5F:F4:B8:DA:81 172.16.105.1 172.16.105.104 ip:icmp 74 2 no
ether3 5.844 16 <- BC:5F:F4:B8:DA:81 D4:CA:6D:AE:39:AE 172.16.105.104 172.16.105.1 ip:icmp 74 15 no
bridge-lan 5.844 17 <- BC:5F:F4:B8:DA:81 D4:CA:6D:AE:39:AE 172.16.105.104 172.16.105.1 ip:icmp 74 15 no
<l2tp-tinos-l2tp> 5.844 18 -> 172.16.105.104 172.16.110.17 ip:icmp 60 15 no
<l2tp-tinos-l2tp> 6.85 19 <- 172.16.110.17 172.16.105.104 ip:icmp 60 2 no
bridge-lan 6.85 20 -> D4:CA:6D:AE:39:AE BC:5F:F4:B8:DA:81 172.16.105.1 172.16.105.104 ip:icmp 74 2 no
ether3 6.85 21 -> D4:CA:6D:AE:39:AE BC:5F:F4:B8:DA:81 172.16.105.1 172.16.105.104 ip:icmp 74 2 no
ether3 6.85 22 <- BC:5F:F4:B8:DA:81 D4:CA:6D:AE:39:AE 172.16.105.104 172.16.105.1 ip:icmp 74 15 no
bridge-lan 6.85 23 <- BC:5F:F4:B8:DA:81 D4:CA:6D:AE:39:AE 172.16.105.104 172.16.105.1 ip:icmp 74 15 no
<l2tp-tinos-l2tp> 6.85 24 -> 172.16.105.104 172.16.110.17 ip:icmp 60 15 no
.
Results ping from Tinos router (172.16.110.1) to Malford PC (172.16.105.104). Ping failure
.
Tinos router sniffer
.
INTERFACE TIME NUM DIR SRC-MAC DST-MAC VLAN SRC-ADDRESS DST-ADDRESS PROTOCOL SIZE CPU FP
l2tp-out1 3.974 1 -> 10.10.20.10 172.16.105.104 ip:icmp 56 3 no
l2tp-out1 4.976 2 -> 10.10.20.10 172.16.105.104 ip:icmp 56 3 no
l2tp-out1 5.978 3 -> 10.10.20.10 172.16.105.104 ip:icmp 56 3 no
l2tp-out1 6.98 4 -> 10.10.20.10 172.16.105.104 ip:icmp 56 3 no
l2tp-out1 7.982 5 -> 10.10.20.10 172.16.105.104 ip:icmp 56 3 no
.
Malford router sniffer
.
INTERFACE TIME NUM DIR SRC-MAC DST-MAC VLAN SRC-ADDRESS DST-ADDRESS PROTOCOL SIZE CPU FP
<l2tp-tinos-l2tp> 2.847 1 <- 10.10.20.10 172.16.105.104 ip:icmp 56 2 no
<l2tp-tinos-l2tp> 3.85 2 <- 10.10.20.10 172.16.105.104 ip:icmp 56 2 no
<l2tp-tinos-l2tp> 4.852 3 <- 10.10.20.10 172.16.105.104 ip:icmp 56 2 no
<l2tp-tinos-l2tp> 5.854 4 <- 10.10.20.10 172.16.105.104 ip:icmp 56 2 no
<l2tp-tinos-l2tp> 6.856 5 <- 10.10.20.10 172.16.105.104 ip:icmp 56 2 no
<l2tp-tinos-l2tp> 7.858 6 <- 10.10.20.10 172.16.105.104 ip:icmp 56 2 no
.
Results ping from Malford PC (172.16.105.104) to Tinos PC (172.16.110.17 ). Ping failure
.
Nothing in either sniffer.
.
Results ping from Malford router (172.16.105.1) to Tinos PC (172.16.110.17 ). Ping is a success
.
Tinos router sniffer
.
INTERFACE TIME NUM DIR SRC-MAC DST-MAC VLAN SRC-ADDRESS DST-ADDRESS PROTOCOL SIZE CPU FP
bridge-lan 6.127 11 <- F8:B1:56:A7:E4:9A D4:CA:6D:AE:5F:A7 172.16.110.17 172.16.110.1 ip:icmp 70 7 no
l2tp-out1 6.127 12 -> 172.16.110.17 172.16.105.1 ip:icmp 56 7 no
l2tp-out1 7.119 13 <- 172.16.105.1 172.16.110.17 ip:icmp 56 12 no
bridge-lan 7.119 14 -> D4:CA:6D:AE:5F:A7 F8:B1:56:A7:E4:9A 172.16.110.1 172.16.110.17 ip:icmp 70 12 no
ether9 7.119 15 -> D4:CA:6D:AE:5F:A7 F8:B1:56:A7:E4:9A 172.16.110.1 172.16.110.17 ip:icmp 70 12 no
ether9 7.119 16 <- F8:B1:56:A7:E4:9A D4:CA:6D:AE:5F:A7 172.16.110.17 172.16.110.1 ip:icmp 70 7 no
bridge-lan 7.119 17 <- F8:B1:56:A7:E4:9A D4:CA:6D:AE:5F:A7 172.16.110.17 172.16.110.1 ip:icmp 70 7 no
.
Malford router sniffer
.
INTERFACE TIME NUM DIR SRC-MAC DST-MAC VLAN SRC-ADDRESS DST-ADDRESS PROTOCOL SIZE CPU FP
<l2tp-tinos-l2tp> 1.762 1 -> 172.16.105.1 172.16.110.17 ip:icmp 56 2 no
<l2tp-tinos-l2tp> 1.859 2 <- 172.16.110.17 172.16.105.1 ip:icmp 56 2 no
<l2tp-tinos-l2tp> 2.761 3 -> 172.16.105.1 172.16.110.17 ip:icmp 56 2 no
<l2tp-tinos-l2tp> 2.864 4 <- 172.16.110.17 172.16.105.1 ip:icmp 56 2 no
<l2tp-tinos-l2tp> 3.766 5 -> 172.16.105.1 172.16.110.17 ip:icmp 56 2 no
<l2tp-tinos-l2tp> 3.858 6 <- 172.16.110.17 172.16.105.1 ip:icmp 56 2 no
<l2tp-tinos-l2tp> 4.77 7 -> 172.16.105.1 172.16.110.17 ip:icmp 56 2 no
<l2tp-tinos-l2tp> 4.866 8 <- 172.16.110.17 172.16.105.1 ip:icmp 56 2 no
<l2tp-tinos-l2tp> 5.778 9 -> 172.16.105.1 172.16.110.17 ip:icmp 56 2 no
<l2tp-tinos-l2tp> 5.869 10 <- 172.16.110.17 172.16.105.1 ip:icmp 56 2 no
.
The net result of this is that Tinos devices can see Malford devices but not the other way round.
.
Thanks,
.
Andrew.
.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Sun Jun 14, 2020 2:50 pm

My problem is that when I run /tool sniffer quick ip-protocol=icmp ip-address=172.16.0.0/16 , permissible keys are only [Q quit I D dump I C-z pause].
On the Tinos network not much happens so its ok, but on the Malford network it captures a ton of data which goes off screen so I cannot see data. I am trying my best but sometimes the Malford capture just creates too much data. How can I display all results in a Terminal.
You can use e.g. /tool sniffer packet print detail where src-address~"172.16.110.17" or dst-address~"172.16.110.17" after stopping the sniffer, or you can use more selective address combinations (172.16.110.17 rather than 172.16.0.0/16) already while sniffing.

You can also specify packet size while pinging (size=1000 in RouterOS, -l 1000 on Windows), and then use /tool sniffer packet print where size>1000 to show only this ping stream. Unfortunately, size cannot be used as capture filter. And you can also limit the number of packets to be sent when pinging (count=1, -n 1)

1. Results ping from Tinos PC (172.16.110.17 ) to Malford PC (172.16.105.104). Ping is a success.
In this scenario, no source NAT takes place at Tinos as the packets leave towards Malford through the L2TP, so filter rules at Malford accept the request packets, and source NAT does take place at Malford as the packets leave towards the destination PC via the bridge-nat, which makes the requests acceptable for the PC or, more likely as case 3 shows, allows the PC to send the responses.

2. Results ping from Tinos router (172.16.110.1) to Malford PC (172.16.105.104). Ping failure
At Tinos, you haven't used src-address=172.16.110.1 with the ping, so the packets' source address is 10.10.20.10, hence the request packets arrive to Malford but the firewall filter drops them.

3. Results ping from Malford PC (172.16.105.104) to Tinos PC (172.16.110.17 ). Ping failure
This clearly shows that a route to 172.16.110.17 is missing or wrong on 172.16.105.104. If the route on the PC was sending the packet using Malford CCR's 172.16.110.1 as a gateway, you would see the packet at ether3 and bridge-lan in Malford sniffer before it could be eventually dropped by the firewall. No wonder that case 1 only works thanks to the src-nat applied on bridge-lan. What exactly is 172.16.105.104, and what is the network configuration on it?

4. Results ping from Malford router (172.16.105.1) to Tinos PC (172.16.110.17 ). Ping is a success
Here, the src-nat on bridge-lan at Tinos makes the 172.16.110.17 see the packets actually coming from Malford CCR appear as if sent by Tinos CCR.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Sun Jun 14, 2020 3:42 pm

You can use e.g. /tool sniffer packet print detail where src-address~"172.16.110.17" or dst-address~"172.16.110.17" after stopping the sniffer, or you can use more selective address combinations (172.16.110.17 rather than 172.16.0.0/16) already while sniffing.
.
Is there no way to force it to scroll the results or save the results in text? When I save to a file I get gobbledegook.

2. Results ping from Tinos router (172.16.110.1) to Malford PC (172.16.105.104). Ping failure
At Tinos, you haven't used src-address=172.16.110.1 with the ping, so the packets' source address is 10.10.20.10, hence the request packets arrive to Malford but the firewall filter drops them.
.
That works with src. Thanks.
.
3. Results ping from Malford PC (172.16.105.104) to Tinos PC (172.16.110.17 ). Ping failure
This clearly shows that a route to 172.16.110.17 is missing or wrong on 172.16.105.104. If the route on the PC was sending the packet using Malford CCR's 172.16.110.1 as a gateway, you would see the packet at ether3 and bridge-lan in Malford sniffer before it could be eventually dropped by the firewall. No wonder that case 1 only works thanks to the src-nat applied on bridge-lan. What exactly is 172.16.105.104, and what is the network configuration on it?
.
This is the big issue, the x.104 is a Win 10 Pro PC, but I have also tried from an Ubuntu box and a Raspberry Pi. Results always negative which means Malford devices cannot see Tinos devices.
.
The Malford PC uses 172.16.105.1 as a gateway, Tinos is x.110.1. What else can I try if in this test the sniffer is not outputting anything. The Gateway on the PC is correct 172.16.105.1.
.
By the way disabling the NAT rule on the Malford router:
/ip firewall nat add src-address=172.16.110.0/24 out-interface=bridge-lan action=masquerade chain: srcnat
.
Makes no difference, presume as it is there to control traffic originating from the Malford netwrok.
.
Thanks,
.
Andrew.
.
EDIT should I look to run sniffer on 10.10.20.10 when I ping from Malford to Tinos? Or would I just see incoming packets?
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Sun Jun 14, 2020 4:09 pm

Is there no way to force it to scroll the results or save the results in text? When I save to a file I get gobbledegook.
While /tool sniffer quick is running, you cannot scroll. Once you stop it and use /tool sniffer packet print, you can scroll, and /tool sniffer packet print detail file=something also shows the information in a useful format, albeit a bit verbose. Without detail, it is too brief (at least on my hAP ac²). The file stored while sniffing (/tool sniffer set file-name) is in a .pcap format which can be open using Wireshark.

This is the big issue, the x.104 is a Win 10 Pro PC, but I have also tried from an Ubuntu box and a Raspberry Pi.
Sorry, of course the gateway must be 172.16.105.1, not 172.16.110.1, I got lost in that. Since there is no switch chip on a CCR 1016, I cannot imagine what would have to happen so that the packets would actually arrive but not be sniffed. When you ping the Malford CCR from each of the three local devices you mentioned, does the sniffer at Malford show the packets? Did you sniff when pinging the Tinos device from each of the Malford devices?

Also, do these devices use DHCP or static configuration? Because your /ip dhcp-server network row for 172.16.105.0/24 at Malford contains no gateway.

By the way disabling the NAT rule on the Malford router:
/ip firewall nat add src-address=172.16.110.0/24 out-interface=bridge-lan action=masquerade chain: srcnat
Makes no difference, presume as it is there to control traffic originating from the Malford netwrok.
Correct. The nat rules only handle the initial packet of each connection, and the rest of packets belonging to the same connection inherits the same handling.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Sun Jun 14, 2020 9:09 pm

Also, do these devices use DHCP or static configuration? Because your /ip dhcp-server network row for 172.16.105.0/24 at Malford contains no gateway.
.
I removed that network as its not used, the DHCP is handled by the Raspberry Pi running Pi-Hole, as it has been for several years. This is where it gets very strange, I swapped to a Pi4 from a Pi3 at the same time as we did the firewall work. This afternoon I swapped back to the old Pi3 and I get connectivity again to the Tinos network.
.
The next problem is that the current configuration is putting a huge overhead on bandwidth, Malford upload is 7mbps and Tinos download is only 1mbps.
.
Can ipv6 cause problems? Can we look at the two NAT rules you added and said might cause some issues.
.
Thanks,
.
Andrew.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Sun Jun 14, 2020 10:18 pm

I removed that network as its not used, the DHCP is handled by the Raspberry Pi running Pi-Hole, as it has been for several years. This is where it gets very strange, I swapped to a Pi4 from a Pi3 at the same time as we did the firewall work. This afternoon I swapped back to the old Pi3 and I get connectivity again to the Tinos network.
It's not enough to remove the row from /ip dhcp-server network, you also have to disable the related dhcp server itself on the Mikrotik - if it leases address faster than the RPi (which might be the reason why the result is different between RPi3 and RPi4), the client may get the lease from the Mikrotik. If you remove just the row from /ip dhcp-server network, the Mikrotik will still lease addresses, but with /32 netmask, so the client would stop talking completely.

The next problem is that the current configuration is putting a huge overhead on bandwidth, Malford upload is 7mbps and Tinos download is only 1mbps.
Almost 100 % of Tinos' raw download contributes to Malford's upload, and about 80 % of that is the traffic decapsulated from the tunnel which contributes to Malford's download, but the rest of Malford's upload cannot come from the Tinos VPN.

Can ipv6 cause problems?
There may be some IPv6 traffic on the Malford uplink, but I haven't seen any configuration items which would suggest that IPv6 is actually in use at either CCR, so I cannot imagine how it could contribute to the upload volume.

All in all, sniffing into a file at Malford and using Wireshark to analyse the file is the only way to find out where those 7 Mbit/s come from.

Can we look at the two NAT rules you added and said might cause some issues.
If pinging between PCs on different sites still works after you disable those two added masquerade rules matching on out-interface=bridge-lan, it's better to keep those rules disabled. But only newly established connections will be affected, so don't disable them while pinging - the connection tracking holds ICMP pinholes for 10 seconds after the last packet. So disable the rules, wait 10 seconds to be sure, and then ping.

I don't know what applications you use between the sites. Some applications won't mind the extra NAT, some may have problems - e.g. VoIP phones at one site connected to a PBX on the other site might exhibit weird behaviour if the L2TP tunnel went down for a short time and the corresponding connections in the firewall got dropped (masqueraded connections are removed if the associated interface goes down).
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Mon Jun 15, 2020 11:25 pm

It's not enough to remove the row from /ip dhcp-server network, you also have to disable the related dhcp server itself on the Mikrotik
.
All the components for the Malford pool were disabled, just use them if the Pi-Hole goes down.
Almost 100 % of Tinos' raw download contributes to Malford's upload, and about 80 % of that is the traffic decapsulated from the tunnel which contributes to Malford's download, but the rest of Malford's upload cannot come from the Tinos VPN.
.
I get that Tinos download is limited by Malford upload but the Tinos could manage barely 20%. One to play around with.
.
If pinging between PCs on different sites still works after you disable those two added masquerade rules matching on out-interface=bridge-lan, it's better to keep those rules disabled. But only newly established connections will be affected, so don't disable them while pinging - the connection tracking holds ICMP pinholes for 10 seconds after the last packet. So disable the rules, wait 10 seconds to be sure, and then ping.
.
I don't know what applications you use between the sites. Some applications won't mind the extra NAT, some may have problems - e.g. VoIP phones at one site connected to a PBX on the other site might exhibit weird behaviour if the L2TP tunnel went down for a short time and the corresponding connections in the firewall got dropped (masqueraded connections are removed if the associated interface goes down).
.
I will try that, unfortunately the Tinos site has gone down again from 11 this morning. Is it possible that the traffic has been captured and used to decipher the password for Tinos? From the same time I am also getting MS365 text verification messages. I will need to send somebody there. If you do not hear from me for a few days please accept my apologies but I need to send somebody there to check which isn't very easy.
.
Thanks again.
.
Regards,
.
Andrew.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Mon Jun 15, 2020 11:55 pm

I get that Tinos download is limited by Malford upload but the Tinos could manage barely 20%. One to play around with.
Sorry, I don't understand what you mean. You've complained that the download of Tinos is about 1 Mbit/s while the upload of Malford is about 7 Mbit/s, so my understanding was that you ask where do those other 6 Mbit/s of Malford's upload come from. Was the actual subject of the complaint different?

the Tinos site has gone down again from 11 this morning. Is it possible that the traffic has been captured and used to decipher the password for Tinos?
Theoretically yes, it depends how interesting a target you are. But one thing is the L2TP password, which would be better used to impersonate Tinos when connecting to Malford as an L2TP client, and another thing is to decipher the management password from Winbox traffic carried inside the L2TP tunnel which would be an additional task. But as Tinos doesn't accept direct management connections coming in via WAN, it would require hijacking the L2TP session by impersonating Malford, which would either require to involve the ISP or to fiddle with routing protocols which would affect many ISP customers, to make use of the knowledge of the admin password, so it doesn't sound likely to me.

But do I read you right that the IP of the outer router at Tinos is public as well, i.e. you could use e.g. an SSTP VPN to connect there in emergency cases?

I will need to send somebody there. If you do not hear from me for a few days please accept my apologies but I need to send somebody there to check which isn't very easy.
On this subject, read my PM from Friday :)
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Tue Jun 16, 2020 12:05 am

Sorry, I don't understand what you mean. You've complained that the download of Tinos is about 1 Mbit/s while the upload of Malford is about 7 Mbit/s, so my understanding was that you ask where do those other 6 Mbit/s of Malford's upload come from. Was the actual subject of the complaint different?
.
Well, maybe. Just where did those 6 Mbps dissapear, say at 100% in theory Malford upload should equal Tinos download (Tinos internet speeds outside the VPN being much higher 30up/12dn).
.
so it doesn't sound likely to me.
.
That is reassuring, thanks.
.
But do I read you right that the IP of the outer router at Tinos is public as well, i.e. you could use e.g. an SSTP VPN to connect there in emergency cases?
.
In theory even if the L2TP connection failed it would fail to normal internet and I would still be able to connect via Teamviewer, which is why I need to check if there was another power cut.
.
On this subject, read my PM from Friday :)
.
I can imagine, I should have mentioned that I might be away for a few days. Its quite remote and not easy to get somebody to go outside the normal rota.
.
I appreciate your patience.
.
Andrew.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Wed Jun 24, 2020 5:42 pm

On this subject, read my PM from Friday :)
.
Finally managed to get somebody to switch the power back on. Back on line and L2TP is connected. Pinging both ways work.
.
Tinos Firewall:
.
/ip firewall filter
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment="defconf: accept to local loopback" dst-address=127.0.0.1
add action=accept chain=input comment="allow access to router itself via VPN tunnels" in-interface-list=VPN
add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new in-interface-list=WAN
.
Tinos Mangle/NAT
.
/ip firewall mangle
add action=change-mss chain=forward comment="Rule to change MTU size to 105.x try not to use" disabled=yes dst-address=172.16.105.0/24 new-mss=1442 passthrough=yes protocol=tcp tcp-flags=syn
add action=mark-routing chain=prerouting new-routing-mark=VOIP passthrough=yes src-address=172.16.110.125
/ip firewall nat
add action=accept chain=srcnat dst-address=172.16.105.0/24 src-address=172.16.110.0/24
add action=accept chain=srcnat dst-address=172.16.100.0/24 src-address=172.16.110.0/24
add action=masquerade chain=srcnat src-address=172.16.110.0/24
add action=dst-nat chain=dstnat disabled=yes dst-port=8443 protocol=tcp src-address=82.17.140.24 to-addresses=172.16.110.125 to-ports=443
add action=masquerade chain=srcnat out-interface=bridge-lan src-address=172.16.105.0/24
Cheers.

Andrew.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Wed Jun 24, 2020 7:23 pm

So the whole issue there was that power was simply down, correct? Or did those remote hands of yours have to power-cycle the device?

Other than that, of course I've already lost track of all that, but if I've parsed the history properly, one open point was whether you could remove those band-aid src-nat rules and still have transparency between hosts in both routers' LANs, and another one was regarding the bandwidth occupation. In the meantime I have seen some complaints here on the forum regarding unexpectedly slow L2TP/IPsec performance on CCRs, but I still don't get what is the essence of your issue - is it that Tinos has a much better throughput to/from the internet when the L2TP tunnel is down (which would be the same like that other complaint), or that Malford traffic volume is much higher than expected whilst the contribution of Tinos traffic is relatively small?
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Wed Jun 24, 2020 9:02 pm

So the whole issue there was that power was simply down, correct? Or did those remote hands of yours have to power-cycle the device?
.
As much as I appreciate the caretaker I would not ask him to touch the equipment, everything powers up on its own. Unfortunately we only have about 15min of UPS time to deal with spikes.
.
Other than that, of course I've already lost track of all that, but if I've parsed the history properly, one open point was whether you could remove those band-aid src-nat rules and still have transparency between hosts in both routers' LANs
.
Indeed, that is certainly a priority. Would you like me to repost both Malford and Tinos rules?
.
and another one was regarding the bandwidth occupation. In the meantime I have seen some complaints here on the forum regarding unexpectedly slow L2TP/IPsec performance on CCRs, but I still don't get what is the essence of your issue
.
The limiting download speed at Tinos is a factor of the upload speed at Malford, and vice versa. So for example is my max Malford upload is 10Mbps, then my max download (whilst the VPN is active) in Tinos is 10-x Mbps as all Tinos traffic is received via Malford.
.
The question is, what protocol should I use to have encryption but also the lowest overheard "x".
.
Much obliged,
.
Andrew.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Unable to update CCR

Thu Jun 25, 2020 12:37 am

Unfortunately we only have about 15min of UPS time to deal with spikes.
I was always thinking the grid outages we've got here in the village a few times a year do not happen in one of the capitals of the most developed world...

Other than that, of course I've already lost track of all that, but if I've parsed the history properly, one open point was whether you could remove those band-aid src-nat rules and still have transparency between hosts in both routers' LANs
Indeed, that is certainly a priority. Would you like me to repost both Malford and Tinos rules?
Not really, the only thing necessary to find out is to disable those src-nat rules referring to LAN out-interfaces; if doing that prevents pings between LANs from working again, there is still an issue with routes on the LAN hosts, not on the CCRs themselves. Whether it needs addressing or not depends on the long-term experience with the applications - if those src-nat rules are still necessary but there is no strange behaviour while they are in place, there is no point in further search except of technical purism.

The limiting download speed at Tinos is a factor of the upload speed at Malford, and vice versa. So for example is my max Malford upload is 10Mbps, then my max download (whilst the VPN is active) in Tinos is 10-x Mbps as all Tinos traffic is received via Malford.

The question is, what protocol should I use to have encryption but also the lowest overheard "x".
We're talking about just a few percent of the bandwidth being wasted on the VPN overhead. There is the second IP header (20 bytes) in any case; for IPsec, there is the ESP overhead consisting of 8 bytes of the ESP header (8 bytes) and the ESP trailer (2 bytes); depending on the integrity check algorithm used, additional bytes are lost on the authentication (12 bytes for SHA1). And if there is a NAT at at least one end (in your case, at Tinos), the ESP is encapsulated into UDP so another 8 bytes gone. So in total, each payload packet is augmented with another 50 bytes for tunneling and IPsec encryption, which accounts for about 80 % for a G.729 RTP packet (compressed voice) or for about 3 % for a TCP packet that fully uses the available MTU after getting encapsulated. ACK-only TCP packets are even smaller than the G.729 ones so the percentage is even less favourable. So the actual percentage of wasted bandwidth depends on the character of the traffic.

By using the L2TP over IPsec encapsulation, you add another 18 bytes per packet for the L2TP headers, or 22 if you use the MPPE which seems to have its own integrity check. But that's still just 40 % of those, say, 10 % in average of IPsec & tunnel overhead, i.e. those average 10 % change to average 14% of the total. If you omit the IPsec part and use plain L2TP with only MPPE as you do now, you only have those 22 bytes of L2TP headers plus 20 of the second IP header, so 42 rather than 50 in case of bare IPsec., i.e. just 9 % of total bandwidth in average rather than 10 %. What a gain.

So - a far better encryption, and a far less CPU spent on encryption as it would be executed in hardware, could be the arguments to move from bare L2TP with MPPE to bare IPsec, definitely not bandwidth efficiency. Use of bare IPsec adds another layer of complexity to the understanding how things work as compared to the L2TP one - instead of adding routes via virtual interfaces to regular routing tables, packets with a proper combination of source and destination address get hijacked by the IPsec engine no matter what the regular routing says.

To summarize: if the L2TP/MPPE throughput is a limiting factor (some CPU cores are reaching 100% of load as handling the MPPE encryption), it makes sense to switch to bare IPsec; if the uplink bandwidth is the limiting factor, you save a tiny bit of it by staying where you are. In any case, use of IPsec significantly improves security as compared to use of MPPE, but adding IPsec around L2TP would cost some additional bandwidth so only a migration to bare IPsec makes sense in that case.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Unable to update CCR

Thu Jun 25, 2020 9:36 am

I was always thinking the grid outages we've got here in the village a few times a year do not happen in one of the capitals of the most developed world...
.
One CCR is in the UK, the other is on a island in Greece called Tinos.
.
Not really, the only thing necessary to find out is to disable those src-nat rules referring to LAN out-interfaces; if doing that prevents pings between LANs from working again, there is still an issue with routes on the LAN hosts, not on the CCRs themselves. Whether it needs addressing or not depends on the long-term experience with the applications - if those src-nat rules are still necessary but there is no strange behaviour while they are in place, there is no point in further search except of technical purism.
.
The only "strange" behaviour left is the inability for the Tinos CCR to ping devices in the Malford network, however Tinos devices can ping Malford devices.
.
We're talking about just a few percent of the bandwidth being wasted on the VPN overhead. There is the second IP header (20 bytes) in any case; for IPsec, there is the ESP overhead consisting of 8 bytes of the ESP header (8 bytes) and the ESP trailer (2 bytes); depending on the integrity check algorithm used, additional bytes are lost on the authentication (12 bytes for SHA1). And if there is a NAT at at least one end (in your case, at Tinos), the ESP is encapsulated into UDP so another 8 bytes gone. So in total, each payload packet is augmented with another 50 bytes for tunneling and IPsec encryption, which accounts for about 80 % for a G.729 RTP packet (compressed voice) or for about 3 % for a TCP packet that fully uses the available MTU after getting encapsulated. ACK-only TCP packets are even smaller than the G.729 ones so the percentage is even less favourable. So the actual percentage of wasted bandwidth depends on the character of the traffic.
.
Thank you for that, which answers why we have terrible VOIP performance and had to try several rules to route that traffic within the Tinos network straight out to rather than using the VPN. I spend a bit of time last night testing both BT and Virgin lines and my "perceived overhead" was significantly higher with Virgin than with BT. This morning the situation is similar between both providers. I can only assume this has something to do with the way that Virgin compresses traffic. Forums seem to be littered with people complaining about VPN performance.
.
By using the L2TP over IPsec encapsulation, you add another 18 bytes per packet for the L2TP headers, or 22 if you use the MPPE which seems to have its own integrity check. But that's still just 40 % of those, say, 10 % in average of IPsec & tunnel overhead, i.e. those average 10 % change to average 14% of the total. If you omit the IPsec part and use plain L2TP with only MPPE as you do now, you only have those 22 bytes of L2TP headers plus 20 of the second IP header, so 42 rather than 50 in case of bare IPsec., i.e. just 9 % of total bandwidth in average rather than 10 %. What a gain.
.
I think I would be keen to use IPSec. Both are CCR 1016s so CPU usage is marginal most of the time.
.
To summarize: if the L2TP/MPPE throughput is a limiting factor (some CPU cores are reaching 100% of load as handling the MPPE encryption), it makes sense to switch to bare IPsec; if the uplink bandwidth is the limiting factor, you save a tiny bit of it by staying where you are. In any case, use of IPsec significantly improves security as compared to use of MPPE, but adding IPsec around L2TP would cost some additional bandwidth so only a migration to bare IPsec makes sense in that case.
.
I would prefer to add move to L2TP over IPSec and test bandwidth. Just need to make sure that 1 IP address (the VOIP phone) 172.16.110.125 ignores the VPN and goes straight to Tinos WAN.
.
Thanks,
.
Andrew.
 
tomislav91
Member
Member
Posts: 303
Joined: Fri May 26, 2017 12:47 pm

Re: Unable to update CCR

Fri Oct 30, 2020 11:11 pm

how is your cpu handle with this rules at the top
/ip firewall filter
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
accept rules goes to about 15GB in 10,15minutes and cpu go to 55%...

Who is online

Users browsing this forum: fragment, RobertsN, smirgo and 84 guests