Hey, i manage to set up IPSec connection between Mikrotik (RB4011iGS+ running RouterOS v7.6) and pfsense box. Both P1 and P2 are being established based on both gateways logs/status. SA are installed as well. The issue is that while i can reach from mikrotik lan any devices from pfsense lan i can’t do the same other way around. The only reachable device from pfsense network is mikrotik itself (via wegui or ssh), but i cant access any device in mikrotik lan. This lead’s me to believe that issue is on mikrotik side in its firewall/nat configuration. So my current configuration for it looks like this:
/ip firewall filter
chain=input action=accept protocol=ipsec-esp
/ip firewall nat
add chain=srcnat action=accept place-before=0 src-address=<mikrotik-network>/24 dst-address=<pfsense_network>/24
/ip firewall raw
add action=notrack chain=prerouting src-address=<mikrotik-network>/24 dst-address=<pfsense_network>/24
add action=notrack chain=prerouting src-address=<pfsense_network>/24 dst-address=<mikrotik-network>/24
I have enabled logs for ipsec and firewall too on mikrotik but i am not seeing anything in them that would indicate any issues.
They only thing that i am seeing in pfsense logs that have any resemblance of issue is this:
Dec 15 10:39:44 charon 79958 09[CFG] vici client 6129 connected
Dec 15 10:39:44 charon 79958 11[CFG] vici client 6129 registered for: list-sa
Dec 15 10:39:44 charon 79958 11[CFG] vici client 6129 requests: list-sas
Dec 15 10:39:44 charon 79958 11[CFG] vici client 6129 disconnected
But expect that i can see in logs that p1 and p2 works, packets are incoming on ipsec to pfsense as packets are being send from pfsense via ipsec.
After fighting with this issue and looking for answers i am completely lost. So any help would be appreciated.
sindy
December 15, 2022, 3:22pm
2
The way you describe it it is indeed an issue with the Mikrotik firewall. So post the complete configuration of the Mikrotik, not just the few bits you assume to be related. Don’t forget to remove the serial number and other personal information (public IP addresses, login names to external services).
Just one point, the rules in the raw table that exclude the Mikrotik LAN to pfSense LAN traffic from connection tracking are redundant to the accept rule for that traffic in the nat table. But it is not harmful and definitely not the cause of your issue.
Mikrotik lan: 10.0.1.0/24
Small note, mangle entries for ipsec + !ipsec from fasttract entries are new - i was testing if they might help - they didn’t
# dec/15/2022 18:17:54 by RouterOS 7.6
# software id = V8U8-SEVN
#
# model = RB4011iGS+
# serial number = xxxxxx
/interface bridge
add admin-mac=redacted_mac_address auto-mac=no comment=defconf name=bridge
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
/interface lte apn
set [ find default=yes ] ip-type=ipv4 use-network-apn=no
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip ipsec peer
add address=<pfsense_wan_ip>/32 exchange-mode=ike2 name=redacted
/ip ipsec profile
set [ find default=yes ] dh-group=modp2048 enc-algorithm=aes-128 \
hash-algorithm=sha256 lifetime=8h
/ip ipsec proposal
set [ find default=yes ] auth-algorithms=sha256 enc-algorithms=aes-128-cbc \
lifetime=1h pfs-group=modp2048
/ip pool
add name=dhcp ranges=10.0.1.50-10.0.1.99
/ip dhcp-server
add address-pool=dhcp interface=bridge lease-time=12h name=defconf
/port
set 0 name=serial0
set 1 name=serial1
/interface bridge port
add bridge=bridge comment=defconf ingress-filtering=no interface=ether2
add bridge=bridge comment=defconf ingress-filtering=no interface=ether3
add bridge=bridge comment=defconf ingress-filtering=no interface=ether4
add bridge=bridge comment=defconf ingress-filtering=no interface=ether5
add bridge=bridge comment=defconf ingress-filtering=no interface=ether6
add bridge=bridge comment=defconf ingress-filtering=no interface=ether7
add bridge=bridge comment=defconf ingress-filtering=no interface=ether8
add bridge=bridge comment=defconf ingress-filtering=no interface=ether9
add bridge=bridge comment=defconf ingress-filtering=no interface=ether10
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus1
/ip neighbor discovery-settings
set discover-interface-list=LAN
/ip settings
set max-neighbor-entries=8192
/ipv6 settings
set disable-ipv6=yes max-neighbor-entries=8192
/interface list member
add comment=defconf interface=bridge list=LAN
add comment=defconf interface=ether1 list=WAN
/interface ovpn-server server
set auth=sha1,md5
/ip address
add address=10.0.1.1/24 comment=defconf interface=bridge network=10.0.1.0
/ip dhcp-client
add comment=defconf interface=ether1
/ip dhcp-server network
add address=10.0.1.0/24 comment=defconf dns-server=10.0.0.110 domain=\
redacted gateway=10.0.1.1 netmask=24
/ip dns
set allow-remote-requests=yes
/ip dns static
add address=10.0.1.1 comment=defconf name=router.lan
/ip firewall filter
add action=accept chain=input protocol=ipsec-esp
add action=accept chain=input comment=\
"defconf: accept established,related,untracked" connection-state=\
established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment=\
"defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=drop chain=input comment="defconf: drop all not coming from LAN" \
in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" \
ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" \
ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" \
connection-mark="" connection-state=established,related disabled=yes \
hw-offload=yes
add action=fasttrack-connection chain=forward connection-mark=!ipsec \
connection-state=established,related hw-offload=yes
add action=accept chain=forward comment=\
"defconf: accept established,related, untracked" connection-state=\
established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" \
connection-state=invalid
add action=drop chain=forward comment=\
"defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat \
connection-state=new in-interface-list=WAN
add action=accept chain=input comment="defconf: accept established,related" \
connection-state=established,related disabled=yes
/ip firewall mangle
add action=mark-connection chain=forward comment="Mark IPsec" ipsec-policy=\
out,ipsec new-connection-mark=ipsec
add action=mark-connection chain=forward comment="Mark IPsec" ipsec-policy=\
in,ipsec new-connection-mark=ipsec
/ip firewall nat
add action=accept chain=srcnat dst-address=10.0.0.0/24 src-address=\
10.0.1.0/24
add action=masquerade chain=srcnat comment="defconf: masquerade" \
ipsec-policy=out,none out-interface-list=WAN
/ip firewall raw
add action=notrack chain=prerouting dst-address=10.0.0.0/24 src-address=\
10.0.1.0/24
add action=notrack chain=prerouting dst-address=10.0.1.0/24 src-address=\
10.0.0.0/24
/ip ipsec identity
add peer=redacted
/ip ipsec policy
add dst-address=10.0.0.0/24 peer=redacted src-address=10.0.1.0/24 tunnel=yes
/ip route
add disabled=no dst-address=10.0.0.0/24 gateway=bridge pref-src=10.0.1.1 \
routing-table=main suppress-hw-offload=no
/ip service
set ftp port=2121
set www port=8080
set ssh port=2222
set www-ssl port=8443
/system clock
set time-zone-name=Europe/Warsaw
/system logging
add prefix=ipsec topics=ipsec
add prefix=firewall topics=firewall
/system resource irq rps
set sfp-sfpplus1 disabled=no
/tool bandwidth-server
set enabled=no
/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=LAN
/tool netwatch
add comment="dns failover" disabled=no down-script=\
"/ip dhcp-server network set 0 dns-server=1.1.1.1" host=10.0.0.110 \
interval=1m test-script="" type=simple up-script=\
"/ip dhcp-server network set 0 dns-server=10.0.0.110"
sindy
December 15, 2022, 5:51pm
4
I cannot spot anything wrong in the configuration.
So the next step is sniffing. First, set hw=no on all the /interface bridge port rows on the 4011. Then, open a command line window on the 4011, make it as wide as your screen allows, and while trying to access 10.0.1.x from 10.0.0.y (none of them being the routers), run /tool sniffer quick ip-address=10.0.0.y in that command line window; you should see the requests coming via ether1 (because extracted IPsec payload is shown as coming in via the same interface like the transport packets carrying it) and leave first through the bridge atd then via the physical ether port through which the destination is connected; the responses should be visible on the physical port and the bridge, but not on ether1 as sniffing does not have access to packets that later get encapsulated into IPsec transport ones.
I will do it on weekend when i have a little bit of time.