Mail server behind mikrotik with two WAN links

joda · March 29, 2011, 7:55am

Hello everybody.
First of all, I want to express my gratitude for all you folks that work on and for mikrotik. This forum and user manuals have helped me allot in the past few weeks. When it comes to routing, there is nothing that comes close, when it comes to hardware based, budget friendly solution.

INFRASTRUCTURE:
I've bought RB750 for my company, with two Internet links. ADSL and Cable internet. Mail server is in our company and we use VPN. The goal was to setup IP mangle, and WAN failover (in the future). Certain IP range from local IP addresses must use always one link, and the other IP range must use the other WAN link.
To safeguard the company from spammers within the company i've used scripts from DesertAdmin.

THE PROBLEM:
When testing the configuration, problem is that the mail server cannot send e-mails. Connection state is U (unreplied). Browsing the web works (Mozilla or IE), Ping, nslookup, tracert too. Here is the link to the screenshot:

My best guess is that the router cannot establish the connection because the reply port isn't the same as a source port.

SETUP:
port 1: cable net
port 2: ADSL (through this port Mikrotik dials PPPoE)
port 3: LAN
Now.. This is done after I've restored original router into place. That's why there is no PPPoe and gateway isn't mikrotik at the moment (192.168.222.222 instead of 192.168.222.1)

Here is the "/ip address print detail", "/ip route print detail", "/ip firewall export".

[admin@MikroTik.netcomp.internal] > ip address print detail
Flags: X - disabled, I - invalid, D - dynamic
0 address=192.168.2.2/24 network=192.168.2.0 broadcast=192.168.2.255
interface=2.eunet actual-interface=2.eunet

1 address=192.168.222.222/24 network=192.168.222.0
broadcast=192.168.222.255 interface=eth3 actual-interface=eth3

2 D address=188.2.108.236/22 network=188.2.108.0 broadcast=188.2.111.255
interface=1.sbb actual-interface=1.sbb

[admin@MikroTik.netcomp.internal] > ip route print detail
Flags: X - disabled, A - active, D - dynamic,
C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme,
B - blackhole, U - unreachable, P - prohibit
0 S ;;; ADSL
dst-address=0.0.0.0/0 gateway=ADSL gateway-status=ADSL unreachable
distance=1 scope=30 target-scope=10 routing-mark=ADSL

1 A S ;;; SBB
dst-address=0.0.0.0/0 gateway=1.sbb gateway-status=1.sbb reachable
distance=1 scope=30 target-scope=10 routing-mark=SBB

2 ADS dst-address=0.0.0.0/0 gateway=188.2.108.1
gateway-status=188.2.108.1 reachable 1.sbb distance=1 scope=30
target-scope=10

3 ADC dst-address=188.2.108.0/22 pref-src=188.2.108.236 gateway=1.sbb
gateway-status=1.sbb reachable distance=0 scope=10

4 ADC dst-address=192.168.2.0/24 pref-src=192.168.2.2 gateway=2.eunet
gateway-status=2.eunet unreachable distance=0 scope=200

5 ADC dst-address=192.168.222.0/24 pref-src=192.168.222.222 gateway=eth3
gateway-status=eth3 reachable distance=0 scope=10

[admin@MikroTik.netcomp.internal] > ip firewall export

jan/02/1970 02:25:29 by RouterOS 4.9

software id = G8XX-H7XX

/ip firewall connection tracking
set enabled=yes generic-timeout=10m icmp-timeout=10s tcp-close-timeout=10s tcp-close-wait-timeout=10s tcp-established-timeout=1d tcp-fin-wait-timeout=10s tcp-last-ack-timeout=10s tcp-syn-received-timeout=5s tcp-syn-sent-timeout=5s tcp-syncookie=no tcp-time-wait-timeout=10s udp-stream-timeout=3m udp-timeout=10s

/ip firewall filter
add action=accept chain=input comment="SYN accept from MAIL server" disabled=no protocol=tcp src-address=192.168.222.xx tcp-flags=syn
add action=accept chain=input comment="SMTP Mail server OK " disabled=no dst-port=25 protocol=tcp src-address=192.168.222.xxx
add action=accept chain=input comment="DNS Mail Server OK" disabled=no protocol=tcp src-address=192.168.222.xxx src-port=53
add action=accept chain=input comment="Ping - OK" disabled=no protocol=icmp
add action=accept chain=input comment="Established - OK" connection-state=established disabled=no
add action=accept chain=input comment="Related - OK" connection-state=related disabled=no
add action=accept chain=input comment=winbox disabled=no dst-port=8291 protocol=tcp
add action=drop chain=virus comment="Drop Spammer" disabled=no dst-port=25 protocol=tcp src-address=!192.168.222.xxx src-address-list=spammer
add action=add-src-to-address-list address-list=spammer address-list-timeout=5d chain=virus comment="add to spammer list" connection-limit=30,32 disabled=no dst-port=25 limit=50,5 protocol=tcp src-address=!192.168.222.xxx
add action=add-src-to-address-list address-list="port scanners" address-list-timeout=2w chain=input comment="Port scanners to list " disabled=no protocol=tcp psd=21,3s,3,1
add action=add-src-to-address-list address-list="port scanners" address-list-timeout=2w chain=input comment="NMAP FIN Stealth scan" disabled=no protocol=tcp tcp-flags=fin,!syn,!rst,!psh,!ack,!urg
add action=drop chain=input comment="dropping port scanners" disabled=no src-address-list="port scanners"
add action=drop chain=input comment="default configuration" disabled=no in-interface=1.sbb
add action=drop chain=input comment="default configuration" disabled=no in-interface=ADSL

/ip firewall mangle
add action=mark-routing chain=prerouting comment="ADSL gateway" disabled=no new-routing-mark=ADSL passthrough=no src-address=192.168.222.1-192.168.222.29
add action=mark-routing chain=prerouting comment="SBB gateway" disabled=no new-routing-mark=SBB passthrough=no src-address=192.168.222.40-192.168.222.254

/ip firewall nat
add action=masquerade chain=srcnat comment="local IP address NAT" disabled=no src-address=192.168.222.0/24
add action=dst-nat chain=dstnat comment="Port FW to Mail server" disabled=no dst-port=25 protocol=tcp to-addresses=192.168.222.xxx to-ports=25
add action=dst-nat chain=dstnat comment="" disabled=no dst-port=995 protocol=tcp src-port=995 to-addresses=192.168.222.xxx
add action=dst-nat chain=dstnat comment="" disabled=no dst-port=443 protocol=tcp src-port=443 to-addresses=192.168.222.xxx to-ports=443
add action=dst-nat chain=dstnat comment=VPN disabled=no protocol=gre to-addresses=192.168.222.xxx
add action=dst-nat chain=dstnat comment="" disabled=no dst-port=1723 protocol=tcp src-port=1723 to-addresses=192.168.222.xxx
add action=src-nat chain=srcnat comment="" disabled=no dst-port=25 protocol=tcp src-address=192.168.222.xxx to-ports=25

/ip firewall service-port
set ftp disabled=no ports=21
set tftp disabled=no ports=69
set irc disabled=no ports=6667
set h323 disabled=no
set sip disabled=no ports=5060,5061
set pptp disabled=no

ditonet · March 29, 2011, 12:24pm

Post ‘/ip route’ export

SurferTim · March 29, 2011, 12:41pm

I think this is the problem:

/ip firewall nat
(snip)
add action=dst-nat chain=dstnat comment=“Port FW to Mail server” disabled=no dst-port=25 protocol=tcp to-addresses=192.168.222.xxx to-ports=25
(snip)

When your email server tries to send email to another email server (relay), the dst-port is 25. So it dstnats the address back to 192.168.222.xxx. Try adding a in-interface parameter to that rule, like “in-interface=ether1”. Add a second rule with a different in-interface. You can also specify a dst-address parameter rather than in-interface. Your call.

joda · March 29, 2011, 12:55pm

I did.. Note: Since MT isn’t in production, ADSL link isn’t active (PPPoE) through port number 2.

joda · March 29, 2011, 1:24pm

Thanks for the quick reply!

I’m a little puzzled by this.. I’ve added those parameters, and it looks like this. Should I remove something?
add action=dst-nat chain=dstnat comment=“Port FW to Mail server ADSL” disabled=no dst-port=25 in-interface=ADSL protocol=tcp to-addresses=192.168.222.xxx to-ports=25
add action=dst-nat chain=dstnat comment=“Port FW to Mail server SBB” disabled=no dst-port=25 in-interface=1.sbb protocol=tcp to-addresses=192.168.222.xxx to-ports=25
The second thing that You wrote.. About dst-address parameter.
The destination address could be any mail server, how could I define it?

Thank You once again!

ditonet · March 30, 2011, 7:48pm

Post ‘/ip route’ export

Old eyes, my old eyes

Remove these NAT rules:

add action=masquerade chain=srcnat comment="local IP address NAT" disabled=no src-address=192.168.222.0/24
add action=src-nat chain=srcnat comment="" disabled=no dst-port=25 protocol=tcp src-address=192.168.222.xxx to-ports=25

and add:

add action=masquerade chain=srcnat comment="default configuration - masquerade" disabled=no out-interface=ADSL
add action=masquerade chain=srcnat comment="default configuration - masquerade" disabled=no out-interface=1.sbb

these DST-NAT rules looks OK

add action=dst-nat chain=dstnat comment="Port FW to Mail server ADSL" disabled=no dst-port=25 in-interface=ADSL protocol=tcp to-addresses=192.168.222.xxx to-ports=25
add action=dst-nat chain=dstnat comment="Port FW to Mail server SBB" disabled=no dst-port=25 in-interface=1.sbb protocol=tcp to-addresses=192.168.222.xxx to-ports=25

You should also add routing marks to incoming connections on ADSL and SBB interfaces respectively.

HTH,

joda · March 31, 2011, 12:15pm

Thank you..

I will try this as soon as possible. I’ll post back the results..

P.S. Go gentle on those eyes, you’ll need them for quite some time.
Cheers!

SurferTim · March 31, 2011, 1:51pm

@ditonet: Thanks! I did not see this response. I knew you would see it. It took me a minute.

The second thing that You wrote.. About dst-address parameter.

I used “in-interface” so it would cover most setups. If you wanted to limit the port forwarding to one public ip (I have a couple public ips on one WAN interface on my core routers for different clients), you can use “dst-address” rather than “in-interface”. That would depend on your setup.

joda · April 5, 2011, 2:49pm

Excellent, it works!
the trick is.. now the PPTP VPN isn't working that's on my Win Server..
I've noticed that I made a mistake in configuring VPN, so here's a fresh list of rules, new rules are bolded and there are old ones that are disabled. GRE and PPTP were NAT-ed to my VPN server, and now I've tried passthrough.
Any thoughts?
Thanx

/ip address print detail
Flags: X - disabled, I - invalid, D - dynamic
0 address=192.168.2.2/24 network=192.168.2.0 broadcast=192.168.2.255 interface=2.eunet
actual-interface=2.eunet
1 address=192.168.222.1/24 network=192.168.222.0 broadcast=192.168.222.255 interface=eth3
actual-interface=eth3
2 D address=188.2.xxx.xxx/22 network=188.2.xxx.xxx broadcast=188.2.xxx.255 interface=1.sbb
actual-interface=1.sbb
3 D address=213.198.xxx.xxx/32 network=194.247.xxx.xxx broadcast=0.0.0.0 interface=ADSL actual-interface=ADSL

[admin@MikroTik.netcomp.internal] > /ip route print detail
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme,
B - blackhole, U - unreachable, P - prohibit
0 A S ;;; ADSL
dst-address=0.0.0.0/0 gateway=ADSL gateway-status=ADSL reachable distance=1 scope=30 target-scope=10
routing-mark=ADSL
1 A S ;;; SBB
dst-address=0.0.0.0/0 gateway=1.sbb gateway-status=1.sbb reachable distance=1 scope=30
target-scope=10 routing-mark=SBB
2 ADS dst-address=0.0.0.0/0 gateway=194.247.xxx.xxx gateway-status=194.247.xxx.xxx reachable ADSL distance=1
scope=30 target-scope=10
3 DS dst-address=0.0.0.0/0 gateway=188.2.xxx.xxx gateway-status=188.2.xxx.xxx reachable 1.sbb distance=1
scope=30 target-scope=10
4 ADC dst-address=188.2.108.0/22 pref-src=188.2.xxx.xxx gateway=1.sbb gateway-status=1.sbb reachable
distance=0 scope=10
5 ADC dst-address=192.168.2.0/24 pref-src=192.168.2.2 gateway=2.eunet gateway-status=2.eunet reachable
distance=0 scope=10
6 ADC dst-address=192.168.222.0/24 pref-src=192.168.222.1 gateway=eth3 gateway-status=eth3 reachable
distance=0 scope=10
7 ADC dst-address=194.247.xxx.xxx/32 pref-src=213.198.xxx.xxx gateway=ADSL gateway-status=ADSL reachable
distance=0 scope=10

[admin@MikroTik.netcomp.internal] > /ip firewall export

apr/05/2011 16:40:58 by RouterOS 4.9

software id = G8J1-XXXX

/ip firewall connection tracking
set enabled=yes generic-timeout=10m icmp-timeout=10s tcp-close-timeout=10s tcp-close-wait-timeout=10s tcp-established-timeout=1d tcp-fin-wait-timeout=10s tcp-last-ack-timeout=10s tcp-syn-received-timeout=5s tcp-syn-sent-timeout=5s tcp-syncookie=no tcp-time-wait-timeout=10s udp-stream-timeout=3m udp-timeout=10s
/ip firewall filter
add action=passthrough chain=VPN comment="PPTP VPN Passthrough" disabled=no dst-address=192.168.222.xxx dst-port=1723 protocol=tcp
add action=passthrough chain=VPN comment="GRE passthrough for VPN" disabled=no dst-address=192.168.222.xxx protocol=gre
add action=accept chain=input comment="SYN accept from MAIL servera" disabled=no protocol=tcp src-address= 192.168.222.xxx tcp-flags=syn
add action=accept chain=input comment="SMTP Mail server OK” disabled=no dst-port=25 protocol=tcp src-address=192.168.222.xxx
add action=accept chain=input comment="DNS Mail Server OK" disabled=no protocol=tcp src-address=192.168.222.xxx src-port=53
add action=accept chain=input comment="Ping - OK" disabled=no protocol=icmp
add action=accept chain=input comment="Established - OK" connection-state=established disabled=no
add action=accept chain=input comment="Related - OK" connection-state=related disabled=no
add action=accept chain=input comment=winbox disabled=no dst-port=8291 protocol=tcp
add action=drop chain=virus comment="Drop Spammer" disabled=no dst-port=25 protocol=tcp src-address= !192.168.222.xxx src-address-list=spammer
add action=add-src-to-address-list address-list=spammer address-list-timeout=5d chain=virus comment= "add to spammer list" connection-limit=30,32 disabled=no dst-port=25 limit=50,5 protocol=tcp src-address= !192.168.222. xxx
add action=add-src-to-address-list address-list="port scanners" address-list-timeout=2w chain=input comment= "Port scanners to list " disabled=no protocol=tcp psd=21,3s,3,1
add action=add-src-to-address-list address-list="port scanners" address-list-timeout=2w chain=input comment= "NMAP FIN Stealth scan" disabled=no protocol=tcp tcp-flags=fin,!syn,!rst,!psh,!ack,!urg
add action=drop chain=input comment="dropping port scanners" disabled=no src-address-list="port scanners"
add action=drop chain=input comment="default configuration" disabled=no in-interface=1.sbb
add action=drop chain=input comment="default configuration" disabled=no in-interface=ADSL
/ip firewall mangle
add action=mark-routing chain=prerouting comment="ADSL" disabled=no new-routing-mark=ADSL passthrough= no src-address=192.168.222.1-192.168.222.29
add action=mark-routing chain=prerouting comment="SBB" disabled=no new-routing-mark=SBB passthrough=no src-address=192.168.222.40-192.168.222.254
/ip firewall nat
add action=dst-nat chain=dstnat comment="Port FW to Mail server ADSL" disabled=no dst-port=25 in-interface= ADSL protocol=tcp to-addresses=192.168.222.xxx to-ports=25
add action=dst-nat chain=dstnat comment="Port FW to Mail server SBB" disabled=no dst-port=25 in-interface= 1.sbb protocol=tcp to-addresses=192.168.222.xxx to-ports=25
add action=dst-nat chain=dstnat comment="NAT POP3 SSL" disabled=no dst-port=995 protocol=tcp src-port=995 to-addresses=192.168.222.xxx
add action=dst-nat chain=dstnat comment="NAT HTTPS" disabled=no dst-port=443 protocol=tcp src-port=443 to-addresses=192.168.222.xxx to-ports=443
add action=dst-nat chain=dstnat comment=VPN disabled=yes protocol=gre to-addresses=192.168.222.xxx
add action=dst-nat chain=dstnat comment="VPN PPTP" disabled=yes dst-port=1723 protocol=tcp to-addresses= 192.168.222.xxx
add action=masquerade chain=srcnat comment="NAT ADSL" disabled=no out-interface=ADSL
add action=masquerade chain=srcnat comment="NAT SBB" disabled=no out-interface=1.sbb
/ip firewall service-port
set ftp disabled=no ports=21
set tftp disabled=no ports=69
set irc disabled=no ports=6667
set h323 disabled=no
set sip disabled=no ports=5060,5061
set pptp disabled=no ports=1723

joda · April 7, 2011, 6:53am

Just to inform You that everything is working like a charm..
For you’re patience with me, and for future newbies like myself, here’s a working config with 2 WAN links, where one group of IP addresses use one link and another group uses second link, with failover. VPN and mail servers are withing LAN.

/ip firewall mangle

add action=mark-routing chain=prerouting comment="ADSL gw" disabled=no new-routing-mark=ADSL passthrough=no src-address=192.168.222.1-192.168.222.30
add action=mark-routing chain=prerouting comment="SBB gw" disabled=no new-routing-mark=SBB passthrough=no src-address=192.168.222.40-192.168.222.254

/ip route print detail

ADSL
        dst-address=0.0.0.0/0 gateway=ADSL gateway-status=ADSL reachable check-gateway=ping distance=1 
        scope=30 target-scope=10 routing-mark=ADSL 
SBB
        dst-address=0.0.0.0/0 gateway=1.sbb gateway-status=1.sbb reachable check-gateway=ping distance=1 
        scope=30 target-scope=10 routing-mark=SBB
ADSL through SBB
        dst-address=0.0.0.0/0 gateway=1.sbb gateway-status=1.sbb reachable distance=2 scope=30 
        target-scope=10 routing-mark=ADSL
SBB through ADSL
        dst-address=0.0.0.0/0 gateway=ADSL gateway-status=ADSL reachable distance=2 scope=30 target-scope=10 
        routing-mark=SBB

/ip firewall filter

add action=accept chain=input comment="Ping - OK" disabled=no protocol=icmp
add action=accept chain=input comment="Established - OK" connection-state=established disabled=no
add action=accept chain=input comment="Related - OK" connection-state=related disabled=no

This is the minimum configuration that works. I’ve added a bit of protection:

add action=drop chain=virus comment="Drop Spammer" disabled=no dst-port=25 protocol=tcp src-address=!192.168.222.5 src-address-list=spammer
add action=add-src-to-address-list address-list=spammer address-list-timeout=5d chain=virus comment="add to spammer list" connection-limit=30,32 disabled=no dst-port=25 limit=50,5 protocol=tcp src-address=!192.168.222.5
add action=add-src-to-address-list address-list="port scanners" address-list-timeout=2w chain=input comment="Port scanners to list " disabled=no protocol=tcp psd=21,3s,3,1
add action=add-src-to-address-list address-list="port scanners" address-list-timeout=2w chain=input comment="NMAP FIN Stealth scan" disabled=no protocol=tcp tcp-flags=fin,!syn,!rst,!psh,!ack,!urg
add action=drop chain=input comment="dropping port scanners" disabled=no src-address-list="port scanners"
add action=drop chain=input comment="default configuration" disabled=no in-interface=1.sbb
add action=drop chain=input comment="default configuration" disabled=no in-interface=ADSL

/ip firewall nat

add action=masquerade chain=srcnat comment="NAT ADSL" disabled=no out-interface=ADSL
add action=masquerade chain=srcnat comment="NAT SBB" disabled=no out-interface=1.sbb
add action=dst-nat chain=dstnat comment="Port FW to Mail server through ADSL" disabled=no dst-port=25 in-interface=ADSL protocol=tcp to-addresses=192.168.222.xxx to-ports=25
add action=dst-nat chain=dstnat comment="Port FW to Mail server through SBB" disabled=no dst-port=25 in-interface=1.sbb protocol=tcp to-addresses=192.168.222.xxx to-ports=25
add action=dst-nat chain=dstnat comment="NAT POP3 SSL" disabled=no dst-port=995 in-interface=ADSL protocol=tcp to-addresses=192.168.222.xxx to-ports=995
add action=dst-nat chain=dstnat comment="NAT HTTPS" disabled=no dst-port=443 in-interface=ADSL protocol=tcp to-addresses=192.168.222.xxx
add action=dst-nat chain=dstnat comment="VPN GRE" disabled=no in-interface=ADSL protocol=gre to-addresses=192.168.222.xxx
add action=dst-nat chain=dstnat comment="VPN PPTP" disabled=no dst-port=1723 in-interface=ADSL protocol=tcp to-addresses=192.168.222.xxx to-ports=1723

Special thanks to SurferTim and ditonet!
P.S. Please tell me how to add to your karma? It is the least I can do.

Best regards from Serbia