hi every one
I have a Routerboard 951Ui-2HnD with RouterOS 6.41
currently I am building a new project (website) and I would like a good internet bandwidth for the servers traffic and in my country we don’t have very high speed internet at reasonable price as compared to other countries
so my first thought was to have mikrotik combine 4 internet connection of the same speed to have more upload speed to give to visitors of my site and after searching a bit i have configured my mikroitk router as follows
### configure my first wan (wan1) as pppoe while the modem is in bridge mode
/interface pppoe-client
add add-default-route=yes comment="1st landline" disabled=no interface=wan1 name=pppoe-out1 password=something user=somthing
/ip address
add address=172.16.0.1/16 interface=local network=172.16.0.0
add address=192.168.1.2/24 interface=wan1 network=192.168.1.0
add address=192.168.2.2/24 interface=wan2 network=192.168.2.0
add address=192.168.3.2/24 interface=wan3 network=192.168.3.0
add address=192.168.4.2/24 interface=wan4 network=192.168.4.0
### set dns for caching and my servers of my isp
/ip dns
set allow-remote-requests=yes cache-size=10240KiB servers=101.102.103.104,101.102.103.105
### adding address list to separate servers while preventing other PCs on my network
/ip firewall address-list
add address=172.16.0.1-172.16.0.75 list=internet_allowed
### i used mangle to combine the 4wans even though i did subscribed for only one to test and till the website building is done
/ip firewall mangle
add action=accept chain=prerouting comment="packet marking for general pcc" in-interface=pppoe-out1
add action=accept chain=prerouting disabled=yes in-interface=wan2
add action=accept chain=prerouting disabled=yes in-interface=wan3
add action=accept chain=prerouting disabled=yes in-interface=wan4
add action=mark-connection chain=input comment="mark connection (in wan1, out wan1)" in-interface=pppoe-out1 new-connection-mark=wan1_conn passthrough=yes
add action=mark-connection chain=input disabled=yes in-interface=wan2 new-connection-mark=wan2_conn passthrough=yes
add action=mark-connection chain=input disabled=yes in-interface=wan3 new-connection-mark=wan3_conn passthrough=yes
add action=mark-connection chain=input disabled=yes in-interface=wan4 new-connection-mark=wan4_conn passthrough=yes
add action=mark-routing chain=output comment="mark routing (in wan1, out wan1)" connection-mark=wan1_conn new-routing-mark=wan1_traffic passthrough=no
add action=mark-routing chain=output connection-mark=wan2_conn disabled=yes new-routing-mark=wan2_traffic passthrough=no
add action=mark-routing chain=output connection-mark=wan3_conn disabled=yes new-routing-mark=wan3_traffic passthrough=no
add action=mark-routing chain=output connection-mark=wan4_conn disabled=yes new-routing-mark=wan4_traffic passthrough=no
add action=mark-connection chain=forward comment="mark packets for new conn packet forward wan1, out wan1" connection-state=new in-interface=pppoe-out1 new-connection-mark=wan1_pfw passthrough=no
add action=mark-connection chain=forward connection-state=new disabled=yes in-interface=wan2 new-connection-mark=wan2_pfw passthrough=no
add action=mark-connection chain=forward connection-state=new disabled=yes in-interface=wan3 new-connection-mark=wan3_pfw passthrough=no
add action=mark-connection chain=forward connection-state=new disabled=yes in-interface=wan4 new-connection-mark=wan4_pfw passthrough=no
add action=mark-routing chain=prerouting comment="mark routing for new conn packet forward wan1, out wan1" connection-mark=wan1_pfw in-interface=local new-routing-mark=wan1_traffic passthrough=no
add action=mark-routing chain=prerouting connection-mark=wan2_pfw disabled=yes in-interface=local new-routing-mark=wan2_traffic passthrough=no
add action=mark-routing chain=prerouting connection-mark=wan3_pfw disabled=yes in-interface=local new-routing-mark=wan3_traffic passthrough=no
add action=mark-routing chain=prerouting connection-mark=wan4_pfw disabled=yes in-interface=local new-routing-mark=wan4_traffic passthrough=no
add action=mark-connection chain=prerouting comment="classifier for quad wan links" dst-address-type=!local new-connection-mark=wan1_conn passthrough=yes per-connection-classifier=src-address:4/0 src-address-list=internet_allowed
add action=mark-connection chain=prerouting disabled=yes dst-address-type=!local new-connection-mark=wan2_conn passthrough=yes per-connection-classifier=src-address:4/1 src-address-list=internet_allowed
add action=mark-connection chain=prerouting disabled=yes dst-address-type=!local new-connection-mark=wan3_conn passthrough=yes per-connection-classifier=src-address:4/2 src-address-list=internet_allowed
add action=mark-connection chain=prerouting disabled=yes dst-address-type=!local new-connection-mark=wan4_conn passthrough=yes per-connection-classifier=src-address:4/3 src-address-list=internet_allowed
add action=mark-routing chain=prerouting comment="final step" connection-mark=wan1_conn new-routing-mark=to_wan1 passthrough=yes src-address-list=internet_allowed
add action=mark-routing chain=prerouting connection-mark=wan2_conn disabled=yes new-routing-mark=to_wan2 passthrough=yes src-address-list=internet_allowed
add action=mark-routing chain=prerouting connection-mark=wan3_conn disabled=yes new-routing-mark=to_wan3 passthrough=yes src-address-list=internet_allowed
add action=mark-routing chain=prerouting connection-mark=wan4_conn disabled=yes new-routing-mark=to_wan4 passthrough=yes src-address-list=internet_allowed
### i expressed my 1st public ip as 1.1.1.1
/ip firewall nat
add action=dst-nat chain=dstnat comment="dns requests for bind" disabled=yes dst-address=1.1.1.1 dst-address-type=local dst-port=53 in-interface=pppoe-out1 protocol=tcp to-addresses=172.16.0.5 to-ports=53
add action=dst-nat chain=dstnat comment="dns requests for bind" disabled=yes dst-address=1.1.1.1 dst-address-type=local dst-port=53 in-interface=pppoe-out1 protocol=udp to-addresses=172.16.0.5 to-ports=53
add action=dst-nat chain=dstnat comment="route web server from wan 1" disabled=yes dst-address=1.1.1.1 dst-address-type=local dst-port=80 in-interface=pppoe-out1 protocol=tcp to-addresses=172.16.0.5 to-ports=80
add action=dst-nat chain=dstnat comment="route ssh for web server from wan 1" disabled=yes dst-address=1.1.1.1 dst-address-type=local dst-port=21 in-interface=pppoe-out1 protocol=tcp to-addresses=172.16.0.5 to-ports=21
add action=dst-nat chain=dstnat comment="route ftp for web server from wan 1" disabled=yes dst-address=1.1.1.1 dst-address-type=local dst-port=22 in-interface=pppoe-out1 protocol=tcp to-addresses=172.16.0.5 to-ports=22
add action=masquerade chain=srcnat comment="allow inter communication (lan to lan)" dst-address-list=internet_allowed src-address-list=internet_allowed
add action=masquerade chain=srcnat comment="allow internat access" out-interface=pppoe-out1 src-address-list=internet_allowed
/ip route
add comment="routes for in/out port forwarding packets" distance=1 gateway=pppoe-out1 routing-mark=wan1_traffic
add disabled=yes distance=2 gateway=wan2 routing-mark=wan2_traffic
add disabled=yes distance=3 gateway=wan3 routing-mark=wan3_traffic
add disabled=yes distance=4 gateway=wan4 routing-mark=wan4_traffic
add check-gateway=ping comment="general pcc routes" distance=1 gateway=pppoe-out1 routing-mark=to_wan1
add check-gateway=ping disabled=yes distance=2 gateway=wan2 routing-mark=to_wan2
add check-gateway=ping disabled=yes distance=3 gateway=wan3 routing-mark=to_wan3
add check-gateway=ping disabled=yes distance=4 gateway=wan4 routing-mark=to_wan4
add check-gateway=ping comment="default routes" distance=1 gateway=pppoe-out1
add check-gateway=ping disabled=yes distance=2 gateway=wan2
add check-gateway=ping disabled=yes distance=3 gateway=wan3
add check-gateway=ping disabled=yes distance=4 gateway=wan4
with this configurations i was able to reach my local servers from my network with public IP or local IP and from outside my LAN I was able to reach my servers using public ip
then had to add more servers and be able to connect to them with ports like 80, so I thought about buying a public IP pool from my isb and assign one of them to my local servers through 1:1 natting
my isb gave my a /29 public pool so now lets say i have 1.1.1.2-1.1.1.6 of public ips and also a matrix LAN IP (lets say):1.1.2.2 which I don’t what is this for and also gateway IP (lets say):1.1.1.1
so I have tried to add all of these public ips to my pppoe client interface and then 1:1 nat them to their respective servers local IPs as follows
###edited the ip addresses to include the new public IPs
/ip address
add address=172.16.0.1/16 interface=local network=172.16.0.0
add address=192.168.1.2/24 interface=wan1 network=192.168.1.0
add address=192.168.2.2/24 interface=wan2 network=192.168.2.0
add address=192.168.3.2/24 interface=wan3 network=192.168.3.0
add address=192.168.4.2/24 interface=wan4 network=192.168.4.0
add address=1.1.1.2/29 interface=pppoe-out1 network=1.1.2.2
add address=1.1.1.3/29 interface=pppoe-out1 network=1.1.2.2
add address=1.1.1.4/29 interface=pppoe-out1 network=1.1.2.2
add address=1.1.1.5/29 interface=pppoe-out1 network=1.1.2.2
add address=1.1.1.6/29 interface=pppoe-out1 network=1.1.2.2
### edited the address list
/ip firewall address-list
add address=172.16.0.1-172.16.0.75 list=nated_clients
add address=172.16.0.1-172.16.75.75 list=internet_allowed
add address=1.1.1.2-1.1.1.6 list=internet_allowed
### editing the firewall dst-nat and src-nat
/ip firewall nat
add action=dst-nat chain=dstnat comment="dns requests for bind" disabled=yes dst-address=1.1.1.2 dst-address-type=local dst-port=53 in-interface=pppoe-out1 protocol=tcp to-addresses=172.16.0.5 to-ports=53
add action=dst-nat chain=dstnat comment="dns requests for bind" disabled=yes dst-address=1.1.1.2 dst-address-type=local dst-port=53 in-interface=pppoe-out1 protocol=udp to-addresses=172.16.0.5 to-ports=53
add action=dst-nat chain=dstnat comment="route web server from wan 1" disabled=yes dst-address=1.1.1.2 dst-address-type=local dst-port=80 in-interface=pppoe-out1 protocol=tcp to-addresses=172.16.0.5 to-ports=80
add action=dst-nat chain=dstnat comment="route ssh for web server from wan 1" disabled=yes dst-address=1.1.1.2 dst-address-type=local dst-port=21 in-interface=pppoe-out1 protocol=tcp to-addresses=172.16.0.5 to-ports=21
add action=dst-nat chain=dstnat comment="route ftp for web server from wan 1" disabled=yes dst-address=1.1.1.2 dst-address-type=local dst-port=22 in-interface=pppoe-out1 protocol=tcp to-addresses=172.16.0.5 to-ports=22
add action=dst-nat chain=dstnat comment="route map server from wan 1" dst-address=1.1.1.3 dst-address-type=local dst-port=80 in-interface=pppoe-out1 protocol=tcp to-addresses=172.16.0.6 to-ports=80
add action=dst-nat chain=dstnat comment="route mail server from wan 1" disabled=yes dst-address=1.1.1.4 dst-address-type=local dst-port=80 in-interface=pppoe-out1 protocol=tcp to-addresses=172.16.0.7 to-ports=80
add action=masquerade chain=srcnat comment="allow inter communication (lan to lan)" dst-address-list=internet_allowed src-address-list=internet_allowed
add action=src-nat chain=srcnat comment="route map server traffic via wan1 public ip" src-address=172.16.0.6 to-addresses=1.1.1.3
add action=src-nat chain=srcnat comment="route mail server traffic via wan1 public ip" src-address=172.16.0.7 to-addresses=1.1.1.4
add action=masquerade chain=srcnat comment="nat here (allow internat access)" out-interface=pppoe-out1 src-address-list=nated_clients
now when I am trying to reach any of my servers from outside my LAN using its public IP it is OK, and when I am trying to reach the servers from my LAN using server local IP it is OK but if i try to reach the server from my LAN using its public IP it is not working
I have searched about this situation and learned that I could do pppoe server to distribute the public IPs to the servers ,but this is not applicable to me as I would do the same setup for all other wan connections (wan2,wan3,wan4) so that I will end up with 4 public IPs for each server and then use the DNS bind server to make use of the round robin feature to circulate between all 4 IPs
so could you review my configurations and tell me what did I do wrong, also if there is any other configuration not related to this issue it could me more efficient that would great to mention.
sorry for the long description of the problem ,
thank you for your time …