Multi IP on WAN & Masq help needed

spookman · November 26, 2016, 10:43pm

EDIT Design UPDATE please see post #8
Hi Guys hoping someone can assist

I am having a problem in getting multiple IP’s working on my mikrotik RB2, if a disable all of the 19.1.16.154-158/29 and cycle them 1 at time enabled and disabled only ever having 1 enabled at a time till I have done them all, I can then enable all of them and they work for a while until only one is left working . I have setup a new site as follows

I am provided an option of hiring a router from the ISP on a monthly basis “75USD” or instead provide my own “RB1” and this router is connected to the ISP on prem, router which is the media converter from fiber to Ethernet, This is connected with a /30 network and VLAN 290, Then from RB1 I assign a single IP from the /29 block, and then assigning the balance of the IP’s to the RB2.

RB1 is configure as a router with no firewall or nat, RB2 acts as the firewall and where all our configurations are made. I am also having another issue where both RB1 and RB2 can not from the terminal resolve dns, internet traffic from inside the network is fine and even the nating to the server on the first ip .154 works fine.

Lastly is it nessesery for my configuration to have 2 seperate RB’s or can I consolidate this config into 1 unit ?
please the network design below with all my out puts.

RB!

/interface vlan
add interface=ether1 name=vlan-290 vlan-id=290
/ip address
add address=19.1.16.153/29 interface=ether2 network=19.1.16.152
add address=19.2.14.130/30 interface=vlan-290 network=19.2.14.128
/ip dns
set servers=8.8.8.8,8.8.4.4
/ip firewall address-list
add address=19.1.16.154 list=admins
/ip firewall filter
add action=accept chain=input protocol=icmp
add action=drop chain=input connection-state=“” src-address-list=!admins
/ip route
add distance=1 gateway=19.2.14.129
add distance=1 dst-address=19.1.16.152/29 gateway=ether2

RB2

/ip address
add address=10.10.0.1/24 interface=ether1 network=10.10.0.0
add address=19.1.16.154/29 interface=ether10-wan network=19.1.16.152
add address=19.1.16.155/29 interface=ether10-wan network=19.1.16.152
add address=19.1.16.156/29 interface=ether10-wan network=19.1.16.152
add address=19.1.16.157/29 interface=ether10-wan network=19.1.16.152
add address=19.1.16.158/29 interface=ether10-wan network=19.1.16.152
/ip dns
set allow-remote-requests=yes servers=8.8.8.8,8.8.4.4
/ip firewall filter
add action=accept chain=input comment=ICMP protocol=icmp
add action=drop chain=input comment=“DROP ALL” connection-state=“” src-address-list=!Admin
add action=accept chain=forward comment=“Services | SUN” connection-state=“” dst-address=10.10.0.100 dst-port=21,22,25,53,80,110,143,443,8080 protocol=tcp
add action=accept chain=forward comment=“Established & Related” connection-state=“” in-interface=ether1
add action=drop chain=forward comment=Invalid connection-state=invalid
add action=accept chain=forward comment=“Allow Internet” connection-state=established,related
add action=drop chain=forward comment=“DROP ALL” log=yes
/ip firewall nat
add action=dst-nat chain=dstnat comment=“WAN1 Web & Mail Traffic | Sun” dst-address=19.1.16.154 dst-port=21,25,53,80,110,143,443,8080 protocol=tcp to-addresses=10.10.0.100
add action=masquerade chain=srcnat out-interface=ether1
add action=masquerade chain=srcnat out-interface=ether10-wan
/ip route
add distance=1 gateway=19.1.16.153

Sob · November 26, 2016, 11:36pm

What exactly do you mean by “not working”? You can’t ping them from RB1? It should work. Do you see their MAC addresses (there will be same one for all) in IP->ARP?

Also you don’t need to manually add:

/ip route
add distance=1 dst-address=19.1.16.152/29 gateway=ether2

The same route should be created dynamically (but the static one should not break anything).

And no, you don’t need two RBs, one is enough. And as a bonus, you’ll be able to use all 8 public addresses from /29 (you could do it with two RBs too, if you really wanted to).

spookman · November 27, 2016, 6:22pm

Hi Sob,

To clarify, from RB1 I can ping the public IP’s, but from the internet I can only ping the first one, the other 4 are not contactable.
I can see in the arp table all 5 ip’s and their MAc’s which is the same for all as expected, but only 1 of the IP’s is ever accessible from the internet.

Sob · November 27, 2016, 7:18pm

I don’t see anything obviously wrong. Try to ping other addresses from outside and then using Tools->Torch check where it gets lost (check both interfaces on RB1 and ether10-wan on RB2).

spookman · November 28, 2016, 8:47am

So I have removed RB1 from the infrastructure and setup RB2 to act as RB1 i.e. moved the vlan290 ont RB2, Still having the same issue, can access first IP but not the rest.
Do I need to instruct the upstream ISP that I am using the entire /29 range, or is there something I need to setup like RIP or BGP, or seomthing with ARP ?

Sent from my iPhone using Tapatalk

kujo · November 28, 2016, 9:39am

If you create vlan interface, than set address to this interface.

Sent from my iPhone using Tapatalk

spookman · November 28, 2016, 11:04am

Thanks Kujo,

That’s not the issue, traffic flows into and out of the network, the problem is the public ip block 19.1.16.152/29 , only 19.1.16.153 is accessable from the internet, rest don’t respond to ICMP. or anything

spookman · November 28, 2016, 12:58pm

Ok I have changed my setup, to consolidate the 2 Mikrotik’s into 1 Mikrotik please see design below aswell as the config setup,

I am still having an issue where only one of the ip’s from the block 19.1.16.152/29 is accessible on the internet at a time.
2nd issue now is that all connection from internal to the internet only work if I masq on the vlan-290 interface, is it possible to get it working on the public ip block “ether10-wan”

/ip address
add address=10.10.0.29/24 interface=ether1 network=10.10.0.0
add address=19.1.16.154/29 disabled=yes interface=ether10-wan network=19.1.16.152
add address=19.1.16.155/29 disabled=yes interface=ether10-wan network=19.1.16.152
add address=19.1.16.156/29 disabled=yes interface=ether10-wan network=19.1.16.152
add address=19.1.16.157/29 disabled=yes interface=ether10-wan network=19.1.16.152
add address=19.1.16.158/29 disabled=yes interface=ether10-wan network=19.1.16.152
add address=19.1.16.153/29 interface=ether10-wan network=196.15.167.152
add address=19.2.14.130/30 interface=vlan-290 network=196.25.145.128
/ip dns
set allow-remote-requests=yes servers=8.8.8.8,8.8.4.4
/ip firewall filter
add action=accept chain=input comment=ICMP protocol=icmp
add action=accept chain=input comment=ICMP dst-port=53 in-interface=ether1 protocol=udp
add action=accept chain=input comment="established, related" connection-state=established,related,new src-address-list=admin_hosts
add action=drop chain=input comment="DROP ALL" connection-state="" src-address-list=!dns
add action=accept chain=forward comment="Services | SUN" connection-state="" dst-address=10.10.0.251 dst-port=21,22,25,53,80,110,143,443,5050,8080 protocol=tcp
add action=accept chain=forward comment="DNS | SUN" connection-state="" dst-address=10.10.0.251 dst-port=53 protocol=udp
add action=accept chain=forward comment="JumpBox RDP" connection-state="" dst-address=10.10.0.222 dst-port=3389 protocol=tcp
add action=accept chain=forward comment="Established & Related" connection-state=established,related
add action=accept chain=forward comment="Allow Internet" connection-state="" in-interface=ether1
add action=drop chain=forward comment=Invalid connection-state=invalid
add action=drop chain=forward comment="DROP ALL" log=yes
/ip firewall mangle
add action=accept chain=forward disabled=yes dst-address-list=public-ips in-interface=ether10-wan src-address-list=internal
add action=accept chain=forward disabled=yes dst-address-list=internal out-interface=ether10-wan
/ip firewall nat
add action=dst-nat chain=dstnat comment="WAN1 Web & Mail Traffic | Sun" dst-address=19.1.16.153 dst-port=21,22,25,53,80,110,143,443 protocol=tcp to-addresses=10.10.0.251
add action=dst-nat chain=dstnat comment="SSH || Sun" dst-address=19.1.16.153 dst-port=5050 protocol=tcp to-addresses=10.10.0.251 to-ports=22
add action=dst-nat chain=dstnat comment="RDP JumpBox" dst-address=19.1.16.153 dst-port=8080 protocol=tcp to-addresses=10.10.0.222 to-ports=3389
add action=dst-nat chain=dstnat comment="DNS 1" dst-address=19.1.16.153 dst-port=53 protocol=udp to-addresses=10.10.0.251
add action=dst-nat chain=dstnat comment=ESX disabled=yes dst-address=19.1.16.153 dst-port=443,902,903 protocol=tcp src-address-list=admin_dns to-addresses=10.10.0.201
add action=dst-nat chain=dstnat comment=vCenter disabled=yes dst-address=19.1.16.154 dst-port=443 protocol=tcp src-address-list=admin_dns to-addresses=10.10.0.200
add action=dst-nat chain=dstnat comment="DNS 2" disabled=yes dst-address=19.1.16.158 dst-port=53 protocol=udp to-addresses=10.10.0.251
add action=masquerade chain=srcnat out-interface=ether10-wan
add action=masquerade chain=srcnat out-interface=ether1
add action=masquerade chain=srcnat out-interface=vlan-290
/ip route
add distance=1 gateway=196.25.145.129

Any Assistance would be appreciated

kujo · November 28, 2016, 1:01pm

Can you turn on packet sniffer on this vlan, and look at icmp traffic, its arrived to you?

Sent from my iPhone using Tapatalk

spookman · November 28, 2016, 1:52pm

If I torch the connection I am not seeing the traffic. As I said, I can disable first IP the the 2nd IP responds, if can re-enable the 1st ip and both will remain responding, after about 15mins, the 2nd ip stops responding.

kujo · November 28, 2016, 2:10pm

Try add

/ip route rule add dst-address 19.1.16.152/29 table=main

Sent from my iPhone using Tapatalk

Sob · November 28, 2016, 4:37pm

From the way it worked with two routers, the /29 is clearly routed subnet. Packets from internet to any of those addresses must come to your router (via vlan-290 interface), no matter if they are active or not (ISP’s router can’t know that). If you use Torch there and ping /29 addresses from outside, you must see something. Are you testing it by pinging those addresses from outside, right?

Btw, you can’t use action=masquerade with multiple addresses, because it just picks one, you need action=src-nat and specify addresses using to-addresses option.

spookman · November 28, 2016, 5:52pm

Thanks Sob, I am sure I tested ping the /29 addresses while running torch on the clan-290, but you have me questioning my self, so I give it a go again.

With regards to the masquerading I am referring to our bound traffic and so currently if I masquerade the vlan-290 interface out bound internet traffic works but then to the outside world all traffic is coming from 19.1.16.130, I was hoping to masquerade ether10-wan so it would look as if all traffic originated from /29, not really a big deal tbh and can live with that but external accessibility of the /29 ips has really got me stumped

Sent from my iPhone using Tapatalk

spookman · November 28, 2016, 7:02pm

So I have to put my hand up and apologize, and just say if I had included the complete details of my installation we might have fixed this much earlier.

Sob and Kujo, I have to say thanks for your patience and effort in assisting,

So to the real problem… Well I forgot to mention that the link between the ISP Router and my Mikrotik is not a fixed copper ether net cable, but instead we have a wireless link between the 2 which we manage, The link is 2 Ubiquity Rocket M5’s configured as bridge mode, and unfortunately I am far to trusting in my techie’s who forgot to enable the WDS (Transparent Brigde Mode), with this not set the up stream switch was only keeping 1 IP and MAC in it’s table, and not all of then as the Rockets proxy the ARP requests. As soon as I enabled the WDS function all /29 IP’s cam alive.

Really sorry about the error on our side…

So that leave my last and final issue, the masquerading of our public ip’s, how can I configure it that the /29 are whats seen as originating traffic and not the VLAN-290 IP ?

Thanks again.

Sob · November 28, 2016, 7:08pm

Easy, set up srcnat rules like this:

/ip firewall nat
add action=src-nat chain=srcnat comment="server1" out-interface=vlan-290 src-address=10.10.0.251 to-addresses=19.1.16.153
add action=src-nat chain=srcnat comment="server2" out-interface=vlan-290 src-address=10.10.0.xxx to-addresses=19.1.16.154
...
add action=src-nat chain=srcnat comment="everything else" out-interface=vlan-290 to-addresses=19.1.16.158

Btw, if you want two extra addresses, you can change their netmask to /32 and then use also .152 and .159:

/ip address
add address=19.1.16.152/32 interface=<interface>
add address=19.1.16.153/32 interface=<interface>
...
add address=19.1.16.158/32 interface=<interface>
add address=19.1.16.159/23 interface=<interface>

spookman · November 28, 2016, 7:12pm

Excellent Sob, thanks for your help