Question on routing between interfaces

RouterOS General
1

Good afternoon! Wasn’t sure which sub-forum to post this in, so hoping it is ok to use general. I’m mostly a firewall and server guy, but got pulled in to helping a client that essentially wants to provide ISP services. They purchased a CCR1009 and asked my company to design and deliver a solution using this device (we actually have 2 of them, though at the moment I’m not sure why we would need 2 routers).

We are being provided a /30 IP from the upstream provider, with a /29 block routed through it.

For the sake of having something tangible to work with, let’s call the /30 block 10.1.1.0 /30, where 10.1.1.1 is our IP, and 10.1.1.2 is the gateway (ISP router).

Let’s say the /29 block being routed to us is 192.168.1.0 /29.

The upstream provider can route as many additional IP blocks to us as we need as we scale this out to more customers. For sake of argument, let us say that we will continue to receive additional /29 blocks routed through our /30.

Ultimately, I think that we’ll need to create virtual interfaces on the router, use one IP out of the first and each additional /29 block and assign that to the virtual interface, and the customer will use that as their gateway, and have use of the remainder of the IPs out of the /29 block.

For now, I simply want to test the configuration using a physical port on the Mikrotik.

So, I have configured one port (Eth1) to have 10.1.1.1 as its IP. I have another port (Eth2) set to use 192.168.1.1.

I have a default route for destination address 0.0.0.0/0 pointing to 10.1.1.2 as its gateway.

I connect Eth1 directly to the device that has 10.1.1.2 as its IP.

I assign a laptop an IP in the 192.168.1.0 /29 range (in this instance, I used 192.168.1.2, with 192.168.1.1 being its gateway) and connect this to Eth2.

If I enable NAT within the MikroTik, I can pass traffic out to the Internet successfully. But, I don’t think I should or could be using NAT in order to provide proper public IP routing out to the clients (assuming that some will want VPN access through their firewall, or will have internal emails servers, web servers, etc).

If I disable the NAT rule, from my laptop I can ping 192.168.1.1 as well as 10.1.1.1, but I cannot ping along to 10.1.1.2 or get out to the Internet.

I feel like I should just somehow be able to create a static route that routes all traffic coming from 192.168.1.1/29 to 10.1.1.1 (or perhaps just to Eth1), and then 10.1.1.1 passes its traffic along to 10.1.1.2 and voila!

I’ve been reading documentation and forum posts for days, and have resorted to just “trying stuff” in the router interface, but no luck so far. Again, no problems passing traffic through from the laptop if I enable NAT, but that isn’t going to work in our scenario. I don’t foresee the client scaling up to 500+ customers, but they will probably very quickly be in a position to bring on 10 to 20 customers, so I do need this to scale out a little, and know that we need to provide reachable public IPs out to the clients.

Can anyone lend some assistance? If anyone thinks that our design is totally off base, don’t hesitate to speak up, I’m open to ideas!

Thank you!

W43

2

You basically have two approaches:

1) Direct assignment to customers (no nat)

In this regime, for maximum utilization of your /29 IP space, you should use /32 per customer as follows:

suppose customer 1 is on ether1, customer2 on ether2, etc… with combo1 being the WAN interface with the /30 on it.

You would add IP addresses as follows:
/ip address add address=192.168.0.1/32 network=192.168.0.2 interface=ether1
/ip address add address=192.168.0.1/32 network=192.168.0.3 interface=ether2
… etc

customers would configure themselves with a normal /29 netmask, and 192.168.0.1 as the gateway.

2) nat-at-the-border

The advantage for the ISP in this regime is that you can use all 8 addresses of each /29 you receive (assuming that they do not overlap the natural classful network or broadcast address…
e.g. 172.16.0.0/29 would not allow you to use 172.16.0.0/32… but 172.16.1.0/29 would allow you to use 172.16.1.0 because 172.16.x.x in classful networking is a /16, thus the natural network address is 172.16.0.0

applying NAT, you could use e.g. 192.168.1.32/29 as 8 useful public IP addresses. Just black-hole route the real subnet so that the Mikrotik won’t ping-pong unused addresses…
/ip route add dst=192.168.1.32/29 type=blackhole

Then use stateless 1:1 nat rules (netmap) at the border, and use 100.64.x.x/10 (cgnat) IP space inside your network. So for instance some customer’s public IP is 192.168.1.32 and the WAN of their router is 100.64.1.32 - your router will perform 1:1 stateless NAT which is lightweight (fast) AND doesn’t break as many things as stateful NAT can…
/ip firewall nat
add chain=dstnat dst-address=192.168.1.32 action=netmap to-address=100.64.1.32
add chain=srcnat src-address=100.64.1.32 action=netmap to-address=192.168.1.32

Somewhere you would want to have a default “src-nat” (stateful) rule for outbound traffic from users who don’t get a static public IP address. (just src-nat to the WAN IP of your router to further conserve your pools of publics)

3

Zero, thank you very much for the info and the quick reply! I guess I am leaning more toward the first scenario that you laid out because that seems more familiar to me given the environments I manage. I’m usually on the other side of things - an ISP gives me a /30 block and whatever block of routed IPs that we request (usually a /29, but for some clients a larger block), and I nat these behind my firewall (usually a Cisco ASA), and occasionally just use pat if we only have a single static IP (/30 with no additional routed block) if we don’t have any overlap with inbound forwards (eg, only have to forward port 443 to a single internal, private IP).

So, putting myself in the position of an ISP, it’s hard to wrap my head around any form of NAT being involved on the router side because I know that as I client, I need public IP’s directly passed through to me. Further complicating all of this in my mind is that I am familiar with dealing with Cisco IOS on firewalls or configuring switches (whether they be Cisco, HP, Dell, Meraki, etc). Nothing in the MikroTik interface, GUI or command line, feels familiar!

I’ll try to lab out what you described in the first scenario tomorrow and see if I can get traffic to flow. Like I said in original post, it was easy enough to use two physical interfaces on the MikroTik, enable NAT, and pass traffic from an inside interface through an outside interface. But, without hitting the check box to enable NAT, I can’t get traffic to pass from one physical interface (the 192.168.1.0/29), through the second physical interface (10.1.1.0/30, MikroTik IP on this interface being 10.1.1.1) and out through the default route (0.0.0.0/0 to gateway of 10.1.1.2 - ISP router) to the Internet.

I’m sure I’ll come up with some more questions, and I really appreciate the help!

4

No problem.

The obvious fast check is to just drop the entire /29 onto a single physical interface and connect something there to see if it works. My suggestion #1 is a way to dole out single IPs to individual customers while keeping them on their own isolated layer2 segments.

If basic “put 192.168.1.1/29 on ether2” doesn’t work, then probably there is either a firewall filter rule blocking the traffic or else your ISP has not properly routed the /29 to the Mikrotik’s WAN interface.

The way to test the latter is simple: in the Mikrotik’s ping tool, set the src address (use the Advanced tab) to be 192.168.1.1 and try pinging things like 8.8.8.8 or even just your ISP’s end of the /30.