MikroTik

Posted: **Fri Apr 15, 2016 2:28 pm**

I'm probably doing something stupid. Or, Movistar has done something stupid (again) with their (weird, custom) router firmware. I'm hoping someone can tell me which, and what

I have Movistar (Spain)'s fibre based Internet service "Fusión", with a VG-8050 Internet router.
Behind that I have a MikroTik RouterBOARD 951Ui 2HnD.

The MikroTik only has Ether2 connected. I am not using the MikroTik's Ether1 "upstream" port.
The MikroTik's sole job in life is to function as a VPN server.

The LAN IP addressing is 192.168.255.0/26, with the VG-8050 Internet router as 192.168.255.7, a Windows Server 2012R2 at 192.168.255.8 and a variety of other boxes and appliances either statically or DHCP configured within that 192.168.255.0/26 range (DHCP server on the Windows Server).

The MikroTik's Ether2 (which appears as ether2-master-local) has two /IP ADDR configured, as 192.168.255.3/26 and as 192.168.255.125/26.
The MikroTik's default route is 192.168.255.7. No pref-src set.

I've configured both a PPTP VPN and an SSTP VPN. An Internet client (a Windows 10 notebook computer) can connect to both successfully, and can navigate through the MikroTik VPN tunnels to the 192.168.255.0/26 LAN segment.

However, any attempt to navigate through the MikroTik VPN tunnel to the Internet results in all packets being lost.
/TOOL SNIFFER captures show packets arriving through the SSTP or PPTP interface on the MikroTik from the IP address which the MikroTik has assigned to the VPN client (/IP POOL VPNpool1 ranges=192.168.255.80-192.168.255.89) and being sent out the bridge-local interface (presumably down the default route to the VG-8050 at 192.168.255.7) ... and then nothing more is ever heard of those packets.

I'm fairly sure that it's some quirk of the VG-8050 rather than something I've done wrong on the MikroTik, but...
Does anyone see anything that I've done wrong in the MikroTik configuration above?
Or have any other suggestion for fixing why the VPN tunnels seem to work but navigating the Internet through those tunnels does not work?

Oh, bugger. I've probably just proved the theory that it's a problem on the VG-8050: I statically assigned an IP address from that 192.168.255.80-89 range to a notebook computer and cabled it by Ethernet to the internal LAN, and it behaves exactly the same as clients when connected through the VPNs...

Well, still, if anyone's got any sage advice?

thanks!

Posted: **Fri Apr 15, 2016 5:36 pm**

Hi libove,

I could be mistaken but ...
If you configured your PC to (lets say) 192.168.255.80/26 than this computer would be able to reach all addresses between 192.168.255.65 and 192.168.255.126.
Your default gateway (192.168.255.7/26) is not within this address space so the computer does not know how to reach it. You need to define a gateway that is within the address space.

If you use your MikroTik (192.168.255.125/26) for that purpose you need to make sure that a froward rule allows that traffic and you would need a nat rule (masquarade). Otherwise the VG-8050 would not be able to send packets back to the computer.
Instead of the NAT rule you also could add a appropriate route on the VG-8050.

Best regards,
Lui

Posted: **Fri Apr 15, 2016 6:51 pm**

Hi Lui,
I apologise, I was sure that I'd mentioned - there already is a static route on the VG-8050 pointing 192.168.255.64/26 -> 192.168.255.3 (the MikroTik).

You mention needing a forwarding rule and a masquerade rule.

Why would a forwarding (firewall, right?) rule be necessary? The MikroTik is a router by default, isn't it?
Here's /ip route print:

[admin@MikroTik1] /interface wireless> /ip route print
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, 
B - blackhole, U - unreachable, P - prohibit 
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 A S  0.0.0.0/0                          192.168.255.7             1
 1 ADC  192.168.255.0/26   192.168.255.3   bridge-local              0
 2 ADC  192.168.255.64/26  192.168.255.125 bridge-local              0
 3 ADC  192.168.255.88/32  192.168.255.125 SSTP1                     0

So, the MikroTik should automatically function as a router, without any specific firewall rules, shouldn't it?
The MikroTik wiki Simple Static Routing page seems to support this (by the absence of any instruction to set up any additional/forwarding rules).

About a masquerade rule, I don't want an additional layer of NAT/masquerade, and with proper routes set up, it seems that it shouldn't be necessary. Everything "inside" of the VG-8050 is consistent with the subnets I'm using (no overlap/conflict), and the MikroTik routes, so no internal masquerading should be needed.
The VG-8050 performs NAT for everything inside.

In terms of the MikroTik's firewall filter, it's completely default, and I've also tested adding (place-before=0) chain=input action=accept, chain=output action=accept, and chain=forward action=accept - these made no difference.

What am I misunderstanding?

thanks!

Posted: **Fri Apr 15, 2016 8:30 pm**

Hi libove,

the route entry on the VG-8050 is the much cleaner solution. I thought the VG-8050 was preconfigured from your provider and you didn't have the possibility to add routes.

Your setup sounds correct. Here the summarized important parts (please check if they match your configuration):

Config on VG-8050 (RouterOS syntax)

/ip address
add address=192.168.255.7/26 interface=ether2 network=192.168.255.0
/ip route
add distance=1 dst-address=192.168.255.64/26 gateway=192.168.255.3

Config on the MikroTik

/ip address
add address=192.168.255.3/26 interface=bridge-local network=192.168.255.0
add address=192.168.255.125/26 interface=bridge-local network=192.168.255.64
/ip route
add distance=1 gateway=192.168.255.7

Config on your test computer
IP-Address: 192.168.255.80
Subnetmask: 255.255.255.192
Default Gateway: 192.168.255.125

With this setup you should be able to ping the VG-8050 (192.168.255.7) and your server (192.168.255.8 ). Are there any firewall rules on the VG-8050 that could block that communication?

To rule out the MikroTik completely, you could ad a route on the Win2012R2 sever so that the server can directly respond to a request from a device in the 192.168.255.64/26 network. You can add such a route by executing the following command on the command line (administrative privileges necessary):

route ADD 192.168.255.64 MASK 255.255.255.192 192.168.255.3

You then should be able to unplug the VG-8050 and still be able to communicate between server (192.168.255.8 ) and your computer (192.168.255.80). If it works, that means that the MikroTik router is routing correctly.

Best regards,
Lui

Posted: **Fri Apr 15, 2016 9:11 pm**

Thanks again Lui.
Yes, I've always been able to connect across the two subnets; effectively, I'd already done the test that you suggest.
And, yes, the configuration script that you provided matches the configuration of the two devices, with one minor difference:
The MikroTik's IP addressing is attached to ether2-master-local instead of to bridge-local.
Would that matter?

The VG-8050 has a (much less friendly!) command line interface, on which I was able to test

VG> ping 192.168.255.125

[success]

So it's not that the VG-8050 can't reach the MikroTik via the VG-8050's static route -> 192.168.255.64/26 via MikroTik[192.168.255.3], it clearly can.

It's either that the VG does not want to listen to packets originating from the MikroTik, or that the MikroTik does not want to emit them for some reason.

On the MikroTik I then did both:

/ping src-address=192.168.255.125 interface=ether2-master-local 192.168.255.7
/ping src-address=192.168.255.3 interface=ether2-master-local 192.168.255.7

.. and neither one got a response. Weird.

Weirder: I also ran a sniffer on the MikroTik while I ran those two ping commands on the MikroTik, and the sniffer picked up nothing, which suggests that the MikroTik itself doesn't want to emit those packets for some reason?
Wait, maybe the sniffer did not pick up nothing, exactly; it did see a multicast coming from the VG 192.168.255.7 as follows:

[admin@MikroTik1] /tool sniffer> pack print det
 0 time=3.34 num=1 direction=rx interface=ether2-master-local src-address=192.168.255.7 dst-address=224.0.0.1 protocol=ip ip-protocol=igmp 
   size=46 cpu=0 fp=no ip-packet-size=32 ip-header-size=24 dscp=0 identification=0 fragment-offset=0 ttl=1 

 1 time=3.34 num=2 direction=rx interface=bridge-local src-address=192.168.255.7 dst-address=224.0.0.1 protocol=ip ip-protocol=igmp size=46 
   cpu=0 fp=no ip-packet-size=32 ip-header-size=24 dscp=0 identification=0 fragment-offset=0 ttl=1

This could be just a coincidence, or, does this suggest that the VG thinks that it needs to do some IGMP magic before it can reply to the packets being sent to it from the MikroTik?

Would it matter that the MikroTik is listening and replying on the same physical and logical port (ether2-master-local)? (It shouldn't; it's not an ideal physical configuration because of possible contention on the one Ethernet cable, but for how this is going to be used that's simply not a problem - after testing, only guest WiFi and VPN clients will come through the MikroTik, so maximum bandwidth will be rather less than one gigabit Ethernet port).

Sorry for all the wild hares. Any other ideas?

thanks again!

Posted: **Sat Apr 16, 2016 12:23 am**

Hi libove,

The think we have to figure out why the MikroTik isn't able to ping the VG. If we solve this, the rest will be probably working as well.

Can you ping the VG from your server (or is the VG blocking ICMP)?
Can you ping the Server from the MikroTik?

The MikroTik's IP addressing is attached to ether2-master-local instead of to bridge-local.
Would that matter?

That shouldn't make any difference.

Would it matter that the MikroTik is listening and replying on the same physical and logical port (ether2-master-local)?

No. This should work just fine.

The thing that alarms me is, that you didn't see any packets when sniffing for icmp. I did some tests and this only happens when the router doesn't know the destination mac (so he cannot sent the ip packet).
What filters did you use (/tool sniffer print)?
What does "/ip arp print" show?

What version of RouterOS do you use?
Can you send me your config * (/export)?

* you certainly can strip the confidential parts but please leave the rest of the config ... I want to put the config onto my test-device and see if I can reproduce the behavior.

Best regards,
Lui

Posted: **Sun Apr 17, 2016 10:44 am**

Hi again Lui,
I've re-tested the PINGing, and now it works as expected between the MikroTik (on both it's 192.168.255.3 and 192.168.255.125 addresses) and the VG-8050 - I can't begin to explain why it didn't work before. (Internal hosts e.g. the Windows server on 192.168.255.8 could always PING the VG-8050 on 192.168.255.7 as well as anything else on either of the two internal subnets).

/ip arp knows the MAC address of the VG-8050. (Given the above, this is no surprise).
/tool sniffer options were similar to:

/tool sniffer> print
                     only-headers: no
                     memory-limit: 100KiB
                    memory-scroll: yes
                        file-name: 
                       file-limit: 1000KiB
                streaming-enabled: no
                 streaming-server: 0.0.0.0
                    filter-stream: no
                 filter-interface: bridge-local
               filter-mac-address: 
              filter-mac-protocol: 
                filter-ip-address: 192.168.255.7/32
              filter-ipv6-address: 
               filter-ip-protocol: 
                      filter-port: 
                       filter-cpu: 
                 filter-direction: any
  filter-operator-between-entries: or
                          running: yes

(I also tried filter-interface: any, variations on filter-ip-address, etc)
Again, I can't explain why at the time I made that earlier post, simple PINGs between the MikroTik and the VG-8050 didn't work; they do work now. So I think that was a red herring.

Outbound navigation to the Internet still doesn't work from 192.168.255.64/26.

I'm fairly sure that I've narrowed the problem down to the VG-8050 and how it decides to (not) NAT packets from the inside which do not come with src-address on the one 192.168.255.0/26 subnet to which the VG-8050 is directly attached.

I added a NAT masquerade rule to the MikroTik to hide anything leaving the MikroTik with an original src-address=192.168.255.64/26 behind the MikroTik's 192.168.255.3 address and navigation now works.

I don't want to keep this configuration over the long term, because double-NAT sometimes causes problems, but I think it does prove that the problem lies with how the VG-8050 decides which exiting packets to NAT.
I've tested and eliminated the possibility that it is a firewall rule on the VG-8050 missing/causing these packets to not be forwarded; I'm convinced that the packets are arriving into the VG-8050 and it's either dropping them or forwarding them withOUT NAT'ing them behind its Internet public IP address.

p.s. Yes, that's definitely the problem. I stuck an ancient 10Mb/s non-duplex hub between the Movistar VG-8050 router's Ethernet upstream port and the Movistar fibre terminator, and I am able to see that the VG-8050 DOES forward packets from the 192.168.255.64/26 network ... withOUT NAT'ing them, so of course they then disappear into the (non-)Ether.

I'm continuing to try to get help on the Movistar forums, and will come back here to report what I find.

thanks again,

Posted: **Sun Apr 17, 2016 11:09 am**

Oh, and:

RouterOS v6.34.4 (stable)
RouterBOARD firmware 3.30

/system license print:
software-id: 68EG-0A8X
nlevel: 4
features:

/system package print
Flags: X - disabled 
 #   NAME                                                  VERSION                                                  SCHEDULED              
 0   routeros-mipsbe                                       6.34.4                                                                          
 1   system                                                6.34.4                                                                          
 2 X wireless-cm2                                          6.34.4                                                                          
 3 X ipv6                                                  6.34.4                                                                          
 4   wireless-fp                                           6.34.4                                                                          
 5   hotspot                                               6.34.4                                                                          
 6   dhcp                                                  6.34.4                                                                          
 7   mpls                                                  6.34.4                                                                          
 8   routing                                               6.34.4                                                                          
 9   ppp                                                   6.34.4                                                                          
10   security                                              6.34.4                                                                          
11   advanced-tools                                        6.34.4

/system resource print 
                   uptime: 21m52s
                  version: 6.34.4 (stable)
               build-time: Mar/24/2016 13:13:08
              free-memory: 106.1MiB
             total-memory: 128.0MiB
                      cpu: MIPS 74Kc V4.12
                cpu-count: 1
            cpu-frequency: 600MHz
                 cpu-load: 0%
           free-hdd-space: 110.6MiB
          total-hdd-space: 128.0MiB
  write-sect-since-reboot: 63
         write-sect-total: 9047
               bad-blocks: 0%
        architecture-name: mipsbe
               board-name: RB951Ui-2HnD
                 platform: MikroTik

MikroTik

Traffic through VPN doesn't reach Internet through ISP's router

Traffic through VPN doesn't reach Internet through ISP's router

Re: Traffic through VPN doesn't reach Internet through ISP's router

Re: Traffic through VPN doesn't reach Internet through ISP's router

Re: Traffic through VPN doesn't reach Internet through ISP's router

Re: Traffic through VPN doesn't reach Internet through ISP's router

Re: Traffic through VPN doesn't reach Internet through ISP's router

Re: Traffic through VPN doesn't reach Internet through ISP's router

Re: Traffic through VPN doesn't reach Internet through ISP's router