Hide ip addresses on gre tunnel

Survival8164 · Sun Jan 01, 2023 7:50 pm

Goodday everyone.

as a first sorry for my bad english.
However, I can read and understand English, but I can't write very well, so I type my message in Google translate and have it translate the text from Dutch to English

I have mikrotik routeros 6.49.7 running on my proxmox vm server.
on this router I have configured a hamnet (radio transmit amatuer) connection according to the following script.

/interface bridge
add comment=HamNet name=bridge
/interface gre
add allow-fast-path=no comment="Link naar GW-44-137" !keepalive name=gre09
gw-44-137-vpn.ampr.org
/routing bgp instance
set default as=4220408304 router-id=44.137.83.78 routing-table=hamnet 
/ip address 
add address=44.137.83.78/28 interface=bridge network=44.137.7.96 
add address=44.137.61.34/30 interface=gre09 network=44.137.61.32 
/ip route 
add distance=1 dst-address=44.137.83.64/28 gateway=bridge pref-src=44.137.83.78 routing-mark=hamnet scope=10 
add distance=1 dst-address=44.137.61.32 gateway=gre09 pref-src=44.137.61.34 routing-mark=hamnet scope=10 
/ip route rule 
add src-address=44.0.0.0/9 table=hamnet 
add src-address=44.128.0.0/10 table=hamnet 
add dst-address=44.0.0.0/9 table=hamnet 
add dst-address=44.128.0.0/10 table=hamnet 
/routing bgp network 
add network=44.137.83.64/28
/routing bgp peer 
add hold-time=15s in-filter=hamnet-in name=gw-44-137 nexthop-choice=force-self out-filter=hamnet-out remote-address=44.137.61.33 remote-as=4220406100 ttl=1
/routing filter
add bgp-communities=44137:10050 chain=hamnet-in set-bgp-local-pref=50 
add bgp-communities=44137:10200 chain=hamnet-in set-bgp-local-pref=200 
add bgp-communities=65535:0 chain=hamnet-in comment=RFC8326 set-bgp-local-pref=0
add chain=hamnet-in set-pref-src=44.137.83.78 
add action=accept chain=hamnet-in prefix=44.0.0.0/8 prefix-length=8-32 
add action=accept bgp-as-path=4220406100 chain=hamnet-in prefix=0.0.0.0/0 
add action=discard chain=hamnet-in 
add action=accept chain=hamnet-out prefix=44.0.0.0/8 prefix-length=8-32 
add action=accept bgp-as-path=4220406100 chain=hamnet-out prefix=0.0.0.0/0 
add action=discard chain=hamnet-out

now I can neatly do a tracert from my router to other hamnet users and my router can also be reached.
but devices connected to the lan side and that therefore have a local ip address cannot go outside via the hamnet connection.
this is probably because they see their local ip address (192.168.200.x) on the hamnet side and do not know what to do with it.
the question now is whether and if so how can I ensure that when traffic goes out through the gre tunnel, the ip locale address is made invisible / is replaced by eg 44.137.83.78 so that hamnet sees an ip address that they know?

sindy · Sun Jan 01, 2023 7:54 pm

Before even getting to the topic, what are the firewall rules of your CHR? Ideally, export the configuration and post it after obfuscating all public IP addresses, serial number, and any logins to external services.

Survival8164 · Sun Jan 01, 2023 8:21 pm

I would like to tell you what is configured on chr.
However, outside the script, I configured exactly 2 lines for the ports 8006 and 3128.
so if you tell me how or where i can find the chr ruls i would love to share them with you but unfortunately i'm not an exeprt and it's purely hobby.
So unfortunately I don't know everything.

sindy · Sun Jan 01, 2023 8:45 pm

That's what I was afraid of. Unlike the SOHO-grade devices, the CHR has no firewall rules by default because it is assumed that only people who know something about networking use it. And your CHR not only runs on a public IP but it even acts as a gateway to another network, so you shouldn't have even connected it to the internet before setting up a firewall.

So the correct procedure now would be to remove this one, install a new one from the installation image while the hardware server would be disconnected from the internet, install a set of firewall rules preventing access to its management interfaces from anywhere else than from a short list of permitted public IP addresses, or even better, only via a VPN connection (which may be the only option if you don't have a public IP address where you connect from to manage that CHR). Although all vendors do their best to make the management services secure, you can never be 100% sure and there have been many cases in the past where malware could get to routers without knowing the user credentials due to vulnerabilities (bugs in the code).

The minimum firewall would like as follows:
/ip firewall address-list
add list=mgmt-ips address=your.home.public.ip

/ip firewall filter
add chain=input connection-state=established,related action=accept
add chain=input connection-state=invalid protocol=!gre action=drop
add chain=input protocol=icmp action=accept
add chain=input protocol=tcp in-interface=ether1 src-address-list=mgmt-ips action=accept
add chain=input action=drop
add chain=forward connection-state=established,related action=accept
add chain=forward connection-state=invalid action=drop
add chain=forward action=drop

Only after setting up this, you can connect the server to the internet. No idea how you manage the proxmox remotely, hopefully it only listens at SSH and HTTPS and the password is not 123?

Survival8164 · Sun Jan 01, 2023 9:45 pm

thank you very much for the rules.
I certainly want to set this up, but I still have a question mark before I completely reinstall the router.
I have now created 2 rules at nat to open the ports of the proxmox https web interface on 1 external ip address.
I have 2 external ip addresses that I can use.
if you also want to give me an example rule how I can ensure that traffic coming in on interface ether1 intended for external ip address 111.111.111.111 can forward port 123 to 192.168.200.2 that would be very nice.

I also had a somewhat stronger password than 1234 and I also use a different password everywhere.

sindy · Sun Jan 01, 2023 10:05 pm

I have now created 2 rules at nat to open the ports of the proxmox https web interface on 1 external ip address.

I don't have much experience running bare metal in public DCs so I have no idea about the network topology. How is the management interface of the Proxmox connected to the internet, if it is? Where did you create those rules, on the CHR or in some firewall provided by the datacenter?

if you also want to give me an example rule how I can ensure that traffic coming in on interface ether1 intended for external ip address 111.111.111.111 can forward port 123 to 192.168.200.2 that would be very nice.

/ip firewall nat add chain=dstnat in-interface=ether1 dst-address=111.111.111.111 protocol=udp dst-port=123 action=dst-nat to-addresses=192.168.200.2
But additionally, you have to add a permissive rule to the forward chain just before the final "drop the rest" one, something like chain=forward connection-nat-state=dstnat action=accept.

Survival8164 · Sun Jan 01, 2023 10:19 pm

thank you very much.
I'm just installing everything on new including proxmox itself.
this because I have been messing around quite a bit here and there that it seemed smarter to me to start again at the beginning, so really.

Survival8164 · Wed Jan 04, 2023 10:52 am

Goodday everyone. In the meantime I've been puzzling again and unfortunately I'm running into 2 problems.
both are dhcp related and they appear as i have followed sindy der advices from post: Sun Jan 01, 2023 8:45 pm.
However, I do want to follow his/her advice, so the question now is how do I ensure that the DHCP problems are solved.
the problems are:
ethernet1 no longer receives an ip address from the provider or there seems to be no more dhcp traffic between the provider and the router.
problem 2: the dhcp server of mikrotik routers no longer wants to hand out ip addresses on interface ether2.

sindy · Wed Jan 04, 2023 8:31 pm

I'm running into 2 problems.
both are dhcp related and they appear as i have followed sindy der advices from post: Sun Jan 01, 2023 8:45 pm.

That's strange. For IP(v4), DHCP is hooked "closer to the wire" than the firewall, so neither the handling of DHCP responses of the server on ether1 nor handling of client requests on ether2 should be affected by any firewall rules. But his may have changed since the current long-term version (6.48.6), so use the following command-line command:
/ip firewall filter add chain=input protocol=udp dst-port=67,68 action=accept place-before=[find where chain=input action=drop !(connection-state~"invalid")]

If that helps, it means something has changed between 6.48.6 and 6.49.7; if it doesn't, something else must be wrong. In the latter case, open a command line window, make it as wide as your screen allows, run /tool sniffer quick port=67,68 in that window and see what traffic it can gather during, let's say, 3 minutes.

Off topic - when referring to a post, it is better to add a link to it than to refer to time, as the forum displays time information in local time, so if someone reads it in a different time zone, it may be misleading. Some skins show order numbers of posts within a topic. So I usually use post #4 (press the "respond with quote" button to see how this link is added).

Survival8164 · Thu Jan 05, 2023 1:56 pm

first of all thank you very much.
coming in it seems to work well now and only ports that I want open are open.
the port forwardings also seem to work fine and get a nice response and can also log in.
dhcp income from the provider also works fine now.
dhcp I suspect that also works, although I still have to connect / install an extra pc that wants an ip address from the dhcp server.

I do have one more probable problem.
namely everything that is on the lan side (ether2 ipranche 192.168.200.0/24) is just allowed to access the internet as it looks now.
So if someone can help me with what I need to adjust to let traffic go outside that would be great.

Survival8164 · Sat Jan 07, 2023 4:22 pm

luckily the security works well now.
so thank you very much for that part.

In the meantime, the connections to the 44 network are also working again.
However, one problem from the very first post and what this topic actually started for has not yet been solved.
and the question still remains how can I ensure that traffic from network 192.168.200.0/24 or one LAN network is hidden so that hamnet (the 44.x network) does not see one's local IP address, but only the address of the router.

sindy · Sat Jan 07, 2023 5:31 pm

the question still remains how can I ensure that traffic from network 192.168.200.0/24 or one LAN network is hidden so that hamnet (the 44.x network) does not see one's local IP address, but only the address of the router.

For that, you need a src-nat rule:
/ip firewall nat add chain=srcnat out-interface=gre09 action=src-nat to-addresses=44.137.61.34

Survival8164 · Sat Jan 07, 2023 6:03 pm

thank you very much for the response.
unfortunately has

/ip firewall nat add chain=srcnat out-interface=gre09 action=src-nat to-addresses=44.137.61.34

this may not work and outgoing traffic will still be blocked.

if I do a trace to eg: 44.137.0.1 then I get a nice response from mikrotik itself but the rest just time outs.
the ip address is also not viewable via the 44 network while it works from the outside.
and via one test connection (openvpn at my home directly to the network) then it works.
So it goes straight to the router.

sindy · Sat Jan 07, 2023 6:48 pm

this may not work and outgoing traffic will still be blocked.
...
if I do a trace to eg: 44.137.0.1 then I get a nice response from mikrotik itself but the rest just time outs.

Post the complete configuration.

the ip address is also not viewable via the 44 network while it works from the outside.
and via one test connection (openvpn at my home directly to the network) then it works.
So it goes straight to the router.

I did not understand these statements, sorry - too many "it"s. Let's start from the configuration.

Survival8164 · Sat Jan 07, 2023 8:13 pm

then let me start with how the configuration is now.
- interface ether1 goes to the internet
- interface ether2 is the local network
- gre is the tunnel to hamnet (hamnet is the 44 network and intended for broadcasting amateurs, i will say hamnet from now on because it is a bit easier)
- bridge is the local part of hamnet and is intended for one hamnet ip ranch from 44.137.83.64 to 79 or a /28 subnet

which ip rachne is for what
- 192.168.100.0/24 has become the network between one provider and one microtik router.
traffic coming in on 85.214.62.188 and on 85.214.172.52 will be forwarded to the router through this network
-192.168.200.0/24 is the lan network.
all servers are in this network.
- 44.137.83.64/28 is one hamnet ip ranche.
where 44.137.83.65 to 44.137.83.69 are reachable both via the internet and via hamnet

what do I ultimately want to have as an end goal.
- that I can serve both to the internet and to hamnet addresses via my local network (lan).
- that I can make port forwardings based on ip address and thus say that for example port 80 can forward external address 85.214.62.188 (192.168.100.2) to server 192.168.200.2 and port 80 to 44.137.83.65 then to 192.168.200.3 and 80 at 44.137.83.70 to 192.168.200.4

what is my configuration right now.
/ip address print detaill

 0   address=192.168.200.1/24 network=192.168.200.0 interface=ether2 actual-interface=ether2
 1   address=192.168.100.2/24 network=192.168.100.0 interface=ether1 actual-interface=ether1
 2   address=192.168.100.3/24 network=192.168.100.0 interface=ether1 actual-interface=ether1
 3   address=44.137.83.78/28 network=44.137.83.64 interface=bridge actual-interface=bridge
 4   address=44.137.61.34/30 network=44.137.61.32 interface=gre09 actual-interface=gre09
 5   address=44.137.83.65/28 network=44.137.83.64 interface=bridge actual-interface=bridge
 6   address=44.137.83.70/32 network=44.137.83.70 interface=bridge actual-interface=bridge

/ip firewall filter print detail

Flags: X - disabled, I - invalid, D - dynamic
 0    chain=input action=accept connection-state=established,related
 1    chain=input action=drop connection-state=invalid protocol=!gre
 2    chain=input action=accept protocol=icmp
 3    chain=input action=accept protocol=tcp src-address-list=mgmnt-ips in-interface=ether1
 4    chain=input action=accept in-interface=ether2
 5    chain=input action=accept protocol=udp dst-port=67,68
 6    chain=input action=drop
 7    chain=forward action=accept connection-state=established,related
 8    chain=forward action=accept in-interface=ether2
 9    chain=forward action=drop connection-state=invalid
10    chain=forward action=drop

/ip firewall addres-list print detail

 0   list=mgmnt-ips address=192.168.200.254 creation-time=jan/07/2023 11:34:56
 1   list=mgmnt-ips address=83.82.141.67 creation-time=jan/07/2023 11:35:54
 2   list=mgmnt-ips address=89.190.141.193 creation-time=jan/07/2023 11:36:32

/ip firewall nat print detail

 0    chain=srcnat action=src-nat to-addresses=44.137.61.34 out-interface=gre09 log=no
      log-prefix=""
 1    chain=srcnat action=masquerade out-interface-list=WAN

and finally
/ip route print detail

 0 ADb  dst-address=0.0.0.0/0 pref-src=44.137.83.78 gateway=44.137.61.33
        gateway-status=44.137.61.33 reachable via  gre09 distance=20 scope=40 target-scope=10
        routing-mark=hamnet bgp-as-path="4220406100" bgp-local-pref=50 bgp-med=0
        bgp-origin=incomplete bgp-communities=44137:10000,44137:10050 received-from=gw-44-137
 1 ADb  dst-address=44.0.0.0/9 pref-src=44.137.83.78 gateway=44.137.61.33
        gateway-status=44.137.61.33 reachable via  gre09 distance=20 scope=40 target-scope=10
        routing-mark=hamnet bgp-as-path="4220406100" bgp-local-pref=50 bgp-med=0
        bgp-origin=incomplete bgp-communities=44137:10000,44137:10050 received-from=gw-44-137
 2 ADb  dst-address=44.128.0.0/10 pref-src=44.137.83.78 gateway=44.137.61.33
        gateway-status=44.137.61.33 reachable via  gre09 distance=20 scope=40 target-scope=10
        routing-mark=hamnet bgp-as-path="4220406100" bgp-local-pref=50 bgp-med=0
        bgp-origin=incomplete bgp-communities=44137:10000,44137:10050 received-from=gw-44-137
 3 ADb  dst-address=44.137.0.0/16 pref-src=44.137.83.78 gateway=44.137.61.33
        gateway-status=44.137.61.33 reachable via  gre09 distance=20 scope=40 target-scope=10
        routing-mark=hamnet bgp-as-path="4220406100" bgp-local-pref=50 bgp-med=0
        bgp-origin=incomplete bgp-communities=44137:10000,44137:10050 received-from=gw-44-137
 4 A S  dst-address=44.137.61.32/32 pref-src=44.137.61.34 gateway=gre09
        gateway-status=gre09 reachable distance=1 scope=10 target-scope=10 routing-mark=hamnet
 5 A S  dst-address=44.137.83.64/28 pref-src=44.137.83.78 gateway=bridge
        gateway-status=bridge reachable distance=1 scope=10 target-scope=10 routing-mark=hamnet
 6 A S  dst-address=0.0.0.0/0 gateway=192.168.100.1
        gateway-status=192.168.100.1 reachable via  ether1 distance=1 scope=30 target-scope=10
 7 ADC  dst-address=44.137.61.32/30 pref-src=44.137.61.34 gateway=gre09
        gateway-status=gre09 reachable distance=0 scope=10
 8 ADC  dst-address=44.137.83.64/28 pref-src=44.137.83.78 gateway=bridge
        gateway-status=bridge reachable distance=0 scope=10
 9 ADC  dst-address=44.137.83.70/32 pref-src=44.137.83.70 gateway=bridge
        gateway-status=bridge reachable distance=0 scope=10
10 A S  dst-address=145.220.78.4/32 gateway=192.168.100.1
        gateway-status=192.168.100.1 reachable via  ether1 distance=1 scope=30 target-scope=10
11 ADC  dst-address=192.168.100.0/24 pref-src=192.168.100.2 gateway=ether1
        gateway-status=ether1 reachable distance=0 scope=10
12 ADC  dst-address=192.168.200.0/24 pref-src=192.168.200.1 gateway=ether2
        gateway-status=ether2 reachable distance=0 scope=10

I hope I have provided enough information. if not, I'd love to hear what else you want to know

sindy · Sat Jan 07, 2023 9:41 pm

- 44.137.83.64/28 is one hamnet ip ranche.
where 44.137.83.65 to 44.137.83.69 are reachable both via the internet and via hamnet
...
what do I ultimately want to have as an end goal.
- that I can serve both to the internet and to hamnet addresses via my local network (lan).
- that I can make port forwardings based on ip address and thus say that for example port 80 can forward external address 85.214.62.188 (192.168.100.2) to server 192.168.200.2 and port 80 to 44.137.83.65 then to 192.168.200.3 and 80 at 44.137.83.70 to 192.168.200.4

So let me rephrase it to make sure I understand the setup and goals properly.

all servers are in the LAN (192.168.200.0/24, ether2).
the addresses from the ham subnet 44.137.83.64/28 are only up on the CHR itself, and all connections arriving to these addresses will be port-forwarded to addresses from 192.169.200.0/24. I.e. the sole purpose of the interface named bridge is to bear these addresses.
when a server in 192.168.200.0/24 initiates a connection towards an address in hamnet (rather than responding to an incoming connection), the local side address of that connection has to be src-nated to an address from 44.137.83.64/28 so that the remote responder would know where to send the response

Is that correct?

What I have a problem with is the requirement that the addresses from the 44.137.83.65 - 44.137.83.69 range "are reachable from the internet" - you would have to advertise this subnet to the internet using BGP in order that the traffic for it would come directly via ether1. So I suppose that there are some border routers that interconnect hamnet with the rest of the internet, and the requests from clients in internet to the 44.137.83.65 - 44.137.83.69 range actually get to the CHR via these border routers and then via gre09. And to prevent non-symmetric routing, you need to make sure that responses to such requests are routed via gre09 even though the destination address is a non-hamnet one, whereas requests that came via ether1 must be responded via ether1. Is this understanding correct?

I hope I have provided enough information. if not, I'd love to hear what else you want to know

I was expecting to see the complete export of the configuration obtained using the /export file=somename hide-sensitive command, but I would have likely asked for actual routing table right next

Survival8164 · Sat Jan 07, 2023 9:47 pm

So let me rephrase it to make sure I understand the setup and goals properly.
all servers are in the LAN (192.168.200.0/24, ether2).

the addresses from the ham subnet 44.137.83.64/28 are only up on the CHR itself, and all connections arriving to these addresses will be port-forwarded to addresses from 192.169.200.0/24. I.e. the sole purpose of the interface named bridge is to bear these addresses.

when a server in 192.168.200.0/24 initiates a connection towards an address in hamnet (rather than responding to an incoming connection), the local side address of that connection has to be src-nated to an address from 44.137.83.64/28 so that the remote responder would know where to send the response

Is that correct?

I couldn't have said it more clearly.
so you totally understand.

sindy · Sun Jan 08, 2023 3:58 pm

What about the other parts, i.e. the access to 44.137.83.64/28 from the internet and the export of the complete configuration?

Survival8164 · Sun Jan 08, 2023 4:44 pm

I suspect you want to know about traffic from and to the router.
therefore here are the results of different tracerts so that the exact problem will hopefully become clear.

the first tracert 2 tracert are from home to the mikrotik router.
about hamnet.
where I immediately make the joke that 44.137.83.65 can be reached both via the internet and via hamnet, but 44.137.83.70 can only be reached via hamnet.

C:\Users\peter>tracert 44.137.83.70

Tracing route to sys10.pd9enp.ampr.org [44.137.83.70]
over a maximum of 30 hops:

  1    12 ms    13 ms    12 ms  gw-vpn1.pi9noz.ampr.org [44.137.60.3]
  2    11 ms    11 ms    11 ms  gw2.pi9noz.ampr.org [44.137.60.2]
  3    22 ms    23 ms    23 ms  sys10.pd9enp.ampr.org [44.137.83.70]

Trace complete.

C:\Users\peter>tracert 44.137.83.65

Tracing route to www.pd9enp.ampr.org [44.137.83.65]
over a maximum of 30 hops:

  1    12 ms    12 ms    11 ms  gw-vpn1.pi9noz.ampr.org [44.137.60.3]
  2    19 ms    11 ms    12 ms  gw2.pi9noz.ampr.org [44.137.60.2]
  3    22 ms    30 ms    31 ms  www.pd9enp.ampr.org [44.137.83.65]

the next tracert is from home via the internet to 44.137.83.65

C:\Users\peter>tracert 44.137.83.65

Tracing route to www.pd9enp.ampr.org [44.137.83.65]
over a maximum of 30 hops:

  1     1 ms    <1 ms    <1 ms  172.16.0.1
  2     2 ms     2 ms     1 ms  192.168.179.1
  3     *        *        *     Request timed out.
  4    10 ms    12 ms    19 ms  ht-rc0001-cr102-et112-251.core.as33915.net [213.51.194.21]
  5    20 ms    13 ms    14 ms  asd-tr0021-cr101-be152-10.core.as9143.net [213.51.158.12]
  6    12 ms    13 ms    12 ms  nl-ams04a-ri3-ae51-0.core.as9143.net [213.51.64.194]
  7    13 ms    12 ms    12 ms  ae78-0.asd001b-jnx-01.surf.net [145.145.166.72]
  8    12 ms    14 ms    15 ms  netsurf-router.customer.surf.net [145.145.2.219]
  9    27 ms    13 ms    12 ms  192.12.54.242
 10    15 ms    14 ms    12 ms  gw-44-137-ext.ampr.org [145.220.78.2]
 11    15 ms    13 ms    12 ms  gw.pi9noz.ampr.org [44.137.42.1]
 12    24 ms    24 ms    23 ms  www.pd9enp.ampr.org [44.137.83.65]

now a tracert to 8.8.8.8 from the lan network of the mikrotik routers

C:\Users\Administrator>tracert 8.8.8.8

Tracing route to dns.google [8.8.8.8]
over a maximum of 30 hops:

  1    <1 ms    <1 ms    <1 ms  192.168.200.1
  2    <1 ms    <1 ms    <1 ms  192.168.100.1
  3    <1 ms    <1 ms    <1 ms  vl500.dcata-b17.as6724.net [85.214.1.25]
  4    <1 ms    <1 ms    <1 ms  433.ae7.core-b1.as6724.net [85.214.0.168]
  5     1 ms    <1 ms    <1 ms  110.ae14.bb-rt1-2.h20.r22.rs.ber.de.as6724.net [85.214.2.90]
  6    <1 ms    <1 ms    <1 ms  ae-1-88.bb-a.rs.ber.de.net.ionos.com [212.227.112.110]
  7    11 ms    11 ms    11 ms  212.227.120.11
  8    11 ms    12 ms    11 ms  ae-22.bb-b.fr7.fra.de.oneandone.net [212.227.120.40]
  9    12 ms    12 ms    12 ms  212.227.112.83
 10    10 ms    10 ms    10 ms  142.251.65.73
 11    10 ms    10 ms    11 ms  142.250.226.149
 12    11 ms    10 ms    11 ms  dns.google [8.8.8.8]

now another tracert from the lan side of the mikrotik router to a hamnet ip address 44.137.80.25

C:\Users\Administrator>tracert 44.137.80.25

Tracing route to pd9enp.ampr.org [44.137.80.25]
over a maximum of 30 hops:

  1    <1 ms    <1 ms    <1 ms  192.168.200.1
  2     *        *        *     Request timed out.
  3     *        *        *     Request timed out.
  4     *        *        *     Request timed out.
  5     *        *        *     Request timed out.
  ----
  28     *        *        *     Request timed out.
  29     *        *        *     Request timed out.
  30     *        *        *     Request timed out.

and finally 2 tracerts from the mikrotik router itself.
the first is to a hamnet ip address 44.137.80.25
the 2nd is to 8.8.8.8

[admin@MikroTik] > /tool traceroute 44.137.80.25
 # ADDRESS                          LOSS SENT    LAST     AVG    BEST   WORST STD-DEV STATUS
 1 44.137.61.33                       0%   95  11.9ms    11.1    10.8    12.5     0.3
 2 44.137.60.3                        0%   94  11.3ms    11.1    10.8    11.9     0.2
 3 44.137.80.25                       0%   94  22.9ms    24.2    21.7      36     2.7

[admin@MikroTik] > /tool traceroute 8.8.8.8
 # ADDRESS                          LOSS SENT    LAST     AVG    BEST   WORST STD-DEV STATUS
 1 192.168.100.1                      0%    5   0.3ms     0.2     0.2     0.3       0
 2 85.214.1.25                        0%    5   0.4ms     4.4     0.3    20.6     8.1
 3 85.214.0.168                       0%    5   0.5ms     0.8     0.4     2.4     0.8
 4 85.214.2.90                        0%    5   0.9ms       1     0.4     2.5     0.8
 5 212.227.112.110                    0%    5   0.9ms     0.8     0.7     0.9     0.1
 6 212.227.120.11                     0%    5  10.8ms    10.9    10.8      11     0.1
 7 212.227.120.40                     0%    5  14.6ms    11.8      11    14.6     1.4
 8 212.227.112.83                     0%    5  12.5ms    12.3    12.2    12.5     0.1
 9 142.251.65.73                      0%    5  10.9ms    10.7    10.5    10.9     0.1
10 142.250.226.149                    0%    5  10.5ms    10.6    10.5    10.7     0.1
11 8.8.8.8                            0%    5  10.6ms    10.5    10.5    10.6       0

I hope it is clear what your problem is

sindy · Sun Jan 08, 2023 5:22 pm

I suspect you want to know about traffic from and to the router.
therefore here are the results of different tracerts so that the exact problem will hopefully become clear.

It wasn't immediate, but OK.

What happens is that you have configured BGP to add the routes into routing table hamnet, but you don't assign a corresponding routing-mark to the traffic anywhere (maybe you do but as you have only posted what you deem relevant rather than posting the complete export, I strongly suspect you indeed don't). So gre09 is only ever used by incoming traffic from the hamnet, whereas all the outgoing traffic, including one to hamnet destiniations, is routed via ether1/192.168.100.1. Hence the src-nat rule I've suggested never matches the connections initiated by hosts in 192.168.200.0/24 towards hamnet destinations. And since they get src-nated to the internet address instead, the response to that internet address from the remote hamnet host being pinged probably gets blocked by some firewall.

Your traceroute shows that the non-symmetric routing doesn't matter (no firewalls on the uncommon network paths), the question is whether you don't mind that you send the traffic to hamnet destinations via internet rather than via the tunnel.

where I immediately make the joke that 44.137.83.65 can be reached both via the internet and via hamnet, but 44.137.83.70 can only be reached via hamnet.

Given that although (or maybe because?) Dutch and English are really close, the translation by Google is far from perfect. So the joke got lost in translation. I haven't even noticed that there is a difference in behaviour between .65 and .70.

I hope it is clear what your problem is

I do have quite a lot of problems, but your traceroutes haven't helped with any of them

Survival8164 · Sun Jan 08, 2023 6:11 pm

thanks again for the explanation.

it could indeed just be that the problem is with the routing mark and its assignments.
Now people don't go far enough to solve this yourself with just this forum thread.
therefore a different question.
it might be an idea that I create an account for you (sindy) on my vm (proxmox) server so that you then have access to, among other things, the console of the mikrotik router.
of course I only give you access to the router so that you can see what is wrong and possibly make the adjustment to get it working.
You do not need to install any separate software to access Proxmox, because everything runs via a website.
if this is an idea, let me know, and how you want me to send you the login details.

Survival8164 · Sun Jan 08, 2023 7:46 pm

in sindy's last post it was indicated that some information was missing and I didn't post the full configuration.
that's why I googled if I could somehow post a more complete configuration.
and I found out that with export I should be able to display everything.
therefore here's the autput of the export command.

# jan/08/2023 17:37:18 by RouterOS 6.49.7
# software id =
#
#
#
/interface bridge
add comment=HamNet name=bridge
/interface gre
add allow-fast-path=no comment="Link naar GW-44-137" !keepalive name=gre09 remote-address=\
    145.220.78.4
/interface list
add name=WAN
add name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip pool
add name=dhcp ranges=192.168.200.100-192.168.200.200
/ip dhcp-server
add address-pool=dhcp disabled=no interface=ether2 name=dhcp1
/routing bgp instance
set default as=4220408304 router-id=44.137.83.78 routing-table=hamnet
/interface list member
add interface=ether1 list=WAN
add list=LAN
/ip address
add address=192.168.200.1/24 interface=ether2 network=192.168.200.0
add address=192.168.100.2/24 interface=ether1 network=192.168.100.0
add address=192.168.100.3/24 interface=ether1 network=192.168.100.0
add address=44.137.83.78/28 interface=bridge network=44.137.83.64
add address=44.137.61.34/30 interface=gre09 network=44.137.61.32
add address=44.137.83.65/28 interface=bridge network=44.137.83.64
add address=44.137.83.70 interface=bridge network=44.137.83.70
/ip dhcp-client
add interface=ether1
/ip dhcp-server network
add address=192.168.200.0/24 gateway=192.168.200.1 netmask=24
/ip dns
set servers=81.169.163.106,85.214.7.22,81.169.148.34,8.8.8.8,8.8.4.4
/ip firewall address-list
add address=192.168.200.254 list=mgmnt-ips
add address=83.82.141.67 list=mgmnt-ips
add address=89.190.141.193 list=mgmnt-ips
/ip firewall filter
add action=accept chain=input connection-state=established,related
add action=drop chain=input connection-state=invalid protocol=!gre
add action=accept chain=input protocol=icmp
add action=accept chain=input in-interface=ether1 protocol=tcp src-address-list=mgmnt-ips
add action=accept chain=input in-interface=ether2
add action=accept chain=input dst-port=67,68 protocol=udp
add action=drop chain=input
add action=accept chain=forward connection-state=established,related
add action=accept chain=forward in-interface=ether2
add action=drop chain=forward connection-state=invalid
add action=drop chain=forward
/ip firewall nat
add action=src-nat chain=srcnat out-interface=gre09 to-addresses=44.137.61.34
add action=masquerade chain=srcnat out-interface-list=WAN
/ip proxy
set anonymous=yes enabled=yes
/ip route
add distance=1 dst-address=44.137.61.32/32 gateway=gre09 pref-src=44.137.61.34 routing-mark=\
    hamnet scope=10
add distance=1 dst-address=44.137.83.64/28 gateway=bridge pref-src=44.137.83.78 routing-mark=\
    hamnet scope=10
add distance=1 gateway=192.168.100.1
add distance=1 dst-address=145.220.78.4/32 gateway=192.168.100.1
/ip route rule
add src-address=44.0.0.0/9 table=hamnet
add src-address=44.128.0.0/10 table=hamnet
add dst-address=44.0.0.0/9 table=hamnet
add dst-address=44.128.0.0/10 table=hamnet
/routing bgp network
add network=44.137.83.64/28
/routing bgp peer
add hold-time=15s in-filter=hamnet-in name=gw-44-137 nexthop-choice=force-self out-filter=\
    hamnet-out remote-address=44.137.61.33 remote-as=4220406100 ttl=1
/routing filter
add bgp-communities=44137:10050 chain=hamnet-in set-bgp-local-pref=50
add bgp-communities=44137:10200 chain=hamnet-in set-bgp-local-pref=200
add bgp-communities=65535:0 chain=hamnet-in comment=RFC8326 set-bgp-local-pref=0
add chain=hamnet-in set-pref-src=44.137.83.78
add action=accept chain=hamnet-in prefix=44.0.0.0/8 prefix-length=8-32
add action=accept bgp-as-path=4220406100 chain=hamnet-in prefix=0.0.0.0/0
add action=discard chain=hamnet-in
add action=accept chain=hamnet-out prefix=44.0.0.0/8 prefix-length=8-32
add action=accept bgp-as-path=4220406100 chain=hamnet-out prefix=0.0.0.0/0
add action=discard chain=hamnet-out

sindy · Sun Jan 08, 2023 8:28 pm

that's why I googled if I could somehow post a more complete configuration.

Have you ever noticed the contents of my automatic signature

?

From here, there are two significantly different paths depending on how you want to deal with the response traffic in connections that come from the internet to the hamnet addresses via gre09.

The first possibility is to keep sending these responses via ether/192.168.100.1 like you do now. If this is the way to go, then it is apparently sufficient to modify the src-nat rule to assign a hamnet source address to connections initiated by hosts in 192.168.200.0/24 that go via ether1 to the hamnet range, and you actually need the whole BGP thing only to advertise your /28 hamnet subnet to the remote hamnet peer (and I am actually surprised this part works because BGP uses information from the indicated routing table also for advertising, and since there are no locally created routes in the hamnet table, it should not advertise any subnets to the peer).

The second possibility is that you want to respond requests that came in via gre09 also via gre09 regardless what the initiator (client) address is; in that case, you need to engage connection marking, because that's the only way to route responses depending on the in-interface of the requests. But even in that case, you still only need BGP to advertise your local hamnet subnets - since there is just a single uplink to hamnet, static routes to 44.0.0.0/9 and 44.128.0.0/10 in routing table main and a single default route in routing table hamnet are sufficient.

So make your choice first, I'll give a more detailed instruction then.

As for direct contact - you can post your e-mail address in some robot-hostile form here and then remove it, but given the current delay of the e-mail notifications, it could stay here for quite a long time. Or you can encrypt the e-mail address or mobile number using the method described here (posts #65 to #68).

Survival8164 · Sun Jan 08, 2023 9:23 pm

thank you again now.

I don't have to think long about the choice.
that will be choice 2.
this because I once learned that for reliable connections it is always best to run everything over the same interface.
so if a request comes through gre9 also the response back through gre9
and what comes in ok ether1 also respond back via ether1.
this is because then the path of the connection remains the same.

sindy · Sun Jan 08, 2023 9:50 pm

OK, let's try then.

First, remove routing-mark=hamnet from the /routing bgp instance row - I still think it is strange that your 44.../28 subnet gets advertised to the peer when this routing-mark is in place.

Next, add the following three routes:
/ip route
add dst-address=44.0.0.0/9 gateway=gre09
add dst-address=44.128.0.0/10 gateway=gre09
add gateway=gre09 routing-mark=hamnet

At this moment, it should become possible to ping addresses in hamnet from hosts in 192.168.200.0/24, because the first two routes make connections initiated by these hosts towards hamnet IPs leave via gre09 and thus match the correct src-nat rule. Later, you may use multiple src-nat rules to use a distinct hamnet IP for each host in 192.168.200.0/24. Or you may assign the servers addresses from a single /28 range within the 192.168.200.0/24 one and use a single netmap rule instead of multiple src-nat ones.

Now, add the following rules to firewall table mangle:
/ip firewall mangle
add chain=prerouting in-interface=gre09 connection-state=new action=mark-connection new-connection-mark=hamnet
add chain=prerouting in-interface=ether2 connection-mark=hamnet action=mark-routing new-routing-mark=hamnet
add chain=output connection-mark=hamnet action=mark-routing new-routing-makr=hamnet

The first one marks any connection that came in via gre09 with a connection-mark hamnet when handling its first packet; the connection tracking module remembers this for the lifetime of the connection (or until you assign another connection-mark value to it using another rule, but that's just for the sake of completeness), and flags any subsequent packet belonging to that connection with that connection-mark, regardless the direction.

The second one assigns a routing-mark (which is in fact a name of a routing table to use) hamnet to all transit packets that come in via ether2 (i.e. from 192.168.200.0/24) and belong to connections with connection-mark hamnet. Both match conditions are essential - if the rule wouldn't check for in-interface=ether2, also packets that just came in via gre09 would be routed back via gre09 rather than to 192.168.200.0/24, because there is no route to 192.168.200.0/24 in routing table hamnet.

The third rule does the same like the second one but for the (response) packets sent by the router itself.

You can use /tool sniffer quick to check whether it really works as expected.

Survival8164 · Sun Jan 08, 2023 10:36 pm

and then I immediately get an error at the very first step.
either if I remove /routing bgp instance

[admin@MikroTik] > /routing bgp instance remove
numbers: 0
failure: cannot remove default bgp instance

sindy · Sun Jan 08, 2023 10:58 pm

routing bgp instance set 0 routing-table=""

Survival8164 · Mon Jan 09, 2023 8:03 am

good day.
thanks again for the response.
to the commands, the problem is still there.
therefore here again the full export of my router.

# jan/09/2023 05:48:58 by RouterOS 6.49.7
# software id =
#
#
#
/interface bridge
add comment=HamNet name=bridge
/interface gre
add allow-fast-path=no comment="Link naar GW-44-137" !keepalive name=gre09 remote-address=\
    145.220.78.4
/interface list
add name=WAN
add name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip pool
add name=dhcp ranges=192.168.200.100-192.168.200.200
/ip dhcp-server
add address-pool=dhcp disabled=no interface=ether2 name=dhcp1
/routing bgp instance
set default as=4220408304 router-id=44.137.83.78
/interface list member
add interface=ether1 list=WAN
add list=LAN
/ip address
add address=192.168.200.1/24 interface=ether2 network=192.168.200.0
add address=192.168.100.2/24 interface=ether1 network=192.168.100.0
add address=192.168.100.3/24 interface=ether1 network=192.168.100.0
add address=44.137.83.78/28 interface=bridge network=44.137.83.64
add address=44.137.61.34/30 interface=gre09 network=44.137.61.32
add address=44.137.83.65/28 interface=bridge network=44.137.83.64
add address=44.137.83.70 interface=bridge network=44.137.83.70
/ip dhcp-client
add interface=ether1
/ip dhcp-server network
add address=192.168.200.0/24 gateway=192.168.200.1 netmask=24
/ip dns
set servers=81.169.163.106,85.214.7.22,81.169.148.34,8.8.8.8,8.8.4.4
/ip firewall address-list
add address=192.168.200.254 list=mgmnt-ips
add address=83.82.141.67 list=mgmnt-ips
add address=89.190.141.193 list=mgmnt-ips
/ip firewall filter
add action=accept chain=input connection-state=established,related
add action=drop chain=input connection-state=invalid protocol=!gre
add action=accept chain=input protocol=icmp
add action=accept chain=input in-interface=ether1 protocol=tcp src-address-list=mgmnt-ips
add action=accept chain=input in-interface=ether2
add action=accept chain=input dst-port=67,68 protocol=udp
add action=drop chain=input
add action=accept chain=forward connection-state=established,related
add action=accept chain=forward in-interface=ether2
add action=drop chain=forward connection-state=invalid
add action=drop chain=forward
/ip firewall mangle
add action=mark-connection chain=prerouting connection-state=new in-interface=gre09 \
    new-connection-mark=hamnet
add action=mark-routing chain=prerouting connection-mark=hamnet in-interface=ether2 \
    new-routing-mark=hamnet
add action=mark-routing chain=output connection-mark=hamnet new-routing-mark=hamnet
/ip firewall nat
add action=src-nat chain=srcnat out-interface=gre09 to-addresses=44.137.61.34
add action=masquerade chain=srcnat out-interface-list=WAN
/ip proxy
set anonymous=yes enabled=yes
/ip route
add distance=1 gateway=gre09 routing-mark=hamnet
add distance=1 dst-address=44.137.61.32/32 gateway=gre09 pref-src=44.137.61.34 routing-mark=\
    hamnet scope=10
add distance=1 dst-address=44.137.83.64/28 gateway=bridge pref-src=44.137.83.78 routing-mark=\
    hamnet scope=10
add distance=1 gateway=192.168.100.1
add distance=1 dst-address=44.0.0.0/9 gateway=gre09
add distance=1 dst-address=44.128.0.0/10 gateway=gre09
add distance=1 dst-address=145.220.78.4/32 gateway=192.168.100.1
/ip route rule
add src-address=44.0.0.0/9 table=hamnet
add src-address=44.128.0.0/10 table=hamnet
add dst-address=44.0.0.0/9 table=hamnet
add dst-address=44.128.0.0/10 table=hamnet
/routing bgp network
add network=44.137.83.64/28
/routing bgp peer
add hold-time=15s in-filter=hamnet-in name=gw-44-137 nexthop-choice=force-self out-filter=\
    hamnet-out remote-address=44.137.61.33 remote-as=4220406100 ttl=1
/routing filter
add bgp-communities=44137:10050 chain=hamnet-in set-bgp-local-pref=50
add bgp-communities=44137:10200 chain=hamnet-in set-bgp-local-pref=200
add bgp-communities=65535:0 chain=hamnet-in comment=RFC8326 set-bgp-local-pref=0
add chain=hamnet-in set-pref-src=44.137.83.78
add action=accept chain=hamnet-in prefix=44.0.0.0/8 prefix-length=8-32
add action=accept bgp-as-path=4220406100 chain=hamnet-in prefix=0.0.0.0/0
add action=discard chain=hamnet-in
add action=accept chain=hamnet-out prefix=44.0.0.0/8 prefix-length=8-32
add action=accept bgp-as-path=4220406100 chain=hamnet-out prefix=0.0.0.0/0
add action=discard chain=hamnet-out

also at this one email address.
I will put this between " change my email address # by @ and ; by a .
I also put spats in it, these must of course also be omitted
"peter vd linden 1976 # hotmail ; com"

I put the email address down so that emails can be sent for an account on my server.

further is it meschien an idea that someone (eg sindy ) puts an export here as it should be so that I can import it?

sindy · Mon Jan 09, 2023 6:11 pm

Just for the case, I have sent you an e-mail and got nothing back, neither an error nor a response - maybe I have deciphered your e-mail address wrong?

Survival8164 · Mon Jan 09, 2023 6:24 pm

Unfortunately I can't find anything in my mail box either, so it seems that I didn't receive anything.
Can you by any chance say which subject you have given the e-mail, then I can search the mail box to see if I can find it by chance.
as far as the descrambling is concerned, it could of course be that you descrambled it wrong.
although that seems strong to me because it is much easier than configuring a mikrotik router.

but just to be sure, use another form of your email address.
before the @ must be: petervdlinden1976 then you get an @ followed by hotmail.com

sindy · Mon Jan 09, 2023 6:28 pm

That's exactly the address I've used - I've copied this new one to the search field for "recipient", removed the extra text (" then you get an "etc.), and it has found the message I've send.

The title of the message was "from the mikrotik forum".

Survival8164 · Mon Jan 09, 2023 7:42 pm

strange.
I just can't find an email with the subject "from the mikrotik forum"
but luckily I have other email addresses so would you like to send an email to the following address.
also now I put it between " and it is remove spatsies and replace # with @ and ; replace with . "jenp peter # gmail ; com"

sindy · Mon Jan 09, 2023 7:50 pm

Still nothing?

Survival8164 · Mon Jan 09, 2023 8:07 pm

yes definitely now.
You have also received an email from me

Hide ip addresses on gre tunnel

Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Re: Hide ip addresses on gre tunnel

Who is online