Community discussions

MikroTik App
 
sawesa
just joined
Topic Author
Posts: 12
Joined: Sat Jun 29, 2019 7:10 pm

understanding packet sniffer

Thu Oct 03, 2019 11:00 am

Hello,

I have a tunneled connection between AWS and a customer endpoint.
I got a Mikrotik that receives by one tunnel the traffic from AWS, NATs and through another tunnel sends the traffic to the customer.
Both are IPsec tunnels.

Communication from the AWS network works (icmp and telnet to certain port) but there is no echo to some tcp packets they are sending to that very same port, and they are blaming the network.

I am 90% sure the network is just fine, since I can see the hits in my NAT rules on both directions, and also the telnet connected to that port should be enough proof that the network is OK. I have even prepared a mangled rule just to record the return traffic and it is being hit each time they try their script: (10.15.1.1 is the customer and the 10.93.0.0 is the AWS network)
09:13:16 firewall,info TO_AWS forward: in:pppoe-out1 out:pppoe-out1, src-mac xx:xx:xx:xx:xx:xx, proto TCP (SYN,ACK), 10.15.1.1:9998->10.93.1.100:33722, NAT (10.15.
1.1:9998->10.1.200.2:9998)->(10.2.200.15:33722->10.93.1.100:33722), len 60 
But I dont manage to sniff the return traffic to the AWS. It seems I only get the way-in traffic, and never the return. It is driving me nuts. I filter with the AWS address, and I expect to see the traffic going from and going back. But I never capture the traffic going back!! Even while I am pinging and I'm getting echo replies, wireshark only shows me one direction packet with the message "no response found"

I prepare the sniffer in the Winbox, choosing the outside interface, putting the AWS IP network, direction: any, operation:and.
(I choose direction any even when I think the direction is always TX in my case (both tunnel enter and exit through the outside interface and traffic is never initiated by us). I have anyway tried to apply filter for the RX and nothing is recorded)
My NAT rules goes:
Fron AWS 10.93.0.0/22 to 10.1.200.2 (which is my Mikrotik)
I dstNAT this traffic to the destination 10.15.1.1
0 chain=dstnat action=dst-nat to-addresses=10.15.1.1 src-address=10.93.0.0/22 dst-address=10.1.200.2 log=yes log-prefix=""
then I do the scrnat
1 chain=srcnat action=src-nat to-addresses=10.2.200.8/29 src-address=10.93.0.0/22 dst-address=10.15.1.1 log=yes log-prefix=""
So if someone can shade some light here: how could I capture traffic in that very last step of the communication, I mean, the return traffic to the AWS?

Thank you very much!
 
User avatar
CZFan
Forum Guru
Forum Guru
Posts: 2098
Joined: Sun Oct 09, 2016 8:25 pm
Location: South Africa, Krugersdorp (Home town of Brad Binder)
Contact:

Re: understanding packet sniffer

Fri Oct 04, 2019 12:28 am

When you set interface in sniffer, do you use physical interface or PPPoE-out1 interface?
 
sawesa
just joined
Topic Author
Posts: 12
Joined: Sat Jun 29, 2019 7:10 pm

Re: understanding packet sniffer

Fri Oct 04, 2019 10:34 am

I select PPPoE-out1
 
pe1chl
Forum Guru
Forum Guru
Posts: 10215
Joined: Mon Jun 08, 2015 12:09 pm

Re: understanding packet sniffer

Fri Oct 04, 2019 10:58 am

When you are tracing traffic that is sent via IPsec, you should trace on the tunnel interface to see the plaintext traffic in both directions.
When you have no tunnel interface (because you use plain IPsec tunnel in both directions instead of e.g. GRE/IPsec) you are out of luck.

However, when looking at the problem you report, it probably is a MTU issue. Those tunneled packets have a smaller MTU than the plain ethernet MTU, especially when using PPPoE that can cause problems.
You can try to put this in the router to see if that fixes it, if so you need to carefully study the MTU and make more appropriate changes:
/ip firewall mangle
add action=change-mss chain=forward new-mss=1280
 
sawesa
just joined
Topic Author
Posts: 12
Joined: Sat Jun 29, 2019 7:10 pm

Re: understanding packet sniffer

Fri Oct 04, 2019 3:10 pm

Thank you for your suggestion.
I have read many times in this forum about netadmins setting up tunnels as GRE/IPsec, I need to dig more into that.

I have applied the MTU rule (had to choose syn TCP for the router to accept it) and they tried their script again with same luck (no response). I saw the hit in the mangle rule.
I have further read about MTU problems and found about the ""Clamp to pmtu" option, do you think it is a good idea to use it instead giving a fixed MTU of 1280?
Anyway I am seriusly suspecting the customer is actually not replying with anything so I have requested them to troubleshoot together.
 
pe1chl
Forum Guru
Forum Guru
Posts: 10215
Joined: Mon Jun 08, 2015 12:09 pm

Re: understanding packet sniffer

Fri Oct 04, 2019 5:17 pm

Sorry I forgot to include the options passthrough=yes protocol=tcp tcp-flags=syn that you may want to use with this rule.
The reason to use a low MSS (1280) is to cover the case that the correct MSS is not really known by the router.
This can happen because you use PPPoE as well (which also costs some MTU when no RFC4638 is used) and combining all those tunnels with different MTU limits usually causes problems with large packets.
With MSS 1280 you should be well below any limits in this configuration.
When it still does not work there apparently is another reason.

Who is online

Users browsing this forum: ddregs, TheCat12 and 48 guests