Community discussions

 
User avatar
hendry
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 60
Joined: Sat Jan 18, 2014 9:59 am
Location: Singapore
Contact:

/tool sniffer Code: 3 (Port unreachable)

Thu Apr 18, 2019 6:41 am

Hi, I'm trying to update a tutorial that uses Mikrotik hardware to capture packets and send to a machine running Wireshark for analysis.

The problem I have is noise. Lots of Code: 3 (Port unreachable) messages that don't make any sense.

Image

Even filtering on tcp only doesn't appear to help!

Here's a screencast: https://s.natalian.org/2019-04-18/mikrotik-sniffing.mp4

I've asked the Wireshark community, and they seem to think I have the Mikrotik misconfigured somehow:
https://ask.wireshark.org/question/8619 ... iff-noise/
RouterBOARD 4xRB952Ui-5ac2nD & 1xRB952Ui-5ac2nD
https://natalian.org/2017/08/20/Choosin ... _Ubiquiti/
 
Sob
Forum Guru
Forum Guru
Posts: 4812
Joined: Mon Apr 20, 2009 9:11 pm

Re: /tool sniffer Code: 3 (Port unreachable)

Thu Apr 18, 2019 11:36 pm

That's sort of correct, because TZSP used by packet sniffer for streaming sends packets to udp port 37008. But on target computer, nothing listens on that port, Wireshark certainly doesn't. And if it's not blocked by firewall, your computer sending back icmp port unreachable is exactly what should happen.

I don't know what's the correct solution. Wireshark sniffing on interface is definitely not it. A clean one would be some helper program that would be actually listenening on udp 37008, receiving packets from RouterOS, removing TZSP wrapping and passing them to Wireshark. There's "UDP Listener remote capture" which sounds good, but it doesn't seem to be it. It listens on port 5555 (I didn't find a way how to change it from Wireshark) and when I forwarded TZSP packets through another router to change the port, the output is weird anyway.
People who quote full posts should be spanked with ethernet cable. Some exceptions for multi-topic threads may apply.
 
McSee
Frequent Visitor
Frequent Visitor
Posts: 67
Joined: Tue Feb 26, 2019 12:49 pm

Re: /tool sniffer Code: 3 (Port unreachable)

Fri Apr 19, 2019 2:52 am

You may try to use "udp port 37008" as a capture filter in Wireshark, it works for me.

And it's also a good idea to filter stream on mikrotik's side at least down to a certain interface as poor Wireshark
gets confused by the same packets captured several times.
 
Sob
Forum Guru
Forum Guru
Posts: 4812
Joined: Mon Apr 20, 2009 9:11 pm

Re: /tool sniffer Code: 3 (Port unreachable)

Fri Apr 19, 2019 4:14 am

Capture filter in Wireshark doesn't change the fact that RouterOS is sending packets to closed port. So if target computer sends back "port unreachable" and you're capturing traffic on same interface where it's connected, you will see it in captured data (if there's not a filter that excludes it).

But it seems that so far nobody cared about proper solution, i.e. something that would actually receive those packets and pass them to Wireshark. I don't know if I'm just using wrong keywords for searching, but I found nothing.
People who quote full posts should be spanked with ethernet cable. Some exceptions for multi-topic threads may apply.
 
McSee
Frequent Visitor
Frequent Visitor
Posts: 67
Joined: Tue Feb 26, 2019 12:49 pm

Re: /tool sniffer Code: 3 (Port unreachable)

Fri Apr 19, 2019 6:00 am

I do not have these "port unreachable" icmp packets in my capture (see below).
Settings as follows:
only-headers: no
memory-limit: 1000KiB
memory-scroll: yes
file-name: 
file-limit: 30000KiB
streaming-enabled: yes
streaming-server: 192.168.10.101
filter-stream: yes
filter-interface: bridge
filter-mac-address: 
filter-mac-protocol: 
filter-ip-address: 
filter-ipv6-address: 
filter-ip-protocol: 
filter-port: !winbox
filter-cpu: 
filter-direction: any
filter-operator-between-entries: or
W1.PNG
You do not have the required permissions to view the files attached to this post.
 
Zwe
just joined
Posts: 11
Joined: Thu Jan 11, 2018 11:19 am
Location: Myanmar,Mandalay
Contact:

Re: /tool sniffer Code: 3 (Port unreachable)

Fri Apr 19, 2019 6:24 am

Good job bro.
 
User avatar
hendry
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 60
Joined: Sat Jan 18, 2014 9:59 am
Location: Singapore
Contact:

Re: /tool sniffer Code: 3 (Port unreachable)

Fri Apr 19, 2019 12:11 pm

Spent some of my good Friday trying some things out between two computers:
  • "udp port 37008" is not a good solution since on both MacOS / Arch I get bytes missing in capture file when I follow the stream
  • filter-stream option seems to have no effect
  • macos's `brew cask install wireshark` does NOT have this Code: 3 (Post unreachable) problem :-? UPDATE: Actually I saw it on MacOS, so now I am confused.
  • think filter-interface: bridge1 works better than the previous interface I tried
Screencast: https://s.natalian.org/2019-04-19/sniffing.mp4
RouterBOARD 4xRB952Ui-5ac2nD & 1xRB952Ui-5ac2nD
https://natalian.org/2017/08/20/Choosin ... _Ubiquiti/
 
McSee
Frequent Visitor
Frequent Visitor
Posts: 67
Joined: Tue Feb 26, 2019 12:49 pm

Re: /tool sniffer Code: 3 (Port unreachable)

Fri Apr 19, 2019 4:40 pm

  • "udp port 37008" is not a good solution since on both MacOS / Arch I get bytes missing in capture file when I follow the stream
  • filter-stream option seems to have no effect
  • macos's `brew cask install wireshark` does NOT have this Code: 3 (Post unreachable) problem :-? UPDATE: Actually I saw it on MacOS, so now I am confused.
  • think filter-interface: bridge1 works better than the previous interface I tried
Following your list:

1. This is strange as Mikrotik's packet sniffer should send all captured packets to UDP/37008. And when you set "TZSP" view filter in Wireshark
you should see only UDP packets with dst port 37008 - the same packets that should be captured with "udp port 37008" capture filter.
It seems I get the same results with either method.

2. Filter-stream very much works for me as without it enabled I get ~170 Mbps of some (fragmented ?) UDP packets captured out of the blue, as on the screenshot below. And it's also cancels out those "port unreachable" packets when I've tested on macOS, but you of course still see them captured locally if do not set capture filter in Wireshark, and you can always tuck them away with "TZSP" view filter.
W2.PNG
3. Yes, I finally have been able to see those "port unreachable" on macOS, but none of them when testing on Windows 10, with any settings of Mikrotik's packet sniffer and Wireshark.
You do not have the required permissions to view the files attached to this post.
 
Sob
Forum Guru
Forum Guru
Posts: 4812
Joined: Mon Apr 20, 2009 9:11 pm

Re: /tool sniffer Code: 3 (Port unreachable)

Fri Apr 19, 2019 6:33 pm

If closed udp port (= no process is listening on it) receives packet, system generates icmp port unreachable and sends it to client. That's normal behaviour. This does not happen when the port is firewalled and incoming packet is silently dropped before it can reach the closed port. I don't know about MacOS, but firewall in Windows by default blocks everything, so that could explain the difference.

Next part is RouterOS, it depends on where you capture data and where you send them. Let's say you have two interfaces on router (LAN and WAN) and your Wireshark machine is connected to LAN. If you capture packets on WAN, then even if Wireshark machine sends icmp port unreachable to router, it doesn't matter, because it's on LAN interface and sniffer doesn't see it. But if you capture packets on LAN, then sniffer sees incoming icmp port unreachable and will send them back to you, wrapped in TZSP.

The main problem is that capturing TZSP with Wireshark this way is simply wrong. What you want to see are packets from router, exactly as the router had them, but it's not what you're getting:
tzsp.png
The whole red part shouldn't be there, it's TZSP wrapping and you're not intersted in that. It's confusing for both user and sometimes also for Wireshark, it seems. But it doesn't look like anyone implemented proper solution yet (something actually receiving TZSP packets and unwrapping originals).
You do not have the required permissions to view the files attached to this post.
People who quote full posts should be spanked with ethernet cable. Some exceptions for multi-topic threads may apply.
 
r00t
Member Candidate
Member Candidate
Posts: 218
Joined: Tue Nov 28, 2017 2:14 am

Re: /tool sniffer Code: 3 (Port unreachable)

Fri Apr 19, 2019 11:15 pm

Yes, this is the problem with TZSP and UDP. Often some packets get lost, meaning IDS can't inspect all of them. Also wireshark doesn't like it and some filters apply to the UDP TZSP stream instead of sniffed packets.
It would be better to have an option to do port mirroring for bridges in ROS, where all traffic would be mirrored to a selected port. You could then set up a tunnel to get these packets to the IDS or receive them on PC on separate network interface (like OpenVPN).
 
Sob
Forum Guru
Forum Guru
Posts: 4812
Joined: Mon Apr 20, 2009 9:11 pm

Re: /tool sniffer Code: 3 (Port unreachable)

Sat Apr 20, 2019 2:24 am

I played with it a little more and made a proof-of-concept of what I consider proper solution. Wireshark supports plugin interface called extcap. It's external executables or scripts that acquire data and feed them to Wireshark. And it's exactly what's needed here, something that receives TZSP packets on port 37008, unwraps original packets and passes them to Wireshark.

It's nowhere near to really be THE solution, it's minimalistic version without any error checks and with ugly shortcuts, but if you're not too unlucky, it works. It's made with PHP, which is probably quite unusual choice, but I really like PHP, especially for quick tests. It's tested with Wireshark 3.0.1 on Windows 7.

<Wireshark directory>\extcap\tzspdump.cmd:
@echo off
<path to php directory>\php.exe "<Wireshark directory>\tzspdump.php" %*
<Wireshark directory>\tzspdump.php:
<?php
  $options = getopt('', Array('extcap-interfaces', 'capture', 'extcap-interface:', 'fifo:', 'extcap-capture-filter:'));
  if(empty($options)) {
    echo 'Wireshark - tzspdump.php v0.0.1'.PHP_EOL;
  } elseif(isset($options['extcap-interfaces'])) {
    echo 'extcap {version=0.0.1}'.PHP_EOL;
    echo 'interface {value=tzspdump}{display=TZSP Listener remote capture}'.PHP_EOL;
  } elseif(isset($options['capture'])) {
    if(isset($options['fifo'])) {
      $pipe = str_replace('\\\\.\\pipe', '\\\\'.php_uname('n').'\\pipe', $options['fifo']);
      $ph = fopen($pipe, 'a+');
      fwrite($ph, "\xd4\xc3\xb2\xa1\x02\x00\x04\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00\x01\x00\x00\x00");
      $sock = socket_create(AF_INET, SOCK_DGRAM, 0);
      socket_bind($sock, '0.0.0.0' , 37008);
      while(true) {
        socket_recvfrom($sock, $buf, 10000, 0, $remote_ip, $remote_port);
        $size = strlen($buf);
        $t = explode(' ', microtime());
        $size -= 5;
        if(!fwrite($ph, pack('L', intval($t[1])).pack('L', intval(substr($t[0], 2))).pack('L', $size).pack('L', $size))) break;
        if(!fwrite($ph, substr($buf, 5))) break;
      }
      socket_close($sock);
    }
  }
?>
Once installed, it will add another interface to Wireshark:
tzsp1.png
If you select it and start capture, Wireshark will start the script and it will listen on udp/37008 (you may need to allow it in firewall) and will pass all received packets to Wireshark. Here is an example of router sending CDP packet (notice that it's the original packet, as was on the router, no TZSP wrapping):
tzsp2.png
And that's all folks. I don't plan to develop this further, because PHP script is a dead end, regular people don't have PHP installed and to get it only for this is unreasonable. Best solution would be to convince Wireshark guys to properly implement this. Or someone else to do it and send them a patch. Feel free to do any of that, I'm not very good with convincing someone and too lazy to write proper code. ;)

But that's not really all, there's one more thing. It seems that there might be a bug in RouterOS. The option to filter stream (to not capture TZSP packets sent by router) doesn't seem to always work. With small packets, everything is ok. But large packets don't fit in one TZSP packet and need to be sent fragmented. And then something like this happens, router captures own TZSP packets (it's captured using regular Wireshark capture on ethernet interface):
tzsp3.png
You do not have the required permissions to view the files attached to this post.
People who quote full posts should be spanked with ethernet cable. Some exceptions for multi-topic threads may apply.
 
McSee
Frequent Visitor
Frequent Visitor
Posts: 67
Joined: Tue Feb 26, 2019 12:49 pm

Re: /tool sniffer Code: 3 (Port unreachable)

Sat Apr 20, 2019 12:30 pm

Sob,

have you seen Mikrotik's very own Trafr utility?
( download link -- http://www.mikrotik.com/download/trafr.tgz )
Which is supposed to make proper ".pcap" from TZSP.

Then there is also Tzsp2pcap ( https://github.com/thefloweringash/tzsp2pcap ).
And it might be possible to remove extra TZSP bits from captured packets with pcap capture file editor Bittwiste
( http://manpages.ubuntu.com/manpages/bio ... ste.1.html )
 
Sob
Forum Guru
Forum Guru
Posts: 4812
Joined: Mon Apr 20, 2009 9:11 pm

Re: /tool sniffer Code: 3 (Port unreachable)

Sat Apr 20, 2019 6:10 pm

I vaguely remember trafr, but looking at it now, it's only Linux binary, so not useful for me on Windows. Tzsp2pcap looks like it can work under Cygwin, so that would be better. They both qualify as proper solution too. Bittwiste looks interesting, but probably not the right tool for this.

But to be honest, I currently prefer my experiment, regardless how ugly it's inside, because it's nicely integrated with Wireshark, just one click away. :)
People who quote full posts should be spanked with ethernet cable. Some exceptions for multi-topic threads may apply.
 
User avatar
hendry
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 60
Joined: Sat Jan 18, 2014 9:59 am
Location: Singapore
Contact:

Re: /tool sniffer Code: 3 (Port unreachable)

Mon Apr 22, 2019 10:47 am

I did manage to create a follow up video: https://www.youtube.com/watch?v=xAJlpSbbIhk

The suggestions here seem to require introducing some intermediate steps to get a nice session in Wireshark. Which is far from ideal.

Same goes in the Wireshark community's comments. https://ask.wireshark.org/question/8619 ... st-id-8635

Oh noes. Was really hoping I could recommend Mikrotik for this sniffing task, but now I'm not so sure.
RouterBOARD 4xRB952Ui-5ac2nD & 1xRB952Ui-5ac2nD
https://natalian.org/2017/08/20/Choosin ... _Ubiquiti/
 
Sob
Forum Guru
Forum Guru
Posts: 4812
Joined: Mon Apr 20, 2009 9:11 pm

Re: /tool sniffer Code: 3 (Port unreachable)

Mon Apr 22, 2019 9:55 pm

But it's not really MikroTik's fault (except that possible bug with fragmented packets, someone should look into that), it's more Wireshark not having the best possible default support for handling TZSP. And maybe it's ok too, because I'm not sure if TZSP is used that much.
People who quote full posts should be spanked with ethernet cable. Some exceptions for multi-topic threads may apply.
 
sindy
Forum Guru
Forum Guru
Posts: 4021
Joined: Mon Dec 04, 2017 9:19 pm

Re: /tool sniffer Code: 3 (Port unreachable)

Mon Apr 22, 2019 10:23 pm

I wonder whether you have tried to follow my advice from ask.wireshark.org - to exclude the interface via which the packets leave towards the wireshark machine from sniffing? I know that the way the /tool sniffer handles the filter is somehow weird, but I believe that even with filter-operator-between-entries set to and, you can set filter-interface to a list of interfaces rather than a single one (if you need it) and still get packets from all the interfaces on the list; however, in your case this should not even be necessary, as you are only interested in the traffic on the wireless interface, aren't you?

As for the rest - if some packets do not make it to the Wireshark capture while others do, the CPU of the Mikrotik is most likely overloaded while sniffing; if none makes it there (which is not your case), it may be some anti-virus software which is hooked closer to the wire than WinPcap/npcap.

Whether the PC sends back its icmp complaints about incoming TZSP packets may depend on a lot of things including firewall settings, so I wouldn't pay too much attention to the fact that your machine does that and some other ones don't.
Instead of writing novels, post /export hide-sensitive. Use find&replace in your favourite text editor to systematically replace all occurrences of each public IP address potentially identifying you by a distinctive pattern such as my.public.ip.1.

Who is online

Users browsing this forum: Google [Bot] and 25 guests