I’m looking for a RouterOs tool to see and possibly save as a log, all http requests, such as visited websites, of my clients CPE’s.
I took a look into manual but I can’t find anything yet!
thanks in advance
If you wanted to just look at the current traffic, and the IP of the web server the clients are visiting, you can just use “/tool torch”.
If you want to see the contents and store (some of?) it, it’s a little more complicated. I’d go for setting up a web server as a “transparent proxy”:
Set up a web server with a server language such as PHP. Then make it make a request to whatever the HTTP request dictates with $_SERVER[‘HTTP_HOST’], $_SERVER[‘REQUEST_URI’] and so on, and output whatever the remote site outputs, with headers and everything. You can then store the request and/or response on the web server. At the router, in the “/ip firewall nat” menu, create a rule that does “dst-nat” to the web server whenever the client’s dst-port is 80 (i.e. HTTP’s default port). If you want to match HTTP content on an arbitrary port, you’d have to set up a layer7 match, but that will make the connection even more drastically slow than the transparent proxy already does.
This approach also allows you to potentially alter the site on-the-fly BTW (but if you do that, do it with care, because it’s very performance costly).
P.S. In case it’s not obvious, you can’t capture HTTPS like that… maybe capture the encrypted traffic, but there’s little use for that.
Uhm you explained everything very clearly.
As I see, it’s very performance costly.
And if I set up an external webserver and then install on it a network protocol analyzer? Without compromising CPE performance?
thanks!
If the protocol analyzer has an option to store per-connection data, sure. Otherwise, you may need to create a custom one.
To make sure that you don’t miss any traffic, you should connect the router and web server on the same switch, on the side of the local network. With this approach, you wouldn’t be able to alter or deny any incoming/outgoing data, though from what I can see, you don’t want to do that anyway.
yes, it can store per-ip address data.
you suggest to put webserver into the same LAN of CPEs?For avoid altering in/out data?
you suggest to put webserver into the same LAN of CPEs?
If you’re going to use a protocol analyzer, it MUST be in the same LAN as the RouterOS router, yes.
If you place it in the same LAN as your public facing interface, you’ll capture traffic that has already been processed by the router, and you wouldn’t be able to tell MAC addresses of CPEs (and if your network is NAT-ted, you wouldn’t be able to tell apart IPs either).
If you instead place it in the same LAN as your private facing interface (where the CPEs are), you’d be able to capture all traffic - allowed or merely attempted - as well as the router’s response to it, which means a more complete data set. You’d also be able to store MACs too (and if the network is NAT-ted, keep private IPs too).
It doesn’t “need” to be on the same switch, but depending on topology, and whether any of your switches are “smart”, not connecting them on the same switch might result in some connections being missed.
For avoid altering in/out data?
With a protocol analyzer, you can’t alter in/out data, even if you wanted to.
If you place it in the same LAN as your public facing interface, you’ll capture traffic that has already been processed by the router, and you wouldn’t be able to tell MAC addresses of CPEs (and if your network is NAT-ted, you wouldn’t be able to tell apart IPs either).
If you instead place it in the same LAN as your private facing interface (where the CPEs are), you’d be able to capture all traffic - allowed or merely attempted - as well as the router’s response to it, which means a more complete data set. You’d also be able to store MACs too (and if the network is NAT-ted, keep private IPs too).
It doesn’t “need” to be on the same switch, but depending on topology, and whether any of your switches are “smart”, not connecting them on the same switch might result in some connections being missed.
Tell me if if my understanding is correct, I need one webserver for each CPE/client?
thanks boen_robot
No.
Just one server that’s “behind” your router, on EQUAL ground with all CPEs, like this:
this is my actual and simplified netword map:
To sniff traffic you would need a switch which can mirror traffic from the monitored port over to the port that the analyser is attached to.
What do you mean for “mirror”?
Sent from my iPhone using Tapatalk - now Free
Basically, use a switch which is NOT a “smart” one. If you want to be 100% sure, use a “Hub” instead of “Switch”, since a “Hub” is by design always “mirror”-ing traffic.
To “mirror” basically means to send the traffic to all of the switch’s ports, as opposed to just the designated one. Many switches maintain a list of MAC addresses and a corresponding port the MAC is supposed to respond from. When a packet is intended for one already in the list, they send it only over its port, and not on other ones. That is, unless the device doesn’t respond in a certain time interval, at which point they send it over all ports again. This is good for performance, bad for capturing traffic.
Some smart switches have the ability (via a web interface or whatever) to turn off this feature for a particular MAC address. If you can’t find a hub, your best bet is one such switch, on which you can blacklist your router’s private facing interface, so that messages intended for the router always reach the server also.
As far as your topology goes - attach it to the “switch” in your diagram.
so if I use a smart switch and place it into “switch” place on my network I could monitor and view all my clients CPEs traffic?
I need just this?Nothing else? Something like wireshark or similar?
what mikrotik switch can I buy?
Rb260gs has port mirror. You will need wireshark an filter port 80 on the capture.
is it better an ethernet hub?
how can I search for a mirror switch? What’s specific name of mirroring in product specs?
Yes, as noted, a hub or a smart switch with mirroring is all you need… in terms of extra hardware. In terms of software, Wireshark or similar, to actually capture the incoming traffic for later analysis.
“Port Mirroring” seems to be the feature you need. It is supported by SwOS, and all MikroTik smart switches by extension.
If you look at SwOS’ manual page, you’ll notice in the “Forwarding” tab, the “Port Mirroring” section. You must simply set the mirror target on the port where your web server is connected, and enable mirroring from all other ports, so that their traffic will always reach the web server, even if the traffic is not intended for it.
yeah, now it’s all clear.
Can I save data being captured?
Port mirroring is the only practical way to go since true hubs are difficult to find and generally are too slow for the Ethernet speeds that are currently in use. In many devices offering port mirroring you may be limited in the throughput that can be monitored. e.g. don’t expect to monitor a saturated 1 Gbps connection in both directions since that would be 2 Gbps traffic and the mirror port can only receive 1 Gbps…
You should also check that what you are proposing is legal in your jurisdiction.
mirroring will work also into a NAT network?
mirroring is separate from ip configuration?