Logging web traffic to remote webserver

RouterOS General
1

I want to send web traffic log to a remote server listening on UDP 514 from a CPE and then save it with a properly software.
CPE config:
ip firewall filter rules

 5   ;;; default configuration
     chain=forward action=accept connection-state=established 

 6   ;;; default configuration
     chain=forward action=accept connection-state=related 

 7   chain=forward action=log tcp-flags=syn protocol=tcp log-prefix="clientlog>

system logging rules

 4    topics=web-proxy prefix="clientlog" action=remote

system logging action

 3 * name="remote" target=remote remote-port=514 src-address=0.0.0.0 
     bsd-syslog=no syslog-facility=daemon syslog-severity=auto 

 4   name="remotelog" target=remote remote=MY PUBLIC IP ADDRESS remote-port=514 
     src-address=0.0.0.0 bsd-syslog=no syslog-facility=daemon 
     syslog-severity=auto

port 514 on webserver firewall open
port 514 on modem firewall open

still no data received!
Something wrong with CPE config?

2

Are you using the buildin proxy-server? because topics=web-proxy selects the log from the buildin web-server not logs from the firewall

3

now I set topic to firewall, but no data yet :frowning:

4

First thing I see is that your logging rule says “remote” and not “remotelog”

5

You’re right . I changed to “remote” now but still no data.
Maybe it’s my network since is natt’d.
Should I open port 514 for each NAT node?

6

ok with topic=firewall now it works.
which is the different with web-proxy topic? Which is better to use?

7

the question is not what is better, but what is working with your actual configuration


remember:
for Italian law, as ISP, you must log:

all tcp and udp new connection with source and destionation ip and port and the exact time on GMT 00:00 and the username used.
and all of the other type of new connection (like ICMP, GRE, etc.)

and you MUST NOT LOG site and page visited for privacy LAW…

8

ok, sorry for my incompetence.
so what method do you think it’s better for logging customer traffic respecting italian privacy law?
I thought to log to a remote server (e.g syslog watcher) listening on udp 514 and saving all firewall topics.
so data captured are like this:

9

I use (and I still to use) Syslog Watcer Pro v2!!!

Put those rules at the end of firewall filter drop on forward rules (if you have any):

lista_ip_clienti = automatic list made by pppoe-server profile on “address list” field
this log only the client traffic, and not all the network traffic.

The first rule log TCP, UDP, ICMP, IGMP etc. new connection,
the second the RELATED connection for the first,
the other rule log the other protocols.

add action=log chain=forward comment="Invio Log connessioni in uscita al Server" connection-state=new src-address-list=lista_ip_clienti
add action=log chain=forward connection-state=related log-prefix=RELATED src-address-list=lista_ip_clienti
add action=log chain=forward comment="Tunnel IPv6 dentro IPv4" protocol=ipv6 src-address-list=lista_ip_clienti
add action=log chain=forward protocol=gre src-address-list=lista_ip_clienti
add action=log chain=forward protocol=ipsec-esp src-address-list=lista_ip_clienti

And configure accordingly system / logging


/system logging action
add bsd-syslog=yes name=LogServerNotice remote=1.1.1.1 syslog-facility=local1 syslog-severity=notice target=remote
add bsd-syslog=yes name=LogServerInfo remote=1.1.1.1 syslog-facility=local1 syslog-severity=info target=remote

/system logging
add action=LogServerNotice topics=pppoe,info,!debug
add action=LogServerInfo topics=firewall,info,!debug

Those rules not only log the firewall, but also the pppoe autentication.

ALL THIS RULE ARE VALID ONLY FOR “PPP” CLIENTS, for HotSpot the method is different…

For HotSpot I suggest you to use “Cloud4Wi”…

10

if users do not log in via ppoe, should I manually configure each client’s cpe for remote logging?

11

No, if you assign static public IP to client, you not need to log anything, each IP is exactly that customer :slight_smile:
You must only track each time you change the “owner” of IP address :slight_smile:

What method you use for login the CPE to Gateway?

any “ppp” method can be logged with that described on previous post, simply change pppoe in the log, with what you want.

12

by now I access cpe’s via routing.
the problem is cpe addresses are assigned by Access Point dhcp server.
so manual config I think is necessary..

13

You decide that yourself or other “ISP starter” company as suggested that to you?

It’s very hard log all easy with this method,
at this point the rule for logging must be put on access point, not on cpe.

14

why on the access point?
I configured a customer cpe to remote log on udp 514 to a pc into my lan and it seems to work as picture I posted in previous post.

15
  1. Because you use user traffic for log the user, and you also log the logging of the user… :wink:
  2. If you want change any, you change only the AP, not all the CPE…
16

multiple cpe’s logging is allowed?
lf I set user cpe for logging this will be heavy on user connection speed?

17

Yes.

Syslog use less space than “packet capture”…

18

Hi rextended, thanks for the conf.
What about hotspot loging configuration?

Thanks in advance, Simone.

19

after this rule:
/ip firewall nat
add action=passthrough chain=unused-hs-chain comment=“place hotspot rules here” disabled=yes

immediately after add this:
/ip firewall nat
add action=log chain=pre-hotspot

and this on system logging:

/system logging action
add bsd-syslog=yes name=LogServerNotice2 remote=1.1.1.1 syslog-facility=local2 syslog-severity=notice target=remote
add bsd-syslog=yes name=LogServerInfo2 remote=1.1.1.1 syslog-facility=local2 syslog-severity=info target=remote

/system logging
set 0 prefix=INFO topics=info,!firewall
add action=LogServerInfo2 topics=firewall,info,!debug
add action=LogServerNotice2 topics=hotspot,info,!debug
20

Hi thanks for the post. I’ll try it tomorrow.

What are the differences between this method?

[admin@Test-HotSpot] > ip firewall filter add chain=forward action=log connection-state=new
[admin@Test-HotSpot] > system logging add topics=firewall action=remote
[admin@Test-HotSpot] > system logging action set remote remote=[remote server ip]