Hello everybody, I hope you all doing well. I have a question to ask. I am using Mikro Tik Model - RB1100AHx2 , OS Level is 6 . How do I save or monitor my client’s browsing history by a remote computer? I have already downloaded MT_Syslog software on my pc. Waiting for a suitable answer. Gimme the easiest way please because I am just beginner, Thanks in advance.
You can log SYN packets:
/ip firewall filter add chain=forward action=log tcp-flags=syn protocol=tcp src-address=192.168.1.0/24 log=yes log-prefix="SYN-FORWARD"
and send log to remote machine:
/system logging set 1 action=remote
where 1 it’s a info level.
Define remote action
/system logging action add name="remote" target=remote remote=192.168.1.1 remote-port=514 src-address=192.168.1.2 bsd-syslog=no syslog-time-format=bsd-syslog syslog-facility=daemon syslog-severity=auto
where remote it’s a syslog server src-address - local interface
Then define rule on your syslog server:
# cat /etc/syslog-ng/conf.d/02-mikrotik.conf
source s_net { udp (); };
filter f_mikrotik_syn { host( "192.168.1.2" ); message("SYN-FORWARD");};
destination df_mikrotik_syn {
file("/home/mikrotik/syn/${YEAR}.${MONTH}.${DAY}.log");
};
log { source ( s_net ); filter( f_mikrotik_syn ); destination ( df_mikrotik_syn ); };
You can also rotate this logs:
#!/bin/bash
set -e
renice 19 $$ > /dev/null
BASE=/home/mikrotik/
DATA=`date -d "yesterday 13:00 " '+%Y.%m.%d'`
DIRS=(syn)
for i in ${DIRS[*]}; do
if [ \! -f ${BASE}${i}/${DATA}.log ]; then
echo "No logfile ${BASE}${i}/${DATA}.log"
exit 1;
fi
gzip -c -9 ${BASE}${i}/${DATA}.log > ${BASE}${i}/${DATA}.log.gz
chown mtlog:mtlog ${BASE}${i}/${DATA}.log.gz
rm ${BASE}${i}/${DATA}.log
done
/etc/init.d/syslog-ng reload | grep -v 'Reload system logging: syslog-ng.'
Great, thank you for the info!
I would like to point that this firewall rule need to be moved to the top in order for this info to be captured.
I have couple of simple questions:
- What is “syn” exactly? Does it capture ANY activity? Basically any TCP packets sent through router will be captured, right? HTTPS/VPN will be captured as well? I want to be able to tell if someone connects and uses encrypted VPN
- I get logs like this. From here I can tell how packets were flowing.
SYN-FORWARD forward: in:ether1-gateway out:bridge-local, src-mac a4:7a:a4:2e:af:50, proto TCP (SYN,ACK), 157.240.2.20:443->192.168.33.106:50785, NAT 157.240.2.20:443->(23.114.87.129:50785->192.168.33.106:50785), len 60
- I got all this info in SQL database. Now I can parse out all pieces. I know local IPs and MACs so it’s easy to detect local clients. For external ones, I need to do reverse DNS lookups, is that something easily done?
My logic to get history report is to collect data for a day, for example.
- Parse out every record, detect from/to
- Eliminate “duplicates” - for example by timestamp if there is multiple packets within a minute, I can just leave 1 row
- Get unique IPs and do reverse DNS lookups to see remote host names.
Then, for each individual internal host I will be able to display history.
Does this sound correct?
Not in all cases. Imagine an DOS from one of your clients.
In short - SYN it’s a init package, please look at three way hanhdshake http://www.inetdaemon.com/tutorials/internet/tcp/3-way_handshake.shtml. Any activity before two hosts starting from SYN package.
And it’s enough - all information about this connection are logged.
First - there is no duplicated entries. If you have two lines with the same content one by one - there was two connections. Second I think it’s a bad idea to make it on the fly. There is a really lot of information, in my case 18095665 lines, 3 GB log file (~ 20 hours, ~ 500 users). Third - revdns It’s useless in this case. For example in my case google.com point to:
$ host google.com
google.com has address 216.58.209.78
$ host 216.58.209.78
78.209.58.216.in-addr.arpa domain name pointer waw02s06-in-f14.1e100.net.
78.209.58.216.in-addr.arpa domain name pointer waw02s06-in-f78.1e100.net.
78.209.58.216.in-addr.arpa domain name pointer waw02s06-in-f14.1e100.net.
78.209.58.216.in-addr.arpa domain name pointer waw02s06-in-f78.1e100.net.
If you have an ip address just use whois. Store logs in file, and find needed informations when it’s necessary.
Where should I put it? At the end (where script put’s it) - there is nothing in log
First - there is no duplicated entries. If you have two lines with the same content one by one - there was two connections.
Good to know. But even if it’s 2 connections, why should I care? Browser can open multiple connections to download pictures, etc.
I think I should clean it up a little, tell me where I might be wrong?
Second I think it’s a bad idea to make it on the fly. There is a really lot of information, in my case 18095665 lines, 3 GB log file (~ 20 hours, ~ 500 users). Third - revdns It’s useless in this case. For If you have an ip address just use whois. Store logs in file, and find needed informations when it’s necessary.
Oh no, I’m not making it on a fly. I’m only parsing out pieces from a string and store them into DB. This part is done and yes, there is a lot!
Then, at the end of day I envision running some job to do post-processing. SQL script to get rid of duplicates. Then get DISTINCT external IPs and then do revdns on those. And I can also cache those in DB so I don’t have to do it all over every day.
What about whois? Is there any programmatic way to pull this data?
ALSO! I guess it’s important to make it even “better”. If HTTP traffic - it would be nice to collect actual URLs which is not visible from TCP packets logging. I get it with secured protocols we can’t but with regular HTTP traffic it would be nice what is being done.
In my rules right behind antiddos/limits/connlimit rules.
If you want to log all conenctions - yes you are wrong. If you only want to know about connection with some host - you can filter logs.
For example in linux systems you can do this:
$ whois 216.58.209.78
#
# ARIN WHOIS data and services are subject to the Terms of Use
# available at: https://www.arin.net/whois_tou.html
#
# If you see inaccuracies in the results, please report at
# https://www.arin.net/public/whoisinaccuracy/index.xhtml
#
#
# The following results may also be obtained via:
# https://whois.arin.net/rest/nets;q=216.58.209.78?showDetails=true&showARIN=false&showNonArinTopLevelNet=false&ext=netref2
#
NetRange: 216.58.192.0 - 216.58.223.255
CIDR: 216.58.192.0/19
NetName: GOOGLE
NetHandle: NET-216-58-192-0-1
Parent: NET216 (NET-216-0-0-0-0)
NetType: Direct Allocation
OriginAS: AS15169
Organization: Google Inc. (GOGL)
RegDate: 2012-01-27
Updated: 2012-01-27
Ref: https://whois.arin.net/rest/net/NET-216-58-192-0-1
OrgName: Google Inc.
OrgId: GOGL
Address: 1600 Amphitheatre Parkway
City: Mountain View
StateProv: CA
PostalCode: 94043
Country: US
RegDate: 2000-03-30
Updated: 2015-11-06
Ref: https://whois.arin.net/rest/org/GOGL
OrgTechHandle: ZG39-ARIN
OrgTechName: Google Inc
OrgTechPhone: +1-650-253-0000
OrgTechEmail: arin-contact@google.com
OrgTechRef: https://whois.arin.net/rest/poc/ZG39-ARIN
OrgAbuseHandle: ABUSE5250-ARIN
OrgAbuseName: Abuse
OrgAbusePhone: +1-650-253-0000
OrgAbuseEmail: network-abuse@google.com
OrgAbuseRef: https://whois.arin.net/rest/poc/ABUSE5250-ARIN
#
# ARIN WHOIS data and services are subject to the Terms of Use
# available at: https://www.arin.net/whois_tou.html
#
# If you see inaccuracies in the results, please report at
# https://www.arin.net/public/whoisinaccuracy/index.xhtml
#
If you just want to know about connections between your clients and some ip you can parse this logs. If you want log information about all connections - you must store this logs.