Community discussions

MikroTik App
 
jberen
Trainer
Trainer
Topic Author
Posts: 13
Joined: Mon May 14, 2007 9:16 pm
Location: Argentina - Monte Caseros
Contact:

Deep Packet Inspection (DPI)

Sat May 28, 2016 9:02 pm

is possible in Mikrotik impplement Deep Packet Inspection (DPI)?
 
TonyJr
Member Candidate
Member Candidate
Posts: 204
Joined: Sat Nov 12, 2011 1:30 am
Location: UK
Contact:

Re: Deep Packet Inspection (DPI)

Mon May 30, 2016 7:20 pm

RouterOS can perform Layer 7 inspection of packets.

TonyJr
 
jberen
Trainer
Trainer
Topic Author
Posts: 13
Joined: Mon May 14, 2007 9:16 pm
Location: Argentina - Monte Caseros
Contact:

Re: Deep Packet Inspection (DPI)

Mon May 30, 2016 8:56 pm

8) yes, but the regex in layer7.org is very Old.
We need the last regex for example youtube Facebook etc. Similar to DPI of ubiquiti
 
TonyJr
Member Candidate
Member Candidate
Posts: 204
Joined: Sat Nov 12, 2011 1:30 am
Location: UK
Contact:

Re: Deep Packet Inspection (DPI)

Tue May 31, 2016 12:41 am

There are many posts on the forum regarding facebook and youtube filtering. However they now mainly use HTTPS which leads you to a dead end. Please however keep this topic active and search for more as it would be beneficial for many providers to be able to shape this kind of traffic (even if the corps don't like it).

TonyJr
 
User avatar
ZeroByte
Forum Guru
Forum Guru
Posts: 4051
Joined: Wed May 11, 2011 6:08 pm

Re: Deep Packet Inspection (DPI)

Tue May 31, 2016 2:58 am

There's a new "domain lists" feature in RC version of ROS that might be useful for you if you want to policy route / block / packet-mark certain domains.

According to (Normis?) it updates the IP addresses in a domain whenever TTL expires.

EDIT: apparently the change is that you may now specify domain names in address lists, and that they are updated when TTL expires.
When given a spoon,
you should not cling to your fork.
The soup will get cold.
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 24749
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: Deep Packet Inspection (DPI)

Tue May 31, 2016 8:19 am

the RouterOS l7 feature has no "regex" included. You can write anything on your own. Inspect the packets with Wireshark and write your own regex.

Inspecting packets just to block facebook is a bad idea. Use address lists, the new domain address lists.
No answer to your question? How to write posts
 
jberen
Trainer
Trainer
Topic Author
Posts: 13
Joined: Mon May 14, 2007 9:16 pm
Location: Argentina - Monte Caseros
Contact:

Re: Deep Packet Inspection (DPI)

Tue May 31, 2016 1:43 pm

when will enable this new functionality in the address list ? In stable release, no RC, you can explain how to use the soft to make the regex ?, thanks
 
User avatar
ZeroByte
Forum Guru
Forum Guru
Posts: 4051
Joined: Wed May 11, 2011 6:08 pm

Re: Deep Packet Inspection (DPI)

Tue May 31, 2016 5:32 pm

when will enable this new functionality in the address list ? In stable release, no RC, you can explain how to use the soft to make the regex ?, thanks
http://wiki.mikrotik.com/wiki/Manual:IP/Firewall/L7

If you read and understand this article, you'll see that L7 regex is not going to work for SSL traffic because the payload is encrypted.

The primary purpose of L7 regex is to recognize "high-layer" protocols (such as ftp, http, smtp, pop3, irc, etc) when they're being used on un-standard ports, or for protocols that are commonly run on whatever port the admin feels like (teamspeak, ventrillo, irc, p2p-trackers, etc).

L7 regex rules can be used to filter websites, but it's a very crude method and with the modern trend to use SSL by default, this method will not be useful anymore. The most effective web filtering solutions involve DNS filtering (with services like OpenDNS) or configuring your own DNS server with RPZ (dns policy).

Using the new domain-name capability of the firewall address-list is another way to use DNS to dictate policy. I haven't tried it yet myself (I don't want to put RC on my own router, and haven't yet updated my sim's version of ROS) so I can't say exactly how it behaves, but I'm optimistic about the usefulness of this feature. Obviously, you'll want to wait at least until 6.36 is released before using this feature in production, but if you have a lab or a test router to play with, then you might like to give this new feature a try.
When given a spoon,
you should not cling to your fork.
The soup will get cold.
 
parham
newbie
Posts: 42
Joined: Sun Feb 15, 2015 11:35 pm

Re: Deep Packet Inspection (DPI)

Wed Jun 22, 2016 12:13 pm

the RouterOS l7 feature has no "regex" included. You can write anything on your own. Inspect the packets with Wireshark and write your own regex.

Inspecting  packets just to block facebook is a bad idea. Use address lists, the new domain address lists.
thanks for adding domain address list which is very helpful in 6.36, but as IT consultant company with over 50+ Mikrotik in client side Which works perfect, I have few client asking for user activity not for blocking just the are interested about what staff doing i can provide the some with Ubiquiti DPI or Meraki AP , but its be very helpful we having this feature in ROS7 which is I already requested.
 
pe1chl
Forum Guru
Forum Guru
Posts: 7050
Joined: Mon Jun 08, 2015 12:09 pm

Re: Deep Packet Inspection (DPI)

Wed Jun 22, 2016 1:48 pm

You will find that you are in a squeeze between company owners who want to know what happens on their
networks and/or block certain activities, and privacy advocates who want to make sure that such inspection is
impossible e.g. by using encryption everywhere.
 
User avatar
nz_monkey
Forum Guru
Forum Guru
Posts: 1896
Joined: Mon Jan 14, 2008 1:53 pm
Location: Over the Rainbow
Contact:

Re: Deep Packet Inspection (DPI)

Wed Jun 22, 2016 2:10 pm

thanks for adding domain address list which is very helpful in 6.36, but as IT consultant company with over 50+ Mikrotik in client side Which works perfect, I have few client asking for user activity not for blocking just the are interested about what staff doing i can provide the some with Ubiquiti DPI or Meraki AP , but its be very helpful we having this feature in ROS7 which is I already requested.
We do a lot of this sort of thing for enterprise clients.  To be honest you need a multi-pronged approach to this.  

You need a UTM/NGFW that can perform:
- SSL Man In The Middle (decryption mid stream, requires a CA and installation of your cert on client machines)
- DNS based identification and categorisation of traffic
- Heuristic/Signature based detection (often part of IPS suite)

Examples of devices that can accurately do this are Palo Alto Networks, Fortigate and Sophos XG.   

In my opinion you will be fighting a losing battle doing this sort of thing, while doing SSL MITM inspection works for now, as technologies like SQRL become more widely adopted performing MITM will become near impossible, and you will need to rely on heuristic/DNS type matching which are not as accurate.
http://thebrotherswisp.com/ | Mikrotik MTCNA, MTCRE, MTCINE | Fortinet NSE7 | Extreme Networks ENA
 
parham
newbie
Posts: 42
Joined: Sun Feb 15, 2015 11:35 pm

Re: Deep Packet Inspection (DPI)

Wed Jun 22, 2016 4:11 pm

thanks for adding domain address list which is very helpful in 6.36, but as IT consultant company with over 50+ Mikrotik in client side Which works perfect, I have few client asking for user activity not for blocking just the are interested about what staff doing i can provide the some with Ubiquiti DPI or Meraki AP , but its be very helpful we having this feature in ROS7 which is I already requested.
We do a lot of this sort of thing for enterprise clients.  To be honest you need a multi-pronged approach to this.  

You need a UTM/NGFW that can perform:
- SSL Man In The Middle (decryption mid stream, requires a CA and installation of your cert on client machines)
- DNS based identification and categorisation of traffic
- Heuristic/Signature based detection (often part of IPS suite)

Examples of devices that can accurately do this are Palo Alto Networks, Fortigate and Sophos XG.   

In my opinion you will be fighting a losing battle doing this sort of thing, while doing SSL MITM inspection works for now, as technologies like SQRL become more widely adopted performing MITM will become near impossible, and you will need to rely on heuristic/DNS type matching which are not as accurate.
Thanks Mate appreciate, we did use the Solarwins, and some others Net Flow analyzers which wasn't great but just ok, rather to mikrotik find a solution and do something  similar to Fortigate or Meraki or Ubiguiti, I worked with Mikrotik for more than 6 year now, all clear to me also I know Fortigate and Cisco  never play with Sophos, since 3 years ago Firewalls features are very changed now you are able to for more thing like virus detect, Malicious, DPI, and we will see more in few years. I really wish Mikrotik do the same thing in new ROS 7.
 
pe1chl
Forum Guru
Forum Guru
Posts: 7050
Joined: Mon Jun 08, 2015 12:09 pm

Re: Deep Packet Inspection (DPI)

Wed Jun 22, 2016 5:03 pm

You need a UTM/NGFW that can perform:
- SSL Man In The Middle (decryption mid stream, requires a CA and installation of your cert on client machines)
How long until that finally stops working?
I expect this solution to be dead in a year or two...
 
User avatar
harry66
newbie
Posts: 48
Joined: Tue Mar 04, 2014 5:29 pm
Location: Germany

Re: Deep Packet Inspection (DPI)

Fri Jun 24, 2016 11:46 am

Is that really the right approach to say that it will no longer work in two years? It is about protecting information from threats we are facing today. Now.
It does not matter at all if this technology is outdated in two years. This is a rapidly developing market and you have to have effective measures now.

The effective measure today is to decrypt SSL on L7 and analyze and filter what's going on there. You need to have a kind of pattern matcher in your communication to filter malicious code. You have to limit the number of applications used on L7. You have to make sure Port 80 is used for http-traffic, 443 for https traffic and DNS for DNS and so on. Everything else is not effective.

I would love if Mtik would spend a thought on prividing this functionality in their products. ARM processing power comes to ranges that allow for such. RB3011 and CCR Tile based platforms should be able to handle a virus DB and a hash based pattern matching. Just hook up to some bitdefender or avira malware pattern base and provide a rocking service.

Honestly I hate to deploy another machine (In my case it is Untangle) to pipe my traffic through to get it inspected for Malware.

@Mikrotik: Do you have plans for further developing RouterOS? Are you going to provide maybe a dedicted appliance? Will you give recommendations on how to peer up with existing technology from other vendor? It would be a worse idea to simply ignore what we see as real life threats...

Hope for an answer. Would like to see a timeline. I have a lot of Mtik devices in the field, but have the impression they are not really prepared for the job.

/Uwe
Here I could list the estate, but who cares?
 
pe1chl
Forum Guru
Forum Guru
Posts: 7050
Joined: Mon Jun 08, 2015 12:09 pm

Re: Deep Packet Inspection (DPI)

Fri Jun 24, 2016 11:57 am

Is that really the right approach to say that it will no longer work in two years? It is about protecting information from threats we are facing today. Now.
It does not matter at all if this technology is outdated in two years. This is a rapidly developing market and you have to have effective measures now.
That "two years" is just a guess when everyone will have deployed counter-measures against it.
But even today the Google Chrome browser detects many of those "solutions", certainly when used on wellknown services.
In the long run it will no longer be accepted to spoof the certificate, and of course that is right.
Protecting your information using pattern matchers is not the best idea anyway, it is much better to install some policies
on your endsystems that protect them against execution of unwanted software (e.g. AppLocker on Windows).  At least that
works against malware that is not yet in the pattern database, a very common situation today.
 
User avatar
harry66
newbie
Posts: 48
Joined: Tue Mar 04, 2014 5:29 pm
Location: Germany

Re: Deep Packet Inspection (DPI)

Fri Jun 24, 2016 12:29 pm

That approach does not take into account, that
  • 90% of your infections come via one single application that you can't control this way: The internet browser
  • There are a lot of systems in the field that are old and/or can't be protected by endpoint protection. (Windows XP, UNIXes e.g.)
Do you have an alternative attempt to protect from cryptolocking malware instead of filtering like the above? Of course it is not ideal to spoof certificates and do intended man in the middle attacks.
As soon as you talk about technology you talk about cost. Doing things centrally is always the most cost efficient solution from investment and from man hours point of view.
Everything else is barely practical or at least only practical in special cases like an up to date Windows/AD environment. But this is not the case in real world.

Don't get me wrong, I am open for discussions. I don't see a way to get around central filtering and policy implementation/enforcement on proxy level. RouterOS capabilities are barely developed in this area, there is lots of room for improvement. And yes, keeping databases up to date if an enormous effort. That's why I would at least expect a proposal to peer with other systems like untangle for example. At the same time I don't see so much higher effort to incorporate this function to RouterOS. Untangle simply buys in a bitdefender service. Sophos buys in a Avira service. All no problem.
Here I could list the estate, but who cares?
 
pe1chl
Forum Guru
Forum Guru
Posts: 7050
Joined: Mon Jun 08, 2015 12:09 pm

Re: Deep Packet Inspection (DPI)

Fri Jun 24, 2016 3:27 pm

I don't consider peeking in encrypted network traffic a solution to the problem that you are facing.
Even with peeking in place you will be passing malware to the user because the pattern is not uptodate.

Solutions that do work:
- let the users work as an ordinary user, not a power user or local administrator
- implement AppLocker policies that only allow software execution from wellknown directories like C:\Windows and C:\Program Files,
  and not from C:\Users (C:\Documents and Settings)
  rule of thumb: execution of software should not be allowed from directories where users can write their data.

It can also help to set the explorer to show extensions for filenames (not the stupid default to hide them).
 
User avatar
harry66
newbie
Posts: 48
Joined: Tue Mar 04, 2014 5:29 pm
Location: Germany

Re: Deep Packet Inspection (DPI)

Fri Jun 24, 2016 4:34 pm

You are talking about the threat situation 5 years ago. You completely neglect the scenarios I described.

Read this and understand it:
https://nakedsecurity.sophos.com/2016/0 ... -required/

/Uwe
Here I could list the estate, but who cares?
 
User avatar
ZeroByte
Forum Guru
Forum Guru
Posts: 4051
Joined: Wed May 11, 2011 6:08 pm

Re: Deep Packet Inspection (DPI)

Fri Jun 24, 2016 6:35 pm

I personally think that DNS-based filtering is going to grow in importance, as threat vectors must ultimately rely on either DNS or a pre-computed hash of IP addresses to communicate with.
As threats are captured and analyzed by security professionals, the pre-computed IP hashes are susceptible to blacklisting. Domain names are much more flexible for the bad actors, and so closing this avenue for "finding home" is key. If the malware cannot find the IP address of its mothership, (or the phishing sandbox website, etc) then the threat is neutralized.

I guess I'm just not looking for a DPI service in Mikrotik for a few reasons.
Firstly, its primary function is to be a router, not a firewall. The firewall rules are quite flexible and allow lots of creative, even ill-advised configurations that perform some task or other. However, the core value of ROS is its ability to be a router, and there are currently many things about this segment which still need work, features which need to be updated, etc. IPv6 functionality is present but very basic at the moment. Failing to keep up with this is going to be a form of creeping death for ROS as a routing platform as adoption progresses.

Secondly (and this is NOT a slam against Mikrotik), I don't think the company has the resources to branch off into yet another highly-specialized field like packet inspecting security appliances. Take CapsMan for instance - it's a wonderful feature, and granted I don't have much experience with it, but it just doesn't seem as evolved and functional as other WiFi controller platforms that have been on the market maturing for several years. It's a Mikrotik solution and allows for an "all-in-one" solution, but it's not going to be as functional/"featureful" as a purpose-built solution.

The ROS software itself is well-known in this community to be easily affected by bugs whenever a new version comes out (hence the bugfix train by Mikrotik - kudos on that, by the way).
It took years and years before development resumed on The Dude - Mikrotik's NMS solution, and my impression from the various threads is that this was due to having no one in-house to work the project.
Mikrotik products are known to have many rough edges when they're new (the 2011 had some issues, the 3011 is becoming more reliable after a year or so, I seem to recall there being tilera platform issues when the CCR line was new, etc)

All of this paints a picture, and if Mikrotik were to roll out a DPI solution for RouterOS (a module, perhaps), then would you really want to trust your network security to a greenhorn product line from Mikrotik? For a home network, sure, or for a student computer lab / guest network / etc - that might be great. This could be a very useful ROS module, and after a few years of maturation, might be a very dependable function for ROS to boast.

Finally, speaking for myself, I am not a fan of "all-in-one" solutions. In my experience, such things either offer lots of B+ quality capabilities (nothing truly excellent) or else they have performance/scaling problems whenever any significant portion of its capabilities are actually put into use at once. (can you imagine actually using a mAP lite as a MPLS PE-router? the feature is available on it). I would rather plug a great firewall into a great router than sit one decent box in front of the network... and this holds doubly-true for anything as involved as security, because a false sense of security is worse (in some ways) than no security at all.

Again, this isn't a slam against Mikrotik - they're doing great things with this product, and what it does well, it does fantastically well for a great price, but they're not a pancea solution. I would much rather they spend their energy on keeping the router functionality as tip-top as possible than spread themselves even thinner into true firewall development.
When given a spoon,
you should not cling to your fork.
The soup will get cold.
 
pe1chl
Forum Guru
Forum Guru
Posts: 7050
Joined: Mon Jun 08, 2015 12:09 pm

Re: Deep Packet Inspection (DPI)

Fri Jun 24, 2016 8:22 pm

You are talking about the threat situation 5 years ago. You completely neglect the scenarios I described.

Read this and understand it:
https://nakedsecurity.sophos.com/2016/0 ... -required/

/Uwe
You have apparently not understood what I suggested.
AppLocker specifies the extensions to be blocked as executable and obviously .JS is one of them.
The user clicking on a .js link results in downloading to %TEMP% by the browser then calling the OS to execute
it and BOOM the execution of this file is prohibited by policy.   Works perfectly.
 
User avatar
ZeroByte
Forum Guru
Forum Guru
Posts: 4051
Joined: Wed May 11, 2011 6:08 pm

Re: Deep Packet Inspection (DPI)

Fri Jun 24, 2016 8:26 pm

If security's a big concern, then it would be wise to implement this in addition to having a DPI firewall anyway.
Depth of defense = ++good
When given a spoon,
you should not cling to your fork.
The soup will get cold.
 
User avatar
harry66
newbie
Posts: 48
Joined: Tue Mar 04, 2014 5:29 pm
Location: Germany

Re: Deep Packet Inspection (DPI)

Mon Jun 27, 2016 9:46 am

You have apparently not understood what I suggested.
AppLocker specifies the extensions to be blocked as executable and obviously .JS is one of them.
The user clicking on a .js link results in downloading to %TEMP% by the browser then calling the OS to execute
it and BOOM the execution of this file is prohibited by policy.   Works perfectly.
I understood this, sure. Most operating systems like OSX and UNIX support this out of the box. Windows has a slight problem with it since everyone not being "Administrator" is seriously disabled in his work.
Let's see what the enhanced DNS filtering option for domains will allow in the upcoming RouterOS release. Until then I have implemented (and can recommend) the following two measures:
  1. Implement domain filtering in the built in web proxy based on common blacklists - guarantees at least ad-free web pages
  2. Tunnel traffic through another appliance like http://www.untangle.com in "bridge mode" to allow for all further traffic filtering and inspection
For the latter I would love to see a sample setup from Mikrotik that explains
  • How to implement a rule set to selectively tunnel traffic through a transparent device that is connected to two ports (in, return) on the Mikrotik router
  • traffic could then be sent through an external filter based on protocol, source or destination or what ever packet mark would be favorite
Other approaches that come to my mind were
  • Implement ICAP to interface a dedicated Antivirus solution that most companies already might have in house
  • Implement support for common blacklist importing. Scripting possibilities on RouterOS are currently too limited to allow for on board processing of existing lists. Mikrotik couls make it a service and provide blacklists ready for import.
Just some innovative ideas to enhance the products capabilities and make it worth much more.
Here I could list the estate, but who cares?
 
int21
just joined
Posts: 19
Joined: Thu Apr 16, 2009 1:31 am

Re: Deep Packet Inspection (DPI)

Mon Nov 07, 2016 4:29 pm

Address is a bad idea, youtube for example use proxy instaled in ISP (provide by google), so, the block in BGP is a waste of time!
 
User avatar
Thor187
newbie
Posts: 49
Joined: Sat Oct 21, 2017 10:21 pm

Re: Deep Packet Inspection (DPI)

Thu Feb 27, 2020 6:29 pm

You need a UTM/NGFW that can perform:
- SSL Man In The Middle (decryption mid stream, requires a CA and installation of your cert on client machines)
How long until that finally stops working?
I expect this solution to be dead in a year or two...
four years on, stil strong.

Who is online

Users browsing this forum: Baidu [Spider] and 80 guests