Hello,
Due to some new corporate cybersecurity policies, my IP Firewall Address List has over 50,000 entries. Many of these entries are address ranges.
What I’d like to do is to find a way to search the list to discover if a particular IP is covered by the range. I can do a manual comparison if I can filter the list down to something manageable.
I’ve tried using the filter feature in winbox. When I search the address field using either “in” or contains" I can only seem to get a search result if I give an exact match for the entire IP address including the CIDR subnet. Is there a way to do a partial match, for instance on the first two octets?
If the filter feature cannot meet my search needs, is there another search method I can use in the router? Perhaps something in the CLI?
I suppose I can export the list and open it in a word processor, I’m just curious if there is a method I can use directly on the router.
Thank you.
Have you used the in operator the right way?
[me@myTik] > ip firewall address-list add list=test address=193.168.0.0/16
[me@myTik] > ip firewall address-list print where 193.168.1.2 in address
Flags: X - disabled, D - dynamic
# LIST ADDRESS CREATION-TIME TIMEOUT
0 test 193.168.0.0/16 mar/24/2021 22:21:17
Hello Sindy,
Thank you for the reply!
I tested your code below. The CLI code works great, and I can search that way. However, when I put the exact same info into the filter in the Winbox GUI under Address Lists, it fails to find anything. Is the GUI supposed to work just like the CLI?
The attached picture shows what I mean, I used your code to create and search for an address list, and I can find it via CLI but not via Window GUI filter.
You’ve solved my issue, since I can search using the CLI. Mostly now I’m wondering if I found a bug in the way the filter works in Winbox that I need to report.
I appreciate your assistance.
Looks like that. Two possible interpretations of the contains operator in Winbox come to my mind:
- a regular expression matching of the column value interpreted as text, which on command line would look like address~“1.2.3”
- a way to simplify the graphical representation of the constant in column expression, allowing to keep the column at the left side and the constant at the right side of the form
None of these works; however, since actually the contains and in operators work exactly the same way in Winbox (in your example, if you fill the constant field with 1.2.0.0/16, the 1.2.3.0/24 item is shown no matter whether you choose in or contains as the operator), I assume the “inverse in” was the designer’s idea, but the coder forgot to swap the roles of the operands.
So yes, your next step should be to raise a support ticket at https://help.mikrotik.com/servicedesk/servicedesk/ or by e-mail to support@mikrotik.com . You can refer to this forum topic.
Thank you, Sindy.
I’ve opened a support ticket bug notification.
Its not so much a bug as it is just a missing feature.
The filter list in winbox gui only allows the order of the command one way - set out by the way the fields are set out
i.e. “where address in 1.2.3.4”
vs the opposite way that you actually want
“where 1.2.3.4 in address”
So this is likely more a feature request to change how the GUI handles creating filter statements to apply to the shown list.
I just want to borrow the thread, how has this affected performance? How much resources do you need for such lists? Im looking to use similar numbers in my firewalls.
@OlofL
I have not noticed a significant resource drain using large address lists. This device is a cloud router x86 architecture with 1 GB of memory and a 3.0 GHz processor. The standard load on the device is around 15% CPU usage and 950 MiB of Free memory.
There is a noticeable delay when using winbox to view such a large address list. It can take 30 to 60 seconds to populate. Managing such lists requires learning how to search and filter in winbox, hence my original question. But I have not noticed any significant delay in the routing. In fact, I now run this list on a least 3 Mikrotik routers to no ill effect.
The list being displayed in Winbox is only done for you. RouterOS most likely uses hashes to handle traffic which is much much faster.
Searching an address take some time and because it takes time you rather use foreach so you can make the change directly on find. Avoiding so a second loop to implement the change. That is 100% faster.
A foreach uses a .id which is pointer to that record so you can use it to make changes instantly.
Unless I’m getting this wrong, I think the filter function in WinBox is broken (WinBox v6.49.13 here). I have a client whose MAC ID is 11:22:33:44:55:66; then, in DHCP Server > Leases, the following filters produce weird results:
[MAC Address] [In] [11:22:33:44] matches “11:22:33:44:55:66”
[MAC Address] [In] [33:44:55:66] does not match “11:22:33:44:55:66”
Finding the address takes some time. Since this is time consuming, it’s better to use a foreach so that you can make changes directly during the search. This avoids a second cycle of implementing changes. This is 100% faster.
I’m a complete noob, how would I use foreach to print a list of all DHCP entries with a MAC ID containing “5D:44:1A”?
That actually consists of two steps - the first one is to obtain such a list and the next one is to print it using foreach, but the second step only makes sense if you actually want to do something else with the items than just print them.
So to obtain the list, you would use /ip dhcp-server lease find where mac-address~“5D:44:1A” . Now to just print these leases without a foreach cycle, you can use just /ip dhcp-server lease print where mac-address~“5D:44:1A”; to use the list obtained by the former command for something else, using the print only as an example, you can use the following:
/ip dhcp-server lease {
:foreach lease in=[find where mac-address~"5D:44:1A"] do={
print where .id=$lease
}
}
But the above makes sense for relatively short lists - as others wrote before, for long lists, it may make sense to use a raw (unfiltered) list for the foreach and use filtering per item; however, printing in particular is not well suited to be used in a foreach cycle.
/ip dhcp-server lease {
:foreach lease in=[find] do={
:if ([get $lease mac-address]~"5D:44:1A") do={print where .id=$lease}
}
}