Community discussions

MikroTik App
 
User avatar
omega-00
Forum Guru
Forum Guru
Topic Author
Posts: 1167
Joined: Sat Jun 06, 2009 4:54 am
Location: Australia
Contact:

address-list script timeout

Thu Jun 10, 2010 1:47 pm

Hey Guys,

Just wondering if there's any way to set an address-list timeout via a script?
Seems as thou you can do it fine via the firewall rules but there's no option to do so in the add command (via terminal or script)

Regards,
Omega-00
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8383
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: address-list script timeout

Thu Jun 10, 2010 5:57 pm

yup, joining the feature request: possibility to add (at least via API, maybe scripting also =) ) dynamic address list items (I'd like not to write them on the disk, my program will recreate them in a minute even in case of reboot :) ). also I can't find a way to see actual timeout of the entry...
Russian-speaking forum: https://forum.mikrotik.by/. Welcome!

For every complex problem, there is a solution that is simple, neat, and wrong.

MikroTik. Your life. Your routing.
 
User avatar
janisk
MikroTik Support
MikroTik Support
Posts: 6283
Joined: Tue Feb 14, 2006 9:46 am
Location: Riga, Latvia

Re: address-list script timeout

Fri Jun 11, 2010 8:27 am

it would be nice if you both could give some examples where to use these static/dynamic entries that will time out, where you cannot remove them when necessary after some time.
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8383
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: address-list script timeout

Fri Jun 11, 2010 1:47 pm

actually, I'd like to see the possibility to create dynamic entries, even without timeouts. because we use USB flash sticks to bootup our routers, and I don't want to write all address lists changes to it. even if router will be rebooted (maybe once-twice a year), our management software will populate those lists in a few minutes, and that downtime isn't worth everyday writes to the flash

p.s. about timeouts - I'd like to see, how long ago the user was added to the 'blocked' list )))
Russian-speaking forum: https://forum.mikrotik.by/. Welcome!

For every complex problem, there is a solution that is simple, neat, and wrong.

MikroTik. Your life. Your routing.
 
User avatar
janisk
MikroTik Support
MikroTik Support
Posts: 6283
Joined: Tue Feb 14, 2006 9:46 am
Location: Riga, Latvia

Re: address-list script timeout

Fri Jun 11, 2010 4:33 pm

as a workaround you can add entry and some comment on it, to remove it later by script.
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8383
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: address-list script timeout

Fri Jun 11, 2010 7:46 pm

entries with timeouts are added via firewall rules... so no way =)
Russian-speaking forum: https://forum.mikrotik.by/. Welcome!

For every complex problem, there is a solution that is simple, neat, and wrong.

MikroTik. Your life. Your routing.
 
User avatar
omega-00
Forum Guru
Forum Guru
Topic Author
Posts: 1167
Joined: Sat Jun 06, 2009 4:54 am
Location: Australia
Contact:

Re: address-list script timeout

Sat Jun 12, 2010 8:49 am

Hey Janis,

Are you asking for specific cases that would require it?

If so: http://forum.mikrotik.com/viewtopic.php?f=9&t=42430 (Wow this guy is awesome, what a great script right? :-D)

Would be really helpful if I could assign dynamic address list entries for *any* ipv6 addresses seen locally by the router.
In this case I'm already using the comment field to store the mac address however so I'll work on storing a time value in there along with it and scrape the time data from it like that periodically.

Your point is valid however wouldn't it make just as much sense to use the pre-existing ability to remove address-list entries rather than running / scheduling my own for everything? :-)

Side notes re IPv6 address list:
[admin@Router] > ipv6 firewall address-list print 
Flags: X - disabled, D - dynamic 
 #   LIST                                                                                                                      ADDRESS                                    
 0   test                                                                                                                      2001:470:8902:1::2/128                     
[admin@Router] > ip firewall address-list print 
Flags: X - disabled, D - dynamic 
 #   LIST                                                                                                                                  ADDRESS                        
 0 D ssh_blacklist                                                                                                                         94.102.9.202 
Will the ipv6 address-list continue to have the mask applied to single addresses?
Just noticed it when writing the above script so when comparing addresses to the address list I have to include the /128 in my code explicitly to match them :-)
It also would appear that the new ipv6 address-list doesn't support ranges, will this be added in future?
 
User avatar
janisk
MikroTik Support
MikroTik Support
Posts: 6283
Joined: Tue Feb 14, 2006 9:46 am
Location: Riga, Latvia

Re: address-list script timeout

Mon Jun 14, 2010 11:18 am

using ranges in IPv6 does not make much sense, because for customers you will use advertise most of the time, and address will be EUI64 standard, so what ranges you will enter there?, you can simply use networks of /64 addresses, as that is what you have to assign to one customer.
 
User avatar
janisk
MikroTik Support
MikroTik Support
Posts: 6283
Joined: Tue Feb 14, 2006 9:46 am
Location: Riga, Latvia

Re: address-list script timeout

Mon Jun 14, 2010 11:24 am

and back to adding addresses to the list - there is one weird way how to add dynamic address-list entries:
* create management port (tunnel or actual interface)
* for this interface add firewall rule, that will catch some interesting traffic, like (in pseudocode) add packets dst-address to list if src-address equals to some value


for this to work you will have to create packets, that are sent over this link to router. That way you can create quite extensive address lists in very short time, and depending on your firewall rule settings you can fill several address lists. with different time settings.
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8383
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: address-list script timeout

Mon Jun 14, 2010 11:35 am

yeah, I was thinking about such setup for making a NET where client gets the internet directly, but must be authorized via some agent program running on his computer ))

then one more feature request - 'action=remove-from-address-list' :D

but it would be much nicer if we can do it via API...
Russian-speaking forum: https://forum.mikrotik.by/. Welcome!

For every complex problem, there is a solution that is simple, neat, and wrong.

MikroTik. Your life. Your routing.
 
User avatar
janisk
MikroTik Support
MikroTik Support
Posts: 6283
Joined: Tue Feb 14, 2006 9:46 am
Location: Riga, Latvia

Re: address-list script timeout

Mon Jun 14, 2010 11:58 am

well, problem with remove is that is much much more expensive than addition and you want your firewall to work superfast. And here lays the problem, adding dynamic remove will slow things down and if used improperly, which it will, it will cause more grief than benefit anyone.

That is why you can somehow work around and add dynamic entries through firewall with time-out, so entries "remove themselves" and removal with some fancy script, that you run once in a minute. Thus maintaining approximately what is needed and still have fast firewall.
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8383
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: address-list script timeout

Mon Jun 14, 2010 12:47 pm

I'm not sure, what will be more expensive: either deleting 1 entry per minute, or "refreshing" 1000 entries per minute %)

anyway, it's fun, not more. can we expect kind of /ip firewall address-list add dynamic=yes? is it so hard not to write changes to the disk? =)
Russian-speaking forum: https://forum.mikrotik.by/. Welcome!

For every complex problem, there is a solution that is simple, neat, and wrong.

MikroTik. Your life. Your routing.
 
User avatar
omega-00
Forum Guru
Forum Guru
Topic Author
Posts: 1167
Joined: Sat Jun 06, 2009 4:54 am
Location: Australia
Contact:

Re: address-list script timeout

Mon Jun 14, 2010 7:33 pm

using ranges in IPv6 does not make much sense, because for customers you will use advertise most of the time, and address will be EUI64 standard, so what ranges you will enter there?, you can simply use networks of /64 addresses, as that is what you have to assign to one customer.
Good point; will the new IPv6 hotspot will have the ability to assign dynamic prefixes (eg a /64 range) per user?
 
dssmiktik
Forum Veteran
Forum Veteran
Posts: 732
Joined: Fri Aug 17, 2007 8:42 am

Re: address-list script timeout

Mon Jun 14, 2010 8:18 pm

If an added feature to the firewall may be used improperly, you might put a big warning on the Wiki about misuse of a particular feature. I would also like to see a remove-from-address-list action in firewall. Options are always good, just make sure users are informed of it's pros and cons (same as layer7 warning about intensive cpu usage if used improperly).

I've been using comment field for array of settings (ex. "time=<time>,setting2=value,setting3=value"). For now, this allows me to get capture settings of the particular address list entry. This however does write to disk though and won't work on dynamic entries (I don't think dynamic entries can have comments).
Doug
 
Feklar
Forum Guru
Forum Guru
Posts: 1726
Joined: Tue Dec 01, 2009 11:46 pm

Re: address-list script timeout

Mon Jun 14, 2010 9:40 pm

I can give you an example of where adding in a timeout feature to the a firewall address list without it being dynamic.

We are running hotspots, and sometimes end users fly bellow some of our filter rules that will block abuse, usually by setting their p2p software with a lower connection limit. We would like to add these users to the "abuse" list for a limited time, but don't want to leave them in there as their DHCP lease can change, or they can check out and someone else will pick up the DHCP lease.

Right now the only way we can do that is make a firewall rule with their IP address listed and with the action of adding them to the address list with a timeout. Then after they pass traffic and are added to the list, remove the rule. It would be much simpler if we could just add an address to the list and set a timeout for that address.
 
User avatar
janisk
MikroTik Support
MikroTik Support
Posts: 6283
Joined: Tue Feb 14, 2006 9:46 am
Location: Riga, Latvia

Re: address-list script timeout

Tue Jun 15, 2010 9:48 am

this is all true, but look on this this way - you can use firewall to add entries dynamically, you detect these abuses using it, so there is no difference, once detected, address is added to list as dynamic with time-out value set, so make firewall in a way, that user can attempt abuse every 30 minutes.

about refresing 1000 or removing one - it is not the case. If you need static list, it is static it should be there, like secure addresses to connect from, bonog IP list to drop packets that should not be in the network etc. If list is dynamic, just manage it to stay dynamic. If you have some differences, just create several lists that will self contain them using action=add-src-to-address-list/add-dst-to-address-list.
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8383
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: address-list script timeout

Tue Jun 15, 2010 12:52 pm

about refresing 1000 or removing one - it is not the case. If you need static list, it is static it should be there, like secure addresses to connect from, bonog IP list to drop packets that should not be in the network etc.
I mean, the main purpose is NOT to write list changes on disk. router reboots => lists are clear => no access for users => BUT! management software fills in them quickly anyway

4-6 million disk writes in 225 days of uptime... is big enough number for me =)
Russian-speaking forum: https://forum.mikrotik.by/. Welcome!

For every complex problem, there is a solution that is simple, neat, and wrong.

MikroTik. Your life. Your routing.
 
User avatar
janisk
MikroTik Support
MikroTik Support
Posts: 6283
Joined: Tue Feb 14, 2006 9:46 am
Location: Riga, Latvia

Re: address-list script timeout

Tue Jun 15, 2010 2:51 pm

adding dynamic does not change that. What i was saying you can already add dynamic entries using firewall filters. These do not generate disk writes.
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8383
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: address-list script timeout

Tue Jun 15, 2010 4:10 pm

adding dynamic does not change that
you mean, creating dynamic rules via, for example, command line should make disk writes? %)
What i was saying you can already add dynamic entries using firewall filters
yeah, you're right. why do we need API at all, if we already have such a nice feature - firewall rules!..
Russian-speaking forum: https://forum.mikrotik.by/. Welcome!

For every complex problem, there is a solution that is simple, neat, and wrong.

MikroTik. Your life. Your routing.
 
Feklar
Forum Guru
Forum Guru
Posts: 1726
Joined: Tue Dec 01, 2009 11:46 pm

Re: address-list script timeout

Tue Jun 15, 2010 7:59 pm

this is all true, but look on this this way - you can use firewall to add entries dynamically, you detect these abuses using it, so there is no difference, once detected, address is added to list as dynamic with time-out value set, so make firewall in a way, that user can attempt abuse every 30 minutes.
I wouldn't care if the list is dynamic or not, but having a timeout feature on certain things such as entries in an address lists would be very nice and useful to us. The main thing that comes to mind is blocking certain people of a specific amount of time. We actually redirect them to a page that lets them know they have been blocked for abuse.

Other areas that we would use a timeout feature would be the following:
1.) Hotspot bypasses for users that couldn't make it through the login process for whatever reason and needed to be bypassed.
2.) Expiring Simple Queue's that were made for the bypass.
3.) Expiring "Static" DHCP leases that were made for the bypass.

There are a few other things I that I can't think off the top of my head.
 
User avatar
janisk
MikroTik Support
MikroTik Support
Posts: 6283
Joined: Tue Feb 14, 2006 9:46 am
Location: Riga, Latvia

Re: address-list script timeout

Wed Jun 16, 2010 9:35 am

there is a lot of word "static" in this thread and static usually is handled by user in direct or indirect way, meaning, you do it by hand or set up scheduler to do it. Once you have set up pattern of using scheduler it is easy to make small adjustments for next "exception".
 
changeip
Forum Guru
Forum Guru
Posts: 3810
Joined: Fri May 28, 2004 5:22 pm

Re: address-list script timeout

Wed Jun 16, 2010 9:46 am

i would like to synchronize dynamic address-lists on routers. i have a fat workaround now, but it would be nice to be able to add entries from the command line (fetch'd from URL) and allow them to timeout on their own. right now i have to ping each one with a 1 TTL to dynamically catch and add it... thats a lot of overhead.
Colo and Wholesale Bandwidth Available! Sales at SanDiegoBroadband dot com
 
blake
Member
Member
Posts: 426
Joined: Mon May 31, 2010 10:46 pm
Location: Arizona

Re: address-list script timeout

Wed Jun 16, 2010 11:28 am

There's a Linux netfilter module called IPSET which has exactly this functionality through its 'iptree' set type; entries can be added to lists via the command line and given timeout values.

How difficult could it be to add this functionality to RouterOS?

There is a huge benefit to be able to perform this functionality via the API from an NMS. An NMS could take into account different metrics before adding a user to a list that the firewall rules cannot.
IT consultant. Network manager. Packet junkie.
1-928-328-1509
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8383
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: address-list script timeout

Wed Jun 16, 2010 2:07 pm

by the way, I forgot to mention one more important thing about dynamic entries: they are not shown in /export =)
Russian-speaking forum: https://forum.mikrotik.by/. Welcome!

For every complex problem, there is a solution that is simple, neat, and wrong.

MikroTik. Your life. Your routing.
 
ekkas
Long time Member
Long time Member
Posts: 562
Joined: Mon Sep 26, 2005 1:01 pm
Location: South Africa

Re: address-list script timeout

Tue Jul 27, 2010 11:44 pm

Sign me up for adding timeoutable (Is that a new word? ;) address-list entries from system scripts or terminal.

Ekkas
 
ekkas
Long time Member
Long time Member
Posts: 562
Joined: Mon Sep 26, 2005 1:01 pm
Location: South Africa

Re: address-list script timeout

Wed Jul 28, 2010 12:19 am

I had to do some routing based on a DDNS host, so as a dirty workaround, I wrote this script.
New to scripting, so feel free to correct me or make it more efficient if needed.
Then I schedule it every 30 minutes or so.
It checks if the address is there, if not it adds it, then it removes any other addres.
I only have 1 address in this list so it works for me.
I thought it might remove any other address in other lists as well. Does the "address-list add" command put it in the context of the YOURLIST list then?
Or to be safe, would there be a way to do somethiing like this:
:foreach i in [/ip firewall address-list find address!=$listip & address-list=YOURLIST] do={ :set checkIPP $i }
to ensure I remove only an IP in the intended list?




:local listip [:resolve "thehost.dyndns.org"]
:local checkIP 0
:foreach i in [/ip firewall address-list find address=$listip] do={ :set checkIP 1 }
:if ($checkIP = 0) do={ /ip firewall address-list add list=YOURLIST address=$listip comment="TrueNW"}
:log info $listip
:local checkIPP -1
:foreach i in [/ip firewall address-list find address!=$listip] do={ :set checkIPP $i }
:if ($checkIP != -1) do {/ip firewall address-list remove numbers=$checkIPP}
 
dssmiktik
Forum Veteran
Forum Veteran
Posts: 732
Joined: Fri Aug 17, 2007 8:42 am

Re: address-list script timeout

Mon Sep 13, 2010 8:00 am

ekkas,
Looks good.

I actually use a similar script. Here is the script I use:
:local hostnames "www.google.com, www.mikrotik.com, routerboard.com"

# Internal processing...
:local Script "Hostname-To-AddressList"
:local hostip
:local oldip
:local dnssearch
:local addrlistsearch

:foreach h in=[:toarray $hostnames] do={
   :set hostip ""
   :set dnssearch 0

   :put ("Resolving " . $h . "...")

# Search DNS cache first
   /ip dns cache all {
      :set dnssearch [find name=$h]
      :if ([:len $dnssearch] > 0) do={
#      Only retrieve DNS A records
         :if ([get $dnssearch type] = "A" || [get $dnssearch type] = "a") do={
            :set hostip [get $dnssearch data]
         }
      }
   }

# If no IPs found, resolve hostname
   :if ([:len $hostip] = 0) do={
      :set hostip [:resolve $h]
   }

# Search address lists
   /ip firewall address-list {
      :set addrlistsearch [find list=$h]
#   Did we find address list with hostname?
      :if ([:len $addrlistsearch] = 1) do={
         :set oldip [get $addrlistsearch address]
         :if ($oldip != $hostip) do={
           :log info ($Script . "   " . $h . " IP changed: (" . $oldip . " -> " . $hostip . ")")
            set $addrlistsearch address=$hostip
         }
      } else={
#      No Address List found with hostname
         :log info ($Script . "   Adding address list " . $h . " address " . $hostip)
         add list=$h address=$hostip disabled=no
      }
   }
}
Doug
 
User avatar
maxrate
Frequent Visitor
Frequent Visitor
Posts: 90
Joined: Mon Oct 23, 2006 10:55 pm
Location: Toronto

Re: address-list script timeout

Wed Mar 20, 2013 3:43 am

It's 3 years later, and MT still hasn't added this!? This would be incredibly useful for a number of scripts with out complicated work-arounds. Mikrotik, please listen to your users and add features.
Mikrotik everywhere!
 
User avatar
skot
Long time Member
Long time Member
Posts: 586
Joined: Wed Nov 30, 2011 3:05 am

[FEATURE REQUEST] address-list script timeout

Wed Mar 20, 2013 5:54 am

+1

This would be a very useful feature.
I don't need any karma... I have Ιησους Χριστος!
 
User avatar
omidkosari
Trainer
Trainer
Posts: 634
Joined: Fri Sep 01, 2006 4:18 pm
Location: Iran , Karaj
Contact:

Re: address-list script timeout

Wed Mar 20, 2013 10:12 am

I think following feature request is also related to this topic .

[Ticket#2012120366000497] Feature request: add dynamic ip to address list by api
Hello,

We will consider adding this feature in the future.

Regards,
Maris

12/4/2012
MTCNA , MTCRE, MTCWE, Mikrotik Certified Trainer
 
User avatar
omidkosari
Trainer
Trainer
Posts: 634
Joined: Fri Sep 01, 2006 4:18 pm
Location: Iran , Karaj
Contact:

Re: address-list script timeout

Thu Jul 03, 2014 11:03 am

There http://forum.mikrotik.com/viewtopic.php?f=19&t=82825 is an official comment
Actually, this will not be feature of 7.x as it is already available in RotuerOS 6.x

ros code

[admin@MikroTik] /ip firewall address-list> add list=lala timeout=30s address=0.0.0.0 
[admin@MikroTik] /ip firewall address-list> print 
Flags: X - disabled, D - dynamic 
 #   LIST                                                                    ADDRESS                         TIMEOUT             
 0 D lala                                                                    0.0.0.0                         28s                 
[admin@MikroTik] /ip firewall address-list> print 
Flags: X - disabled, D - dynamic 
 #   LIST                                                                    ADDRESS                         TIMEOUT             
 0 D lala                                                                    0.0.0.0                         25s
MTCNA , MTCRE, MTCWE, Mikrotik Certified Trainer

Who is online

Users browsing this forum: No registered users and 19 guests