All correct, but there are two ways:
fast and error prone | precise and error free
Then you have to move the cursor wherever you prefer...
(the cursor cannot be completely at the bottom of either mode)
The full script clearly specifies that elements are searched for and short-lived ones are skipped; it doesn't actually apply to everything...
You misunderstand. In both cases, only the deletion of 2 items, .id=10000 and .id=30000 was attempted. Both non-existent, both times.
And lo and behold... timeout>60 comes into play... It's no casual...
OK, misreaded: is not from 10000 to 30000 but 10000 and 30000
I have to self-correct:
foreach i in={10000; 30000} [..] remove $i
This example line is interpretable:
there is no .id specified,
it simply deletes lines 10000 and 30000 if a "print" was performed before,
since $i is a number, not an index.
Therefore, "remove 10000" does not attempt to remove the index *10000 (or *2710, for that matter),
but the object located on line 10000 of the "print" statement.
More (?) fast and...
/ip fire conn
:foreach idc in=[find] do={
:execute "/ip fire conn remove $idc"
}
Yep. Should have been
> foreach i in={[:toid "*10000"]; [:toid "*30000"]} do= { /ip firewall connection remove $i } on-error= { :put "Skipped $i (type: $[:typeof $i ])" }
Skipped *10000 (type: id)
Skipped *30000 (type: id)
I think it still writes logs for errors? And :execute spawns a background task, which I assume involves a new thread or process? and would be quite resource intensive. I tested with:
:for i from=1 to=20 do={ [:execute { :delay 30; :local x (20 / 0); }] }
to confirm that errors are logged and the background task runner is NOT reused (each :execute has its own job).
It remains to be seen whether parallel execution improves speed when the records are 200k and whether more than one processor is used or not.
I didn't miss it. The issue is that find fails even by itself.
[sin3vil@MASTER] > ip firewall/connection/find where src-address=10.222.3.124 src-port=5060
no such item (4)
Also, I might want to remove actually live connections. For example, 10.222.3.124 is source-natted with the wrong IP, I update the NAT rules and want to flush it's specific connections so they get natted properly.
I'm not looping over 2 specific ids that might happen to work. I'm finding all matching ids that fails, before the actual remove command.
Also, it might be a cause of your specific setup. How many connections do you have?
I'm at ~450k at the moment.
Also guys, this isn't a general discussion about using on-error constructs, but why these seem to work elsewhere but not specifically in /ip firewall connection.
Sorry for pollution on the topic
Mikrotik probably uses the conntrack-tools way of streaming conntrack data to user space for filtering. That implementation itself is pretty solid (and widely used). You're probably running into some sort of weirdness, like your operation taking too long (timeout), too much memory, or some extended feature (fasttrack, fasttrack with L3 offload) causes some unknown condition.
That normal error handling can't catch non-normal errors is not exactly news worthy. Try catching and dealing with an out of memory in c++.
I've done similar for somewhat under 200k connections (admittedly on a router with relatively good single core performance) and - although it took a while - there wasn't any problem with it.