Help with script for DNS lookups and address lists

RouterOS Scripting
1

I’m using the script from
http://adminsline.wordpress.com/2012/08/03/mikrotik-youtube/
in order to make address list with YouTube video server IPs from DNS cache so I can make some redirections.

I wish to use the same script so I could grab IP addresses of some other video streaming sites,
like vimeo.com, imdb.com and so on. While for YouTube video servers I get A records in DNS cache, for Vimeo or IMDB
I actually get CNAME records and the above script can’t resolve them.

There is other script on Mikrotik Wiki:
http://wiki.mikrotik.com/wiki/Sync_Address_List_from_DNS_Lookup_Results_-_CNAME_and_A_Records
which works with CNAME records as well but is not suitable as such for my situation (the first script as search string
takes part of the server name, for example “googlevideo”, “vimeocdn” etc).

Unfortunately I’m not that good at scripting and can’t figure out how to incorporate the CNAME resolving part of the second
script into the first one.

Has anyone done something like that and is willing to help? It would be much appreciated.

2

I’ll look at the scripts once I get back to my computer.

Sent from my SCH-I545 using Tapatalk

3

Thank you very much!

4

It wouldn’t be that hard to merge the two scripts, but I don’t have the time to do it right this second. It will most likely be at least a week or two before I would have time to tackle it, I had/have some family stuff going on that is taking up most of my time. I figure it would take about 2 hours to get it done.

5

I have tried merging it but with no success.
It’s not a pressing matter so if you could do it in a two weeks time,
when you have free time it would be great.

Best regards!

6

I’ll keep it on the to do list when I have some time. If you want it done sooner I could “consult” on it… if your interested in going that route just email me.

7

Also can you give me a run through of EXACTLY what you are wanting the script to do? You just want the script with CNAMEs to work on partial names?

8

Here is the exact script I’m running on the router at the moment:

:foreach i in=[/ip dns cache find] do={
    :local bNew "true";
    :local cacheName [/ip dns cache all get $i name] ;
#    :put $cacheName;

    :if (([:find $cacheName "youtube"] != 0) || ([:find $cacheName "googlevideo"] != 0)) do={

        :local tmpAddress [/ip dns cache get $i address] ;
#   :put $tmpAddress;

# if address list is empty do not check
        :if ( [/ip firewall address-list find ] = "") do={
            :log info ("added entry: $[/ip dns cache get $i name] IP $tmpAddress");
            /ip firewall address-list add address=$tmpAddress list=VC-YouTube comment=$cacheName;
        } else={
            :foreach j in=[/ip firewall address-list find ] do={
                :if ( [/ip firewall address-list get $j address] = $tmpAddress ) do={
                    :set bNew "false";
                }
            }
            :if ( $bNew = "true" ) do={
                :log info ("added entry: $[/ip dns cache get $i name] IP $tmpAddress");
                /ip firewall address-list add address=$tmpAddress list=VC-YouTube comment=$cacheName;
            }
        }
    }
}

It searches through DNS cache looking for servers which have words “youtube” or “googlevideo” in their name and then adds IP
address of the server to address list named VC-YouTube.

If I change it to search for servers that have in their name “vimeocdn” it will find, lets say server10.vimeocdn.com (I’ve made
that up) but the Type field in DNS cache will be CNAME and the Data field will be something like a1806.dwskn.akamai.net.
In order to be able to add that to an address list I would need that a1806.dwskn.akamai.net further resolved to numeric IP address.

The second script I mentioned in my first post can do that, but I can’t feed it a partial server name and let it find all the
servers that has the keyword in its name so it’s unusable in my case.

I guess this might be interesting to others as well as it allows some fancy dynamic redirecting or routing of the sites that can be
bandwidth hogs. As I’ve said there is no rush about it so whenever you have free time, it would be nice to have this done and apply
it to more sites…

Regards!

9

:local myServers { “google” }
:local myListName “myList”
:local myTimeout “00:01:00”

/ip dns cache all {
:foreach i in=$myServers do={
:foreach j in=[find where (name~$i)] do={
:local myName [get $j name]
:local myType [get $j type]
:local myData [get $j data]

        :if ($myType = "A") do={
            :do {
                /ip firewall address-list add address=$myData list=$myListName comment="DNSCACHE-$i-$myName" timeout=$myTimeout
            } on-error={
                :put "DNSCACHE: Error on $myName $myData $myType"
            }
        }

        :if ($myType = "CNAME") do={
            :local currentName $j
            :local nextName [find where (name=$myData && type="CNAME")]

            :while ($nextName != "") do={
                :set currentName $nextName
                :set nextName [find where (name=[get $nextName data] && type="CNAME")]
            }

            :resolve [get $currentName data]

            :foreach k in=[find where (name=[get $currentName data] && type="A")] do={
                :set myData [get $k data]
                :do {
                    /ip firewall address-list add address=$myData list=$myListName comment="DNSCACHE-$i-$myName" timeout=$myTimeout
                } on-error={
                    :put "DNSCACHE: Error on $myName $myData $myType"
                }
            }
        }
    }
}

}

10

Thank you very very much Efaden!!! The script works like a charm.

I had to takeout the following part of the script though, as it was giving me an error,
but that doesn’t decrease its usefulness at all!

timeout=$myTimeout
     } on-error={
        :put "DNSCACHE: Error on $myName $myData $myType"
     }
11

The error catch will only work on 6.5 (I think) and beyond. … I would actually suggest upgrading then using that statement. Otherwise the script will crash on duplicate entries. If you want me to make it work with duplicate entries and no on-error let me know… I can re-work it a bit…

And timeout is also only on the newer versions.. I think 6.7 and up.

Right now the script doesn’t clear the list… so you will run into problems if you remove those and run it on an older version. If you want me to make it work on an older ROS let me know what version and I can retrofit.

-Eric

12

Oh, I see.. I’ve tried it on a router with 6.3 version.
I don’t want to bother you more than I have already, I will just upgrade the OS to a higher version.
Thanks again!

13

Up to you… won’t take me long to retrofit it.

14

Not tested… but something like this.
:local myServers { “google” }
:local myListName “myList”

/ip firewall address-list {
remove [find list=$myListName]
}

/ip dns cache all {
:foreach i in=$myServers do={
:foreach j in=[find where (name~$i)] do={
:local myName [get $j name]
:local myType [get $j type]
:local myData [get $j data]

		:if ($myType = "A") do={
			:local inList false
			:foreach k in=[find where (address=$myData && list=$myListName)] {
				:set inList true
			}

			:if (!inList) do={
				/ip firewall address-list add address=$myData list=$myListName comment="DNSCACHE-$i-$myName"
			}
		}

		:if ($myType = "CNAME") do={
			:local currentName $j
			:local nextName [find where (name=$myData && type="CNAME")]

			:while ($nextName != "") do={
				:set currentName $nextName
				:set nextName [find where (name=[get $nextName data] && type="CNAME")]
			}

			:resolve [get $currentName data] 

			:foreach k in=[find where (name=[get $currentName data] && type="A")] do={
				:set myData [get $k data]
				:local inList false
				:foreach l in=[find where (address=$myData && list=$myListName)] do={
					:set inList true
				}

				:if (!inList) do={
					/ip firewall address-list add address=$myData list=$myListName comment="DNSCACHE-$i-$myName"
				}
			}
		}
	}
}

}

15

Thank you for the modified version as well.

I have upgraded OS to v6.7 in the meantime and the original script now works without any modification.
However, when I run if for the second time (soon after the first run) it ends up in error. Please have a look:

DNSCACHE: Error on r4---sn-nvm-cxbe.googlevideo.com 193.105.163.207 CNAME
DNSCACHE: Error on r1---sn-nvm-cxbe.googlevideo.com 193.105.163.204 CNAME
DNSCACHE: Error on r1.sn-nvm-cxbe.googlevideo.com 193.105.163.204 A
DNSCACHE: Error on r8---sn-c0q7lne7.googlevideo.com 74.125.108.173 CNAME
DNSCACHE: Error on r8.sn-c0q7lne7.googlevideo.com 74.125.108.173 A
DNSCACHE: Error on r8.sn-c0q7lner.googlevideo.com 173.194.1.237 A
DNSCACHE: Error on r11.sn-c0q7ln7y.googlevideo.com 74.125.108.144 A
DNSCACHE: Error on r4.sn-nvm-cxbe.googlevideo.com 193.105.163.207 A
DNSCACHE: Error on r3.sn-nvm-cxbe.googlevideo.com 193.105.163.206 A
DNSCACHE: Error on r10.sn-c0q7lnee.googlevideo.com 74.125.108.207 A
DNSCACHE: Error on r10.sn-c0q7lney.googlevideo.com 173.194.2.143 A
DNSCACHE: Error on r11.sn-c0q7lnez.googlevideo.com 173.194.1.48 A
DNSCACHE: Error on r2.sn-nvm-cxbe.googlevideo.com 193.105.163.205 A
DNSCACHE: Error on r5.sn-c0q7lnek.googlevideo.com 173.194.1.202 A
DNSCACHE: Error on r5.sn-c0q7lnez.googlevideo.com 173.194.1.42 A
DNSCACHE: Error on r7.sn-c0q7lney.googlevideo.com 173.194.2.140 A
DNSCACHE: Error on r5.sn-c0q7lney.googlevideo.com 173.194.2.138 A
DNSCACHE: Error on r19.sn-c0q7lnek.googlevideo.com 173.194.1.216 A
DNSCACHE: Error on r9.sn-c0q7lne6.googlevideo.com 173.194.1.142 A
DNSCACHE: Error on r13.sn-c0q7lnee.googlevideo.com 74.125.108.210 A
DNSCACHE: Error on r18.sn-c0q7lnez.googlevideo.com 173.194.1.55 A
interrupted
           no such item (4)

I understand that those DNSCACHE: Error are because the address already exists in the address-list and that
is okay I presume? But I don’t know why it got interrupted and what item is missing. An expired entry from the
DNS cache??

16

Hmmm. That’s my guess. I’ll have to investigate and add a mechanism to catch that.

Sent from my SCH-I545 using Tapatalk

17

Try this… I added a “:resolve” in the middle to make sure entries are valid when I loop… also added something to detect if an entry vanished while it was looping and just ignore that entry… I haven’t tested it… let me know.
:local myServers { “google” }
:local myListName “myList”
:local myTimeout “00:01:00”

/ip dns cache all {
:foreach i in=$myServers do={
:foreach j in=[find where (name~$i)] do={
:local exists false
:do {
:local myName [get $j name]
:local myType [get $j type]
:local myData [get $j data]
:set exists true
} on-error={
:put “DNSCACHE: Missing Entry”
}

        :if ($exists && $myType = "A") do={
            :do {
                /ip firewall address-list add address=$myData list=$myListName comment="DNSCACHE-$i-$myName" timeout=$myTimeout
            } on-error={
                :put "DNSCACHE: Error on $myName $myData $myType"
            }
        }

        :if ($exists && $myType = "CNAME") do={
            :local currentName $j
            :local nextName [find where (name=$myData && type="CNAME")]

            :while ($nextName != "") do={
                :set currentName $nextName
                :resolve $currentName
                :set nextName [find where (name=[get $nextName data] && type="CNAME")]
            }

            :resolve [get $currentName data]

            :foreach k in=[find where (name=[get $currentName data] && type="A")] do={
                :set myData [get $k data]
                :do {
                    /ip firewall address-list add address=$myData list=$myListName comment="DNSCACHE-$i-$myName" timeout=$myTimeout
                } on-error={
                    :put "DNSCACHE: Error on $myName $myData $myType"
                }
            }
        }
    }
}

}

18

It gives me the following error:

expected variable name (line 39 column 26)
19

Whoops… that was stupid. Scope problem…
:local myServers { “google” }
:local myListName “myList”
:local myTimeout “00:01:00”

/ip dns cache all {
:foreach i in=$myServers do={
:foreach j in=[find where (name~$i)] do={
:do {
:local myName [get $j name]
:local myType [get $j type]
:local myData [get $j data]

            :if ($myType = "A") do={
                :do {
                    /ip firewall address-list add address=$myData list=$myListName comment="DNSCACHE-$i-$myName" timeout=$myTimeout
                } on-error={
                    :put "DNSCACHE: Error on $myName $myData $myType"
                }
            }

            :if ($myType = "CNAME") do={
                :local currentName $j
                :local nextName [find where (name=$myData && type="CNAME")]

                :while ($nextName != "") do={
                    :set currentName $nextName
                    :resolve $currentName
                    :set nextName [find where (name=[get $nextName data] && type="CNAME")]
                }

                :resolve [get $currentName data]

                :foreach k in=[find where (name=[get $currentName data] && type="A")] do={
                    :set myData [get $k data]
                    :do {
                        /ip firewall address-list add address=$myData list=$myListName comment="DNSCACHE-$i-$myName" timeout=$myTimeout
                    } on-error={
                        :put "DNSCACHE: Error on $myName $myData $myType"
                    }
                }
            }
        } on-error={
            :put "DNSCACHE: Missing Entry"
        }
    }
}

}

20

This time it runs through with no problem. Even when I runnit for the second time immediately after the
first run it still doesn’t get aborted and runs through to the end.

One funny thing, when I run it, the first run, it showed 30 of those “DNSCACHE: Error” errors, and that’s exactly how many
entries it added to the address list. But please note, I already had those IP addresses but in another
address-list (collected by the old script I used before yours) so that might be the cause for such behaviour.

In any case, all addresses are there regardles of the “DNSCACHE: Error”.

Thank you very much for your effort!