Community discussions

MikroTik App
 
troysorzano
just joined
Topic Author
Posts: 1
Joined: Fri Jan 28, 2011 3:24 pm

Simple Web Scraper for The Dude

Sun Dec 29, 2013 8:40 am

I needed a way to automate the export of device names and IP's to use with other automated processes. I know you can use the The Dude Gui and right click to export but I needed something that could run on a schedule.

http://nocwiz.com/2013/12/29/using-mikr ... epository/
#
# NOCwiz.com presents The Dude Network Device Repository
#
# dudendr.py allows you to use Mikrotik's The Dude
# as your NDR.  This script will screen scrape the
# device IP's and names from The Dude web interface.
#
# Requires Requests and BeautifulSoup Python Modules
#
 
# For this example just hard code all the parameters
username    = 'admin'
password    = 'supersecret'
dudeaddr    = '10.0.19.59'
filename    = 'dudendr.csv'
 
import requests, urllib
from BeautifulSoup import BeautifulSoup
 
devicename = []
deviceurl = []
deviceip = []
 
#Create the parameters to submit to the login form.
payload = {
    'process': 'login',
    'page': 'start',
    'password': password,
    'user': username
}
 
# Use urllib to encode the form payload then post
params = urllib.urlencode(payload)
page = 'http://'+dudeaddr+'/dude/main.html'
resp = requests.post(page, params=payload)
# save the cookie since we are now logged in
cookies = resp.cookies
 
# Get the page of all devices
page = 'http://'+dudeaddr+'/dude/main.html?page=devices'
resp = requests.get(page,cookies=cookies)
 
# Process the page results
# Gather the devicename and deviceurl
soup = BeautifulSoup(resp.text)
links = soup.findAll('a')
for a in links:
    if 'device_info' in a['href'] :
        devicename.append(a.string)
        deviceurl.append(a['href'])
 
# Process all the device URL's to get the IPs
for url in deviceurl:
    page = 'http://'+dudeaddr+url
    resp = requests.get(page,cookies=cookies)
    soup = BeautifulSoup(resp.text)
    result = soup.find(attrs={"name":"address"})
    deviceip.append(result['value'])
 
# Save the results to a CSV file
file = open(filename, "w")
for x in range(0,len(deviceurl)):
    file.write(devicename[x]+","+deviceip[x]+"\n")
file.close()

Who is online

Users browsing this forum: SGBIPL and 64 guests