Securely storing apikey/tokens for /tool/fetch... Approaches? == $SECRET

Amm0 · Tue Feb 22, 2022 8:17 pm

In V7, I've been trying to "port" some bash/javascript scripts we use. Most just call various REST APIs. Been using RouterOS script using /tool/fetch to replicate some of them, since running them directly on a router make scripting SOME stuff easier.

Problem is some REST API I call need an "apikey" or "token", which is essentially a fancy password needed for the API.
e.g.

:global apikey [???????]
/tool/fetch url=... http-header-field="Authorization: bearer $apikey" ...

Since I'm just experiment with this approach, I've used them as :global variables loaded by script & this obviously works. But not very secure. Since the script code is pretty visible in the config (e.g. via :export or other uses). And, even load them from a file in /files, the files can't be restricted to a single user either AFAIK.

While for stuff like AWS and other API, you can use the X.509 certificates – which are supported in /tool/fetch under V7, e.g.

/tool/fetch url=... certificate=...

But not all REST APIs support X.509 client certs.

Basically... I want to "stash" an external REST API's apikey/token/password on a Mikrotik, that will be there after reboot, but not show up in an ":export". Similar to how /certificate stuff work (e.g. in "backup" but not ":export"), except I'm dealing with 8-64 char strings, not certificates. Or, the concept of "encrypted-secrets", which store "private data" used by GitHub repo/actions/etc (e.g. to avoid needing password aren't kept in files/code) .

Curious if any one has any "nifty" solutions to this?

Amm0 · Wed Feb 23, 2022 12:23 am

I'd seen "/ip/firewall/layer7-protocol/..." used to "cache" things, but that's also not very "hidden".

But that gave me an idea. I don't normally use PPP, but they do have "secrets" and the password field there would at least hidden in most exports, but still persist. Anyway wrote a function that use

/ppp/secret $name set/get password=$apikey

for storing, well, "secrets". Maybe there is a better approach – really was hoping there be something in /certificates to securely store a basic string, but couldn't figure out any tricks there. Anyway, this is what I came up so far:

Updated: See code in viewtopic.php?p=916159#p916159

Which then can be used like:

> $SECRET set MTforumpw password=ItIsASecretDontYouKnow

 > :put [$SECRET get MTforumpw]
ItIsASecretDontYouKnow
 
 > $SECRET print
Columns: NAME, SERVICE, PASSWORD, PROFILE
# NAME       SERVICE  PASSWORD                          PROFILE
;;; used by $SECRET
1 MTforumpw  async    ItIsASecretDontYouKnow            null

And my main use is for /tool/fetch, so looks like this with /tool/fetch's headers:

{
# ...
:local headers "Authorization: bearer $[$SECRET get mtforumpw]"        
:local resp [/tool/fetch url="$url" http-method="$method" http-header-field="$headers" http-data=($payload) output="user" as-value]
:put $resp

Seems to work in a few tests – at least avoids the /export showing REST/etc API keys without show-sensitive, which was my biggest concern.
Still feel pretty "hack-ish", so curious if anyone has any better ideas here...

Sob · Wed Feb 23, 2022 2:48 am

Your (mis)use of PPP secrets is almost there. Since v7 passwords show up in export only when show-sensitive is specified (unlike in v6, where they were exported by default and hide-sensitive was needed to not export them), so nobody can export them by accident. And user must have "sensitive" permission to see them, otherwise it exports password=***** without showing actual password.

It's probably good idea to make it official and add some dedicated storage with same properties. I'm not sure about possible per-user access, that could complicate things (e.g. it would probably require some "superadmin" permission for making backups containing secrets of all users).

I would very much hate following in steps of unexportable certificates and other things, that can be exported only as part of binary backup, which is terribly impractical (restoring it is all or nothing, it's not possible to see what's inside, so there's no simple way to see differences between backups).

Amm0 · Wed Feb 23, 2022 5:25 am

Yes, PPP is a poor knock-off of AAA.

It's probably good idea to make it official and add some dedicated storage with same properties.

You won't get an argument from me

. Basically trying to not have "passwords" is my ROS script code.

While I do find the approach of "/ip/firewall/layer7-protocol/set $attr regexp=$value" from this post very cleaver – it shows the same need of "persisted variables" as built-in to RouterOS (with some "is-sensitive=yes" option I suppose

).

But good news is it's easy to change my $SECRET function to do={# something else in future} – without changing any of the code that USED any "secrets". As you point out, /ppp/secret has some /user/group policy for it ("sensitive"), although the :export with /system/script/... containing "passwords" was WAY bigger concern, than other admin users seeing anything.

I would very much hate following in steps of unexportable certificates and other things, that can be exported only as part of binary backup, which is terribly impractical (restoring it is all or nothing, it's not possible to see what's inside, so there's no simple way to see differences between backups).

Again, no argument.

But this is why I started my hunt for some "hack" in /certificates ... I knew cert stuff is NOT in an /export – so TRIED to turn that negative, into a positive

... no such luck.

Amm0 · Wed Feb 23, 2022 6:08 am

Did a quick test on this. /ppp/secret's password can be up to 64k it seems – I would have though it be much lower. At least under V7, this test script show the limit:

:global ppppwdmax do={
    :for i from=1 to=[:tonum $1] step=($1/10) do={
        /ppp/secret/remove [find where comment="#removeme"]
        :local expected [:rndstr length=$i from=abc]
        /ppp/secret/add name="pwd$i" password=$expected comment="#removeme"
        :local actual [/ppp/secret/get "pwd$i" password]
        :put "/ppp/secret test loop=$i expected=$[:len $expected] actual=$[:len $expected]"
        /terminal/cuu
        :if ($expected!=$actual) do={
            :error "failed to created new /ppp/secret with password lengths of loop=$i expected=$[:len $expected] actual=$[:len $actual] "
        }  
    } 
}

# this will work
$ppppwdmax 60000
# /ppp/secret test loop=54001 expected=54001 actual=54001

# this won't and gets a very clear error with limit
$ppppwdmax 100000
# afraid to create strings larger than 64kB

Amm0 · Mon Feb 28, 2022 9:04 pm

Updated the "persisted store using ppp secrets" script so it should work on both V6 and V7. Although again persisted variables are a "needed feature" – since this is still a hack, works well enough for my purposes, but no warranties here.

### $SECRET
#   get <name>
#   set <name> password=<password>
# . remove <name
#   print
:global SECRET
:set $SECRET do={
    :global SECRET

    # helpers
    :local fixprofile do={
        :if ([/ppp profile find name="null"]) do={:put "nothing"} else={
            /ppp profile add bridge-learning=no change-tcp-mss=no local-address=0.0.0.0 name="null" only-one=yes remote-address=0.0.0.0 session-timeout=1s use-compression=no use-encryption=no use-mpls=no use-upnp=no
        }
    }
    :local lppp [:len [/ppp secret find where name=$2]]
    :local checkexist do={
        :if (lppp=0) do={
            :error "\$SECRET: cannot find $2 in secret store"
        }
    }

    # $SECRET
    :if ([:typeof $1]!="str") do={
        :put "\$SECRET"
        :put "   uses /ppp/secrets to store stuff like REST apikeys, or other sensative data"
        :put "\t\$SECRET print - prints stored secret passwords"
        :put "\t\$SECRET get <name> - gets a stored secret"
        :put "\t\$SECRET set <name> password=\"YOUR_SECRET\" - sets a secret password" 
        :put "\t\$SECRET remove <name> - removes a secret" 
    }

    # $SECRET print
    :if ($1~"^pr") do={
        /ppp secret print where comment~"\\\$SECRET"
        :return [:nothing] 
    }

    # $SECRET get
    :if ($1~"get") do={
        $checkexist
       :return [/ppp secret get $2 password] 
    }

    # $SECRET set
    :if ($1~"set|add") do={
        :if ([:typeof $password]="str") do={} else={:error "\$SECRET: password= required"}
        :if (lppp=0) do={
            /ppp secret add name=$2 password=$password 
        } else={
            /ppp secret set $2 password=$password
        }
        $fixprofile
        /ppp secret set $2 comment="used by \$SECRET"
        /ppp secret set $2 profile="null"
        /ppp secret set $2 service="async"
        :return [$SECRET get $2]
    } 

    # $SECRET remove
    :if ($1~"rm|rem|del") do={
        $checkexist
        :return [/ppp secret remove $2]
    }
    :error "\$SECRET: bad command"
}

Here is an example of using the function:

$SECRET 
#$SECRET
#   uses /ppp/secrets to store stuff like REST apikeys, or other sensative data
#        $SECRET print - prints stored secret passwords
#        $SECRET get <name> - gets a stored secret
#        $SECRET set <name> password="YOUR_SECRET" - sets a secret password
#        $SECRET remove <name> - removes a secret
#$SECRET: bad command

$SECRET print
#Flags: X - disabled 
# #   NAME         SERVICE CALLER-ID      PASSWORD      PROFILE      REMOTE-ADDRESS 

$SECRET add "rest_apikey" password="mikrotik"
#

$SECRET print
#Flags: X - disabled 
# #   NAME         SERVICE CALLER-ID      PASSWORD      PROFILE      REMOTE-ADDRESS 
# 0   ;;; used by $SECRET
#     rest_apikey  async                  mikrotik      null        

:put [$SECRET get rest_apikey]
# mikrotik

$SECRET remove rest_apikey
# 

:put [$SECRET get rest_apikey]
# no such item

and more specific example from above of using as in /tool/fetch for common "API Keys" (TLS with Bearer auth header):

{
# ...
:local headers "Authorization: bearer $[$SECRET get mtforumpw]"        
:local resp [/tool/fetch url="$url" http-method="$method" http-header-field="$headers" http-data=($payload) output="user" as-value]
:put $resp
# ...
}

abligh · Fri Dec 22, 2023 3:28 am

Updated the "persisted store using ppp secrets" script so it should work on both V6 and V7. Although again persisted variables are a "needed feature" – since this is still a hack, works well enough for my purposes, but no warranties here.

This is very useful. I think it would be possible to enhance this by storing usernames in the caller-id field. That would allow storing username/password pairs that could be looked up by "name". If this seems sensible I might have a go at that.

Securely storing apikey/tokens for /tool/fetch... Approaches? == $SECRET

Securely storing apikey/tokens for /tool/fetch... Approaches? == $SECRET

Re: Securely storing apikey/tokens for /tool/fetch... Approaches?

Re: Securely storing apikey/tokens for /tool/fetch... Approaches?

Re: Securely storing apikey/tokens for /tool/fetch... Approaches?

Re: Securely storing apikey/tokens for /tool/fetch... Approaches?

Re: Securely storing apikey/tokens for /tool/fetch... Approaches?

Re: Securely storing apikey/tokens for /tool/fetch... Approaches?

Who is online