Replace characters in string (url encode)

Hi all!

I am trying to do a “/tool fetch url=”$urlstring" dst-path=“/$file” where $urlstring is a string with bunch of data including the different values gathered from “/system routerboard print”. This in turn is a poor mans way of making a http post to a server to register data about every router.

Problem is that many of the values contains spaces, hyphens and other characters that does not work well in url’s and they need to be escaped (url encoded). So, have anyone done that before? What I basically need is a way to replace different characters in a string, ie - with %2D, space with %20 etc.

I would be grateful for any help.

You can loop through the string, checking each character and substituting as needed. Add additional :if statements for any other desired characters. The starting and ending brackets are for copying / pasting into the terminal (because of :local variables), and they can be removed in your final script.

{
:local urlstring "http://www.mikrotik.com/foo bar/foo-bar/"
:local urlEncoded

:for i from=0 to=([:len $urlstring] - 1) do={ 
  :local char [:pick $urlstring $i]
  :if ($char = " ") do={
    :set $char "%20"
  }
  :if ($char = "-") do={
    :set $char "%2D"
  }
  :set urlEncoded ($urlEncoded . $char)
    
}
:put $urlEncoded
}

Thank you, just what i needed. I am still trying to get my head around the string functions in the script language :slight_smile:

Very good, skot!
I’m writing a script to receive SMS and send inbox text via http via “fetch” command.
I just need something to handle spaces in sms text…

What does the SMS text look like that you are receiving?

I solved the issue, when I was working at this.
I remember that it wil be necessary to reformat sms text by inserting “?” instead of sms spaces.
It was working well, but unfortunately the RB750UP I was working on get destroyed by some GSM modem issues, as you can see on those posts:
http://forum.mikrotik.com/t/rb750up-3g-usb-freezes-completely/54120/1
http://forum.mikrotik.com/t/rb750up-dead/53960/1
So I lost all my work (not saved yet) and I lost also any big interest in using mikrotik as SMS gateway, beacause Routeros is too much unreliable when used with GSM modems.
I noticed lot of issues also with microSD storage, smb service and also some issue with USB storage.
I tested lot of Routeros versions, on RB750UP and RB493G.
:frowning:

I tried to replace a url “0/24” from “192.168.1.0/24”

Unfortunately it doesnt work with your script.

I have tried to do a test with urlstring, it works. Only when comes to replacing IP it doesnt work. It only replace 1 character and not more than that?

Correct. That script above loops through the url one character at a time, replacing the ones you want. If you want to replace a substring, it will have to be done differently, something like this:

{
:local ip 192.168.1.0/24
:local ipSub [:pick $ip 0 [:find $ip "0/24"]]
:put $ipSub
:put ($ipSub."254/27")
}

I have written a function, based on this article, to urlEncode Strings.

:global urlEncode "   :local output \"\"
          :set input [:toarray \$input]
          :if ([:len \$input] > 0) do={
                    :local input1 [:tostr [:pick \$input 0]]
                    :for i from=0 to=([:len \$input1] - 1) do={ 
                              :local char [:pick \$input1 \$i]
                              :if (\$char = \" \") do={
                                      :set \$char \"%20\"
                              }
                              :if (\$char = \"-\") do={
                                     :set \$char \"%2D\"
                              }
                              :if (\$char = \"(\") do={
                                     :set \$char \"%28\"
                              }
                              :if (\$char = \")\") do={
                                     :set \$char \"%29\"
                              }
                              :if (\$char = \":\") do={
                                     :set \$char \"%3A\"
                              }

                              :set output (\$output . \$char)
                    }
                    :set output [:tostr \$output]
                    :set output [:toarray \$output]
          };"

To use it u place the following at the beginning of your script:

/system script run "Functions"
:global urlEncode

And at the point you want to urlEncode something the following:

:global Result ""
:local runFunc [:parse (":global Result; :local input \"Place your text to urlEncode here!\"; " . $urlEncode . ":set Result \$output")]
$runFunc

You can then pickup the urlencoded string in $Result

Hope it helps someone :wink:

Let me suggest to make it nicer :slight_smile:.

Instead of:

:if (\$char = \" \") do={
                                      :set \$char \"%20\"
                              }
                              :if (\$char = \"-\") do={
                                     :set \$char \"%2D\"
                              }
                              :if (\$char = \"(\") do={
                                     :set \$char \"%28\"
                              }
                              :if (\$char = \")\") do={
                                     :set \$char \"%29\"
                              }
                              :if (\$char = \":\") do={
                                     :set \$char \"%3A\"
                              }

You can use:

:local conversion {" "="%20";"-"="%2D";"("="%28";")"="%29";":"="%3A"}
:foreach old,new in=$conversion do={
  :if ($char=$old) do={
    :set $char $new;
  }
}

(writing it as “internal” part of script, so to substitute it in export you need to escape all " and $ characters, of course)

Also - check this out, this is how we now use the functions :slight_smile:. (please, see the “sticky” Functions and function parameters thread in this forum section!)

:global urlEncode do={
  :local output ""
  :local input [:toarray $1]
  :if ([:len $input] > 0) do={
    :local input1 [:tostr [:pick $input 0]]
    :for i from=0 to=([:len $input1] - 1) do={ 
      :local char [:pick $input1 $i]
      :local conversion {" "="%20";"-"="%2D";"("="%28";")"="%29";":"="%3A"}
      :foreach old,new in=$conversion do={
        :if ($char=$old) do={
          :set $char $new;
        }
      }
      :set output ($output . $char)
    }
    :set output [:tostr $output]
    :set output [:toarray $output]
  }
  :return $output
}

Then you put it in “Functions” script, and - when needed - you do the same in the beginning of your script:

/system script run "Functions"
:global urlEncode

But then, using it is much simpler, for example:

:local original "This is a (simple) string";
:local result [$urlEncode $original];
:put "The original text is:";
:put $original;
:put "And the escaped result is:";
:put $result;

Or even:

:put "This is a quickly escaped string: $[$urlEncode "Escape this: text"] - simple, isn't it?";

@dasiu: many thanks for pointing me to the optimized form of it :wink:

:local conversion {" "="%20";"!"="%21";"\""="%22";"#"="%23";"$"="%24";"%"="%25";"&"="%26";"'"="%27";"("="%28";")"="%29";"*"="%2A";"+"="%2B";","="%2C";"-"="%2D";"."="%2E";"/"="%2F";":"="%3A";";"="%3B";"<"="%3C";"="="%3D";">"="%3E";"\?"="%3F";"@"="%40";"["="%5B";"\\"="%5C";"]"="%5D";"^"="%5E";"`"="%60";"{"="%7B";"|"="%7C";"}"="%7D";"~"="%7E"}

the above line has more conversation codes now.

# ------------------- fURLEncode ----------------------
:global fURLEncode
:if (!any $fURLEncode) do={ :global fURLEncode do={
  :local Chars {" "="%20";"!"="%21";"\""="%22";"#"="%23";"$"="%24";"%"="%25";"&"="%26";"'"="%27";"("="%28";")"="%29";"*"="%2A";"+"="%2B";","="%2C";"-"="%2D";"."="%2E";"/"="%2F";":"="%3A";";"="%3B";"<"="%3C";"="="%3D";">"="%3E";"\?"="%3F";"@"="%40";"["="%5B";"\\"="%5C";"]"="%5D";"^"="%5E";"`"="%60";"{"="%7B";"|"="%7C";"}"="%7D";"~"="%7E"}
  :local URLEncodeStr
  :local Char
  :local EncChar
  :for i from=0 to=([:len $1]-1) do={
    :set Char [:pick $1 $i]
    :set EncChar ($Chars->$Char)
    :if (any $EncChar) do={
      :set URLEncodeStr ($URLEncodeStr . $EncChar)
    } else={
      :set URLEncodeStr ($URLEncodeStr . $Char)
    }
  }
  :return $URLEncodeStr
}}

Dollar char replacement does not work in this function

fixed version:

# ------------------- fURLEncode ----------------------
:global fURLEncode
:if (!any $fURLEncode) do={ :global fURLEncode do={
  :local Chars {" "="%20";"!"="%21";"\""="%22";"#"="%23";"%"="%25";"&"="%26";"'"="%27";"("="%28";")"="%29";"*"="%2A";"+"="%2B";","="%2C";"-"="%2D";"."="%2E";"/"="%2F";":"="%3A";";"="%3B";"<"="%3C";"="="%3D";">"="%3E";"\?"="%3F";"@"="%40";"["="%5B";"\\"="%5C";"]"="%5D";"^"="%5E";"`"="%60";"{"="%7B";"|"="%7C";"}"="%7D";"~"="%7E"}
  :local URLEncodeStr
  :local Char
  :local EncChar
  :for i from=0 to=([:len $1]-1) do={
    :set Char [:pick $1 $i]
    :set EncChar ($Chars->$Char)
    :if (any $EncChar) do={
      :set URLEncodeStr ($URLEncodeStr . $EncChar)
    } else={
      :if ($Char="\$") do={
        :set URLEncodeStr ($URLEncodeStr . "%24";)
      } else={
        :set URLEncodeStr ($URLEncodeStr . $Char)
      }
    }
  }
  :return $URLEncodeStr
}}

This mikrotik parser bug!

works correctly:
[admin@MikroTik] > :local arr1 ({}); :set ($arr1->“$”) “%24”; :put ($arr1->“$”)
%24

correct and not working!
[admin@MikroTik] > :local arr1 {“$”=“%24”}; :put ($arr1->“$”)

Wrong but next works!
But without one quotes: “$” work again
[admin@MikroTik] > :local arr1 {“$=”%24"}; :put ($arr1->“$”)
%24

Replaced with new version:
http://forum.mikrotik.com/t/rextended-fragments-of-snippets/151033/1