We are trying to make API-calls to FreeIPAs / (Red Hats IdM) via the fetch tool. The procedure is done in two parts, one call to authenticate and get a cookie, and the other call is to run a api-call to update i.e. dns-records in our case. Below is a working example using curl, using headers to pass the cookie information (curl has a built-in cookie feature, that the fetch tool don’t have):
First we authenticate:
curl -s -k -i \
-H "Accept: text/plain" \
-H "Content-Type: application/x-www-form-urlencoded" \
-H "Referer: https://IPA-SERVER.DNS.NAME/ipa" \
-H "X-IPA-API-Version 2.247" \
--data-urlencode "user=IPA_API_USER" \
--data-urlencode "password=IPA_API_PASSWORD" \
"https://IPA-SERVER.DNS.NAME/ipa/session/login_password"
HTTP/1.1 200 Success
Date: Wed, 12 Feb 2025 11:11:54 GMT
Server: Apache/2.4.62 (Rocky Linux) OpenSSL/3.2.2 mod_auth_gssapi/1.6.3 mod_wsgi/4.7.1 Python/3.9
Set-Cookie: ipa_session=MagBearerToken=f0DSZeog3u%2f0XNiT2eDtCRO2gWMwj4cDLTAM00AIeBzDubBbkguD4lcU7ngxnxP0g%2fQDOvCryNLWjpPKp2gmNnGbTV0qKnY%2ftB0G%2bC9XqIqYiol7wkO72%2fn0fCKhniCEwdLaGBREvrzBIXAHsVzdbHdA%2bdnKGRfoHOS%2b6NpB%2bJlM8%2bZbpsleF4Pm6hr7BoXt9DA%2bW1XIMe4bVhUn5m3%2fOh9mLOUN33tXhur3GHSgYscyhDvGGv0cyThAbJOHRs%2bxUZ5pc5iuTrUDtKiOD4uFGSlOIrv%2bEAIR4%2bLA9PvDbUk%3d;path=/ipa;httponly;secure;
X-Frame-Options: DENY
Content-Security-Policy: frame-ancestors 'none'
Cache-Control: no-cache, private
Vary: Accept-Encoding
Transfer-Encoding: chunked
Content-Type: text/plain; charset=UTF-8
Then we send json-data to update the record:
curl -s -k -i \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-H "Referer: https://IPA-SERVER.DNS.NAME/ipa" \
-H "X-IPA-API-Version: 2.247" \
-H "Cookie: ipa_session=MagBearerToken=f0DSZeog3u%2f0XNiT2eDtCRO2gWMwj4cDLTAM00AIeBzDubBbkguD4lcU7ngxnxP0g%2fQDOvCryNLWjpPKp2gmNnGbTV0qKnY%2ftB0G%2bC9XqIqYiol7wkO72%2fn0fCKhniCEwdLaGBREvrzBIXAHsVzdbHdA%2bdnKGRfoHOS%2b6NpB%2bJlM8%2bZbpsleF4Pm6hr7BoXt9DA%2bW1XIMe4bVhUn5m3%2fOh9mLOUN33tXhur3GHSgYscyhDvGGv0cyThAbJOHRs%2bxUZ5pc5iuTrUDtKiOD4uFGSlOIrv%2bEAIR4%2bLA9PvDbUk%3d;path=/ipa;httponly;secure;" \
--data "{\"method\":\"dnsrecord_mod\",\"params\":[[\"ZONE_NAME\",\"RECORD_NAME\"],{\"arecord\":\"1.1.1.1\", \"dnsttl\": 300}]}" \
"https://IPA-SERVER.DNS.NAME/ipa/session/json"
HTTP/1.1 200 Success
Date: Wed, 12 Feb 2025 11:12:43 GMT
Server: Apache/2.4.62 (Rocky Linux) OpenSSL/3.2.2 mod_auth_gssapi/1.6.3 mod_wsgi/4.7.1 Python/3.9
X-Frame-Options: DENY
Content-Security-Policy: frame-ancestors 'none'
Cache-Control: no-cache, private
Vary: Accept-Encoding
Transfer-Encoding: chunked
Content-Type: application/json; charset=utf-8
{"result": {"result": {"arecord": ["1.1.1.1"], "dnsttl": ["300"], "idnsname": ["test"]}, "value": "test", "messages": [{"type": "warning", "name": "VersionMissing", "message": "API Version number was not sent, forward compatibility not guaranteed. Assuming server's API version, 2.254", "code": 13001, "data": {"server_version": "2.254"}}], "summary": null}, "error": null, "id": null, "principal": "IPA_API_USER@IPA_DOMAIN", "version": "4.12.2"}
The problem with the fetch tool arises when we want to extract the cookie-information from the headers, what I want is output like this.
/tool fetch url="https://IPA-SERVER.DNS.NAME/ipa/ui/" output=user-with-headers
status: finished
downloaded: 2KiB
duration: 1s
data: <!DOCTYPE html>, i.e
http-headers: Accept-Ranges: bytes
Cache-Control: no-cache, private, max-age=0
Content-Encoding: gzip
Content-Length: 741
Content-Security-Policy: frame-ancestors 'none'
Content-Type: text/html; charset=UTF-8
Date: Wed, 12 Feb 2025 10:43:11 GMT
Expires: Wed, 12 Feb 2025 10:43:11 GMT
Last-Modified: Wed, 21 Aug 2024 15:06:37 GMT
Server: Apache/2.4.62 (Rocky Linux) OpenSSL/3.2.2 mod_auth_gssapi/1.6.3 mod_wsgi/4.7.1 Python/3.9
Vary: Accept-Encoding
X-Frame-Options: DENY
Here is another example with a random Norwegian newspaper, but we can see the cookie-information from the header-output:
/tool fetch url="https://www.vg.no" output=user-with-headers
status: finished
downloaded: 20KiB
duration: 1s
data: <!DOCTYPE html>, i.e
http-headers: accept-ch: sec-ch-ua-model,sec-ch-ua-platform-version
Accept-Ranges: bytes
Age: 0
Cache-Control: max-age=0, must-revalidate, public
Connection: keep-alive
Content-Encoding: gzip
Content-Security-Policy: frame-ancestors https://admarket.no https://admarket.schibsted.se https://schibsted.dredition.aptoma.no/; upgrade-insecure-requests
Content-Type: text/html; charset=UTF-8
Date: Wed, 12 Feb 2025 11:44:46 GMT
ETag: W/"d456341b09be46d77967e93f95ba953d-br"
Link: <https://cmp.vg.no/unified/wrapperMessagingWithoutDetection.js>; rel=preload; as=script,... i.e
Permissions-Policy: unload=(), fullscreen=(self),ch-ua-model=*,ch-ua-platform-version=*
Set-Cookie: clientBucket=77; Expires=Wed, 26 Feb 2025 11:44:46 GMT; Domain=vg.no; Path=/; Secure; samesite=strict
strict-transport-security: max-age=15552000
Transfer-Encoding: chunked
Vary: Accept-Encoding,User-Agent
x-age: 103
x-cache: HIT:2888
x-clue: aHR0cHM6Ly9kaXNjb3JkLmdnL3Znbm8gb2cgc2kgJ2hlYWRlcm5lc2UnCg==
X-Content-Type-Options: nosniff
x-varnish-director: vg_frimand_udo
X-VG-TLSProxy: u89-tlsproxy-01.int.vgnett.no
X-VG-WebCache: oa68-varnish-02
X-XSS-Protection: 1
But when I try to do fetch tool-authentication, I receive no headers back:
/tool fetch url="https://IPA-SERVER.DNS.NAME/ipa/session/login_password" \
mode=https \
http-method=post \
http-header-field="Accept: text/plain,\
Content-Type: application/x-www-form-urlencoded,\
Referer: https://IPA-SERVER.DNS.NAME/ipa,\
X-IPA-API-Version: 2.247" \
http-data="user=IPA_API_USER&password=IPA_API_PASSWORD" \
output=user-with-headers
status: finished
downloaded: 0KiB
data:
http-headers:
The best clue I have is that with curl we cannot use “–head” to view the headers, we have to use “–include”, some internet comments explain that http-method post and head, cannot be combined.
I find one other forum-post that seems to do the same thing we are attempting:
http://forum.mikrotik.com/t/cookie-based-auth-using-fetch/172277/1
But he explicitly explains that with output=user-with-headers, he sees the header-information.
Any ideas/tips, anyone?