Why yq? Adventures in XML, (Thu, May 16th)

Category :

SANS Full Feed

Posted On :

I was recently asked to <ahem> “recover” a RADIUS key from a Microsoft NPS server.  No problem I think, just export the config and it’s all there in clear text right?

… yes, sort of …

The XML file that gets output is of course perfect XML, but that doesn’t mean it’s easy to read for a human, it’s as scrambled as my weekend eggs.  I got my answer, but then of course started to look for a way to get the answer in an easier way, something I could document and hand off to my client.  In other words, I started on the quest for a “jq” like tool for XML.  Often security work involves taking input in one text format and converting it to something that’s human readable, or more easily parsed by the next tool in the pipeline.  (see below)

XMLLint is a pretty standard one that’s in Linux, you can get it by installing libxml2.  Kali has it installed by default – usage is very straightforward:

xmllint < file.xml


cat file.xml | xmllint

There are a bunch of output options, but because it’s not-so windows friendly I didn’t dig to far – run man xmllint or browse here: https://gnome.pages.gitlab.gnome.org/libxml2/xmllint.html  if you need more than the basics on this.

However, finding something like this for Windows turned into an adventure, there’s a port of xmllint for Windows but it’s in that 10-year age range that makes me a bit leary to install it.  With a bit of googling I found yq.

This is a standalone install on most Linux distro’s (sudo apt-get install yq or whatever), and has a few standard install methods for windows:

you can just download the binary and put it in your path
choco install yq
winget install –id MikeFarah.yq

yq is written to mimic jq like you’d expect from the name, but will take json, yaml, xml, csv and tsv files.  It’s not as feature-heavy as jq, but it’s got enough, and let’s face it, most of us use these for pretty print output, so that we can grep against that anyway.
I especially liked it for today’s problem because I can adjust the indent, since the NPS XML export has a fairly deep heirarchy I went with an indent of 1:

type nps-export.xml | yq –input-format xml –output-format xml –indent 1 > pretty.xml

A quick peek at the file found me my answwer in the pretty (and grep-able) format that I wanted.  A typical RADIUS Client section looks lke:

<Clients name=”Clients”>
<Client_ _Template_Guid=”_Template_Guid” xmlns:dt=”urn:schemas-microsoft-com:datatypes” dt:dt=”string”>{00000000-0000-0000-0000-000000000000}</Client_>
<IP_Address xmlns:dt=”urn:schemas-microsoft-com:datatypes” dt:dt=”string”>IP.Address.Goes.Here</IP_Address>
<NAS_Manufacturer xmlns:dt=”urn:schemas-microsoft-com:datatypes” dt:dt=”int”>0</NAS_Manufacturer>
<Opaque_Data xmlns:dt=”urn:schemas-microsoft-com:datatypes” dt:dt=”string”></Opaque_Data>
<Radius_Client_Enabled xmlns:dt=”urn:schemas-microsoft-com:datatypes” dt:dt=”boolean”>1</Radius_Client_Enabled>
<Require_Signature xmlns:dt=”urn:schemas-microsoft-com:datatypes” dt:dt=”boolean”>0</Require_Signature>
<Shared_Secret xmlns:dt=”urn:schemas-microsoft-com:datatypes” dt:dt=”string”>SuperSecretSharedKeyGoesHere</Shared_Secret>
<Template_Guid xmlns:dt=”urn:schemas-microsoft-com:datatypes” dt:dt=”string”>{1A1917B8-D2C0-43B3-8144-FAE167CE9316}</Template_Guid>

Or I could dump all the shared secrets with the associated IP Addresses with:

type pretty.xml | findstr “IP_Address Shared_Secret”


cat pretty.xml | grep ‘IP_address|Shared_Secret’

After all that, I think I’ve found my go-to for text file conversions – in particular xml or yaml, especially in Windows ..

Full details on these two tools discussed:

If you’ve got a different text formatter (or un-formatter), or if you’ve used xmllint or yq in an interesting way, please let us know about it in our comment form!

Rob VandenBrink
[email protected]

(c) SANS Internet Storm Center. https://isc.sans.edu Creative Commons Attribution-Noncommercial 3.0 United States License.