I'm constantly looking for ways to improve my development workflow. I couldn't say that I am obsessed with the concept of efficiency, rather I am just a lazy person! If something could be done easier (with help of new tools or automation) and rough cost-benefit analysis shows that it is worth doing I'd give it a try.
Recent examples:
- mergepbx to not bother with project file merges.
- Sourcery to generate some of the code I'm really not keen to write over and over again. More on this tool next time.
The task
Currently I'm working on a fairly complicated feature which involves processing and interpreting deeply nested JSON payload. Even though there is a bit of documentation I prefer to see the data myself and infer as much of requirements directly from it. Luckily, API developers have created a test endpoint which returns all the supported "forms". It's quite big payload:
wc -l payload.json
> 7481
{
sections: {
...
type: "..."
alternatives: {
...
fields: {
...
group: {
...
}
}
}
}
}
First assumption I wanted to verify is if sections
have more than 1 nested alternatives
and if they do I wanted to quickly find this section to inspect visually (just knowing its type
should be enough for this).
Rather than just browsing 7k lines json and believe that I haven't missed something I decided to use jq.
jq
jq
is described "a lightweight and flexible command-line JSON processor".
After quickly studying the manual, I've figured how to get the answer:
jq '.sections[] | .type + ", alternatives: " + (.alternatives | length | tostring)' payload.json
"ssn, alternatives: 1"
"identity_upload, alternatives: 1"
"business_owners, alternatives: 2"
...
"address_upload, alternatives: 1"
Good! Then I wanted to know how many fields each section had:
jq '.sections[] | .type + ", fields: " + (.alternatives[] |.fields | length | tostring)' payload.json
"ssn, fields: 2"
"identity_upload, fields: 3"
"business_owners, fields: 6"
"business_owners, fields: 1"
...
"address_upload, fields: 2"
I've spent not more than 10 minutes writing these predicates. Well, I haven’t figured yet how to make more complex queries involving select(.type == "")
, but I was able to analyse my assumptions about the payload structure. I am certain next time jq
would come in handy again, say, to determine how many distinct values a field can have, etc.