JSON - Jq Command line processor

About

jq is a command line json parser.

Data in jq is represented as streams of JSON values.

Management

Get / Reference

by name

• produces the value at the properties key “foo”

jq .["foo$"] # The full qualified name
jq '.foo' # Short name

• or

.foo|.bar  (.foo.bar is equivalent)

• Optional (no error)

jq '.foo?'  

by key

Arrays are zero-based

.[1] # returns the second element.

• Slice 10 included, 15 excluded

.[10:15]

• No Error

.[]?

by Alias

• as - An Json object can be saved for later use with the as keyword in a pipe

jq '. as $alias | $alias '

Function

Map (To Array)

Map(x) will run the input filter x for each element of the input array, and return the outputs in a new array

Example: The Map function returns the value of Json objects array in an array

[
    {"name":"JSON", "good":true}, 
    {"name":"XML", "good":false}
]

• Process

jq 'map(.name)' input.json

[
  "JSON",
  "XML"
]

To BASH array

The transformation below is equivalent to the

declare -A name=( [key1]=value1 [key2]=value2 … )

Note that the -r is fucking important.

eval "declare -A data=($(echo '{
    "total_count": "203",
    "bundle_count": "10"
}' | jq -r '. | to_entries | .[] | @sh "[\(.key)]=\(.value)"'))"
echo ${data[total_count]}
echo ${data[bundle_count]}

To list

• Create a list

[
    {"name":"JSON", "good":true}, 
    {"name":"XML", "good":false}
]

• Process

jq '.[] | .name, .good' input.json

"JSON"
true
"XML"
false

Pipe

The | operator combines two filters by feeding the output(s) of the one on the left into the input of the one on the right.

[
    {"name":"JSON", "good":true}, 
    {"name":"XML", "good":false}
]

• Process

jq '.[] | .name' input.json

"JSON"
"XML"

Join (From Array to String)

From an array

[
  "1",
  "2"
]

join (",")

"1,2"

grouping operator

jq '(. + 2) * 5'

input: 1 output 15

Get keys

Keys

Keys Returns:

• the index of an array
• or the keys of an object

keys

To Entry

• In an array of object, get the keys by pivoting a key to a value with the to_entries

cat myJson.json | jq '."foo" | to_entries[] | .key

To Array from Keys

• From an array of object, returns the keys in an array

[
  { key1:Hello;key2:Nico },
  { key1:Hello;key2:Moi }
]

jq 'map(keys) | add | unique' input.json

• Return

[
  key1,
  key2
]

Output

Raw string

jq -r # To get the result as a raw string

Bash/Shell output

The -r is fucking imporant.

jq -r ".Accounts[].Id | @sh"
# or
jq -r '.SITE_DATA | to_entries | .[] | .key + "=" + (.value | @sh)'

Csv

Don't forget the -r options

Without headers

[
    {"name":"JSON", "good":true}, 
    {"name":"XML", "good":false}
]

• Process (@csv need an array of an array as input)

jq -r '.[] | [.name, .good] | @csv' input.json

"JSON",true
"XML",false

With headers

jq -r '(map(keys) | add | unique) as $cols | map(. as $row | $cols | map($row[.])) as $rows | $cols, $rows[] | @csv'

See https://stackoverflow.com/questions/32960857/how-to-convert-arbirtrary-simple-json-to-csv-using-jq