What is jq? Processing JSON on the command line explained

jq is a lightweight command-line program that parses, filters, and transforms JSON using a small purpose-built query language. It reads JSON from standard input or a file, applies a filter you supply, and writes JSON (or raw text) to standard output. Because it behaves like any other Unix filter, jq slots directly into shell pipelines alongside tools such as curl, grep, and sort, which makes it the standard way to work with JSON from the terminal or inside scripts.

Key takeaways

• jq is a standalone binary, not a language library, and must be installed separately.
• Its core concept is the filter: an expression that takes JSON in and produces JSON out.
• The identity filter . pretty-prints and validates JSON with no other change.
• Use -r for raw string output and -c for compact single-line output.
• Filters compose with the pipe |, mirroring how shell pipelines chain commands.
• jq never edits files in place; redirect to a temporary file to persist changes.

What problem does jq actually solve?

APIs, config files, and structured logs increasingly speak JSON. Extracting a single field or reshaping a document with generic text tools like grep and sed is brittle because JSON is nested and whitespace-insensitive. jq understands JSON structurally, so you can address data by key and index rather than by line and column.

A typical use is pulling one value out of an API response:

curl -s https://api.example.com/user/42 | jq '.name'

Here curl fetches the document and jq extracts the name field. The filter is quoted so the shell does not interpret the special characters.

Because jq is a well-behaved filter, its output feeds naturally into the rest of the toolchain. With the raw flag it emits plain lines that xargs, while read loops, or sort can consume, which lets you extract identifiers with jq and act on each one with another command. That composability, rather than any single feature, is what makes jq feel like a native part of the shell rather than a separate program you shell out to.

How do jq filters work?

Everything in jq is a filter. The simplest is the identity filter ., which returns its input unchanged but pretty-prints it:

echo '{"a":1,"b":2}' | jq '.'

You navigate into structures by chaining accessors. .user.email descends two levels; .items[0] takes the first array element; .items[] iterates over every element, emitting each as a separate result.

Filters combine with the pipe operator, exactly like a shell pipeline but operating on JSON values:

jq '.users[] | .email'

This iterates the users array and, for each user, emits the email field.

Common building blocks

• .foo — access object key foo.
• .foo? — access foo but stay silent if the input is not an object.
• .[] — iterate an array or object’s values.
• select(condition) — pass through only inputs that match.
• map(f) — apply filter f to each element of an array.
• length, keys, has("k") — inspect structures.

How do I reshape JSON into a new structure?

jq can construct new objects and arrays. Wrapping an expression in {} builds an object; [] builds an array. This is how you slim a large response down to just the fields you need:

jq '.users[] | {name: .name, city: .address.city}'

To collect all results back into a single array rather than a stream of separate objects, wrap the whole filter in brackets:

jq '[.users[] | {name, city: .address.city}]'

Note the shorthand {name}, which is equivalent to {name: .name}.

How do I filter and search with select?

The select function keeps only inputs for which a condition is true, discarding the rest. Combined with iteration it works like a structural grep:

jq '.orders[] | select(.total > 100)'

You can combine conditions with and, or, and not, and test membership or string content:

jq '.users[] | select(.active and (.role == "admin"))'

Because select emits nothing for non-matching items, chaining it before a constructor is a clean way to build a filtered, reshaped result set.

What are the most useful command-line flags?

jq’s flags change how input is read and output is written. These are the ones you will reach for constantly.

The --arg flag matters for correctness and safety: it binds a shell value to a jq variable as a proper JSON string, avoiding the quoting mistakes that arise from splicing values into the filter text.

name="Ada Lovelace"
jq -n --arg n "$name" '{greeting: ("Hello, " + $n)}'

How do I aggregate and summarize data with jq?

Beyond extraction, jq can compute summaries over arrays, which is where it starts to feel like a query language. Three functions do most of the work: map transforms elements, add sums a list of numbers, and group_by partitions a list by a key.

To total a numeric field across an array:

jq '[.orders[].total] | add'

The inner [...] collects every total into one array, then add reduces it to a single number. To count how many orders each customer placed, group and then measure each group:

jq 'group_by(.customer) | map({customer: .[0].customer, count: length})'

For fully custom aggregation, reduce walks a stream while accumulating a value, similar to a fold in functional languages:

jq 'reduce .items[] as $i (0; . + $i.price)'

Here the accumulator starts at 0 and each item’s price is added to it. A frequent pitfall is running a numeric function over a stream instead of an array; wrap the stream in brackets first so add receives one array rather than many separate values.

What are common jq mistakes and pitfalls?

• Forgetting quotes. The filter contains characters the shell treats specially. Always single-quote the filter, and use --arg for dynamic values.
• Confusing a stream with an array. .items[] emits many values; if a later stage expects one array, wrap it as [.items[]].
• Expecting in-place edits. jq writes to stdout only. To update a file, write to a temp file and move it back:

jq '.version = "2.0"' config.json > config.tmp && mv config.tmp config.json

• Silent failures on missing keys. Accessing a key on a non-object raises an error; use the ? operator or // default to supply a fallback, as in .nickname // "unknown".
• Assuming raw output is JSON. With -r, the output is plain text and no longer safe to pipe into another jq stage expecting JSON.

How does jq compare to alternatives?

jq is the most widely used tool, but it is not the only option. The right choice depends on the data format and whether you want a dependency.

For pure JSON in shells and CI pipelines, jq remains the default because it is fast, dependency-free at runtime, and portable across platforms.

Frequently asked questions

Is jq installed by default?

No. jq is a separate utility you install via a package manager such as apt, dnf, brew, or by downloading a static binary. It is not part of the POSIX toolset.

What is the difference between jq and JSON.parse?

JSON.parse is a language function that loads JSON into memory; jq is a standalone command-line program with its own filter language for querying and transforming JSON in the shell.

Does jq modify files in place?

No. jq writes to standard output and never edits files in place. To update a file you redirect output to a temporary file and move it back over the original.

How do I stop jq from adding color codes?

jq only colorizes when output goes to a terminal. When piped or redirected it emits plain text, and you can force monochrome with the -M flag.

Can jq read multiple JSON objects from one stream?

Yes. jq processes a stream of whitespace-separated JSON values, applying your filter to each one, which is ideal for newline-delimited JSON logs.

Why does jq print quotes around my string output?

By default jq emits valid JSON, so strings are quoted. Use the -r (raw) flag to print string values without surrounding quotes.

Is jq safe to use on untrusted JSON?

Parsing untrusted JSON with jq is generally safe because the filter language has no file or network access, but always validate the shape of data before acting on extracted values.