---
type: reference
title: jq --stream for Huge JSON Files
summary: How to use jq's --stream mode to process JSON files too large to load into memory whole.
tags:
  - type/reference
  - tool/jq
  - domain/data-processing
scope: global
last_updated: 2026-06-28
date: 2026-06-28
---

`jq` normally parses an entire JSON document into memory before filtering it, which falls
over on multi-gigabyte files. `--stream` mode avoids this by emitting a flat sequence of
`[path, leaf-value]` pairs as it parses, instead of building the full in-memory tree.

## Basic usage

```
jq --stream 'select(.[0][0] == "records")' huge-file.json
```

This walks the file incrementally and only ever holds the current path/value pair in memory,
not the whole document.

## Reconstructing structure

Streamed output is flat, so reassembling a filtered subset back into normal JSON needs
`fromstream` on the way out:

```
jq -n --stream 'fromstream(inputs | select(.[0][0] == "records"))' huge-file.json
```

## When to reach for this

- The file is larger than you're willing to hold in memory (multi-GB exports, log dumps).
- You only need a small slice of a huge document (e.g., one array's worth of records) and
  don't want to pay the cost of parsing the rest.
- Note that `--stream` filters are noticeably slower per-byte than plain `jq` on data that
  *does* fit in memory — it's a memory/time tradeoff, not a free win.