Self-Hosting Firecrawl (Docker Compose)

Prereqs: Docker + Compose, 4 GB RAM headroom (2 GB absolute floor), a few GB disk for images. See firecrawl-self-hosted-capabilities for what you get/don't get.

Install

git clone https://github.com/firecrawl/firecrawl.git
cd firecrawl
cp apps/api/.env.example .env   # check SELF_HOST.md for current layout
docker compose build && docker compose up -d
# API serves on http://localhost:3002

Follow the current SELF_HOST.md — the v1.5.0 self-host overhaul reworked the compose file and added a production-ready image; details drift.

Env vars that matter

Var	Why
`PORT` / `HOST`	Default 3002 / 0.0.0.0
`OPENAI_API_KEY`	Only needed for `/extract` / JSON LLM extraction; OpenAI-compatible endpoints and Ollama work
`BULL_AUTH_KEY`	Secures the otherwise-open Bull queue admin UI — always set
`POSTGRES_USER/PASSWORD/DB`	Change from dev defaults
`PROXY_SERVER`, `PROXY_USERNAME`, `PROXY_PASSWORD` + `stealthProxy`	Bring-your-own proxies (no built-in pool self-hosted)
`MAX_CPU`, `MAX_RAM`	Reject new jobs under load
SearXNG endpoint	Only if you want `/search`

Security posture (critical)

Self-host has no working auth (Supabase auth is cloud-only; the "Supabase client not configured" warning means auth is bypassed). Therefore:

Bind to LAN/Tailscale/Docker-internal networks only — never the public internet.
On a public VPS behind Traefik: do NOT give it a public router; keep it on an internal Docker network only, or add reverse-proxy basic-auth/forward-auth if a route is unavoidable.

Smoke test

curl -s -X POST http://localhost:3002/v1/scrape \
  -H 'Content-Type: application/json' \
  -d '{"url":"https://example.com","formats":["markdown"]}'

Wiring to Claude Code / agents — CLI + skill (preferred over MCP)

The official CLI (npm i -g firecrawl-cli, repo github.com/firecrawl/cli) supports self-host cleanly and avoids the ~10–20k-token MCP schema tax:

export FIRECRAWL_API_URL=http://<host>:3002 (or --api-url) — when the URL isn't https://api.firecrawl.dev, auth is automatically skipped; no key or placeholder needed.
Commands: scrape, search, map, crawl, agent, interact, monitor.
Official Claude Code integration: firecrawl/firecrawl-claude-plugin (install via /plugin, search "firecrawl") wraps the CLI as skills — it inherits FIRECRAWL_API_URL, so it works self-hosted. Requires the CLI installed on the host machine. firecrawl/skills is a separate repo for embedding Firecrawl in application code.
Sensible routing: Firecrawl first, WebFetch as fallback when Firecrawl fails or isn't needed.

Wiring via MCP (heavier alternative)

claude mcp add firecrawl \
  -e FIRECRAWL_API_URL=http://<host>:3002 \
  -e FIRECRAWL_API_KEY=self-hosted-placeholder \
  -- npx -y firecrawl-mcp

FIRECRAWL_API_KEY must be a non-empty placeholder even for self-host — the MCP server errors without it (mcp-server#126); the self-hosted API doesn't validate it.
MCP schema costs ~10–20k tokens/session — add the server per-project where scraping is actually used, not globally.
Any non-MCP agent (e.g. Hermes) can hit the REST API directly (POST /v1/scrape, /v1/crawl) — same surface as cloud, so official SDKs work by overriding the API URL.

Ops notes

Update: git pull && docker compose build && docker compose up -d (watch the v1.x release notes; compose layout has changed before).
Keep WebFetch as the default for simple static pages; route JS-heavy pages, crawls, and structured extraction to Firecrawl.

firecrawl-self-hosted-capabilities — feature parity and value assessment

4.3 KiB Raw Blame History Unescape Escape