--- type: reference subtype: pattern/framework title: "Firecrawl self-hosted — feature parity, value vs WebFetch, and gotchas" summary: "What the open-source self-hosted Firecrawl actually delivers vs the cloud product and vs Claude Code's WebFetch — which features work, which are cloud-only (Fire-engine anti-bot, FIRE-1 agent), where the marketing claims break down, and the security/licensing gotchas." tags: - type/reference - tool/firecrawl - domain/web-scraping - domain/ai-agents scope: global last_updated: 2026-07-08 --- # Firecrawl Self-Hosted — Capabilities & Value Assessment Research snapshot 2026-07-08. Sources: [SELF_HOST.md](https://github.com/firecrawl/firecrawl/blob/main/SELF_HOST.md), [Self-Host Overhaul v1.5.0](https://github.com/firecrawl/firecrawl/discussions/1222), issues [#2257](https://github.com/firecrawl/firecrawl/issues/2257), [#2881](https://github.com/firecrawl/firecrawl/issues/2881), [mcp-server#126](https://github.com/firecrawl/firecrawl-mcp-server/issues/126). ## Feature parity: self-hosted vs cloud | Feature | Self-hosted | Notes | |---|---|---| | `/scrape` (JS rendering via Playwright) | ✅ | The core value — real headless Chromium | | `/crawl`, `/map`, batch scrape | ✅ | Full multi-page crawling works | | `/search` | ⚠️ | Only via a self-run SearXNG instance or your own search API key | | `/extract` (LLM extraction) | ⚠️ | Works, but requires your own `OPENAI_API_KEY` (OpenAI-compatible/Ollama OK) | | Actions (click/scroll), screenshot | ✅ (degraded reliability) | Basic Playwright actions | | PDF parsing | ⚠️ | Works; anti-bot-protected PDFs less reliable than cloud | | Stealth / anti-bot / residential proxies | ❌ mostly | No Fire-engine. v1.5.0 added `PROXY_*` env vars + `stealthProxy` flag but you must bring your own proxy pool. Self-host fails on aggressively protected sites where cloud succeeds (#2257) | | FIRE-1 / `/agent` autonomous browsing | ❌ | Cloud-only | | Browser sandbox, change tracking | ❌ / undocumented | Cloud-leaning | ## Value vs Claude Code's WebFetch (fact-check of Firecrawl's blog) - **JS rendering — real advantage.** WebFetch does a plain HTTP fetch; JS-only SPAs return empty shells. Firecrawl's headless Chromium fixes this, and it works self-hosted. - **Multi-page crawl/map — real advantage**, works self-hosted. WebFetch is one URL per call. - **Anti-bot / proxy rotation — cloud-only.** The blog markets this without disclosing that Fire-engine is excluded from self-host. The biggest omission in their comparison. - **Token efficiency — mostly marketing.** The circulated "80–94% savings" figures compare Firecrawl markdown vs *raw HTML*, not vs WebFetch's output, which is already markdown-converted by a fast model. No published benchmark compares the real pair. Additionally, the Firecrawl MCP server costs ~10–20k tokens of tool schema per session, which built-in WebFetch doesn't. Firecrawl only wins on tokens if you fetch many pages per session or need pages WebFetch can't render at all. - **Practical stance:** self-hosted Firecrawl complements rather than replaces WebFetch. Use Firecrawl for JS-heavy pages, crawls, and structured extraction; keep WebFetch for simple static pages (zero setup, no MCP overhead). ## Gotchas - **No real auth in self-host.** Supabase auth can't be configured self-hosted → auth is effectively bypassed (documented behavior, not a bug). Never expose to the public internet; keep LAN/Tailscale/Docker-internal, or add reverse-proxy auth. - **MCP server bug:** `firecrawl-mcp-server` demands `FIRECRAWL_API_KEY` even with `FIRECRAWL_API_URL` set to a self-hosted endpoint (mcp-server#126). Workaround: set any non-empty placeholder — the self-hosted server doesn't validate it. - **Bull queue admin UI unauthenticated** unless `BULL_AUTH_KEY` set. - **License:** AGPL-3.0 core (SDKs/UI MIT). Fine for private self-hosting; source-release obligation only if you offer a modified version as a network service. - **Actively maintained** (v1.5.0 self-host overhaul), but self-host permanently trails cloud on anti-bot robustness. ## Resource footprint Docker Compose stack (API + worker + Playwright/Chromium + Redis + Postgres/RabbitMQ). Floor ~2 GB RAM; realistic light personal use 4 GB RAM / 2 cores; comfortable 8 GB. Headless Chromium is the biggest consumer. Images total several GB on disk. ## Related - [[firecrawl-self-host-setup]] — how to install and wire it to Claude Code / agents