--- type: howto title: "Use the self-hosted Firecrawl instance (any client, any human)" summary: "How to actually call the tailnet-only self-hosted Firecrawl endpoint — REST API, CLI, and the two clients already wired up — as opposed to how to install/deploy it." tags: - type/howto - tool/firecrawl - domain/web-scraping scope: global last_updated: 2026-07-09 date: 2026-07-09 update_note: experience-driven related: - firecrawl-self-host-setup - firecrawl-self-hosted-capabilities source: ovh-prod --- # Using the Self-Hosted Firecrawl Instance For installing/deploying, see [[firecrawl-self-host-setup]]. For what works vs cloud, see [[firecrawl-self-hosted-capabilities]]. This note is for actually calling it — any human, any client, any language. ## Endpoint - `http://100.100.43.95:3002` — Tailscale tailnet IP of the ovh-prod VPS (`vps-5d79b132`). - `firecrawl.hyperthrive.io:3002` also resolves to the same tailnet IP for convenience — this is **not** a public route, the DNS name doesn't change the fact that the port is bound only to the tailnet interface (confirmed via `ss` — unreachable on the public IP). - Any caller (human or agent) must be joined to the same tailnet to reach it. ## No auth required Self-hosted Firecrawl has no working auth (Supabase auth is cloud-only — see [[firecrawl-self-hosted-capabilities]] for why). POST directly, no API key needed. Security is enforced entirely by the tailnet-only network bind, not by the app — never expose this port beyond the tailnet. ## Direct REST usage (any language/client) - Scrape: `POST /v1/scrape` — `{"url": "...", "formats": ["markdown"]}` - Crawl: `POST /v1/crawl` — `{"url": "..."}` (async, returns a job id; poll `GET /v1/crawl/{id}`) - Map: `POST /v1/map` — `{"url": "..."}` — discover URLs on a site - `/v1/search` and `/v1/extract` are **not reliably available** on this deployment — search needs a SearXNG instance (not deployed here), extract needs `OPENAI_API_KEY` (not currently set in this stack's `.env`). ```bash curl -s -X POST http://100.100.43.95:3002/v1/scrape \ -H 'Content-Type: application/json' \ -d '{"url":"https://example.com","formats":["markdown"]}' ``` ## Via firecrawl-cli On any machine with the CLI installed and on the tailnet: ```bash export FIRECRAWL_API_URL=http://100.100.43.95:3002 firecrawl scrape https://example.com firecrawl crawl https://example.com firecrawl map https://example.com ``` No `FIRECRAWL_API_KEY` needed once `FIRECRAWL_API_URL` points somewhere other than the cloud default — auth is skipped automatically. ## Clients already wired up (as of 2026-07-09) - **Desktop Claude Code** — official firecrawl plugin + `firecrawl-cli` installed; `FIRECRAWL_API_URL` set via `~/.credentials` (sourced into shell env). Verified working via `firecrawl scrape`. - **hermes01** (Nous Hermes agent on Proxmox) — native Firecrawl provider built into hermes-agent (`plugins/web/firecrawl/provider.py`) reads `FIRECRAWL_API_URL` directly. Set in `~/.hermes/.env` on hermes01, with `web.extract_backend: firecrawl` in `~/.hermes/config.yaml` — deliberately **not** `web.backend` (the shared fallback), which would also route web *search* to firecrawl and break it, since search needs SearXNG which isn't deployed. Verified end-to-end via a one-shot `hermes chat -z` extract call that returned real scraped markdown. - Both reach the endpoint over Tailscale directly — no public exposure, no per-client auth needed. ## What it's good for / not good for Real headless-Chromium JS rendering and multi-page crawl/map are genuine wins over a plain HTTP fetch (e.g. Claude Code's WebFetch). No anti-bot/proxy rotation, no FIRE-1 autonomous browsing, no working `/search` or `/extract` without extra setup (SearXNG / `OPENAI_API_KEY` respectively). Full comparison in [[firecrawl-self-hosted-capabilities]]. ## Related - [[firecrawl-self-host-setup]] — install/deploy and env-var reference - [[firecrawl-self-hosted-capabilities]] — feature parity vs cloud, value vs WebFetch