Skip to content
Firecrawl Scrape

Firecrawl Scrape Skill

Firecrawl's skill for reading known web pages with its CLI: scrape a URL to markdown, links or schema-typed JSON, batch-scrape URL lists, cap PDF pages, and discover and run data-provider tools for structured results.

GitHub Docs

Install

Install with the skills CLI

One command for Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf and dozens of other agents. Needs Node.js; the skill is fetched straight from firecrawl/cli.

$ npx skills add firecrawl/cli --skill firecrawl-scrape

Run it in a terminal. The CLI finds the agents installed on your machine, asks which ones to add the skill to, and whether to install it into the folder you ran it from or for all your projects.

Other ways to install

Copy the whole skill folder — SKILL.md and any scripts or references beside it — into ~/.claude/skills/firecrawl-scrape/, where Claude Code picks it up in every project. Only git is needed.

tmp=$(mktemp -d) && \
  git clone --depth 1 --branch main https://github.com/firecrawl/cli.git "$tmp" && \
  mkdir -p ~/.claude/skills && \
  cp -R "$tmp"/skills/firecrawl-scrape ~/.claude/skills/firecrawl-scrape

No scripts or references next to it? Saving the SKILL.md alone as ~/.claude/skills/firecrawl-scrape/SKILL.md works too.

Using another agent? Its personal skills folder
Codex
~/.agents/skills/
Cursor
~/.cursor/skills/
GitHub Copilot
~/.copilot/skills/
Gemini CLI
~/.gemini/skills/
Windsurf
~/.codeium/windsurf/skills/
OpenCode
~/.config/opencode/skills/

About

  • Scrape a URL to markdown, full page, links or several formats at once
  • Schema-typed JSON extraction with --schema or --schema-file
  • batch-scrape jobs for URL lists, with webhooks and per-URL schemas
  • Discover, inspect and execute Firecrawl data-provider tools by contract
  • PDF page budgets with --max-pages and large-result recovery by job ID
  • Saves output to .firecrawl/ files for grep-and-read workflows

Firecrawl Scrape is one of the focused skills shipped in Firecrawl's own CLI repository, next to the umbrella firecrawl skill and siblings for search, interaction and downloads. It covers the moment an agent already knows which page or tool it wants and needs the content: it runs firecrawl scrape and saves the result to a file under .firecrawl/ instead of pasting it into the conversation.

The quick-start covers the common cases: main-content markdown by default or the full page with --full-page, waiting for JavaScript with --wait-for, several URLs at once, markdown and links together, and a targeted --query question about a page. For typed data it passes a JSON schema (inline or from a file) with the json format. For longer lists it moves to batch-scrape, which starts a job you check later, reads URLs from a file and can apply one schema to every page or call a webhook.

Beyond plain pages, the skill explains Firecrawl's catalogue of data-provider tools: discover them with search and list, read the selected tool's contract (required inputs, examples, response key, pagination), then execute it through scrape with exactly the declared fields, checking each result for errors. It also covers recovering large results by job ID rather than repeating a paid call, and limiting PDFs with --max-pages, since PDFs cost one credit per parsed page. Tips favour scraping to a file and grepping it over --query, and escalating to interact when a page needs clicks or forms. It requires the Firecrawl CLI (firecrawl or npx firecrawl-cli) and a Firecrawl account. ISC licensed.

SKILL.md

The instructions your agent loads, as published by Firecrawl.

SKILL.md
name
firecrawl-scrape
description
Read a known webpage or execute a discovered workflow or data-provider capability. Use for page content or structured results once the URL or tool is selected.
allowed-tools
- Bash(firecrawl *) - Bash(npx firecrawl-cli *)

firecrawl scrape

Read a URL for page content, or execute a selected provider tool for structured data. Discover tools with search and inspect their inputs with list before execution. Multiple URLs can be scraped concurrently.

For structured datasets, first check for a suitable workflow or data provider using the [search skill](../firecrawl-search/SKILL.md). Read a known page directly; reuse a selected contract instead of repeating discovery.

Quick start
# Basic markdown extraction
firecrawl scrape "<url>" -o .firecrawl/page.md

# Keep the whole page, including nav/footer (main content only is the default)
firecrawl scrape "<url>" --full-page -o .firecrawl/page.md

# Wait for JS to render, then scrape
firecrawl scrape "<url>" --wait-for 3000 -o .firecrawl/page.md

# A few URLs (markdown only; each saved to .firecrawl/; -o is ignored)
firecrawl scrape https://example.com https://example.com/blog https://example.com/docs

# Get markdown and links together
firecrawl scrape "<url>" --format markdown,links -o .firecrawl/page.json

# Ask a question about the page
firecrawl scrape "https://example.com/pricing" --query "What is the enterprise plan price?"

Run firecrawl scrape --help for the full option list.

Done when: the page content or provider result has been checked for errors and inspected in bounded sections to answer the request. Preserve source links and disclose partial results.

Structured JSON from a page

Pass a JSON schema with the json format to get typed fields instead of markdown:

firecrawl scrape "<url>" --format json \
  --schema '{"type":"object","properties":{"plan":{"type":"string"},"price":{"type":"string"}},"required":["plan"]}' \
  -o .firecrawl/pricing.json

# Large schemas: keep them in a file
firecrawl scrape "<url>" --format json --schema-file schema.json -o .firecrawl/page.json

The result is under .json in the output. --schema only applies when json is in --format; add other formats with commas (--format markdown,json). For the same schema across many URLs, use batch-scrape --schema below.

Many known URLs

Use firecrawl batch-scrape for a list of known URLs when you need formats other than markdown, JSON extraction, a webhook, or a job you can check later. Keep the multi-URL scrape above for a few pages of markdown.

# Start a job (prints a job ID), then check it later
firecrawl batch-scrape https://example.com/a https://example.com/b
firecrawl batch-scrape <job-id>

# Wait and save every document as JSON (urls.txt: one URL per line, # comments ok)
firecrawl batch-scrape --urls-file urls.txt --format markdown,links --wait -o .firecrawl/batch.json

# Same extraction for every URL (--schema or --prompt adds the json format)
firecrawl batch-scrape --urls-file urls.txt --wait \
  --schema '{"type":"object","properties":{"title":{"type":"string"}}}' -o .firecrawl/batch.json

Results are in .data[], one document per URL with metadata.sourceURL. Add --webhook <url> to get notified, --ignore-invalid-urls to skip bad entries, and --cancel with a job ID to stop a job. Run firecrawl batch-scrape --help for all options.

Find tools, inspect inputs, and get help

Use the CLI help to check supported options rather than guessing:

firecrawl search --help
firecrawl list --help
firecrawl scrape --help

Domain discovery with --domain-tools returns tool summaries by default. Add --tool-detail full for contracts upfront, or inspect one selected tool with list as shown below. Use --tool-detail compact for only provider, capability and description; inspect by those two IDs with list. Summary remains the default. Prefer full when several related contracts will be needed immediately.

For structured data, search for the task, inspect a matching tool's contract, then execute with the exact input fields it declares:

# Web + domain matching + semantic tools
firecrawl search '<user question>'

# Semantic tools only
firecrawl search alexandria '<user question>'

# Categories → providers → tools → contract
firecrawl list
firecrawl list <category-id> --category
firecrawl list <provider-id>
firecrawl list <provider-id> <capability-id> --pretty

# Execute a tool
firecrawl scrape <provider-id>/<capability-id> --options '<JSON matching the selected contract>'

Normal search includes web results and tool matches; search alexandria searches tools only. list <provider> <capability> --pretty shows the selected contract; use --json for machine-readable output. To browse progressively, use list, then list <category> --category, then list <provider>. Search and list do not execute the selected provider tool. Read only the contracts needed for the task; use returned identifiers rather than guessing them.

Read the expanded contract before building inputs or parsing results:

  • required: true requires that input; each requiresOneOf group requires at least one member, not all of them.
  • Selected-contract inspection already requests examples. Read the singular example.request and example.response when present; an empty request can be valid for tools with optional inputs.
  • response.key identifies the records field inside data.alexandria[i].data; an empty key means that data object itself. Do not assume every provider returns records.
  • Provider pagination differs from catalogue next: use the contract's continuation input and the returned page/cursor, preserve filters, and stop at its exhaustion signal. paginated: true alone does not specify that mapping.
Execution and large results

URL scraping does not execute provider tools automatically. Use exact discovered input fields and resolve record IDs with lookup tools rather than inventing them. Check each data.alexandria[] result for errors, not just the outer success flag.

If the client reports an output/context limit, the upstream request may have succeeded. Preserve the request or scrape ID and recover the retained result before repeating the provider call. For large datasets and PDFs, save output with --json -o when a local filesystem is available and inspect bounded sections with jq or other file tools. Keep stderr separate from JSON stdout; do not merge streams with 2>&1 when piping to a JSON parser. Where remote processing is preferable, use firecrawl scrape firecrawl/bash to select from a retained result. Read [large-result recovery](references/large-results.md) for IDs, command examples, expiry, and errors. This is explicit recovery, not automatic overflow detection.

PDFs and page budgets

PDFs cost 1 credit per parsed page. Use --max-pages (an integer from 1 to 10000) to limit PDF parsing, especially for large or unknown documents:

firecrawl scrape "https://example.com/report.pdf" --max-pages 5 --json -o .firecrawl/report.json

The cap applies to each PDF, not the whole command or total credits. Extra formats and options can add charges. The CLI does not quote page counts or costs before execution. Use JSON output to inspect the returned metadata.numPages (parsed), metadata.totalPages (document total), and metadata.creditsUsed when present; a smaller parsed count means the result is partial.

Tips
  • Prefer plain scrape over `--query`. Scrape to a file, then use grep, head, or read the markdown directly — you can search and reason over the full content yourself. Use --query only when you want a single targeted answer without saving the page (costs 5 extra credits).
  • Scrape handles static pages and JS-rendered SPAs. Escalate to interact when the page needs interaction (clicks, form fills, pagination) or scrape misses content.
  • Multiple URLs are scraped concurrently — check firecrawl --status for your concurrency limit. This mode saves markdown only and ignores -o; other requested formats are dropped. If markdown wasn't requested, the whole JSON response is written into the .md file. Use batch-scrape when you need other formats or JSON extraction.
  • Single format outputs raw content. Multiple formats (e.g., --format markdown,links) output JSON.
  • Always quote URLs — shell interprets ? and & as special characters.
  • Naming convention: .firecrawl/{site}-{path}.md
See also
  • [firecrawl-search](../firecrawl-search/SKILL.md) — find pages when you don't have a URL
  • [firecrawl-interact](../firecrawl-interact/SKILL.md) — when scrape can't get the content, use interact to click, fill forms, etc.
  • [firecrawl-download](../firecrawl-download/SKILL.md) — bulk download an entire site to local files
  • firecrawl-build-scrape — building scrape into an app instead of running it here
Install
New

Firecrawl's official routing skill for its CLI: lets an agent search the web, scrape and map sites, crawl sections, monitor pages for changes and interact with pages behind clicks or logins, saving clean markdown to .firecrawl/.

Integrate Firecrawl's /scrape endpoint into product code — turn a known URL into markdown, HTML, links, metadata or screenshots, with sensible defaults baked in.

Skill: PDF

by Anthropic

Read, create, and manipulate PDF files — extract text and tables, merge, split, watermark, fill forms, encrypt, and OCR scanned pages.

Browse all skills →