MCP: jevcrawl
by Rayen Mlayeh
Goal-directed web crawler MCP server that returns only the passages answering your agent's question, verbatim and with sources.
Local-first web MCP server for AI agents — search, fetch, crawl, extract and deep research with no API keys, no cloud round-trip and no per-query bill.
Set up this MCP server in your favorite AI agent — copy a ready-made configuration below.
wigolo gives a coding agent a single surface for everything web-related: search, fetch, crawl, extract, cache, find-similar, research and autonomous gather loops. The core tools need no API keys and nothing they touch leaves ~/.wigolo/, which makes it a practical alternative to metered search-and-scrape APIs for agents that browse heavily.
The toolset is unusually broad for a single server. search runs multi-engine web search across 18 direct adapters with rank fusion, ML reranking and an explainable per-result score, and accepts an array of queries for parallel breadth. fetch loads a URL through a tiered router that escalates from plain HTTP to a headless browser only when it meets an anti-bot challenge or an SPA shell, returning clean markdown with metadata and links; it also handles PDFs, single-heading sections, authenticated sessions and page actions such as click, type, scroll and screenshot. crawl walks multiple pages by BFS, DFS or sitemap with per-domain rate limits, robots.txt compliance and boilerplate dedup. extract pulls tables, metadata, JSON-LD, brand identity, named schemas or any custom JSON Schema, while diff and watch report what changed on a page since the last visit and can deliver changes to a webhook.
Setup is one command: npx wigolo init provisions the local engine, and --agents=claude-code,cursor also writes the MCP config for the agents you name. Supported targets include Claude Code, Cursor, Codex, Gemini CLI, OpenCode, VS Code, Windsurf, Zed and Antigravity, plus any MCP client, LangChain, CrewAI, LlamaIndex, the Vercel AI SDK, n8n or plain REST. Every tool also runs from the terminal, an interactive NDJSON shell and the SDKs.
Requires Node 20 or newer and roughly 1.5 GB of free disk on macOS, Linux or Windows. AGPL-3.0-only and currently in public beta.
by Rayen Mlayeh
Goal-directed web crawler MCP server that returns only the passages answering your agent's question, verbatim and with sources.
by SerpApi
Official SerpApi MCP server: real-time Google, Bing, YouTube and other search results as structured JSON or token-saving markdown.
by Crawlbase
Official Crawlbase MCP server giving agents live web access — fetch any URL as raw HTML, clean Markdown or a screenshot, with JS rendering and proxy rotation handled for you.
by Exa
Real-time web search and crawling built for AI — neural search with configurable tools and live content retrieval.