Web Scraping & Data Listings
Compare the best open-source listings for web scraping & data. Crawl, scrape and extract structured data. Ranked by GitHub stars, with install commands, what each one can touch and scan reports where available.
Boss Zhipin Scraper
Boss直聘爬虫 / BOSS直聘职位数据抓取工具，基于 Chrome CDP 协议复用真实登录态，绕过字体反爬，输出明文薪资 JSON/CSV + 薪资技能分析。A Chrome-CDP-based BOSS Zhipin job scraper/crawler.
Duckduckgo MCP Server
A Model Context Protocol (MCP) server that provides web search capabilities through DuckDuckGo, with additional features for content fetching and parsing.
MCP Searxng
Private web search for AI assistants via SearXNG — supports Claude, Cursor, and any MCP client
Crw
Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.
Scrapecraft
🤖 AI-powered web scraping editor with visual workflow builder. Build, test & deploy web scrapers using natural language. Powered by ScrapeGraphAI & LangGraph.
Mantic Sh
A structural code search engine for Al agents.
Kindly Web Search MCP Server
Kindly Web Search MCP Server: Web search + robust content retrieval for AI coding tools (Claude Code, Codex, Cursor, GitHub Copilot, Gemini, etc.) and AI agents (Claude Desktop, OpenClaw, Hermes, etc.). Supports Serper, Tavily, and SearXNG.
Graphlit MCP Server
Model Context Protocol (MCP) Server for Graphlit Platform
MCP Omnisearch
🔍 A Model Context Protocol (MCP) server providing unified access to multiple search engines (Tavily, Brave, Kagi, Exa), AI tools (Kagi FastGPT, Exa, Linkup), and content extraction services (Firecrawl, Tavily, Kagi). Includes GitHub search. All through a single interface.
SpotifyScraper
Extract public Spotify data — tracks, albums, artists, playlists, podcasts & lyrics — without the official API. Sync + async, typed models, one dependency.
LLM Reader
Turn Webpage to LLM friendly input text. Similar to Firecrawl and Jina Reader API. Makes RAG, AI web scraping, image & webpage links extraction easy.
Teracrawl
High-performance web crawler API optimized for LLMs. Turn any search or website into clean Markdown using remote browsers. Firecrawl alternative
Alfanous
Alfanous is an Arabic search engine API provides the simple and advanced search in Quran , more features and many interfaces...
PyScrappy
Adaptive Python web scraping toolkit + MCP server for AI agents. Self-healing selectors that survive site changes, TLS-fingerprint stealth to bypass anti-bot filters, CSS/XPath parsing, and 24 built-in scrapers, clean, structured, LLM-ready data from any URL.
OpenPlexity Pages
SearchGPT / Perplexity Pages clone, but personalised for you.
Firecrawl Claude Plugin
Firecrawl plugin for Claude Code: search the live web, developer docs, government sources and research papers, run structured data providers, and scrape a page into clean Markdown. Firecrawl is the context API for AI agents.
Vexor
A semantic search engine for files and code.
Skilless AI
skilless.ai gives your AI Agents real data capabilities - web search, web scraping, video download/subtitle extraction, and media processing. One-line install, works with OpenCode, Cursor, Claude Code, and any AI agent that reads skill files.
Amazon Scraper V2
Agent-ready Amazon Scraper API: real-time product details (batch ASIN), offers & stock, reviews, search, best sellers, deals and seller data across 20 marketplaces. One pasted line for Claude Code / Codex / Cursor, an MCP server, or plain HTTP with 3 free calls a day and no key. Python examples inside.
Argo
专门为 agent 打造的开源 agent 搜索工具，具备 200+的搜索源及多语言搜索能力，覆盖中文/英文/学术/代码/购物/金融/新闻/百科。
Open Extract
Structured Data Extractor for AI Agents. Search your documents or the web for specific data and get it back in JSON or Markdown in a single tool call.
Deepsearch Academic
An implementation of Google Deep Search with support for 1000+ references, local inference, chatting with your scraping session using RAPTOR, and report generation.
SearxNcrawl
Open-source MCP server that gives AI agents web search (SearXNG) and clean Markdown crawling of pages, whole sites and logged-in areas. Self-hosted, no API key. Firecrawl-compatible API planned.
Serpapi MCP
SerpApi MCP Server for Google and other search engine results
List refreshed November 15, 2026. Data comes from public GitHub repositories.