Web Scraping & Data AI AgentsWeb Scraping & Data

Firecrawl

Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

Crawl4ai

Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key. Join our Discord https://discord.gg/VuxzM8mtf

Vane

Vane is an AI-powered answering engine.

Maxun

Turn any website into a structured API. Extract, automate, search and monitor the web.

Sparrow

Structured data extraction, instruction calling and agentic workflows with ML, LLM and Vision LLM

Noria

An agentic extractor for messaging platforms. Noria receives messages & URLs via chat; scrapes web data to evaluate the content against custom scoring matrices (e.g., jobs, scholarships); and sends a structured summary with a calculated match score directly to the assigned messenger number.

LLM Wiki Agent

A personal knowledge base that builds and maintains itself. Drop in sources — Claude (or Codex/Gemini) reads them, extracts knowledge, and maintains a persistent interlinked wiki. Works with Claude Code, Codex, OpenCode, Gemini CLI. No API key needed.

Spider

Foundational low latency web data collecting in Rust

AIWriteX

AIWriteX - 微信公众号全自动AI工具：全网热搜舆情聚合+趋势分析+爆款选题+文章采集+一键生成排版发布 | AI自动配图 | 去AI味、过朱雀检测 | 支持小红书/百家号/头条等多平台 | 洗稿润色支持多账号 | 专家赛道 | 小绿书贴图 | 短视频文案 | 手机控制 | 小说连载 | 爆文10w+神器

Scrapecraft

🤖 AI-powered web scraping editor with visual workflow builder. Build, test & deploy web scrapers using natural language. Powered by ScrapeGraphAI & LangGraph.

LLM Reader

Turn Webpage to LLM friendly input text. Similar to Firecrawl and Jina Reader API. Makes RAG, AI web scraping, image & webpage links extraction easy.

OpenPlexity Pages

SearchGPT / Perplexity Pages clone, but personalised for you.

Readwren

An adaptive multi-agent system that extracts your literary DNA through conversation and generates actionable reading profiles.

Open Extract

Structured Data Extractor for AI Agents. Search your documents or the web for specific data and get it back in JSON or Markdown in a single tool call.

BrowserPilot

Open‑source alternative to Perplexity Comet, director.ai and firecrawl combined

Structsense

A multi-agent system for extracting structured information from unstructured text and documents. It orchestrates a configurable pipeline of AI agents — extractor → alignment → judge → human feedback — each driven by a single YAML config file.