web-extraction

Fast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data — all from Rust. CLI, REST API, and MCP server.

Give your Hermes agent the web as real sources, never a made-up answer — multi-provider search and extraction with an optional local, key-free DonSeTch option.

Use LLMs to robustly extract web data

Fully automated and hands-free, accurately extracting and understanding web content — powered by machine learning agents.

Windows desktop app for collecting, reviewing, ranking, and exporting Google Scholar search results.

DeepSeek Harness URL reader: fetch any page and return clean main-content text/Markdown. Auto charset (GBK/GB2312/UTF-8/Big5), token-efficient (6000-char cap, cache, offset), zero deps, no API key. 网页一键读全文 → 干净正文 / 结构化 Markdown

Low-Cost Cross-Domain Web Structured Information Extraction using specialized LoRA adapters.

Replayable Browser Agent

基于Scala Akka的分布式主题网络爬虫

Hermes plugin to improve web search with SearXNG for higher-quality search results and Crawl4AI for LLM-optimized webpage extraction.

A swipeable shortlist of live San Francisco apartment listings, powered by Context.dev.

Keyless-first MCP for better web research — DuckDuckGo out of the box, Brave/Tavily optional, 3-tier extraction (HTTP → hydration → Playwright)

Self-hosted web scraping and Markdown extraction for AI agents

Quota-aware web search for AI agents: 10 providers with local quota tracking (fallback before 429), intent routing, free local-first extraction cascade (gh CLI native for github.com, JSON API passthrough, Russian Trusted CA). Benchmarks included.

Automatic extraction of the information on local event from a webpage with Machine Learning

Give your agent the web as real sources, never made-up answers — an MCP server with 15 search and 9 extract providers plus an optional local, key-free DonSeTch integration.

Web Fetch benchmark: versioned tasks, frozen references and deterministic acceptance scoring for page content, records, resources and files.

Free local web search/extraction router for AI agents. Go CLI + MCP, BYOK/free-first routing, keyless DDGS/Scrapling fallback, setup writers and client guides.

A Diffbot MCP server with a variety of helpful web knowledge fetching tools for your agent or LLM pipeline.

Official Firecrawl CLI and Python toolkit for coding agents: scrape URLs to clean Markdown, crawl websites, query Alexandria, and verify saved web evidence.