Angler: an open-source search, crawl & research gateway you self-host
Phạm Hải Anh · AI engineer··2 min read
Tetra Software introduces Angler — an open-source (MIT) search + crawl + research gateway that runs on your own machine. Send a query or a URL, get clean markdown back: multi-source, bias-resistant, Firecrawl-compatible. Free, local-first, no API key.
What is Angler?
Angler folds together the things you usually stitch by hand — search, scraping pages (even ones behind Cloudflare), crawling a whole site, and multi-round research with citations — behind a single API gateway (port 17300). The output is tidy, low-token markdown, ready for research and for AI agents.
A self-hosted Firecrawl alternative
Angler speaks the Firecrawl API: it is Firecrawl-SDK compatible. Any agent or script already using Firecrawl just points FIRECRAWL_API_URL at your Angler instance and works with no code changes. The core difference: you self-host it — your data and infrastructure, no vendor lock-in.
Free, local-first, MIT open source
Angler is MIT-licensed — free to download and use for anything, no license fees. Docker + Docker Compose is all you need; it is up in minutes. It runs local by default: no API key, no rate limits, your research queries never leave your machine. To share it with a team, set ANGLER_API_KEY to enable Bearer auth.
Bias-resistant, multi-source — academia too
Instead of following a few top-SEO results that share one viewpoint, Angler actively diversifies sources on several axes: many source types (news, academic, official, community), multiple languages, enforced opposing sides, and a per-domain cap — with diversity stats returned so you see the full picture. A great fit for serious market research and information synthesis.
What is Angler used for?
Market & competitor research: gather many-sided sources and synthesize with citations instead of reading page by page.
Information gathering & synthesis: one query to clean markdown from across the web, even hard pages.
Feeding AI agents / RAG: tidy, low-token markdown — plug in the Firecrawl SDK and go.
Give your AI agents web access
Angler is also web access for AI coding agents. Because it is Firecrawl-compatible, agents like Claude Code, Cursor, Codex CLI, Gemini CLI, Cline, OpenHands, Goose, Aider and Windsurf call Angler over MCP (Model Context Protocol — point the Firecrawl MCP server at Angler) or the Firecrawl SDK: just set FIRECRAWL_API_URL to your Angler instance. Angler is a Firecrawl-compatible REST API (it does not speak MCP itself); paired with the Firecrawl MCP server you get a self-hosted web-search / web-scraping MCP server — web access for your agent while data stays local.
Building agents with a framework? LangChain/LangGraph, CrewAI, LlamaIndex, Mastra and the Vercel AI SDK can all call Angler's API for LLM-ready markdown — for agentic RAG.
Get started
See the product page for Angler, or grab the source and self-host it from the tetravn/angler-crawl GitHub repository. A single docker compose up -d gives you your own research gateway on port 17300.
Related articles
Google’s Open Knowledge Format (OKF): packaging knowledge for AI agents — what we learned deploying it
Google just published OKF — an open standard that turns organizational knowledge into a Markdown directory agents read directly. We took the samples apart and pulled out deployment lessons for several domains.
Read ↗
WebMCP: we made tetra.vn callable by AI agents
tetra.vn now exposes tools to AI agents via the WebMCP standard: agents (Claude Code, Cursor…) can call the site for products, knowledge and navigation instead of scraping HTML. The next step of AEO — and how an agent-ready site differs from an ordinary one.
Read ↗
Vietnamese OCR: turning document images into usable data (safely)
OCR only reads text; businesses need structured data. The difference between OCR and data extraction, why Vietnamese OCR is hard, the risk of free online OCR with sensitive data, and how to do it safely on your own server.
Read ↗Personal Data Protection checklist
Review your business before the law takes effect on 01/01/2026.