feat: add SearXNG as a native web search backend

Adds SearXNG (https://docs.searxng.org) as a self-hosted, privacy-first
web search backend alongside Firecrawl, Tavily, Exa, and Parallel.

SearXNG is a meta-search engine that aggregates results from 70+ search
engines. No API key needed -- just set SEARXNG_URL to your instance.

Changes:
- tools/web_tools.py: _get_searxng_url(), _searxng_search(), search
  dispatch, extract falls back to Firecrawl (SearXNG is search-only)
- hermes_cli/tools_config.py: SearXNG provider in web tool picker
- hermes_cli/config.py: SEARXNG_URL env var, diagnostics, set command
- tests/tools/test_web_tools_searxng.py: 15 tests
- optional-skills/research/searxng-search/: agent-guided skill
- Docs: configuration.md, environment-variables.md, skills catalogs

Based on #6071 by @gnanam1990, #8106 by @cro, #2572 by @bhovig,
#2710 and #9961 by @StreamOfRon, #7258 by @coldxiangyu163
This commit is contained in:
kshitijk4poor 2026-04-17 14:37:59 +05:30
parent 2367c6ffd5
commit 130b021d74
10 changed files with 600 additions and 6 deletions

View file

@ -1102,21 +1102,24 @@ code_execution:
## Web Search Backends
The `web_search`, `web_extract`, and `web_crawl` tools support four backend providers. Configure the backend in `config.yaml` or via `hermes tools`:
The `web_search`, `web_extract`, and `web_crawl` tools support five backend providers. Configure the backend in `config.yaml` or via `hermes tools`:
```yaml
web:
backend: firecrawl # firecrawl | parallel | tavily | exa
backend: firecrawl # firecrawl | searxng | parallel | tavily | exa
```
| Backend | Env Var | Search | Extract | Crawl |
|---------|---------|--------|---------|-------|
| **Firecrawl** (default) | `FIRECRAWL_API_KEY` | ✔ | ✔ | ✔ |
| **SearXNG** | `SEARXNG_URL` | ✔ | — (falls back to Firecrawl) | — |
| **Parallel** | `PARALLEL_API_KEY` | ✔ | ✔ | — |
| **Tavily** | `TAVILY_API_KEY` | ✔ | ✔ | ✔ |
| **Exa** | `EXA_API_KEY` | ✔ | ✔ | — |
**Backend selection:** If `web.backend` is not set, the backend is auto-detected from available API keys. If only `EXA_API_KEY` is set, Exa is used. If only `TAVILY_API_KEY` is set, Tavily is used. If only `PARALLEL_API_KEY` is set, Parallel is used. Otherwise Firecrawl is the default.
**Backend selection:** If `web.backend` is not set, the backend is auto-detected from available API keys. If only `SEARXNG_URL` is set, SearXNG is used. If only `EXA_API_KEY` is set, Exa is used. If only `TAVILY_API_KEY` is set, Tavily is used. If only `PARALLEL_API_KEY` is set, Parallel is used. Otherwise Firecrawl is the default.
**SearXNG** is a self-hosted open-source meta-search engine that aggregates results from 70+ search engines. No API key is needed — just set `SEARXNG_URL` to your instance (e.g., `https://searx.example.com`). SearXNG only supports search; extract and crawl automatically fall back to Firecrawl.
**Self-hosted Firecrawl:** Set `FIRECRAWL_API_URL` to point at your own instance. When a custom URL is set, the API key becomes optional (set `USE_DB_AUTHENTICATION=false` on the server to disable auth).