mirrors/hermes-agent
1
0
Fork 0
mirror of https://github.com/NousResearch/hermes-agent.git synced 2026-06-03 07:21:54 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 11

chore(web): remove web_crawl tool + provider crawl plumbing (#33824)

Browse source 
The web_crawl_tool() function was an orphan — no model schema registered
it, no skill or CLI command called it, and the agent had no way to invoke
it. PR #32608 proposed wiring it up as a model-callable tool; we've
decided not to expose crawl as a separate capability since web_search +
web_extract cover the use cases we want models to have.

Removed:
- tools/web_tools.py: web_crawl_tool() (~230 LOC)
- plugins/web/firecrawl/provider.py: supports_crawl() + crawl()
- plugins/web/tavily/provider.py: supports_crawl() + crawl()
- plugins/web/xai/provider.py: supports_crawl() override
- agent/web_search_provider.py: supports_crawl() + crawl() ABC methods
- agent/web_search_registry.py: get_active_crawl_provider() +
  the 'crawl' branch in _resolve()
- agent/display.py: web_crawl tool-progress rendering
- hermes_cli/config.py: 'web_crawl' from TAVILY_API_KEY.tools
- tools/website_policy.py: stale comment reference
- Tests: removed TestWebCrawlTavily class, the two website-policy
  web_crawl tests, the searxng/ddgs/brave-free crawl-error tests,
  the integration test_web_crawl method, and the
  test_unconfigured_crawl_emits_top_level_error test. Trimmed the
  capability-flag parametrize list and the WebSearchProvider ABC
  conformance tests.
- Docs: trimmed the Crawl column from capability tables in both EN
  and zh-Hans, updated the developer-guide ABC table.

Net: 25 files, +115/-1067.

Closes #33762 (the schema-text bug only existed if #32608 landed).
Supersedes #32608.
This commit is contained in:
Teknium 2026-05-28 04:52:42 -07:00 • committed by GitHub
parent b243afb68b
commit 5e1f793430
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
25 changed files with 115 additions and 1067 deletions
Download patch file Download diff file Expand all files Collapse all files

7
website/docs/developer-guide/web-search-provider-plugin.md
View file

@ -80,9 +80,6 @@ class MyBackendWebSearchProvider(WebSearchProvider):
def supports_extract(self) -> bool:
return False
def supports_crawl(self) -> bool:
return False
def search(self, query: str, limit: int = 5) -> Dict[str, Any]:
import httpx
@ -157,12 +154,10 @@ Full contract in `agent/web_search_provider.py`. Methods you may override:
| `is_available()` | ✅ | — | Cheap availability gate — env vars, optional deps |
| `supports_search()` | — | `True` | Capability flag for `web_search` routing |
| `supports_extract()` | — | `False` | Capability flag for `web_extract` routing |
| `supports_crawl()` | — | `False` | Capability flag for deep-crawl modes |
| `search(query, limit)` | conditional | raises | Required when `supports_search()` returns `True` |
| `extract(urls, **kwargs)` | conditional | raises | Required when `supports_extract()` returns `True` |
| `crawl(url, **kwargs)` | conditional | raises | Required when `supports_crawl()` returns `True` |
Providers can advertise multiple capabilities from a single class — Firecrawl, Tavily, Exa, and Parallel all implement all three of search/extract/crawl. Brave Search and DDGS are search-only; SearXNG is search-only with a documented "pair me with an extract provider" workflow.
Providers can advertise multiple capabilities from a single class — Firecrawl, Tavily, Exa, and Parallel all implement both search and extract. Brave Search and DDGS are search-only; SearXNG is search-only with a documented "pair me with an extract provider" workflow.
## Response shape