mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-07-31 19:16:29 +00:00
feat(skills): add optional last30days skill
Hermes-native port of mvanhorn/last30days-skill (MIT): research what people said about a topic across HN, Reddit, Polymarket, X, YouTube, and the web over the last 30 days. Keyless pure-stdlib fetcher for HN Algolia / Reddit public JSON / Polymarket Gamma; X, YouTube, and web coverage reframed onto Hermes web_search/web_extract. Credit: Matt Van Horn (@mvanhorn).
This commit is contained in:
teknium1
parent
f4df260f26
commit
abd847d80f
6 changed files with 700 additions and 0 deletions
122
optional-skills/research/last30days/SKILL.md
Normal file
122
optional-skills/research/last30days/SKILL.md
Normal file
|
|
@ -0,0 +1,122 @@
|
|||
---
|
||||
name: last30days
|
||||
description: "Research what people said about a topic in the last 30 days."
|
||||
version: 0.1.0
|
||||
author: Matt Van Horn (mvanhorn), Hermes Agent
|
||||
license: MIT
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [Research, Social-Media, Trends, News]
|
||||
related_skills: [polymarket, duckduckgo-search]
|
||||
---
|
||||
|
||||
# last30days
|
||||
|
||||
Researches a topic across Hacker News, Reddit, Polymarket, X, YouTube, and the
|
||||
general web over a recent time window (default 30 days), then synthesizes a
|
||||
grounded summary of what people are actually saying, with engagement signals
|
||||
and citations. The bundled fetch script is pure Python stdlib and fully
|
||||
keyless; API keys are optional enhancers only. This is a slimmed Hermes port of
|
||||
the upstream last30days skill (github.com/mvanhorn/last30days-skill, MIT) — it
|
||||
does not include upstream's TikTok/Instagram/Bluesky/Digg integrations or its
|
||||
setup wizard.
|
||||
|
||||
## When to Use
|
||||
|
||||
- "What are people saying about X lately?" / community-sentiment questions
|
||||
- Recency-bound research: launches, earnings reactions, drama, hiring signals
|
||||
- Comparing tools/products by recent community chatter ("X vs Y")
|
||||
- Checking prediction-market odds alongside social discussion
|
||||
- NOT for deep historical research or single-document lookups — use plain
|
||||
`web_search`/`web_extract` for those
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python 3.10+ (stdlib only — no pip installs)
|
||||
- Network access to reddit.com, hn.algolia.com, gamma-api.polymarket.com
|
||||
- No env vars required. Everything runs keyless in degraded-but-useful mode.
|
||||
Optional enhancers (document in the user's own `.env`, never required):
|
||||
- `XAI_API_KEY` — richer X/Twitter coverage upstream; in this port, X
|
||||
coverage comes from `web_search` instead
|
||||
- `BRAVE_API_KEY` — upstream web-search backend; unnecessary here because
|
||||
Hermes has native `web_search`
|
||||
|
||||
## How to Run
|
||||
|
||||
Run the fetch script through the `terminal` tool, then synthesize:
|
||||
|
||||
```
|
||||
python3 ~/.hermes/skills/research/last30days/scripts/fetch_sources.py "your topic" --days 30 --format md
|
||||
```
|
||||
|
||||
Cover X, YouTube, and the general web with Hermes `web_search` (the script
|
||||
deliberately does not scrape those), then write the summary per the Procedure.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Command | Purpose |
|
||||
|---|---|
|
||||
| `fetch_sources.py "topic"` | All keyless sources, markdown output |
|
||||
| `fetch_sources.py "topic" --format json` | Machine-readable output |
|
||||
| `fetch_sources.py "topic" --days 7` | Narrower recency window |
|
||||
| `fetch_sources.py "topic" --sources hackernews,polymarket` | Subset of sources |
|
||||
| `fetch_sources.py "topic" --subreddit LocalLLaMA` | Scope Reddit to one sub |
|
||||
| `fetch_sources.py "topic" --limit 10` | Cap items per source |
|
||||
|
||||
Sources: `hackernews` (Algolia, date-filtered), `reddit` (public JSON search,
|
||||
`t=month`), `polymarket` (active events by 30-day volume).
|
||||
|
||||
## Procedure
|
||||
|
||||
1. Run the fetcher via `terminal`:
|
||||
`python3 ~/.hermes/skills/research/last30days/scripts/fetch_sources.py "TOPIC" --format md`
|
||||
Note each source's `status:` line — `error` means degraded coverage for
|
||||
that source, not a failed run.
|
||||
2. Fill the gaps with Hermes-native tools (batch these searches):
|
||||
- `web_search`: `TOPIC site:x.com` or `TOPIC twitter reaction` for X chatter
|
||||
- `web_search`: `TOPIC site:youtube.com` (or `TOPIC review video`) for YouTube
|
||||
- `web_search`: `TOPIC` plus a recency phrase like "this month" for news/web
|
||||
3. Pull full text of the 2-4 most-cited pages with `web_extract` when a
|
||||
headline alone can't support a claim.
|
||||
4. Discard anything clearly older than the window. The script date-filters HN
|
||||
and Reddit; web results need manual checking against publish dates.
|
||||
5. Synthesize. Upstream's output contract, kept here because it works:
|
||||
- Lead with `What I learned:` then bold-lead-in paragraphs — no invented
|
||||
title, no `##` section headers in the body
|
||||
- Quote real people verbatim with source attribution (subreddit, HN
|
||||
thread, channel); prefer high-engagement items (the script pre-sorts
|
||||
by an engagement score: 0.6·score + 0.4·comments, each capped)
|
||||
- End with a numbered `KEY PATTERNS from the research:` list
|
||||
- Never fabricate quotes, titles, or engagement numbers; if a source
|
||||
returned nothing or errored, say "partial coverage" — not "nothing
|
||||
was said on <source>"
|
||||
- Include Polymarket odds only when a market genuinely matches the topic
|
||||
6. For comparison topics ("A vs B"), run steps 1-3 once per entity, then
|
||||
structure as: verdict, per-entity findings, head-to-head, bottom line.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **Reddit public JSON returns HTTP 403 intermittently** — upstream migrated
|
||||
off it for this reason. When the `reddit` source reports an error, fall back
|
||||
to `web_search` with `TOPIC site:reddit.com` and treat Reddit as partial
|
||||
coverage. Do not retry in a loop.
|
||||
- HN Algolia rejects `points>N` in `numericFilters` (HTTP 400) — the script
|
||||
filters low-engagement stories client-side instead; don't "fix" that.
|
||||
- Polymarket's search matches loosely; a topic like "React hooks" can return
|
||||
unrelated political markets. Drop non-matching events during synthesis.
|
||||
- Engagement scores compare items *within* a source only — a 500-point HN
|
||||
story and a 500-upvote Reddit post are not equivalent audiences.
|
||||
- Windows: invoke as `python` if `python3` is not on PATH; the script itself
|
||||
is portable (no `/tmp`, no shell pipelines).
|
||||
- Don't dump the script's raw item list as the answer — it is evidence for
|
||||
synthesis, not output.
|
||||
|
||||
## Verification
|
||||
|
||||
```
|
||||
python3 ~/.hermes/skills/research/last30days/scripts/fetch_sources.py "artificial intelligence" --sources hackernews --limit 3 --format json
|
||||
```
|
||||
|
||||
Should print JSON with `"status": "ok"` and up to 3 dated HN stories from the
|
||||
last 30 days. Exit code 0 even on partial source failures.
|
||||
222
optional-skills/research/last30days/scripts/fetch_sources.py
Normal file
222
optional-skills/research/last30days/scripts/fetch_sources.py
Normal file
|
|
@ -0,0 +1,222 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Keyless multi-source fetcher for the last30days skill.
|
||||
|
||||
Queries three free JSON APIs over a recency window and emits scored,
|
||||
normalized items as JSON or markdown:
|
||||
|
||||
- Hacker News : hn.algolia.com/api/v1/search (created_at_i numeric filters)
|
||||
- Reddit : www.reddit.com/search.json (t=month; may 403 — see SKILL.md)
|
||||
- Polymarket : gamma-api.polymarket.com/public-search (active events)
|
||||
|
||||
Pure stdlib. No API keys. Endpoints and query parameters ported from
|
||||
mvanhorn/last30days-skill (MIT). X/YouTube/web coverage is handled by the
|
||||
hosting agent's own search tools, not this script.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import datetime as _dt
|
||||
import json
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
|
||||
USER_AGENT = "last30days-hermes/0.1 (research skill; +https://github.com/mvanhorn/last30days-skill)"
|
||||
|
||||
HN_SEARCH_URL = "https://hn.algolia.com/api/v1/search"
|
||||
REDDIT_SEARCH_URL = "https://www.reddit.com/search.json"
|
||||
POLYMARKET_SEARCH_URL = "https://gamma-api.polymarket.com/public-search"
|
||||
|
||||
MIN_HN_POINTS = 2 # upstream filters low-engagement stories client-side
|
||||
|
||||
|
||||
def _get_json(url: str, timeout: int = 20):
|
||||
req = urllib.request.Request(url, headers={"User-Agent": USER_AGENT})
|
||||
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
||||
return json.loads(resp.read().decode("utf-8", errors="replace"))
|
||||
|
||||
|
||||
def engagement_score(score: int, num_comments: int) -> float:
|
||||
"""Engagement-based relevance in [0, 1] (ported from upstream reddit_public)."""
|
||||
score_component = min(1.0, max(0.0, score / 500.0))
|
||||
comments_component = min(1.0, max(0.0, num_comments / 200.0))
|
||||
return round((score_component * 0.6) + (comments_component * 0.4), 3)
|
||||
|
||||
|
||||
def date_to_unix(date_str: str) -> int:
|
||||
"""YYYY-MM-DD -> Unix timestamp at start of day UTC."""
|
||||
y, m, d = (int(p) for p in date_str.split("-"))
|
||||
return int(_dt.datetime(y, m, d, tzinfo=_dt.timezone.utc).timestamp())
|
||||
|
||||
|
||||
def window(days: int) -> tuple[str, str]:
|
||||
today = _dt.datetime.now(_dt.timezone.utc).date()
|
||||
return ((today - _dt.timedelta(days=days)).isoformat(), today.isoformat())
|
||||
|
||||
|
||||
def search_hackernews(topic: str, from_date: str, to_date: str, limit: int = 30) -> list[dict]:
|
||||
from_ts = date_to_unix(from_date)
|
||||
to_ts = date_to_unix(to_date) + 86400 # include end date
|
||||
params = {
|
||||
"query": topic,
|
||||
"tags": "story",
|
||||
"numericFilters": f"created_at_i>{from_ts},created_at_i<{to_ts}",
|
||||
"hitsPerPage": str(limit * 2), # overfetch, then filter low engagement
|
||||
}
|
||||
# Algolia ANDs query tokens; mark all-but-first optional so multi-word
|
||||
# topics rank by token overlap instead of requiring every word.
|
||||
tokens = topic.split()
|
||||
if len(tokens) > 1:
|
||||
params["optionalWords"] = " ".join(tokens[1:])
|
||||
data = _get_json(f"{HN_SEARCH_URL}?{urllib.parse.urlencode(params)}")
|
||||
items = []
|
||||
for hit in data.get("hits", []):
|
||||
points = hit.get("points") or 0
|
||||
if points <= MIN_HN_POINTS:
|
||||
continue
|
||||
object_id = hit.get("objectID", "")
|
||||
items.append({
|
||||
"source": "hackernews",
|
||||
"title": hit.get("title") or "",
|
||||
"url": hit.get("url") or f"https://news.ycombinator.com/item?id={object_id}",
|
||||
"discussion_url": f"https://news.ycombinator.com/item?id={object_id}",
|
||||
"date": (hit.get("created_at") or "")[:10],
|
||||
"points": points,
|
||||
"comments": hit.get("num_comments") or 0,
|
||||
"relevance": engagement_score(points, hit.get("num_comments") or 0),
|
||||
})
|
||||
items.sort(key=lambda i: i["relevance"], reverse=True)
|
||||
return items[:limit]
|
||||
|
||||
|
||||
def search_reddit(topic: str, limit: int = 25, subreddit: str | None = None) -> list[dict]:
|
||||
"""Reddit public JSON search, t=month. May return HTTP 403 (endpoint is
|
||||
unreliable per upstream); callers should fall back to agent web search."""
|
||||
if subreddit:
|
||||
base = f"https://www.reddit.com/r/{subreddit.removeprefix('r/')}/search.json"
|
||||
params = {"q": topic, "restrict_sr": "on", "sort": "relevance", "t": "month",
|
||||
"limit": str(limit), "raw_json": "1"}
|
||||
else:
|
||||
base = REDDIT_SEARCH_URL
|
||||
params = {"q": topic, "sort": "relevance", "t": "month",
|
||||
"limit": str(limit), "raw_json": "1"}
|
||||
data = _get_json(f"{base}?{urllib.parse.urlencode(params)}")
|
||||
items, seen = [], set()
|
||||
for child in (data.get("data") or {}).get("children", []):
|
||||
post = child.get("data") or {}
|
||||
permalink = post.get("permalink") or ""
|
||||
url = f"https://www.reddit.com{permalink}"
|
||||
if url in seen:
|
||||
continue
|
||||
seen.add(url)
|
||||
score = post.get("score") or 0
|
||||
num_comments = post.get("num_comments") or 0
|
||||
created = post.get("created_utc")
|
||||
items.append({
|
||||
"source": "reddit",
|
||||
"title": post.get("title") or "",
|
||||
"url": url,
|
||||
"subreddit": post.get("subreddit") or "",
|
||||
"date": _dt.datetime.fromtimestamp(created, tz=_dt.timezone.utc).date().isoformat()
|
||||
if created else "",
|
||||
"points": score,
|
||||
"comments": num_comments,
|
||||
"relevance": engagement_score(score, num_comments),
|
||||
})
|
||||
items.sort(key=lambda i: i["relevance"], reverse=True)
|
||||
return items[:limit]
|
||||
|
||||
|
||||
def search_polymarket(topic: str, pages: int = 3, limit: int = 15) -> list[dict]:
|
||||
"""Active prediction markets matching the topic, sorted by volume."""
|
||||
events: dict[str, dict] = {}
|
||||
for page in range(1, pages + 1):
|
||||
params = {"q": topic, "page": str(page),
|
||||
"events_status": "active", "keep_closed_markets": "0"}
|
||||
try:
|
||||
data = _get_json(f"{POLYMARKET_SEARCH_URL}?{urllib.parse.urlencode(params)}")
|
||||
except (urllib.error.URLError, urllib.error.HTTPError, OSError):
|
||||
break
|
||||
batch = data.get("events") or []
|
||||
if not batch:
|
||||
break
|
||||
for ev in batch:
|
||||
eid = str(ev.get("id") or ev.get("slug") or "")
|
||||
if eid and eid not in events:
|
||||
events[eid] = ev
|
||||
items = []
|
||||
for eid, ev in events.items():
|
||||
slug = ev.get("slug") or ""
|
||||
volume = float(ev.get("volume1mo") or ev.get("volume") or 0)
|
||||
items.append({
|
||||
"source": "polymarket",
|
||||
"title": ev.get("title") or "",
|
||||
"url": f"https://polymarket.com/event/{slug}" if slug
|
||||
else f"https://polymarket.com/event/{eid}",
|
||||
"volume_1mo": volume,
|
||||
"liquidity": float(ev.get("liquidity") or 0),
|
||||
})
|
||||
items.sort(key=lambda i: i["volume_1mo"], reverse=True)
|
||||
return items[:limit]
|
||||
|
||||
|
||||
def render_markdown(topic: str, from_date: str, to_date: str, results: dict) -> str:
|
||||
lines = [f"# last30days: {topic} ({from_date} to {to_date})", ""]
|
||||
for source, payload in results.items():
|
||||
items = payload["items"]
|
||||
status = payload["status"]
|
||||
lines.append(f"## {source} ({len(items)} items, status: {status})")
|
||||
for it in items:
|
||||
meta = []
|
||||
if "points" in it:
|
||||
meta.append(f"{it['points']} pts, {it['comments']} comments")
|
||||
if "volume_1mo" in it:
|
||||
meta.append(f"${it['volume_1mo']:,.0f} 30-day volume")
|
||||
if it.get("date"):
|
||||
meta.append(it["date"])
|
||||
lines.append(f"- [{it['title']}]({it['url']}) ({'; '.join(meta)})")
|
||||
lines.append("")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
ap = argparse.ArgumentParser(description="Keyless last-30-days source fetcher")
|
||||
ap.add_argument("topic")
|
||||
ap.add_argument("--days", type=int, default=30)
|
||||
ap.add_argument("--sources", default="hackernews,reddit,polymarket",
|
||||
help="comma list: hackernews,reddit,polymarket")
|
||||
ap.add_argument("--limit", type=int, default=20, help="max items per source")
|
||||
ap.add_argument("--subreddit", default=None, help="scope reddit search to one subreddit")
|
||||
ap.add_argument("--format", choices=["json", "md"], default="md")
|
||||
args = ap.parse_args(argv)
|
||||
|
||||
from_date, to_date = window(args.days)
|
||||
wanted = {s.strip() for s in args.sources.split(",") if s.strip()}
|
||||
fetchers = {
|
||||
"hackernews": lambda: search_hackernews(args.topic, from_date, to_date, args.limit),
|
||||
"reddit": lambda: search_reddit(args.topic, args.limit, args.subreddit),
|
||||
"polymarket": lambda: search_polymarket(args.topic, limit=args.limit),
|
||||
}
|
||||
results: dict[str, dict] = {}
|
||||
for name in ("hackernews", "reddit", "polymarket"):
|
||||
if name not in wanted:
|
||||
continue
|
||||
try:
|
||||
results[name] = {"status": "ok", "items": fetchers[name]()}
|
||||
except Exception as exc: # degraded, not fatal — mirror upstream behavior
|
||||
results[name] = {"status": f"error: {exc}", "items": []}
|
||||
print(f"[last30days] {name} failed: {exc}", file=sys.stderr)
|
||||
|
||||
if args.format == "json":
|
||||
print(json.dumps({"topic": args.topic, "from": from_date, "to": to_date,
|
||||
"results": results}, indent=2))
|
||||
else:
|
||||
print(render_markdown(args.topic, from_date, to_date, results))
|
||||
# exit 0 even on partial coverage (upstream default); failures are annotated
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
214
tests/skills/test_last30days_skill.py
Normal file
214
tests/skills/test_last30days_skill.py
Normal file
|
|
@ -0,0 +1,214 @@
|
|||
"""Tests for the last30days optional skill (frontmatter + script logic).
|
||||
|
||||
No live network calls: urllib.request.urlopen is mocked everywhere.
|
||||
"""
|
||||
|
||||
import io
|
||||
import json
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
import yaml
|
||||
|
||||
TESTS_DIR = Path(__file__).resolve().parent
|
||||
SKILL_DIR = TESTS_DIR.parent.parent / "optional-skills" / "research" / "last30days"
|
||||
SKILL_MD = SKILL_DIR / "SKILL.md"
|
||||
SCRIPT = SKILL_DIR / "scripts" / "fetch_sources.py"
|
||||
|
||||
sys.path.insert(0, str(SCRIPT.parent))
|
||||
import fetch_sources # noqa: E402
|
||||
|
||||
|
||||
def _fake_response(payload):
|
||||
body = json.dumps(payload).encode("utf-8")
|
||||
|
||||
class _Resp(io.BytesIO):
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, *args):
|
||||
self.close()
|
||||
|
||||
return _Resp(body)
|
||||
|
||||
|
||||
class TestFrontmatter(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
text = SKILL_MD.read_text(encoding="utf-8")
|
||||
assert text.startswith("---\n"), "SKILL.md must start with YAML frontmatter"
|
||||
cls.fm = yaml.safe_load(text.split("---", 2)[1])
|
||||
cls.body = text.split("---", 2)[2]
|
||||
|
||||
def test_required_fields(self):
|
||||
for field in ("name", "description", "version", "author", "license", "platforms"):
|
||||
self.assertIn(field, self.fm, f"missing frontmatter field: {field}")
|
||||
|
||||
def test_name(self):
|
||||
self.assertEqual(self.fm["name"], "last30days")
|
||||
|
||||
def test_description_length_and_shape(self):
|
||||
desc = self.fm["description"]
|
||||
self.assertLessEqual(len(desc), 60, f"description is {len(desc)} chars (max 60)")
|
||||
self.assertTrue(desc.endswith("."), "description must end with a period")
|
||||
|
||||
def test_version_and_license(self):
|
||||
self.assertEqual(str(self.fm["version"]), "0.1.0")
|
||||
self.assertEqual(self.fm["license"], "MIT")
|
||||
|
||||
def test_platforms(self):
|
||||
self.assertEqual(set(self.fm["platforms"]), {"linux", "macos", "windows"})
|
||||
|
||||
def test_hermes_metadata(self):
|
||||
hermes = self.fm["metadata"]["hermes"]
|
||||
self.assertIsInstance(hermes["tags"], list)
|
||||
self.assertTrue(hermes["tags"])
|
||||
self.assertIsInstance(hermes["related_skills"], list)
|
||||
|
||||
def test_body_sections_present(self):
|
||||
for section in ("## When to Use", "## Prerequisites", "## How to Run",
|
||||
"## Quick Reference", "## Procedure", "## Pitfalls",
|
||||
"## Verification"):
|
||||
self.assertIn(section, self.body, f"missing section: {section}")
|
||||
|
||||
|
||||
class TestEngagementScore(unittest.TestCase):
|
||||
def test_zero(self):
|
||||
self.assertEqual(fetch_sources.engagement_score(0, 0), 0.0)
|
||||
|
||||
def test_capped_at_one(self):
|
||||
self.assertEqual(fetch_sources.engagement_score(10_000, 10_000), 1.0)
|
||||
|
||||
def test_weighting(self):
|
||||
# 250/500 * 0.6 + 100/200 * 0.4 = 0.3 + 0.2 = 0.5
|
||||
self.assertEqual(fetch_sources.engagement_score(250, 100), 0.5)
|
||||
|
||||
def test_negative_clamped(self):
|
||||
self.assertEqual(fetch_sources.engagement_score(-50, -5), 0.0)
|
||||
|
||||
|
||||
class TestWindow(unittest.TestCase):
|
||||
def test_date_to_unix_utc(self):
|
||||
self.assertEqual(fetch_sources.date_to_unix("1970-01-02"), 86400)
|
||||
|
||||
def test_window_ordering(self):
|
||||
start, end = fetch_sources.window(30)
|
||||
self.assertLess(start, end)
|
||||
|
||||
|
||||
class TestHackerNews(unittest.TestCase):
|
||||
def test_filters_low_points_and_sorts(self):
|
||||
payload = {"hits": [
|
||||
{"title": "Low", "objectID": "1", "points": 1, "num_comments": 0,
|
||||
"created_at": "2026-07-01T00:00:00Z"},
|
||||
{"title": "Mid", "objectID": "2", "points": 50, "num_comments": 10,
|
||||
"created_at": "2026-07-02T00:00:00Z", "url": "https://example.com/mid"},
|
||||
{"title": "High", "objectID": "3", "points": 400, "num_comments": 150,
|
||||
"created_at": "2026-07-03T00:00:00Z"},
|
||||
]}
|
||||
with mock.patch.object(fetch_sources.urllib.request, "urlopen",
|
||||
return_value=_fake_response(payload)):
|
||||
items = fetch_sources.search_hackernews("test topic", "2026-06-21", "2026-07-21")
|
||||
self.assertEqual([i["title"] for i in items], ["High", "Mid"])
|
||||
self.assertEqual(items[0]["discussion_url"],
|
||||
"https://news.ycombinator.com/item?id=3")
|
||||
# story without a url falls back to the HN discussion link
|
||||
self.assertEqual(items[0]["url"], "https://news.ycombinator.com/item?id=3")
|
||||
self.assertEqual(items[1]["url"], "https://example.com/mid")
|
||||
|
||||
def test_multiword_query_sets_optional_words(self):
|
||||
captured = {}
|
||||
|
||||
def fake_urlopen(req, timeout=0):
|
||||
captured["url"] = req.full_url
|
||||
return _fake_response({"hits": []})
|
||||
|
||||
with mock.patch.object(fetch_sources.urllib.request, "urlopen", fake_urlopen):
|
||||
fetch_sources.search_hackernews("alpha beta gamma", "2026-06-21", "2026-07-21")
|
||||
self.assertIn("optionalWords=beta+gamma", captured["url"])
|
||||
self.assertIn("numericFilters=created_at_i%3E", captured["url"])
|
||||
|
||||
|
||||
class TestReddit(unittest.TestCase):
|
||||
PAYLOAD = {"data": {"children": [
|
||||
{"data": {"title": "Post A", "permalink": "/r/test/comments/a/", "score": 500,
|
||||
"num_comments": 200, "subreddit": "test", "created_utc": 86400}},
|
||||
{"data": {"title": "Dup A", "permalink": "/r/test/comments/a/", "score": 1,
|
||||
"num_comments": 0, "subreddit": "test", "created_utc": 86400}},
|
||||
{"data": {"title": "Post B", "permalink": "/r/test/comments/b/", "score": 10,
|
||||
"num_comments": 2, "subreddit": "test", "created_utc": 86400}},
|
||||
]}}
|
||||
|
||||
def test_dedupes_by_url_and_scores(self):
|
||||
with mock.patch.object(fetch_sources.urllib.request, "urlopen",
|
||||
return_value=_fake_response(self.PAYLOAD)):
|
||||
items = fetch_sources.search_reddit("test")
|
||||
self.assertEqual(len(items), 2)
|
||||
self.assertEqual(items[0]["title"], "Post A")
|
||||
self.assertEqual(items[0]["relevance"], 1.0)
|
||||
self.assertEqual(items[0]["date"], "1970-01-02")
|
||||
|
||||
def test_subreddit_scoping(self):
|
||||
captured = {}
|
||||
|
||||
def fake_urlopen(req, timeout=0):
|
||||
captured["url"] = req.full_url
|
||||
return _fake_response({"data": {"children": []}})
|
||||
|
||||
with mock.patch.object(fetch_sources.urllib.request, "urlopen", fake_urlopen):
|
||||
fetch_sources.search_reddit("topic", subreddit="r/LocalLLaMA")
|
||||
self.assertIn("/r/LocalLLaMA/search.json", captured["url"])
|
||||
self.assertIn("restrict_sr=on", captured["url"])
|
||||
|
||||
|
||||
class TestPolymarket(unittest.TestCase):
|
||||
def test_dedupes_across_pages_and_sorts_by_volume(self):
|
||||
pages = [
|
||||
{"events": [{"id": "1", "slug": "small", "title": "Small", "volume1mo": 100},
|
||||
{"id": "2", "slug": "big", "title": "Big", "volume1mo": 9000}]},
|
||||
{"events": [{"id": "1", "slug": "small", "title": "Small", "volume1mo": 100}]},
|
||||
{"events": []},
|
||||
]
|
||||
responses = iter(pages)
|
||||
with mock.patch.object(fetch_sources.urllib.request, "urlopen",
|
||||
side_effect=lambda *a, **k: _fake_response(next(responses))):
|
||||
items = fetch_sources.search_polymarket("election")
|
||||
self.assertEqual([i["title"] for i in items], ["Big", "Small"])
|
||||
self.assertEqual(items[0]["url"], "https://polymarket.com/event/big")
|
||||
|
||||
|
||||
class TestMainDegradedMode(unittest.TestCase):
|
||||
def test_source_error_is_partial_not_fatal(self):
|
||||
"""A failing source annotates status and exit stays 0 (upstream behavior)."""
|
||||
def fake_urlopen(req, timeout=0):
|
||||
raise fetch_sources.urllib.error.HTTPError(
|
||||
req.full_url, 403, "Forbidden", None, None)
|
||||
|
||||
stdout = io.StringIO()
|
||||
with mock.patch.object(fetch_sources.urllib.request, "urlopen", fake_urlopen), \
|
||||
mock.patch.object(sys, "stdout", stdout), \
|
||||
mock.patch.object(sys, "stderr", io.StringIO()):
|
||||
rc = fetch_sources.main(["topic", "--sources", "reddit", "--format", "json"])
|
||||
self.assertEqual(rc, 0)
|
||||
out = json.loads(stdout.getvalue())
|
||||
self.assertIn("error", out["results"]["reddit"]["status"])
|
||||
self.assertEqual(out["results"]["reddit"]["items"], [])
|
||||
|
||||
def test_markdown_output_shape(self):
|
||||
payload = {"hits": [{"title": "T", "objectID": "9", "points": 10,
|
||||
"num_comments": 3, "created_at": "2026-07-01T00:00:00Z"}]}
|
||||
stdout = io.StringIO()
|
||||
with mock.patch.object(fetch_sources.urllib.request, "urlopen",
|
||||
return_value=_fake_response(payload)), \
|
||||
mock.patch.object(sys, "stdout", stdout):
|
||||
rc = fetch_sources.main(["topic", "--sources", "hackernews"])
|
||||
self.assertEqual(rc, 0)
|
||||
text = stdout.getvalue()
|
||||
self.assertIn("# last30days: topic", text)
|
||||
self.assertIn("## hackernews (1 items, status: ok)", text)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
|
@ -193,6 +193,7 @@ hermes skills uninstall <skill-name>
|
|||
| [**drug-discovery**](/docs/user-guide/skills/optional/research/research-drug-discovery) | Pharmaceutical research assistant for drug discovery workflows. Search bioactive compounds on ChEMBL, calculate drug-likeness (Lipinski Ro5, QED, TPSA, synthetic accessibility), look up drug-drug interactions via OpenFDA, interpret ADMET... |
|
||||
| [**duckduckgo-search**](/docs/user-guide/skills/optional/research/research-duckduckgo-search) | Free web search via DuckDuckGo — text, news, images, videos. No API key needed. Prefer the `ddgs` CLI when installed; use the Python DDGS library only after verifying that `ddgs` is available in the current runtime. |
|
||||
| [**gitnexus-explorer**](/docs/user-guide/skills/optional/research/research-gitnexus-explorer) | Index a codebase with GitNexus and serve an interactive knowledge graph via web UI + Cloudflare tunnel. |
|
||||
| [**last30days**](/docs/user-guide/skills/optional/research/research-last30days) | Research what people said about a topic in the last 30 days. |
|
||||
| [**osint-investigation**](/docs/user-guide/skills/optional/research/research-osint-investigation) | Public-records OSINT investigation framework — SEC EDGAR filings, USAspending contracts, Senate lobbying, OFAC sanctions, ICIJ offshore leaks, NYC property records (ACRIS), OpenCorporates registries, CourtListener court records, Wayback... |
|
||||
| [**parallel-cli**](/docs/user-guide/skills/optional/research/research-parallel-cli) | Optional vendor skill for Parallel CLI — agent-native web search, extraction, deep research, enrichment, FindAll, and monitoring. Prefer JSON output and non-interactive flows. |
|
||||
| [**qmd**](/docs/user-guide/skills/optional/research/research-qmd) | Search personal knowledge bases, notes, docs, and meeting transcripts locally using qmd — a hybrid retrieval engine with BM25, vector search, and LLM reranking. Supports CLI and MCP integration. |
|
||||
|
|
|
|||
|
|
@ -0,0 +1,140 @@
|
|||
---
|
||||
title: "Last30Days — Research what people said about a topic in the last 30 days"
|
||||
sidebar_label: "Last30Days"
|
||||
description: "Research what people said about a topic in the last 30 days"
|
||||
---
|
||||
|
||||
{/* This page is auto-generated from the skill's SKILL.md by website/scripts/generate-skill-docs.py. Edit the source SKILL.md, not this page. */}
|
||||
|
||||
# Last30Days
|
||||
|
||||
Research what people said about a topic in the last 30 days.
|
||||
|
||||
## Skill metadata
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Source | Optional — install with `hermes skills install official/research/last30days` |
|
||||
| Path | `optional-skills/research/last30days` |
|
||||
| Version | `0.1.0` |
|
||||
| Author | Matt Van Horn (mvanhorn), Hermes Agent |
|
||||
| License | MIT |
|
||||
| Platforms | linux, macos, windows |
|
||||
| Tags | `Research`, `Social-Media`, `Trends`, `News` |
|
||||
| Related skills | [`polymarket`](/docs/user-guide/skills/bundled/research/research-polymarket), [`duckduckgo-search`](/docs/user-guide/skills/optional/research/research-duckduckgo-search) |
|
||||
|
||||
## Reference: full SKILL.md
|
||||
|
||||
:::info
|
||||
The following is the complete skill definition that Hermes loads when this skill is triggered. This is what the agent sees as instructions when the skill is active.
|
||||
:::
|
||||
|
||||
# last30days
|
||||
|
||||
Researches a topic across Hacker News, Reddit, Polymarket, X, YouTube, and the
|
||||
general web over a recent time window (default 30 days), then synthesizes a
|
||||
grounded summary of what people are actually saying, with engagement signals
|
||||
and citations. The bundled fetch script is pure Python stdlib and fully
|
||||
keyless; API keys are optional enhancers only. This is a slimmed Hermes port of
|
||||
the upstream last30days skill (github.com/mvanhorn/last30days-skill, MIT) — it
|
||||
does not include upstream's TikTok/Instagram/Bluesky/Digg integrations or its
|
||||
setup wizard.
|
||||
|
||||
## When to Use
|
||||
|
||||
- "What are people saying about X lately?" / community-sentiment questions
|
||||
- Recency-bound research: launches, earnings reactions, drama, hiring signals
|
||||
- Comparing tools/products by recent community chatter ("X vs Y")
|
||||
- Checking prediction-market odds alongside social discussion
|
||||
- NOT for deep historical research or single-document lookups — use plain
|
||||
`web_search`/`web_extract` for those
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python 3.10+ (stdlib only — no pip installs)
|
||||
- Network access to reddit.com, hn.algolia.com, gamma-api.polymarket.com
|
||||
- No env vars required. Everything runs keyless in degraded-but-useful mode.
|
||||
Optional enhancers (document in the user's own `.env`, never required):
|
||||
- `XAI_API_KEY` — richer X/Twitter coverage upstream; in this port, X
|
||||
coverage comes from `web_search` instead
|
||||
- `BRAVE_API_KEY` — upstream web-search backend; unnecessary here because
|
||||
Hermes has native `web_search`
|
||||
|
||||
## How to Run
|
||||
|
||||
Run the fetch script through the `terminal` tool, then synthesize:
|
||||
|
||||
```
|
||||
python3 ~/.hermes/skills/research/last30days/scripts/fetch_sources.py "your topic" --days 30 --format md
|
||||
```
|
||||
|
||||
Cover X, YouTube, and the general web with Hermes `web_search` (the script
|
||||
deliberately does not scrape those), then write the summary per the Procedure.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Command | Purpose |
|
||||
|---|---|
|
||||
| `fetch_sources.py "topic"` | All keyless sources, markdown output |
|
||||
| `fetch_sources.py "topic" --format json` | Machine-readable output |
|
||||
| `fetch_sources.py "topic" --days 7` | Narrower recency window |
|
||||
| `fetch_sources.py "topic" --sources hackernews,polymarket` | Subset of sources |
|
||||
| `fetch_sources.py "topic" --subreddit LocalLLaMA` | Scope Reddit to one sub |
|
||||
| `fetch_sources.py "topic" --limit 10` | Cap items per source |
|
||||
|
||||
Sources: `hackernews` (Algolia, date-filtered), `reddit` (public JSON search,
|
||||
`t=month`), `polymarket` (active events by 30-day volume).
|
||||
|
||||
## Procedure
|
||||
|
||||
1. Run the fetcher via `terminal`:
|
||||
`python3 ~/.hermes/skills/research/last30days/scripts/fetch_sources.py "TOPIC" --format md`
|
||||
Note each source's `status:` line — `error` means degraded coverage for
|
||||
that source, not a failed run.
|
||||
2. Fill the gaps with Hermes-native tools (batch these searches):
|
||||
- `web_search`: `TOPIC site:x.com` or `TOPIC twitter reaction` for X chatter
|
||||
- `web_search`: `TOPIC site:youtube.com` (or `TOPIC review video`) for YouTube
|
||||
- `web_search`: `TOPIC` plus a recency phrase like "this month" for news/web
|
||||
3. Pull full text of the 2-4 most-cited pages with `web_extract` when a
|
||||
headline alone can't support a claim.
|
||||
4. Discard anything clearly older than the window. The script date-filters HN
|
||||
and Reddit; web results need manual checking against publish dates.
|
||||
5. Synthesize. Upstream's output contract, kept here because it works:
|
||||
- Lead with `What I learned:` then bold-lead-in paragraphs — no invented
|
||||
title, no `##` section headers in the body
|
||||
- Quote real people verbatim with source attribution (subreddit, HN
|
||||
thread, channel); prefer high-engagement items (the script pre-sorts
|
||||
by an engagement score: 0.6·score + 0.4·comments, each capped)
|
||||
- End with a numbered `KEY PATTERNS from the research:` list
|
||||
- Never fabricate quotes, titles, or engagement numbers; if a source
|
||||
returned nothing or errored, say "partial coverage" — not "nothing
|
||||
was said on <source>"
|
||||
- Include Polymarket odds only when a market genuinely matches the topic
|
||||
6. For comparison topics ("A vs B"), run steps 1-3 once per entity, then
|
||||
structure as: verdict, per-entity findings, head-to-head, bottom line.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **Reddit public JSON returns HTTP 403 intermittently** — upstream migrated
|
||||
off it for this reason. When the `reddit` source reports an error, fall back
|
||||
to `web_search` with `TOPIC site:reddit.com` and treat Reddit as partial
|
||||
coverage. Do not retry in a loop.
|
||||
- HN Algolia rejects `points>N` in `numericFilters` (HTTP 400) — the script
|
||||
filters low-engagement stories client-side instead; don't "fix" that.
|
||||
- Polymarket's search matches loosely; a topic like "React hooks" can return
|
||||
unrelated political markets. Drop non-matching events during synthesis.
|
||||
- Engagement scores compare items *within* a source only — a 500-point HN
|
||||
story and a 500-upvote Reddit post are not equivalent audiences.
|
||||
- Windows: invoke as `python` if `python3` is not on PATH; the script itself
|
||||
is portable (no `/tmp`, no shell pipelines).
|
||||
- Don't dump the script's raw item list as the answer — it is evidence for
|
||||
synthesis, not output.
|
||||
|
||||
## Verification
|
||||
|
||||
```
|
||||
python3 ~/.hermes/skills/research/last30days/scripts/fetch_sources.py "artificial intelligence" --sources hackernews --limit 3 --format json
|
||||
```
|
||||
|
||||
Should print JSON with `"status": "ok"` and up to 3 dated HN stories from the
|
||||
last 30 days. Exit code 0 even on partial source failures.
|
||||
|
|
@ -554,6 +554,7 @@ const sidebars: SidebarsConfig = {
|
|||
'user-guide/skills/optional/research/research-drug-discovery',
|
||||
'user-guide/skills/optional/research/research-duckduckgo-search',
|
||||
'user-guide/skills/optional/research/research-gitnexus-explorer',
|
||||
'user-guide/skills/optional/research/research-last30days',
|
||||
'user-guide/skills/optional/research/research-osint-investigation',
|
||||
'user-guide/skills/optional/research/research-parallel-cli',
|
||||
'user-guide/skills/optional/research/research-qmd',
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue