diff --git a/optional-skills/research/last30days/SKILL.md b/optional-skills/research/last30days/SKILL.md new file mode 100644 index 00000000000..1a09314b2b2 --- /dev/null +++ b/optional-skills/research/last30days/SKILL.md @@ -0,0 +1,122 @@ +--- +name: last30days +description: "Research what people said about a topic in the last 30 days." +version: 0.1.0 +author: Matt Van Horn (mvanhorn), Hermes Agent +license: MIT +platforms: [linux, macos, windows] +metadata: + hermes: + tags: [Research, Social-Media, Trends, News] + related_skills: [polymarket, duckduckgo-search] +--- + +# last30days + +Researches a topic across Hacker News, Reddit, Polymarket, X, YouTube, and the +general web over a recent time window (default 30 days), then synthesizes a +grounded summary of what people are actually saying, with engagement signals +and citations. The bundled fetch script is pure Python stdlib and fully +keyless; API keys are optional enhancers only. This is a slimmed Hermes port of +the upstream last30days skill (github.com/mvanhorn/last30days-skill, MIT) — it +does not include upstream's TikTok/Instagram/Bluesky/Digg integrations or its +setup wizard. + +## When to Use + +- "What are people saying about X lately?" / community-sentiment questions +- Recency-bound research: launches, earnings reactions, drama, hiring signals +- Comparing tools/products by recent community chatter ("X vs Y") +- Checking prediction-market odds alongside social discussion +- NOT for deep historical research or single-document lookups — use plain + `web_search`/`web_extract` for those + +## Prerequisites + +- Python 3.10+ (stdlib only — no pip installs) +- Network access to reddit.com, hn.algolia.com, gamma-api.polymarket.com +- No env vars required. Everything runs keyless in degraded-but-useful mode. + Optional enhancers (document in the user's own `.env`, never required): + - `XAI_API_KEY` — richer X/Twitter coverage upstream; in this port, X + coverage comes from `web_search` instead + - `BRAVE_API_KEY` — upstream web-search backend; unnecessary here because + Hermes has native `web_search` + +## How to Run + +Run the fetch script through the `terminal` tool, then synthesize: + +``` +python3 ~/.hermes/skills/research/last30days/scripts/fetch_sources.py "your topic" --days 30 --format md +``` + +Cover X, YouTube, and the general web with Hermes `web_search` (the script +deliberately does not scrape those), then write the summary per the Procedure. + +## Quick Reference + +| Command | Purpose | +|---|---| +| `fetch_sources.py "topic"` | All keyless sources, markdown output | +| `fetch_sources.py "topic" --format json` | Machine-readable output | +| `fetch_sources.py "topic" --days 7` | Narrower recency window | +| `fetch_sources.py "topic" --sources hackernews,polymarket` | Subset of sources | +| `fetch_sources.py "topic" --subreddit LocalLLaMA` | Scope Reddit to one sub | +| `fetch_sources.py "topic" --limit 10` | Cap items per source | + +Sources: `hackernews` (Algolia, date-filtered), `reddit` (public JSON search, +`t=month`), `polymarket` (active events by 30-day volume). + +## Procedure + +1. Run the fetcher via `terminal`: + `python3 ~/.hermes/skills/research/last30days/scripts/fetch_sources.py "TOPIC" --format md` + Note each source's `status:` line — `error` means degraded coverage for + that source, not a failed run. +2. Fill the gaps with Hermes-native tools (batch these searches): + - `web_search`: `TOPIC site:x.com` or `TOPIC twitter reaction` for X chatter + - `web_search`: `TOPIC site:youtube.com` (or `TOPIC review video`) for YouTube + - `web_search`: `TOPIC` plus a recency phrase like "this month" for news/web +3. Pull full text of the 2-4 most-cited pages with `web_extract` when a + headline alone can't support a claim. +4. Discard anything clearly older than the window. The script date-filters HN + and Reddit; web results need manual checking against publish dates. +5. Synthesize. Upstream's output contract, kept here because it works: + - Lead with `What I learned:` then bold-lead-in paragraphs — no invented + title, no `##` section headers in the body + - Quote real people verbatim with source attribution (subreddit, HN + thread, channel); prefer high-engagement items (the script pre-sorts + by an engagement score: 0.6·score + 0.4·comments, each capped) + - End with a numbered `KEY PATTERNS from the research:` list + - Never fabricate quotes, titles, or engagement numbers; if a source + returned nothing or errored, say "partial coverage" — not "nothing + was said on " + - Include Polymarket odds only when a market genuinely matches the topic +6. For comparison topics ("A vs B"), run steps 1-3 once per entity, then + structure as: verdict, per-entity findings, head-to-head, bottom line. + +## Pitfalls + +- **Reddit public JSON returns HTTP 403 intermittently** — upstream migrated + off it for this reason. When the `reddit` source reports an error, fall back + to `web_search` with `TOPIC site:reddit.com` and treat Reddit as partial + coverage. Do not retry in a loop. +- HN Algolia rejects `points>N` in `numericFilters` (HTTP 400) — the script + filters low-engagement stories client-side instead; don't "fix" that. +- Polymarket's search matches loosely; a topic like "React hooks" can return + unrelated political markets. Drop non-matching events during synthesis. +- Engagement scores compare items *within* a source only — a 500-point HN + story and a 500-upvote Reddit post are not equivalent audiences. +- Windows: invoke as `python` if `python3` is not on PATH; the script itself + is portable (no `/tmp`, no shell pipelines). +- Don't dump the script's raw item list as the answer — it is evidence for + synthesis, not output. + +## Verification + +``` +python3 ~/.hermes/skills/research/last30days/scripts/fetch_sources.py "artificial intelligence" --sources hackernews --limit 3 --format json +``` + +Should print JSON with `"status": "ok"` and up to 3 dated HN stories from the +last 30 days. Exit code 0 even on partial source failures. diff --git a/optional-skills/research/last30days/scripts/fetch_sources.py b/optional-skills/research/last30days/scripts/fetch_sources.py new file mode 100644 index 00000000000..212da92ad0a --- /dev/null +++ b/optional-skills/research/last30days/scripts/fetch_sources.py @@ -0,0 +1,222 @@ +#!/usr/bin/env python3 +"""Keyless multi-source fetcher for the last30days skill. + +Queries three free JSON APIs over a recency window and emits scored, +normalized items as JSON or markdown: + +- Hacker News : hn.algolia.com/api/v1/search (created_at_i numeric filters) +- Reddit : www.reddit.com/search.json (t=month; may 403 — see SKILL.md) +- Polymarket : gamma-api.polymarket.com/public-search (active events) + +Pure stdlib. No API keys. Endpoints and query parameters ported from +mvanhorn/last30days-skill (MIT). X/YouTube/web coverage is handled by the +hosting agent's own search tools, not this script. +""" + +from __future__ import annotations + +import argparse +import datetime as _dt +import json +import sys +import urllib.error +import urllib.parse +import urllib.request + +USER_AGENT = "last30days-hermes/0.1 (research skill; +https://github.com/mvanhorn/last30days-skill)" + +HN_SEARCH_URL = "https://hn.algolia.com/api/v1/search" +REDDIT_SEARCH_URL = "https://www.reddit.com/search.json" +POLYMARKET_SEARCH_URL = "https://gamma-api.polymarket.com/public-search" + +MIN_HN_POINTS = 2 # upstream filters low-engagement stories client-side + + +def _get_json(url: str, timeout: int = 20): + req = urllib.request.Request(url, headers={"User-Agent": USER_AGENT}) + with urllib.request.urlopen(req, timeout=timeout) as resp: + return json.loads(resp.read().decode("utf-8", errors="replace")) + + +def engagement_score(score: int, num_comments: int) -> float: + """Engagement-based relevance in [0, 1] (ported from upstream reddit_public).""" + score_component = min(1.0, max(0.0, score / 500.0)) + comments_component = min(1.0, max(0.0, num_comments / 200.0)) + return round((score_component * 0.6) + (comments_component * 0.4), 3) + + +def date_to_unix(date_str: str) -> int: + """YYYY-MM-DD -> Unix timestamp at start of day UTC.""" + y, m, d = (int(p) for p in date_str.split("-")) + return int(_dt.datetime(y, m, d, tzinfo=_dt.timezone.utc).timestamp()) + + +def window(days: int) -> tuple[str, str]: + today = _dt.datetime.now(_dt.timezone.utc).date() + return ((today - _dt.timedelta(days=days)).isoformat(), today.isoformat()) + + +def search_hackernews(topic: str, from_date: str, to_date: str, limit: int = 30) -> list[dict]: + from_ts = date_to_unix(from_date) + to_ts = date_to_unix(to_date) + 86400 # include end date + params = { + "query": topic, + "tags": "story", + "numericFilters": f"created_at_i>{from_ts},created_at_i<{to_ts}", + "hitsPerPage": str(limit * 2), # overfetch, then filter low engagement + } + # Algolia ANDs query tokens; mark all-but-first optional so multi-word + # topics rank by token overlap instead of requiring every word. + tokens = topic.split() + if len(tokens) > 1: + params["optionalWords"] = " ".join(tokens[1:]) + data = _get_json(f"{HN_SEARCH_URL}?{urllib.parse.urlencode(params)}") + items = [] + for hit in data.get("hits", []): + points = hit.get("points") or 0 + if points <= MIN_HN_POINTS: + continue + object_id = hit.get("objectID", "") + items.append({ + "source": "hackernews", + "title": hit.get("title") or "", + "url": hit.get("url") or f"https://news.ycombinator.com/item?id={object_id}", + "discussion_url": f"https://news.ycombinator.com/item?id={object_id}", + "date": (hit.get("created_at") or "")[:10], + "points": points, + "comments": hit.get("num_comments") or 0, + "relevance": engagement_score(points, hit.get("num_comments") or 0), + }) + items.sort(key=lambda i: i["relevance"], reverse=True) + return items[:limit] + + +def search_reddit(topic: str, limit: int = 25, subreddit: str | None = None) -> list[dict]: + """Reddit public JSON search, t=month. May return HTTP 403 (endpoint is + unreliable per upstream); callers should fall back to agent web search.""" + if subreddit: + base = f"https://www.reddit.com/r/{subreddit.removeprefix('r/')}/search.json" + params = {"q": topic, "restrict_sr": "on", "sort": "relevance", "t": "month", + "limit": str(limit), "raw_json": "1"} + else: + base = REDDIT_SEARCH_URL + params = {"q": topic, "sort": "relevance", "t": "month", + "limit": str(limit), "raw_json": "1"} + data = _get_json(f"{base}?{urllib.parse.urlencode(params)}") + items, seen = [], set() + for child in (data.get("data") or {}).get("children", []): + post = child.get("data") or {} + permalink = post.get("permalink") or "" + url = f"https://www.reddit.com{permalink}" + if url in seen: + continue + seen.add(url) + score = post.get("score") or 0 + num_comments = post.get("num_comments") or 0 + created = post.get("created_utc") + items.append({ + "source": "reddit", + "title": post.get("title") or "", + "url": url, + "subreddit": post.get("subreddit") or "", + "date": _dt.datetime.fromtimestamp(created, tz=_dt.timezone.utc).date().isoformat() + if created else "", + "points": score, + "comments": num_comments, + "relevance": engagement_score(score, num_comments), + }) + items.sort(key=lambda i: i["relevance"], reverse=True) + return items[:limit] + + +def search_polymarket(topic: str, pages: int = 3, limit: int = 15) -> list[dict]: + """Active prediction markets matching the topic, sorted by volume.""" + events: dict[str, dict] = {} + for page in range(1, pages + 1): + params = {"q": topic, "page": str(page), + "events_status": "active", "keep_closed_markets": "0"} + try: + data = _get_json(f"{POLYMARKET_SEARCH_URL}?{urllib.parse.urlencode(params)}") + except (urllib.error.URLError, urllib.error.HTTPError, OSError): + break + batch = data.get("events") or [] + if not batch: + break + for ev in batch: + eid = str(ev.get("id") or ev.get("slug") or "") + if eid and eid not in events: + events[eid] = ev + items = [] + for eid, ev in events.items(): + slug = ev.get("slug") or "" + volume = float(ev.get("volume1mo") or ev.get("volume") or 0) + items.append({ + "source": "polymarket", + "title": ev.get("title") or "", + "url": f"https://polymarket.com/event/{slug}" if slug + else f"https://polymarket.com/event/{eid}", + "volume_1mo": volume, + "liquidity": float(ev.get("liquidity") or 0), + }) + items.sort(key=lambda i: i["volume_1mo"], reverse=True) + return items[:limit] + + +def render_markdown(topic: str, from_date: str, to_date: str, results: dict) -> str: + lines = [f"# last30days: {topic} ({from_date} to {to_date})", ""] + for source, payload in results.items(): + items = payload["items"] + status = payload["status"] + lines.append(f"## {source} ({len(items)} items, status: {status})") + for it in items: + meta = [] + if "points" in it: + meta.append(f"{it['points']} pts, {it['comments']} comments") + if "volume_1mo" in it: + meta.append(f"${it['volume_1mo']:,.0f} 30-day volume") + if it.get("date"): + meta.append(it["date"]) + lines.append(f"- [{it['title']}]({it['url']}) ({'; '.join(meta)})") + lines.append("") + return "\n".join(lines) + + +def main(argv: list[str] | None = None) -> int: + ap = argparse.ArgumentParser(description="Keyless last-30-days source fetcher") + ap.add_argument("topic") + ap.add_argument("--days", type=int, default=30) + ap.add_argument("--sources", default="hackernews,reddit,polymarket", + help="comma list: hackernews,reddit,polymarket") + ap.add_argument("--limit", type=int, default=20, help="max items per source") + ap.add_argument("--subreddit", default=None, help="scope reddit search to one subreddit") + ap.add_argument("--format", choices=["json", "md"], default="md") + args = ap.parse_args(argv) + + from_date, to_date = window(args.days) + wanted = {s.strip() for s in args.sources.split(",") if s.strip()} + fetchers = { + "hackernews": lambda: search_hackernews(args.topic, from_date, to_date, args.limit), + "reddit": lambda: search_reddit(args.topic, args.limit, args.subreddit), + "polymarket": lambda: search_polymarket(args.topic, limit=args.limit), + } + results: dict[str, dict] = {} + for name in ("hackernews", "reddit", "polymarket"): + if name not in wanted: + continue + try: + results[name] = {"status": "ok", "items": fetchers[name]()} + except Exception as exc: # degraded, not fatal — mirror upstream behavior + results[name] = {"status": f"error: {exc}", "items": []} + print(f"[last30days] {name} failed: {exc}", file=sys.stderr) + + if args.format == "json": + print(json.dumps({"topic": args.topic, "from": from_date, "to": to_date, + "results": results}, indent=2)) + else: + print(render_markdown(args.topic, from_date, to_date, results)) + # exit 0 even on partial coverage (upstream default); failures are annotated + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tests/skills/test_last30days_skill.py b/tests/skills/test_last30days_skill.py new file mode 100644 index 00000000000..9ffe43f0197 --- /dev/null +++ b/tests/skills/test_last30days_skill.py @@ -0,0 +1,214 @@ +"""Tests for the last30days optional skill (frontmatter + script logic). + +No live network calls: urllib.request.urlopen is mocked everywhere. +""" + +import io +import json +import sys +import unittest +from pathlib import Path +from unittest import mock + +import yaml + +TESTS_DIR = Path(__file__).resolve().parent +SKILL_DIR = TESTS_DIR.parent.parent / "optional-skills" / "research" / "last30days" +SKILL_MD = SKILL_DIR / "SKILL.md" +SCRIPT = SKILL_DIR / "scripts" / "fetch_sources.py" + +sys.path.insert(0, str(SCRIPT.parent)) +import fetch_sources # noqa: E402 + + +def _fake_response(payload): + body = json.dumps(payload).encode("utf-8") + + class _Resp(io.BytesIO): + def __enter__(self): + return self + + def __exit__(self, *args): + self.close() + + return _Resp(body) + + +class TestFrontmatter(unittest.TestCase): + @classmethod + def setUpClass(cls): + text = SKILL_MD.read_text(encoding="utf-8") + assert text.startswith("---\n"), "SKILL.md must start with YAML frontmatter" + cls.fm = yaml.safe_load(text.split("---", 2)[1]) + cls.body = text.split("---", 2)[2] + + def test_required_fields(self): + for field in ("name", "description", "version", "author", "license", "platforms"): + self.assertIn(field, self.fm, f"missing frontmatter field: {field}") + + def test_name(self): + self.assertEqual(self.fm["name"], "last30days") + + def test_description_length_and_shape(self): + desc = self.fm["description"] + self.assertLessEqual(len(desc), 60, f"description is {len(desc)} chars (max 60)") + self.assertTrue(desc.endswith("."), "description must end with a period") + + def test_version_and_license(self): + self.assertEqual(str(self.fm["version"]), "0.1.0") + self.assertEqual(self.fm["license"], "MIT") + + def test_platforms(self): + self.assertEqual(set(self.fm["platforms"]), {"linux", "macos", "windows"}) + + def test_hermes_metadata(self): + hermes = self.fm["metadata"]["hermes"] + self.assertIsInstance(hermes["tags"], list) + self.assertTrue(hermes["tags"]) + self.assertIsInstance(hermes["related_skills"], list) + + def test_body_sections_present(self): + for section in ("## When to Use", "## Prerequisites", "## How to Run", + "## Quick Reference", "## Procedure", "## Pitfalls", + "## Verification"): + self.assertIn(section, self.body, f"missing section: {section}") + + +class TestEngagementScore(unittest.TestCase): + def test_zero(self): + self.assertEqual(fetch_sources.engagement_score(0, 0), 0.0) + + def test_capped_at_one(self): + self.assertEqual(fetch_sources.engagement_score(10_000, 10_000), 1.0) + + def test_weighting(self): + # 250/500 * 0.6 + 100/200 * 0.4 = 0.3 + 0.2 = 0.5 + self.assertEqual(fetch_sources.engagement_score(250, 100), 0.5) + + def test_negative_clamped(self): + self.assertEqual(fetch_sources.engagement_score(-50, -5), 0.0) + + +class TestWindow(unittest.TestCase): + def test_date_to_unix_utc(self): + self.assertEqual(fetch_sources.date_to_unix("1970-01-02"), 86400) + + def test_window_ordering(self): + start, end = fetch_sources.window(30) + self.assertLess(start, end) + + +class TestHackerNews(unittest.TestCase): + def test_filters_low_points_and_sorts(self): + payload = {"hits": [ + {"title": "Low", "objectID": "1", "points": 1, "num_comments": 0, + "created_at": "2026-07-01T00:00:00Z"}, + {"title": "Mid", "objectID": "2", "points": 50, "num_comments": 10, + "created_at": "2026-07-02T00:00:00Z", "url": "https://example.com/mid"}, + {"title": "High", "objectID": "3", "points": 400, "num_comments": 150, + "created_at": "2026-07-03T00:00:00Z"}, + ]} + with mock.patch.object(fetch_sources.urllib.request, "urlopen", + return_value=_fake_response(payload)): + items = fetch_sources.search_hackernews("test topic", "2026-06-21", "2026-07-21") + self.assertEqual([i["title"] for i in items], ["High", "Mid"]) + self.assertEqual(items[0]["discussion_url"], + "https://news.ycombinator.com/item?id=3") + # story without a url falls back to the HN discussion link + self.assertEqual(items[0]["url"], "https://news.ycombinator.com/item?id=3") + self.assertEqual(items[1]["url"], "https://example.com/mid") + + def test_multiword_query_sets_optional_words(self): + captured = {} + + def fake_urlopen(req, timeout=0): + captured["url"] = req.full_url + return _fake_response({"hits": []}) + + with mock.patch.object(fetch_sources.urllib.request, "urlopen", fake_urlopen): + fetch_sources.search_hackernews("alpha beta gamma", "2026-06-21", "2026-07-21") + self.assertIn("optionalWords=beta+gamma", captured["url"]) + self.assertIn("numericFilters=created_at_i%3E", captured["url"]) + + +class TestReddit(unittest.TestCase): + PAYLOAD = {"data": {"children": [ + {"data": {"title": "Post A", "permalink": "/r/test/comments/a/", "score": 500, + "num_comments": 200, "subreddit": "test", "created_utc": 86400}}, + {"data": {"title": "Dup A", "permalink": "/r/test/comments/a/", "score": 1, + "num_comments": 0, "subreddit": "test", "created_utc": 86400}}, + {"data": {"title": "Post B", "permalink": "/r/test/comments/b/", "score": 10, + "num_comments": 2, "subreddit": "test", "created_utc": 86400}}, + ]}} + + def test_dedupes_by_url_and_scores(self): + with mock.patch.object(fetch_sources.urllib.request, "urlopen", + return_value=_fake_response(self.PAYLOAD)): + items = fetch_sources.search_reddit("test") + self.assertEqual(len(items), 2) + self.assertEqual(items[0]["title"], "Post A") + self.assertEqual(items[0]["relevance"], 1.0) + self.assertEqual(items[0]["date"], "1970-01-02") + + def test_subreddit_scoping(self): + captured = {} + + def fake_urlopen(req, timeout=0): + captured["url"] = req.full_url + return _fake_response({"data": {"children": []}}) + + with mock.patch.object(fetch_sources.urllib.request, "urlopen", fake_urlopen): + fetch_sources.search_reddit("topic", subreddit="r/LocalLLaMA") + self.assertIn("/r/LocalLLaMA/search.json", captured["url"]) + self.assertIn("restrict_sr=on", captured["url"]) + + +class TestPolymarket(unittest.TestCase): + def test_dedupes_across_pages_and_sorts_by_volume(self): + pages = [ + {"events": [{"id": "1", "slug": "small", "title": "Small", "volume1mo": 100}, + {"id": "2", "slug": "big", "title": "Big", "volume1mo": 9000}]}, + {"events": [{"id": "1", "slug": "small", "title": "Small", "volume1mo": 100}]}, + {"events": []}, + ] + responses = iter(pages) + with mock.patch.object(fetch_sources.urllib.request, "urlopen", + side_effect=lambda *a, **k: _fake_response(next(responses))): + items = fetch_sources.search_polymarket("election") + self.assertEqual([i["title"] for i in items], ["Big", "Small"]) + self.assertEqual(items[0]["url"], "https://polymarket.com/event/big") + + +class TestMainDegradedMode(unittest.TestCase): + def test_source_error_is_partial_not_fatal(self): + """A failing source annotates status and exit stays 0 (upstream behavior).""" + def fake_urlopen(req, timeout=0): + raise fetch_sources.urllib.error.HTTPError( + req.full_url, 403, "Forbidden", None, None) + + stdout = io.StringIO() + with mock.patch.object(fetch_sources.urllib.request, "urlopen", fake_urlopen), \ + mock.patch.object(sys, "stdout", stdout), \ + mock.patch.object(sys, "stderr", io.StringIO()): + rc = fetch_sources.main(["topic", "--sources", "reddit", "--format", "json"]) + self.assertEqual(rc, 0) + out = json.loads(stdout.getvalue()) + self.assertIn("error", out["results"]["reddit"]["status"]) + self.assertEqual(out["results"]["reddit"]["items"], []) + + def test_markdown_output_shape(self): + payload = {"hits": [{"title": "T", "objectID": "9", "points": 10, + "num_comments": 3, "created_at": "2026-07-01T00:00:00Z"}]} + stdout = io.StringIO() + with mock.patch.object(fetch_sources.urllib.request, "urlopen", + return_value=_fake_response(payload)), \ + mock.patch.object(sys, "stdout", stdout): + rc = fetch_sources.main(["topic", "--sources", "hackernews"]) + self.assertEqual(rc, 0) + text = stdout.getvalue() + self.assertIn("# last30days: topic", text) + self.assertIn("## hackernews (1 items, status: ok)", text) + + +if __name__ == "__main__": + unittest.main() diff --git a/website/docs/reference/optional-skills-catalog.md b/website/docs/reference/optional-skills-catalog.md index 94416573dc2..8d50467a95d 100644 --- a/website/docs/reference/optional-skills-catalog.md +++ b/website/docs/reference/optional-skills-catalog.md @@ -193,6 +193,7 @@ hermes skills uninstall | [**drug-discovery**](/docs/user-guide/skills/optional/research/research-drug-discovery) | Pharmaceutical research assistant for drug discovery workflows. Search bioactive compounds on ChEMBL, calculate drug-likeness (Lipinski Ro5, QED, TPSA, synthetic accessibility), look up drug-drug interactions via OpenFDA, interpret ADMET... | | [**duckduckgo-search**](/docs/user-guide/skills/optional/research/research-duckduckgo-search) | Free web search via DuckDuckGo — text, news, images, videos. No API key needed. Prefer the `ddgs` CLI when installed; use the Python DDGS library only after verifying that `ddgs` is available in the current runtime. | | [**gitnexus-explorer**](/docs/user-guide/skills/optional/research/research-gitnexus-explorer) | Index a codebase with GitNexus and serve an interactive knowledge graph via web UI + Cloudflare tunnel. | +| [**last30days**](/docs/user-guide/skills/optional/research/research-last30days) | Research what people said about a topic in the last 30 days. | | [**osint-investigation**](/docs/user-guide/skills/optional/research/research-osint-investigation) | Public-records OSINT investigation framework — SEC EDGAR filings, USAspending contracts, Senate lobbying, OFAC sanctions, ICIJ offshore leaks, NYC property records (ACRIS), OpenCorporates registries, CourtListener court records, Wayback... | | [**parallel-cli**](/docs/user-guide/skills/optional/research/research-parallel-cli) | Optional vendor skill for Parallel CLI — agent-native web search, extraction, deep research, enrichment, FindAll, and monitoring. Prefer JSON output and non-interactive flows. | | [**qmd**](/docs/user-guide/skills/optional/research/research-qmd) | Search personal knowledge bases, notes, docs, and meeting transcripts locally using qmd — a hybrid retrieval engine with BM25, vector search, and LLM reranking. Supports CLI and MCP integration. | diff --git a/website/docs/user-guide/skills/optional/research/research-last30days.md b/website/docs/user-guide/skills/optional/research/research-last30days.md new file mode 100644 index 00000000000..68838f08318 --- /dev/null +++ b/website/docs/user-guide/skills/optional/research/research-last30days.md @@ -0,0 +1,140 @@ +--- +title: "Last30Days — Research what people said about a topic in the last 30 days" +sidebar_label: "Last30Days" +description: "Research what people said about a topic in the last 30 days" +--- + +{/* This page is auto-generated from the skill's SKILL.md by website/scripts/generate-skill-docs.py. Edit the source SKILL.md, not this page. */} + +# Last30Days + +Research what people said about a topic in the last 30 days. + +## Skill metadata + +| | | +|---|---| +| Source | Optional — install with `hermes skills install official/research/last30days` | +| Path | `optional-skills/research/last30days` | +| Version | `0.1.0` | +| Author | Matt Van Horn (mvanhorn), Hermes Agent | +| License | MIT | +| Platforms | linux, macos, windows | +| Tags | `Research`, `Social-Media`, `Trends`, `News` | +| Related skills | [`polymarket`](/docs/user-guide/skills/bundled/research/research-polymarket), [`duckduckgo-search`](/docs/user-guide/skills/optional/research/research-duckduckgo-search) | + +## Reference: full SKILL.md + +:::info +The following is the complete skill definition that Hermes loads when this skill is triggered. This is what the agent sees as instructions when the skill is active. +::: + +# last30days + +Researches a topic across Hacker News, Reddit, Polymarket, X, YouTube, and the +general web over a recent time window (default 30 days), then synthesizes a +grounded summary of what people are actually saying, with engagement signals +and citations. The bundled fetch script is pure Python stdlib and fully +keyless; API keys are optional enhancers only. This is a slimmed Hermes port of +the upstream last30days skill (github.com/mvanhorn/last30days-skill, MIT) — it +does not include upstream's TikTok/Instagram/Bluesky/Digg integrations or its +setup wizard. + +## When to Use + +- "What are people saying about X lately?" / community-sentiment questions +- Recency-bound research: launches, earnings reactions, drama, hiring signals +- Comparing tools/products by recent community chatter ("X vs Y") +- Checking prediction-market odds alongside social discussion +- NOT for deep historical research or single-document lookups — use plain + `web_search`/`web_extract` for those + +## Prerequisites + +- Python 3.10+ (stdlib only — no pip installs) +- Network access to reddit.com, hn.algolia.com, gamma-api.polymarket.com +- No env vars required. Everything runs keyless in degraded-but-useful mode. + Optional enhancers (document in the user's own `.env`, never required): + - `XAI_API_KEY` — richer X/Twitter coverage upstream; in this port, X + coverage comes from `web_search` instead + - `BRAVE_API_KEY` — upstream web-search backend; unnecessary here because + Hermes has native `web_search` + +## How to Run + +Run the fetch script through the `terminal` tool, then synthesize: + +``` +python3 ~/.hermes/skills/research/last30days/scripts/fetch_sources.py "your topic" --days 30 --format md +``` + +Cover X, YouTube, and the general web with Hermes `web_search` (the script +deliberately does not scrape those), then write the summary per the Procedure. + +## Quick Reference + +| Command | Purpose | +|---|---| +| `fetch_sources.py "topic"` | All keyless sources, markdown output | +| `fetch_sources.py "topic" --format json` | Machine-readable output | +| `fetch_sources.py "topic" --days 7` | Narrower recency window | +| `fetch_sources.py "topic" --sources hackernews,polymarket` | Subset of sources | +| `fetch_sources.py "topic" --subreddit LocalLLaMA` | Scope Reddit to one sub | +| `fetch_sources.py "topic" --limit 10` | Cap items per source | + +Sources: `hackernews` (Algolia, date-filtered), `reddit` (public JSON search, +`t=month`), `polymarket` (active events by 30-day volume). + +## Procedure + +1. Run the fetcher via `terminal`: + `python3 ~/.hermes/skills/research/last30days/scripts/fetch_sources.py "TOPIC" --format md` + Note each source's `status:` line — `error` means degraded coverage for + that source, not a failed run. +2. Fill the gaps with Hermes-native tools (batch these searches): + - `web_search`: `TOPIC site:x.com` or `TOPIC twitter reaction` for X chatter + - `web_search`: `TOPIC site:youtube.com` (or `TOPIC review video`) for YouTube + - `web_search`: `TOPIC` plus a recency phrase like "this month" for news/web +3. Pull full text of the 2-4 most-cited pages with `web_extract` when a + headline alone can't support a claim. +4. Discard anything clearly older than the window. The script date-filters HN + and Reddit; web results need manual checking against publish dates. +5. Synthesize. Upstream's output contract, kept here because it works: + - Lead with `What I learned:` then bold-lead-in paragraphs — no invented + title, no `##` section headers in the body + - Quote real people verbatim with source attribution (subreddit, HN + thread, channel); prefer high-engagement items (the script pre-sorts + by an engagement score: 0.6·score + 0.4·comments, each capped) + - End with a numbered `KEY PATTERNS from the research:` list + - Never fabricate quotes, titles, or engagement numbers; if a source + returned nothing or errored, say "partial coverage" — not "nothing + was said on <source>" + - Include Polymarket odds only when a market genuinely matches the topic +6. For comparison topics ("A vs B"), run steps 1-3 once per entity, then + structure as: verdict, per-entity findings, head-to-head, bottom line. + +## Pitfalls + +- **Reddit public JSON returns HTTP 403 intermittently** — upstream migrated + off it for this reason. When the `reddit` source reports an error, fall back + to `web_search` with `TOPIC site:reddit.com` and treat Reddit as partial + coverage. Do not retry in a loop. +- HN Algolia rejects `points>N` in `numericFilters` (HTTP 400) — the script + filters low-engagement stories client-side instead; don't "fix" that. +- Polymarket's search matches loosely; a topic like "React hooks" can return + unrelated political markets. Drop non-matching events during synthesis. +- Engagement scores compare items *within* a source only — a 500-point HN + story and a 500-upvote Reddit post are not equivalent audiences. +- Windows: invoke as `python` if `python3` is not on PATH; the script itself + is portable (no `/tmp`, no shell pipelines). +- Don't dump the script's raw item list as the answer — it is evidence for + synthesis, not output. + +## Verification + +``` +python3 ~/.hermes/skills/research/last30days/scripts/fetch_sources.py "artificial intelligence" --sources hackernews --limit 3 --format json +``` + +Should print JSON with `"status": "ok"` and up to 3 dated HN stories from the +last 30 days. Exit code 0 even on partial source failures. diff --git a/website/sidebars.ts b/website/sidebars.ts index 327b2296149..4afcf57ea30 100644 --- a/website/sidebars.ts +++ b/website/sidebars.ts @@ -554,6 +554,7 @@ const sidebars: SidebarsConfig = { 'user-guide/skills/optional/research/research-drug-discovery', 'user-guide/skills/optional/research/research-duckduckgo-search', 'user-guide/skills/optional/research/research-gitnexus-explorer', + 'user-guide/skills/optional/research/research-last30days', 'user-guide/skills/optional/research/research-osint-investigation', 'user-guide/skills/optional/research/research-parallel-cli', 'user-guide/skills/optional/research/research-qmd',