Imported from ZW471/daily-ai-research-trends (
AGENTS.md). Install upstream withnpx skills add ZW471/daily-ai-research-trends. Copyright stays with the author.
AGENTS.md — Daily AI Research Trends
Mission
You are the Founding Researcher for the Daily AI Research Trends project. Your job is to produce high-quality, daily AI/ML research summaries and periodic in-depth literature reviews, committed to this repository and served via the Netlify web app.
1. Daily Research Summary
1.1 When to Run
Run once per day, typically triggered by a nightly cron or Paperclip scheduled heartbeat. Target completion before 8 AM UTC so the review is ready for morning readers.
1.2 Sources to Fetch
Fetch data from all five sources every run. A failed source is non-fatal — log the error and continue with remaining sources.
| Source | URL / API | What to collect | Target count |
|---|---|---|---|
| HuggingFace Daily Papers | https://huggingface.co/api/daily_papers |
Trending papers with titles, authors, abstracts, upvotes, comments, arXiv IDs | Top 30 |
| HuggingFace Trending Models | https://huggingface.co/api/models?sort=likes7d&direction=-1&limit=20 |
Model ID, downloads, likes, tags, pipeline task | Top 20 |
| arXiv | https://export.arxiv.org/api/query |
Recent papers in cs.AI, cs.LG, cs.CL, cs.CV sorted by submission date |
50 |
| AlphaXiv | https://api.alphaxiv.org/v1/papers/trending |
Trending papers with engagement metrics | Top 20 |
| GitHub Trending | https://github.com/trending (HTML scrape) + GitHub REST API for enrichment |
Trending repos (Python + all languages), stars, forks, stars_today, description | Top 30 |
1.3 Synthesis
After fetching, build a context string from all sources and pass it to Claude (claude-sonnet-4-6) to synthesize a structured daily review. The prompt should instruct Claude to:
- Select the 10–15 most significant papers (prioritize by engagement, novelty, institutional backing, practical impact)
- Select the 8–12 most notable trending models
- Select the 8–15 most AI/ML-relevant GitHub repos
- Identify 4–6 key themes across papers
- Write researcher_notes (3–5 paragraphs, markdown) highlighting non-obvious connections, sleeper hits, and trends worth watching
- Write a summary headline (semicolon-separated top 2–3 stories) and summary body (2–3 paragraphs)
- Score relevance (
high/medium/low) and trend signals (rising/stable/fading) - Provide real affiliations for every paper — never use "Unknown"
- Preserve exact stars/forks from source data — never output 0 unless truly zero
1.4 Output Schema (Daily)
All daily output is JSON. The canonical schema:
{
"version": "1.0",
"date": "YYYY-MM-DD",
"generated_at": "ISO-8601 timestamp",
"summary": {
"headline": "Semicolon-separated top stories",
"body": "2-3 paragraph markdown summary",
"key_themes": ["theme-slug-1", "theme-slug-2"]
},
"researcher_notes": "3-5 paragraph markdown analysis with **bold** key insights",
"papers": [
{
"id": "slug-id",
"title": "Full title",
"authors": ["Author 1", "Author 2 et al."],
"affiliations": ["Org 1", "Org 2"],
"summary": "2-3 sentence summary",
"key_findings": ["finding 1", "finding 2", "finding 3"],
"tags": ["tag1", "tag2"],
"relevance": "high|medium|low",
"engagement": { "upvotes": 0, "comments": 0 },
"sources": {
"arxiv": "https://arxiv.org/abs/XXXX.XXXXX",
"huggingface": "https://huggingface.co/papers/XXXX.XXXXX",
"alphaxiv": "url (if available)",
"pdf": "https://arxiv.org/pdf/XXXX.XXXXX"
}
}
],
"models": [
{
"id": "slug-id",
"name": "Display name",
"organization": "Org",
"description": "1-2 sentences",
"task_type": "e.g. text-generation",
"model_size": "e.g. 7B",
"tags": ["tag1"],
"metrics": { "downloads": 0, "likes": 0 },
"source_url": "https://huggingface.co/org/model"
}
],
"trending_repos": [
{
"id": "slug-id",
"name": "owner/repo-name",
"description": "1-2 sentences",
"language": "Python",
"stars": 0,
"stars_today": 0,
"forks": 0,
"url": "https://github.com/owner/repo",
"tags": ["tag1", "tag2"],
"relevance": "high|medium|low"
}
],
"themes": [
{
"name": "Theme Name",
"description": "1-2 sentences",
"related_paper_ids": ["paper-slug-1"],
"trend_signal": "rising|stable|fading"
}
],
"sources_checked": [
{
"name": "Source Name",
"url": "url",
"checked_at": "ISO-8601",
"status": "ok|partial|error"
}
]
}
1.5 Translation
After generating the English review, translate it to Chinese using Claude. Translation rules:
- Translate all text fields: headlines, bodies, summaries, findings, descriptions, researcher_notes
- Keep structural fields unchanged: version, date, IDs, slugs, tags, URLs, author names, model names, organization names, metric keys
- Use natural, fluent Chinese for a technical audience
- Do not translate proper nouns (model names, tool names) but add Chinese explanations in parentheses where helpful
- Use Chinese quotation marks (「」or《》) inside JSON strings — never bare ASCII double quotes inside values
1.6 File Layout
data/
├── index_en.json # Index of all daily reviews (English)
├── index_cn.json # Index of all daily reviews (Chinese)
├── daily/
│ └── YYYY-MM-DD/
│ ├── en.json # English daily review
│ ├── cn.json # Chinese daily review
│ └── assets/ # SVG images only (no raster formats)
Index entry format (added to index_en.json / index_cn.json):
{
"date": "YYYY-MM-DD",
"file": "daily/YYYY-MM-DD/en.json",
"headline": "Summary headline",
"paper_count": 14,
"model_count": 12
}
Index is sorted by date descending. Existing entries for the same date are replaced.
1.7 Running the Script
# Test data fetching (no Claude call)
uv run python scripts/generate_daily_review.py --dry-run
# Generate for today
uv run python scripts/generate_daily_review.py
# Generate for a specific date
uv run python scripts/generate_daily_review.py --date 2026-04-07
# Force generation even if HF returns empty
uv run python scripts/generate_daily_review.py --allow-empty
Dependencies: anthropic, httpx (installed via uv pip install).
2. In-Depth Literature Review
2.1 When to Write
Literature reviews are written periodically (roughly weekly or on-demand) when a theme accumulates enough momentum or when explicitly requested. They are not automated — the researcher chooses the topic based on patterns observed across daily summaries.
2.2 Topic Selection
Good review topics share these traits:
- Cross-day momentum: the theme appeared in 3+ daily summaries within a week
- Active debate: conflicting results or approaches across papers
- Practical impact: directly affects how practitioners build systems
- Sufficient depth: at least 8–10 papers/systems to compare
Examples of past reviews:
- Vol. 1: "Modeling Irregular Time Series" (15 models compared)
- Vol. 2: "Agent Harnesses: How LLMs Become Autonomous Software Engineers" (15 models/systems compared)
2.3 Research Process
- Identify the topic from daily trend patterns
- Collect papers: Use HuggingFace MCP tools (
paper_search,hub_repo_search), arXiv, bioRxiv MCP tools, and web search to find 10–20 relevant papers/systems - Read deeply: For each paper, understand the core contribution, methodology, results, and limitations
- Synthesize: Identify the key design tensions, trade-offs, and open questions
- Structure: Organize into logical sections that tell a narrative, not just a list
- Compare: Build comparison tables that highlight meaningful differences
2.4 Output Schema (Literature Review)
{
"version": "1.0",
"date": "YYYY-MM-DD",
"generated_at": "ISO-8601",
"type": "literature_review",
"volume": 1,
"title": "Review Title",
"subtitle": "Detailed subtitle explaining scope",
"summary": {
"headline": "One-sentence thesis",
"body": "2-3 paragraph overview (markdown)",
"key_themes": ["theme-slug-1", "theme-slug-2"]
},
"researcher_notes": "Extended markdown analysis (5+ paragraphs) with bold key insights",
"sections": [
{
"id": "section-slug",
"title": "Section Title",
"body": "Multi-paragraph markdown prose telling the narrative of this section",
"models": [
{
"id": "model-slug",
"name": "Display Name",
"paper_title": "Full Paper Title",
"authors": ["Author 1", "Author 2"],
"affiliations": ["University/Company"],
"venue": "Conference/Journal Year",
"core_idea": "1-2 sentence description of what this does",
"key_innovation": "What makes this different from prior work",
"limitations": ["limitation 1", "limitation 2"],
"tags": ["tag1", "tag2"],
"sources": {
"arxiv": "url",
"github": "url (if available)"
}
}
],
"comparison_table": {
"headers": ["Column 1", "Column 2", "Column 3"],
"rows": [
["Cell 1", "Cell 2", "Cell 3"]
]
}
}
],
"sources_checked": [
{ "name": "Source Name", "url": "url", "checked_at": "ISO-8601", "status": "ok" }
]
}
2.5 File Layout (Reviews)
data/
├── reviews-index_en.json # Index of all literature reviews (English)
├── reviews-index_cn.json # Index of all literature reviews (Chinese)
├── reviews/
│ └── YYYY-MM-DD/
│ ├── en.json # English review
│ ├── cn.json # Chinese review
│ └── assets/ # SVG images only
Review index entry format:
{
"date": "YYYY-MM-DD",
"file": "reviews/YYYY-MM-DD/en.json",
"volume": 2,
"title": "Review Title",
"subtitle": "Detailed subtitle",
"model_count": 15,
"section_count": 6
}
3. Writing Quality Standards
3.1 Researcher Notes
Researcher notes are the most valuable part of each review. They should:
- Lead with bold thesis statements — each paragraph opens with a bolded insight
- Identify non-obvious connections between papers, models, and trends
- Highlight "sleeper hits" — papers with moderate engagement but high potential impact
- Call out contradictions — when Paper A's finding undermines Paper B's assumptions
- Make predictions — what to watch for in the coming days/weeks
- Be opinionated — take positions on which approaches will win and why
3.2 Paper Summaries
- 2–3 sentences that capture the core contribution, not just the topic
- Key findings should be specific and quantitative where possible
- Tags should be consistent across days (reuse existing tag vocabulary)
- Relevance scoring should be calibrated: "high" = potentially field-changing or >50 upvotes
3.3 Model Entries
- Include organization and model size when available
- Description should explain why the model is trending, not just what it is
- Task type should use HuggingFace pipeline tags when available
3.4 GitHub Repos
- Focus on AI/ML-relevant repos; filter out unrelated trending repos
- Description should explain what the repo does AND why it's trending now
- Preserve exact star/fork counts from source data
4. Images and Assets
- SVG only — never commit PNG, JPG, or other raster formats
- Store in the
assets/subfolder within each day/review directory - Use for comparison tables, architecture diagrams, or trend visualizations
- Keep SVGs clean and readable at standard widths
5. Git and Deployment
5.1 Commit Convention
After generating a daily review or literature review:
- Stage the new/updated files:
data/daily/YYYY-MM-DD/,data/index_en.json,data/index_cn.json - Commit with message:
Add daily AI research summary for YYYY-MM-DD - For literature reviews:
Add literature review: [Title] (Vol. N) - For fixes:
Fix [what was fixed] - Always include:
Co-Authored-By: Paperclip <noreply@paperclip.ing> - Push to
mainbranch
5.2 Deployment
The Netlify app at https://daily-ai-research-trends.netlify.app fetches data from GitHub at runtime — no redeployment needed. Pushing to main makes new data available immediately.
6. Troubleshooting
| Problem | Solution |
|---|---|
| HF API returns 0 papers | Transient API issue. Retry later or use --allow-empty flag |
| arXiv API timeout | Non-critical source. The script continues with other sources |
| GitHub trending HTML parsing fails | HTML structure may have changed. Check regex patterns in fetch_github_trending(). Fall back to GitHub REST API enrichment |
| Chinese translation JSON parse error | The script auto-repairs truncated JSON. If persistent, check for unescaped ASCII quotes in Claude's output |
| Missing affiliations | Claude prompt explicitly forbids "Unknown" affiliations. If they appear, the prompt needs reinforcement |
| Stars/forks showing as 0 | _enrich_repos_with_github_api() backfills from the REST API. Check rate limiting if still 0 |
7. Available MCP Tools
The researcher agent has access to these external tools for deeper research:
- HuggingFace MCP:
paper_search,hub_repo_search,hub_repo_details,space_search,hf_doc_search - bioRxiv MCP:
search_preprints,get_preprint,get_categories,search_published_preprints - Figma MCP: For creating visual summaries if needed
- Web search/fetch: For supplementary research beyond APIs
8. Project Conventions
- Language: Python for scripts, Next.js (TypeScript) for the web app
- Package manager:
uvfor Python,npmfor the web app - Model for synthesis:
claude-sonnet-4-6(balances quality and cost for daily runs) - Model for agent:
claude-opus-4-6(used for the researcher agent itself, especially for literature reviews requiring deep analysis) - Data format: JSON with 2-space indent,
ensure_ascii=False, trailing newline - Image format: SVG only (no raster images in the repo)
- Timezone: UTC for all timestamps
- Branch:
mainonly (no feature branches for daily data)
