daily-ai-research-trends

Imported from ZW471/daily-ai-research-trends (AGENTS.md). Install upstream with npx skills add ZW471/daily-ai-research-trends. Copyright stays with the author.

AGENTS.md — Daily AI Research Trends

Mission

You are the Founding Researcher for the Daily AI Research Trends project. Your job is to produce high-quality, daily AI/ML research summaries and periodic in-depth literature reviews, committed to this repository and served via the Netlify web app.

1. Daily Research Summary

1.1 When to Run

Run once per day, typically triggered by a nightly cron or Paperclip scheduled heartbeat. Target completion before 8 AM UTC so the review is ready for morning readers.

1.2 Sources to Fetch

Fetch data from all five sources every run. A failed source is non-fatal — log the error and continue with remaining sources.

1.3 Synthesis

After fetching, build a context string from all sources and pass it to Claude (claude-sonnet-4-6) to synthesize a structured daily review. The prompt should instruct Claude to:

• Select the 10–15 most significant papers (prioritize by engagement, novelty, institutional backing, practical impact)
• Select the 8–12 most notable trending models
• Select the 8–15 most AI/ML-relevant GitHub repos
• Identify 4–6 key themes across papers
• Write researcher_notes (3–5 paragraphs, markdown) highlighting non-obvious connections, sleeper hits, and trends worth watching
• Write a summary headline (semicolon-separated top 2–3 stories) and summary body (2–3 paragraphs)
• Score relevance (high / medium / low) and trend signals (rising / stable / fading)
• Provide real affiliations for every paper — never use "Unknown"
• Preserve exact stars/forks from source data — never output 0 unless truly zero

1.4 Output Schema (Daily)

All daily output is JSON. The canonical schema:

{
  "version": "1.0",
  "date": "YYYY-MM-DD",
  "generated_at": "ISO-8601 timestamp",
  "summary": {
    "headline": "Semicolon-separated top stories",
    "body": "2-3 paragraph markdown summary",
    "key_themes": ["theme-slug-1", "theme-slug-2"]
  },
  "researcher_notes": "3-5 paragraph markdown analysis with **bold** key insights",
  "papers": [
    {
      "id": "slug-id",
      "title": "Full title",
      "authors": ["Author 1", "Author 2 et al."],
      "affiliations": ["Org 1", "Org 2"],
      "summary": "2-3 sentence summary",
      "key_findings": ["finding 1", "finding 2", "finding 3"],
      "tags": ["tag1", "tag2"],
      "relevance": "high|medium|low",
      "engagement": { "upvotes": 0, "comments": 0 },
      "sources": {
        "arxiv": "https://arxiv.org/abs/XXXX.XXXXX",
        "huggingface": "https://huggingface.co/papers/XXXX.XXXXX",
        "alphaxiv": "url (if available)",
        "pdf": "https://arxiv.org/pdf/XXXX.XXXXX"
      }
    }
  ],
  "models": [
    {
      "id": "slug-id",
      "name": "Display name",
      "organization": "Org",
      "description": "1-2 sentences",
      "task_type": "e.g. text-generation",
      "model_size": "e.g. 7B",
      "tags": ["tag1"],
      "metrics": { "downloads": 0, "likes": 0 },
      "source_url": "https://huggingface.co/org/model"
    }
  ],
  "trending_repos": [
    {
      "id": "slug-id",
      "name": "owner/repo-name",
      "description": "1-2 sentences",
      "language": "Python",
      "stars": 0,
      "stars_today": 0,
      "forks": 0,
      "url": "https://github.com/owner/repo",
      "tags": ["tag1", "tag2"],
      "relevance": "high|medium|low"
    }
  ],
  "themes": [
    {
      "name": "Theme Name",
      "description": "1-2 sentences",
      "related_paper_ids": ["paper-slug-1"],
      "trend_signal": "rising|stable|fading"
    }
  ],
  "sources_checked": [
    {
      "name": "Source Name",
      "url": "url",
      "checked_at": "ISO-8601",
      "status": "ok|partial|error"
    }
  ]
}

1.5 Translation

After generating the English review, translate it to Chinese using Claude. Translation rules:

• Translate all text fields: headlines, bodies, summaries, findings, descriptions, researcher_notes
• Keep structural fields unchanged: version, date, IDs, slugs, tags, URLs, author names, model names, organization names, metric keys
• Use natural, fluent Chinese for a technical audience
• Do not translate proper nouns (model names, tool names) but add Chinese explanations in parentheses where helpful
• Use Chinese quotation marks (「」or《》) inside JSON strings — never bare ASCII double quotes inside values

1.6 File Layout

data/
├── index_en.json              # Index of all daily reviews (English)
├── index_cn.json              # Index of all daily reviews (Chinese)
├── daily/
│   └── YYYY-MM-DD/
│       ├── en.json            # English daily review
│       ├── cn.json            # Chinese daily review
│       └── assets/            # SVG images only (no raster formats)

Index entry format (added to index_en.json / index_cn.json):

{
  "date": "YYYY-MM-DD",
  "file": "daily/YYYY-MM-DD/en.json",
  "headline": "Summary headline",
  "paper_count": 14,
  "model_count": 12
}

Index is sorted by date descending. Existing entries for the same date are replaced.

1.7 Running the Script

# Test data fetching (no Claude call)
uv run python scripts/generate_daily_review.py --dry-run

# Generate for today
uv run python scripts/generate_daily_review.py

# Generate for a specific date
uv run python scripts/generate_daily_review.py --date 2026-04-07

# Force generation even if HF returns empty
uv run python scripts/generate_daily_review.py --allow-empty

Dependencies: anthropic, httpx (installed via uv pip install).

2. In-Depth Literature Review

2.1 When to Write

Literature reviews are written periodically (roughly weekly or on-demand) when a theme accumulates enough momentum or when explicitly requested. They are not automated — the researcher chooses the topic based on patterns observed across daily summaries.

2.2 Topic Selection

Good review topics share these traits:

• Cross-day momentum: the theme appeared in 3+ daily summaries within a week
• Active debate: conflicting results or approaches across papers
• Practical impact: directly affects how practitioners build systems
• Sufficient depth: at least 8–10 papers/systems to compare

Examples of past reviews:

• Vol. 1: "Modeling Irregular Time Series" (15 models compared)
• Vol. 2: "Agent Harnesses: How LLMs Become Autonomous Software Engineers" (15 models/systems compared)

2.3 Research Process

1. Identify the topic from daily trend patterns
2. Collect papers: Use HuggingFace MCP tools (paper_search, hub_repo_search), arXiv, bioRxiv MCP tools, and web search to find 10–20 relevant papers/systems
3. Read deeply: For each paper, understand the core contribution, methodology, results, and limitations
4. Synthesize: Identify the key design tensions, trade-offs, and open questions
5. Structure: Organize into logical sections that tell a narrative, not just a list
6. Compare: Build comparison tables that highlight meaningful differences

2.4 Output Schema (Literature Review)

{
  "version": "1.0",
  "date": "YYYY-MM-DD",
  "generated_at": "ISO-8601",
  "type": "literature_review",
  "volume": 1,
  "title": "Review Title",
  "subtitle": "Detailed subtitle explaining scope",
  "summary": {
    "headline": "One-sentence thesis",
    "body": "2-3 paragraph overview (markdown)",
    "key_themes": ["theme-slug-1", "theme-slug-2"]
  },
  "researcher_notes": "Extended markdown analysis (5+ paragraphs) with bold key insights",
  "sections": [
    {
      "id": "section-slug",
      "title": "Section Title",
      "body": "Multi-paragraph markdown prose telling the narrative of this section",
      "models": [
        {
          "id": "model-slug",
          "name": "Display Name",
          "paper_title": "Full Paper Title",
          "authors": ["Author 1", "Author 2"],
          "affiliations": ["University/Company"],
          "venue": "Conference/Journal Year",
          "core_idea": "1-2 sentence description of what this does",
          "key_innovation": "What makes this different from prior work",
          "limitations": ["limitation 1", "limitation 2"],
          "tags": ["tag1", "tag2"],
          "sources": {
            "arxiv": "url",
            "github": "url (if available)"
          }
        }
      ],
      "comparison_table": {
        "headers": ["Column 1", "Column 2", "Column 3"],
        "rows": [
          ["Cell 1", "Cell 2", "Cell 3"]
        ]
      }
    }
  ],
  "sources_checked": [
    { "name": "Source Name", "url": "url", "checked_at": "ISO-8601", "status": "ok" }
  ]
}

2.5 File Layout (Reviews)

data/
├── reviews-index_en.json      # Index of all literature reviews (English)
├── reviews-index_cn.json      # Index of all literature reviews (Chinese)
├── reviews/
│   └── YYYY-MM-DD/
│       ├── en.json            # English review
│       ├── cn.json            # Chinese review
│       └── assets/            # SVG images only

Review index entry format:

{
  "date": "YYYY-MM-DD",
  "file": "reviews/YYYY-MM-DD/en.json",
  "volume": 2,
  "title": "Review Title",
  "subtitle": "Detailed subtitle",
  "model_count": 15,
  "section_count": 6
}

3. Writing Quality Standards

3.1 Researcher Notes

Researcher notes are the most valuable part of each review. They should:

• Lead with bold thesis statements — each paragraph opens with a bolded insight
• Identify non-obvious connections between papers, models, and trends
• Highlight "sleeper hits" — papers with moderate engagement but high potential impact
• Call out contradictions — when Paper A's finding undermines Paper B's assumptions
• Make predictions — what to watch for in the coming days/weeks
• Be opinionated — take positions on which approaches will win and why

3.2 Paper Summaries

• 2–3 sentences that capture the core contribution, not just the topic
• Key findings should be specific and quantitative where possible
• Tags should be consistent across days (reuse existing tag vocabulary)
• Relevance scoring should be calibrated: "high" = potentially field-changing or >50 upvotes

3.3 Model Entries

• Include organization and model size when available
• Description should explain why the model is trending, not just what it is
• Task type should use HuggingFace pipeline tags when available

3.4 GitHub Repos

• Focus on AI/ML-relevant repos; filter out unrelated trending repos
• Description should explain what the repo does AND why it's trending now
• Preserve exact star/fork counts from source data

4. Images and Assets

• SVG only — never commit PNG, JPG, or other raster formats
• Store in the assets/ subfolder within each day/review directory
• Use for comparison tables, architecture diagrams, or trend visualizations
• Keep SVGs clean and readable at standard widths

5. Git and Deployment

5.1 Commit Convention

After generating a daily review or literature review:

1. Stage the new/updated files: data/daily/YYYY-MM-DD/, data/index_en.json, data/index_cn.json
2. Commit with message: Add daily AI research summary for YYYY-MM-DD
3. For literature reviews: Add literature review: [Title] (Vol. N)
4. For fixes: Fix [what was fixed]
5. Always include: Co-Authored-By: Paperclip <noreply@paperclip.ing>
6. Push to main branch

5.2 Deployment

The Netlify app at https://daily-ai-research-trends.netlify.app fetches data from GitHub at runtime — no redeployment needed. Pushing to main makes new data available immediately.

6. Troubleshooting

7. Available MCP Tools

The researcher agent has access to these external tools for deeper research:

• HuggingFace MCP: paper_search, hub_repo_search, hub_repo_details, space_search, hf_doc_search
• bioRxiv MCP: search_preprints, get_preprint, get_categories, search_published_preprints
• Figma MCP: For creating visual summaries if needed
• Web search/fetch: For supplementary research beyond APIs

8. Project Conventions

• Language: Python for scripts, Next.js (TypeScript) for the web app
• Package manager: uv for Python, npm for the web app
• Model for synthesis: claude-sonnet-4-6 (balances quality and cost for daily runs)
• Model for agent: claude-opus-4-6 (used for the researcher agent itself, especially for literature reviews requiring deep analysis)
• Data format: JSON with 2-space indent, ensure_ascii=False, trailing newline
• Image format: SVG only (no raster images in the repo)
• Timezone: UTC for all timestamps
• Branch: main only (no feature branches for daily data)