Imported from anayy09/claude-research-skills (evidence-synthesis/SKILL.md). Install upstream with npx skills add anayy09/claude-research-skills --skill evidence-synthesis. Copyright stays with the author (MIT).

Evidence Synthesis

A review is a research design, not a reading exercise. It has a protocol, a sampling frame (the search), an eligibility rule, a measurement instrument (the appraisal tool), an analysis, and a reporting standard. Treating it as anything less produces a document that reads like a review and cannot be reproduced.

This skill is written to be run inside one conversation with executable checks at the points where reviews usually break. It does not delegate to subagents. Citation verification is delegated to the collection's one checker, check_citations.py in investigating-sources, through the scripts/verify_citations.py shim here; the release zip ships a copy of the checker so the skill installs on its own.

This skill is for the formal review: a protocol, a search that is reported, a screening log, an appraisal instrument, and a reporting guideline. A related-work section, a literature sweep, a cited brief or report, a reference audit, and a reference diet are investigating-sources. When a related-work section grows into a review with a screening log, the hand-over is at the protocol: write it here, and keep the source log shared.

Four rules with mechanisms behind them

1. A citation you cannot confirm does not go in the document. Not flagged as uncertain, not softened with a hedge. Removed. Distinct failures need distinct checks: fabrication (the work does not exist, or the DOI points somewhere else), retraction (the work exists but has been withdrawn), and supersession (the work exists as a preprint that has since been published after review). A DOI that resolves proves only that something is there. Run scripts/verify_citations.py on every reference list before it goes anywhere.

2. The numbers in the flow diagram must reconcile. Not approximately. Records that vanish between boxes are the single most common arithmetic error in published reviews, and it is the first thing a methods reviewer recomputes. Log decisions as you screen with scripts/screening_log.py, and derive the diagram from the log rather than counting by hand at the end.

3. Cite the peer-reviewed version of record. A preprint is something to find during searching and, when a published version exists, not something to cite. The numbers move during review, so citing the preprint reports figures the authors have since revised, and a preprint and its published version are one study that must be counted once in the flow diagram. Find the published version before extraction with scripts/verify_citations.py --upgrade-preprints. Preprints are eligible only when the protocol says so in advance, and then they are labelled as preprints, versioned, and handled in a sensitivity analysis.

The rule cannot be implemented by publisher name. SSRN sits under Elsevier's DOI prefix and TechRxiv under IEEE's, so "prefer IEEE and Elsevier" applied to publishers would admit working papers while excluding peer-reviewed society journals. Peer-review status is read from the record type, never from the imprint. See references/peer-reviewed-sources.md.

4. Disclose AI use where it made or informed a judgment. Screening, extraction, appraisal, and interpretive summarizing all qualify. This is now an explicit expectation: Cochrane, the Campbell Collaboration, JBI and the Collaboration for Environmental Evidence issued a joint position statement in 2025 supporting the RAISE recommendations (Thomas et al., 2025), which hold the synthesist responsible for the output, require human oversight, and require transparent reporting of any AI use that makes or suggests judgments. See references/ai-use-reporting.md and templates/ai-disclosure.md.

Step 1: choose the review type before anything else

The type determines the protocol, the reporting guideline, the appraisal tool, and how long this will take. Getting it wrong is expensive to correct later.

Full decision guidance, effort estimates, and what each type may and may not claim are in references/review-types.md.

Step 2: protocol first

Write the protocol before screening, and register it. PROSPERO for health reviews, OSF or the Open Science Framework registries otherwise. A protocol written after screening is a description, not a preregistration, and the difference is visible in the timestamps.

Use templates/protocol.md. The elements that matter most and are most often vague: the eligibility criteria stated so a second person would apply them identically, the primary outcome fixed in advance, and the planned synthesis method chosen before the data are seen.

Step 3: build and report the search

python scripts/search_builder.py --example > search.yaml   # edit the blocks
python scripts/search_builder.py --spec search.yaml --prisma-s
python scripts/search_builder.py --spec search.yaml --peer-reviewed-only

One concept-block specification renders to PubMed, Scopus, Web of Science, Cochrane CENTRAL, Ovid Embase, and Europe PMC, to the publisher platforms that hold the primary literature (IEEE Xplore, SpringerLink and the Nature portfolio, Elsevier ScienceDirect, the ACM Digital Library, Wiley Online Library), and to a Crossref REST query for scripted supplementary searching, plus a PRISMA-S reporting record. Hand-translating a strategy per database is where strategies silently diverge and become unreproducible.

Run at least two federated indexes plus the publisher platforms that own the literature for the question. For engineering and computer science that means IEEE Xplore and the ACM Digital Library as primary sources, not supplements, because the conference proceedings there are the peer-reviewed venue of record.

--peer-reviewed-only applies each platform's publication-type restriction (NOT preprint[pt] in PubMed, DOCTYPE in Scopus, DT= in Web of Science, NOT SRC:PPR in Europe PMC, filter=type:journal-article in Crossref) and, for the platforms where the restriction is a UI facet rather than query syntax, says which facet to use rather than emitting a filter the platform cannot parse.

The script also warns about the mechanical failures that return a plausible but wrong result set: SpringerLink has no truncation operator, ScienceDirect caps Boolean connectors per field and will silently reject a systematic strategy, and IEEE Xplore command search truncates very long OR-chains.

Two checks worth two minutes each: have a librarian or second reviewer look at the strategy (PRESS), and confirm the search retrieves the key papers you already know about. A search that misses a known paper is broken.

references/search-strategy.md covers block construction, controlled vocabulary versus free text, sensitivity against precision, grey literature, citation chasing, and the justification a language restriction requires. references/peer-reviewed-sources.md covers the publisher platforms, their syntax quirks, and Crossref member IDs for publisher-scoped queries.

Step 4: screen with a log

python scripts/screening_log.py init --title "..."
python scripts/screening_log.py identified --source PubMed --n 412
python scripts/screening_log.py dedup --removed 291
python scripts/screening_log.py screen --id S001 --decision exclude --reason "..."
python scripts/screening_log.py fulltext --id S002 --decision exclude --reason "..."
python scripts/screening_log.py flow            # reconciliation check
python scripts/screening_log.py exclusions      # PRISMA item 16b table

Dual independent screening is the standard; where a second human is unavailable, say so as a limitation rather than implying it happened. Every full-text exclusion needs a specific reason. "Did not meet inclusion criteria" is not a reason, and the script flags exclusions with none recorded.

Merge preprint-and-published pairs here, before extraction. They are one study, and automated deduplication misses them because the title, author list, and year can all differ between the two versions. Keep the published version as the record, and state in the PRISMA-S dedup item how the pairs were identified.

Step 5: appraise with the right instrument

Applying one generic checklist to every design is a methodological failure, not a simplification. Match the tool to the design:

references/appraisal-tools.md gives the domains, the common misapplications, and why PROBAST+AI (Moons et al., BMJ 2025;388:e082505) matters for any review that includes machine-learning prediction models: it separates model development from model evaluation and adds explicit fairness and real-world-performance considerations that PROBAST 2019 did not cover.

Step 6: synthesize, and do not pool by reflex

Meta-analysis is one option among several and is often the wrong one. Clinically or methodologically diverse studies pooled into a single number produce a precise answer to no question. When pooling is inappropriate, use a structured narrative synthesis and report it against SWiM rather than writing an unstructured discussion.

references/certainty-and-synthesis.md covers the decision to pool, fixed versus random effects, heterogeneity interpretation (including why I-squared is not a measure of how much heterogeneity there is), prediction intervals, small study effects, and the SWiM elements.

Step 7: rate certainty with GRADE, not with a pyramid

The evidence pyramid is a teaching aid for where to start looking. It is not a verdict. A randomized trial with serious risk of bias, imprecision, and indirectness supports a weaker conclusion than a well-conducted observational study with a large consistent effect. GRADE encodes this: start from the design, then rate down for risk of bias, inconsistency, indirectness, imprecision and publication bias, and rate up for large effect, dose-response, or when plausible confounding would reduce the observed effect.

Report certainty per outcome, not for the review as a whole.

Step 8: verify, then report

python scripts/verify_citations.py --refs references.md --mailto you@uni.edu
python scripts/verify_citations.py --refs references.md --mailto you@uni.edu \
    --upgrade-preprints --require-peer-reviewed
python scripts/verify_citations.py --self-test    # checks the verifier's logic

verify_citations.py parses the reference list (one per line, or a .bib) into the source-log schema and hands it to check_citations.py, which queries Crossref (with the Retraction Watch data it ingested, exposed through updated-by), falls back to DataCite because arXiv registers there (a Crossref-only check reports every arXiv citation as a fabrication), corroborates retractions against OpenAlex, classifies peer-review status from the record rather than the publisher's name, and looks up the published version of every preprint. A reference that could not be checked because the service was unreachable is reported as skipped, never as failed: a firewall is not evidence of fabrication. --write-log sources.json keeps the resulting source log so audit_report.py can cross-check the finished manuscript against it. The checker needs requests for live lookups.

Then report against the guideline for the review type, using templates/evidence-table.md for the study characteristics table and templates/ai-disclosure.md for the AI-use statement.

Anti-patterns

What this skill will not do

It will not invent a citation to fill a gap, produce a systematic review from a single conversation without the user doing the screening, claim dual independent screening that did not happen, assert an effect size it did not compute from reported data, or claim a source is peer reviewed on the strength of the publisher's name rather than the record. Where a step needs a human or a second reviewer, it says so and the limitation goes in the report.

Files

Key sources

Page et al. (2021), PRISMA 2020, BMJ 372:n71. Tricco et al. (2018), PRISMA-ScR, Ann Intern Med 169:467-473. Rethlefsen et al. (2021), PRISMA-S. Collins et al. (2024), TRIPOD+AI, BMJ 385:q902. Moons et al. (2025), PROBAST+AI, BMJ 388:e082505. Thomas et al. (2025), RAISE recommendations, and the joint Cochrane/Campbell/JBI/CEE position statement on AI use in evidence synthesis (2025). Verify current versions before citing: reporting guidelines are revised, and a partial update of PRISMA 2020 covering AI tool use was in development as of mid-2026.

Loading discipline

Load this skill once per session, before the step it governs, and do not invoke it again when it is already in context; a second load re-injects the same text and nothing else. When a repository carries docs/SKILL-ROUTING.md (project-ledger), it names the skill for each step and file; follow it, and record the skill in that step's progress entry. When a brief names several skills, each is loaded at the step it governs, not all at the start.