Imported from JelleZijlstra/pycroscope (
AGENTS.md). Install upstream withnpx skills add JelleZijlstra/pycroscope. Copyright stays with the author.
Agent Instructions
- If behavior, CLI output/options, or configuration changes, update the relevant docs in
docs/in the same change. Do not updatedocs/for minor changes. - When asked to make a PR, first rebase your changes on latest
main, then open a PR. Verify that the PR body is formatted correctly, then monitor CI and push fixes for any CI failures or merge conflicts.
Changelog
- Any change with a user-visible effect must include a Scriv fragment in
changelog.d/. - Create fragments with
uv run --locked --group release scriv create. Do not editdocs/changelog.mddirectly except while preparing a release. - Fragments should contain a single bullet in plain language that explains the user-visible effect (not internal refactors).
- Bugfixes for bugs that were not in any release do not need changelog entries.
uv lockfile
- Use
uv run --lockedanduv sync --lockedfor ordinary development and test commands so they fail instead of modifyinguv.lock. - Modify
uv.lockonly whenpyproject.tomldependency declarations or the project version change. Runuv lockexplicitly and include the resulting lockfile change in the same commit.
Architecture
- Major files include:
name_check_visitor.pycontains the core visitor that walks over code being type checked. It should visit every AST node exactly twice (in the collect and check phases). This file should contain the core checker logic, but delegate to other files (e.g.,attributes.py,type_object.py,stacked_scopes.py) for narrower logic specific to those files.value.pycontains the core data structures representing types.relations.pyperforms basic operations on types, such as subtyping, assignability, and intersections.type_object.pycontains rich data about classes.annotations.pycontains code for understanding annotations and typing-related objects and parsing them into pycroscope's internal data structures.arg_spec.pyparses callable objects and extracts their callable signatures.signature.pycontains representations of callable signatures and operations on them, such as type checking calls.checker.pyorchestrates type checking across multiple files and maintains caches that span across files.implementation.pycontains callbacks ("impls") that perform more precise checks or type inference on specific callables.stacked_scopes.pycontains logic for tracking which names are in scope and for local narrowing.attributes.pyretrieves attributes on objects.
- Techniques for keeping
name_check_visitor.pyclean include:- Collect short-lived data in attributes on the
NameCheckVisitor, then aggregate it later. For example, to determine whether a function is a generator, set some marker when visitingyieldexpressions, and read the result at the end of visiting theFunctionDef. To handle nesting, stash any existing state when entering a newFunctionDef. - Make canonical internal representations (such as
ValueandTypeObjectobjects) rich enough to store the state you need; avoid creating additional complex objects internal toname_check_visitor.py.
- Collect short-lived data in attributes on the
Testing
- Before finishing, run the linting/tests relevant to the files you changed.
test_self.pyis often useful for finding regressions. Run it with thefullextra enabled or it will be skipped.- Run conformance CI with the same interpreter you want pycroscope itself to use, because
tools/conformance_ci.pyinvokessys.executable -m pycroscopeunder the hood. - Use this form from the repo root:
uv run --locked --python 3.12 python tools/conformance_ci.py --typing-repo ~/py/typing(optionally prefixUV_CACHE_DIR=/tmp/uv-cache). - When fixing regressions found by
test_self.py, add separate test cases instead of just relying ontest_self.py. - When writing test cases, always use code samples (
@assert_passes()) instead of tests that directly invoke pycroscope functions. Code samples should represent user-written code that triggers the pycroscope feature under test and should not import internal pycroscope functions (except where needed for e.g.assert_is_value). Use direct unit tests (calling internal functions) only if testing a widely used API with a clear contract, such asTypeObject.get_attributeorhas_relation. - Prefer
assert_type()overassert_is_value()for type assertions against types where possible; it's OK to keepassert_is_valuefor more complicated types that cannot be directly represented in user code.
Coverage
- To reproduce the CI coverage run locally, use Python 3.14 and
pytest-cov, for example:UV_CACHE_DIR=/tmp/uv-cache uv run --locked --python 3.14 --extra tests --extra asynq --extra codemod --with pytest-cov pytest --cov=pycroscope --cov-report=term-missing --cov-report=json:coverage.json pycroscope - To generate an HTML line-by-line coverage report locally, add
--cov-report=htmlto the coverage command above. The report is written tohtmlcov/index.html. - The pinned-full-coverage check is checked in as
tools/check_full_coverage.py; run it withpython tools/check_full_coverage.py coverage.json tools/full_coverage_files.txt.
Code Conventions
- Types and type signatures should have a single canonical internal representation,
which is normalized when we parse annotations (e.g. in
annotations.py). We should use the same representation regardless of whether an object was created in importable or unimportable mode. - All type inference should go through the main type inference visitor in
name_check_visitor.py. Other code should generally not perform AST walks. (Exceptions include the pattern matching visitor inpatma.py, stub visitor intypeshed.py, and annotation visitor inannotations.py.) Always prefer using Values derived from type inference over doing direct checks on the AST. - The code should avoid special-casing specific functions or standard library classes outside of impl functions in
implementation.py. Instead of special-casing individual symbols, find more general solutions. implementation.pyshould contain only impl functions used by extended argspecs. Shared non-impl helpers should live in the owning module even when they support special-casing logic.- Code that branches on different subclasses of
Valueshould take care to cover all cases. Where possible, replace code that dispatches on different kinds of values with a call to a general helper function that already knows how to deal with all Values, such ashas_relation,is_assignable, oris_subtype. If dispatching onValueis necessary, use the following procedure:- Call
pycroscope.value.gradualizeorpycroscope.value.replace_fallbackto narrow the Value to a fixed set of classes. This will raiseNotAGradualTypewhen it encounters unexpected Value subclasses. Do not catch this exception; instead add specific handling for those classes. Non-GradualType Value subclasses should be used only in narrow places.replace_fallbacknarrows to a smaller set of Value subclasses (BasicTypeinstead ofGradualType). If you need special handling for some types that are GradualType but not BasicType, do that here. - Perform special handing for
MultiValuedValue(unions) andIntersectionValue(intersections), usually calling the same logic for each member value and aggregating the results. - Now dispatch over all remaining types, which are described by the
SimpleTypeunion. Useassert_never()to ensure all types are covered.
- Call
- Avoid using
@staticmethodfor local helper functions. Use private module-level functions instead. - Avoid using function-local imports, except where necessary to avoid an import cycle.
- Never catch
NotAGradualType. Instead refactor the code so that non-gradual types do not escape narrow parts of the checker, or add specific handling for individual types. - Do not use the names of symbols or parameters (such as
self) for logic. Instead use type inference or figure out whether a parameter (for example) logically representsselfwithout relying on the name. Exception: opinionated lint-style checks may still enforce spelling conventions for parameter names, such asmethod_first_argrequiringself/cls. - The behavior in import failure mode (where we cannot load the runtime module) should match that in normal mode
as much as possible. As a corollary, whenever testing import failure-related issues,
write tests with
run_in_both_module_modes=Trueif possible; avoidallow_import_failures=True. - Types and other internal objects should only be represented in one canonical way in pycroscope's internal logic. There should not be two equivalent ways to represent the same concept.