pyxtrackers

Imported from chanwutk/pyxtrackers (AGENTS.md). Install upstream with npx skills add chanwutk/pyxtrackers. Copyright stays with the author.

Review this plan thoroughly before making any code changes. For every issue or recommendation, explain the concrete tradeoffs, give me an opinionated recommendation, and ask for my input before assuming a direction. My engineering preferences (use these to guide your recommendations):

• DRY is important-flag repetition aggressively.
• Well-tested code is non-negotiable, I'd rather have too many tests than too few.
• I want code that's "engineered enough"--not under-engineered (fragile, hacky) and not over-engineered (premature abstraction, unnecessary complexity).
• l err on the side of handling more edge cases, not fewer; thoughtfulness > speed.
• Bias toward explicit over cleverness.

1. Architecture review Evaluate:

• Overall system design and component boundaries.
• Dependency graph and coupling concerns.
• Data flow patterns and potential bottlenecks.
• Scaling characteristics and single points of failure.
• Security architecture (auth, data access, API boundaries)

2. Code quality review Evaluate:

• Code organization and module structure.
• DRY violations–be aggressive here.
• Error handling patterns and missing edge cases (call these out explicitly).
• Technical debt hotspots.
• Areas that are over-engineered or under-engineered relative to my preferences.

3. Test review Evaluate:

• Test coverage gaps (unit, integration, e2e). test quay and assertion strength
• Missing edge case coverage–be thorough.
• Untested failure modes and error paths.

4. Performance review Evaluate:

• N+1 queries and database access patterns.
• Memory-usage concerns.
• Caching opportunities.
• Slow or high-complexity code paths.

For each issue you find For every specific issue (bug, smell, design concern, or risk):

• Describe the problem concretely, with file and line references.
• Presence-s options, including do nothing where that's reasonable.
• For each option, specify: implementation effort, risk, impact on other code, and maintenance burden.
• Give me your recommended option and why, mapped to my preferences above.
• Then explicitly ask whether i agree or want to choose a different direction before proceeding Workflow and interaction
• Do not assume my priorities on timeline or scale.
• After each section, pause and ask for my feedback before moving on.

BEFORE YOU START: Ask it I want one of two options: 1/ BIG CHANGE: Work through this interactively, one section at a time (Architecture → Code Quality → Tests → Performance) with at most 4 top issues in each section. SMALL CHANGE: work through interactively oNe question per review section

FOR EACH STAGE OF REVIEW: output the explanation and pros and cons of each stage's questions AND your opinionated recommendation and why, and then use AskUserQuestion. Also NUMBER issues and then give LETTERS for options and when using AskUserQuestion make sure each option clearly labels the issue NUMBER and option LETTER so the user doesn't get confused. Make the recommended option always the 1st option.

Project Overview

PyxTrackers is a high-performance Cython reimplementation of three multi-object tracking algorithms: SORT, ByteTrack, and OC-SORT. The goal is numerical equivalence with the Python originals while achieving significant speedups. It is a standalone pip-installable package under the pyxtrackers namespace.

Package Name & Namespace

from pyxtrackers import Sort, BYTETracker, OCSort

# Or directly
from pyxtrackers.sort import Sort
from pyxtrackers.bytetrack import BYTETracker
from pyxtrackers.ocsort import OCSort

Development Workflow

The build backend is setuptools. Use pip for installation and dependency management. No third-party project managers (uv, poetry, etc.) — standard pip + venv is sufficient for this project's scope.

First-Time Setup

python -m venv .venv                     # Create virtual environment
source .venv/bin/activate                # Activate (Linux/macOS)
# .venv\Scripts\activate                 # Activate (Windows)
pip install -e ".[dev]"                  # Install package + dev dependencies + Cython

Common Commands

pytest tests/ -v                         # Run tests
python setup.py build_ext --inplace      # Rebuild after changing .pyx files

Why editable install (-e)

We use pip install -e . (editable) for development instead of pip install . (non-editable):

• Editable (-e): Python imports resolve directly from the source tree. Editing a .py file takes effect immediately without reinstalling. With setuptools>=68 (required in pyproject.toml), editable installs correctly place .so files in the source tree next to the .pyx files.
• Non-editable: Copies everything to site-packages/. Every change (even .py) requires a full pip install ., which recompiles all Cython extensions from scratch. Too slow for iterative development.

The dev workflow is:

• Edited .py → do nothing, changes are picked up immediately
• Edited .pyx → run python setup.py build_ext --inplace (incremental, only recompiles changed files)

Note: build_ext --inplace requires Cython, which is why Cython is included in the dev extras.

Dependency management

Build dependencies (Cython, numpy, setuptools-scm) are declared in [build-system].requires in pyproject.toml. pip installs them automatically in an isolated build environment during pip install -e . or pip install ., then discards that environment. Cython is additionally listed in [project.optional-dependencies].dev so it remains available in the development environment for running python setup.py build_ext --inplace after editing .pyx files.

• Runtime dependencies: edit [project].dependencies in pyproject.toml
• Build dependencies: edit [build-system].requires in pyproject.toml
• Dev-only dependencies: edit [project.optional-dependencies].dev in pyproject.toml
• Then run pip install -e ".[dev]" to apply changes

Versioning

Version is managed by setuptools-scm with CalVer (YYYY.M.D) derived from git tags automatically.

git tag v2026.2.23                       # Tag a release (first of the day)
git tag v2026.2.23.1                     # Second release same day
git push origin v2026.2.23              # Push tag → triggers CI release pipeline

Untagged commits get dev versions like 2026.2.24.0.dev3. The version is available at runtime via pyxtrackers.__version__.

Note: PEP 440 normalizes leading zeros, so the version is 2026.2.23 not 2026.02.23.

Releasing

1. Tag the commit: git tag v2026.2.23 && git push origin v2026.2.23
2. GitHub Actions builds sdist + wheels for all platforms and publishes to PyPI
3. GitHub Release is created with wheels + standalone CLI binaries attached
4. For conda-forge: submit conda-recipe/meta.yaml to conda-forge/staged-recipes (first release only; subsequent releases are auto-detected)

Build Flags

setup.py auto-detects the platform and build context via _get_arch_args():

• macOS universal2 Python (CFLAGS contains both -arch arm64 and -arch x86_64): No arch-specific flags — the compiler runs for both architectures in one pass, so no single -march/-mcpu is valid
• macOS ARM (non-universal): Uses -mcpu=apple-m1 (Apple Clang doesn't support -march=native on ARM)
• Source install on single-arch (non-portable, non-universal): Uses -march=native -mtune=native for max performance on the user's CPU
• Binary wheels (cibuildwheel/conda-build, detected via CIBUILDWHEEL or CONDA_BUILD env vars): Portable flags only (no arch-specific flags, except -mcpu=apple-m1 on ARM)
• Windows (MSVC): Uses /O2 /fp:fast instead of GCC/Clang flags

Running Tests

Tests compare Python reference implementations against Cython implementations on shared detection data.

pytest                                   # All tests
pytest tests/ -v                         # Verbose
pytest tests/test_sort_comparison.py -v  # Single file
pytest tests/ -v --cli-binary dist/pyxtrackers   # Include binary-CLI tests

--cli-binary takes the path to a PyInstaller-built binary (dist/pyxtrackers or dist/pyxtrackers.exe on Windows). Without it, binary-CLI tests are skipped.

Numerical comparison tolerance: 1e-6 pixels.

Design Decisions

Key decisions made during development. Understand these before suggesting changes.

Cython Reference Annotation Convention

All Cython implementation files (.pyx) must include navigation references back to the original Python implementation under references/.

Required format:

• A reference line must be exactly Ref: <url> on its own line.
• Explanatory text may be added after the reference. For example, Ref: <url> (Explanatory text). However, the explanatory text should be used rarely, unless it is crucial for understanding the reference.
• When possible, include line anchors (for example #L120-L145) to point to the exact source span in references/.
• If the reference line refers to the whole function, put the reference line inside the docstring, instead of a separate in-line comment.
• The <url> is in the format https://github.com/chanwutk/pyxtrackers/blob/main/<PATH_TO_FILE>#L<LINE_FROM>-L<LINE_TO>. When reading the reference file, do not fetch the file from github. Instead, look at <PATH_TO_FILE> from line <LINE_FROM> to line <LINE_TO> locally.

CLI I/O format

CLI output is x1,y1,x2,y2,id (track ID last), matching the library's [x1, y1, x2, y2, track_id] numpy column order.

CLI implementation strategy

The CLI is implemented in pyxtrackers/cli.pyx (Cython), not Python. The hot path uses native C stdio (fgets/fwrite) and C numeric parsing (strtod) to reduce per-frame overhead versus Python readline()/split()/float() loops.

Behavioral requirements:

• Keep strict fail-fast parsing semantics: malformed input raises ValueError and exits non-zero.
• Preserve 1:1 input/output line correspondence (including empty input lines).
• Preserve exact CLI schema (x1,y1,x2,y2,score in, x1,y1,x2,y2,id out).

PyInstaller entrypoint shim

PyInstaller still uses a tiny Python launcher (pyxtrackers/cli_launcher.py) as the script entrypoint. The launcher imports and calls pyxtrackers.cli:main from the compiled Cython module.

Rationale: keep standalone binary builds stable across platforms while still running the compiled CLI at runtime.

PyPI publish downloads only wheels and sdist

The publish-pypi job uses two separate actions/download-artifact steps (one for sdist, one with pattern: wheels-*) to exclude PyInstaller binary artifacts (binary-*). Downloading all artifacts causes twine to choke on the binary files. The publish-github job correctly downloads everything for the GitHub Release.

numpy version markers

numpy>=2.0 was lowered because the codebase uses only stable C API patterns available since numpy 1.19. The actual floor is set by Python version because numpy 1.x does not support Python 3.13:

• Python 3.10–3.11: numpy>=1.22
• Python 3.12: numpy>=1.26
• Python 3.13+: numpy>=2.1

NPY_NO_DEPRECATED_API is set to NPY_1_7_API_VERSION (not NPY_2_3_API_VERSION) so the code builds with any numpy >= 1.22. This macro hides deprecated C API symbols for forward-compatibility; it is a build-time hygiene measure, not a runtime version check.

Note: If you change numpy versions in an existing venv, reinstall scipy via pip install --force-reinstall scipy — its compiled extensions must match the numpy ABI.

Architecture

Three-Layer Design

1. pyxtrackers/ — Cython reimplementations (the installable package)
2. references/ — Pure Python reference implementations (for testing, not installed)
3. tests/ — Comparison tests that run both implementations on identical input and verify equivalence

Installable Package Structure

pyxtrackers/
├── __init__.py              # Re-exports Sort, BYTETracker, OCSort
├── cli.pyx                  # Cython stdin/stdout CLI
├── cli_launcher.py          # Python launcher for PyInstaller entrypoint
├── sort/
│   ├── __init__.py          # Re-exports Sort
│   ├── sort.pyx             # Main SORT tracker
│   ├── sort.pyi             # Type stubs
│   ├── kalman_filter.pyx    # 7D Kalman filter
│   └── kalman_filter.pxd    # C-level interface
├── bytetrack/
│   ├── __init__.py          # Re-exports BYTETracker
│   ├── bytetrack.pyx        # Main ByteTrack tracker
│   ├── kalman_filter.pyx    # 8D Kalman filter
│   ├── kalman_filter.pxd
│   ├── matching.pyx         # IOU + linear assignment
│   └── matching.pxd
└── ocsort/
    ├── __init__.py           # Re-exports OCSort (wrapper)
    ├── ocsort.pyx            # Main OC-SORT tracker
    ├── ocsort.pyi            # Type stubs
    ├── ocsort_wrapper.py     # Python wrapper for SORT-compatible interface
    ├── kalman_filter.pyx     # 7D Kalman filter with freeze/unfreeze
    ├── kalman_filter.pxd
    ├── association.pyx       # IOU + linear assignment
    └── association.pxd

.pxd files expose cdef functions/structs for cross-module cimport without Python overhead.

Build System

• pyproject.toml — Package metadata (PEP 621), build deps, pip config
• setup.py — Cython extension definitions (setuptools build backend)
• vendor/lapjv/ — Vendored LAPJV C++ linear assignment solver

Tracker Specifics

Key Cython Patterns

• C structs over Python classes for track state (e.g., STrack in ByteTrack) — no Python object overhead
• Flat arrays for matrices (e.g., double covariance[64] for 8x8) — cache-friendly layout
• nogil sections for GIL-free computation in hot paths
• malloc/free for manual memory management; libcpp.vector for dynamic collections
• cdef functions for C-only internal calls; def functions as Python-callable wrappers
• View classes (e.g., STrackView) provide non-owning Python access to C structs without copies

Bounding Box Conventions

Three representations used throughout:

• tlbr: [x1, y1, x2, y2] — for IOU calculation
• tlwh: [x, y, w, h] — for storage
• xyah: [cx, cy, aspect_ratio, h] — for Kalman filter observation

ByteTrack IOU uses PASCAL VOC formula (+1 to dimensions); SORT/OC-SORT use standard formula.

External Dependencies

• LAPJV (vendor/lapjv/lapjv.cpp): Vendored C++ linear assignment solver, linked at compile time
• NumPy: Runtime dependency; C headers used for compilation
• Cython: Build-time requirement
• setuptools: Build backend

Track Lifecycle (all trackers)

New → Tracked → Lost → Removed
       ↑          |
       └──────────┘  (re-activated on match)