Imported from weecology/MillionTrees (AGENTS.md). Install upstream with npx skills add weecology/MillionTrees. Copyright stays with the author.

Agent Instructions

To set up a development environment with all dependencies (including extras for development and testing),

use uv (https://github.com/astral-sh/uv):

uv sync --group dev
uv run python

Pull requests

CI runs pre-commit hooks (YAPF and docformatter on src/milliontrees/). Pull requests must pass pre-commit before merge; fix style locally instead of relying on CI to fail.

uv sync --group dev
uv run pre-commit run --all-files

Use uv run pre-commit install once per clone if you want the same checks to run automatically on git commit. Equivalent manual commands are documented in docs/contributing.md.

Agent gate before PR

• Before opening or updating a PR, always run uv run pre-commit run --all-files.
• If any hook fails (including docformatter), fix the files and rerun until all hooks pass.
• Do not hand off a PR with known pre-commit failures.

Coding Philosophy

Code Style Preferences

• Pythonic: Write idiomatic Python code that follows PEP 8 and Python best practices
• Concise: Favor brevity and clarity over verbosity
• Minimal: Use the simplest solution that works; avoid over-engineering
• Functional: Prefer functional programming patterns where appropriate

Error Handling Philosophy

• Fail Fast: Let code fail quickly and visibly rather than hiding errors
• Minimal Try/Except: Avoid excessive try/except blocks that mask underlying issues
• Explicit Failures: When errors occur, they should be obvious and informative
• No Silent Failures: Don't catch exceptions unless you can meaningfully handle them. In particular, never swallow I/O errors in dataset get_input() methods by returning blank/zero images — a missing or unreadable file is a real data problem that must surface as a hard error, not be silently skipped.

Testing Strategy

• Minimal Testing: Write only essential tests that catch critical functionality
• Quality over Quantity: Focus on meaningful tests rather than high coverage numbers
• Integration over Unit: Prefer integration tests that test real workflows
• Avoid Test Bloat: Don't write tests for trivial getters/setters or obvious functionality

Code Generation Guidelines

What TO Do

• Use list/dict comprehensions instead of explicit loops when clearer
• Leverage Python's built-in functions and standard library
• Use type hints for function signatures
• Write docstrings for public functions and classes
• Use pathlib instead of os.path for file operations
• Prefer f-strings over .format() or % formatting
• Use context managers (with statements) for resource management
• Handle edge cases with early returns rather than nested conditions

What NOT To Do

• Don't wrap every operation in try/except "just in case"
• Don't write defensive code for inputs that should be validated elsewhere
• Don't add unnecessary abstraction layers
• Don't write tests for trivial operations
• Don't catch broad exceptions (Exception, BaseException) unless absolutely necessary
• Don't suppress errors with pass statements in except blocks
• Don't add configuration options for things that don't need to be configurable
• Avoid over-use of argparse and CLI for simple scripts.

Specific Patterns

Preferred Error Handling

# Good: Let it fail fast
def process_file(filepath):
    with open(filepath) as f:  # FileNotFoundError will bubble up naturally
        return json.load(f)    # JSONDecodeError will bubble up naturally

# Avoid: Excessive defensive programming
def process_file(filepath):
    try:
        if not os.path.exists(filepath):
            # Handle missing file
        with open(filepath) as f:
            try:
                return json.load(f)
            except JSONDecodeError:
                # Handle invalid JSON
    except Exception as e:
        # Handle everything else

Preferred Function Design

# Good: Simple, direct, functional
def filter_valid_annotations(annotations: list[dict]) -> list[dict]:
    return [ann for ann in annotations if ann.get('x') and ann.get('y')]

# Avoid: Over-engineered with unnecessary error handling
def filter_valid_annotations(annotations: list[dict]) -> list[dict]:
    try:
        if not annotations:
            return []
        result = []
        for ann in annotations:
            try:
                if ann.get('x') is not None and ann.get('y') is not None:
                    result.append(ann)
            except (KeyError, AttributeError, TypeError):
                continue
        return result
    except Exception:
        return []

Domain-Specific Guidelines

For Data Processing Scripts

• Use pandas operations instead of manual loops
• Leverage numpy vectorization
• Use pathlib for file path operations
• Let pandas/numpy raise their own exceptions rather than catching them

For CLI Scripts

• Use argparse for command-line interfaces
• Let missing required arguments fail naturally
• Use sys.exit() for expected failure cases
• Don't catch keyboard interrupts unless necessary

For API Interactions

• Use requests library directly without excessive retry logic
• Let HTTP errors bubble up unless you can meaningfully handle them
• Use requests.Session for connection pooling when making multiple requests

Comments and Documentation

When to Document

• Complex algorithms or business logic
• Non-obvious performance considerations
• API contracts and expected input/output formats
• Workarounds for external library limitations

When NOT to Document

• Obvious operations (x = x + 1)
• Standard library usage
• Simple getter/setter methods
• Code that is self-explanatory

Where write-ups go: notes/, not docs/

docs/ is the readthedocs site. Every page in it must be listed in the toctree in docs/index.rst, and CI builds the docs with -W, so a markdown file dropped into docs/ without a toctree entry fails the build.

Analysis write-ups, diagnostic reports, experiment logs and generated table fragments go in notes/ instead — see notes/README.md. Scripts that emit a report should default their output path there. Figures stay in docs/public/ (notes reference them as ../docs/public/...) so published pages can use the same images. Only move a note into docs/ if a benchmark user should read it, and add it to the toctree in the same commit.

Summary

Write code that is clear, direct, and fails obviously when something goes wrong. Prefer simplicity over robustness, and trust that proper system design and monitoring will catch issues rather than trying to handle every possible error case in the code itself.

File I/O Patterns

Preferred File Reading

# Good: Let file operations fail fast and explicitly
def process_annotations(data_dir: str) -> pd.DataFrame:
    treetops_file = os.path.join(data_dir, 'treetops.gpkg')
    return gpd.read_file(treetops_file)  # FileNotFoundError will bubble up clearly

# Good: Use optional return types when files may legitimately not exist
def load_optional_config(config_path: str) -> Optional[dict]:
    try:
        with open(config_path) as f:
            return json.load(f)
    except FileNotFoundError:
        return None  # Explicit handling of expected case

# Avoid: Silent failures with existence checks
def process_annotations(data_dir: str) -> Optional[pd.DataFrame]:
    treetops_file = os.path.join(data_dir, 'treetops.gpkg')
    if os.path.exists(treetops_file):
        return gpd.read_file(treetops_file)
    return None  # Silent failure makes debugging harder

Adding Datasets to MillionTrees

Follow the docs/contributing.md for adding new datasets.

Update the leaderboard

For a reproducible leaderboard update, run the fine-tuned models in training/ and the pretrained baselines in existing_models/ (see docs/repository_structure.md). For SLURM systems, slurm/submit_all.sh launches both, then scripts/make_benchmark_table.py regenerates the tables. The logs are available in /logs and will report the performance of each model. Update the leaderboard with the top-level metrics.

SLURM Workflow

After submitting any sbatch job, always:

1. Note the job ID returned by sbatch.
2. Append an entry to the job ledger at /home/b.weinstein/logs/job_ledger.md (see below).
3. Wait 60 seconds (use ScheduleWakeup with delaySeconds=60).
4. Run squeue -j <JOBID> and check the .out/.err logs to confirm the job is running (ST=R) with no errors.

Job ledger

Every time Claude submits a SLURM job, append one entry to /home/b.weinstein/logs/job_ledger.md so that checklog can explain why each run exists and what comes next. Format:

## <JOBID> — <YYYY-MM-DD HH:MM> — <script name>
Why: <one line — the goal/hypothesis behind this run>
Next: <one line — what to do when it finishes (read which log, kick off which dependent job, update which table)>

• Keep it to those two lines (Why / Next); no more.
• For job arrays or chained submissions, write one entry per job ID, and use Next to record the dependency (e.g. "blocks JOBID 12345 polygon train").
• Never edit or delete prior entries; the ledger is an append-only history.