bq-inspector

Imported from yu-iskw/bq-inspector (AGENTS.md). Install upstream with npx skills add yu-iskw/bq-inspector. Copyright stays with the author.

bq-inspector — project instructions

Authoritative shared instructions for humans and coding agents. How each product loads this repo: Coding agents & instruction files.

Project overview

Read-only BigQuery job and metadata inspection CLI (src/bq_inspector/). Tooling:

• Package manager: uv (via requirements.setup.txt, not mise)
• CLI toolchain: mise — Trunk, Trivy, OSV-Scanner, Grype, CodeQL (mise.toml [tasks], mise.lock, minimum_release_age = "7d")
• Build system: Hatchling
• Linting/formatting: Trunk (Ruff, Pyright, Pylint, Bandit, Semgrep; Ruff is the formatter; Black is not used)
• Testing: pytest
• Python: 3.10+ (see .python-version for the pinned version)

Quick commands

make setup-tools  # mise: trunk, trivy, osv-scanner, grype, codeql (+ trunk install)
make setup        # setup-tools + Python venv (uv sync)
make setup-python # Python venv only (skip CLI toolchain)
make lint         # mise run lint (Trunk check)
make lint-python  # Same as `make lint`
make format       # mise run format-trunk + uv ssort
make dead-code    # Find unused code with Vulture (see pyproject [tool.vulture])
make vulture      # Same as make dead-code
make test         # Run pytest with coverage (pytest-cov); alias: `make coverage`
make codeql       # Run local CodeQL analysis
make build        # Build the package
make clean        # Clean build artifacts

Mise toolchain (local vs CI)

• Local / agents: Install mise on PATH, then make setup-tools or make setup. Commands use mise run <task> from mise.toml—there is no dev/mise-exec.sh or other shell wrapper to invoke mise.
• CI: .github/workflows/mise_toolchain.yml runs jdx/mise-action and dev/test-mise-toolchain.sh. Lint in PRs still uses Trunk Action; Python tests use uv.
• make setup-python works without mise (Python/uv only). make setup requires mise because it runs setup-tools first.

Code style

• Follow the Google Python Style Guide (see .pylintrc)
• Use type hints for all public functions
• Imports sorted by Ruff (rule I)
• Max line length: 100 characters (Ruff)
• snake_case for functions and variables, PascalCase for classes

Testing

• Tests live under src/bq_inspector/tests/ (colocated with the package)
• Test files must match test_*.py
• Run make test before commits
• Aim for meaningful coverage on critical paths

Security

• Static analysis: Trunk runs Ruff, Pyright (types), Pylint, Bandit, Semgrep, and Trivy for quick feedback
• Deep analysis: GitHub CodeQL path analysis (see .github/workflows/codeql.yml)
• Dependencies: OSV-Scanner, Trivy, and Grype (make scan-vulnerabilities; runs serially via mise; versions from mise)
• Local CodeQL: make codeql (CodeQL CLI via mise); on Linux or macOS ARM64, make setup-tools skips the CodeQL version check (x64 bundle in mise.lock)
• make scan-vulnerabilities: OSV-Scanner exits 1 when it reports vulnerabilities (expected); fix deps or document accepted risk.
• Use trunk check before pushing

AI guardrails & code quality

• Cyclomatic complexity: max 10 per function (Ruff C901)
• Maintainability: CodeQL security-and-quality tracks long-term health
• If an edit pushes complexity over 10, refactor into smaller functions before finishing

Session postmortem (coding agents)

• Purpose: After a substantive session, run a retrospective so failures and inefficiencies surface as ranked Must / Should / Consider improvements. Template: .agents/skills/postmortem/references/postmortem-report-template.md.
• Invocation: Invoke the postmortem skill (e.g. /postmortem in Claude Code). Load from .claude/skills/postmortem/ or .agents/skills/postmortem/ depending on your tool. Keep output in chat unless the user asks to persist; the skill does not authorize editing AGENTS.md, CLAUDE.md, or skills without a separate request.
• Skip: Do not run after purely mechanical work with no learning signal (e.g. obvious typo, format-only pass, trivial dependency bump with no retries). If the session included debugging, ambiguity, or retries, run a postmortem anyway.

Git workflow

• Branch from main
• Run make lint && make test before commits
• Conventional commits: type(scope): description (e.g. feat(api): add user endpoint)
• Types: feat, fix, docs, style, refactor, test, chore
• For releases, record changes with the manage-changelog skill when Changie is available (fragments, batch, merge into CHANGELOG.md)

Architecture

• Package source: src/bq_inspector/
• Dev scripts: dev/
• CI/CD: .github/workflows/
• Claude Code automation: .claude/ — see CLAUDE.md for how Claude loads this repo and the directory layout
• Architecture decision records (ADRs): docs/adr/. Use the manage-adr skill when the adr CLI is installed

Common gotchas

• Do not add shell wrappers (e.g. mise-exec.sh) to call mise; use mise.toml [tasks] and mise run.

• After clone: run mise trust, then mise install --locked (or make setup-tools / make setup); workflows are [tasks] in mise.toml (mise run lint, mise tasks)

• Refresh the toolchain lock with mise lock and commit mise.lock when bumping CLI tools

• Keep mise scanner versions aligned with .trunk/trunk.yaml (Trivy, OSV-Scanner) when bumping either side

• Run Python tools with uv run … in the project virtualenv

• Trunk pins linter versions under .trunk/; make setup-tools runs mise run trunk-install

• Commit uv.lock and mise.lock (do not gitignore them)

• If mise install --locked fails locally, extra tools in ~/.config/mise/config.toml may be missing from mise.lock; make setup-tools retries without --locked, or run MISE_LOCKED=false mise install

• If Trunk errors about a missing managed linter, run mise run trunk-install (via make setup-tools)

Parallel or multi-step work (Claude Code)

• This repo does not ship a built-in parallel orchestration subagent. For concurrent work, use multiple Task invocations, your editor’s multi-agent features, or your own scripts.
• After substantial or overlapping edits, use the verifier subagent (.claude/agents/verifier.md) to run build → lint → test → dependency scan → CodeQL by delegating each phase to build-and-fix, lint-and-fix, test-and-fix, security-scan, and codeql-fix.

Claude Code subagents

Invoked from Claude Code (Task tool or slash flows). Definitions: .claude/agents/*.md

• verifier — Five-phase verification via preload skills; see .claude/agents/verifier.md.

Claude Code skills

Slash-invoked skills live under .claude/skills/<name>/SKILL.md. Use a skill when it matches the task; each SKILL.md lists prerequisites (some require a CLI on PATH). Skills cite this file and Makefile targets rather than linking peer-to-peer to other SKILL.md files.

Some tools load mirrored skills under .agents/skills/ instead of .claude/. Other repos may add manage-changelog when Changie is configured (see Git workflow).

Coding agents & instruction files

Copilot / GitHub.com: This repo does not add .github/copilot-instructions.md; build, test, and style narrative stay in this file. On GitHub.com, personal instructions override repository content, then path-scoped rules and copilot-instructions.md apply (see Custom instructions). Edit policy: shared rules here; Claude Code-only behavior, @AGENTS.md import, and .claude/ details in CLAUDE.md.

Where things live (quick map)

• This file — Stack, make targets, style, testing, security, git, ADR pointers, Claude subagent/skill tables
• CLAUDE.md + .claude/ — Claude Code entrypoint and automation layout (see CLAUDE.md for directory breakdown and self-improvement rules)
• .agents/skills/ — Skills for tools that do not read .claude/ (e.g. postmortem); may mirror .claude/skills/
• .gemini/settings.json — Gemini CLI project context
• .cursor/rules/ — Optional Cursor rules (e.g. Always Apply); see Cursor: Rules
• .codex/config.toml — Optional Codex defaults (sandbox, approvals); links above under OpenAI Codex

Learned User Preferences

• Prefer minimal, focused fixes over broad refactors; do not over-implement beyond stated scope when adding CLI or adapter features.
• Do not commit local uv.lock or mise.lock churn from setup when unrelated to the task.
• Exclude unrelated untracked paths (e.g. .cursor/hooks/) from commits unless explicitly requested.

Learned Workspace Facts

• This repo is bq-inspector (BigQuery CLI); package source is src/bq_inspector/, not template src/your_package/.
• In git worktrees, run mise trust before make setup-python or other mise tasks.
• Pyright resolves imports from .venv; run make setup-python before make lint or third-party imports fail spuriously.
• BigQuery job subcommands are nested under jobs (e.g. bq-inspector jobs summary); flat shorthand also works (e.g. bq-inspector summary).
• The legacy bq-inspector schema subcommand was removed; use per-command --input-schema and --output-schema.
• BigQuery composite job IDs (projectId:location.jobId) may be passed in the jobId field; normalize_job_ref in core/shared/job_ref.py splits them.
• SDK adapter uses _resource_to_api_dict() (._properties with to_api_repr fallback) for jobs, datasets, and tables so responses include full REST fields.
• For job-local lineage/impact verification, use a job that references tables; SELECT 1-style jobs legitimately have empty lineage/impact fields.
• jobs lineage is job-local (jobs.get projection); lineage links and lineage graph use the Data Lineage API via datalineage/ (google-cloud-datacatalog-lineage).
• Data Lineage BigQuery table FQNs use Dataplex dot-delimited format bigquery:{projectId}.{datasetId}.{tableId} (not REST path-style).
• Lineage commands request cloud-platform OAuth scope (required by searchLinks); IAM still requires roles/datalineage.viewer on clientProjectId.
• clientProjectId defaults to projectId for lineage commands when omitted; bare lineage --help shows the lineage group while bare lineage with --params/--input-schema still flat-aliases to jobs lineage.