kq

Imported from jboelter/kq (AGENTS.md). Install upstream with npx skills add jboelter/kq. Copyright stays with the author.

Copilot instructions for kq

todo.md tracks current follow-up work and recent completions.

Current repository state

• This repository contains a working Go implementation of the kq/kqr Kusto CLI as described in spec.md.
• spec.md is the authoritative source for product design, command surface, safety rules, and package responsibilities.
• The codebase currently ships two binaries under cmd/: kq and kqr.
• kq and kqr share the internal package set and differ by injected capability profile.
• internal/definition validation rejects .yaml query definitions that declare KQL parameters without matching YAML parameter metadata, and it also rejects parameterized YAML definitions that omit declare query_parameters(...) from kqlTemplate.
• auth show reports interactive based on implemented interactive auth modes, so the current shipped profiles report false.
• catalog list emits a structured empty Services NDJSON object when no catalog is configured or the loaded catalog is empty.

Build, test, and lint commands

• go build -o bin/kq ./cmd/kq and go build -o bin/kqr ./cmd/kqr to produce local binaries.
• build.cmd and build.sh cross-compile kq and kqr for Windows, Linux, and macOS, and inject version metadata via ldflags.
• go build ./... for compile validation across the whole module.
• go test ./... for unit tests.
• go test -tags=e2e ./internal/client for tagged client integration tests.
• go vet ./... for static analysis.
• gofmt -l . to check formatting; gofmt -w . to apply.
• GitHub Actions workflows live under .github/workflows/: ci.yml validates formatting, tests, vet, and builds on pushes/PRs, and release.yml builds tagged release artifacts. Use the local commands above for interactive validation and debugging.

Kusto emulator workflow

• The tagged client e2e suite is now designed around the Azure Data Explorer Kusto emulator, not the public Samples cluster.
• The default automated path is suite-style setup/teardown in internal/client/client_e2e_test.go via TestMain and testcontainers-go/modules/compose.
• The managed test path uses the repo's docker-compose.yml, not an ad hoc container definition, so local/manual and automated workflows stay aligned.
• The compose stack now includes a one-shot seed sidecar; the seed script (testdata/kusto-emulator/compose-seed.sh) and seed data (testdata/kusto-emulator/events.csv) are co-located and checked in.
• Run the managed tagged suite with:
  • KQ_E2E_EMULATOR=1
  • go test -tags=e2e ./internal/client
• In managed mode, the test suite creates its own temporary emulator data directory, randomizes the published port and container name, waits for readiness, seeds the database, and tears the stack down after the package finishes.
• Manual/scripted runs should set or inherit:
  • KQ_E2E_EMULATOR=1
  • KQ_E2E_CLUSTER=http://127.0.0.1:8080
  • KQ_E2E_DATABASE=KQE2E
  • KQ_E2E_AUTH_MODE=emulator
  • KQ_E2E_MANAGED=0
• The compose file accepts overrides for KQ_EMULATOR_PORT, KQ_EMULATOR_DATA_DIR, KQ_EMULATOR_SEED_DIR, and KQ_EMULATOR_CONTAINER.
• If Docker is installed but the current shell cannot access the daemon, refresh group membership before running the managed suite.
• The emulator is HTTP-only and unauthenticated. Keep that behavior isolated to explicit emulator mode; normal cloud flows must remain HTTPS + authenticated.

Go best practices

• Use semantic version tags for releases and keep module paths compatible with Go's versioning rules, including major-version suffixes if the module ever reaches v2+.
• Keep cmd/kq and cmd/kqr thin. They only inject the profile and call app.Main.
• Design packages around clear ownership boundaries from the spec. Avoid cross-package reach-in helpers and avoid circular dependencies.
• Accept interfaces where they are consumed, not where they are produced. Keep interfaces small and purpose-built so shared logic stays testable without leaking adapter details.
• Pass context.Context as the first parameter on request-scoped operations and propagate cancellation through auth, client, and output pipelines.
• Return errors instead of panicking. Wrap errors with %w when adding context, and preserve enough structure for command layers to map failures to stable exit codes.
• Keep structs and APIs idiomatic Go: favor zero-value-safe types when practical, prefer concrete types until an abstraction is needed, and avoid configuration via hidden global state.
• Keep concurrency explicit and minimal. Only introduce goroutines when they materially improve UX or throughput, and ensure ownership, cancellation, and error propagation are obvious.
• Prefer table-driven tests for shared parsing, validation, profile, and parameter logic. Keep tests close to package boundaries and cover kq/kqr profile differences directly.
• Preserve deterministic CLI behavior. Machine-readable stdout, diagnostics on stderr, and exit-code semantics are part of the public contract and should be validated in tests.
• Do not let azure-kusto-go types escape internal/client. Translate external models into project-owned request, result, and error types before they reach command code.

Architecture

• One shared Go codebase produces two profile-gated CLI binaries:
  • kq: full-featured Kusto CLI for interactive operator and developer workflows
  • kqr: reduced-functionality, read-only CLI for automation, agents, CI, and scripting
• kq and kqr differ by injected capability profile, not by duplicated business logic.
• Entrypoints are thin main packages in cmd/kq and cmd/kqr. Each creates the appropriate profile and calls app.Main.
• internal/profile is the core policy boundary. Profiles gate command registration, request categories, auth modes, stdout output policy, and interactive features.
• CLI flag parsing and subcommand dispatch is handled by jessevdk/go-flags (v1.6.1). The subcommand tree is built in buildParser() using parser.AddCommand() with profile-conditional registration.
• Commands implement the flags.Commander interface (Execute(args []string) error). Flags are defined via struct tags (long:"...", description:"...", required:"true", default:"...").
• Structured command/result stdout defaults to NDJSON. Parser help/usage text, llm prompt, query template, catalog template, and completion output are intentional plain-text exceptions. Query-shaped commands use --spill, --spill-dir, --format, --all-tables, --timeout, --request-option, --request-properties, and --request-parameter to control delivery and execution; --format csv allows single-table query results to stream as CSV to stdout or spill as CSV files. Shared option structs (ConnectionOpts, AuthOpts, OutputOpts, ExecutionOpts, RequestPropertyOpts, QueryInputOpts) are embedded in a baseCmd type that also carries execution pipeline helpers.
• All command handlers live in internal/app, organized by domain file (query.go, database.go, entity.go, etc.). There are no separate cmd* packages.
• azure-kusto-go is isolated behind internal/client. Command handlers never import the SDK directly.
• The repo carries a local patched copy of azure-kusto-go/azkustodata under third_party/ and redirects to it with a replace in go.mod. That local patch prevents bound query parameters from auto-prepending declare query_parameters(...); authored query text remains authoritative while we wait for an upstream change.
• No service catalog is embedded in the binary. App starts with services.Empty() and loads entries from a user-supplied YAML or JSON catalog file via --catalog or KQ_CATALOG_FILE. --service resolution and catalog list require an explicit catalog file; catalog list emits a structured empty Services object when no catalog is configured or the catalog is empty.

Command tree

Profile command names use tree paths. These are the strings used in internal/profile/profile.go:

Shared (ReadOnlyProfile + FullProfile)

• query run
• query template
• query validate
• database list
• database describe
• entity list
• entity describe
• entity sample
• auth show
• catalog list
• catalog status
• catalog template
• completion
• llm prompt
• version

kq-only (FullProfile only)

• mgmt run

The subcommand tree is built in buildParser() using parser.AddCommand() with profile-conditional registration.

catalog list reads from the externally supplied catalog file if one is configured; otherwise it emits a single structured empty Services object with schema metadata and rowCount: 0.

Key conventions

• kqr must stay read-only by construction. Do not add mutable commands, ingestion paths, interactive prompts, or prompt-based auth behind flags or hidden paths.
• kq is a superset for operator workflows, but shared read-only commands must keep the same request, validation, and output behavior across both binaries.
• Do not assume bundled services exist. Catalog-driven features must behave correctly when the catalog is empty and when it is loaded from --catalog or KQ_CATALOG_FILE.
• Preserve the output contract:
  • outside explicit text-output exceptions, kqr stdout is reserved for result data or result manifests
  • kqr stderr is reserved for diagnostics, warnings, and errors
  • structured command/result stdout is NDJSON; help text, llm prompt, query template, and completion are textual exceptions
  • machine-readable output is a first-class feature, not an afterthought
• Keep capability enforcement in code, not just documentation. Request categories and auth policies are validated centrally through profiles.
• All command handlers must emit output through emit(), resultFromMap(), or resultFromSlice() in internal/app/helpers.go. Do not write directly to stdout with json.Encoder or fmt.Fprint except for the spill-manifest path inside the shared output helpers and the intentional text-output paths for parser help/usage, llm prompt, query template, and completion.
• Do not let azure-kusto-go types or behaviors leak into command handler code.
• Parameter handling is a first-class subsystem: merge repeated --param inputs plus repeated --params object sources (inline JSON or @file); validate required parameters; map friendly names to Kusto parameter names; and serialize typed values into Kusto-ready literals.
• Error handling preserves stable, scriptable exit behavior. Baseline exit codes: 0 success, 2 usage or validation error, 3 authentication failure, 4 request or service failure, 5 partial query failure detected in-band, 6 output handling failure.

Package responsibilities

• internal/app — go-flags parser construction, subcommand registration, handler implementations, shared option structs, execution helpers
• internal/profile — capability profiles and policy gating
• internal/auth — auth mode resolution and token acquisition
• internal/client — Kusto SDK adapter, result normalization, and service error extraction (normalizeServiceError wraps SDK errors into project-owned ServiceError before they escape to command code)
• internal/definition — .kql and .yaml loading, parameter schema, typed serialization
• internal/format — NDJSON stdout rendering plus spill-file rendering
• internal/manifest — spill-to-disk and manifest generation
• internal/model — shared request and result types
• internal/errors — structured errors with exit codes; includes ServiceError for normalized Kusto service diagnostics
• internal/services — external JSON service catalog loading, normalization, sorting, and alias resolution
• internal/version — build metadata