Imported from atlanhq/application-sdk (
.claude/skills/adopt-preflight-gate/SKILL.md). Install upstream withnpx skills add atlanhq/application-sdk --skill adopt-preflight-gate. Copyright stays with the author.
Adopt the SDK-native preflight gate
Scope boundary — check first
This skill is for v3 apps only (subclasses App, @entrypoint methods,
Handler in app/handler.py). If the app is v2 (Argo-era layout,
application_sdk.workflows/handlers imports), STOP and run /upgrade-v3
first; this skill picks up after.
Reference implementation for everything below: atlan-mysql-app (PR #340) —
short-circuiting auth check with typed error, advisory tables check, PARTIAL
status. Read its app/handler.py before proposing changes.
What changed (context you state to the developer up front)
- The SDK injects
{app}:preflightas the first activity of every extraction workflow. It calls the app's oneHandler.preflight_check. PreflightOutput.statusis the gate verdict:NOT_READYis always reported (asoutcome="would_block"), and aborts the run only when the app has opted into hard mode (preflight_gate_mode = "hard", typedPreflightFailed, red activity);READYandPARTIALproceed.PARTIALis display-only — use it for "advisory check failed, run anyway".- There is no per-check
blockingflag. Importance is expressed by control flow: required checks short-circuit (returnNOT_READYearly), advisory checks run and only influencePARTIAL. - Return the verdict; don't raise it. The returned status is what both
surfaces render, so a block belongs there. But raising is not a no-op, and
what it does depends on the error's type: a typed plumbing error
(
RateLimitedError,DependencyUnavailableError,ResourceExhaustedError) means "I could not determine readiness" and fails open in both postures; anything else — an untyped crash, a typed source error, overrunning the budget — is treated as an unverifiable source and blocks in hard mode. So an uncaught probe exception is a run-aborting bug for a hard app, not a harmless fail-open. - A failed check should carry
error=<SDK leaf>(...).to_failure_details()— category/code/audience/suggested_action flow to the Automation Engine and dashboards. Untyped failures fall back to thePREFLIGHT_CHECK_FAILEDsentinel. - Behavior to say plainly: by default the gate is soft — a handler that
returns
NOT_READYdoes not block the run; the verdict is always reported (asoutcome="would_block") but the run proceeds. Blocking real runs is a per-app opt-in (preflight_gate_mode = "hard", next section) taken once the app's checks are trusted. - The two surfaces receive credentials differently — a nuance the "one
implementation" principle hides. The UI's Test-Connection sends credentials
inline in the request body; the gate has no body and resolves credentials
from the extraction input's top-level triple (
credential_guid/credential_ref/agent_json). It does not readconnection.attributes.defaultCredentialGuid. So a green UI check is not proof the gate will resolve the same credential — if the guid only reaches the input via the connection, the gate sees none (phase-0 bucket 9).
Hard mode — the gate-level opt-in (CNCT-81)
Enforcement is a gate property, not a handler property. The handler always
returns the honest verdict; the gate decides what to do with NOT_READY:
- soft (default): never raise — the run proceeds and the dodged block is
emitted as
outcome="would_block"(withgate_mode="soft"and the per-checkcheck_matrix) on the gate outcome event. The verdict is always reported, so connector-pulse can rank apps by how often they would have blocked real runs; that list is the "your checks are ready to enforce" queue. - hard: raise
PreflightFailed, run aborts onNOT_READY. The opt-in for every app whose checks are trusted to gate real runs.
Opting in is deliberately explicit and deliberately small:
class MyApp(App):
preflight_gate_mode = "hard" # git-blamed: checks are trusted to block runs
or, ops-side without an app release: ATLAN_PREFLIGHT_GATE_MODE=hard on the
worker deployment (env wins over the attribute; any value other than the
literal hard resolves to soft — malformed config never blocks a run by
accident). The worker logs an INFO line per hard app at boot, and emits a
queryable Preflight gate posture event per app carrying the resolved mode and
budget. Prefer the per-app attribute: the env lever applies to every app on that
worker, including ones whose checks have never been validated against real runs.
Hard mode covers every outcome the gate attributes to the source — a
NOT_READY verdict, a probe overrunning the budget, a handler crash, a provably
absent credential. Failures of the gate's own plumbing (rate limit,
secret-store outage, a credential lookup that failed for any other reason, worker
unavailable) always fail open, in both postures. The outcome event
carries gate_classification (source_unverifiable vs gate_broken) so the two
are separable in pulse.
The check budget — size it before flipping to hard
Handler.preflight_check gets App.preflight_gate_timeout_seconds (default 150,
clamped 5-300) and the SDK enforces it: the gate cancels the handler when it
elapses. In hard mode an overrun blocks the run, so this is not a formality.
It bounds the whole handler call, not each check — one slow probe can consume the budget and leave the rest unrun. And it is a deadline, not a reservation: a handler returning in 3s holds its worker slot for 3s regardless of the budget. So a generous budget costs nothing on a healthy run; it only changes the run that would otherwise have been cut short.
App.preflight_gate_max_attempts (default 2, clamped 1-3) sets the retries. A
retry rescues a transient — a cold pool, a cluster resuming — by trying again;
it cannot rescue a systematically slow check, which needs a bigger budget instead.
Both timeouts derive from these two numbers, so an app declaring a large budget
usually wants 1: at the 300s ceiling, two attempts reserve a ~10 minute
schedule_to_close.
class MyApp(App):
preflight_gate_mode = "hard"
preflight_gate_timeout_seconds = 250 # this source's probe is genuinely slow
preflight_gate_max_attempts = 1 # a retry won't rescue a slow check
What the skill checks during adoption:
- Size from the p99 of successful runs, not the max. Sizing to the worst
observed run makes the timeout decorative — nothing ever overruns, so hard mode
is back to enforcing only
NOT_READYverdicts. Sizing to p95 blocks 5% of runs. - Read
gate_duration_ms, nevercheck_matrixdurations. Per-checkduration_msis written by the app, and rows predating SDK 3.25 carry durations from abandoned attempts (see the orphaned-attempt note below), so they read far above any budget that was ever in force.gate_duration_msis measured by the SDK. Headroom isgate_duration_ms / (gate_timeout_seconds * 1000). - Measure the handler's real cost before flipping to hard. If the app runs a
comparable probe as a
@taskelsewhere, itstimeout_secondsis the honest estimate — a check that mirrors a 600s task will not fit in the default. - Size probes to
PreflightInput.timeout_seconds, don't defeat it. That field carries what remains after credential resolution.max(input.timeout_seconds, <bigger constant>)discards it; adeadlinewhose per-probe floor never forces an early return makes it decorative. Both read as "budget honoured" in review and are not. - Bound the whole handler, not just the probes. Client build, connect, and auth are network I/O and count against the budget.
- Keep probes awaitable. Cancellation lands at an
await, so blocking synchronous I/O on the event loop escapes the budget entirely and stalls the worker's other activities. Run blocking drivers in a thread. - Fan-out is where budgets die. A per-catalog/per-schema loop scales with the source, not with the code. Prefer a bounded scope, concurrency, or an early exit once the check is satisfied.
Rules the skill enforces during adoption:
- Never soften the handler to dodge the gate. Returning
PARTIALfor a failure that should block hides the truth from every surface; posture belongs on the App class, verdicts belong in the handler. - Never return
NOT_READYfor a transient. A 429 or a dependency outage is "ask me later", not "the source is not ready" — collapsing them makes hard mode fail closed on a blip. Raise a typedRateLimitedError/DependencyUnavailableErrorinstead; the gate routes those to fail-open. - Soft is the default landing state with two exit conditions, both required
before adding
preflight_gate_mode = "hard":- the app's
would_blockrows track real workflow failures (the checks are right, not just loud), and - the app's p99
gate_duration_mssits comfortably inside its declared budget (the checks fit, so an overrun is signal rather than routine). Flipping on (1) alone converts a fail-open into a block on every slow run.
- the app's
- An app with no
preflight_checkhandler needs no posture: the DefaultHandler never returnsNOT_READY, so the gate never has anything to enforce.
What the gate emits, and which numbers to trust
Every outcome writes one Preflight gate outcome row. v3 apps log to
otel_logs.service_logs (not combined_workflow_logs, which is the Argo path),
and LogAttributes is a Map, so LogAttributes['outcome'] works directly while
check_matrix needs JSONExtractArrayRaw on the string value.
| attribute | meaning |
|---|---|
outcome |
proceeded / would_block / blocked / no_verdict / skipped |
gate_mode |
resolved posture; absent on the workflow-emitted rows |
gate_classification |
verdict / source_unverifiable / gate_broken |
gate_duration_ms |
SDK-measured elapsed; the only number that can size a budget |
gate_timeout_seconds |
the budget in force, so headroom needs no join |
gate_attempt |
distinguishes a first-try pass from a retry rescue |
check_matrix |
per-check name/passed/error_code/duration_ms; [] where no check ran |
check_matrix is present on every outcome, so parse it unconditionally rather
than branching on field presence — a branch mishandled in the dropping direction
is how a gate that never reached a verdict vanishes from the numerator.
The orphaned-attempt caveat, for anything sized on historical data. Before SDK
3.25 the gate had no timeout of its own. Temporal's start_to_close is enforced
server-side, and a non-heartbeating activity's coroutine is not stopped by it — so
the workflow gave up at 25s and failed open while the abandoned handler kept
running, finished minutes later, and emitted its own outcome row. Consequences when
reading old data:
- a
blockedrow does not prove the run was blocked; the workflow may have moved on long before it was written - the same
workflow_run_idcan carry bothblockedandno_verdict, both at attempt 1 — that pair is the signature check_matrixdurations far above any budget come from those orphans, not from probes that were permitted to run that long
The SDK-side cancel fixes this: the gate's own timer fires before Temporal's, so the
work actually stops and the row means what it says. Size budgets from
gate_duration_ms going forward, or from temporal.activity.duration_ms on the
activity.ended event, which the log interceptor has always measured honestly.
The design principle every decision flows from
One implementation, two surfaces. Handler.preflight_check is the single
authority on "is this source ready" — the UI's Test-Connection button and the
gate at the head of every run both execute the same function. Two copies of
readiness logic (a handler AND an activity, or checks buried in extraction
code) inevitably drift: the UI says ready while the run fails, or worse the
inverse. Drift is the anti-pattern this whole feature exists to kill.
Corollaries the skill acts on:
- Consolidation always flows activity → handler, never the other way. The handler is reachable by both surfaces; an activity is reachable by one.
- Anything that behaves like preflight IS preflight. If code verifies source readiness before extraction — regardless of what it is named or where it lives — its natural home is the handler, where the UI benefits from it too. Phase 0 hunts for these by behavior, not by name.
- Deletion requires a coverage diff. Consolidating or deleting duplicate readiness logic must never lose a check: enumerate what the old code verified, prove each item exists in the handler, fold gaps in first.
Frame it to the developer as what they gain: write a check once, and the Test-Connection button, every scheduled run, the Temporal failure pane, and the failure dashboards all get it — with a typed, actionable error — for free.
Phase 0 — Classify the app
Run these detections and report the bucket(s) before changing anything:
- Collision class — a
@taskwhose activity name resolves to{app_name}:preflight(a task method literally namedpreflight, or explicitname="preflight"). The worker will REFUSE TO BOOT on the bumped SDK (WorkerActivityNameCollisionError). Known fleet members: atlan-presto-app, clickhouse, power-bi-app, qlik-sense-cloud-app, redshift-app, teradata-app, trino. - Coexistence class (named) — other
*preflight*-named@tasks (e.g.miner_preflight_check). Non-breaking: they keep running alongside the gate. Each is a consolidation candidate for phase 2. - Hidden preflight (semantic hunt) — readiness logic that is preflight in
behavior but not in name. Scan
@taskbodies and entrypoint code that runs BEFORE the extraction fan-out for:- connect-and-probe patterns (
SELECT 1, ping, token validation, list-one API call) whose result only gates whether to continue; - tasks named
test_*,validate_*,check_*,verify_*,*_probe; - early-exit guards in
run()that abort before any data is extracted. Classify each hit with the developer in mind: - True preflight — read-only source-readiness verification, no side effects the extraction depends on → consolidation candidate: it belongs in the handler so the UI check runs it too.
- Execution guard owned by the SDK (e.g. the sql template's
prime_sql_authwarm-up) → leave alone; it is infrastructure, not app preflight, and the SDK maintains it. - Business logic wearing a check's clothes (produces state extraction consumes, seeds caches the run needs) → leave in the workflow; moving it to the handler would make the UI path perform work. When unsure which of the three, ask the developer — that is a phase-2 question, not a guess.
- connect-and-probe patterns (
- Gate eligibility — the entrypoint input contract must carry the
credential-routing triple (
extraction_method,credential_guid,agent_json), normally by extending the toolkitExtractionInput(checkapp/generated/*_input.pyorcontract/). Missing triple on a source-ful app = the gate silently skips; fix the contract. - Handler presence — no
Handler.preflight_checkat all → the gate runs the SDK DefaultHandler no-op (never blocks). Valid state; offer to write a handler in phase 2 but do not require it. - Silent-drift audit — list every
input.metadata/input.connection_configkey read insidepreflight_check, and cross-check each against the input contract's fields. On the gate path, metadata is rebuilt from the extraction input'smodel_dump; a UI-form-only key is absent — a hard[...]read crashes, which is a handler crash and therefore blocks every run in hard mode; a defensive.get(..., default)silently runs the check with wrong config. Every unmatched key needs a decision in phase 2. - Multi-credential class — an app that needs more than one credential to
verify a source (e.g. an API token AND an object-store credential), whose
per-auth-type guids live in separate input fields, not on the single
top-level
credential_guidtriple. The gate resolves only that one triple, so the symptom on the gate path is the handler receivingcredentials=[], defaulting to one auth type, and raising missing-credential on every gated run — reported on every run in soft mode, and aborting every run in hard mode. Detect by: multiple*_credential_guidfields on the input contract, or a handler that resolves guids itself (CredentialResolver/get_credentialscalled insidepreflight_check). If found, the app adopts the SDKpreflight_credential_refsprimitive in phase 2 (see 2g) — it must NOT hand-roll credential resolution or the fail-open taxonomy. - Multi-entrypoint class (crawler + miner, etc.) — more than one
@entrypointmethod on the App class (grep@entrypoint). The gate is injected per workflow type andPreflightInput.entrypointis baked from the entry-point's registered name — so a miner run reliably arrives withentrypoint="miner", a crawler run withentrypoint="crawler". The singlepreflight_checkmust branch oninput.entrypointand run only that entrypoint's checks; each entrypoint gets its own tree in phase 2 (see 2h). Two traps: (a) agetattr(input, "entrypoint", "crawler")-style default is dead code — the field always exists, defaulting to"", which is falsy against== "miner", so a mis-set default silently routes miner runs down the crawler branch; (b) entrypoints usually differ in how credentials arrive (next bucket). - Late-credential-derivation class — the input carries the credential
triple, but
credential_guid/credential_refis populated inside the entrypoint body rather than at input construction. Classic case: a miner that reuses the crawler's connection, so the guid arrives only onconnection.attributes.defaultCredentialGuidand the body derives it (input.model_copy(update={"credential_guid": ...})). The gate runs before any body code, so it resolves an empty triple →credentials=[]→ the handler's connect step fails → softwould_block, while extraction (which derives the guid late) still succeeds. Tell-tale triad: UI Test-Connection green, miner gate soft-fails, extraction runs anyway. Detect by grepping entrypoint bodies for credential assignment ordefaultCredentialGuidreads before the first extraction activity. Fix in phase 2 by lifting the derivation to input construction (see 2h). SDK-side fix tracked in CNCT-92. - Non-routable
extraction_methodclass — the credential router (CredentialRef.resolve) readsextraction_methodas the routing selector and accepts onlydirect(+credential_guid) oragent(+ populatedagent_json); anything else raisesCredentialRoutingError. Query-extraction / miner-family manifests commonly setextraction-methodto an extraction kind instead — e.g.query_history— which names what is extracted, not how creds are routed. Left as-is, the gate'sresolve()raises on every run and survives only via the deprecatedlegacy_credential_reffallback (removed in SDK v4.0) — so it looks like it works (a caughtCredentialRoutingErrorstacktrace on every run) until v4.0 removes the fallback or the app flips to hard mode, at which point the gate resolvescredentials=[]and soft-skips or hard-aborts. Detect by grepping the manifest (app/generated/**/*.json) and the input model forextraction-method/extraction_methodvalues that are notdirect/agent. Fix in phase 2 by normalizing it at input construction (see 2h). Same SDK ticket: CNCT-92.
Phase 1 — Bump and boot
-
Daft-cliff pre-check (mandatory when the lock resolves SDK <3.20.0). SDK v3.20.0 removed daft and moved the transformation layer to DuckDB/pyarrow — the bump below drags the app across that cliff. Check
uv.lock's resolvedatlan-application-sdkversion; if it is below 3.20.0, orgrep -rn "import daft" app/hits, invoke/migrate-off-daftnow and complete it before proceeding — it owns the breakage taxonomy (empty[daft]extra, changed transformer contracts, missingduckdb, the silent literal-vs-column precedence flip) and its done-bar is output parity: transformed output structurally identical, no attribute lost. Only when its step-4 evidence is green does phase 1 continue; the daft migration and the gate adoption land as separable changes, in that order.Applies to apps already past the cliff too: on every SDK release 3.20.0-3.25.0 the DuckDB transform raises
AttributeError: DuckDBPyConnection ... has no attribute 'to_arrow_table'when the lock resolvesduckdb<1.5.0(application-sdk#2940). The bump below cannot escape it — the target version is inside that range. Pinduckdb>=1.5.0,<1.6.0as a direct dependency, or wait for the release carrying #2940; see/migrate-off-daftclass 3b. -
Always resolve the current latest SDK first.
.info.versionalone may still be inside the release cooldown, so list versions with upload dates and pick the newest one older than 7 days:curl -s https://pypi.org/pypi/atlan-application-sdk/json | jq -r '.releases | to_entries[] | [.key, .value[0].upload_time] | @tsv'. Pin that version — it must also satisfy the gate floor,>=3.24.1— inpyproject.toml. (Pinning an in-cooldown version and then locking with the--exclude-newerbound below is unsatisfiable and fails with an unhelpful resolution error.) Tiebreak when the floor itself is still inside the cooldown (no version satisfies both): the floor wins. Adopt the in-cooldown version, drop the--exclude-newerbound for the SDK only (keep it for everything else viauv lock --upgrade-package atlan-application-sdkafter the bounded full refresh), and flag the fresh pick for human review in the PR — per the org's fresh-dependency rule — noting the date the cooldown clears. Then refresh the whole lock, not just the SDK:uv lock --upgrade --exclude-newer "$(date -u -v-7d +%Y-%m-%dT%H:%M:%SZ)"(GNU date:date -u -d '7 days ago' ...) followed byuv sync --all-extras --all-groups— the--exclude-newerbound keeps every transitive pick inside the org's release-age cooldown, and the full refresh means stale transitive pins don't surface as unrelated breakage mid-adoption. (If the full upgrade drags in an unrelated failure, bisect by falling back touv lock --upgrade-package atlan-application-sdkand report the offending dep — but the full refresh is the default.) 3.24.0 carries the gate, the multi-credential primitive (preflight_credential_refs/credentials_by_name), and the per-entrypoint credential handling this skill relies on; 3.24.1 adds the budget knobs (preflight_gate_timeout_seconds/preflight_gate_max_attempts), which the sizing steps below need — hence the floor. Raise the declared floor if it sits below>=3.24.1. (Earlier revisions of this skill said "don't raise the floor", then pinned>=3.24.0— both retired.) After the bump, confirm the surfaces landed:PreflightInput.credentials_by_name/preflight_credential_refsmust be importable, or the upgrade didn't take. -
Boot the worker. Boot is the collision detector.
-
Collision fix (if bucket 1): delete the app's
@task preflight, its call site in the workflow, and any now-orphaned private preflight input/output contracts. THEN — mandatory — diff the deleted activity's checks againstHandler.preflight_check: every check the activity performed must exist in the handler (the gate runs the handler, so coverage moves — it must not vanish). Fold gaps into the handler before proceeding. -
Run the app's existing test suite; fix fallout from the bump before touching preflight logic, so phase-2 diffs stay clean.
Phase 2 — Interactive check-design session
This is the heart of the skill. Do not silently rewrite the handler — the blocking/advisory structure is the developer's judgment; you propose and they decide.
2a. Render the current state as a tree
Parse the existing preflight_check and draw what it does today, in
execution order, as a decision tree the developer can react to:
current structure (as-implemented):
[auth probe] ──fail──> return NOT_READY (blocks: yes)
│ pass
▼
[tables check] ──fail──> status = NOT_READY (!) (blocks: yes — is this intended?)
│ pass
▼
all pass ──> READY
Annotate every place where the implemented behavior may not match intent —
the classic bug is status = READY if all(c.passed) else NOT_READY, which
promotes every advisory check to run-blocking.
Include the phase-0 consolidation candidates in the same picture, marked as living outside the handler, so the developer sees the whole readiness surface at once:
outside the handler today (phase-0 findings):
[crawler_preflight @task] duplicates auth+tables → consolidate?
[validate_filters (in run())] read-only filter probe → consolidate?
[warm_cache @task] seeds extraction state → stays (business logic)
2a-bis. The consolidation question, per candidate
For every named-coexistence activity and every "true preflight" semantic hit,
ask: "This verifies before extraction, but only runs in the workflow —
the UI Test-Connection never sees it. Move it into preflight_check so both
surfaces run it, and delete the activity?" On yes: fold the logic into the
handler as a check in the tree below, then delete the activity with the
coverage diff (north-star corollary 3). On no: record why in the phase
report — it stays as accepted, documented duplication (phase-4 cleanup list).
2b. Ask the classification questions, per check
For each check, ask (AskUserQuestion where available, plain questions otherwise), using these litmus tests:
- "If this check fails, can extraction produce anything useful?"
No → required (short-circuit
NOT_READY). Yes → advisory (PARTIAL). - "Does every later check depend on this one?" Yes → it must run first and short-circuit (no point checking permissions when auth failed).
- "Who fixes a failure of this check?" Customer (credentials, network, grants) → the error's default USER audience is right. The app team or Atlan (internal state, unexpected config) → the error must use an APP_OWNER / PLATFORM-audience leaf, otherwise the blocked run is attributed to the customer's configuration in the SLA split.
2c. Propose the target tree and the canonical ordering
Present the proposed structure the same visual way, and let the developer edit it before you write code. Canonical ordering (from the design review): reachability → authentication → authorization → advisory probes — each tier short-circuits the tiers after it.
proposed:
[connectivity] ──fail──> NOT_READY (error=SourceUnavailableError, stop)
│ pass
[auth] ──fail──> NOT_READY (error=AuthError, stop)
│ pass
[read access] ──fail──> NOT_READY (error=AppPermissionDeniedError, stop)
│ pass
├─[server version] ──fail──> mark failed, continue (advisory)
└─[optional feature] ──fail──> mark failed, continue (advisory)
status: NOT_READY only via a short-circuit above;
PARTIAL if any advisory check failed; READY otherwise.
2d. Suggest missing checks
Compare against the family baseline and propose (never force) additions:
- SQL sources: reachability/DNS of the host; authentication (connect + trivial query); read authorization (information_schema / catalog probe scoped by the configured filters); server version (advisory); optional extensions the miner needs (advisory — e.g. pg_stat_statements).
- API/BI sources: token validity; required scopes/permissions; access to the configured workspace/project/site; API version or tenant feature flags (advisory).
- Any source: if the check uses filter/config metadata, it must come from
contract fields (phase-0 silent-drift audit) — suggest adding missing fields to
contract/app.pkland regenerating, rather than reading form-only keys.
2e. Implement
Write the handler to the agreed tree. Requirements:
- Checks append to a list as they run; short-circuits return the list built so
far (
PreflightOutput(status=NOT_READY, checks=checks)). - Status logic is explicit — never
all(c.passed)when any check is advisory. - Time each check and set
duration_ms; the UI shows it. error=only on failed checks, statically-clean form:error=AuthError(message="...", suggested_action="...", cause=exc).to_failure_details()message= one clean sentence, no hosts/credentials/stack traces (thecause=chain carries a redacted, capped repr for diagnostics).suggested_actiononly when there is a concrete user-fixable step.
Leaf selection quick table (application_sdk.errors):
| Failure | Leaf | Audience default |
|---|---|---|
| bad credentials / login rejected | AuthError |
USER |
| host unreachable / refused / DNS | SourceUnavailableError |
USER |
| missing grants / scopes | AppPermissionDeniedError |
USER |
| required state/extension/version absent | PreconditionError |
USER |
| app-internal inconsistency | InternalError (or subclass) |
APP_OWNER |
2f. App-specific error classes (when the leaves aren't enough)
Three tiers, lightest first — pick the lightest that fits:
- Bare leaf —
AuthError(message="..."). Semantics (category/code/ audience/retryable) come from the leaf. Fine for standard failures. - Per-instance overrides —
AuthError(message="...", suggested_action="...", retryable=True, cause=exc).message,suggested_action,retryable,causeare instance fields on every leaf. Right for one-off checks; no class needed. - App subclass — when the same message/action is reused across checks,
when the app wants its own
codefor aggregation (dashboards and the Automation Engine group by code), or when the failure carries app-specific evidence. What you CANNOT override per-instance:category,code,audience— those are ClassVars (fixed aggregation keys), and changing them is exactly what a subclass is for.
Subclasses live in app/failures.py (same home the /typed-failures
skill establishes — reuse the file if it exists). Pattern:
from dataclasses import dataclass
from typing import ClassVar
from application_sdk.errors import AuthError
@dataclass(kw_only=True)
class MySQLAccountLockedError(AuthError):
"""MySQL rejected login due to FAILED_LOGIN_ATTEMPTS lockout."""
# reusable defaults — override AppError's instance fields
message: str = "MySQL account is locked out after repeated failed logins."
suggested_action: str | None = (
"Wait for auto-unlock or have a DBA run ALTER USER ... ACCOUNT UNLOCK, "
"then re-run preflight."
)
# distinct code for aggregation; category/audience inherit from AuthError
code: ClassVar[str] = "MYSQL_ACCOUNT_LOCKED"
# any extra dataclass field lands in FailureDetails.evidence automatically
lockout_seconds_remaining: int | None = None
Usage in a check is identical to a leaf:
error=MySQLAccountLockedError(cause=exc, lockout_seconds_remaining=300).to_failure_details()
What the owner gets for free: category AUTH and audience USER inherited
(no re-declaring), the distinct code shows up in the gate outcome event's
reason and in AE/top-error-code dashboards, lockout_seconds_remaining
rides in evidence, and the cause chain is redacted and capped by the SDK.
Two guardrails: evidence field names must not look secret-bearing
(*_secret / *_password / *_token are rejected at the wire boundary), and
if the failure is app-internal rather than customer-fixable, base the subclass
on an APP_OWNER-audience leaf (InternalError) — not on AuthError — or the
SLA split misattributes it (see the audience litmus question in 2b).
During the interactive session: when the developer picks the same leaf with
the same custom message/action for two or more checks, proactively offer to
extract a subclass into app/failures.py.
One module, no exceptions. Every typed class the app raises — from preflight
checks and from extraction tasks — lives in app/failures.py (or whatever
single module the app already uses; match it, don't add a second). Scattered
definitions are how the same root cause ends up with two different codes
depending on which surface hit it, which silently splits its aggregation in AE
and the dashboards. If the app has untyped raise sites left over, run
/typed-failures first — it owns the sweep and this skill assumes its output.
Pick the leaf by who must act, because that is what the SLA split reads.
audience is a ClassVar, so the leaf choice is the routing decision:
| Audience | Leaves | Means |
|---|---|---|
USER |
AuthError, AppPermissionDeniedError, NotFoundError, PreconditionError, InvalidInputError, AlreadyExistsError, SourceUnavailableError, RateLimitedError |
the connector owner must fix credentials, permissions, or source config |
PLATFORM |
DependencyUnavailableError, ColdStartRaceError, ResourceExhaustedError |
infra ops must act |
APP_OWNER |
InternalError, DataIntegrityError, UnimplementedError, AppTimeoutError, bare AppError |
the app team owns it — a connector or SDK bug |
Two traps that land a source problem on the app's ledger:
- Untyped is not neutral. A bare
Exception, a bareAppError, orInternalErrorresolves toAPP_OWNER. Leaving a source failure untyped does not defer the decision — it silently files the bug against the app. - A slow source is not
AppTimeoutError. That leaf isAPP_OWNER(and retryable). A source that will not answer belongs inSourceUnavailableError(USER). Reach forAppTimeoutErroronly when our own deadline is the fact being reported.
2f-bis. Soft mode and the failure-rate SLA (do this before flipping to hard)
Soft mode deliberately lets a run proceed after the gate has already said the
source is not ready or could not be verified. That run will often fail later, in
extraction, on the same root cause the gate just reported — and if those
downstream raise sites are untyped, each of those failures is filed against the
app (APP_OWNER, per the table above) even though the gate's own row already
attributed it correctly. The app's failure rate absorbs a source problem it
diagnosed but was told not to block on.
So while the app is still soft, walk the failure paths for each check the gate can report and confirm the extraction-side failure for that same cause carries the same leaf the check does:
- Take each blocking check and ask: if the gate reports this and the run proceeds anyway, where does extraction die? Name the raise site.
- Make it the same leaf. If the check reports
AuthError, the extraction failure for bad credentials must also beAuthError— notInternalError, not untyped. One root cause, one code, one audience, on both surfaces. - Do not swallow it into a retry loop that eventually raises something
generic. The last error standing is the one the Automation Engine attributes,
so a typed cause wrapped in an untyped retry-exhausted error routes to
APP_OWNER. - Do not pre-empt the gate. Deleting a check because "extraction will fail anyway" loses the early, cheap, correctly-attributed signal and keeps only the late, expensive one.
Why this matters beyond tidiness: the gate's outcome row and the workflow's
failure share workflow_run_id, so a run that failed downstream of a
would_block is identifiable — the pair is the evidence that the failure was an
accepted-risk proceed, not a regression. That evidence is only usable if the
downstream failure is typed; otherwise the two rows disagree about whose fault
it was and the app's is the one that counts.
Logging on those paths, per docs/standards/logging.md: log the typed error
(pass the exception, not str(exc) interpolated into a message), keep the level
at error only for genuine failures, and never log credential values or a
driver message that embeds a connection string — the SDK redacts
FailureDetails.cause_repr, but a hand-built log line is yours to keep clean.
2g. Multi-credential apps — declare named refs, don't hand-roll
Only if phase 0 flagged the multi-credential class. The gate resolves exactly one credential off the top-level triple, so an app with per-auth-type guids must tell the gate which guids to resolve — declaratively. Do not re-implement resolution or the outage-vs-not-found taxonomy in the handler: that boilerplate is dangerous (a store outage misread as a bad credential blocks healthy runs), and the SDK now owns the one correct implementation.
Declare a ClassVar map of ref-name → the input field holding that guid, on
the extraction input:
class MyAppExtractionInput(ExtractionInput):
preflight_credential_refs: ClassVar[dict[str, str]] = {
"api": "api_credential_guid",
"object_store": "object_store_credential_guid",
}
The gate resolves each guid inside the activity under one fail-open taxonomy
(confirmed outage → the workflow fails open, never blocks; genuine not-found →
an empty group so the handler decides NOT_READY) and hands the handler the
results grouped by name:
async def preflight_check(self, input: PreflightInput) -> PreflightOutput:
api = input.credentials_by_name["api"]
obj = input.credentials_by_name.get("object_store", [])
...
Guardrails:
- It MUST be a
ClassVar, not a pydantic field. Declared as a field, the gate reads{}and silently falls back to the single-triple path (the SDK logs a warning if you get this wrong, but it fires per gated run while building the gate input — not once at boot — so declare it correctly up front rather than relying on spotting the warning). - The named guids must be top-level fields on the input contract (the gate reads
them from the extraction-input snapshot). If they arrive nested under AE
metadata, lift them onto the contract — the same fix as the phase-0 silent-drift audit. - Delete the app's hand-rolled credential resolution + taxonomy once the primitive is in; that consolidation is the point (one correct implementation in the SDK). Coverage diff still applies (north-star corollary 3).
- Single-credential apps declare nothing and keep the top-level triple path — zero change.
This primitive ships in the SDK release the phase-1 lock upgrade pulls; if
PreflightInput.credentials_by_name / preflight_credential_refs aren't present
on the installed SDK, the upgrade didn't land — re-run phase 1 before adopting.
2h. Multi-entrypoint apps — one handler, per-entrypoint trees + per-entrypoint creds
Only if phase 0 flagged the multi-entrypoint class (bucket 8). One
preflight_check, branching on input.entrypoint; design each entrypoint's
tree separately in 2c (a crawler validates schemas/tables; a miner validates
its query-history prerequisites, e.g. pg_stat_statements). Keep the branches
disjoint — never run the crawler's tables probe on a miner run, or vice versa.
The load-bearing gate-path issue is credential sourcing per entrypoint
(bucket 9). If an entrypoint's guid arrives only on the reused connection, lift
it onto the top-level credential_guid at input construction so the gate —
which runs before the body — resolves it:
class MyMinerInput(QueryExtractionInput):
@model_validator(mode="after")
def _lift_credential_guid_from_connection(self):
# gate reads the top-level triple before any body code runs, so the
# connection's guid must be lifted here, not in run()
if not self.credential_guid and self.connection is not None:
derived = (
getattr(self.connection.attributes, "default_credential_guid", "")
or getattr(self.connection.attributes, "defaultCredentialGuid", "")
)
if derived:
self.credential_guid = derived
return self
Precedence — get this exactly, especially for SDR apps:
- an explicit top-level
credential_guid/credential_refalways wins — never overwrite it; agent_json(SDR) wins over any guid sitting on the connection;- fall back to
connection.attributes.defaultCredentialGuidonly when the whole triple is empty.
Then delete the old body-side derivation (coverage diff, corollary 3) so there
is one source of truth. Temporal deserializes the workflow input through
pydantic before dispatching the gate, so a mode="after" validator has already
run — verify it (don't assume) by feeding a connection-only input through
PreflightGateInput.from_extraction_input(input, "<entrypoint>") and asserting
.credential_guid is populated.
This is an app-side stopgap. The connection-carries-its-default-guid pattern is platform-wide, so the durable fix is the gate resolving it directly (CNCT-92); when that lands, delete the validator.
Also normalize extraction_method if it isn't router-routable (phase-0
bucket 10). Lifting the guid is not enough if the router can't route it: the
gate calls CredentialRef.resolve, which only accepts direct / agent. A
manifest that sets extraction-method to an extraction kind (e.g.
query_history) makes resolve() raise, and it limps along only on the
deprecated legacy_credential_ref fallback. Map it to the routing selector at
construction, mirroring resolve()'s own precedence (agent creds win over a
guid):
@model_validator(mode="after")
def _route_extraction_method_for_gate(self):
if self.extraction_method not in ("direct", "agent"):
agent = self.agent_json
if agent is not None and agent.is_populated():
self.extraction_method = "agent"
elif self.credential_guid:
self.extraction_method = "direct"
return self
Leave it untouched when there's nothing to route (no guid, no agent) — let the
gate decide on an empty credential rather than inventing a route. When it bites
if skipped: works today via the caught fallback, but breaks at SDK v4.0 (fallback
removed) or the hard-mode flip (every run aborts). Nothing reads the extraction
kind value for behaviour, so remapping is safe — verify with a grep. Same
durable SDK fix: resolve() should route "non-agent + guid → GUID" itself
(CNCT-92), after which this validator is deleted too.
Phase 3 — Tests (mandatory, thorough)
Update or write unit tests so every verdict path is pinned. Minimum matrix:
- All checks pass →
READY, full check list, durations set. - Each required check failing →
NOT_READY, short-circuit proven (later checks absent from the list),errorpresent with the agreed category/audience andsuggested_action. - Advisory-only failure →
PARTIAL, run-proceeding semantics documented in the test name, failed advisory check present withpassed=False. - No
blocking=/category=/suggested_action=kwargs anywhere (grep-clean — those fields no longer exist onPreflightCheck). - Gate-path input shape:
preflight_checkcalled with aPreflightInputbuilt only from contract fields + credentials (no form-only keys) behaves correctly — this is the regression test for the phase-0 silent-drift audit.
Then the full app suite green, pre-commit clean, and one final worker boot. Do not claim done with anything less.
For a multi-entrypoint or late-credential-derivation app, add a gate-path check
the unit suite can't give you: start the real workflow (uv run main.py,
provision a guid via /dev/local-vault, then /start?entrypoint=<ep> with the
guid only on connection.attributes.defaultCredentialGuid) and read the
gate verdict from the "Preflight gate outcome" event. The console drops the
event's structured extras, so read the local observability logs
(local/dapr/objectstore/artifacts/apps/observability/non-sdr/logs/**.json.gz,
gzipped JSONL) and confirm outcome="proceeded" with the entrypoint's checks in
check_matrix. Read gate_classification to tell the two non-verdicts apart:
gate_broken on a no_verdict row is the gate's own plumbing (e.g. flaky
embedded Dapr locally) — not a real verdict, re-run. source_unverifiable on a
would_block/blocked row means the handler crashed or overran its budget, which
is about your checks and will abort the run once the app is on hard mode. An
outcome="skipped" row means the gate never ran at all — check the reason
(input_not_credential_resolvable is a contract problem, see step 4).
Pitfalls (each observed live during the rollout — check all of them)
all(c.passed)status logic → advisory checks silently promoted to run-blocking.- Raise-to-block → the verdict belongs in the returned status; a raise is classified by error type instead, so it either fails open (typed plumbing error) or blocks with the wrong attribution (anything else).
- Returning
NOT_READYfor a transient (429, dependency outage) → makes a hard gate fail closed on a blip. Raise a typed plumbing error instead. - Sizing checks to a budget the handler can't meet, or reading
input.timeout_secondsand then overriding it (max(input.timeout_seconds, <bigger constant>), or a deadline whose per-probe floor never forces an early return) → the overrun blocks every run in hard mode. erroron a passed check → ignored by the gate; remove it.- Form-only metadata keys → silently absent on the gate path; defensive
.gethides it (checks pass with wrong config), hard access crashes — which blocks every run in hard mode. Fix the contract, not the symptom. - Deleting a colliding activity without the coverage diff → checks vanish.
- Wrong audience on an internal failure → customer's SLA split absorbs an app bug.
- Untyped extraction failure for a cause the gate already reports → in soft mode
the run proceeds, dies later on that same cause, and defaults to
APP_OWNER; the app's failure rate absorbs a source problem it correctly diagnosed (2f-bis). - Typed cause swallowed by a retry loop that finally raises something generic → the last error standing is what AE attributes, so the typing is wasted.
- Typed classes spread across modules → one root cause aggregates under two
codes depending on which surface raised it. One
app/failures.py. - Skipping the boot check before pushing a bump → collision discovered as a prod crash-loop instead of locally.
- Multi-credential app hand-rolling credential resolution + fail-open taxonomy
in the handler → dangerous duplication (a store outage misread as a bad
credential blocks healthy runs). Use
preflight_credential_refs+credentials_by_name(2g); declare it as aClassVar— a pydantic field silently no-ops back to the single-triple path. - Credential derived from the connection in the entrypoint body → the gate sees
an empty triple and soft-fails
would_blockwhile extraction succeeds. Lift the guid at input construction, not inrun()(phase-0 bucket 9, 2h). - Trusting a green UI Test-Connection as gate readiness → the UI gets creds
inline; the gate resolves them from the triple, so they can disagree. Read the
"Preflight gate outcome" event (
outcome/reason/check_matrix), not the UI, to know what the gate actually did. - Sizing a budget from
check_matrixduration_ms→ those are handler-authored, and on pre-3.25 rows they come from abandoned attempts that kept running after the workflow gave up, so they read far above any budget that was in force. Usegate_duration_ms. - Sizing a budget from the worst observed run → the timeout becomes decorative
(nothing ever overruns, so hard mode is back to enforcing only
NOT_READY). Size from the p99 of successful runs; p95 blocks 5% of them. - Reading a
blockedrow on pre-3.25 data as proof the run was blocked → an orphaned attempt writes that row minutes after the workflow already failed open. The signature isblockedandno_verdicton the sameworkflow_run_id, both at attempt 1. - Raising
preflight_gate_timeout_secondstoward the ceiling while leavingpreflight_gate_max_attemptsat 2 → at 300s that reserves a ~10 minuteschedule_to_close, and a retry cannot rescue a systematically slow check anyway. Pair a large budget with1. - Mis-set
entrypointdefault inpreflight_check→ multi-entrypoint runs routed down the wrong branch (a miner silently running the crawler's checks). extraction_methodnames an extraction kind (e.g.query_history), not a routing selector →CredentialRef.resolveraises and the gate survives only on the deprecatedlegacy_credential_reffallback (removed in v4.0) / hard- aborts under hard mode. Reads green today because the error is caught — a false all-clear. Normalize todirect/agentat input construction (phase-0 bucket 10, 2h).
Agent protocol — stop points and what to report at each
The skill is a conversation with checkpoints, not a batch job. Three hard stops:
- After phase 0 — report the bucket(s), the consolidation candidates (with your three-way classification and reasoning), the eligibility verdict, and the silent-drift key list. No edits yet. The developer may already know some candidates are intentional; let them say so here.
- After 2c/2a-bis — the agreed target tree and consolidation decisions, restated as the plan of record ("these checks, this order, these block, these are advisory, these activities fold in, these stay"). Get an explicit yes before writing code. This restatement goes verbatim into the PR description later.
- After phase 3 — the full verify evidence: test matrix results, suite + pre-commit output, worker boot confirmation. Then hand off for PR review.
Between stops, work autonomously. If anything contradicts these instructions (an SDK surface that moved, a pattern that doesn't fit the buckets), STOP and report rather than improvising — the buckets came from a fleet audit, but the fleet has ~80 apps and this skill has met seven of them.
Done means
Bucket reported → bumped and booting → check tree agreed with the developer
and implemented → typed errors on failed checks, all defined in one module
(app/failures.py) → each blocking check's extraction-side counterpart raises
the same leaf, so a soft-mode proceed cannot file a source failure against the
app → budget sized to what the handler actually costs → test matrix green →
suite + pre-commit green. Summarize the final tree in the PR description so
reviewers see the blocking structure at a glance.