Imported from plow-pbc/latch (
.claude/skills/latch-smoke/SKILL.md). Install upstream withnpx skills add plow-pbc/latch --skill latch-smoke. Copyright stays with the author.
Latch smoke — one real call, verified in the audit log
The unit suite never opens a socket to a real relay (README-ts.md
§ Integration coverage). This is the leg that does: an MCP client → the Plow
relay → the device WebSocket → @domo/mcp-server → the executor.
The audit log is the proof, not the client's reply. A reply can come from a
deferred handle or a half-working path; an exec_end carrying this call's
intent id cannot.
One command does it: scripts/latch-smoke. It sends and then verifies in the
same process, so there is no state to copy between steps. Its outcome table
lives in a pure verdict() and is covered by e2e/latchSmoke.test.ts — read
that file to see exactly which records produce which exit code.
Before you start: the credential
You need an MCP client registration for the target install — endpoint plus
bearer token. It is minted once, in the app's Agents tab on the target Mac
(a GUI step), and then recorded at a known path so no later run needs the
GUI: save the block _mcpConfig renders, verbatim, to
~/.latch/<client>.json — 0600, directory 0700, one file per agent credential
(docs/AUTONOMOUS-OPERATION.md Stop 3 owns this convention). The block:
{"mcpServers":{"plow-mbp":{"type":"http","url":"<mcpUrl>","headers":{"Authorization":"Bearer <token>"}}}}Pass it as --config ~/.latch/<client>.json --server <name>, which selects one
device when the config contains several and supplies both URL and token. The
--server flag is optional for a one-server config. --url + --token-file
(token alone, one line, 0600) remains for a
credential that is not recorded. Treat the file the way this repo treats every
credential: never echo it, never put it in a log line or a commit, reference
the token by its last 3 characters only. The script prints an HTTP status on
a refusal and never the response body, because an authenticated response can
repeat the credential back.
Run it
scripts/latch-smoke --config ~/.latch/<client>.json --server plow-mbp \
--home "~/Library/Application Support/Plow-Latch"-
--home <dir>— required. The instance home to read. There is no default: every wrong home this script has produced came from it choosing one, and a chosen home that is wrong reads as a fresh install rather than as an error.--home "~/Library/Application Support/Plow-Latch" # a packaged install --home "$(just --evaluate apphome)" # from sourceEvaluate
apphomefrom the same checkout and the same shell environmentjust appsaw —branchcomes fromscripts/worktree-name.sh, so it follows the checkout, and the-localsuffix followsDOMO_API_BASE_URL. From a different worktree, or with that variable unset when the app had it, the answer is a different install. If in doubt, use theDOMO_HOME=valuejust appechoes in its recipe line.Do not build that path by hand.
justfile:28picks between three homes and the one people forget is the local-relayPlow-Latch-<branch>-local, which exists so a credential minted against one relay never lands in the other's home.A wrong home is the usual cause of a
TIMEOUTthat found nothing, which is why the run printshome=and refuses one that does not exist — on either side of--ssh. -
--ssh <user@host>— read that Mac's audit log over ssh instead of this one's. Host map: thetailscale-sshskill. The call itself always goes over the relay, so only the log read is remote. -
--timeout <seconds>— default 120, and must be more than 2: the log read always consumes some of it, so at or below that a call has no time to answer and the script says so rather than sending. Keep it near the default against an adversarial-mode target: a review is allowed up to 90s (REVIEWER_TIMEOUT_MS), so a short window reports TIMEOUT on a decision that was still coming. One deadline covers the log read, the send and every poll, rather than starting after the send: a relay that accepts and never answers costs roughly what you asked for instead of the 90 seconds a hard-coded socket timeout used to spend. It is a bound on each blocking wait, not a wall-clock guarantee —urlopen's timeout is per socket operation, so name resolution is outside it and a trickled response body resets it — and every log read is allowed 5s even past the deadline, so a timed-out run reports its real verdict rather than an unreadable log. -
Everything after
--replaces the command. The default is a fixed/bin/echo latch-smoke, and that is deliberate: the run's nonce ridesgoal, whichRuleKey.computeexcludes from the rule key, so one pre-seeded Always Allow rule matches every run. A nonce in the argv would mint a new key each time and raise a dialog forever.
What actually answers depends on the target's approval mode. The default is adversarial: the AI reviewer decides without a dialog — unattended, but not deterministic, and stored Always Allow rules are deliberately vetoed in that mode. A pre-seeded rule makes the smoke deterministic only under Ask or Approve mode; Ask without one raises a dialog that an unattended run times out at. That is the product working, not a failure — say which mode the target is in rather than waiting on a dialog nobody is watching.
Reading the result
Only success exits 0.
| Output | Exit | Means |
|---|---|---|
OK + exec_start/exec_end |
0 | it worked — quote both lines as the verification |
FAILED — it ran and … |
1 | it executed and exited nonzero, or this Mac reaped it. Not a plumbing fault |
FAILED — the executor threw |
1 | exec_error names why; nothing ran |
DENIED |
1 | the owner refused it. The relay and the device both worked |
REFUSED — HTTP 3xx |
1 | the relay tried to redirect and this follows none — urllib carries the bearer across a redirect, so following one would hand it to wherever it pointed. Check --url |
REFUSED — HTTP 4xx |
1 | refused before an intent existed, so there is no audit line. 401/403 is the relay or the credential; another 4xx is the MCP handler |
UNVERIFIED — … |
— | not an outcome. The send did not settle the question: anything that is not a response (a timeout, a dropped socket), a 5xx, or an isError — which is also how an ordinary denial comes back. So the script does not stop; it polls (up to 20s to see it arrive, then the rest of the window), and one of the rows above is still the answer |
TIMEOUT — … no decision was recorded |
1 | it arrived and nothing decided it in the window: an adversarial review still running, or an Ask dialog sitting unanswered. Not a plumbing problem — raise --timeout first |
TIMEOUT — approved, never started |
1 | exec_start is written before the spawn, so its absence means the executor was never reached. Check the app is running |
TIMEOUT — started, still running |
1 | re-run, or raise --timeout |
TIMEOUT — nothing carrying … |
1 | it never arrived; the output names the three causes |
TIMEOUT — the audit log stopped being readable |
1 | the call WAS sent — re-read the log for the nonce once the host is reachable |
REFUSED — the relay answered with a JSON-RPC error |
1 | refused before it became an intent; nothing was written to the log. A bad envelope or an unknown tool. The relay's own text is deliberately not quoted — that response is authenticated and can reflect the credential back — so read the relay's logs for the detail |
REFUSED — could not reach <url> |
1 | the request never left this Mac — a URL with no scheme is rejected before any socket exists. Check --url; the message quotes what was wrong with it |
REFUSED — <file> has a line break inside the token |
1 | nothing was sent. A token pasted across two lines keeps its newline, and the header it would build is refused — rewrite the file as a single line |
REFUSED — cannot read the audit log |
1 | nothing was sent, deliberately: a call this cannot verify would still raise a dialog. Check --ssh, and that --home is readable — a missing log is not this |
Smoke-testing the gog provider specifically
gog is a bundled plugin on main (plow-pbc/latch#183), driven through
plow-gog — a bare gog argv is refused with a sentence naming it. A build
has the plugin staged; a from-source checkout needs just stage-plugins gog
first. The stored Plow credential must authorize gmail:access-token.
Same command, its own argv:
scripts/latch-smoke --config ~/.latch/<client>.json --server plow-mbp \
--home "~/Library/Application Support/Plow-Latch" -- plow-gog gmail search newer_than:1d --jsonThe example above is a fan-out. Smoke output omits its account-level degraded
reasons; inspect the normal tool response for those details.
| Output | Means |
|---|---|
FAILED — the executor threw … not installed |
the gog plugin is not staged — run just stage-plugins gog and repackage |
FAILED — the executor threw … could not reach Plow / returned 4xx |
the mint failed; check that the owner's stored Plow login session is still live. Session authority does not grant Google access |
FAILED — it ran and exited 1 on a fan-out |
no account answered; this includes every account lacking read access. Inspect degraded in the normal tool response |
FAILED — it ran and exited 6 on a selected-account command |
gog denied access. Inspect the normal tool response for Google's error; this can be a resource permission denial, so do not infer a missing write grant |
OK |
an eligible account answered; smoke output does not show skipped accounts |
A 403 inside gog's own output comes from Google. Tokens carry only the
owner's grants: Gmail and Calendar read/write access differ per account.
Failure triage
REFUSED — HTTP 401/403→ the credential is wrong or the device was re-paired. Scopes freeze at mint; a Mac paired before a scope grant needs to re-activate.UNVERIFIED — …→ not a result. Read the row the script lands on afterwards, not this line — the table above says which outputs those are and what waiting it does. The only thing worth adding here: this is the one prefix that is not a verdict, so treating it as one is the mistake to avoid.TIMEOUT — nothing carrying …→ the three causes the output names, in likelihood order: a different install's log (check--home; a branch-suffixed home is the usual cause), a refusal in the MCP layer before an intent existed, or avalidaterejection — wrong device, expired, replayed nonce — which writesintent_rejectedwithout the nonce, so the script quotes any written since the send.- Approval dialog never answered → expected on an unattended run; see above.
