dsh-autoresearch

Imported from EveGoodEvening/dsh-autoresearch (AGENTS.md). Install upstream with npx skills add EveGoodEvening/dsh-autoresearch. Copyright stays with the author.

AGENTS.md

Lessons

• DeepSeek Harness out-of-tree features ship as opt-in bundles: package.json declares dsh.bundle.patch, and the stable patch inserts an ordinary Cordis plugin row whose config is replaced whole, not deep-merged.
• Install out-of-tree bundles with dsh plugin --profile <name> add <package-spec>: the command forwards to pnpm inside the profile and reconciles dsh.profile.bundles. A bare package name targets the configured registry, so unpublished bundles need a local directory or packed tarball spec; plain pnpm add does not activate the bundle.
• Autoresearch runtime authority belongs to AutoresearchRunController; compose the existing agents, jobs, subprocess, systemPrompt, and tools services and generic dsh-tool-jobs controls instead of adding a workflow engine or subagent service.
• The Web profile moves model-facing job_* controls into per-agent presets and disables the host tool-jobs row; a host plugin must not require ctx.tools.get('job_list' | 'job_output' | 'job_kill') during apply(). Let owner-relative ctx.jobs.start() enforce controller availability or validate against the calling Agent at execution time.
• DSH prerelease packages publish as one synchronized family, but pnpm peer auto-install can select stale dist-tag versions for Service Definition packages. Pin the full direct development peer closure to one exact DSH release and require pnpm peers check to pass with no mixed release family.
• Karpathy-style autoresearch depends on a narrow mutable surface, immutable shell-free evaluator argv and provenance, one scalar metric, baseline-first execution, strict keep/reject decisions, and durable SQLite evidence.
• Public-facing descriptions identify dsh-autoresearch as an independently maintained, unofficial third-party DeepSeek Harness plugin. Keep the README and package description consistent, preserve Karpathy inspiration attribution, and do not imply affiliation, endorsement, or sponsorship by DeepSeek or Andrej Karpathy.
• Release verification must exercise the packed artifact outside the checkout: inspect the allowlist, install without local links, import generated ESM/declarations, install/dump the real named dsh profile, boot the actual Web profile, and fetch its HTML surface.
• Resume identity is the canonical repository plus durable run state: canonicalize the repository root in the policy hash, exclude foreground/background dispatch mode, rehydrate start_commit from SQLite, and never bind recovery to the original caller subdirectory.
• Evaluator recovery treats the classified attempt outcome as authority: parse live unredacted stdout, atomically persist the metric or typed failure with exit facts and artifacts, and use redacted logs only as integrity evidence; truncated stderr alone does not invalidate a measured result.
• Retention is lazy and repository-local: controller startup sweeps unowned, quiescent terminal runs, while current-run cleanup happens only after terminal lock release. Artifact pruning keeps SQLite identity/hash metadata with retention = 'pruned'; retainWorktrees: false removes all safe terminal worktrees unless cleanupWorktreesOnSuccess narrows cleanup to target-reached and budget-limited.
• maxResultChars is presentation-only: canonical run result decoding validates structure independently of size, while render and job adapters bound their own output.
• Model activation no longer accepts evaluator argv, metric, or environment authority: new runs select a Host-owned evaluatorRegistrations entry, and resume revalidates its durable fingerprint before mutation or spawn.
• Evaluator cwd uses normalized repository-relative paths: explicit . is invalid, while omitting cwd selects the isolated evaluation worktree root. Copyable root-level registration examples must omit the field and prove Config/real Loader plus installed evaluator execution, not weaken path normalization or manufacture a dot default.
• Controller provenance now carries registered evaluator files and dataset metadata through initialization and every attempt; local declared bytes are manifest-checked, while external dataset digests and executable/provider identity remain trusted Host assertions rather than independently verified runtime facts.
• Declared Git manifest path comparison must use the same deterministic UTF-16 ordering as normalized declarations and canonical fingerprints; localeCompare can falsely reject tracked mixed-case/non-ASCII blobs. Prove exact immutable-start inventory and within-fixture manifest/hash equality across reordered declarations, without renaming paths or rewriting durable hashes.
• Each proposal child is fresh, but bounded durable research memory now retains its untrusted hypothesis/intended edits/summary plus Host-derived commits, changed paths, metrics, decisions, and failure facts. Proven-quiescent evaluator failures consume an ordinal and continue; proposal/report/path-validation failures still terminate the run as round-failed.
• Real DSH filesystem observation policy requires reading an existing file before overwriting it; deterministic child providers must perform read → write → report and verify actual edits rather than treating a denied tool call as a candidate.
• Nonterminal resume retains the common immutable activation policy fields (objective, mutable globs, limits and optional constraints/target); replace only the new-start selectors run_tag/evaluator_id with resume_run_id. A tracker ID alone is not a valid resume request.
• Git control-plane budgets are independent Host settings: gitTimeoutMs defaults to 900000, and gitMaxStdoutBytes/gitMaxStderrBytes each default to 1048576; timeout_ms, maxStdoutBytes, and maxStderrBytes govern evaluator attempts only. Production preflight and runtime/recovery Git commands retain the Host Git options, so small evaluator budgets do not constrain repository discovery, allocation, commits, or reconciliation.
• npm Trusted Publishing can publish this pnpm-managed package through GitHub Actions without a long-lived npm token: use a GitHub-hosted runner, id-token: write, Node >=22.14.0 (also satisfy this package's stricter engine range), and npm >=11.5.1 for the publish step. Match the npm trusted publisher to the repository and workflow filename exactly; new configurations default to staged publishing, so explicitly allow npm publish for direct releases. Source: https://docs.npmjs.com/trusted-publishers
• Clean release CI must build before running coverage tests: composition and release tests import the package's generated lib/ exports. Local check, compatibility and publish gates now run typecheck → build → test:coverage, with coverage supplying the single full suite. Coordinator clean-lib/check proof confirms this order works from a clean-generated tree; do not restore test-before-build ordering or add a duplicate full suite.
• npm 11.16.0 checks registry version availability even for npm publish --dry-run: an already-published package version is rejected without uploading anything. A publish dry run on the unchanged 0.1.6 checkout therefore cannot demonstrate a successful new release; keep version bumps explicit rather than changing package identity just to make verification pass.
• The publish workflow does not need broad default GITHUB_TOKEN write permissions: keep the repository default read-only and grant contents: read plus id-token: write in the job. An Actions allowlist must permit the workflow's GitHub and pnpm actions; a full-SHA-only policy requires replacing its @v6 references with commit SHAs. GitHub-controlled repository/account-disabled banners require GitHub Support rather than broader workflow permissions. Source: https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-github-actions-settings-for-a-repository
• Keep existing release tags immutable when retrying after Actions settings changes: settings do not replay a past push, and this workflow has no manual dispatch trigger. A new unpublished version and matching tag provide a fresh push event; verify an actual Actions run rather than treating an active workflow registration as proof that release execution began.
• fs.promises.writeFile() exposes a newly created file before all content is written. The holding evaluator fixture must write its PID to a sibling temporary file and rename it into place: a reader that accepts file existence can otherwise parse an empty marker as PID 0, causing the HMR liveness assertion to fail intermittently on CI. A delayed-write smoke reproduces the empty read before the fix and observes the real child PID after atomic publication.
• Never run pnpm pack inside the parallel test suite: the package's prepack hook deletes and rebuilds shared lib/, racing real Loader imports. Keep packed-artifact execution in the sequential release:smoke step; README wording/copy and exact-script assertions are not substitutes for consumer-behavior verification.
• The released dsh-autoresearch@0.2.0 baseline pins DSH 0.1.7-rc.2, but a fresh npm lookup later on 2026-09-29 resolved CLI latest to 0.2.0-rc.2 (previously observed on next). Dist-tags move even within one day: resolve the intended channel at check time and pin the direct DSH family to that CLI version rather than trusting older channel observations or independent service dist-tags.
• The current Web Agent composition puts preset-standard in the Agent preset plane, while agent-preset-registry and subagent-model-selection-settings remain Host rows. Mount the preset through agentPresets; its model-facing Jobs tools do not make job_* available in the Host tool registry.
• Latest Agent child creation uses agents.create({ sessionId: SessionId(...), parentAgent, meta, agentOptions, setup }); inherit the parent's current session.requestHeader() route and commit proposal reports from tools/result, not tool invocation. Jobs require the owner's SessionId; producer done is not registry settlement. Before jobs.start(), subscribe to non-consuming JobEvents in an explicitly jobs-injected owner child scope. Yield the exact unsubscribe disposer first and a registered-job terminal-event barrier second in a Cordis composite effect: reverse-order awaited teardown retains the listener through owner closure, producer HMR, and full Host shutdown. The event proves settlement even if owner cleanup has already dropped the job, so do not query jobs.get() after it; no internal jobs.wait() (marks awaited, suppressing the owner's completion notice) or polling. Real owner and Host shutdown smokes verified terminal event, owner record removal, and evaluator quiescence.
• The public DSH subprocess handle has no portable PID. Persist spawnedAt only after ctx.subprocess.spawn() returns, keep provider PID absent when not supplied, and decode legacy nullable provider_pid separately; neither a fabricated PID nor spawn intent proves a process started.
• Cordis Include/HMR disposal can leave an old fiber draining after its row changes: retain the previous fiber and await its await() before asserting that the replacement is active. Real installed CLI-profile boot keeps its profile-config import origin; do not force a fifth base-package URL into boot(). Resolve plugin imports through native PluginPackages/createRuntimeResolution in a temporary profile without mutating shared dsh-base/node_modules or linking installed peers back to the checkout. A cold standalone import or checkout link does not prove an independently installed bundle.
• Web's official /?token=… bootstrap is an authentication exchange, not the final HTML URL: the clean root is 401 without a session, the token request returns 303 with Set-Cookie and Location: ./, and the cookie authenticates the clean root as 200 HTML. This package's Node range is ^22.19.0 || >=24.2.0; the Node 24 floor for import.meta.main is 24.2.0, not 24.0.0 (https://nodejs.org/api/esm.html#importmetamain).
• The README ships inside the npm tarball: describe the tested version and date-stamped host channel without embedding transient prepublication status (candidate / not yet published) in release-facing copy. Pin the intended install spec so the same artifact remains accurate after publication, even if the registry latest tag changes.
• A successful publish workflow is not itself proof of registry availability, and one early 404 or old dist-tag read is not a final verdict. Verify with a fresh exact-version/dist-tag registry query and a downloaded tarball integrity match; compare the available provenance subject/source/workflow/tag identity without claiming cryptographic signature verification.
• Latest-upstream monitoring is a disposable forward-compatibility probe, not a published-support claim: repin both development and peer dependencies in a copied workspace so packed consumers cannot silently reinstall the old family; retain the original peer declarations and candidate lockfile as evidence. Keep dependency execution read-only and grant issues: write only to a separate reporting job; serialize and deduplicate default-branch incidents, and distinguish infrastructure failures from demonstrated API incompatibility.
• A job_output(wait: true) return is not proof of completion: its finite wait may expire with job.status = 'running' and empty text. A real DSH 0.2.0-rc.2 accepted lifecycle took about 28 seconds across 236 managed subprocess calls, exceeding a 20-second tool wait. Subscribe before start, await the non-consuming settled event, then collect the final result; retain a gated running/empty-output regression.
• Evaluator fixture readiness must race the evaluator outcome, not just poll for file existence: a delayed Node startup can lose to the watchdog and never publish a PID. Arm a watcher before starting the job and observe atomic marker publication; a separate 10–15 second polling limit can expire during legitimate Git preparation before the evaluator starts. Keep the case's existing overall deadline. Use a child readiness acknowledgement for real-tree termination coverage, control only its watchdog clock, and separately verify real-clock timeout before readiness. Assert descendants are quiescent when persisting the outcome, not merely afterward.
• Vitest deep equality over SQLite main/WAL Buffers can dominate snapshot tests: measured backups took 33–90 ms while byte-diff assertions introduced second-scale gaps. Use native Buffer.equals() for exact bytes in SQLite and controller snapshots. A test timeout does not settle its promises: drain owned controllers/subprocesses before restoring prototype spies or deleting roots, and keep finite operation/cleanup limits so the next case cannot inherit live work. Keep live WAL churn and all 20 snapshots; install the worker-stop listener before signalling it.
• Proposal children receive exact intentional-public report identity/protocol, Host metric name/direction/parser and operational workspace/branch/commit identity plus bounded policy/provenance fingerprints, even when bytes overlap configured environment values. Host must keep credentials out of designated public fields: actual credentials there are disclosed. Do not project raw argv/environment or infer universal confidentiality from registration authority/fingerprints. Apply exact known-value redaction only to permitted untrusted/display/history leaves before bounds; never redact or reject whole critical public context for coincidental collisions.
• Persist typed machine evidence and authoritative transition facts by field origin, never through generic known-value string redaction: generated IDs, ISO timestamps, exit/signal/failure-code facts, provenance SHA, already-hashed environment digests, machine snapshots and Git-baseline paths/SHA must remain exact even for environment values 0, 1 or :. Keep raw argv/cwd/error-message text and untrusted annotations/display/history separately scrubbed. Preserve strict conflicts, Git/recovery validators and canonical hash construction; already-corrupted rows must fail validation, never be reverse-redacted or guessed. A managed provider can omit a portable PID; persist truthful absence and prove actual execution/quiescence without inventing provider identity.
• Release observers must use API-specific Accept headers: reusing GitHub's application/vnd.github+json header against npm's latest endpoint produced HTTP 406 after a successful publish. Resume registry verification with application/json; an observer's content-negotiation error is not evidence that publication failed.
• Keep the README concise and user-facing: show supported DSH and scoped Cordis versions near the top, distinguish declared ranges from tested versions, and link to configuration/workflows instead of accumulating implementation, test-debugging, or release-operation notes.
• GitHub commit autolinks can display only seven SHA characters while retaining the full forty-character commit href. Rendered evidence checks must bind the exact approved repository/full-SHA href to its legitimate visible prefix before comparing paragraphs; stripping links and requiring full visible SHA rejects valid comments. Verify publication and downstream comment effects separately: a failed reporting job does not undo a successful immutable npm release.
• Candidate-phase cancellation must use validated recovery/reconciliation with a non-aborted cleanup signal before restoring accepted HEAD or releasing locks. Select terminal cancelled experiments as well as unresolved ones, finish interrupted acceptance transactions against their durable measurement, and retain authority on uncertain process state or unexpected refs. Preserve the canonical controller result in background job output instead of replacing a settled cancellation with background-start-failed. Real candidate-readiness cancellation now proves accepted HEAD restoration, quiescent attempts, lock release, unchanged caller and identical terminal resume; keep this boundary in packed release evidence.
• DSH --dump-config emits native !!js YAML tags. Plain js-yaml with its default schema rejects that valid output; inspect with a Cordis-aware parser or an inert custom scalar-tag schema, never by evaluating configuration expressions merely to inspect plugin registration.
• In Cordis 4.0.4, calling an effect disposer again after disposal has started returns undefined, not its in-flight completion promise. Proposal cleanup must join its memoized AgentHandle.dispose() independently of releaseOwner() before checking registry/job quiescence, and owner cancellation must remain distinguishable from a missing report even before the request signal aborts. Gated concurrent-disposal regressions preserve genuine teardown failures; real held-child HMR now settles killed/cancelled, releases authority, leaves no child/jobs and restores exactly one tool/prompt registration.