onesie-stream

Imported from frodi-karlsson/onesie (.agents/skills/onesie-stream/SKILL.md). Install upstream with npx skills add frodi-karlsson/onesie --skill onesie-stream. Copyright stays with the author (MIT).

onesie streams over many records with -i jsonl, -i lines, -i csv, -i tsv or -i request, one record per line or row in and one answer per line out.

A failed record usually does not stop the run. It prints an error record in its place and the run keeps going, so a consumer has to check which kind of line it is looking at before it reads an answer out of it. An auth failure and --stop-on-error do stop it, and the next rules list what else cuts the output short. See the onesie skill's failures reference for the exit code table.

onesie --ask urgent='is this urgent' -i jsonl -j 8 < tickets.jsonl | jq -c 'select(.error == null)'

For a flag this skill does not cover, run onesie --help or onesie calibrate --help, and onesie -V for the version and the built in limits.

Rules

Read the error before you read any answer.

A failed record still prints a line, carrying an error rather than answers. A missing field reads as null in jq, and null sorts below every number, so a threshold check with < silently admits a failed record as if it had passed. The error sits at .error, but under --merge it sits at .answers.error, or under the --merge-key name, since the input's own fields stay at the top. In -o csv and -o tsv it is the error column, empty on a row that answered.

Bad:

onesie --ask urgent='is this urgent' -i jsonl < in.jsonl | jq -c 'select(.urgent.value < 0.8)'

Good:

onesie --ask urgent='is this urgent' -i jsonl < in.jsonl | jq -c 'select(.error == null and .urgent.value < 0.8)'

Match output line N to input line N, unless the exit code says the run stopped early.

Output keeps input order under any -j, so line N answers record N. A failed record still gets its line, and so does a blank line in jsonl or lines unless --skip-blank drops it. A blank line in csv or tsv is skipped with no output line. Some stops leave only a prefix of the input: an auth failure, exit 3, --stop-on-error with the failing record's code, --stop-on-assert, exit 1, or 6 when an earlier record failed, an interrupt, exit 130, a csv row over the length limit or an input column that clashes with an output column, exit 2, and a consumer that closes the pipe, such as head, which exits 0. A csv or tsv header that is not valid exits 2 before any line. --unordered drops the ordering for throughput.

Under --merge read answers from .answers, and rename it with --merge-key if the record has one.

Answers sit at the top level keyed by question id. --merge folds them under an answers key instead, and --merge-key renames that key, which matters when the record already has one called answers. Folding only means something in a structured output, so -o table, -o raw and -o markdown are rejected under --merge.

On live input, filter each line on its answer, since a stream's exit code arrives only at the end.

A stream exits once its input ends, with 1 when any assertion was false, and tail -f never ends. So nothing chained after it with && or || runs, and -q is refused on a stream with exit 2 anyway. Act on each line as it arrives instead: under --assert every record carries its gate result at .assert, true or false, and under --merge at .answers.assert. Pass jq --unbuffered so each match leaves jq at once rather than when a buffer fills. A failed record carries .answers.error under --merge and no assert, so a filter on .answers.assert == false alone skips it. Route it too, as the good example does, since a record with no answer is not an all clear.

Bad:

tail -f app.log | onesie 'does this line report a failure a person must act on' -i lines --assert 'answer.value < 0.8' -q && page_on_call

Good:

tail -f app.log | onesie 'does this line report a failure a person must act on' \
    -i lines --merge -o json --assert 'answer.value < 0.8' \
  | jq -c --unbuffered 'select(.answers.assert == false or .answers.error != null)'

Freeze a run with --print-request and replay it with -i request.

--print-request over a stream writes one body per input line, and -i request reads those bodies back. It checks that each line is one JSON object and still applies the flag rules, but nothing inside the body: no question checks, no normalization and no policy. So a bad body is sent as is and comes back as an error record, not an exit 2, and a good one comes back as the raw response body, not the usual record. Under -i request, --print-request prints its input unchanged, so adding it to a pipeline turns the whole run into a dry run: onesie --ask urgent='is this urgent' -i jsonl --print-request < in.jsonl > bodies.jsonl && onesie -i request < bodies.jsonl.

Good:

onesie --ask urgent='is this urgent' -i jsonl --print-request

Read csv or tsv with -i and write it back with -o csv --merge.

-i csv and -i tsv read a header row and send each row as an object keyed by it, so a question can name a column such as body. A byte order mark before the header is dropped, and a blank or repeated column name exits 2. A csv field may be quoted to hold commas and newlines. TSV has no quoting: a quote is ordinary text and every non blank line is a row. -o csv and -o tsv write one header row: the input columns under --merge, an id column under --id, one column per question holding its -o values answer, an assert column holding true, false or abstain when there is an assertion, empty on a failed row, and an error column. In -o tsv a tab or line break inside a cell becomes a space. Under --merge, an input column named like a question id or error, or assert under a gate, exits 2 at the first row. An input id column never clashes. --merge into -o csv or -o tsv needs csv or tsv input, since jsonl has no fixed columns, and both refuse --usage, and --merge-key even beside --merge.

Write -o markdown for a PR comment or a job summary, and never as a file to resume or parse.

-o markdown, or -o md, writes a stream as one GitHub table: a row per record as it finishes, in input order unless --unordered, an id column under --id, a column per question, a gate column holding passed, failed or unsure under an assertion, and an error column. After the last row comes an alert that counts the records, a [!TIP] when every one passed or answered, a [!WARNING] when the worst is an unsure and a [!CAUTION] when any failed or has no answer, then the models and, under --usage, the summed tokens and the cost when the provider reports one. A stream with no records still writes the alert, 0 records., so a comment is never empty. A run that ends early, by an interrupt, an abort such as a refused key, --stop-on-error or --stop-on-assert, keeps the rows that finished, and its alert reads stopped after 2 records: 1 passed, 1 failed. instead. Ids, option names, question ids, models and error messages sit in code spans, so an id such as @someone pings no one and #12 links nowhere. A table cannot be read back into answers, so --resume, --merge, -r and -q exit 2, while --out alone writes the table to a file. Read answers with -o json or -o csv, and keep markdown for the people reading the result. A GitHub comment holds at most 65536 characters, so send a large stream to $GITHUB_STEP_SUMMARY or write it as csv.

Bad:

onesie 'is this urgent' -i jsonl --id '.id' -o markdown --out answers.md --resume

Good:

onesie 'is this urgent' -i jsonl --id '.id' -o markdown

Choose the state with --map and name each record with --id.

--map EXPR runs a jq expression on each record and sends its result as the state, so --map '.body' keeps a customer name or an internal id away from the model, and --map '{subject, body}' or --map '.subject + "\n\n" + .body' builds a smaller one. It works on every input but -i request, a csv or tsv row is the object keyed by its header, and --merge still folds the answers into the whole record. An object the expression builds reaches the model with its keys sorted, and onesie -V lists the depth and size caps on the result. A syntax error exits 2 before any request. A record where the expression fails, yields nothing, yields more than one value, or yields null, a number or a boolean gets an error line, and the stream carries on, since a state is a string, an object or an array. An empty or all whitespace string, an empty object or an empty array is refused as an error line too, which --skip-blank does not drop, with a message such as --map: empty string, an empty state is a request the model cannot answer, and for one record it exits 2 before any request. A jsonl line holding one of them is an error line too. --id EXPR takes streaming input only and must yield one string or one number. It runs one record at a time as the input is read, so keep it cheap, such as a field lookup. Ids match by their text, so 7, 7.0 and the string "7" are one id in jsonl, while in csv and tsv every cell is text and 7 and 7.0 stay two. The empty string is an id like any other. A missing id, one of another type, one longer than the cap onesie -V lists, or a repeat of an earlier record's id is an error line, and so is an id -o tsv or -o csv could not write back, such as a tab in tsv. Every other output line carries its id, an "id" key in json and values and an id first column in csv and tsv. Under --merge the record already holds its fields, so nothing is added. The question ids id, answer, answers, error, model, usage, state, questions, assert, abstain, abstain_if and any id starting with __ are reserved.

Bad:

onesie 'is this urgent' -i jsonl

Good:

onesie 'is this urgent' -i jsonl --map '.body' --id '.id'

Write a long stream with --out and rerun it with --resume and --id after a failure.

--out FILE writes the answers to a file instead of stdout, with a fingerprint in FILE.onesie beside it. The fingerprint covers the questions, the provider, the model, --map, --id, the input and output modes, the merge key, the policy flags, --assert and --abstain-if, and --skip-blank when a resume counts lines, which is without --id on any input but csv and tsv. So -j, --timeout and --retries can change between runs, and --skip-blank can change with --id or on csv or tsv input. --resume exits 2 when any of those changed, saying the questions, flags or gate changed, or when a non empty file has no fingerprint or one from another onesie version, and says to drop --resume to start over. With --id a resume skips every record whose id the file already answers, with no request, and asks the rest, including a record whose last line was an error. Each answer is appended as it arrives, so an interrupted run loses nothing. Only a run with both --resume and --id rewrites the file once it completes, in input order with the newest line per id, and keeps the answered ids the input no longer holds after the rest, so a cut short input never deletes an answer. --prune drops those instead. --resume on a missing or empty file starts fresh, so pass it on the first run too. Until the rewrite the file can hold a record twice and out of order. Only the id is compared, so a record whose content changed but whose id stayed the same keeps its old answer. A skipped record keeps its stored assertion outcome, so it still counts toward exit 1 or 7 and toward --stats, and under --stop-on-assert a skipped false assertion stops the run with nothing after it asked. Under --stop-on-error, a resume without --id drops a trailing error line and asks its record again, since a run stopped by --stop-on-error writes nothing after it, so a rerun makes progress once the cause is gone. An error line with more lines after it came from a run without --stop-on-error, and the resume stops there before any request, with the code that run exited with, or with exit 2 for a csv or tsv row, which keeps only the failure's message. Pass --id or drop --stop-on-error to carry on. With --id the failed record is asked again and stops the run only if it fails again. --unordered is allowed since the rewrite restores the order. Raw output is refused in every resume, with --resume needs output that keeps each record's outcome, which raw lines do not, since a raw line keeps no id, failure or gate outcome. Every --out run into a regular file holds an OS lock on FILE.onesie.lock for the whole run, so a second run into the same file exits 2 until the first ends. Through a symlink, the fingerprint, the lock and the rewrite sit beside the link's target. A device or a pipe, such as /dev/stdout, gets no fingerprint and no lock, and --resume refuses one. A file or a directory onesie cannot write exits 2 before any request. Without --id, --resume counts the complete lines instead, drops a line cut off mid write, skips that many input records, retries no failed one but a trailing error line under --stop-on-error, and refuses --unordered. A stored error line it skips still counts as a failed record, so the run exits 6 and --stats counts it as failed, under -i request too. A full resume by id: onesie 'is this urgent' -i jsonl -j 8 --map '.body' --id '.id' --out answers.jsonl --resume < tickets.jsonl.

Dedup shares a request within one run, --cache shares it across runs, and --resume skips the ids an --out file already answers.

Within one stream, records whose request is identical, the same state after --map, the same questions and the same model, are asked once, and each still gets its own line. --no-dedup asks every record, and --stats counts the records deduplicated. That lasts one run. --cache, or ONESIE_CACHE=1, keeps each successful response on disk, so a later run asks nothing for a request it has seen, whatever the record's id. --resume with --id goes by id, not by request: it skips every record whose id the --out file already answers, even one whose content changed, and asks the rest. So finish a run that stopped with --resume, and rerun records you edited with --cache, which asks only the ones that changed.

Bad:

onesie 'is this urgent' -i jsonl --map '.body' --id '.id' --out answers.jsonl --resume

Good:

onesie 'is this urgent' -i jsonl --map '.body' --id '.id' --cache -o json