grading

Imported from chaz-clark/canvas-toolbox (.claude/skills/grading/SKILL.md). Install upstream with npx skills add chaz-clark/canvas-toolbox --skill grading. Copyright stays with the author.

Grading

Grading with AI is decision support, not autonomy. A human decides every grade that reaches a student. This skill is the operating manual for that discipline.

Fresh data before any grading decision

A grading decision is only as good as the data under it. A stale cache — a _computed_grades.csv from an earlier sync, a day-old export — has produced confidently-wrong conclusions ("KC3 blocker" when 19/31 had actually completed it). So: pull fresh Canvas data before you reason about grades, and never trust a cache without checking its age.

The toolkit already builds this in — do NOT hand-roll a verify_data_freshness.sh or a timestamped CSV:

• grader_fetch_gradebook.py mirrors the whole gradebook, stamps it with fetched_at, and skips the fetch only if the cache is younger than --max-age-hours (default 6) — --force always refreshes. Use it (or its cache) as the source of truth for gradebook-wide reasoning.
• grader_fetch.py re-fetches a challenge's submissions; when in doubt, re-run it rather than reusing an old fetch.

If a course keeps a computed file (final grades, standings), regenerate it from a fresh fetch each run — never read a cached custom CSV whose age you can't see.

The constitution (AGENTS.md) already binds you at all times: never read FERPA Zone-2 files, and grades reach Canvas ONLY through a sanctioned lib/tools/ writer. This skill is the how; those are the never.

Final letters are READ, not parsed

A student's grade-request letter (or self-assessment, reflection — any prose where they make a case) is comprehension data, not structured data. Never regex/NLP-extract the requested grade, the evidence, or any claim you'll repeat back to them. A parser on prose fabricates: a field script "extracted" "you requested an A" for students who asked for a C, and it reached them before anyone caught it.

• Read each letter in full — every one, no sampling — before you write feedback that references what it says.
• Ground or abstain. If you can't tie a claim like "you requested X" to the actual words of that student's letter, do not write it — leave it out. Never default.
• Files may be parsed; letters may not. Code, notebooks, CSVs, the gradebook → programmatic extraction is fine. Prose where a human makes a claim → read it.

HG-5 — the instructor is the top layer

Principle HG-5 of the hybrid grading architecture: the agent drafts; the instructor reviews and confirms; only then does anything post.

The push protocol — you run all of it; the human clicks to approve, twice

You are not blocked from pushing — but you may not skip the review. Run the whole flow yourself (never send the instructor to a terminal), and the instructor clicks an in-chat pop-up at TWO points: to attest the review, and to authorize the push. You cannot click either for them, and you cannot self-attest.

1. Grade — grader_fetch.py --challenge-dir grading/<name> → 3-pass consensus (grader_grade.py --bulk → grader_consensus.py). Single-pass LLM grading drifts; consensus is the default.
2. Show the review surface in the conversation — mandatory, never skip it. Present feedback/_all_comments.md + the per-student <KEY>.md and the old→new grade preview (dry-run) right here in chat. This IS the review. Do not run --mark-reviewed until you have actually shown the comments.
3. Attest — run grader_push.py --challenge-dir grading/<name> --mark-reviewed --yes. The grade_guardian hook fires a permission pop-up — the instructor clicks Allow to attest they reviewed _all_comments.md (Deny if you skipped showing it, #265). The .reviewed marker records the attested state (auto-invalidates if a file changes).
4. Push — run grader_push.py --challenge-dir grading/<name> --push --yes. The grade_guardian hook fires a second pop-up; the instructor clicks Allow — that click authorizes the write (#264). No terminal, ever.

What the code enforces (so the protocol isn't docs-only)

• --yes is honored on every path — including AI-drafted comments (#264). You run --mark-reviewed --yes then --push --yes. Do not tell the instructor to open a terminal or type reviewed/push at a shell; do not pipe answers on stdin.
• The human gate is TWO in-chat pop-ups you cannot skip (#264, #265). On the AI-drafted path, grade_guardian returns an ask at BOTH --mark-reviewed and --push, so Claude Code prompts the instructor to click at each. --yes does not bypass these — the hook fires above the tool. This is what stops an agent from running --mark-reviewed --yes and self-attesting a review the human never saw. (Value-only / grader_standing pushes don't prompt — the human is the grader there.)
• Duplicate-comment Andon. Canvas appends comments, so re-runs stack them. Default mode refuses an already-graded submission; --regrade admits ONLY genuine resubmissions and supersedes (deletes old grader comments) rather than appending.
• When rows are SKIPPED, read why — never reach for the API. grader_push skips already-pushed rows (re-push with --regrade), rows that would LOWER an existing grade (--allow-lower), and inactive/Test students (#61 — --include-inactive). The fix is always a flag on the tool; a hand-written requests/python -c write skips every safeguard and the guardian blocks it anyway.
• Add or FIX comments after grading → --comments-only. To attach feedback to ALREADY-GRADED work (grade now, comment later) or replace a wrong comment, use --comments-only: it posts the comment, leaves the grade untouched, bypasses the regrade gate, and supersedes the prior grader comment so nothing stacks. Never clear grades to "get past" the gate — that's destructive and re-opens the sync-mismatch bugs.
• Comment on a NON-SUBMITTER (0 / no submission) → --roster-csv. The file-keyed push can't see a student with no submission file. --roster-csv <user_id,comment> posts comment-only straight to /submissions/<user_id> — their empty submission object still accepts it. Grade untouched, guardian pop-up still gates it. For an instructor-written note there, pass --disclosure script (it's not AI-drafted).
• Disclosure is mandatory (see the menu below), appended automatically.

The in-chat review is the gate, not a rubber-stamp. The failure an RCA was written about was students getting AI-drafted grades with no review. --yes --push is the sanctioned path now — but only because the review moved into the conversation and the guardian prompt makes the instructor consciously approve the write. So actually show the comments and the old→new grades and get a real "yes" before you push; never stack overrides to skip a gate you hit.

Never hand-write a Canvas write (the field failure mode)

Grades and comments reach Canvas only through grader_push.py (feedback) or grader_standing.py (the standing column) — never a custom requests/curl script, an inline python -c, or a /tmp/*.py. A direct write skips every safeguard at once: the review gate, the duplicate-comment Andon, Test-Student exclusion (#61), grade validation, canvas_course_guard, the disclosure tag. Duplicate comments, grades on Test Student, and wrong grade scales in the field were all hand-written scripts bypassing the tool.

The grade_guardian PreToolUse hook enforces this deterministically — it blocks creating such a script (Write), editing one (Edit), and running one (python x.py whose body writes to Canvas). If it blocks you: use the tool, or surface the blocker to the instructor. Do not route around it.

The disclosure-tag menu — say honestly what graded vs commented

--disclosure picks the provenance tag appended to each comment:

A course that grades one way every time can set CANVAS_DISCLOSURE_DEFAULT=hybrid in its .env and skip the flag; an explicit --disclosure still wins. The chosen tag prints in the pre-push banner. It won't stack if you switch graders mid-stream.

grader_standing — the "your grade" column

Which push tool? Ask: does the assignment take a student submission? Yes (online upload / text / quiz) → grader_push (keyed on submission files). No (a No-Submission "Your Grade" / standing column) → grader_standing (roster-keyed by user_id, overwrites freely, no regrade gate). Using grader_push on a No-Submission column hits the regrade gate and dead-ends — grader_push now detects this and refuses with a pointer here, but pick the right tool up front.

For a No-Submission column (often weighted 100%) that you compute from a syllabus table and refresh weekly — instructor-computed, value-only, roster-keyed. It sidesteps Canvas's auto-zero and the regrade gate. It computes nothing — your script produces the values (from the gradebook — grader_fetch_gradebook.py mirrors it locally, de-identified and cached, as that upstream input); the tool pushes them, dry-run by default, with strict guards (unmatched/ambiguous key → hard fail; out-of-bounds → abort; big drop → abort unless --allow-swings).

Confirming a standing push — use --yes, NOT a terminal. Because standing is value-only and instructor-computed (no AI-drafted feedback to review), --yes is allowed here — the safe side of HG-5, unlike grader_push. The right flow: run the dry-run, show the instructor the old → new preview, and when they confirm (e.g. they say "push"), re-run with --yes yourself. Do not tell the instructor to open a terminal and type push — our audience is non-technical faculty; that's a dead end. Their confirmation of the preview is the attestation; --yes captures it. (You still never generate the grades yourself and --yes them — the values come from the instructor's script/CSV.)

grader_standing.py --csv standing.csv --assignment-id <id>                    # dry-run diff
grader_standing.py --csv standing.csv --assignment-id <id> --push --allow-enrolled
grader_standing.py --csv standing.csv --assignment-id <id> --push --yes --allow-enrolled   # weekly

FERPA during grading — de-identify before the LLM, re-identify by key

• grader_fetch.py de-identifies submissions (strips names → opaque keys) before anything reaches the LLM. Names live only in the gitignored keymap/master.
• Re-identify by KEY, never by sort order. grader_reidentify.py maps key → user_id via .keymap.json, and is duplicate-aware (user_id → [keys]) — one student legitimately has many keys across submission batches. Positional / sort-order mapping is wrong and will misattribute grades.
• build_deid_master.py builds the course-wide .deid_master.csv (one row per student, keyed by user_id; dedups multi-section students).
• Zone-2 files (.deid_master.csv, .keymap.json, .known_names.txt, .review.csv, submissions_raw/, feedback/_grader*.csv) are never read or displayed. Verify with wc -l / ls, never cat/head/grep.
• The working files you do read here (_computed_grades.csv, _gradebook_canvas.csv, _actual_grades.csv) carry names. Reading them is fine; echoing a name out of them is not — report every student as user_id or deid_code. See the constitution's FERPA section (Zone 2-Adjacent).

Grading a course with no Canvas API (#339 Phase A)

Some courses run on another LMS entirely (Brightspace/D2L) with no API and no export — inputs are local-only: the instructor's notes/posts as files, and a roster CSV. The Canvas de-id/re-id pipeline above doesn't apply (there's no fetch step to hook a rename into), but the re-identify half does, adapted:

• Declare the course's own Zone-2 files first. .claude/ferpa_zone2.txt is not Canvas-specific — it's a per-course extension point (#278). A non-Canvas course has different name-bearing files than a Canvas one (a roster CSV, a mixed discussion export, per-student submission .txt files) and MUST list them there before any of this applies. An installed hook covering none of your files is worse than no hook.
• The roster CSV is the map, read only by local_feedback_join.py. Work from codes: the roster's own id column (a D2L Classlist export's OrgDefinedId, not a name) plays the same role Canvas's user_id does elsewhere in this skill — draft results keyed by that code, never a name.
• local_feedback_join.py joins names back in at output time: first name only by default, "First L." only for students who actually share a first name (computed against the whole roster, so a student's display name is stable run to run). Its own console output — dry run and --apply alike — never prints a name; the joined text exists only in the file --output writes, for the instructor to open themselves.
• Splitting the source files is not yet built (Phase B, #339) — a mixed discussion export with per-student separators, or renaming already- per-student submission files to codes, still needs to happen by hand or by a not-yet-built tool. local_feedback_join.py only covers the re-identify side once you already have code-keyed results.
• Why grade_guardian doesn't need new awareness for this: its Canvas- write bypass detector requires BOTH an HTTP write verb (requests.post, .put(, etc.) AND Canvas-specific submission/grade-endpoint context (posted_grade, /assignments/.../submissions). A purely local script — reads a roster CSV, writes a local file, makes no network calls — never matches either, so it's never mistaken for a hand-written Canvas bypass. The roster/notes files are still protected, generically, by the ordinary Zone-2 path-read block once declared above.

Final-letter grading — split the write, two sanctioned tools

A "final letter" workflow has two Canvas writes; do NOT hand-write a fix_push.py (the grade_guardian hook blocks it, correctly). Keep your course-specific computation (the tier/stretch logic in calc_final_grades.py), and route both writes:

1. The grade (Course Grade column, value-only) → grader_standing.py — emit a user_id,final_grade CSV, then --push --yes (instructor-computed, value-only).
2. The comment (End Letter, comment-only, preserves the existing grade) → grader_letter_comments.py — a user_id,comment CSV → comment-only writes, no grade change. --yes allowed because these are the instructor's own notes.

AI-drafted per-student feedback is NOT a final-letter comment — that goes through grader_push (the HG-5 review gate). grader_letter_comments is for instructor- authored notes only.

Known case: an auto-scored quiz stuck in "To Do"

A classic quiz with an essay/file-upload question auto-scores on submission (workflow_state: graded) but stays in the instructor's To-Do because the manual question is pending_review. Do NOT reach for grader_push — regrade_gate correctly refuses (it's already graded; this isn't a grade push), and stacking --force/--regrade is the wrong move. If that manual question is worth 0 points, use grader_quiz_clear_pending.py: it posts a 0 to the 0-point manual question to clear the flag — and it can't change a grade because it only ever touches 0-point questions. If the pending question is worth points, that's real grading → SpeedGrader.

Quick command map

Full grading knowledge: lib/agents/knowledge/grader_hybrid_architecture.md and lib/agents/knowledge/toolkit_reuse_knowledge.md.