Instruction file imported from johnlicataptbiz/bizcloser (
.github/instructions/ml-workflow.instructions.md). Copyright stays with the author.
BizCloser ML Workflow Instructions
This file provides instructions for the BizCloser ML optimization loop.
Workflow Overview
The BizCloser agent uses a "Perfection Loop" for continuous model improvement. Follow these steps when modifying model behavior:
- Baseline Evaluation: Run the platform eval suite using
npm run eval:platform:agent1. - Failure Analysis: Identify the specific "Qualifier" or "Style" mismatch in
backend/data/eval/openai-platform-agent1-run-details.json. - Failure Slicing: Create a
.jsonlsuite containing only the failure cases to speed up iteration. - Prompt vs. Fine-Tune Decision:
- If the model misses broad style or context, prefer Fine-Tuning.
- If the model misses specific gating logic, attempt Prompt Tuning (v2 system prompt) first.
- When using the OpenAI platform eval flow, prefer the platform file-backed dataset path over inline rows so runs are reproducible from a persisted eval dataset artifact.
- Perfection Loop: Iteratively run the failure slice until 100% pass rate is achieved on the subset.
- Full Regression: Run the original full suite to ensure no regressions.
Current Repo Artifacts
- Latest baseline run:
backend/data/eval/openai-platform-agent1-run-details.json - Latest failure-slice suite:
backend/data/eval/platform-agent1-failure-slices-v2.jsonl - Prompt tuning draft:
backend/data/eval/system-prompt-v2-draft.txt - Prompt-tuning run:
backend/data/eval/openai-platform-agent1-run-details-v3-prompt-tuning.json - Negative-example fine-tune seed:
backend/data/fine-tune/bizcloser-v7-negative-examples.jsonl - Latest analysis report:
backend/data/eval/failure-slice-v2-analysis.md
Tools
- Eval Runner:
backend/scripts/run-openai-platform-eval.ts - Skill:
.roo/skills/autonomous-machine-learning-dev-suite/SKILL.md
Quality Gates
Route & Gating> 90%Qualification Precision> 90%Call Pitch Structure(Fidelity) > 95%Human SMS Style> 85%
Decision Rule
If prompt tuning does not improve the failure slice materially, stop adding prompts and move to a targeted fine-tune with negative examples instead.
Decision Maker / Spouse Check Clarification
When evaluating AGENT-based threads, treat the spouse question as a decision-maker inclusion check, not as a relationship screening question.
The purpose is to verify whether anyone else needs to be on the call because the most common objection is:
- "I need to talk to my spouse"
Use wording that asks whether another decision maker should join the call so the lead does not have to relay details afterward.
Fifth Qualifier Clarification
After the 4 business-fit qualifiers are known, treat the 5th qualifier as:
- whether the lead is actually evaluating business coaching, clinic-growth support, or PT Biz help right now
This qualifier is about intent and evaluation stage, not general curiosity alone.
Examples:
- YES: actively looking at coaching/help, comparing options, wants to see if support makes sense
- NO: just chatting, unrelated support request, no real evaluation intent
- UNKNOWN: some interest is present, but it is still not clear whether they are seriously evaluating help