How long does the integration take?

Change one base URL in an OpenAI-compatible client, or install the Python or TypeScript SDK; the quickstart routes a first request in a few minutes.

Will this lock us into another vendor?

No. The router is Apache-2.0 open source, your provider keys stay yours, and the API is OpenAI-compatible, so leaving is a change of base URL.

What happens when a provider fails?

The router falls back to the next model your policy allows and records the fallback in the trace, so an outage costs a retry, not an incident.

OP5---LLM-Cost-Latency-Quality-Optimize - Skill

Name: OP5---LLM-Cost-Latency-Quality-Optimize
Author: angWindy

Imported from angWindy/OP5---LLM-Cost-Latency-Quality-Optimize (AGENTS.md). Install upstream with npx skills add angWindy/OP5---LLM-Cost-Latency-Quality-Optimize. Copyright stays with the author.

AGENTS.md — Instructions for AI agents working on OP5

Read this first. The plan is in doc/plan/op5-llm-cost-latency-quality-plan.md. The current phase and recent worklog are in doc/phases/INDEX.md and doc/worklog/.

TL;DR

Side-project. No company data, no company infra. Demo providers only.
Conda env: always conda activate vsf before running anything Python.
Large folders are ignored by default (see .cursorignore): data/raw/, data/processed/, data/cache/, results/. Don't try to Read or Grep them wholesale. Ask the user to fetch a slice if you need it.
Every experiment writes JSONL paired by case_id (see master plan §4.4).
Held-out set is read-once-only — do not run experiments on it during tuning.

Workspace conventions

Need to…	Use
Run an experiment	Write a script under `scripts/phase-XX/`, invoke from repo root
Add a schema	JSON Schema under `src/op5/schemas/`, reference from `doc/plan` §4.1
Add a scorer	`src/op5/scorers/<name>.py`, expose `score(pred, ref) -> dict`
Log eval results	`results/<phase>-<desc>.jsonl` matching EvalLog schema
Document a decision	Append to `doc/worklog/YYYY-MM-DD-phase-XX-<topic>.md`
Start a new phase	Add `doc/phases/phase-XX-<slug>.md`, mark `active` in `INDEX.md`

What NOT to do

Don't Read anything under data/raw/ or data/processed/ without the user requesting it explicitly. These can be gigabytes.
Don't run anything on the held-out set until the phase that uses it says so.
Don't commit .env or any file containing API keys.
Don't change the deterministic router thresholds without a paired experiment on dev set + a worklog entry justifying the change.
Don't start the next phase before closing out the current one (worklog written, INDEX.md updated).

When you start a session

Skim doc/phases/INDEX.md — find the active phase.
Read its plan file.
Read the most recent worklog.
Confirm vsf is available (conda activate vsf && python --version).
Confirm the user wants you to keep going on the active phase, or pivot.

When you finish a session

Update the worklog with what actually happened, even if it's just "tried X, didn't work".
Update doc/phases/INDEX.md if a phase moved status.
Surface any blockers to the user explicitly.

Install OP5---LLM-Cost-Latency-Quality-Optimize and it becomes one more candidate the router can pick - when it fits.

A listing is a routing target with a manifest: what it does, which domains it covers, what it costs and who publishes it. Once installed it sits beside your own models and tools, is scored like any other candidate for each request, and shows up in the trace when it wins. Ratings come from workspaces that installed it, one per account.

{ "ocm": "1", "id": "angwindy-op5-llm-cost-latency-quality-optimize-op5-llm-c-a2c488", "kind": "skill", "name": "OP5---LLM-Cost-Latency-Quality-Optimize", "description": "1. Skim `doc/phases/INDEX.md` — find the active phase. 2. Read its plan file. 3. Read the most recent worklog. 4. Confirm `vsf` is available (`conda activate vsf && python --version`). 5. Confirm the user wants you to keep going on the active phase, or pivot.", "publisher": "angWindy", "version": "1.0.0", "capabilities": { "domains": [ "coding" ], "tags": [ "skill-md", "github-agents" ], "languages": [ "en" ] }, "quality_prior": 0.6, "examples": [ "1. Skim `doc/phases/INDEX.md` — find the active phase. 2. Read its plan file. 3. Read the most recent worklog. 4. Confirm `vsf` is available (`conda activate vsf && python --version`). 5. Confirm the user wants you to keep going on the active phase, or pivot." ], "primary": false, "metadata": { "source": { "provider": "github-agents", "repository": "https://github.com/angWindy/OP5---LLM-Cost-Latency-Quality-Optimize", "path": "AGENTS.md", "ref": "5288bcc5e2e0d85cc10116c4a99bf32d366da9c3", "url": "https://github.com/angWindy/OP5---LLM-Cost-Latency-Quality-Optimize/blob/5288bcc5e2e0d85cc10116c4a99bf32d366da9c3/AGENTS.md", "key": "angWindy/OP5---LLM-Cost-Latency-Quality-Optimize/AGENTS.md" } }, "instructions": "# AGENTS.md — Instructions for AI agents working on OP5\n\n> Read this first. The plan is in `doc/plan/op5-llm-cost-latency-quality-plan.md`. The\n> current phase and recent worklog are in `doc/phases/INDEX.md` and `doc/worklog/`.\n\n## TL;DR\n\n- Side-project. **No company data, no company infra.** Demo providers only.\n- **Conda env:** always `conda activate vsf` before running anything Python.\n- **Large folders are ignored by default** (see `.cursorignore`): `data/raw/`, `data/processed/`,\n `data/cache/`, `results/`. Don't try to `Read` or `Grep` them wholesale. Ask the user to\n fetch a slice if ", "cost": { "context_tokens": 594 } }

OP5---LLM-Cost-Latency-Quality-Optimize

About

AGENTS.md — Instructions for AI agents working on OP5

TL;DR

Workspace conventions

What NOT to do

When you start a session

When you finish a session

Install OP5---LLM-Cost-Latency-Quality-Optimize and it becomes one more candidate the router can pick - when it fits.

What it costs to adopt - in engineering time.

One base URL

No vendor lock-in

Failover built in

Use it

Manifest

Reviews