metric-pack-designer

Name: metric-pack-designer
Author: openai

Design custom metric packs for plugin-eval so teams can add local evaluation rubrics that emit schema-compatible checks and metrics. Use when the user wants their own evaluation criteria or visualizat

by openai(0) 0 installs

Free

Free account. Installing gives you the manifest plus copy-paste snippets.

See reviews

About

Imported from openai/plugins (plugins/plugin-eval/skills/metric-pack-designer/SKILL.md). Install upstream with npx skills add openai/plugins --skill metric-pack-designer. Copyright stays with the author.

Metric Pack Designer

Use this skill when the user wants to extend plugin-eval with a local rubric.

Workflow

Clarify the custom rubric categories and target kinds.
Define the smallest useful checks[] and metrics[] payload.
Create a metric-pack manifest plus a script that prints JSON to stdout.
Run the pack through plugin-eval analyze <path> --metric-pack <manifest.json>.

Design Rules

Keep IDs stable across runs so comparisons stay meaningful.
Emit only checks[], metrics[], and optional artifacts[].
Do not try to overwrite the core score or summary.
Prefer deterministic local signals over subjective text generation.

Reference

../../references/metric-pack-manifest.md

Use it

Copy one of these into your project. Installing also returns the manifest and these snippets.

# after Install: the listing is in your workspace's routing pool - a plan picks it for its slot
curl -s -X POST https://api.opensmartroute.ai/api/v1/route -H 'Authorization: Bearer $OSR_API_KEY' -H 'Content-Type: application/json' -d '{"text": "...", "plan": true}'

Manifest

An Open Capability Manifest: the router reads it to know what this does, what it costs and when to pick it.

openai-plugins-metric-pack-designer.ocm.jsonjson

{
  "ocm": "1",
  "id": "openai-plugins-metric-pack-designer",
  "kind": "skill",
  "name": "metric-pack-designer",
  "description": "Design custom metric packs for plugin-eval so teams can add local evaluation rubrics that emit schema-compatible checks and metrics. Use when the user wants their own evaluation criteria or visualizations.",
  "publisher": "openai",
  "version": "1.0.0",
  "capabilities": {
    "domains": [
      "general"
    ],
    "tags": [
      "skill-md",
      "skills-sh"
    ],
    "languages": [
      "en"
    ]
  },
  "quality_prior": 0.6,
  "examples": [
    "Design custom metric packs for plugin-eval so teams can add local evaluation rubrics that emit schema-compatible checks and metrics. Use when the user wants their own evaluation criteria or visualizations."
  ],
  "primary": false,
  "metadata": {
    "source": {
      "provider": "skills.sh",
      "repository": "https://github.com/openai/plugins",
      "path": "plugins/plugin-eval/skills/metric-pack-designer/SKILL.md",
      "ref": "HEAD",
      "url": "https://github.com/openai/plugins/blob/HEAD/plugins/plugin-eval/skills/metric-pack-designer/SKILL.md",
      "key": "openai/plugins/plugins/plugin-eval/skills/metric-pack-designer/SKILL.md"
    }
  },
  "instructions": "# Metric Pack Designer\n\nUse this skill when the user wants to extend `plugin-eval` with a local rubric.\n\n## Workflow\n\n1. Clarify the custom rubric categories and target kinds.\n2. Define the smallest useful `checks[]` and `metrics[]` payload.\n3. Create a metric-pack manifest plus a script that prints JSON to stdout.\n4. Run the pack through `plugin-eval analyze <path> --metric-pack <manifest.json>`.\n\n## Design Rules\n\n- Keep IDs stable across runs so comparisons stay meaningful.\n- Emit only `checks[]`, `metrics[]`, and optional `artifacts[]`.\n- Do not try to overwrite the core score or summary.\n-",
  "cost": {
    "context_tokens": 182
  }
}

Fetch it by URL: GET /api/v1/registry/openai-plugins-metric-pack-designer/manifest?version=1.0.0