How long does the integration take?

Change one base URL in an OpenAI-compatible client, or install the Python or TypeScript SDK; the quickstart routes a first request in a few minutes.

Will this lock us into another vendor?

No. The router is Apache-2.0 open source, your provider keys stay yours, and the API is OpenAI-compatible, so leaving is a change of base URL.

What happens when a provider fails?

The router falls back to the next model your policy allows and records the fallback in the trace, so an outage costs a retry, not an incident.

srt-benchmark - Prompt - OpenSmartRoute

Name: srt-benchmark
Author: vinayakv22

Prompt file imported from vinayakv22/insanely-fast-whisper-rocm (.windsurf/workflows/srt-benchmark.md). Copyright stays with the author.

SRT quality benchmark workflow

Purpose

Guide to regenerate benchmarks, collect SRT quality metrics, and compare word vs chunk timestamp behavior.

Steps

Reset artifacts (only after recent code changes that could effect benchmarks)
```
scripts/clean_benchmark.sh
```
Removes cached benchmark JSON, SRT, and TXT files so the new run is isolated.
Run word-level benchmark variants

Choose one of the files below that best matches what kind of benchmark we want to test:
- uploads/Weighted_Scorecard_vs_5_minute_test.mp3 (short)
- uploads/Weighted_Scorecard_vs_medium_test.mp3 (medium)
- uploads/Weighted_Scorecard_vs.mp3 (large +- 17 minutes)
```
scripts/benchmark.sh -a "uploads/Weighted_Scorecard_vs_5_minute_test.mp3" \
  -m distil-whisper/distil-large-v3.5 \
  --timestamp-types word \
  --batch-word 4
```
Captures baseline, stabilize, demucs, VAD, and combination variants with word timestamps.

Run chunk-level baseline (fallback validation)

scripts/benchmark.sh -a "uploads/Weighted_Scorecard_vs_5_minute_test.mp3" \
  -m distil-whisper/distil-large-v3.5 \
  --timestamp-types chunk

Produces JSON/SRT outputs that rely on the chunk fallback path to confirm duration splitting behavior.

Inspect benchmark JSON metrics
```
# Example: open baseline JSON
sed -n '1,160p' benchmarks/Weighted_Scorecard_vs_5_minute_test_transcribe_<timestamp>.json
```
Focus on format_quality.srt.score, duration_stats, boundary_counts, cps_histogram, and sample_offenders.
Compare word vs chunk results
- Quality score: confirm chunk fallback raises score when word segmentation still has long spans.
- Durations: note duration_stats.max_seconds and boundary_counts.too_long differences.
- CPS: verify cps_histogram and offender lists reflect improvements with fallback.
Review SRT outputs for timing accuracy
```
head -n 80 transcripts-srt/Weighted_Scorecard_vs_5_minute_test_transcribe_<timestamp>.srt
```
Compare word-based vs chunk-based SRT files to assess alignment vs readability trade-offs.
Document findings

Summarize observed improvements or regressions (e.g., more balanced durations, remaining boundary violations) and note follow-up actions for segment_words() or compute_srt_quality().

Install srt-benchmark and it becomes one more candidate the router can pick - when it fits.

A listing is a routing target with a manifest: what it does, which domains it covers, what it costs and who publishes it. Once installed it sits beside your own models and tools, is scored like any other candidate for each request, and shows up in the trace when it wins. Ratings come from workspaces that installed it, one per account.

# SRT quality benchmark workflow ## Purpose Guide to regenerate benchmarks, collect SRT quality metrics, and compare word vs chunk timestamp behavior. ## Steps 1. **Reset artifacts** (only after recent code changes that could effect benchmarks) ```bash scripts/clean_benchmark.sh ``` Removes cached benchmark JSON, SRT, and TXT files so the new run is isolated. 2. **Run word-level benchmark variants** Choose one of the files below that best matches what kind of benchmark we want to test: - `uploads/Weighted_Scorecard_vs_5_minute_test.mp3` (short) - `uploads/Weighted_Scorecard_vs_medium_test.mp3` (medium) - `uploads/Weighted_Scorecard_vs.mp3` (large +- 17 minutes) ```bash scripts/benchmark.sh -a "uploads/Weighted_Scorecard_vs_5_minute_test.mp3" \ -m distil-whisper/distil-large-v3.5 \ --timestamp-types word \ --batch-word 4 ``` Captures baseline, stabilize, demucs, VAD, and combination variants with word timestamps. 3. **Run chunk-level baseline (fallback validation)** ```bash scripts/benchmark.sh -a "uploads/Weighted_Scorecard_vs_5_minute_test.mp3" \ -m distil-whisper/distil-large-v3.5 \ --timestamp-types chunk ``` Produces JSON/SRT outputs that rely on the chunk fallback path to confirm duration splitting behavior. 4. **Inspect benchmark JSON metrics** ```bash # Example: open baseline JSON sed -n '1,160p' benchmarks/Weighted_Scorecard_vs_5_minute_test_transcribe_<timestamp>.json ``` Focus on `format_quality.srt.score`, `duration_stats`, `boundary_counts`, `cps_histogram`, and `sample_offenders`. 5. **Compare word vs chunk results** - **Quality score**: confirm chunk fallback raises `score` when word segmentation still has long spans. - **Durations**: note `duration_stats.max_seconds` and `boundary_counts.too_long` differences. - **CPS**: verify `cps_histogram` and offender lists reflect improvements with fallback. 6. **Review SRT outputs for timing accuracy** ```bash head -n 80 transcripts-srt/Weighted_Scorecard_vs_5_minute_test_transcribe_<timestamp>.srt ``` Compare word-based vs chunk-based SRT files to assess alignment vs readability trade-offs. 7. **Document findings** Summarize observed improvements or regressions (e.g., more balanced durations, remaining boundary violations) and note follow-up actions for `segment_words()` or `compute_srt_quality()`.

{ "prompt": "# SRT quality benchmark workflow\n\n## Purpose\n\nGuide to regenerate benchmarks, collect SRT quality metrics, and compare word vs chunk timestamp behavior.\n\n## Steps\n\n1. **Reset artifacts** (only after recent code changes that could effect benchmarks)\n\n ```bash\n scripts/clean_benchmark.sh\n ```\n\n Removes cached benchmark JSON, SRT, and TXT files so the new run is isolated.\n\n2. **Run word-level benchmark variants**\n\n Choose one of the files below that best matches what kind of benchmark we want to test:\n\n - `uploads/Weighted_Scorecard_vs_5_minute_test.mp3` (short)\n - `uploads/Weighted_Scorecard_vs_medium_test.mp3` (medium)\n - `uploads/Weighted_Scorecard_vs.mp3` (large +- 17 minutes)\n\n ```bash\n scripts/benchmark.sh -a \"uploads/Weighted_Scorecard_vs_5_minute_test.mp3\" \\\n -m distil-whisper/distil-large-v3.5 \\\n --timestamp-types word \\\n --batch-word 4\n ```\n\n Captures baseline, stabilize, demucs, VAD, and combination variants with word timestamps.\n\n3. **Run chunk-level baseline (fallback validation)**\n\n ```bash\n scripts/benchmark.sh -a \"uploads/Weighted_Scorecard_vs_5_minute_test.mp3\" \\\n -m distil-whisper/distil-large-v3.5 \\\n --timestamp-types chunk\n ```\n\n Produces JSON/SRT outputs that rely on the chunk fallback path to confirm duration splitting behavior.\n\n4. **Inspect benchmark JSON metrics**\n\n ```bash\n # Example: open baseline JSON\n sed -n '1,160p' benchmarks/Weighted_Scorecard_vs_5_minute_test_transcribe_<timestamp>.json\n ```\n\n Focus on `format_quality.srt.score`, `duration_stats`, `boundary_counts`, `cps_histogram`, and `sample_offenders`.\n\n5. **Compare word vs chunk results**\n\n - **Quality score**: confirm chunk fallback raises `score` when word segmentation still has long spans.\n - **Durations**: note `duration_stats.max_seconds` and `boundary_counts.too_long` differences.\n - **CPS**: verify `cps_histogram` and offender lists reflect improvements with fallback.\n\n6. **Review SRT outputs for timing accuracy**\n\n ```bash\n head -n 80 transcripts-srt/Weighted_Scorecard_vs_5_minute_test_transcribe_<timestamp>.srt\n ```\n\n Compare word-based vs chunk-based SRT files to assess alignment vs readability trade-offs.\n\n7. **Document findings**\n\n Summarize observed improvements or regressions (e.g., more balanced durations, remaining boundary violations) and note follow-up actions for `segment_words()` or `compute_srt_quality()`.", "variables": [], "notes": "Benchmark subtitles and inspect srt_quality metrics", "metadata": { "source": { "provider": "github-windsurf-workflows", "repository": "https://github.com/vinayakv22/insanely-fast-whisper-rocm", "path": ".windsurf/workflows/srt-benchmark.md", "ref": "5209159b1f45b0d25987100f1dec5f215d997dbf", "url": "https://github.com/vinayakv22/insanely-fast-whisper-rocm/blob/5209159b1f45b0d25987100f1dec5f215d997dbf/.windsurf/workflows/srt-benchmark.md", "key": "vinayakv22/insanely-fast-whisper-rocm/.windsurf/workflows/srt-benchmark.md" } } }

srt-benchmark

About

SRT quality benchmark workflow

Purpose

Steps

Install srt-benchmark and it becomes one more candidate the router can pick - when it fits.

What it costs to adopt - in engineering time.

One base URL

No vendor lock-in

Failover built in

Use it

Manifest

Reviews