Skip to content
OpenSmartRoute
Skillv1.0.0

deepeval-otel

Export raw OpenTelemetry traces from an AI application to Confident AI's Observatory. TRIGGER when the user wants to send OpenTelemetry or OTLP traces/spans from an LLM app, agent, RAG pipeline, or ch

by confident-ai(0) 0 installs
Free
Sign in to install

Free account. Installing gives you the manifest plus copy-paste snippets.

See reviews

About

Imported from confident-ai/deepeval (skills/deepeval-otel/SKILL.md). Install upstream with npx skills add confident-ai/deepeval --skill deepeval-otel. Copyright stays with the author (Apache-2.0).

DeepEval OpenTelemetry Export

Use this skill to instrument an AI application — an LLM app, agent, RAG pipeline, or chatbot — with raw OpenTelemetry so its traces land in Confident AI's Observatory. No deepeval package is needed — it works with any OTLP-capable OpenTelemetry SDK. The job is exactly two things: export to the correct Confident AI OTLP endpoint, and set the confident.* attributes Confident AI reads off each span.

Scope: AI Applications Only

This skill instruments AI applications only. The confident.* attributes and span types — agent, llm, retriever, tool — describe AI components, and Confident AI's Observatory is built to evaluate and monitor AI behavior.

Instrument only the AI parts of the system: agent loops and planning, LLM calls, retrieval / vector search, and tool calls. Do not apply confident.* attributes to non-AI software (web servers, CRUD backends, database layers, infrastructure) or to non-AI spans inside an otherwise-AI app — that data does not belong in Confident AI and will not render meaningfully. If the target has no LLM, agent, retrieval, or tool-calling component, this skill does not apply.

When to Use vs the deepeval Skill

Use this skill for vendor-neutral OTLP export to Confident AI — pointing an OpenTelemetry exporter at Confident AI and setting confident.* attributes.

Use the deepeval skill when the user wants to build a Python pytest eval suite, generate datasets or goldens, write metrics, run deepeval test run, or instrument with the deepeval SDK's @observe decorator. The two skills are complementary, not alternatives.

Prerequisites

  • A Confident AI account and a CONFIDENT_API_KEY.
  • An OpenTelemetry SDK for the application's language. For Python: opentelemetry-sdk and opentelemetry-exporter-otlp-proto-http.
  • The Confident AI OTLP endpoint accepts HTTP only — never gRPC.

How It Works

Confident AI exposes an OTLP/HTTP traces endpoint. Point any OpenTelemetry span exporter at it with the x-confident-api-key header. Confident AI's exporter then reads confident.* attributes off each span to build the trace and span structure. Parent/child nesting comes from native OpenTelemetry span context, not from any attribute.

Workflow

  1. Confirm the target is an AI application (it has LLM calls, an agent loop, retrieval, or tool calls). If it has none of these, stop — this skill does not apply. Then inspect for an existing OpenTelemetry setup (a TracerProvider, span exporters, or an OpenTelemetry Collector) and prefer repointing what exists over adding a parallel pipeline.
  2. Choose the endpoint from the API key's region prefix. Read references/endpoint-and-exporter.md.
  3. Wire (or repoint) an OTLP/HTTP span exporter with the x-confident-api-key header. For Python, start from templates/confident_otel_setup.py.
  4. If the process runs other OpenTelemetry instrumentation or an APM agent (auto-instrumentation for HTTP/DB, Datadog, etc.), isolate the Confident AI export so only AI spans reach it — a dedicated pipeline or a span filter. Read "Export Only AI Spans" in references/endpoint-and-exporter.md.
  5. Set confident.span.* attributes on spans; set confident.trace.* for trace-wide fields. Read references/span-attributes.md and references/trace-attributes.md.
  6. Honor the OTLP data-type rules: JSON-encode dicts/metadata, use native arrays for string lists. See the Data-Type Rules in span-attributes.md.
  7. If the app already emits OpenTelemetry GenAI semantic conventions, read references/gen-ai-fallbacks.md before adding redundant attributes.
  8. Verify traces appear in the Confident AI Observatory.

Core Principles

  1. Instrument AI components only — agent, LLM, retriever, and tool spans. Never apply confident.* attributes to non-AI software or non-AI spans.
  2. Export only AI spans. If the process has other OpenTelemetry instrumentation or an APM agent, isolate the Confident AI pipeline (a dedicated provider or a span filter) so non-AI spans — HTTP requests, DB queries, infra — are never exported to Confident AI.
  3. Prefer repointing an existing OTLP exporter over adding a parallel one.
  4. The confident.* attribute keys are the entire contract — they are the same in every language, so language choice is irrelevant.
  5. Always use OTLP/HTTP. Confident AI's endpoint does not accept gRPC.
  6. Honor OTLP data-type rules: attribute values must be primitives or homogeneous primitive lists; JSON-encode dicts and metadata.
  7. Set confident.span.type explicitly when it is known; rely on gen_ai.* inference only as a fallback.
  8. Never put secrets, credentials, or raw sensitive data into span attributes.

References

Topic File
Endpoints, region selection, auth, exporter wiring references/endpoint-and-exporter.md
Trace-level confident.trace.* attributes references/trace-attributes.md
Span-level confident.span.* attributes and data-type rules references/span-attributes.md
Standard OTel gen_ai.* fallback behavior references/gen-ai-fallbacks.md

Templates

Purpose Template
Minimal Python OTLP exporter setup + example trace templates/confident_otel_setup.py

Use it

Copy one of these into your project. Installing also returns the manifest and these snippets.

yaml
targets:
  - https://api.opensmartroute.ai/api/v1/registry/confident-ai-deepeval-deepeval-otel/manifest   # or paste the manifest below

Manifest

An Open Capability Manifest: the router reads it to know what this does, what it costs and when to pick it.

confident-ai-deepeval-deepeval-otel.ocm.jsonjson
{
  "ocm": "1",
  "id": "confident-ai-deepeval-deepeval-otel",
  "kind": "skill",
  "name": "deepeval-otel",
  "description": "Export raw OpenTelemetry traces from an AI application to Confident AI's Observatory. TRIGGER when the user wants to send OpenTelemetry or OTLP traces/spans from an LLM app, agent, RAG pipeline, or chatbot to Confident AI; configure the Confident AI OTLP endpoint; set confident.span.* or confident.trace.* attributes; export AI-app traces without the deepeval Python package; wire an OTLPSpanExporter, OpenTelemetry Collector, or vendor-neutral OTel SDK to Confident AI; or pick the US vs EU OTLP endpoint. Language-agnostic: the mechanism is OTLP attribute keys plus an exporter endpoint. DO NOT TRIGGER for building DeepEval pytest eval suites, datasets, goldens, metrics, or deepeval test run (use the `deepeval` skill); for instrumenting with the DeepEval SDK's @observe decorator or framework integrations (use the `deepeval-tracing` skill); or for non-AI software such as web servers, CRUD backends, or infrastructure: the confident.* attributes describe AI components only.",
  "publisher": "confident-ai",
  "version": "1.0.0",
  "capabilities": {
    "domains": [
      "coding"
    ],
    "tags": [
      "skill-md",
      "opentelemetry",
      "otel",
      "otlp",
      "tracing",
      "confident-ai",
      "observatory",
      "spans",
      "skills-sh"
    ],
    "languages": [
      "en"
    ]
  },
  "quality_prior": 0.6,
  "examples": [
    "Export raw OpenTelemetry traces from an AI application to Confident AI's Observatory. TRIGGER when the user wants to send OpenTelemetry or OTLP traces/spans from an LLM app, agent, RAG pipeline, or chatbot to Confident AI; configure the Confident AI OTLP endpoint; set confident.span.* or confident.trace.* attributes; export AI-app traces without the deepeval Python package; wire an OTLPSpanExporter, OpenTelemetry Collector, or vendor-neutral OTel SDK to Confident AI; or pick the US vs EU OTLP endpoint. Language-agnostic: the mechanism is OTLP attribute keys plus an exporter endpoint. DO NOT TRIGGER for building DeepEval pytest eval suites, datasets, goldens, metrics, or deepeval test run (use the `deepeval` skill); for instrumenting with the DeepEval SDK's @observe decorator or framework integrations (use the `deepeval-tracing` skill); or for non-AI software such as web servers, CRUD backends, or infrastructure: the confident.* attributes describe AI components only."
  ],
  "primary": false,
  "metadata": {
    "source": {
      "provider": "skills.sh",
      "repository": "https://github.com/confident-ai/deepeval",
      "path": "skills/deepeval-otel/SKILL.md",
      "ref": "HEAD",
      "url": "https://github.com/confident-ai/deepeval/blob/HEAD/skills/deepeval-otel/SKILL.md",
      "key": "confident-ai/deepeval/skills/deepeval-otel/SKILL.md"
    },
    "license": "Apache-2.0"
  },
  "instructions": "# DeepEval OpenTelemetry Export\n\nUse this skill to instrument an **AI application** — an LLM app, agent, RAG\npipeline, or chatbot — with **raw OpenTelemetry** so its traces land in\n**Confident AI's Observatory**. No `deepeval` package is needed — it works with\nany OTLP-capable OpenTelemetry SDK. The job is exactly two things: export to\nthe correct Confident AI OTLP endpoint, and set the `confident.*` attributes\nConfident AI reads off each span.\n\n## Scope: AI Applications Only\n\nThis skill instruments **AI applications only**. The `confident.*` attributes\nand span types — `agent`, `llm`, `retrie",
  "cost": {
    "context_tokens": 1335
  }
}

Fetch it by URL: GET /api/v1/registry/confident-ai-deepeval-deepeval-otel/manifest?version=1.0.0

Reviews

Star ratings from people who tried it. One review per account; edit yours any time.

No reviews yet. Install it, try it, and be the first to rate it.