About

Imported from jeremylongshore/tons-of-skills-marketplace (plugins/saas-packs/clickhouse-pack/skills/clickhouse-observability/SKILL.md). Install upstream with npx skills add jeremylongshore/tons-of-skills-marketplace --skill clickhouse-observability. Copyright stays with the author (MIT).

ClickHouse Observability

Name: clickhouse-observability
Author: jeremylongshore

Overview

Set up comprehensive monitoring for ClickHouse using built-in system tables, Prometheus integration, Grafana dashboards, and alerting rules. The workflow layers four signal sources: system.* tables (always available, zero dependencies), a Prometheus scrape endpoint, application-level client instrumentation, and alert rules that fire on the failure modes that actually page an on-call — high error rate, latency creep, merge backlog, and resource exhaustion.

Deep configs live in references/ so this file stays a fast, followable map.

Prerequisites

ClickHouse instance with system.* table access
Prometheus (or compatible: Grafana Alloy, Victoria Metrics)
Grafana for dashboards
AlertManager or PagerDuty for alerts

Instructions

Step 1: Query system tables for a health snapshot

Start with zero dependencies — the system.* tables already hold everything. Run this for an instant server-health read:

SELECT
    (SELECT count() FROM system.processes) AS running_queries,
    (SELECT value FROM system.metrics WHERE metric = 'MemoryTracking') AS memory_bytes,
    (SELECT count() FROM system.merges) AS active_merges;

Query throughput, insert rates, and per-table part counts (the merge-health signal), plus a full table of which system.* table to poll at what frequency: system table queries & reference.

Step 2: Wire up Prometheus scraping

ClickHouse Cloud exposes a managed Prometheus endpoint (Basic auth with a Cloud API key); self-hosted uses the built-in :9363 /metrics endpoint enabled in config.xml. Write the scrape config to your prometheus.yml. Full Cloud + self-hosted scrape configs and the config.xml block: Prometheus scrape config & Grafana dashboards.

Step 3: Instrument the application client

Server metrics show what ClickHouse does; client metrics attribute latency, error codes, and insert volume to your own code. Wrap queries in a prom-client histogram/counter and expose /metrics. Full instrumentation + structured logging: application-level instrumentation.

Step 4: Build Grafana dashboard panels

Panels for QPS, P50/P95/P99 latency, error rate, and insert throughput are in the Grafana dashboards reference. Or import the official community dashboard: https://grafana.com/grafana/dashboards/23415.

Step 5: Load alert rules

Write Prometheus alert rules for the five production failure modes (error rate, latency, part count, memory, disk) to a rules file loaded by AlertManager. Full rule set plus per-alert tuning notes: Prometheus alert rules.

Output

Applying this skill produces a set of monitoring config artifacts you write to your infrastructure repo:

prometheus.yml — scrape config targeting your ClickHouse endpoint
clickhouse-alerts.yml — the five-rule alert group loaded by AlertManager
A Grafana dashboard (imported ID 23415 or the custom JSON panels)
Client instrumentation exposing clickhouse_query_duration_seconds, clickhouse_query_errors_total, and clickhouse_insert_rows_total
Ad-hoc system.* queries for on-demand health snapshots

Error Handling

Issue	Cause	Solution
Metrics endpoint empty	Prometheus not configured	Enable `/metrics` in config
High cardinality alerts	Too many label values	Reduce label cardinality
Missing query_log data	Logging disabled	Set `log_queries = 1` in config
Dashboard gaps	Scrape interval too long	Use 10-15s scrape interval

Examples

Snapshot server health right now — run the Step 1 query against any instance with system.* access; no exporter or scrape needed. See system-tables.md for throughput and merge-health variants.

Alert when merges fall behind — the ClickHouseTooManyParts rule fires when a table exceeds 300 active parts for 10 minutes, the classic inserts-outpacing-merges signal. Full rule + tuning guidance: alerting.md.

Attribute slow queries to your service — wrap calls in instrumentedQuery() so P95 latency and error codes land in Prometheus labeled by query type. See instrumentation.md.

Resources

Prometheus Integration
ClickHouse Grafana Dashboard
System Tables Reference
Cloud Monitoring
For incident response, see the clickhouse-incident-runbook skill.

Use it

Copy one of these into your project. Installing also returns the manifest and these snippets.

yaml

targets:
  - https://api.opensmartroute.ai/api/v1/registry/jeremylongshore-tons-of-skills-marketplace-clickhouse-ob-853a88/manifest   # or paste the manifest below

Manifest

An Open Capability Manifest: the router reads it to know what this does, what it costs and when to pick it.

jeremylongshore-tons-of-skills-marketplace-clickhouse-ob-853a88.ocm.jsonjson

{
  "ocm": "1",
  "id": "jeremylongshore-tons-of-skills-marketplace-clickhouse-ob-853a88",
  "kind": "skill",
  "name": "clickhouse-observability",
  "description": "Monitor ClickHouse with Prometheus metrics, Grafana dashboards, system table queries, and alerting for query performance, merge health, and resource usage. Use when setting up ClickHouse monitoring, building Grafana dashboards, or configuring alerts for production ClickHouse deployments. Trigger with \"clickhouse monitoring\", \"clickhouse metrics\", \"clickhouse Grafana\", \"clickhouse observability\", \"monitor clickhouse\", \"clickhouse Prometheus\".",
  "publisher": "jeremylongshore",
  "version": "1.0.0",
  "capabilities": {
    "domains": [
      "data_analysis"
    ],
    "tags": [
      "skill-md",
      "saas",
      "database",
      "analytics",
      "clickhouse",
      "olap",
      "skills-sh"
    ],
    "languages": [
      "en"
    ]
  },
  "quality_prior": 0.6,
  "examples": [
    "Monitor ClickHouse with Prometheus metrics, Grafana dashboards, system table queries, and alerting for query performance, merge health, and resource usage. Use when setting up ClickHouse monitoring, building Grafana dashboards, or configuring alerts for production ClickHouse deployments. Trigger with \"clickhouse monitoring\", \"clickhouse metrics\", \"clickhouse Grafana\", \"clickhouse observability\", \"monitor clickhouse\", \"clickhouse Prometheus\"."
  ],
  "primary": false,
  "metadata": {
    "source": {
      "provider": "skills.sh",
      "repository": "https://github.com/jeremylongshore/tons-of-skills-marketplace",
      "path": "plugins/saas-packs/clickhouse-pack/skills/clickhouse-observability/SKILL.md",
      "ref": "HEAD",
      "url": "https://github.com/jeremylongshore/tons-of-skills-marketplace/blob/HEAD/plugins/saas-packs/clickhouse-pack/skills/clickhouse-observability/SKILL.md",
      "key": "jeremylongshore/tons-of-skills-marketplace/plugins/saas-packs/clickhouse-pack/skills/clickhouse-observability/SKILL.md"
    },
    "compatibility": "Designed for Claude Code",
    "allowed_tools": [
      "Read,",
      "Write,",
      "Edit"
    ],
    "license": "MIT"
  },
  "instructions": "# ClickHouse Observability\n\n## Overview\n\nSet up comprehensive monitoring for ClickHouse using built-in system tables,\nPrometheus integration, Grafana dashboards, and alerting rules. The workflow\nlayers four signal sources: `system.*` tables (always available, zero\ndependencies), a Prometheus scrape endpoint, application-level client\ninstrumentation, and alert rules that fire on the failure modes that actually\npage an on-call — high error rate, latency creep, merge backlog, and resource\nexhaustion.\n\nDeep configs live in `references/` so this file stays a fast, followable map.\n\n## Prerequisites\n",
  "cost": {
    "context_tokens": 1204
  }
}

Fetch it by URL: GET /api/v1/registry/jeremylongshore-tons-of-skills-marketplace-clickhouse-ob-853a88/manifest?version=1.0.0

Reviews

Star ratings from people who tried it. One review per account; edit yours any time.

No reviews yet. Install it, try it, and be the first to rate it.