Imported from reason-machines/data-skills (skills/altimate-data-engineering-skills/SKILL.md) via skills.sh. Install upstream with npx skills add reason-machines/data-skills --skill altimate-data-engineering-skills. Copyright stays with the author.

Altimate Data Engineering Skills

Skill by ara.so — Data Skills collection.

Altimate Data Engineering Skills is a collection of Claude Code skills that encode the workflows and best practices of experienced analytics engineers. These skills transform Claude from a code generator into a capable data engineering assistant by teaching how to approach tasks, not just what syntax to use.

The project demonstrates 53% accuracy on ADE-bench (43 real-world dbt tasks), 3x improvement on model creation tasks, and 84% pass rate on Snowflake query optimization.

What This Project Does

Data Engineering Skills provides:

7 dbt skills: Model creation, debugging, testing, documentation, migration, refactoring, incremental models
3 Snowflake skills: Cost analysis, query optimization by ID, query optimization by text
1 delegation skill: Hand off complex tasks to altimate-code CLI tool
Workflow automation: Skills trigger automatically based on user intent
Best practices: Encoded patterns from experienced analytics engineers

Skills are markdown files with YAML frontmatter that define trigger conditions and step-by-step workflows.

Installation

Installing Skills in Claude Code

# Add the marketplace plugin
/plugin marketplace add AltimateAI/data-engineering-skills

# Install all skills
/plugin install dbt-skills@data-engineering-skills
/plugin install snowflake-skills@data-engineering-skills
/plugin install altimate-code@data-engineering-skills

Installing Kits

Kits bundle skills, MCP servers, and instructions:

# Install altimate-code CLI (required for kits)
npm install -g altimate-code

# Install kit system
altimate-code kit install AltimateAI/data-engineering-skills

# Activate the dbt-snowflake kit
altimate-code kit activate dbt-snowflake

# Check status
altimate-code kit status

Manual Installation

Clone and reference skills directly:

git clone https://github.com/AltimateAI/data-engineering-skills.git
cd data-engineering-skills

Available Skills

dbt Skills

creating-dbt-models: Creates new dbt models following project conventions

Discovers existing patterns before writing
Runs dbt build after creation (not just compile)
Verifies output with dbt show
Handles staging, intermediate, and mart models

debugging-dbt-errors: Troubleshoots dbt compilation and runtime errors

Reads full error messages carefully
Checks upstream dependencies
Applies fixes and rebuilds
Stops after 3 failed attempts to reassess

testing-dbt-models: Adds schema tests to models

Studies existing test patterns in project
Matches project testing style
Covers uniqueness, not_null, relationships, accepted_values

documenting-dbt-models: Generates dbt model documentation

Analyzes model logic and columns
Creates descriptions for models and fields
Follows project documentation patterns

migrating-sql-to-dbt: Converts legacy SQL to dbt models

Parses raw SQL queries
Creates proper dbt model structure
Handles CTEs, refs, and sources

refactoring-dbt-models: Safely restructures dbt models

Tracks all dependencies before changes
Applies refactoring
Verifies downstream models still work

developing-incremental-models: Creates incremental dbt models

Selects appropriate strategy (append, merge, delete+insert)
Designs proper unique_key
Handles late-arriving data and edge cases

Snowflake Skills

finding-expensive-queries: Identifies costly Snowflake queries

Finds queries by cost, time, or data scanned
Ranks by impact
Provides query IDs for optimization

optimizing-query-by-id: Optimizes using Snowflake query history ID

Retrieves query profile from ID
Applies optimization patterns
Validates semantic preservation

optimizing-query-text: Optimizes raw SQL query text

Profiles query execution
Identifies bottlenecks (scans, joins, aggregations)
Applies anti-pattern fixes
Tests performance improvement

Delegation Skill

altimate-code: Delegates complex data tasks to altimate-code CLI

Verifies altimate-code installation
Invokes altimate-code run --yolo non-interactively
Reads output file and summarizes results
Requires: npm install -g altimate-code (Node 20+)

Skill Structure

Skills are markdown files with YAML frontmatter:

---
name: creating-dbt-models
description: |
  Guide for creating dbt models. ALWAYS use this skill when:
  (1) Creating ANY new model (staging, intermediate, mart)
  (2) Task mentions "create", "build", "add" with model/table
  (3) Modifying model logic or columns
---

Followed by workflow instructions:

# dbt Model Development

**Read before you write. Build after you write. Verify your output.**

## Critical Rules
1. ALWAYS run `dbt build` after creating models - compile is NOT enough
2. ALWAYS verify output after build using `dbt show`
3. If build fails 3+ times, stop and reassess your approach

## Workflow
1. Discover Conventions
   - Check existing models in same layer (staging/intermediate/mart)
   - Note naming patterns, CTE style, column ordering
   
2. Write Model
   - Follow discovered patterns
   - Use proper refs and sources
   
3. Build and Verify
   ```bash
   dbt build --select model_name
   dbt show --select model_name --limit 10


## Usage Examples

### Creating a dbt Model

```yaml
# User request:
"Create a staging model for raw customers data"

# Skill triggers: creating-dbt-models
# AI workflow:
# 1. Checks models/staging/ for naming patterns
# 2. Creates models/staging/stg_customers.sql
# 3. Runs: dbt build --select stg_customers
# 4. Verifies: dbt show --select stg_customers

Example model created:

-- models/staging/stg_customers.sql
with source as (
    select * from {{ source('jaffle_shop', 'customers') }}
),

renamed as (
    select
        id as customer_id,
        first_name,
        last_name,
        email,
        created_at
    from source
)

select * from renamed

Debugging dbt Errors

# User request:
"Fix this error: Compilation Error in model customers (models/customers.sql)"

# Skill triggers: debugging-dbt-errors
# AI workflow:
# 1. Reads full error message
# 2. Checks upstream model dependencies
# 3. Identifies issue (e.g., missing ref)
# 4. Applies fix
# 5. Rebuilds: dbt build --select customers

Optimizing Snowflake Query

# User request:
"This query is taking 5 minutes, can you optimize it?"

# Skill triggers: optimizing-query-text
# AI workflow:
# 1. Profiles query (EXPLAIN, execution stats)
# 2. Identifies bottlenecks (table scans, inefficient joins)
# 3. Applies optimizations (clustering, filters, join order)
# 4. Tests and measures improvement

Example optimization:

-- Before (slow)
SELECT 
    o.order_id,
    c.customer_name,
    SUM(oi.amount) as total
FROM orders o
JOIN customers c ON o.customer_id = c.customer_id
JOIN order_items oi ON o.order_id = oi.order_id
WHERE o.order_date >= '2024-01-01'
GROUP BY 1, 2;

-- After (optimized)
SELECT 
    o.order_id,
    c.customer_name,
    SUM(oi.amount) as total
FROM orders o
INNER JOIN order_items oi ON o.order_id = oi.order_id
LEFT JOIN customers c ON o.customer_id = c.customer_id
WHERE o.order_date >= '2024-01-01'
  AND o.order_date < '2024-12-31'  -- Added upper bound for partition pruning
GROUP BY 1, 2;
-- Assumes orders table is clustered by order_date

Using Kits

# Activate dbt-snowflake kit for a project
cd /path/to/dbt-project
altimate-code kit activate dbt-snowflake

# This configures:
# - All dbt and Snowflake skills
# - MCP server for dbt (live project access)
# - Project-specific instructions

# Deactivate when done
altimate-code kit deactivate

Kit configuration (.altimate/kits/active.yaml):

kit: dbt-snowflake
version: 1.0.0
skills:
  - creating-dbt-models
  - debugging-dbt-errors
  - testing-dbt-models
  - documenting-dbt-models
  - migrating-sql-to-dbt
  - refactoring-dbt-models
  - developing-incremental-models
  - finding-expensive-queries
  - optimizing-query-by-id
  - optimizing-query-text
mcp_servers:
  - dbt
instructions: |
  You are working on a dbt + Snowflake project.
  Always check project conventions before creating models.
  Run dbt build (not compile) after creating models.

Configuration

Skill Configuration

Skills are auto-triggered based on user requests. No explicit configuration needed.

MCP Integration

Skills work best with Altimate MCP server for live project access:

// claude_desktop_config.json or similar
{
  "mcpServers": {
    "altimate-dbt": {
      "command": "npx",
      "args": ["-y", "@altimate/mcp-server-dbt"],
      "env": {
        "DBT_PROJECT_DIR": "/path/to/dbt/project",
        "DBT_PROFILES_DIR": "/path/to/.dbt"
      }
    }
  }
}

MCP tools available to skills:

dbt_project_info: Project structure, model list, sources
dbt_model_details: Column types, dependencies, compiled SQL
dbt_compile: Compile models without CLI
snowflake_query_history: Recent query executions and stats
snowflake_table_stats: Row counts, clustering info

Common Patterns

Pattern 1: Multi-Step Model Creation

# User: "Create a mart model for monthly revenue by customer"
# Skills auto-chains:
# 1. creating-dbt-models: Creates initial model
# 2. testing-dbt-models: Adds schema tests
# 3. documenting-dbt-models: Adds documentation

Pattern 2: Debug-Fix-Verify Loop

# User: "My model won't compile"
# Skill: debugging-dbt-errors
# Workflow:
# 1. Read error
# 2. Check dependencies
# 3. Apply fix
# 4. Rebuild
# 5. If still fails, repeat up to 3x
# 6. If 3x fails, stop and reassess approach

Pattern 3: Safe Refactoring

# User: "Break this large model into smaller ones"
# Skill: refactoring-dbt-models
# Workflow:
# 1. Map all downstream dependencies
# 2. Create new intermediate models
# 3. Update refs in downstream models
# 4. Run dbt build on entire DAG
# 5. Verify no breakage

Pattern 4: Incremental Model Development

-- Pattern: merge strategy with unique_key
{{ config(
    materialized='incremental',
    unique_key='event_id',
    merge_update_columns=['status', 'updated_at']
) }}

select
    event_id,
    user_id,
    event_type,
    status,
    created_at,
    updated_at
from {{ source('events', 'raw_events') }}
{% if is_incremental() %}
    where updated_at > (select max(updated_at) from {{ this }})
{% endif %}

Pattern 5: Query Optimization Workflow

# Skill: optimizing-query-text
# Steps encoded:
# 1. Profile query
# 2. Identify bottleneck type:
#    - Full table scan → Add filters/clustering
#    - Inefficient join → Reorder, change type
#    - Large aggregation → Pre-aggregate or partition
# 3. Apply pattern-based fix
# 4. Validate semantics unchanged
# 5. Measure improvement

Troubleshooting

Skills Not Triggering

Problem: Skills don't activate for your request

Solution:

Be explicit: "Create a dbt model" not "make a model"
Check skill installation: /plugin list
Review trigger phrases in skill YAML
Use exact trigger phrases from skill description

dbt Build Failures

Problem: Models fail to build after creation

Solution:

Skill will auto-retry up to 3 times
After 3 failures, skill stops to reassess
Check dbt debug for connection issues
Verify refs and sources exist
Check for SQL syntax errors

MCP Server Not Connected

Problem: Skills can't access live project data

Solution:

# Check MCP server config
cat claude_desktop_config.json | grep altimate

# Verify dbt project path
export DBT_PROJECT_DIR=/correct/path
export DBT_PROFILES_DIR=/correct/.dbt/path

# Restart Claude Code

altimate-code Delegation Fails

Problem: "altimate-code not found" error

Solution:

# Install altimate-code CLI
npm install -g altimate-code

# Verify installation
altimate-code --version

# Check Node version (requires 20+)
node --version

Query Optimization No Improvement

Problem: Optimized query performs the same

Solution:

Skill checks for anti-patterns but can't fix all issues
May need manual clustering/indexing setup
Check if bottleneck is data volume (consider aggregation)
Review Snowflake warehouse size
Use finding-expensive-queries to compare before/after

Kit Activation Issues

Problem: Kit won't activate

Solution:

# Check kit is installed
altimate-code kit list

# Reinstall if needed
altimate-code kit install AltimateAI/data-engineering-skills

# Check for conflicting active kits
altimate-code kit status

# Deactivate others first
altimate-code kit deactivate

Performance

Benchmark results (ADE-bench, 43 real-world dbt tasks):

Baseline Claude: 46.5% accuracy (20/43 tasks)
Claude + Skills: 53.5% accuracy (23/43 tasks)
Model Creation: 3x improvement (40% → 65%)

Snowflake optimization (TPC-H SF1000, 62 queries):

Baseline: 77.4% pass rate, 4.7% avg improvement
With Skills: 83.9% pass rate, 16.8% avg improvement (3.6x better)

Additional Resources

Documentation: https://docs.myaltimate.com/
ADE-bench Framework: https://github.com/dbt-labs/ade-bench
altimate-code CLI: https://github.com/AltimateAI/altimate-code
dbt Slack: https://getdbt.slack.com/archives/C05KPDGRMDW
Contact: https://app.myaltimate.com/contactus

altimate-data-engineering-skills

About