About

Imported from borghei/claude-skills (engineering/senior-data-scientist/SKILL.md). Install upstream with npx skills add borghei/claude-skills --skill senior-data-scientist. Copyright stays with the author (MIT + Commons Clause).

Senior Data Scientist

Name: senior-data-scientist
Author: borghei

Expert data science for statistical modeling, experimentation, ML deployment, and data-driven decision making — A/B test design and analysis, feature engineering, model training/evaluation, production deployment, and causal inference.

Keywords

data-science, machine-learning, statistics, a-b-testing, causal-inference, feature-engineering, mlops, experiment-design, model-deployment, python, scikit-learn, pytorch, tensorflow, spark, airflow

Core Capabilities

Experiment design & analysis — hypothesis framing, power analysis and sample sizing, randomization, SRM monitoring, and post-hoc significance testing.
Feature engineering — profiling, candidate generation (temporal/aggregation/interaction/text), selection (variance, correlation, SHAP/RFE), and leakage validation.
Model training & evaluation — stratified/temporal splits, baselines, hyperparameter tuning, cross-validation, calibration, and fairness checks.
Production deployment — containerized serving, input/output drift monitoring (KS/PSI), canary rollouts, and latency/error SLAs.
Causal inference — propensity score matching, difference-in-differences, regression discontinuity, instrumental variables, and assumption/placebo testing.

When to Use

Designing or analyzing an A/B test.
Building a feature engineering pipeline.
Training, evaluating, or deploying an ML model.
Estimating treatment effects from observational data.

Clarify First

Before running an analysis or pipeline, confirm these inputs. If any is unknown or vague, ASK — do not assume:

Task — A/B test design / feature engineering / model evaluation / causal inference (selects the script and workflow)
Dataset & target variable — what you are modeling or measuring (drives feature generation and leakage validation)
Decision metric & minimum effect — the metric and the smallest effect worth detecting (drives power analysis and sample size)

Stop rule: ask only the 2-3 that most change the output. If the user says "just draft it," proceed and list your assumptions at the top of the artifact.

Tools

Script	Purpose
`scripts/experiment_designer.py`	A/B test design, power analysis, sample size calculation
`scripts/feature_engineering_pipeline.py`	Automated feature generation, correlation analysis, feature selection
`scripts/statistical_analyzer.py`	Hypothesis testing, causal inference, regression analysis
`scripts/model_evaluation_suite.py`	Model comparison, cross-validation, deployment readiness checks

statistical_analyzer.py is referenced but not yet present in the repo — see the note in references/ds-operations.md. Use inline scipy/statsmodels in the meantime.

References

Load the reference that matches the task — keep this file lean and pull detail on demand:

references/ds-workflows.md — quick-start commands, tech stack, the five end-to-end workflows (A/B testing, feature pipeline, train/evaluate, deploy, causal inference) with Python snippets, performance targets, and common commands. Read when executing any data-science task.
references/ds-operations.md — troubleshooting table, success criteria, and the full CLI flag reference for each script. Read when diagnosing issues or running the tools.
references/statistical_methods_advanced.md — advanced statistical methods reference (hypothesis testing, causal inference, regression). Read for statistical depth.
references/experiment_design_frameworks.md — experiment design frameworks and power-analysis foundations. Read when designing rigorous experiments.
references/feature_engineering_patterns.md — feature engineering patterns and selection techniques. Read when building features.

Scope & Limitations

This skill covers:

End-to-end experiment design including power analysis, randomization, and post-hoc analysis
Feature engineering pipelines with profiling, generation, selection, and validation
Model training evaluation including cross-validation, calibration, and fairness checks
Production model deployment with monitoring, drift detection, and canary rollouts

This skill does NOT cover:

Data engineering infrastructure (ETL orchestration, pipeline scheduling, data lake management) -- see senior-data-engineer
Deep learning model architecture design and training at scale (distributed GPU training, custom layers) -- see senior-ml-engineer
Prompt engineering, RAG systems, and LLM fine-tuning workflows -- see senior-prompt-engineer
Computer vision pipelines (object detection, segmentation, video processing) -- see senior-computer-vision

Integration Points

Skill	Integration	Data Flow
`senior-data-engineer`	Feature pipeline ingests data from ETL outputs; shares data quality validation patterns	Raw data stores --> feature engineering pipeline --> feature store
`senior-ml-engineer`	Trained models handed off for MLOps deployment; shares model registry and serving configs	Evaluated model artifacts --> deployment pipeline --> production serving
`senior-prompt-engineer`	Embedding features from LLMs feed into ML pipelines; experiment frameworks apply to prompt A/B tests	LLM embeddings --> feature vectors; experiment designs --> prompt evaluation
`senior-architect`	Model serving architecture reviewed for scalability; data platform design aligned with training infrastructure	Architecture specs --> deployment topology --> monitoring dashboards
`senior-backend`	Model inference endpoints integrated into backend services; API contracts defined for prediction requests	REST/gRPC model API --> backend service layer --> client applications
`senior-devops`	CI/CD pipelines extended for model retraining triggers; containerized model images deployed via infrastructure-as-code	Docker images --> Kubernetes manifests --> production clusters

Use it

Copy one of these into your project. Installing also returns the manifest and these snippets.

yaml

targets:
  - https://api.opensmartroute.ai/api/v1/registry/borghei-claude-skills-senior-data-scientist/manifest   # or paste the manifest below

Manifest

An Open Capability Manifest: the router reads it to know what this does, what it costs and when to pick it.

borghei-claude-skills-senior-data-scientist.ocm.jsonjson

{
  "ocm": "1",
  "id": "borghei-claude-skills-senior-data-scientist",
  "kind": "skill",
  "name": "senior-data-scientist",
  "description": "Expert data science for statistical modeling, experimentation, ML deployment, and data-driven decision making — A/B test design and analysis, feature engineering, model training/evaluation, production deployment, and causal inference.",
  "publisher": "borghei",
  "version": "1.0.0",
  "capabilities": {
    "domains": [
      "general"
    ],
    "tags": [
      "skill-md",
      "data-science",
      "ml",
      "statistics",
      "experimentation",
      "python",
      "mlops",
      "skills-sh"
    ],
    "languages": [
      "en"
    ]
  },
  "quality_prior": 0.6,
  "examples": [
    "Expert data science for statistical modeling, experimentation, ML deployment, and data-driven decision making — A/B test design and analysis, feature engineering, model training/evaluation, production deployment, and causal inference."
  ],
  "primary": false,
  "metadata": {
    "source": {
      "provider": "skills.sh",
      "repository": "https://github.com/borghei/claude-skills",
      "path": "engineering/senior-data-scientist/SKILL.md",
      "ref": "HEAD",
      "url": "https://github.com/borghei/claude-skills/blob/HEAD/engineering/senior-data-scientist/SKILL.md",
      "key": "borghei/claude-skills/engineering/senior-data-scientist/SKILL.md"
    },
    "license": "MIT + Commons Clause"
  },
  "instructions": "# Senior Data Scientist\n\nExpert data science for statistical modeling, experimentation, ML deployment, and data-driven decision making — A/B test design and analysis, feature engineering, model training/evaluation, production deployment, and causal inference.\n\n## Keywords\n\ndata-science, machine-learning, statistics, a-b-testing, causal-inference,\nfeature-engineering, mlops, experiment-design, model-deployment, python,\nscikit-learn, pytorch, tensorflow, spark, airflow\n\n## Core Capabilities\n\n- **Experiment design & analysis** — hypothesis framing, power analysis and sample sizing, randomization,",
  "cost": {
    "context_tokens": 1558
  }
}

Fetch it by URL: GET /api/v1/registry/borghei-claude-skills-senior-data-scientist/manifest?version=1.0.0

Reviews

Star ratings from people who tried it. One review per account; edit yours any time.

No reviews yet. Install it, try it, and be the first to rate it.