Imported from ajmeyers42/loom (skills/weave-train/SKILL.md). Install upstream with npx skills add ajmeyers42/loom --skill weave-train. Copyright stays with the author.

Demo ML Designer

You are configuring Elastic ML jobs for a pre-sales demo. The goal is not just a working ML job — it's a job that produces a compelling anomaly at the right moment for the right entity, with enough baseline training data to make the anomaly visually obvious.

A demo ML job that doesn't show a clear red cell is worse than no ML job. Design for the wow moment first, then work backwards to the job config.

Step 1: Read the Required Inputs

Read both files before designing any job configuration:

• demo/{slug}-demo-script.md — required. Identifies ML scenes, anomaly narratives, and demo timing.
• data/{slug}-data-model.json — required. All detector field names, partition/by field names, datafeed source index names, and any geo_point fields that need shadow mappings must match the index mappings defined here exactly. A mismatch causes job-creation failures or datafeed schema errors at bootstrap time. If this file does not yet exist, stop and run weave-model first.

Step 2: Identify the ML Scenes

For each scene in the script that uses ML:

• What is the anomaly narrative? (e.g., "inventory gap grows faster than normal sales velocity", "unusual authentication volume from a single source", "SLA breach rate spikes outside normal pattern")
• What entity is anomalous? (store + SKU, IP address, user account, vendor, etc.)
• What metric is anomalous? (count, mean, sum, rare event, absence of event)
• What is the normal pattern the job must learn? (daily seasonal rhythm, per-entity baseline, steady-state rate)
• When in the demo does the anomaly need to appear? (T-minus from demo start, or live during the presentation)

Step 3: Design the Anomaly Detection Job

Detector selection

Choose the detector function that matches the anomaly narrative:

Partition by — always partition the detector by the entity that varies. This gives each entity its own baseline:

mean(discrepancy_pct) PARTITION BY store_id, sku_group

Influence fields — add fields that help explain the anomaly after detection:

"influencers": ["store_id", "sku", "event_type"]

Bucket span selection

The bucket span must match the granularity of the anomaly narrative:

• Event-level anomalies (fraud, auth spikes): 5m or 15m
• Hourly operational anomalies (inventory drift, SLA patterns): 1h
• Daily business anomalies (sales patterns, shrink): 4h or 1d

Rule: bucket span should be 1/10th to 1/20th of the anomaly's natural duration. If a shrink event builds over 4 hours, a 15m–30m bucket span detects it while it's still forming rather than after it's complete.

Minimum training data

The job needs enough data to learn a stable baseline before the demo anomaly appears. Calculate: training_hours = (anomaly_detection_lag_factor) * bucket_span_hours * 72

For most demo jobs: 24–48 hours of baseline data at realistic volume, with the anomaly injected in the last 2–4 hours before demo start.

Do not train on the anomaly period — the model must learn what "normal" looks like first.

Complete job config

{
  "job_id": "inventory-shrink-monitor",
  "description": "Detects abnormal inventory discrepancy growth by store and SKU group",
  "analysis_config": {
    "bucket_span": "1h",
    "detectors": [
      {
        "function": "mean",
        "field_name": "discrepancy_pct",
        "partition_field_name": "sku_group",
        "by_field_name": "store_id",
        "detector_description": "Mean discrepancy_pct by store and SKU group"
      }
    ],
    "influencers": ["store_id", "sku", "event_type"]
  },
  "data_description": {
    "time_field": "@timestamp",
    "time_format": "epoch_ms"
  },
  "model_plot_config": {
    "enabled": true,
    "annotations_enabled": true
  },
  "analysis_limits": {
    "model_memory_limit": "256mb"
  }
}

Datafeed config

{
  "datafeed_id": "datafeed-inventory-shrink-monitor",
  "job_id": "inventory-shrink-monitor",
  "indices": ["inventory-positions"],
  "query": {
    "bool": {
      "must": [
        { "range": { "stock_on_hand": { "gte": 0 } } },
        { "range": { "system_on_hand": { "gt": 0 } } }
      ]
    }
  },
  "scroll_size": 1000,
  "delayed_data_check_config": { "enabled": true }
}

Important — geo_point fields in datafeed source indices: If the datafeed source index contains a geo_point field (e.g., store_location), the ML datafeed will fail when it encounters it. Shadow the field via runtime_mappings in the datafeed config, or exclude it from _source:

"runtime_mappings": {
  "store_location": {
    "type": "keyword",
    "script": "emit('')"
  }
}

Serverless: ML field names differ from documentation

When querying .ml-anomalies-* on Serverless, the actual field names differ from what the self-managed documentation shows:

Before writing any dashboard panel or ES|QL query against .ml-anomalies-*, confirm field names with: GET .ml-anomalies-*/_mapping

Serverless: ML Anomaly Explorer UI not available

The Kibana ML Anomaly Explorer (swimlane UI) is not available on Serverless. Replace it with a custom Kibana dashboard panel using ES|QL or Lens querying .ml-anomalies-* directly. Update the demo script accordingly — any scene that references the swimlane UI must use a custom dashboard instead. The record_score field renders identically when displayed in a Kibana heatmap visualization.

Step 4: Design the Anomaly Injection Plan

This is what makes the demo work. Define exactly what data to insert, for which entities, at what time, to guarantee the anomaly appears.

Injection strategy — skew the signal:

For an inventory shrink scenario:

• Normal period (T-48h to T-2h): balanced POS/receive/stock events at realistic ratios (e.g., 3 sales per receive, discrepancy_pct drifts 5–15% by end of day)
• Anomaly period (T-2h to demo time): for the target store+SKU, suppress receive events, inject POS events at 3–5× normal velocity. discrepancy_pct climbs to 40–60% while typical value stays at 10–12%. Anomaly score crosses 75+ threshold.

For an authentication anomaly:

• Normal period: auth events at steady per-user rates (Poisson distribution)
• Anomaly period: one source IP generates 10–20× normal auth volume, plus new user accounts not seen in training period (rare detector fires)

Injection timing:

demo_start = T+0
anomaly_visible = T-30min (ML job processes up to current time)
inject_start = T-3h (gives model time to process and flag)
training_end = T-3h (clean separation between train and anomaly periods)

Injection specification document:

{
  "job_id": "inventory-shrink-monitor",
  "anomaly_scenario": "Inventory shrink — store 1842, SKU group: fasteners",
  "target_entities": [
    { "store_id": "1842", "sku": "174239", "anomaly_type": "high_discrepancy" },
    { "store_id": "1842", "sku": "182041", "anomaly_type": "high_discrepancy" },
    { "store_id": "1842", "sku": "196723", "anomaly_type": "high_discrepancy" }
  ],
  "normal_period": {
    "duration_hours": 46,
    "pos_receive_ratio": "3:1",
    "discrepancy_pct_range": [5, 15]
  },
  "anomaly_period": {
    "duration_hours": 2,
    "trigger_time": "T-2h from demo start",
    "pos_events_multiplier": 4,
    "receive_events": 0,
    "expected_discrepancy_pct": [40, 60],
    "expected_anomaly_score": [75, 95]
  },
  "verification": "Open ML swimlane 30 min before demo — confirm red cells on store 1842"
}

Step 5: NLP / Trained Model Config (if applicable)

When the script includes ELSER semantic search or NLP features:

ELSER v2 deployment (if not already in the data model):

For Serverless — use the managed endpoint (no model_id):

PUT /_inference/sparse_embedding/elser-v2-endpoint
{
  "service": "elser",
  "service_settings": {
    "num_allocations": 1,
    "num_threads": 1
  }
}

For ECH / self-managed — deploy the model explicitly:

PUT /_inference/sparse_embedding/elser-v2-endpoint
{
  "service": "elasticsearch",
  "service_settings": {
    "model_id": ".elser_model_2_linux-x86_64",
    "num_allocations": 1,
    "num_threads": 1
  }
}

Warmup query (run 30 min before demo — first inference is always slow):

POST /associate-knowledge-base/_search
{
  "query": {
    "semantic": {
      "field": "body_semantic",
      "query": "warmup query for model initialization"
    }
  },
  "size": 1
}

Wait for sub-2s response before considering the endpoint warm.

NER / classification models — if the script includes entity extraction or classification on text fields, specify:

• Model ID (from Elastic's model hub or a custom uploaded model)
• Target field and output field
• Inference pipeline processor config

Step 6: Write the Outputs

Output 1: data/{slug}-ml-config.json

Machine-readable collection of all ML artifacts:

{
  "anomaly_detection_jobs": [ { ...job config... } ],
  "datafeeds": [ { ...datafeed config... } ],
  "inference_endpoints": [ { ...endpoint config... } ],
  "anomaly_injection_plan": [ { ...per-entity injection spec... } ],
  "training_data_requirements": {
    "minimum_hours": 48,
    "anomaly_injection_start": "T-2h from demo",
    "verification_checklist": [ ... ]
  }
}

Output 2: data/{slug}-ml-setup.md

Step-by-step ML setup guide for the SE:

# ML Setup — [Demo Name]

## Jobs to Configure
[Table: job_id | detector | partition | bucket_span | index]

## Setup Sequence
1. Verify index has data (minimum N documents)
2. Create job via API or Kibana ML UI
3. Create and start datafeed
4. Wait for first bucket to complete (bucket_span time)
5. Run datafeed on historical data (open job → start datafeed from T-48h)
6. Verify swimlane shows clean baseline (no anomalies in normal period)
7. Run anomaly injection script
8. Wait 2× bucket_span for anomaly to surface
9. Verify red cells appear on target entities

## Anomaly Injection
[Exact script command or API calls to inject the demo anomaly]

## Verification Checklist
- [ ] Swimlane open in Kibana — confirm correct time range
- [ ] Red/orange cells visible on [specific store/entity]
- [ ] Anomaly score ≥ 75 on at least one record
- [ ] `typical` value visible and clearly lower than `actual`
- [ ] ES|QL join query returns the same SKUs as the ML anomaly

What Good Looks Like

Inventory shrink pattern: mean(discrepancy_pct) PARTITION BY sku_group BY store_id, 1h bucket, 48h training on 4 stores. Three specific SKUs at store 1842 injected with 4× POS velocity and 0 receives for 2 hours pre-demo. Expected: anomaly score 85–95, swimlane shows 3 red cells for store 1842 fasteners group. All other stores stay green.

Auth anomaly pattern (SOC): high_count PARTITION BY source_ip, 15m bucket, 24h training on per-IP auth patterns. Inject 1,000 auth attempts from a single IP that had zero history in training. rare detector also fires. Anomaly visible within two buckets (30 min) of injection start.

SLA breach pattern (financial): high_mean(days_since_filed) PARTITION BY claim_type BY vendor_id, 4h bucket. Inject vendor VND-0412 online debit claims with day 8–11 values (above SLA) while other vendors stay at day 1–3. Anomaly score crosses threshold after one bucket (4h). Time the injection for T-8h from demo so the swimlane is populated by demo time.