Imported from lzwei196/KISS-Knowledge-Infrastructure-for-Scientific-Simulation (models/Raven/SKILL.md). Install upstream with npx skills add lzwei196/KISS-Knowledge-Infrastructure-for-Scientific-Simulation --skill Raven. Copyright stays with the author.

MANDATORY EXECUTION POLICY — READ BEFORE PROCEEDING

You MUST run the actual model binary or package described in this document. If the model fails to import, compile, or execute, you MUST:

1. Check diagnostics/triplets.yaml for a matching error pattern
2. Try the fix suggested in the triplet's remedy section
3. If still failing, report the error to the user with full details

You MUST NOT substitute a simplified Python formula, regression equation, or hand-coded approximation in place of the real model. Doing so produces scientifically invalid results and defeats the purpose of the KI.

Before starting, run: python preflight_check.py (in this KI directory) to verify that the model binary/package and required data are available.

DEBUGGING PROTOCOL — When something goes wrong, follow this order:

1. Check triplets — diagnostics/triplets.yaml may already cover this error
2. Read official docs — The model's own documentation for expected formats/units
3. Find working examples — Check outputs/ or the model's shipped test data
4. Fix the tool — With knowledge of what "correct" looks like

Do NOT write custom debug scripts. The answers are in the docs and examples.

KI map — what to read, and when

Projected 2026-08-17 from the KI's actual contents — 10 components present. Refresh: python3 ki_tools_common/generate_skill_map.py --ki_dir <this KI>.

Executable tool index (projected — complete by construction)

Every public tool in this KI, by exact path. What each is FOR lives in the human-written Tool Inventory above; --help on any of these prints its arguments.

11 public tools; _-prefixed helpers and packaging files excluded.

Data Preparation

Forcing data

Data Sources: Use from ki_tools_common.load_forcing import load_daily_forcing for CMFD/MSWX/NASA POWER. Then convert to Raven .rvt format using this KI's tool: tools/s3_forcing/convert_forcing_to_rvt.py

Data Validation Reference: See data_ki/CMFD/SKILL.md for CMFD unit documentation and known traps.

Raven Hydrological Modelling Framework v4.1 — Knowledge Infrastructure

Package: hydrocraft-raven v1.0.0 Model: Raven v4.1 (University of Waterloo, Prof. James Craig) Created by: Jianyun Zhang Research Group, Hohai University Last updated: 2026-03-21 Stats: 11 tools | 7 skill documents | 28 diagnostic triplets | 9 error log entries | ~4,500 lines of validated Python Validation Status: production_validated -- Full HydroCraft data run on Bengbu basin (118,358 km2, CMFD 224 cells, 2000-2005)

Overview

This knowledge infrastructure enables fully autonomous construction and execution of the Raven Hydrological Modelling Framework on any basin. Raven is unique among HydroCraft models because it is a meta-framework that can emulate 15+ different hydrological model structures (GR4J, HBV, HMETS, SAC-SMA, HYMOD, UBC, etc.) from a single codebase.

What Raven does: Assembles custom hydrological models from a library of 120+ process algorithms. Each hydrological process (infiltration, ET, snowmelt, baseflow, routing) is an interchangeable module. Users select algorithms via the .rvi configuration file.

Key differentiator for HydroCraft: Run 8 model structures on the same basin, same forcing, same period and compare performance. This quantifies structural uncertainty — how much results vary by model choice. No other HydroCraft model can do this.

Models Raven Can Emulate (Level 1 — near-exact)

Installation

Binary

Raven v4.1 executable: KISSPATH_BINARIES/raven/Raven.exe
Source code:            KISSPATH_BINARIES/raven/*.cpp (118 files)
Makefile:               KISSPATH_BINARIES/raven/Makefile
User manual (PDF):      KISSPATH_BINARIES/raven/RavenUsersManual.pdf

Compilation

cd KISSPATH_BINARIES/raven
g++ -std=c++11 -Wno-deprecated -fPIC -O2 -o Raven.exe *.cpp
chmod +x Raven.exe

No external dependencies needed for basic mode. For NetCDF I/O support, uncomment -Dnetcdf in Makefile and link libnetcdf.

Python dependencies

numpy, pandas, geopandas, rasterio, shapely, netCDF4 (all in HydroCraft venv)

Pipeline (11 stages)

Total: 11 tools, ~4,200 lines of validated Python code.

Parallelism

• s1, s2, s3 can run in parallel after s0
• s5 depends on s1 and s2
• s8 (ensemble): all model runs execute in parallel (same forcing/HRU, different .rvi/.rvp)

6. Output Description

This section restates dag.yaml; if this section and dag.yaml ever disagree, dag.yaml wins. The headline output is the dag's validation_rank: 1 variable, which is the quantity this KI is judged by when observations are supplied.

Headline output:

discharge (STREAMFLOW) -- Simulated streamflow hydrograph at each gauged SubBasin outlet; observed series co-reported when supplied. (m3/s)

When asked what Raven predicts, answer first with discharge (STREAMFLOW). Use parse_raven_output.py for Hydrographs.csv because the raw file is period-ending and must be shifted to calendar dates before external metrics are computed.

8. Unit Conversion Table

Exact input/output shapes live in docs/format_spec.yaml. Raven performs no unit conversion on .rvt values, so these conversions must happen before model execution, normally in tools/s3_forcing/convert_forcing_to_rvt.py.

8c. Sign Conventions and Output Units

Site-adaptive s0 flags — READ BEFORE RUNNING A MOUNTAIN BASIN

select_model_template.py reads the .rvh and .rvt already staged in --output_dir, so stage the basin and forcing files BEFORE calling s0. It then makes three site-dependent decisions and reports each in its JSON:

With orographic corrections on, build_rvp_parameters.py emits ADIABATIC_LAPSE and PRECIP_LAPSE automatically (it queries the binary via :CreateRVPTemplate) and calibrate_raven_dds.py calibrates them.

calibrate_raven_dds.py takes --seed, --cal_start, --cal_end — always pass the calibration window, otherwise the objective is Raven's whole-simulation Diagnostics.csv and any held-out score you report afterwards is a fitted number. It reports skipped_parameters_absent_from_rvp: check that list, a name that is absent was never optimised. Each template's OWN melt parameters are calibrated (UBCWM multipliers, HMETS min/max melt factors, HBV refreeze) — a wider set needs a bigger --n_iterations, not fewer parameters (dt_rav_043).

Select the reported emulation with parse_raven_output.select_best_member(..., cal_key="cal") — ranking on the held-out window makes structure choice a form of tuning (dt_rav_036).

Tools Reference

Skill Documents

Critical Domain Knowledge

These non-obvious facts cause silent failures. Each is a diagnostic triplet.

1. RAVEN IGNORES UNITS (dt_001 through dt_004) — THE #1 TRAP

"Raven ignores units and will not do units conversion" — stated explicitly on the Raven cheat sheet. This means:

All conversions MUST happen in convert_forcing_to_rvt.py. The tool has bounds checking to catch violations.

2. Missing forcing filled with zeros silently (dt_010)

If PRECIP is not in the .rvt file, Raven fills it with zeros. Zero precip = zero discharge. No warning. Always verify PRECIP is present.

3. Snow template for snow basins (dt_012)

GR4J, HYMOD, and basic SAC-SMA have NO snow processes. In cold basins, all winter precipitation becomes rain, producing completely wrong seasonal patterns. Use HBV-EC, HMETS, or UBC for snow basins.

4. Class names must match exactly between .rvh and .rvp (dt_007, dt_008)

Soil profile names, land use classes, vegetation classes, and terrain classes in the .rvh HRU table must be defined in .rvp. Case-sensitive. Always generate .rvp AFTER .rvh using --rvh_file flag.

5. Forcing generators produce silent approximations (dt_025)

Raven has 40+ internal forcing generators that estimate missing variables. If WIND_VEL is not provided, Raven estimates it from latitude. These approximations are reasonable but can differ significantly from measurements. Check ForcingFunctions.csv output to see what was generated internally.

6. Cold start spinup (dt_023)

Discard first 1-2 years of simulation. Default initial conditions need equilibration time. Or use solution.rvc from a previous run for warm start.

7. Diagnostics require observed data (dt_011)

If no observed discharge is provided in .rvt, Diagnostics.csv reports -9999 for all metrics. This is not an error — it means "not computed."

8. .rvh HRU table requires EXACTLY 13 columns (err_001 — discovered during validation)

The HRU data lines in .rvh must have 13 comma-separated fields, not the 12 shown in the dissection plan. The undocumented 10th column is AQUIFER_PROFILE (set to [NONE] if not using aquifer model). Without it, Raven segfaults with "line N is wrong length."

Correct column order: ID, AREA, ELEVATION, LATITUDE, LONGITUDE, BASIN_ID, LAND_USE_CLASS, VEG_CLASS, SOIL_PROFILE, AQUIFER_PROFILE, TERRAIN_CLASS, SLOPE, ASPECT

HMETS default RUNOFF_COEFF: The default 0.3 is suitable for humid basins. For semi-arid basins, reduce to 0.1-0.2. This is a LandUseParameter, not a GlobalParameter.

9. :GlobalParameter is a single-line command, NOT a block (err_002)

Wrong: :GlobalParameter / PARAM1 val / :EndGlobalParameter Correct: :GlobalParameter PARAM1 val (one parameter per line, no block delimiters)

10. .rvc file is REQUIRED even if empty (err_004)

Raven v4.1 exits with "Cannot find or read .rvc file" if no .rvc exists. Create a minimal file with just a comment line. Use generate_rvc_initial.py.

11. Hydrographs.csv is PERIOD-ENDING — never label-join it to a gauge (dt_rav_034)

Hydrographs.csv, ForcingFunctions.csv and the precip rates in WatershedStorage.csv are period-ending: the row stamped date d holds the time-averaged value for the timestep preceding d (RavenUsersManual v4.1, "Output files"), and row 0 (stamped :StartDate) is only the initial condition. Observation input (:ObservationData) has been period-starting since v2.7, so a gauge value for calendar day d is written at date d and Raven prints it in the row stamped d+1. Consequence: joining a calendar-dated gauge series on Raven's raw date label scores sim(day d) against obs(day d-1) — a silent one-timestep lag in every external metric, and it will not match Raven's own Diagnostics.csv.

# THE canonical reader — applies the period-ending -> calendar-day correction.
sys.path.insert(0, f"{KI}/tools/s7_output")
from parse_raven_output import load_discharge_series, compute_water_balance
sim, raven_obs = load_discharge_series(f"{run_dir}/output", basin_name)   # calendar dates

Do not write your own read_hydrograph() helper in a run script; that is how the lag gets reintroduced. Proof the correction is right: after the shift, Raven's own (observed) column equals the raw gauge file on identical calendar dates (Tangnaihai, 3286 days, max abs diff 0.0).

12. WatershedStorage "Total [mm]" is NOT physical storage (dt_rav_035)

Total [mm] bundles the Cum. Losses to Atmosphere accumulator (6744.8 of 6828.6 mm after 12 alpine years at Tangnaihai). Using delta(Total) as ΔS double-counts ET and reports a ~77% closure FAIL on a run whose native MB Error [mm] closes to machine zero. Use the KI tool:

wb = compute_water_balance(f"{run_dir}/output", basin_name, start=..., end=...)
# P/ET/Q from Cum. Inputs / Cum. Losses to Atmosphere / Cum. Outflow;
# dS from the physical stores only. Also returns raven_mb_error_mm —
# if THAT is large, the fault is genuinely in the model/config.

CLI: parse_raven_output.py --water_balance --wb_start ... --wb_end ...

13. Never select the reported emulation on the held-out window (dt_rav_036)

Choosing "best of N emulations" by its validation-period score (or by Raven's Diagnostics.csv NASH_SUTCLIFFE, which spans the whole :ObservationData record, held-out years included) makes the reported held-out number a fitted statistic. Rank on the calibration window and report the held-out score of the cal-selected member:

from parse_raven_output import select_best_member
best, ranked = select_best_member(emulation_table, cal_key="cal", metric="nse")

run_ensemble_comparison.py now emits rank_basis (and a rank_warning when it had to fall back to the full-period diagnostic). Drive DDS with calibrate_raven_dds.py --cal_start/--cal_end so the objective itself never sees the held-out years.

Raven Input File Architecture

All files share a common prefix (basin name). Raven is invoked as:

./Raven.exe <basin_name> -o output/

Quick Start

# Activate HydroCraft venv
source KISSPATH_PYTHON_ENV/bin/activate

KI=KISSPATH_KI_ROOT/Raven/knowledge_infrastructure

# 1. Select template
python $KI/tools/s0_config/select_model_template.py \
    --template hbv_ec --basin_name chaohe \
    --output_dir outputs/chaohe_raven/ \
    --start_date 2000-01-01 --end_date 2010-12-31

# 2. Generate .rvh (basin/HRU definition)
python $KI/tools/s1_basin_setup/build_rvh_from_shapefile.py \
    --basin_shp data/shp/chaohe_zhangjiaofen_shp/chaohe_zhangjiaofen_boundary_shp/chaohe_zhangjiaofen_boundary.shp \
    --dem data/dem/china_dem_90m/china_dem_90m.tif \
    --output_dir outputs/chaohe_raven/ \
    --basin_name chaohe --strategy elevation_bands --n_bands 5

# 3. Generate .rvp (parameters)
python $KI/tools/s2_parameters/build_rvp_parameters.py \
    --template hbv_ec --basin_name chaohe \
    --rvh_file outputs/chaohe_raven/chaohe.rvh \
    --output_dir outputs/chaohe_raven/

# 4. Convert forcing to .rvt (CRITICAL — unit conversions happen here)
python $KI/tools/s3_forcing/convert_forcing_to_rvt.py \
    --forcing_dir outputs/chaohe_2000_2010_025deg/vic_temp/forcing/forcing_final \
    --grid_nc outputs/chaohe_2000_2010_025deg/vic_temp/grid/basin_grid.nc \
    --output_dir outputs/chaohe_raven/ --basin_name chaohe \
    --start_year 2000 --end_year 2010 --forcing_source cmfd

# 5. Generate .rvc (initial conditions)
python $KI/tools/s5_initial_conditions/generate_rvc_initial.py \
    --output_dir outputs/chaohe_raven/ --basin_name chaohe \
    --n_hrus 5 --climate semi_humid

# 6. Validate all inputs
python $KI/tools/common/validate_raven_inputs.py \
    --run_dir outputs/chaohe_raven/ --basin_name chaohe

# 7. Run Raven
python $KI/tools/s6_execution/run_raven.py \
    --run_dir outputs/chaohe_raven/ --basin_name chaohe

# 8. Parse output
python $KI/tools/s7_output/parse_raven_output.py \
    --output_dir outputs/chaohe_raven/output/ --basin_name chaohe

# 9. Multi-model ensemble (Raven's killer feature)
python $KI/tools/s8_ensemble/run_ensemble_comparison.py \
    --base_dir outputs/chaohe_raven/ --basin_name chaohe \
    --templates gr4j,hbv_ec,hmets,hymod,sac_sma \
    --start_date 2000-01-01 --end_date 2010-12-31

# 10. Compare with VIC
python $KI/tools/s10_coupling/raven_vic_comparison.py \
    --raven_hydro outputs/chaohe_raven/output/Hydrographs.csv \
    --vic_discharge outputs/chaohe_2000_2010_025deg/routing_param/rout_out/ZJF\ \ .day \
    --output_dir outputs/chaohe_comparison/ --basin_name chaohe

Diagnostic Triplets

25 triplets covering 7 failure domains. See diagnostics/triplets.yaml for full details.

11. Validated Results -- Bengbu Basin (Step 3 Production Validation)

Basin: Bengbu (Huai River, 118,358 km2, humid subtropical monsoon) Period: 2000-01-01 to 2005-12-31, CMFD forcing (224 grid cells aggregated to basin mean) Data: ALL inputs from HydroCraft global datasets (CMFD forcing, China DEM 90m, AVHRR land cover) Reference: VIC 5.1.0 + Lohmann routing (mean Q = 1,509 m3/s)

Performance Metrics -- Convention Bars

These bars restate docs/validation_convention.yaml; if this section and the convention file disagree, docs/validation_convention.yaml wins. The Bengbu production run below compares Raven to VIC, while the headline field bar for discharge (STREAMFLOW) applies to gauged discharge validation after spinup.

Ensemble Results (5 model structures, uncalibrated)

Validation Criteria (all PASSED)

Key Findings

1. MOHYSE is the best uncalibrated model for Bengbu (NSE=0.787, r=0.920). Its simple structure (10 params) matches the humid subtropical climate well.
2. HBV-Light has highest correlation (r=0.944) but overshoots (PBIAS=+49%) due to insufficient ET in default parameterization.
3. All 5 models capture monsoon seasonality correctly (56-66% of flow in Jun-Sep vs VIC's 61%).
4. Structural uncertainty (ensemble spread): mean Q ranges from 974 to 2,291 m3/s across models (CV=0.35), demonstrating Raven's value for quantifying model choice uncertainty.

Data Replacement Tracking (Step 2 Complete)

Errors Found During Validation

1. err_005 (CRITICAL): convert_forcing_to_rvt.py had WRONG VIC forcing column mapping. Tool assumed [PRECIP, TMAX, TMIN, WIND, SW, LW, PRESSURE] but actual VIC output from process_forcing.py is [AIR_TEMP, PREC, PRESSURE, SWDOWN, LWDOWN, VP, WIND]. Caused 44,551 mm/yr precip (44x too high), temperature read as 50C (pressure column). Fixed by correcting column indices.
2. err_006 (MODERATE): convert_forcing_to_rvt.py could not read HydroCraft VIC grid NC (uses y/x with 2D mask, not lat/lon). Fixed by adding y/x + mask support.
3. err_007 (MODERATE): build_rvp_parameters.py HRU parser used line.split() on comma-separated .rvh files, producing wrong class names. Fixed by detecting and handling CSV format.
4. err_008 (MODERATE): build_rvp_parameters.py read column [9] (AQUIFER_PROFILE = [NONE]) as terrain class instead of column [10]. Fixed by reading correct column index.
5. err_009 (CRITICAL): select_model_template.py GR4J template put GR4J_X1-X4 as :GlobalParameter but Raven v4.1 requires them as soil/land-use properties. Template also used :SWCanopyCorrect (not recognized in v4.1). Workaround: Used hand-crafted templates from working Chaohe run (run_ensemble_v2.py). Generic tools need refactoring.

Comparison with Other HydroCraft Models

File Structure

knowledge_infrastructure/
  SKILL.md                                    # This file (agent entry point)
  DISSECTION_PLAN.md                         # Planning document
  knowledge_infrastructure.yaml              # Schema-compliant package definition
  workflow/
    workflow.md                              # Pipeline workflow document
  tools/
    s0_config/select_model_template.py       # Select emulation template
    s1_basin_setup/build_rvh_from_shapefile.py  # Generate .rvh
    s2_parameters/build_rvp_parameters.py    # Generate .rvp
    s3_forcing/convert_forcing_to_rvt.py     # Convert forcing (UNIT-CRITICAL)
    s5_initial_conditions/generate_rvc_initial.py  # Generate .rvc
    s6_execution/run_raven.py                # Execution wrapper
    s7_output/parse_raven_output.py          # Parse output files
    s8_ensemble/run_ensemble_comparison.py   # Multi-model ensemble
    s9_calibration/calibrate_raven_dds.py    # DDS calibration
    s10_coupling/raven_vic_comparison.py     # Raven vs VIC comparison
    common/validate_raven_inputs.py          # Cross-file validation
  docs/
    s0_model_selection_skill.md
    s1_basin_hru_setup_skill.md
    s3_forcing_conversion_skill.md
    s4_process_algorithm_guide.md
    s8_model_intercomparison_skill.md
    s9_calibration_skill.md
    coupling_skill.md
  diagnostics/
    triplets.yaml                           # 25 diagnostic triplets
    error_log.yaml                          # Error log from real runs