Prompt file imported from nevedomski/SAS_to_PY (
.github/prompts/07-normalization-utils.prompt.md). Copyright stays with the author.
Scaffold normalization helpers to ensure deterministic DF comparisons
Dependencies:
- Requires schema contracts from
02-schema-contracts.prompt.md. - Feeds into
08-test-chunk-compare.prompt.mdand10-test-e2e.prompt.md.
Instructions:
Implement normalization utilities to ensure deterministic test comparisons. Include:
- Data type normalization (e.g., date formats, numeric precision).
- Column ordering enforcement.
- Missing value handling rules.
Create tests/test_utils/normalize.py with:
from __future__ import annotations
from typing import Iterable, Mapping
import pandas as pd
import numpy as np
def normalize_df(
df: pd.DataFrame,
*,
sort_keys: Iterable[str] | None = None,
column_order: Iterable[str] | None = None,
datetime_cols: Iterable[str] = (),
strip_string_cols: Iterable[str] | None = None,
) -> pd.DataFrame:
"""
Apply canonical ordering, dtype normalization, NA normalization, and optional string trimming.
Returns a new DataFrame.
"""
...
- Ensure immutability (don't mutate the argument in place).
- Avoid expensive copies where they are not needed.
- Ensure the helper is unit-test friendly and well-documented.