Imported from caixinyu2017-star/xixi (plugins/auto-empirical-research-skills/skills/17-DAAF-Contribution-Community-daaf/dot-claude/skills/education-data-source-scorecard/SKILL.md). Install upstream with npx skills add caixinyu2017-star/xixi --skill education-data-source-scorecard. Copyright stays with the author.

Scorecard Data Source Reference

College Scorecard — the primary institutional-level source for post-enrollment labor market outcomes, linking NSLDS financial aid records to IRS/Treasury earnings data. Use when comparing institutions on post-graduation earnings, loan repayment, or student debt, or when actual tax-record-based earnings are required rather than survey estimates. Covers six sub-datasets accessed via Portal mirrors. Critical limitation: tracks only Title IV federal aid recipients, not all students.

Federal data on post-college outcomes including earnings, debt, and repayment for students who received Title IV financial aid. Links education records to IRS tax data for actual earnings, making it the primary source for post-college labor market outcomes.

CRITICAL: Value Encoding and Missing Data

The Education Data Portal uses integer encodings for all categorical variables and lowercase, restructured variable names that differ from the original Scorecard column names. Suppression encoding differs by dataset:

• Earnings/counts: -3 integer code is the primary suppression indicator
• Yes/No flags (institutional characteristics): null for missing, 0/1 for valid
• Rates (repayment, default): null for missing
• The original Scorecard string "PrivacySuppressed" does NOT appear in Portal data

Context	pred_degree_awarded_ipeds	HBCU / tribal flags	religious_affiliation
Portal (integer)	0-4	0 / 1	Integer codes 22-200
Original Scorecard	String labels	String labels	String labels

See ./references/variable-definitions.md for complete encoding tables.

What is College Scorecard?

• Publisher: U.S. Department of Education
• Primary value: Post-college labor market outcomes (earnings) and debt/repayment metrics
• Data sources: NSLDS (loans/aid), IRS/Treasury (earnings), IPEDS (institutional characteristics)
• Coverage: Title IV federal aid recipients only — not all students
• Unique feature: Links education to IRS tax records for actual earnings data
• Access: Education Data Portal mirrors (parquet/CSV); see datasets-reference.md for paths, mirrors.yaml for mirror config, fetch-patterns.md for fetch code
• Primary identifier: unitid (IPEDS institution ID)

Reference File Structure

Decision Trees

What outcome am I researching?

Outcome type?
├─ Post-college earnings
│   ├─ Institution-level → ./references/earnings-data.md
│   └─ By field of study → ./references/field-of-study.md
├─ Student debt levels
│   ├─ Cumulative borrowing → ./references/debt-repayment.md
│   └─ Debt by field → ./references/field-of-study.md
├─ Loan repayment/default
│   └─ Repayment rates → ./references/debt-repayment.md
├─ Completion rates
│   └─ Scorecard completion → ./references/completion-rates.md
└─ Understanding limitations
    ├─ Who is included → ./references/population-coverage.md
    └─ Data quality issues → ./references/data-quality.md

How do I interpret this data?

Interpretation question?
├─ Why are earnings suppressed?
│   └─ Privacy thresholds → ./references/data-quality.md
├─ What does "6-year earnings" mean?
│   └─ Cohort timing → ./references/earnings-data.md
├─ Why don't Scorecard rates match IPEDS?
│   └─ Different cohorts → ./references/completion-rates.md
├─ What loans are included in debt?
│   └─ Federal only → ./references/debt-repayment.md
└─ How representative is this data?
    └─ Title IV coverage → ./references/population-coverage.md

Building a query?

Query construction?
├─ Variable names and codes → ./references/variable-definitions.md
├─ Suppression flags to handle → ./references/data-quality.md
├─ Understanding cohort years → ./references/earnings-data.md
└─ Field-level queries → ./references/field-of-study.md

Quick Reference: Scorecard Variables

Portal Data Structure (CRITICAL)

The Portal uses LONG format with time horizon as a column, NOT the WIDE format from original Scorecard bulk download files. Portal column names are all lowercase and differ significantly from original Scorecard names.

Earnings Dataset Columns (Actual Portal Names)

Source dataset: scorecard/colleges_scorecard_earnings (203,066 rows x 33 columns)

Key Identifiers

Data Timing

"After entry" means after first enrollment, not after graduation.

Categorical Value Encodings (Institutional Characteristics Dataset)

Missing Data Codes

import polars as pl

# Filter for valid earnings (handle -3 suppression code)
valid = df.filter(
    (pl.col("earnings_med").is_not_null()) &
    (pl.col("earnings_med") != -3)
)

# Filter for 6-year earnings specifically
six_yr_valid = valid.filter(pl.col("years_after_entry") == 6)

Data Access

Datasets for Scorecard are available via the mirror system. See datasets-reference.md for canonical paths, mirrors.yaml for mirror configuration, and fetch-patterns.md for fetch code patterns.

Codebooks are .xls files co-located with data in all mirrors. Use get_codebook_url() from fetch-patterns.md to construct download URLs.

Truth Hierarchy: When interpreting variable values, apply this priority:

1. Actual data file (what you observe in the parquet/CSV) — this IS the truth
2. Live codebook (.xls in mirror) — authoritative documentation, may lag
3. This skill documentation — convenient summary, may drift from codebook

If this documentation contradicts the codebook, trust the codebook. If the codebook contradicts observed data, trust the data and investigate.

All Scorecard Datasets (6 total)

Scorecard naming note: Data file paths differ significantly from codebook paths. Notable mismatches: data repayment_fsa vs codebook default; data inst_characteristics vs codebook institutional-characteristics; data repayment_nslds vs codebook repayment; data student_body_nslds vs codebook student-characteristics_aid-applicants; data student_body_treasury vs codebook student-characteristics_home-neighborhood. Always use the exact paths shown above.

Fetching Data

import polars as pl
from fetch_utils import fetch_from_mirrors  # See fetch-patterns.md

# Fetch earnings data
earnings = fetch_from_mirrors("scorecard/colleges_scorecard_earnings")

# Filter by time horizon (LONG format — filter, don't use wide column names)
six_yr = earnings.filter(pl.col("years_after_entry") == 6)

# Filter for valid earnings (exclude -3 suppression code)
valid = six_yr.filter(
    (pl.col("earnings_med").is_not_null()) &
    (pl.col("earnings_med") != -3)
)

# Institution names/control are NOT in the earnings dataset.
# Join to inst_characteristics or IPEDS directory:
inst = fetch_from_mirrors("scorecard/colleges_scorecard_inst_characteristics",
                          years=[2020])
valid = valid.join(
    inst.select("unitid", "inst_name", "pred_degree_awarded_ipeds"),
    on="unitid", how="left"
)

Common Pitfalls

Critical Limitation: Title IV Recipients Only

The single most important caveat for all Scorecard analysis:

Scorecard tracks ONLY students who received federal financial aid (Title IV):

• Pell Grants
• Federal student loans (Direct, Perkins, PLUS)
• Federal work-study

Coverage varies dramatically by institution type:

Data systematically overrepresents lower-income students who are more likely to need federal aid.

What Scorecard Data Does NOT Include

Comparison: Scorecard vs IPEDS

Related Data Sources

Topic Index