Quadrivium: Reconstructive Harmonization of U.S. Higher-Education Survey Data

Abstract

An integrated database harmonizing multiple NSF/NCSES higher-education surveys into reproducible analytical panels, using Reconstructive Harmonization: each era is reconstructed on its own terms, methodological discontinuities are decomposed into named, quantified components (real growth, definitional change, population expansion, bounded residual), and both are published with documentation a cold reader can use without misreading the discontinuity. Version 4.0 contains three datasets joined into one integrated object. (a) HERD — NSF Higher Education Research and Development survey data (FY 1972–2024) across the 2010 instrument redesign: a 50-year field-level R&D expenditure panel (FY 1975–2024), a Q15/Q16 research-personnel panel (FY 2022–2024), and a Q4/Q5 attribute sibling; this is the expenditure-OUT face (R&D spent, by institution / field / source class). (b) Federal S&E Support — a 53-year (FY 1971–FY 2023) institution-level panel of federal science-and-engineering obligations to higher-education institutions, drawn from the NSF Survey of Federal Science and Engineering Support to Universities, Colleges, and Nonprofit Institutions and carrying a native IPEDS UnitID; this is the funding-IN face (federal S&E dollars committed to individual higher-education institutions). (c) GSS — NSF Survey of Graduate Students and Postdoctorates in Science and Engineering (FY 1972–2024), carrying a native IPEDS UnitID across all 53 years: three long-format panels — graduate-student federal support by mechanism and federal agency, enrollment by degree level / gender / race, and postdoctoral plus non-faculty-researcher counts; this is the human-capital face (the federally-supported graduate students and postdocs that research funding trains). The value of the database is the integrated whole: HERD (expenditure-OUT), Federal S&E Support (funding-IN), and GSS (human-capital) are linked on the institution-year hub via the cross-survey institution-identity spine (and, for GSS, the native IPEDS UnitID it carries), so funding committed to an institution can be read against the R&D it spent and the researchers it trained — the funding → people → productivity picture, joined on the same federal agencies (NIH, NSF, DOD, DOE, USDA, NASA). The funding-IN ↔ expenditure-OUT join carries documented misalignments (federal-obligation fiscal-year basis vs. institution reporting-year basis, and institutional-grain differences); each survey also carries its own instrument discontinuities (the 2010 HERD redesign; the 2017 GSS redesign and the 1984–87 / 2014 GSS frame shifts), decomposed in the methods notes. Each dataset is validated against published NSF/NCSES ground truth; GSS reproduces published federal graduate-support counts for all 49 years (FY1975–2023) with zero mismatches and reconciles its 2023 cross-sections cell-for-cell. GSS field names are provisional in v4.0: the dedicated NCSES field-code reference was unavailable at release, so gss_code → field was reconstructed by count-matching each code's 2023 enrollment to published NSF totals — exact and unique for the 91 of 131 codes active in 2023; field_coarse is the Science / Engineering / Health super-broad only, and the ~40 historical-only codes are left NULL (no invented mappings) pending the reference, to be confirmed in a later patch. Version 4.0.1 corrects a defect in the HERD financial panel (herd_panel.parquet) present in v1.0.0-v4.0.0: 43,306 exact duplicate rows in FY2010-2023, caused by a year-range-blind crosswalk join that doubled seven era-B labels (aerospace, electrical engineering, and the five Geosciences rows). Geosciences field totals were double-counted in FY2010-2023 (e.g., FY2023 $8.07B vs the corrected $4.04B), and electrical/aerospace engineering sub-field totals were inflated; Engineering-all, Computer and information sciences, institution-level totals, FY2024, and the attribute, personnel, Federal S&E Support and GSS panels are unaffected. Users of v4.0.0 or earlier should re-pull herd_panel.parquet.