Bhavik Pandit

Auditors, Not Oracles: Sparse Human Grounding for Bias-Aware Self-Alignment

Reinforcement learning from human feedback (RLHF) treats annotators as preference oracles. This is expensive and can encode annotator-pool bias into the reward signal: rater demographics, positional artifacts, and misspecified helpfulness criteria can become part of what the model is trained to produce. A growing body...