Towards Identifying the Dataset Biases Causing Phantom Transfer
This work embeds a dataset's completions with Sentence-BERT, subtract the embeddings of clean reference completions, and compares the result to an open vocabulary of candidate topics, observing that different teacher models appear to express the same bias through different vocabulary.
Jonas Jürß, Pietro Lió
· 0 citations