John D. Kelleher

Analysing the Linearity of Linguistic Relations in Language Model Embedding Spaces

We propose a framework to analyse how strongly different linguistic relations are linearly encoded in language model embedding spaces. We formalise linear encoding via a constrained linear approximation over related and unrelated word pairs and apply this to an extended BATS dataset covering inflectional, derivational,...