The Grammar of Transformers: A Systematic Review of Interpretability Research on Syntactic Knowledge in Language Models
It is taken that TLMs encode a non-trivial amount of syntactic knowledge, which shows strong performance on formal syntactic phenomena, but weaker and more variable performance on phenomena at the syntax-semantics interface.
Nora Graichen, Iria de-Dios-Flores, Gemma Boleda
· arXiv.org · 2 citations