The role of grammar in transformer encoders: Does it really matter for language understanding process?
Transformer-based methods achieve strong performance on language understanding tasks, yet the role of grammar in encoder representations remains insufficiently explained. While decoder-only models implicitly learn language structure through next-token prediction, encoder-only models still dominate many classification a...