Self-Architecting Protein Transformers: An Empirical Study
This work asks whether a self-architecting transformer, which grows its own width and depth from quantitative signals derived from the attention matrices, can extract competitive protein representations from a single reference proteome.