Autoregressive transformers remain comparatively weak for protein sequence and structure generation. We study the role of target representation: amino acid tokens encode residue identities without explicit contextual semantics, while backbone coordinates require a discrete representation in our framework. We introduce...
Mahdi Pourmirzaei, Farzaneh Esmaili, Amir Ziashahabi et al.· 0 citations
This paper derives closed-form optimal transforms for keys and values from calibration statistics, under a high-resolution model, and shows that the optimal key transform is not orthogonal and satisfies a generalized Parseval relation: the attention-aware distortion becomes mean-squared error in the transform domain.
Samuel Fernández-Menduiña, Amir Ziashahabi, Eduardo Pavez et al.· 2 citations
This work proposes ASPIRE, a non-synchronized batched self-speculative decoding framework built on three components, which achieves speedup in decoding throughput over autoregressive baselines and improves average speedup by approximately $27\% over the strongest prior self-speculative baselines.
Amir Ziashahabi, Hossein Entezari Zarch, Lei Gao et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.