RT-J: Large-Scale Pretraining of Relational Transformers for Context-Efficient Predictions
This work introduces RT-J, an 86M-parameter relational foundation model (RFM) that makes accurate predictions even with hundreds of in-context examples or fewer, and highlights the importance of real-world semantics, context engineering and large-scale pretraining for RFMs.