Pformer: a plug-and-play periodicity-aware embedding for crystal property prediction

Abstract

Predicting macroscopic properties of crystalline materials from atomic structure remains a central challenge in computational materials science. We introduce Pformer, a modular crystal representation framework built around a Fourier-inspired source encoder. The source model is trained by supervised scalar-property regression from atom attributes and Cartesian coordinates; its FAN transformation applies sine, cosine, and GELU branches to learned latent projections before Transformer encoding. After source training, Pformer extracts the learned atom-embedding weight matrix, adapts its width when required, and uses it to initialize message-passing graph neural networks and transformer-based crystal encoders for downstream property prediction. We denote each initialized downstream variant as FAN-\emph{Backbone} (e.g., FAN-PotNet); here FAN identifies the source-encoder component that produced the transferred weights, whereas Pformer denotes the complete transfer framework. On the Materials Project benchmark, FAN-PotNet reduces the bandgap MAE from 0.204 to 0.149 eV, and FAN-GATGNN lowers the shear modulus MAE from 0.075 to 0.022 log(GPa). On JARVIS-DFT, FAN-PotNet decreases the total energy MAE from 0.032 to 0.017 eV/atom, and FAN-ComFormer improves the $E_{\text{hull}}$ MAE from 0.048 to 0.045 eV. These benchmark comparisons show lower MAE for selected backbone--property combinations and ties for others, indicating that the effect of FAN-derived initialization depends on the downstream architecture and target property.