What should a linear optical frontend compute? Assessing the role of meta-optics, nonlocality, and coherence in hybrid inference systems

Abstract

Hybrid inference systems that pair an optical frontend with a digital backend offer a route to offload computation to the physical layer. Yet what the optics should compute, and when this is beneficial, has remained unclear. Here, we show that, for classification tasks, a well-designed optical frontend reshapes the statistics of the sensor intensity readout to improve class separability, quantified by the Bhattacharyya distance. This training-free metric predicts downstream accuracy and reveals that much of the discriminative information resides in inter-pixel correlations. We then identify the roles of coherence and different forms of nonlocality. Because the sensor measures intensity, a linear frontend produces features that are quadratic in the input field; however, only nonlocal, coherent optical systems can exploit the associated information. Such systems can yield significant performance gains, surpassing the best trained linear preprocessor. These results provide physical insights and new design principles for optimal optical--electronic inference systems.