Robert Feldt

Foreword: Pre-Generation Prediction and Characterisation of LLM Test Failures using the Jacobian Lens

Testing LLM-based software is expensive: each input needs several sampled responses, each judged by a costly oracle. Further, every prompt or model change calls for re-testing. Pre-generation adequacy measures cut this cost by ranking inputs by failure risk based on the model's internal state. However, the state of the...