P. Kefalas

A Multi-Agent Benchmarking Framework for Evaluating the Performance of Large Language Models in Logic Programming

A configurable multi-agent framework for benchmarking LLMs in Prolog code generation that combines a Code Generator Agent, a deterministic execution layer using SWI-Prolog, and an evaluator based on the LLM-as-a-Judge paradigm that supports model-agnostic experimentation and evaluates outputs across functional correctn...