Kenan Alkiek

HARISSA: Inference-Time Self-Checks for Efficient and Safe Local Language Model Deployment

Running a language model locally offers advantages in privacy, latency, and cost, but local hardware fits only small models, which are less capable than frontier models. The usual remedy for a hard query, escalating it to a cloud model, gives up the privacy and cost advantages of running locally. A deployment that stay...