Mehmet Emin Küçüker

LLM Inference Performance Optimization in Limited-Resource Environments

With the widespread adoption of LLM-based chatbots, cloud-dependent solutions have come to dominate the market. However, open-source pre-trained LLMs are enabling implementation of local solutions. Achieving competitive performance locally requires the ability to run high-parameter models. Here, the primary bottleneck...