ML Inference Acceleration via Scalable In-network Computing
Modern inference engines have significantly advanced the scalability of large models through innovations in serving techniques, proving instrumental in driving the widespread adoption of AI. However their true potential is constrained by traditional server architecture where reliance on host CPUs and operating systems...