Pedro Cuenca demonstrated running MiMo 2.6 Flash with native mxfp4 weights across an RTX 6000 GPU and an M5 laptop over 10 GbE at 40 tokens per second, utilizing out-of-the-box support in llama.cpp.
Prime Intellect has introduced Prime Inference and says it has served trillions of tokens for reinforcement learning and dedicated customer deployments.
Jiantao Jiao says an NVIDIA team built SWE-Serve to test whether AI agents can develop an inference engine and serve real models. The benchmark has 53 tasks drawn from real sgl_project engineering work.
SambaNova said SN50 is built for premium inference by pairing RDU decode with existing GPUs for prefill. It cited ROI in 6 months at a 70% billable utilization planning assumption.
