As the artificial intelligence infrastructure landscape shifts from foundational model training to operational execution, developers of high-volume consumer products face intense economic and performance pressures. Inworld has officially launched Realtime Inference, a specialized first-party inference service designed to run open models with the strict latency, reliability, and cost-efficiency required by consumer-facing AI applications.
Inworld launched Realtime Inference, a first-party service providing optimized open models, intelligent routing, caching, and failover for consumer AI apps.
The service is accessible through an OpenAI-compatible API and Inworld’s Realtime Router, supporting integration with over 100 third-party models.
Core platform features include cache-aware serving, model-specific decoding for improved time-to-first-token latency, and production-grade observability.
Production measurements show performance improvements, such as a drop in median time to first token from 178ms to 161ms for Gemma 4 26B, and a 99.9995 percent completion rate for GLM 5.3 Flash across 20 million requests.
Consumer AI social simulation app Status scaled to one million users in 19 days while achieving a 20x cost reduction using Inworld's infrastructure.
The service complements Inworld's broader stack, which includes Realtime TTS, STT, Router, and managed compute APIs.
Consumer AI products introduce unique infrastructural challenges compared to enterprise batch tasks. Applications frequently process long conversational histories, memories, safety rules, and prompt instructions while users expect near-instantaneous responses. Furthermore, a single user-visible output often requires multiple concurrent model calls covering intent detection, safety screening, content retrieval, ranking, and speech generation.
To combat these bottlenecks, Inworld's Realtime Inference is built around complete interaction workflows rather than isolated tokens. Key architectural components include optimized open models from families such as Gemma, DeepSeek, GLM, Qwen, and MiMo, alongside cache-aware serving designed to reuse repeated prompt contexts and lower cached-input rates. Additionally, model-specific decoding enhancements maximize streaming pace and throughput without sacrificing response quality.
"Cost is the first thing standing in these developers’ way, so it is the first thing we are taking down," said Kylan Gibbs, CEO and co-founder of Inworld. "Pero a consumer app does not succeed on cheap inference alone. These teams need help with growth and distribution. We are building toward a model where a small team can come to Inworld and get the technology, the economics, and the support to build a real consumer business."
Real-world deployments demonstrate the platform's capacity to handle heavy, unpredictable consumer traffic. For instance, AI social simulation platform Status scaled from a prototype to one million users in just 19 days by leveraging Inworld's model and routing infrastructure. Fai Nur, CEO of Status, noted that the collaboration enabled a 20x cost reduction through combined prompt design, model selection, and intelligent routing.
To maintain visibility across high-volume deployments, Inworld's Realtime Router provides conditional routing, automatic provider failover, traffic splitting, and deep per-request observability. Developers can access the service immediately through the Inworld developer platform at inworld.ai/models.
About Inworld
Inworld is a research lab and inference provider focused on realtime AI for consumer-facing applications. The company builds first-party speech models, serves text, image, video, and decision models, and operates the infrastructure behind modular APIs for speech recognition, voice generation, routing, inference, realtime interactions, and compute. Inworld helps developers meet the quality, latency, reliability, and cost requirements of high-use consumer AI products.
