ModelSphere

4Paradigm open-sources ModelSphere, a production LLM deployment platform that schedules across Chinese AI chips

4Paradigm published this on Oct 16, 2026 (source: github.com).

Hong Kong-listed 4Paradigm and Beijing Ruisi Zhilian have released ModelSphere, a free, Apache 2.0 platform on GitHub that combines compute management, model deployment and tuning for enterprise LLM serving. It works alongside engines such as vLLM and SGLang, adding cluster-level features like cache-aware request routing, a unified KV cache pool, prefill-decode disaggregation, autoscaling and automatic performance tuning. Through its EngineX inference engine it can schedule mixed Huawei Ascend, Iluvatar CoreX, Moore Threads and Hygon DCU hardware, and it supports models including DeepSeek V4, Qwen, Kimi, MiniMax and GLM. The companies claim about 285% higher peak throughput than a baseline.