DeepSeek has open-sourced DeepInfer, the engine that runs its services
Prepared by Sintelligent with the help of SI. Published in SI: . Updated: .
The engine behind a major SI service has become available to everyone, and running large models will get easier.
The DeepInfer repository on GitHub is described as a framework for large-scale inference of DeepSeek models; it is licensed under Apache 2.0 and supports Nvidia SM100 GPUs and Huawei Ascend 950 NPUs. According to the authors, the release contains offline inference and the core features for running at scale, but does not include some components needed for a full service. By default it uses the DeepSeek-V4.1-Flash-1017 checkpoint; the code is built mainly on vLLM with ideas from SGLang. The repository includes charts of prefill and decode throughput measured on 4 GPUs for prefill and 64 for decode.
Sources 2