DeepSeek Open-Sources DeepInfer Inference Framework

On October 20, Hangzhou-based AI company DeepSeek announced that it is fully open-sourcing DeepInfer, the internal inference framework that powers its large-scale online AI services. The code is released under the Apache 2.0 license on GitHub. DeepInfer is the engine behind the company’s V4.1-Flash model and is paired with an open-source token codec component called deepseek-recipe.

DeepSeek Fully Opens DeepInfer Framework Used for V4.1 Flash

DeepSeek describes DeepInfer as a collection of high-performance kernels and a deeply optimized inference engine. The framework supports distributed inference, speculative decoding, structured outputs, asynchronous scheduling, batch-invariant inference, and deterministic sampling. It runs on both NVIDIA SM100-generation GPUs and Huawei Ascend 950 NPUs, selectable via the environment variable `DEEPINFER_ACCELERATOR`.

DeepSeek Open-Sources DeepInfer Inference Framework, Discloses approximately $38 in Revenue per GPU per Hour — BigGo Finance Global TimesDeepSeek Fully Open-Sources Inference Framework: NVIDIA & Huawei Ascend Compatible, 155T Tokens Daily Single-Cluster Throughput, $38 Per Card Per Hour Revenue DeepSeek Open-Sourced the Engine That Runs Its Models – on Nvidia and Huawei Silicon Alike DeepSeek Fully Opens DeepInfer Framework Used for V4.1 Flash DeepSeek open-sources DeepInfer software to curb AI deployment costs DeepSeek Open Sources Inference Tech to Lower AI Costs

According to DeepSeek’s technical report, a single DeepInfer-enabled cluster serves an average of 155 trillion tokens per day and a GPU generates roughly $38 in revenue per hour at off-peak electricity rates. The design separates pre-fill and decode stages, allowing independent resource allocation: eight-card machines handle pre-fill, while 64- to 128-card supernodes handle decode. KV-Cache reuse (up to seven days for agent traffic and three days for chatbot traffic) and a “SWA Bounded Replay” mechanism further reduce redundant computation.

DeepSeek Fully Open-Sources Inference Framework: NVIDIA & Huawei Ascend Compatible, 155T Tokens Daily Single-Cluster Throughput, $38 Per Card Per Hour Revenue

DeepSeek positions DeepInfer as a tool to lower compute costs and energy consumption for AI inference. By improving hardware utilization and reducing per-request latency, the company argues that operators can extract more output from existing compute investments. The company’s pricing for the V4.1-Flash model is $0.035 per million input tokens and $0.12 per million output tokens (as listed on OpenRouter), markedly lower than quoted U.S. competitor rates of $10–$50 per million tokens. Independent benchmarking data are not yet available; all performance figures are reported by DeepSeek.

DeepSeek Fully Open-Sources Inference Framework: NVIDIA & Huawei Ascend Compatible, 155T Tokens Daily Single-Cluster Throughput, $38 Per Card Per Hour Revenue

The same day as the open-source release, the U.S. Department of Homeland Security (DHS) issued an exclusion order that bars federal civilian agencies from acquiring or using DeepSeek products and requires removal of existing DeepSeek tools from agency IT systems. DHS cites “multiple threat vectors” and the company’s location in the People’s Republic of China as security concerns. The order applies only to federal civilian agencies; private sector use remains unrestricted.

DeepSeek Open Sources Inference Tech to Lower AI Costs

• The company added that future model versions will be added to the framework as they become publicly available.

DeepSeek Fully Opens DeepInfer Framework Used for V4.1 Flash

• DHS framed the exclusion order as a protective measure against potential security risks associated with foreign-developed AI models.

DeepSeek Open Sources Inference Tech to Lower AI Costs

• “We’re sharing our production stack to help the community lower compute costs, reduce energy use and make AI more affordable,” — DeepSeek spokesperson

DeepSeek Open Sources Inference Tech to Lower AI Costs

• “From private deployments to public clouds, we’ll keep working with the community to make large-scale inference more efficient,” — DeepSeek spokesperson

DeepSeek Open Sources Inference Tech to Lower AI Costs

• Performance benchmarks: DeepSeek provides internal test results (e.g., 4 SM100 GPUs for pre-fill, 64 for decode) but no independent third-party measurements have been published.

DeepSeek Open-Sourced the Engine That Runs Its Models – on Nvidia and Huawei Silicon Alike

• Cost impact: While DeepSeek claims substantial margin relief for cloud providers, the absence of external validation leaves the actual savings unverified.

DeepSeek Open Sources Inference Tech to Lower AI Costs

The open-source release of an inference engine that runs on both U.S. and Chinese silicon creates a shared technical foundation that could lower barriers for developers worldwide. At the same time, the DHS exclusion order highlights growing geopolitical tension over AI supply chains, forcing U.S. federal entities to weigh security concerns against potential cost advantages. The juxtaposition of open-source collaboration and regulatory restriction underscores the complex interplay between technological openness and national security policy in the rapidly evolving AI ecosystem.

Gizmodo