DeepInfer vs LMDeploy: DeepSeek V4.1-Flash Serving Compared

DeepInfer and LMDeploy are both optimized for DeepSeek models, but they are not the same product. DeepInfer is DeepSeek's Apache-2.0 engine for one model, DeepSeek-V4.1-Flash, with an offline-only release and support for NVIDIA SM100 plus Huawei Ascend 950. LMDeploy is InternLM's open-source toolkit for compressing, deploying, and serving LLMs, with its own TurboMind and PyTorch engines, a full OpenAI-compatible API server, and documented DeepSeek V3/R1 work. There is no public head-to-head for V4.1-Flash.

The short answer

Choose LMDeploy if you need a served API today, want an existing DeepSeek deployment path, or need an Ascend-capable stack with a mature serving layer. Consider DeepInfer if you specifically want DeepSeek's own V4.1-Flash engine, its DSpark/SWA/EPLB implementation, and you are prepared to build the service layer yourself.

The key caveat: LMDeploy's README documents DeepSeek V3/R1, not V4.1-Flash. Do not assume the new model is supported just because the older DeepSeek models are. Check the current support matrix and the engine — TurboMind vs PyTorch — before committing.

What each one is

DeepInfer

DeepInfer is DeepSeek's open-source inference framework for serving DeepSeek-V4.1-Flash at scale. It is Apache-2.0 and supports NVIDIA SM100 and Huawei Ascend 950. It ships an offline engine with optimized kernels, Wide-EP/EPLB, DSpark, SWA bounded replay, asynchronous scheduling, deterministic modes, and structured outputs.

Its release omits the KV cache system, prefill/decode disaggregation, and data-parallel load balancing. It supports only one model. The value is the DeepSeek-specific tuning and the visibility into a production inference stack.

LMDeploy

LMDeploy is an open-source toolkit for compressing, deploying, and serving LLMs. It provides two engines: TurboMind, its high-performance CUDA engine, and a PyTorch engine with broader hardware support. Its documented features include persistent batching (continuous batching), blocked KV cache, tensor parallelism, high-performance CUDA kernels, KV-cache quantization, AWQ, and automatic prefix caching.

For DeepSeek, LMDeploy documents V3/R1 support, DeepSeek optimizations such as FlashMLA, DeepGemm, DeepEP, MicroBatch, and EPLB, and PD disaggregation through DLSlime and Mooncake. It also ships an OpenAI-compatible API server with /v1/chat/completions, /v1/models, and /v1/completions. The PyTorch engine documents Huawei Ascend support.

Head-to-head: scope, not speed

Sources: DeepInfer README/Q&A; LMDeploy README and OpenAI-compatible API docs. Accessed 2026-10-20.

Where DeepInfer wins

• V4.1-Flash is the design target. The engine is built around V4.1's CED architecture, DSpark draft path, SWA bounded replay, and Wide-EP decode layout. LMDeploy's public DeepSeek work is still described around V3/R1.
• DeepSeek's own production notes. The Q&A documents why operations are split 64+64 in the reference deployment, how EPLB reacts to hot experts, and how the KV cache design works. That is a different kind of evidence from a general toolkit's feature list.
• Deterministic modes. Batch-invariant inference and deterministic sampling (NVIDIA only) are exposed. If you need reproducible evaluation or auditing, that matters.
• Ascend 950 in the official release. LMDeploy also has an Ascend path, but DeepInfer lists Ascend 950 as a first-class target alongside SM100.

Where LMDeploy wins

• A real serving layer. OpenAI-compatible endpoints, request handling, and a documented API server. DeepInfer stops at offline inference.
• PD disaggregation already integrated. LMDeploy documents DeepSeek PD deployment through DLSlime and Mooncake. DeepInfer leaves PD to the operator.
• Quantization and cache controls. KV-cache quantization, AWQ, and prefix caching are part of the toolkit. DeepInfer's release does not include a service-level KV cache.
• DeepSeek V3/R1 experience. If you are serving those models today, LMDeploy has a documented path. V4.1 is the open question, not the baseline.
• Hardware breadth. NVIDIA CUDA plus a PyTorch-engine Ascend path; TurboMind is the CUDA-focused engine.

The benchmark trap

LMDeploy's README claims “up to 1.8× higher request throughput than vLLM” using continuous batching, blocked KV cache, dynamic split-and-fuse, tensor parallelism, and CUDA kernels. That is LMDeploy's own comparison against vLLM on its own test — not against DeepInfer.

DeepInfer's charts use 4 SM100 GPUs for prefill and 64 for decode, a 131,072-token context, and a decode configuration with nextn=3 and 75% draft acceptance. The two numbers cannot be placed on the same leaderboard without normalizing model, hardware, context, batch policy, cache hit rate, and SLO.

We have not run either engine on identical hardware. If you need a decision, control the checkpoint, GPU/NPU model and count, quantization, input/output distribution, and concurrency; then measure TTFT and TPOT at p50/p95/p99, throughput under a latency constraint, and cost per million delivered output tokens.

Decision guide

FAQ

Sources

• LMDeploy README — engines, DeepSeek work, PD integration, Ascend, quantization, license.
• LMDeploy OpenAI-compatible API docs — endpoints and launch command.
• DeepInfer README — scope, hardware, missing components, charts.
• DeepInfer Limits — the missing service layer.
• All ten comparisons — vLLM, SGLang, TensorRT-LLM, TGI, Xinference, Dynamo, llama.cpp, Ollama, and MLC-LLM.

Next: DeepInfer vs TGI, vs Xinference, or the comparison hub.