Skip to content
FluxServe

OPEN SOURCE · DIFFUSION INFERENCE

Serving diffusion,
one block
at a time.

A lightweight serving engine for diffusion language models. From your first request to multi-GPU inference.

Get started View on GitHub

Docker setup ↗

docker pull flxoss/fluxserve:v0.1-cu130-fa4
FluxServeSGLang
Mini · BigCodeBenchDecode throughput versus request rate, reconstructed from approximate points in the published figure. The vertical axis starts at 690, matching the original chart's range.Decode throughput (tokens/s)7008009001000110012001300481216Request rate (req/s)

LLaDA2.0-mini · TP=1 · EP=1

INSIDE FLUXSERVE

Built for the way diffusion generates.

Attention, scheduling, and execution. Designed around the block.

01 / ATTENTION

Every block counts.

Block-causal attention for variable-length prefill and decoding, with paged KV caching and CUDA graph support.

Explore block attention
02 / SCHEDULING

Keep requests moving.

A C++ control plane and Python execution plane coordinate requests with fine-grained, block-level scheduling.

Understand the scheduler
03 / EXECUTION

Room for larger models.

Tensor, data, and expert parallelism bring diffusion inference to documented multi-GPU configurations.

Deploy across GPUs

FROM SOURCE TO SERVING

Your next step starts here.

01 Send your first request 02 Find a deployment guide 03 Benchmark your workload