# FluxServe

FluxServe is a lightweight serving engine for diffusion language models. It combines block-level scheduling, attention kernels, and distributed execution to serve autoregressive (AR) diffusion models on NVIDIA GPUs.

## Start here

- [Quickstart](/docs/guides/quickstart/): launch a server and send your first request.
- [Docker installation](/docs/guides/getting_started/): build the CUDA environment and install FluxServe.
- [Architecture](/docs/architecture/): understand how requests move through the engine.
- [Benchmarking](/docs/guides/benchmark/): measure offline and online performance.

## Core capabilities

### Block attention

Block-causal attention supports variable-length prefill and block decoding. The paged FlashInfer path reuses committed KV cache entries; CUDA graph execution is available through explicit launch options.

### Dynamic scheduling

The native C++ scheduler manages requests at block granularity. The Python runtime consumes execution plans and coordinates model execution and cache updates.

### Multi-GPU execution

Tensor, data, and expert parallelism support larger serving configurations. Use a documented combination for your model and hardware; the parallelism sizes are not independently interchangeable.

## Deployment guides

- [LLaDA2.0-mini on one GPU](/docs/serving/llada2-mini/)
- [LLaDA2.0-flash on four GPUs](/docs/serving/llada2-flash/)
- [LLaDA2.1 decoding presets](/docs/serving/llada2.1/)

## Project

FluxServe is developed by [FLX-OSS](https://github.com/FLX-OSS). Source code is available on [GitHub](https://github.com/FLX-OSS/FluxServe) under the [MIT license](https://github.com/FLX-OSS/FluxServe/blob/7183e38ea5689ca3afdda4fdf7c084b8a0f6d3f3/LICENSE).

---

Source revision: [7183e38](https://github.com/FLX-OSS/FluxServe/tree/7183e38ea5689ca3afdda4fdf7c084b8a0f6d3f3).