Choose an acceleration approach¶
On this page
Start with the problem you need to solve.
Get a working baseline first. Complete a generation using the model guides and save the inputs, configuration and output. Change one setting at a time to compare latency, resource use and generation quality.
Choose by goal¶
Fit available memory¶
Start with parameter offload and model quantization. Offload trades GPU memory for host memory and transfer latency; quantization needs compatible weight formats and compute backends.
For Wan2.1, start from a complete configuration in low-resource deployment.
Reduce generation latency¶
Use low-latency deployment to explore the options: use fewer sampling steps with matching distilled weights, or evaluate attention backends and feature caching with the original model.
If the bottleneck is unclear, inspect time spent in each stage with performance profiling.
Use multiple GPUs¶
To have multiple GPUs work on one generation, see parallel inference. To run separate services, see inference services.
The first approach needs matching parallel settings and process counts. The second needs separate model, GPU and port configurations.
Compare the main methods¶
Scroll horizontally to see the full comparison.
| Method | What it changes | Check before using it |
|---|---|---|
| Model quantization | Weight and compute precision | Compatible weight formats, backends and GPUs; compare output quality |
| Parameter offload | Weights resident on the GPU | Host memory, disk throughput and transfer overhead; activations still need GPU memory |
| Attention | The Attention implementation | Support for the chosen backend in your GPU, model and dependencies |
| Feature caching | Repeated computation during denoising | Supported algorithms, reuse thresholds and output differences |
| Step distillation | Sampling steps and model weights | Matching distilled weights, step counts and CFG settings |
| Parallel inference | Computation across GPUs | Process counts must match the parallel configuration; CFG parallelism requires CFG |
| Variable resolution | Resolution in intermediate denoising stages | Model support, resolution transitions and output quality |
Check each guide and model example for supported configurations and combination limits. Validate individual changes before combining them.
Verify the result¶
- Record a baseline. Fix the model, input, seed, dimensions and frame count. Keep the full configuration and output.
- Compare the same timing scope. Separate model loading, the first run and subsequent generations. Also observe peak GPU and host memory. Less denoising computation does not imply a proportional reduction in total latency.
- Inspect the output. Compare detail, motion consistency and adherence to the prompt before changing another setting.
Once results are reproducible, consult the testing methods in benchmarks or use performance profiling to investigate further.