[docs] Optimize and scale pt. 1 by stevhliu · Pull Request #14867 · huggingface/diffusers

stevhliu · 2026-09-24T21:50:30Z

Refreshes the Optimize and scale section:

add an Overview with a starter path (dtype + device → offload) and links to other optimization techniques such as kernels and quantization
more accurate doc titles like "Precision and compilation" that reflect what they actually teach vs a more vague promise like "Reduce memory usage"
adds a section to guide cache method selection and also adds TextKVCache (can remove if this is too niche, but good to document for completeness)
reduce noisy prose (tighter intros, fewer callouts, etc.) and more consistent navigation
make sure hardware/API claims (FA3/Hopper, kernels requirements) matches code

HuggingFaceDocBuilderDev · 2026-09-24T21:57:01Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

sayakpaul

Didn't review it fully but some changes are too intrusive, IMO. Hopefully, the existing comments will be helpful to get an idea of what I am talking about and help you make changes to the files I didn't review yet.

sayakpaul · 2026-09-26T02:52:42Z


 ## Optimization

-You can optimize the pipeline's runtime and memory consumption with torch.compile and feed-forward chunking. To learn about other optimization methods, check out the [Speed up inference](../../optimization/fp16) and [Reduce memory usage](../../optimization/memory) guides.


I think precision and compilation as a title is less exciting as a reader in the sense that it doesn't tell me that it's related to speeding up of inference.

i think it's a little strange to broadly name it "Accelerate inference" but then it doesn't include things like caching or the attention backends, which may cause users to overlook those last two options. so i'd rather keep it as "Precision and compilation"

sayakpaul · 2026-09-26T02:54:36Z

 <hfoption id="memory">

-Refer to the [Reduce memory usage](../../optimization/memory) guide for more details about the various memory saving techniques.
+Refer to the [Memory and offloading](../../optimization/memory) guide for more details about the various memory saving techniques.


I think this change makes it less obvious to the reader the page is about something they would actually care about (i.e., reducing memory usage).

i agree here that "Reduce memory usage" is better since it has all the memory techniques on one page :)

stevhliu · 2026-09-28T17:26:01Z

Hopefully, the existing comments will be helpful to get an idea of what I am talking about and help you make changes to the files I didn't review yet.

Softened some overly absolute claims, hopefully this is what you were referring to 😄

sayakpaul

Left some further comments. Thanks for iterating!

sayakpaul · 2026-09-30T03:42:46Z

+
+| Method | Use when | Tradeoff |
+|--------|----------|----------|
+| Text KV Cache | NucleusMoE image only, need exact text K/V reuse across steps | Lossless |


Hmm, it was added alongside NucleusMoE but I don't see anything that would prevent it from being used in other pipelines.

Trade-offs are usually brittle because quality is subjective. So, maybe we should suggest to the readers that they should inspect the quality of the outputs because each caching technique has its own trade-off (better memory consumption, better speed, better preservation of the original outputs, etc.). Not sure how to present this faithfully in the table.

The current table and wording have most of it. My biggest worry is about terms like "loseless". Open to ideas.

it seems like apply_text_kv_cache only applies its hooks to NucleusMoEImageTransformerBlock so on other models it doesn't do anything?

sayakpaul · 2026-09-30T03:43:49Z

+| Method | Use when | Tradeoff |
+|--------|----------|----------|
+| Text KV Cache | NucleusMoE image only, need exact text K/V reuse across steps | Lossless |
+| SeaCache | Video transformers that already have a SeaCache path | Approximate, settings often do not transfer across models |


As per the Cosmos3 documentation page, it seems like it's supposed to only work with Cosmos3. @yzhautouskay can probably confirm.

sayakpaul · 2026-09-30T03:50:43Z

    attention_weight_callback=lambda _: 0.3,
    unconditional_batch_skip_range=5,
-    unconditional_batch_timestep_skip_range=(-1, 781),
+    unconditional_batch_timestep_skip_range=(-1, 641),


What happened here? 👀

i used the default value from below, but happy to change it back to 781 if thats better

diffusers/src/diffusers/hooks/faster_cache.py

Line 151 in c60830e

unconditional_batch_timestep_skip_range: tuple[int, int] = (-1, 641)

sayakpaul · 2026-09-30T04:02:23Z

 ```

-To learn more, take a look at the [Reduce memory usage](../optimization/memory) and [Accelerate inference](../optimization/fp16) guides.
+To learn more, take a look at [Optimize and scale](../stable_diffusion#optimization-techniques).


../stable_diffusion#optimization-techniques

Do we still have a reason keep these under "stable_diffusion". I don't see any reason to.

stable_diffusion.md is actually the overview, so it may be nice to point users toward an overview of optimization techniques for a particular task and then they can choose what they want

github-actions Bot added documentation Improvements or additions to documentation size/L PR with diff > 200 LOC labels Sep 24, 2026

stevhliu requested a review from sayakpaul September 24, 2026 21:57

sayakpaul reviewed Sep 26, 2026

View reviewed changes

stevhliu mentioned this pull request Sep 29, 2026

[docs] Optimize and scale pt. 2 #14909

Open

sayakpaul reviewed Sep 30, 2026

View reviewed changes

stevhliu added 9 commits September 30, 2026 09:12

overview

963d5bf

better titles

8a50078

compile

cbe03c5

cache

1bf3971

attention

ae0e1ef

feedback - titles

1f849b5

feedback - attention

7049181

feedback - soften absolutes

83d101c

feedback - cache integration, update table

342b599

stevhliu force-pushed the optimize-scale branch from fedf28e to 342b599 Compare September 30, 2026 17:04


		## Optimization

		You can optimize the pipeline's runtime and memory consumption with torch.compile and feed-forward chunking. To learn about other optimization methods, check out the [Speed up inference](../../optimization/fp16) and [Reduce memory usage](../../optimization/memory) guides.

Conversation

stevhliu commented Sep 24, 2026

Uh oh!

HuggingFaceDocBuilderDev commented Sep 24, 2026

Uh oh!

sayakpaul left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

stevhliu commented Sep 28, 2026

Uh oh!

sayakpaul left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants