Question about caching implementation

[Barry0121]

Hi! I'm doing a deep dive into the implementation and have some questions.

1. I see that the key caching-related codes are in DeepCache.sd.pipeline_stable_diffusion.py under StableDiffusionPipeline's call() around lines 726 to 740 (It says "7. Denoising loop" at the top, so I assume it must be around here). However, I am struggling to see how/where you are storing the cached up-sample block output? Are you doing this through sample_from_quad_center()? But this function is only for non-uniform sampling right?
2. This question is related to Q1: So far all the hyperparameters belonging to the actual caching part of deepcache are: cache_interval, cache_layer_id (I believe this is the skip connection layer), cache_block_id (this should be the up-sampling block), center, and power (these are for non-uniform sampling). However, if we specify which block we want to cache, shouldn't that also specify which skip connection is being connected to the output? So, why do we need to specify both?
3. Is there anywhere else I should check for stuff you have changed from the original stable_diffusion pipeline (outside of pipeline_stable_diffusion.py)?

There are parts of the paper I'm not sure about, so I apologize if my questions seem dumb. Thank you!

[horseee]

Hi @Barry0121,

Q1: The variable responsible for storing cached features is named as prv_features at Line 723 and undergoes updates at Line 753. To understand the mechanism of feature caching, you can delve into the inference code of the UNet architecture. The function sample_from_quad_center exclusively serves to adjust the caching schedule.

Q2: Each combination of cache_layer_id and cache_block_id corresponds uniquely to a skip connection.

Q3: I documented the modifications compared to the original pipeline in ReadMe, available here. You can find detailed information there.

[Barry0121]

Thank you for the response! Appreciate your insight.