Model Guidance
Guidance is a method of controlling the output of a diffusion model by providing additional information or constraints during the generation process. It can be used to improve the quality of generated images, enforce specific styles, or guide the model towards desired outputs
Basic prompt conditioning is a form of guidance, but guidance can also be applied in other ways, such as through attention mechanisms, classifier-free guidance, or other techniques that influence the model's behavior during inference
Tip
Typical guidance scale/strength values are in the range of 0.0 to 14.0 with any value at 1.0 or below simply disables the guidance method
- Using guidance on a model that was tuned to skip guidance may result in "overcooked" output
- Skipping guidance on a model that was tuned to use guidance may result in "under-cooked" output
Tip
Guidance requires additional calculations, so Turbo style models are typically tuned to skip additional guidance for faster generation
Note
Some guidance methods are only available for standard model pipelines while others are only available for modular pipelines.
Guidance methods:
- CFG: ClassifierFreeGuidance
- Auto: AutoGuidance
- Zero: ClassifierFreeZeroStar
- PAG: PerturbedAttentionGuidance
- APG: AdaptiveProjectedGuidance
- SLG: SkipLayerGuidance
- SEG: SmoothedEnergyGuidance
- TCFG: TangentialClassifierFreeGuidance
- FDG: FrequencyDecoupledGuidance
See https://huggingface.co/docs/diffusers/v0.40.0/en/api/modular_diffusers/guiders for more information