LoRA Technical

How LoRA and the wider LyCORIS adapter family are loaded and applied on the Diffusers backend. This is a code map for contributors. For usage and settings, see LoRA and Networks.

All code lives in modules/lora/, with per-architecture native loaders in pipelines/<arch>/<arch>_lora.py.

Files

File	Responsibility
extra_networks_lora.py	Entry point. Parses <lora:name:weight> from the prompt, including component tags, and drives activate/deactivate. Registered as an ExtraNetwork.
lora_overrides.py	Method selection (get_method), the NATIVE_DISPATCH registry, the allow_native arch list derived from it, and the force lists.
lora_load.py	Load orchestration: network_load, the native loader load_safetensors, disk scanning, the key dump, and lora_cache.
lora_diffusers.py	Diffusers/PEFT path: load_diffusers, adapter and scale tracking.
lora_nunchaku.py	Nunchaku path for Nunchaku-quantized models.
native_adapter.py	Generic LyCORIS family loaders and key-parsing scaffolding shared by the per-arch native loaders.
network.py	Data model: NetworkOnDisk, Network, NetworkModule.
network_*.py	Per-family NetworkModule subclasses that implement calc_updown (lora, lokr, hada, oft, ia3, glora, norm, full, boft).
lora_apply.py	Weight patching: backup, network_calc_weights (accumulate updown), and the two apply functions.
networks.py	Apply lifecycle: network_activate, network_deactivate, promotion of staged per-network settings, the native_active flag.
lora_common.py	Shared module-level state, imported everywhere as l: loaded_networks, previously_loaded_networks, timer, debug.
lora_convert.py	Network-name stamping on the loaded pipeline (assign_network_names_to_compvis_modules) and the CompVis-to-diffusers UNet name table.
pipelines/<arch>/<arch>_lora.py	Per-arch native loaders, one per arch in NATIVE_DISPATCH.

Load lifecycle

Activation runs once per generation through the extra-networks system, after the prompt is parsed.

flowchart TD
    P["prompt with <lora:name:weight>"] --> A["ExtraNetworkLora.activate"]
    A --> PA["parse: names, te/unet multipliers, dyn_dims, lora_modules"]
    PA --> GM["lora_overrides.get_method"]
    GM --> SW["switch_method: a changed method removes the previous method's sets"]
    SW -->|diffusers| ND1["network_load -> load_diffusers per net"]
    SW -->|nunchaku| NK["lora_nunchaku.load_nunchaku"]
    SW -->|native| ND2["network_load -> load_safetensors per net"]
    ND1 --> SA["pipe.set_adapters, optional fuse_lora"]
    ND2 --> CH{"changed?"}
    CH -->|yes| DE["network_deactivate (when the applying pass or this request fuses)"]
    DE --> AC["network_activate: patch module weights"]
    CH -->|no| SKIP["skip, weights already applied"]
    SA --> DONE["generation runs with adapters live"]
    AC --> DONE
    NK --> DONE
    SKIP --> DONE

network_load runs the per-network load:

• l.loaded_networks is cleared and each name is resolved to a NetworkOnDisk through gather_networks.
• Per network, get_method(shorthash) is called again and the load is dispatched to load_diffusers, the nunchaku no-op (handled separately), or load_safetensors.
• Each loaded Network is appended to l.loaded_networks. Its multipliers, block weights and requested components are staged in pending_config, and network_activate promotes them after the removal pass, which still runs with the previous request's values.
• The native loader returns a clone of the cached file for every reference (Network.clone, sharing the bound weights), so one file named twice in a prompt is two networks with their own strengths and components. The Nth reference to a name is keyed name#N (Network.key), and that key identifies the network in the apply stamp, stack selection and the factor cache; references on the same layers add up.
• lora_cache is trimmed to lora_in_memory_limit.

The apply step differs by path:

• Diffusers adapters are activated by set_adapters, optionally fused.
• Native networks are applied by network_activate, which patches the model's module weights directly.
• Nunchaku is handled inside its own loader.

Method selection

get_method returns (method, reason). The reason is logged so an automatic fallback can be told apart from a user opt-in. Method is decided per network so a single file hash can be forced even when the arch supports native.

flowchart TD
    S["get_method(shorthash)"] --> NQ{"Nunchaku transformer or unet loaded?"}
    NQ -->|yes| RN["nunchaku"]
    NQ -->|no| FD{"opts.lora_force_diffusers?"}
    FD -->|yes| RD1["diffusers: opt-in"]
    FD -->|no| FC{"pipeline class in force_classes_diffusers?"}
    FC -->|yes| RD2["diffusers: class-forced"]
    FC -->|no| AN{"sd_model_type in allow_native?"}
    AN -->|no| RD3["diffusers: arch-unsupported"]
    AN -->|yes| FH{"file hash in force_hashes_diffusers?"}
    FH -->|yes| RD4["diffusers: hash-forced"]
    FH -->|no| RNAT["native: default"]

allow_native is list(NATIVE_DISPATCH), the arches with a per-arch native loader: sd, sdxl, sd3, f1, zimage, chroma, ernieimage, f2, f3, anima, krea2, minimaxh3, mossimage, boogu, fibo2, ideogram4, qwen21. Any other arch always takes the diffusers path. force_classes_diffusers is empty; a pipeline class listed there always takes the diffusers path.

Native path

load_safetensors is the native loader. It looks up shared.sd_model_type in NATIVE_DISPATCH and hands the file to that arch's module.

flowchart TD
    LS["load_safetensors(name, network_on_disk)"] --> C{"name in lora_cache?"}
    C -->|yes| RET["return a clone of the cached Network"]
    C -->|no| D["NATIVE_DISPATCH[sd_model_type]"]
    D --> PA["import pipelines/<arch>/<arch>_lora.py"]
    PA --> TL["mod.try_load -> native_adapter.try_load_chain"]
    TL --> NET["Network with NetworkModule per matched key"]

Each module exposes try_load(name, network_on_disk, lora_scale). The returned Network has a modules dict keyed by stamped layer name. The diffusers path does not build modules, which is why an applied network is detected by len(net.bound()) > 0, the modules the file bound whatever components the request selects.

native_adapter

native_adapter.py holds the generic, arch-independent half of the per-arch loaders:

• Family loaders, one per adapter type: try_load_lora (with DoRA), try_load_lokr, try_load_loha, try_load_oft, try_load_ia3, try_load_glora, try_load_norm, try_load_full.
• try_load_chain(name, nod, scale, family_loaders) runs each family loader in order and merges the non-empty results into one Network, binding every merged module to that network so the strength set on it reaches all families.
• Key-parsing helpers: parse_key, group_by_suffixes, prefix detection, marker checks, PEFT name stripping.
• ChunkSpec slices a fused delta onto several targets: equal chunks (idx/total), explicit row ranges (start/end), or a row reorder. A per-output DoRA magnitude slices with the rows; a per-input one spans every chunk, so the chunks share one column norm through network.DoraGroup, computed once per activate pass from each chunk's pristine weight.
• TextEncoderRouter maps text-encoder prefixes (lora_te_, lora_te1_/2_/3_, text_encoder., clip_l., t5. and the like) onto the loaded encoder that fills the role, read from the encoder's structure (CLIP-L, CLIP-G, CLIP-H or T5).
• delta_fits checks each family's delta against the target module's shape before binding. widen_input (LoRA and DoRA) and widen_full (full-weight deltas) zero-pad a narrower input onto an input-widened target such as an inpaint conv_in, reported as expanded=N; the added channels stay unchanged. narrow_input and narrow_full drop input columns past the target's width when every dropped column is zero, as in a fused save that padded a SwiGLU down projection (ComfyUI Fibo 2 w2, 9728 columns against 9536), reported as trimmed=N; a nonzero column past the width is a mismatch. LoKr and LoHA deltas with a different input width count as mismatches.

Shared rules applied by every family loader:

• A key whose target the loaded model lacks counts as unmapped and is reported in one warning per file.
• A delta that does not fit its target counts as a mismatch, and a file with any mismatch is refused whole.
• An unreadable or truncated file is declined.

A per-arch module is thin. In it:

• The arch's key prefixes, a network_prefix and a resolve_targets remap are set.
• The family loaders are bound.
• A single try_load that calls try_load_chain is exposed.

Every per-arch module binds the full family set: LoRA (with DoRA), LoKR, LoHA, OFT and BOFT, IA3, GLoRA, Norm and Full.

Arch	Module
sd, sdxl	pipelines/sd/sd_lora.py
sd3	pipelines/sd3/sd3_lora.py
f1	pipelines/flux/flux1_lora.py
zimage	pipelines/z_image/zimage_lora.py
chroma	pipelines/chroma/chroma_lora.py
ernieimage	pipelines/ernie/ernie_lora.py
f2	pipelines/flux/flux2_lora.py
f3	pipelines/flux3/flux3_lora.py
anima	pipelines/anima/anima_lora.py
krea2	pipelines/krea2/krea2_lora.py
minimaxh3	pipelines/minimax/minimax_lora.py
mossimage	pipelines/moss/moss_lora.py
boogu	pipelines/boogu/boogu_lora.py
fibo2	pipelines/bria/fibo2_lora.py
ideogram4	pipelines/ideogram/ideogram_lora.py
qwen21	pipelines/qwen/qwen21_lora.py

resolve_targets is where an arch rewrites trainer layouts to the current diffusers module names. Examples:

• Z-Image rewrites fused attention.qkv and bare attention.out to to_q/to_k/to_v and to_out.0, chunking the fused up-weight at load time.
• Flux.1 splits the BFL qkv and single-block linear1 by the widths of the loaded block, renames XLabs processor keys, and swaps the halves of the final-layer modulation.
• Fibo 2 chunks the ComfyUI fused attention.qkv and to_kv onto the split projections, slices the padded fused w13 into w1 and w3, and renames the original checkpoint names (cross_q, self_out, all_x_embedder.2-1).
• SD3 routes the kohya context_block stream to add_*_proj and the x_block stream to to_*, including attn2 on dual-attention blocks.
• SD and SDXL rename CompVis block names (input_blocks_4_1_proj_in) through the lora_convert table, with the loaded UNet deciding where SD 1.x holds an upsampler.

Data model

classDiagram
    class NetworkOnDisk {
        name
        filename
        shorthash
        metadata
        sd_version
        detect_version()
    }
    class Network {
        name
        network_on_disk
        te_multiplier
        unet_multiplier
        block_spec
        components
        pending_config
        modules
        bundle_embeddings
    }
    class NetworkModule {
        network
        sd_module
        alpha
        scale
        dora_scale
        calc_updown()
        calc_scale()
        multiplier()
    }
    NetworkOnDisk "1" --> "*" Network : loaded into
    Network "1" --> "*" NetworkModule : modules
    NetworkModule <|-- NetworkModuleLora
    NetworkModule <|-- NetworkModuleLokr
    NetworkModule <|-- NetworkModuleHada
    NetworkModule <|-- NetworkModuleOFT
    NetworkModule <|-- NetworkModuleIa3
    NetworkModule <|-- NetworkModuleGLora
    NetworkModule <|-- NetworkModuleNorm
    NetworkModule <|-- NetworkModuleFull

• NetworkOnDisk describes a file: name, hash, parsed safetensors metadata, and a detect_version heuristic over metadata and filename.
• Network is one loaded adapter: its multipliers, block weights (block_spec), requested components, and a modules dict keyed by layer name.
• NetworkModule is one layer's weights inside one network. It owns the family math in calc_updown, plus alpha, scale, and DoRA handling.

NetworkModule.multiplier() gives each layer its strength:

• Layers stamped lora_te* take te_multiplier; every other layer (UNet, transformer, llm_adapter) takes unet_multiplier, split into in, mid and out by the UNet's down_blocks, mid_block and up_blocks.
• The result is scaled by the per-block factor when block_spec is set (lora_blocks).
• A network given components (module=, high, low, cond, uncond) keeps only its modules on those components' layers for the request (Network.select_layers); bound_modules holds every module the file bound, and layers outside the components are not touched for that network.
• uncond re-targets the network's transformer modules onto Ideogram 4's unconditional_transformer (networks.tower_aliases), rebuilding split DoRA groups for that tower's layers. A reference without it never reaches the tower, so a bare reference applies to the conditional transformer only.

Apply and restore

Layer names are stamped on the loaded pipeline by lora_convert.assign_network_names_to_compvis_modules: lora_te_ (a lone encoder) or lora_te1_, lora_te2_ and lora_te3_, lora_unet_, lora_transformer_, lora_unconditional_transformer_ and lora_llm_adapter_, each followed by the module path with dots as underscores.

network_activate iterates every module of the active components (text_encoder*, unet, transformer*, unconditional_transformer, llm_adapter). For each module that carries a network_layer_name and whose stamp differs from the wanted set, it backs up, computes the combined delta, and applies it. The stamp covers the networks with their multipliers plus the stack, block-weight, component and SDNQ mechanism signatures, so changing any of them re-applies the layer.

flowchart TD
    NA["network_activate"] --> IT["for each module with network_layer_name"]
    IT --> SK{"current_names == wanted_names?"}
    SK -->|yes| NEXT["skip"]
    SK -->|no| BK["network_backup_weights"]
    BK --> CW["network_calc_weights: sum updown over loaded_networks"]
    CW --> FU{"opts.lora_fuse_native?"}
    FU -->|yes| AD["network_apply_direct: write delta into weight"]
    FU -->|no| AW["network_apply_weights: restore from backup then add delta"]
    AD --> MARK["set network_current_names = wanted_names"]
    AW --> MARK

network_calc_weights walks l.loaded_networks (or previously_loaded_networks when use_previous is set), pulls each network's NetworkModule for this layer, calls calc_updown on the layer's pristine weight, scales by the module multiplier, and accumulates one combined delta. pristine_weight is the stored copy when the layer has one, dequantized with the stored SDNQ state, and the live weight otherwise.

Fuse and backup are a tradeoff set by lora_fuse_native:

• Backup mode: network_backup_weights clones the original weight to CPU. network_apply_weights restores from that backup and adds the new delta, so switching or removing a LoRA recomputes from the backup. network_deactivate returns early when both the pass that applied the set and the current request run in this mode. A quantized model always runs in backup mode.
• Fuse mode (on by default): no tensor backup is kept, only the sentinel True. network_apply_direct writes the delta into the weight in place for lower peak VRAM. Removal needs an explicit subtract, so network_deactivate recomputes the previous delta with use_previous=True and applies it with deactivate=True.
• A set whose delta depends on the weight (DoRA, OFT, BOFT, GLoRA, IA3) keeps a tensor backup for the layers it touches in fuse mode too, because a delta recomputed on the fused weight is not the one that was applied; network_deactivate restores those layers from the copy. The unload log reports such a pass as mode=backup.
• Removal follows the mode a set was applied in. The activation pass records its fuse decision in lora_common.last_fuse, so a fused set is subtracted even when the next request runs with lora_fuse_native off (a per-request override_settings or the settings toggle), and backup-mode copies restore before a fuse-mode pass replaces them with markers.
• A request whose load method differs from the previous one on the same checkpoint (lora_force_diffusers toggled) first removes the previous method's sets in ExtraNetworkLora.switch_method: networks.network_remove for native sets, an adapter unload for diffusers ones. It also clears the change cache, so the same tags apply again under the new method.

Storing True versus a tensor also lets a mid-session toggle of the setting invalidate the stale backup. networks.native_active records whether native weights are currently applied and gates restoration when LoRAs are removed.

Diffusers path

When get_method returns diffusers, lora_diffusers.load_diffusers calls the pipeline's own load_lora_weights and records the adapter name and scale in diffuser_loaded and diffuser_scales. After the load loop, network_load calls set_adapters with the collected names and weights, and fuses when configured. No NetworkModule objects are built, and lora_apply is not involved. unload_diffusers tears the adapters down.

Adding native support for a new arch

Native LoRA is the adapter step of the larger process of adding a new architecture. The LoRA-specific touchpoints are:

• pipelines/<arch>/<arch>_lora.py: the arch's key prefixes, network_prefix and a resolve_targets remap are set, the native_adapter family loaders are bound, and a single try_load is exposed via try_load_chain.
• lora_overrides.NATIVE_DISPATCH: the arch's sd_model_type is mapped to the new module, which also puts it in allow_native.
• lora_convert.assign_network_names_to_compvis_modules: a component outside the stamped set gets its prefix here.

Key fixups belong in resolve_targets, the arch's grouping hook or the family loader, not in load_safetensors. A format that every arch shares (a trainer's suffix spelling, a wrapper prefix) belongs in native_adapter. Arches without a dispatch entry keep working through the diffusers path.

Debugging

• SD_LORA_DEBUG=1 enables verbose load, activate, and deactivate logs, including per-family module counts, the list of unmapped keys, per-key shape mismatches, applied layer counts, backup size, and timings.
• SD_LORA_DUMP=1 writes two sorted key lists to the temp directory for every arch: LoRA-<type>-<class>-<file>.txt with the file's keys and Model-<type>-<class>.txt with the stamped layer names, so the two can be diffed when keys fail to match.
• Network load: type=LoRA name="x" native modules=N unmapped=M is a warning naming how many of a file's targets the loaded model lacks.
• Network load: type=LoRA name="x" modules=N mismatch=M shapes do not match the loaded model is followed by not loaded: the file was trained for another base.
• Network load: type=LoRA name="x" requested=<component> missing reports a module=, high, low, cond or uncond tag naming a component the loaded pipeline lacks.
• The status line Network status: type=LoRA ... method=actual(reason) reports native only if modules were built.
• Relevant settings: lora_force_diffusers, lora_fuse_native, lora_fuse_diffusers, lora_in_memory_limit, lora_force_reload, extra_networks_default_multiplier.