roboflow · Borda · Sep 28, 2026 · Sep 26, 2026
@@ -17,6 +17,10 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
     - Pretrained RF-DETR Nano on COCO val2017 scored 0.4802 mAP through `evaluate()` and now scores 0.4842, the same as pycocotools on the same predictions.
     - Training is unchanged, and only COCO-format datasets with crowd labels get different numbers — **runs from before and after this change are not directly comparable on crowd-labelled COCO data.**
     - Not covered: WebDataset shards and a hand-built `Trainer.validate(model, dataloaders=...)` call with no datamodule attached are both still scored the old way, and `val/F1` only ignores a detection whose IoU with the crowd is at least 0.5 (measured against the crowd box for detection, the crowd mask for segmentation). ([#1531](https://github.com/roboflow/rf-detr/issues/1531))
+- A checkpoint trained with `backbone_lora=True` reloads with its trained backbone. `RFDETR.from_checkpoint()`, `RFDETR<Size>(pretrain_weights=...)` and `train()` loaded the checkpoint before wrapping the encoder with LoRA. The checkpoint stores the encoder under PEFT names, so none of those weights matched and the encoder stayed at random init. `evaluate()` scored about 0, `predict()` and exports ran with an untrained backbone, and training restarted the encoder. The only sign was a partial-load warning (`223 model parameter(s) not in checkpoint … 331 checkpoint key(s) not consumed`).
+    - The encoder is now wrapped before such a checkpoint loads. With `backbone_lora=True` it keeps the trained adapters. The flag can be passed explicitly, or `from_checkpoint()` restores it from a checkpoint's `model_config`; `checkpoint_best_total.pth` does not carry one yet (#1533), so pass the flag for that file. With `backbone_lora=False`, the constructor default, the adapters are merged into the encoder weights, and the encoder keeps the trainable flags of a model built without LoRA.
+    - Without `peft` installed, loading such a checkpoint as `pretrain_weights` or through `from_checkpoint()` now raises `ImportError`. Before, a model configured without LoRA loaded it with a random encoder instead. The patch-size check added in #965 now also covers these checkpoints.
+    - Measured case: RF-DETR Large at 640 px, LoRA fine-tuned on 262 COCO train2017 person images. Reloaded and evaluated on 300 val2017 person images, it scored val mAP50:95 0.0001 before this fix and 0.6221 after, the same as the in-memory model. ([#1540](https://github.com/roboflow/rf-detr/issues/1540))
 
 ### Changed
 

@@ -101,7 +101,7 @@ RF-DETR provides several optional extras for additional functionality:
 | `coreml`     | `pip install "rfdetr[coreml]"`                          | Native CoreML export (.mlpackage; macOS only)                                                                                                                              |
 | `tensorrt`   | `pip install "rfdetr[tensorrt]"`                        | TensorRT inference (tensorrt, polygraphy, onnxruntime-gpu, plus onnx and onnxconverter-common to cast the graph to FP16 on TensorRT 11+; pycuda lives in `tensorrt-bench`) |
 | `augment`    | `pip install "rfdetr[augment]"`                         | Custom CPU (Albumentations) + GPU (Kornia) augmentations                                                                                                                   |
-| `lora`       | `pip install "rfdetr[lora]"`                            | LoRA fine-tuning with PEFT                                                                                                                                                 |
+| `lora`       | `pip install "rfdetr[lora]"`                            | LoRA fine-tuning with PEFT, and loading a checkpoint trained with `backbone_lora=True`                                                                                     |
 | `visual`     | `pip install "rfdetr[visual]"`                          | Visualization utilities (matplotlib, pandas, seaborn)                                                                                                                      |
 | `cli`        | `pip install "rfdetr[cli]"`                             | CLI with typed argument parsing (jsonargparse)                                                                                                                             |
 | `plus`       | `pip install "rfdetr[plus]"`                            | XLarge and 2XLarge detection models (PML 1.0 license)                                                                                                                      |

@@ -547,6 +547,10 @@ def __init__(self, *, trust_checkpoint: bool = False, **kwargs: Any) -> None:
                 weights.
             **kwargs: ModelConfig field values (e.g. ``resolution``, ``num_classes``,
                 ``pretrain_weights``, ``gradient_checkpointing``).
+
+        Raises:
+            ImportError: If ``backbone_lora=True`` is set, or ``pretrain_weights`` was saved by a
+                ``backbone_lora=True`` run, and ``peft`` is not installed (``pip install "rfdetr[lora]"``).
         """
         self.model_config = self.get_model_config(**kwargs)
         self.maybe_download_pretrain_weights()
@@ -666,6 +670,8 @@ def from_checkpoint(cls, path: str | os.PathLike[str], *, trust_checkpoint: bool
             KeyError: If the checkpoint does not contain an ``"args"`` key.
             ValueError: If the model class cannot be inferred from ``model_name``,
                 ``pretrain_weights``, or the checkpoint filename.
+            ImportError: If the checkpoint was saved by a ``backbone_lora=True`` run and ``peft``
+                is not installed (``pip install "rfdetr[lora]"``).
 
         Examples:
             >>> model = RFDETR.from_checkpoint("checkpoint_best_total.pth")  # doctest: +SKIP

@@ -157,6 +157,7 @@ def _build_model_context(model_config: ModelConfig, *, trust_checkpoint: bool =
             args.num_keypoints_per_class = _mc_kp
 
     if model_config.backbone_lora:
+        # No-op when load_pretrain_weights already wrapped the encoder to load a LoRA checkpoint.
         apply_lora(nn_model)
 
     # Adapt patch-embedding projection for non-RGB channel counts

@@ -29,7 +29,12 @@
 from rfdetr.models.backbone.dinov2 import DinoV2
 from rfdetr.models.lwdetr import LWDETR
 from rfdetr.utilities.logger import get_logger
-from rfdetr.utilities.state_dict import _ckpt_args_get, remap_projector_to_cross_attn, validate_checkpoint_compatibility
+from rfdetr.utilities.state_dict import (
+    _LORA_ENCODER_KEY_PREFIX,
+    _ckpt_args_get,
+    remap_projector_to_cross_attn,
+    validate_checkpoint_compatibility,
+)
 
 logger = get_logger()
 
@@ -329,6 +334,10 @@ class default) larger than the checkpoint provides, the head is temporarily alig
       scenario), both reinitializations are applied: expand to checkpoint size for loading, then trim to configured
       size.
 
+    A checkpoint saved by a ``backbone_lora=True`` run is loaded into a LoRA-wrapped encoder. With
+    ``model_config.backbone_lora`` set the encoder stays wrapped with the trained adapters; otherwise the adapters are
+    merged into the encoder weights, unless the caller had already wrapped the encoder (see :func:`_load_model_state`).
+
     Class names stored in the checkpoint ``args`` are extracted and returned.
 
     Args:
@@ -346,6 +355,7 @@ class default) larger than the checkpoint provides, the head is temporarily alig
 
     Raises:
         Exception: If the checkpoint file cannot be loaded even after a re-download.
+        ImportError: If the checkpoint was saved by a ``backbone_lora=True`` run and ``peft`` is not installed.
     """
     mc = model_config
     if mc.pretrain_weights is None:
@@ -621,7 +631,7 @@ class default) larger than the checkpoint provides, the head is temporarily alig
         )
         checkpoint["model"].pop("_kp_active_mask", None)
     interpolate_position_embeddings(checkpoint["model"], mc.positional_encoding_size)
-    incompatible = nn_model.load_state_dict(checkpoint["model"], strict=False)
+    incompatible = _load_model_state(nn_model, checkpoint["model"], backbone_lora=mc.backbone_lora)
     _warn_on_partial_load(incompatible, pretrain_weights)
 
     if should_restore_config_keypoint_schema and hasattr(nn_model, "reinitialize_keypoint_head"):
@@ -644,10 +654,57 @@ class default) larger than the checkpoint provides, the head is temporarily alig
     return class_names
 
 
+def _load_model_state(nn_model: LWDETR, model_state: dict[str, Tensor], *, backbone_lora: bool) -> Any:
+    """Load *model_state* into *nn_model* with the encoder in the layout the checkpoint was saved from.
+
+    A ``backbone_lora=True`` run saves the encoder under PEFT names (``backbone.0.encoder.base_model.model.…``). A plain
+    encoder consumes none of them, so ``load_state_dict(strict=False)`` would skip them all and leave the encoder at
+    random init (#1540). The encoder is therefore wrapped with LoRA before such a checkpoint loads. A model configured
+    without LoRA then gets the loaded adapters merged into its encoder weights, and the trainable flags that peft froze
+    when wrapping are restored. An encoder the caller had already wrapped is loaded as is and stays wrapped.
+
+    Args:
+        nn_model: Model to load into, in-place.
+        model_state: Checkpoint weights, already aligned to the model's heads and query counts.
+        backbone_lora: Whether the model is configured to keep LoRA adapters on its encoder.
+
+    Returns:
+        The ``load_state_dict`` result listing missing and unexpected keys.
+
+    Raises:
+        ImportError: If *model_state* comes from a ``backbone_lora=True`` run and ``peft`` is not installed.
+    """
+    if not any(key.startswith(_LORA_ENCODER_KEY_PREFIX) for key in model_state):
+        return nn_model.load_state_dict(model_state, strict=False)
+
+    backbone = cast(Backbone, nn_model.backbone[0])
+    encoder = backbone.encoder
+    encoder_requires_grad = {name: param.requires_grad for name, param in encoder.named_parameters()}
+    try:
+        apply_lora(nn_model)
+    except ImportError as err:
+        raise ImportError(
+            "This checkpoint was saved by a backbone_lora=True run, and loading it needs the 'peft' dependency. "
+            'Install it with pip install "rfdetr[lora]".'
+        ) from err
+    incompatible = nn_model.load_state_dict(model_state, strict=False)
+    # Merge only a wrap made here: an encoder the caller had already wrapped stays as the caller left it.
+    if not backbone_lora and backbone.encoder is not encoder:
+        from peft import PeftModel  # optional dependency; apply_lora above has already required it
+
+        backbone.encoder = cast(DinoV2, cast(PeftModel, backbone.encoder).merge_and_unload())
+        for name, param in backbone.encoder.named_parameters():
+            param.requires_grad = encoder_requires_grad[name]
+        logger.info("Merged the checkpoint's LoRA adapters into the backbone encoder, since backbone_lora=False.")
+    return incompatible
+
+
 def apply_lora(nn_model: LWDETR) -> None:
     """Apply LoRA adapters to the backbone encoder of *nn_model*.
 
     Replaces ``nn_model.backbone[0].encoder`` in-place with a PEFT-wrapped encoder using DoRA with rank 16 and alpha 16.
+    Does nothing when the encoder is already wrapped, which :func:`load_pretrain_weights` does itself to load a
+    checkpoint saved by a ``backbone_lora=True`` run.
 
     Args:
         nn_model: LWDETR model whose backbone encoder will receive LoRA adapters.
@@ -661,7 +718,7 @@ def apply_lora(nn_model: LWDETR) -> None:
                 pip install "rfdetr[train]"
     """
     try:
-        from peft import LoraConfig, get_peft_model
+        from peft import LoraConfig, PeftModel, get_peft_model
         from transformers import PreTrainedModel
     except ImportError as exc:
         raise ImportError(
@@ -670,6 +727,10 @@ def apply_lora(nn_model: LWDETR) -> None:
             'pip install "rfdetr[lora]" or pip install "rfdetr[train]".'
         ) from exc
 
+    backbone = cast(Backbone, nn_model.backbone[0])
+    if isinstance(backbone.encoder, PeftModel):
+        return
+
     lora_config = LoraConfig(
         r=16,
         lora_alpha=16,
@@ -686,7 +747,6 @@ def apply_lora(nn_model: LWDETR) -> None:
             "register_tokens",
         ],
     )
-    backbone = cast(Backbone, nn_model.backbone[0])
     # PEFT's type signature requires a PreTrainedModel, but DinoV2 is a compatible nn.Module
     # wrapper at runtime. Cast both sides of this dynamic wrapper boundary instead of relying
     # on an environment-sensitive ignore for PEFT's evolving type annotations.

@@ -441,6 +441,7 @@ def __init__(self, model_config: ModelConfig, train_config: TrainConfig) -> None
                         "Reset keypoint Gaussian precision outputs to unit values after pretrained weight load."
                     )
         if model_config.backbone_lora:
+            # No-op when load_pretrain_weights already wrapped the encoder to load a LoRA checkpoint.
             apply_lora(self.model)
 
         # Build criterion/postprocessors after potential num_classes alignment so

@@ -34,6 +34,11 @@
     "lr_schedulers",
 )
 
+#: Where a ``backbone_lora=True`` run saves the DINOv2 encoder's keys. PEFT nests the wrapped encoder at
+#: ``PeftModel.base_model.model``, so ``backbone.0.encoder.encoder.…`` is saved as
+#: ``backbone.0.encoder.base_model.model.encoder.…``.
+_LORA_ENCODER_KEY_PREFIX = "backbone.0.encoder.base_model.model."
+
 
 def _raise_patch_size_mismatch(ckpt_patch_size: int, model_patch_size: int) -> None:
     """Raise a descriptive ValueError for a patch_size incompatibility.
@@ -305,11 +310,12 @@ def validate_checkpoint_compatibility(checkpoint: dict[str, Any], model_args: An
 
         When ``"args"`` is absent or ``args.patch_size`` is not set, a fallback infers ``patch_size`` from the DINOv2
         patch-embedding projection weight shape (key
-        ``backbone.0.encoder.encoder.embeddings.patch_embeddings.projection.weight``). This fallback **can raise**
-        :class:`ValueError` on a mismatch, providing a clear error before the cryptic :class:`RuntimeError` from
-        :meth:`~torch.nn.Module.load_state_dict` would otherwise fire. For all other attributes (e.g.
-        ``segmentation_head``), if either side is missing, that check is skipped silently — preserving backward
-        compatibility.
+        ``backbone.0.encoder.encoder.embeddings.patch_embeddings.projection.weight``, or
+        ``backbone.0.encoder.base_model.model.encoder.embeddings.patch_embeddings.projection.weight`` in a checkpoint
+        saved by a ``backbone_lora=True`` run). This fallback **can raise** :class:`ValueError` on a mismatch,
+        providing a clear error before the cryptic :class:`RuntimeError` from :meth:`~torch.nn.Module.load_state_dict`
+        would otherwise fire. For all other attributes (e.g. ``segmentation_head``), if either side is missing, that
+        check is skipped silently — preserving backward compatibility.
 
         Two class-count scenarios are distinguished:
 
@@ -362,8 +368,13 @@ def validate_checkpoint_compatibility(checkpoint: dict[str, Any], model_args: An
         _ckpt_patch_size_from_args = _ckpt_args_get(_ckpt_args, "patch_size")
 
     if _ckpt_patch_size_from_args is None:
-        _patch_proj_key = "backbone.0.encoder.encoder.embeddings.patch_embeddings.projection.weight"
-        _ckpt_proj_w = checkpoint.get("model", {}).get(_patch_proj_key)
+        _patch_proj_suffix = "encoder.embeddings.patch_embeddings.projection.weight"
+        _ckpt_model = checkpoint.get("model", {})
+        # A backbone_lora=True run saves the same weight under the PEFT prefix, and its args (a TrainConfig dump)
+        # carry no patch_size, so this fallback is its only clear error before load_state_dict's (#1540).
+        _ckpt_proj_w = _ckpt_model.get(f"backbone.0.encoder.{_patch_proj_suffix}")
+        if _ckpt_proj_w is None:
+            _ckpt_proj_w = _ckpt_model.get(f"{_LORA_ENCODER_KEY_PREFIX}{_patch_proj_suffix}")
         _ckpt_proj_shape = getattr(_ckpt_proj_w, "shape", None)
         if _ckpt_proj_shape is not None and len(_ckpt_proj_shape) == 4 and _ckpt_proj_shape[2] == _ckpt_proj_shape[3]:
             _inferred_ps = int(_ckpt_proj_shape[-1])