FedCKA: Representation-Guided Layer Personalization for Federated 3D Perception

📄 Paper (arXiv)

Code accompanying FedCKA: Representation-Guided Layer Personalization for Federated 3D Perception Across Driving Domains.

FedCKA is a personalized federated learning method for 3D object detection under heterogeneous driving domains. It uses Centered Kernel Alignment (CKA) to measure layer-wise representation similarity, creating dynamic client-specific aggregation masks that determine which layers remain globally shared and which are locally personalized.

FedCKA dynamically identifies client-specific layers through representation similarity. After local training, layer-wise CKA scores determine which layers remain personalized and which are shared through global aggregation.

.
├── mmdet/                              # Main project directory
│   ├── image/                          # Apptainer definition file and container setup
│   │   └── README.md                   # Container build and verification instructions
│   └── mmdetection3d/                  # Core framework and project workspace
│       ├── mmdet3d/                    # Customized mmdet3d runner
│       ├── projects/                   # All experiments and configurations
│       │   ├── analysis/               # Evaluation, class balance, and qualitative results
│       │   ├── cmt_40_epoch/           # Main federated experiments and model merging tools
│       │   ├── cmt_full/               # Full CMT baseline replication
│       │   ├── mmdet3d_plugin/         # Custom FP16 CMT plugin for small batch sizes
│       │   ├── subsets_creation/       # nuScenes split generation and .pkl metadata files
│       │   └── README.md               # Projects guide, path configurations, and setups
│       ├── tools/                      # Adjusted train and evaluation scripts
│       └── README.md                   # Base MMDetection3D clone and checkpoint setup
└── README.md                           # Main project overview, NDS results, and citations

Comparison of Federated Learning methods across nuScenes domains. Performance is reported as nuScenes Detection Score (NDS).

FedCKA particularly benefits challenging domains with strong distribution shifts. The example below shows predictions on the Singapore night-rain domain, comparing Ground Truth, FedAvg, and FedCKA.

Qualitative comparison under severe domain shift. FedCKA recovers detections missed by the standard FedAvg baseline.

The required .sif container, Flash Attention wheel, and pretrained checkpoints are hosted on Hugging Face.

Download the image components directly into the /image directory:

cd /YOUR_PATH_HERE/mmdet/image
wget -c [https://huggingface.co/datasets/Anon-fedcka-ICRA/essentials/resolve/main/unibev_cuda113_ubuntu2004_gcc10.sif](https://huggingface.co/datasets/Anon-fedcka-ICRA/essentials/resolve/main/unibev_cuda113_ubuntu2004_gcc10.sif)
wget -c [https://huggingface.co/datasets/Anon-fedcka-ICRA/essentials/resolve/main/flash_attn-0.2.2+cu113torch1.11.0-cp38-cp38-linux_x86_64.whl](https://huggingface.co/datasets/Anon-fedcka-ICRA/essentials/resolve/main/flash_attn-0.2.2+cu113torch1.11.0-cp38-cp38-linux_x86_64.whl)

Create the checkpoints directory and download the required .pth weights into it:

mkdir -p /YOUR_PATH_HERE/mmdet/mmdetection3d/ckpts
cd /YOUR_PATH_HERE/mmdet/mmdetection3d/ckpts
wget -c [https://huggingface.co/datasets/Anon-fedcka-ICRA/essentials/resolve/main/nuim_r50.pth](https://huggingface.co/datasets/Anon-fedcka-ICRA/essentials/resolve/main/nuim_r50.pth)
wget -c [https://huggingface.co/datasets/Anon-fedcka-ICRA/essentials/resolve/main/fcos3d_vovnet_imgbackbone-remapped.pth](https://huggingface.co/datasets/Anon-fedcka-ICRA/essentials/resolve/main/fcos3d_vovnet_imgbackbone-remapped.pth)

Paper: FedCKA: Representation-Guided Layer Personalization for Federated 3D Perception Across Driving Domains

If you use this work, please cite:

@misc{verhoog2026fedcka,
  title={FedCKA: Representation-Guided Layer Personalization for Federated {3D} Perception Across Driving Domains},
  author={Verhoog, Jolle and Burak Ünal, Ali and Caesar, Holger},
  year={2026},
  eprint={ARXIV_ID},
  archivePrefix={arXiv},
  primaryClass={PRIMARY_CATEGORY},
  url={https://arxiv.org/abs/ARXIV_ID}
}

During the development of this codebase, the authors utilized ChatGPT, Google Gemini, and GitHub Copilot in Agent Mode as programming assistants. The system was used strictly as an assistive tool for debugging, expanding existing functions, generating boilerplate code, and replacing repetitive tasks. All AI-generated code was thoroughly reviewed, tested, and modified by the authors to ensure correctness and compatibility within the MMDetection3D environment. The authors assume full responsibility for the functionality, logic, and integrity of the code in this repository.

• [1] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y. Arcas, "Communication efficient learning of deep networks from decentralized data," in Proc. AISTATS, 2017, pp. 1273–1282.
• [2] D. A. E. Acar, Y. Zhao, R. Matas Navarro, M. Mattina, P. N. Whatmough, and V. Saligrama, "Federated learning based on dynamic regularization," in Proc. ICLR, 2021.
• [3] T. Yu, S. Kumar, A. Gupta, S. Levine, K. Hausman, and C. Finn, "Gradient surgery for multi-task learning," in Proc. NeurIPS, 2020, pp. 5824–5836; and X. Zhang, W. Sun, and Y. Chen, "Tackling the non-IID issue in heterogeneous federated learning by gradient harmonization," IEEE Signal Processing Letters, vol. 31, pp. 2595–2599, 2024.
• [4] L. Collins, H. Hassani, A. Mokhtari, and S. Shakkottai, "Exploiting shared representations for personalized federated learning," in Proc. ICML, 2021, pp. 2089–2099.
• [5] X. Li, M. Jiang, X. Zhang, M. Kamp, and Q. Dou, "FedBN: Federated learning on non-IID features via local batch normalization," in Proc. ICLR, 2021.
• [6] Y. Gao, X. He, and Y. Chen, "Personalized federated learning algorithm based on information content model customization," in Proc. CAMMIC, 2025, pp. 834–838.
• [7] R. Tamirisa, C. Xie, W. Bao, A. Zhou, R. Arel, and A. Shamsian, "FedSelect: Personalized federated learning with customized selection of parameters for fine-tuning," in Proc. CVPR, 2024, pp. 29485–29494.
• [8] MMDetection3D Contributors, "MMDetection3D: OpenMMLab next-generation platform for general 3D object detection," 2020. [Online]. Available: https://github.com/open-mmlab/mmdetection3d
• [9] J. Yan et al., "Cross modal transformer: Towards fast and robust 3D object detection," in Proc. ICCV, 2023, pp. 18268–18278.

Original FedCKA contributions are licensed under the Apache License, Version 2.0; see LICENSE. Third-party code retains its applicable licenses and copyright notices. The FedCKA license does not replace those terms.

This repository builds upon MMDetection3D and CMT. Both projects are distributed under Apache-2.0. Their retained code includes contributions originating from other projects, including DETR3D and UVTR.

Paths in the table are relative to mmdet/mmdetection3d/. Existing source-file notices remain applicable. When redistributing this code, preserve the applicable copyright, attribution, and license notices, including those reproduced here. Modified upstream files must carry a notice identifying their modification for FedCKA.

The following license applies to retained DETR3D-derived portions:

MIT License

Copyright (c) 2021 Wang, Yue

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

The Apache-2.0 license for FedCKA does not grant additional rights to nuScenes data, pretrained checkpoints, container images, or third-party binary dependencies linked from this repository. Those materials remain subject to their respective licenses and terms. Obtain and use nuScenes through its official website.