CUDA C++ Performance Engineering Portfolio

This is an in-progress portfolio in correctness-first performance engineering for scientific and high-performance computing. Project 1 Phases A and B are complete for their established finite-float64 Windows contracts. Phase A took a four-dimensional fixed median from an authoritative Python/4Denoise reference through C++17, CPU profiling and optimization, OpenMP, profiled CUDA experiments, transfer/residency analysis, and Python integration. Phase B extended that process to adaptive median filtering, ending with validated persistent adaptive CUDA ownership for Python.

The retained Phase B split CUDA path includes a measured balanced nine-value min/max reduction. A lower-dependency median network was tested and rejected after a repeatable regression. Native transfer/residency characterization justified explicit GPU ownership. Project 2, general-purpose CUDA K-means, is complete for its tested Windows scope: its Python reference, serial C++ baseline, native profile, measured assignment-addressing improvement, assignment-only OpenMP scaling, correctness-first CUDA baseline, focused count/update profiling, deterministic tiled FP64 centroid reduction, parallel integer counting, native lifecycle characterization, supported persistent native ownership, correctness-first Python CUDA integration, and controlled scikit-learn comparison are complete. The retained CUDA path preserves exact per-update counts, labels/termination, and centroid bits on the tested workloads. The native owner validates/uploads once, runs independent resident fits, and downloads explicitly; both workflows are exposed through pybind11, with explicit host copies; the controlled scikit-learn comparison and final documentation/hygiene closeout are complete. Project 3 now has a proposed conventional FP32 GEMM contract awaiting approval before reference/fixture work. The earlier 3D-median concept is deferred because its incremental portfolio value is currently smaller and its scientific transformation remains undefined.

Project 1 studies scientific neighborhoods and exact numerical semantics. Project 2 studies iterative assignment, deterministic reduction, convergence and residency; its closeout separates stage-specific improvements, complete one-shot calls and amortized resident workflows. Project 3 proposes dense FP32 arithmetic, data reuse and arithmetic intensity against a same-mode cuBLAS ceiling. Tensor Core/TF32 work is optional later under a separate contract. Project 2's broader crossover, GPU-library comparison and Linux validation remain deferred, not completion blockers.

The canonical workload contains 47,228,125 outputs with shape (85, 35, 127, 125). All reported measurements below came from one Windows 11 laptop with an Intel Core i7-13700H (14 physical cores, 20 logical processors) and an RTX 4070 Laptop GPU. CPU measurements used MSVC x64 Release /O2 and included internal output allocation; CUDA values distinguish kernel execution from the H2D-plus-kernel-plus-D2H path. They are machine-specific results, not universal performance claims.

• The original correctness-first C++ median was 3.756627 s.
• Sampling identified general-purpose std::nth_element as the dominant original hotspot.
• A fixed 19-comparator median-of-nine network produced a measured 1.135× speedup against its fresh same-session baseline.
• Direct reusable C-order addressing produced another measured 1.105× speedup against its fresh median-of-nine baseline.
• Reprofiling found selection still dominant at approximately 70% of relevant samples.
• Static OpenMP decomposition reached 319.583 ms at 20 threads: 8.729× versus the fresh 2.789567 s optimized-serial baseline and 147.781 million outputs/s.
• The CUDA baseline matched all 47,228,125 canonical optimized-serial outputs bit for bit. A timing audit established a 6.888 ms reusable-buffer steady-state kernel median; three subsequent kernel candidates did not justify replacing it. In the native transfer study, the pageable one-call median was 99.866 ms versus a fresh 336.586 ms OpenMP-20 median (3.370×). Transfers consumed a median 93.456% of that path; pinned staging cut the transfer pair by about one-third, while ten native device-resident operations reduced effective time to 17.083 ms/filter.
• The pybind11 interface exposes optimized serial, explicit-thread-count OpenMP, optional one-shot CUDA, and CudaMedianBuffer resident workflows with strict finite, four-dimensional, C-contiguous float64 validation. Direct NumPy buffers made serial 1.097× and OpenMP-20 1.658× faster than their copied binding baselines. Persistent CUDA reduced median effective Python workflow time from 0.835 s/filter for repeated one-shot calls to 0.0436 s/filter across ten resident calls (19.170×) in the measured session. All 47,228,125 canonical outputs remained bit-for-bit exact.

The optimized candidates matched the established Python and preceding C++ outputs bit for bit. Raw runs, benchmark boundaries, full scaling data, profiling caveats, and implementation progression are in the Project 1 technical report.

For Phase B adaptive median, the persistent Python CUDA owner matched all 47,228,125 canonical outputs bit for bit. In one controlled session, a new one-call owner took 184.567 ms median; reusing an owner for 20 independent filters with one upload and final download took 28.221 ms/filter (6.540× lower effective Python wall time). The resident measurement excludes owner construction; full timing boundaries and variable-laptop caveats are in the technical report.

Python reference
  → correctness-first C++17
  → profiled and optimized serial C++
  → portable OpenMP multicore CPU
  → correctness-first CUDA C++  [complete]
  → CUDA baseline profiling     [complete]
  → CUDA optimization checkpoint [complete]
  → transfer/residency analysis  [complete]
  → correctness-first Python interface [complete]
  → direct NumPy-buffer CPU bindings [complete]
  → persistent-GPU Python ownership [complete]
  → final Phase A validation/report [complete]
  → Phase B correctness-first native C++ [complete]
  → Phase B native benchmark and profile [complete]
  → Phase B fixed-stack storage [complete]
  → Phase B specialized nine-value selection [complete]
  → Phase B direct-row gathering [complete]
  → Phase B portable OpenMP scaling [complete]
  → Phase B correctness-first adaptive CUDA [complete]
  → Phase B adaptive CUDA profiling [complete]
  → Phase B split common-path CUDA [complete]
  → Phase B split-kernel profiling [complete]
  → Phase B min/max reduction experiment [complete]
  → Phase B median-network experiment [complete; rejected]
  → Phase B CUDA transfer/residency characterization [complete]
  → Phase B persistent adaptive Python ownership [complete]
  → Phase B validation and closeout [complete]
  → Project 2 K-means Python reference and deterministic fixtures [complete]
  → Project 2 straightforward serial C++ [complete]
  → Project 2 native CPU profiling [complete]
  → Project 2 isolated assignment-addressing experiment [complete]
  → Project 2 ordered distance-loop experiment [complete; rejected]
  → Project 2 assignment-only portable OpenMP CPU [complete]
  → Project 2 correctness-first CUDA [complete]
  → Project 2 CUDA count/update profiling [complete]
  → Project 2 deterministic centroid reduction [complete]
  → Project 2 parallel integer count experiment [complete]
  → Project 2 transfer/residency and native host-overhead characterization [complete]
  → Project 2 supported persistent native CUDA ownership [complete]
  → Project 2 pybind11 one-shot and resident-owner integration [complete]
  → Project 2 fair scikit-learn comparison [complete]
  → Project 2 final closeout [complete]
  → Project 3 FP32 GEMM contract [proposed; approval pending]
  → Project 3 authoritative reference and fixtures [after approval]

The work follows a controlled loop: define numerical behavior, validate exactly, establish a fresh baseline, profile, change one meaningful variable, and remeasure.

01_4DSTEM_Median_Filter_Acceleration/
    cpp/                         C++17 serial/OpenMP/CUDA source and CMake build
        include/adaptive_median.hpp
                                 correctness-first Phase B native API
        include/adaptive_median_detail.hpp
                                 tested selector primitive and internal experiment hooks
        include/adaptive_median_cuda.hpp
                                 Phase B CUDA API and timing result types
        include/adaptive_median_cuda_transfer_experiment.hpp
                                 internal Phase B transfer/residency measurements
        include/fixed_median_cuda.hpp
                                 CUDA interface, RAII owner, and timing result types
        src/fixed_median_cuda.cu baseline kernel, one-shot path, and resident buffers
        src/python_bindings.cpp  pybind11 fixed CPU/CUDA and persistent adaptive CUDA module
        src/cuda_validation.cpp  exact fixture-validation executable
        src/cuda_benchmark.cpp   serial/OpenMP/CUDA benchmark executable
        src/cuda_kernel_benchmark.cpp
                                 reusable-buffer steady-state kernel benchmark
        src/cuda_transfer_characterization.cpp
                                 pageable/pinned, residency, and crossover benchmark
        src/adaptive_median.cpp  retained adaptive baselines and optimized path
        src/adaptive_benchmark.cpp
                                 native timing and diagnostic-counter harness
        src/adaptive_validation.cpp
                                 public exact Phase B validation executable
        src/adaptive_median_cuda.cu
                                 adaptive baseline, retained split/balanced kernels, and internal transfer study
        src/adaptive_cuda_validation.cpp / adaptive_cuda_benchmark.cpp
                                 exact adaptive CUDA validation and timing harnesses
        src/adaptive_cuda_transfer_characterization.cpp
                                 native pageable/pinned/resident adaptive CUDA study
    reference_data/public_adaptive/
                                 public deterministic adaptive branch fixture
    reference_data/public_synthetic/
                                 public deterministic correctness fixture
    benchmark_data/              instructions for local compatible inputs
    benchmarks/                  Python/4Denoise benchmark harness
    python/validate_bindings.py  public binding smoke and correctness test
    python/validate_adaptive_cuda_owner.py
                                 adaptive Python owner correctness and timing
    *.ipynb                      output-cleared scientific reference notebooks
02_4DSTEM_3D_Median_Filter/      deferred concept; legacy directory prefix
03_CUDA_KMeans/                  completed Project 2; Python oracle, public workloads, serial/OpenMP/CUDA C++ paths and benchmark harness; legacy prefix
04_CUDA_Matrix_Multiplication/   proposed Project 3 FP32 GEMM contract; awaiting approval; legacy prefix
ROADMAP.md                       staged development plan and completion state

The placeholder directories retain their original numeric prefixes for now; no project files have been moved. The roadmap and project README titles give the intended project order.

The default CPU build requires CMake 3.24+, a C++17 compiler, and a compiler-supported OpenMP runtime. CUDA targets are opt-in and additionally require a compatible CUDA toolkit. The pybind11 extension is separately opt-in and requires pybind11 in the selected Python environment.

Windows with Visual Studio 2022:

cmake -S 01_4DSTEM_Median_Filter_Acceleration/cpp -B 01_4DSTEM_Median_Filter_Acceleration/cpp/build -G "Visual Studio 17 2022" -A x64
cmake --build 01_4DSTEM_Median_Filter_Acceleration/cpp/build --config Release
01_4DSTEM_Median_Filter_Acceleration\cpp\build\Release\phase_a_fixed_median.exe

Linux with GCC or Clang and an installed OpenMP runtime:

cmake -S 01_4DSTEM_Median_Filter_Acceleration/cpp -B 01_4DSTEM_Median_Filter_Acceleration/cpp/build -DCMAKE_BUILD_TYPE=Release
cmake --build 01_4DSTEM_Median_Filter_Acceleration/cpp/build
./01_4DSTEM_Median_Filter_Acceleration/cpp/build/phase_a_fixed_median

The default executable validates the retained straightforward, median-of-nine, optimized-serial, and OpenMP implementations against a small deterministic synthetic fixture. Linux compilation and scaling are intended portability targets but have not yet been verified on the planned Lambda environment.

The Phase A and Phase B CUDA validation/benchmark targets are enabled with PHASE_A_ENABLE_CUDA=ON; platform-specific CUDA configuration and reproduction commands are kept in the Project 1 technical report.

Experimental arrays are intentionally not distributed. The benchmark accepts another little-endian float64, C-contiguous, nonempty four-dimensional NumPy array with axis order (scan_y, scan_x, detector_y, detector_x):

phase_a_baseline_benchmark <compatible-input.npy> --threads 1,2,4,8

Use thread counts supported by the current machine. The program performs a full serial-versus-OpenMP bitwise comparison before timing, then runs one warm-up and three filter-only trials per configuration. See benchmark data guidance.

The upstream experimental array, prepared canonical benchmark input, and experimental correctness slices remain local and are ignored by Git because redistribution rights are not established. The committed synthetic fixture verifies executable behavior but does not reproduce the documented experimental benchmark. Notebook outputs were cleared before publication to avoid embedding local paths or experimental renderings; their source cells document the reference methodology and require separately obtained 4Denoise software and local data to rerun.

AI coding agents were used as development tools. I remained responsible for problem specification, scientific and numerical contracts, algorithm and optimization decisions, code review, debugging, correctness validation, benchmark design, and performance interpretation. This repository does not claim that every generated line was manually authored.

See ROADMAP.md for completed and planned milestones and LEARNING_NOTES.md for the evolving technical study outline.