SLM-MLA Real-Hardware Toolkit

This repository addresses one concrete problem:

A complex optical element can be recovered with gradient-based optimization in simulation, but the same method does not predict measurements from the real SLM-object-camera bench. Which parts of the simulation are valid, which hardware effects are missing, and what code can be reused to continue?

The code therefore has two equally important parts:

1. A controlled simulator and optimizer. It generates camera intensities from known SLM patterns and a known complex target, then reconstructs the target with PyTorch complex autograd. This reproduces the case in which SGD is expected to work.
2. A measured-hardware validation pipeline. It reads Sony RAW data, subtracts measured backgrounds, estimates illumination and common brightness drift, recovers a supported relative phase from symmetric multi-plane data, and tests whether that recovered field predicts a matched measurement with a microlens array installed.

The project is not presented as a completed diffuser reconstruction. The final matched experiment found that the incident field recovered without the array did not yet predict pattern-specific measurements with the array. The code is useful precisely because it preserves that boundary and provides the next researcher with tested simulation, calibration, phase-recovery, and validation components instead of only a final optimization script.

• Measuring the real illumination is necessary; a uniform-beam assumption can make a model fail before optimization begins.
• Symmetric measurements at several camera positions recovered a band-limited relative phase. Adding it improved prediction at a camera distance excluded from that phase fit by 12.99% and 18.38% for two comparison patterns.
• Propagation to the recorded 10.5 cm microlens plane and back has a numerical round-trip error of about 5.3e-7. This checks the implementation of the propagation operator; it does not independently validate the physical relay, aperture, sampling, or distance measurement.
• In the matched experiment with the microlens array installed, the current absolute NRMSE remained 18.73% and 19.95%, and only 1 of 8 input patterns was identified, equal to random chance.
• Measurements made before and after an SLM-display reconnect changed different reference patterns by different amounts. The observation is associated with a display-state change, but the available images do not isolate which device component caused it. A single brightness correction cannot absorb the change.

The current evidence points to incomplete real-hardware calibration, stability, and field transfer as the main remaining problems. The numerical propagation operator passes its tests, but that alone does not prove that the complete physical relay is represented correctly.

jpsim/       Reusable optics, camera, reconstruction, RAW, and validation code
examples/    Small synthetic programs that run without experimental data
workflows/   Formal command-line analyses for real RAW datasets
configs/     Documented EXULUS-SE1 / 635 nm hardware parameters
data/        Mapping schema and a tiny example; no private RAW files
tests/       Synthetic unit and regression tests
docs/        Model explanation, code map, data contract, results, and handoff

Read docs/CODE_MAP.md before modifying the package. It explains which modules are simulation primitives, which are measurement corrections, and which are decision gates.

Researchers who want to reuse the professor's original simulation and complex target solver should start with docs/SIMULATION_REUSE_GUIDE.md. It explains what to replace for a new MLA or diffuser, how to change geometry and sampling, and how to distinguish an optimizer failure from a forward-model mismatch.

Python 3.10 or later is required.

git clone <repository-url>
cd slm-mla-real-hardware-toolkit
python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -e ".[dev]"
pytest -q

The same reproducible environment can be created with uv:

uv sync --extra dev
uv run pytest -q

rawpy is only needed when Sony .ARW files are decoded. The synthetic examples use NumPy, PyTorch, SciPy, and Matplotlib only.

With version 0.1.0, the default synthetic recovery should converge to a phase RMSE below 1e-3 rad and an amplitude NRMSE below 1e-3. These thresholds are control checks, not expected error levels for real measurements.

python examples/synthetic_complex_transmission_recovery.py \
  --iterations 300 --output outputs/synthetic_recovery

This example:

1. creates known phase patterns on a simulated SLM;
2. propagates them to a known microlens-array transmission;
3. records intensity-only camera frames;
4. rebuilds the same forward operator used to generate the data;
5. optimizes the complex transmission with Adam; and
6. reports amplitude NRMSE and gauge-aligned phase RMSE.

It is the clean control experiment that should pass before real data are used.

To see why hardware-model mismatch matters:

python examples/simulation_model_mismatch.py \
  --iterations 220 --output outputs/model_mismatch

The second example generates data with a shifted or incorrectly spaced optical system while the solver uses the nominal geometry. It is a controlled demonstration of model mismatch, not a claim that geometry was the only error in the real experiment.

python examples/lab_parameter_forward_model.py \
  --config configs/exulus_se1_635nm.yaml \
  --output outputs/lab_forward

This forward-only example uses the recorded wavelength, microlens pitch and focal length, and the 10.5 cm microlens-to-camera distance. Its analysis grid is coarser than the physical SLM pixels, which keeps the example small enough for a laptop. Use the measured sampling and relay model before comparing it with RAW frames.

The RAW archive is deliberately not included in Git. Each dataset needs:

• the original .ARW files;
• a frame_mapping.csv following docs/DATA_FORMAT.md;
• the exact displayed BMP files and their SHA-256 hashes; and
• a dataset directory containing the acquisition protocol and notes.

The formal order is:

# 1. Check a symmetric no-array axial scan.
slm-audit-zscan \
  --package-dir /path/to/run04d \
  --mapping /path/to/run04d/frame_mapping.csv \
  --raw-dir /path/to/run04d/raw \
  --output-dir analysis/zscan

# 2. Recover the supported incident phase from no-array multi-plane data.
slm-recover-phase \
  --run04d-dir /path/to/run04d \
  --run05-checkpoint /path/to/run05 \
  --data-dir /path/to/run05/raw \
  --output-dir analysis/incident_phase \
  --manufacturer-phase-prior 0:0,128:pi

# 3. Test whether that field predicts matched with-array measurements.
slm-validate-mla \
  --run05-package /path/to/run05 \
  --run06-package /path/to/run06 \
  --run05-phase-dir analysis/incident_phase \
  --raw-dir /path/to/raw_archive \
  --output-dir analysis/matched_mla

The commands enforce important restrictions: no per-pattern brightness fit, no per-pattern registration, background subtraction remains signed, and comparison patterns are not used to select the model.

• The simulator and complex-transmission optimizer for a new MLA or diffuser.
• The analytic microlens transmission and angular-spectrum propagation tests.
• The Sony RAW and frame-mapping pipeline.
• Common-flux, background, registration, and repeated-frame noise estimates.
• Symmetric multi-plane relative-phase recovery.
• The matched no-array / with-array validation and pattern-identification test.
• The explicit stop rules that prevent a low training loss from being reported as successful hardware prediction.

For the recommended continuation, see docs/NEXT_STEPS.md. For a clean-room reproduction checklist, see docs/REPRODUCIBILITY.md. Development and review conventions are documented in CONTRIBUTING.md, and release changes are listed in CHANGELOG.md.

Gray levels 0 and 128 are treated as a nominal 0 and pi pair using the manufacturer's typical 633 nm curve. This is an engineering prior, not a panel-specific calibration. The recovered phase is relative and band-limited; global phase and unsupported spatial frequencies cannot be obtained from these intensity measurements.

The quantitative evidence behind these statements, and the limits of each claim, are summarised in docs/RESULTS_AND_LIMITS.md. The full project report is not distributed with this repository.