ORB Feature-Extraction Accelerator (VHDL)

Synthesizable VHDL for the ORB-SLAM3 feature-extraction front end, targeting the programmable logic of an AMD Kria KR260 (Zynq UltraScale+ MPSoC, xck26-sfvc784-2LV-c). The accelerator takes over FAST detection, non-maximum suppression, corner scoring and keypoint orientation from ORBextractor. The processing system streams one image-pyramid level at a time into it over AXI DMA and reads back packed 128-bit keypoints.

Written for a Master's thesis at the University of the Bundeswehr Munich. The work was carried out during a research visit to San Diego State University.

This repository holds the RTL, its testbenches, the non-regenerable Vivado project sources, and the hand-written files the Kria needs to load the accelerator. The generated bitstreams are not here, and neither is the software side.

The matching ORB-SLAM3 fork lives on the kr260_hw_accelerator branch of brdsdsu/ORB_SLAM3. It holds the userspace driver (src/ORBHwAccelerator.cc), the ORBextractor changes that call into the hardware, and the stereo two-instance path. What it needs from this design is the interface contract documented below: the AXI4-Lite register map, the 128-bit keypoint layout, and the build ID that couples them.

Generic defaults are sized so that two instances fit on a K26, which is what a stereo configuration needs.

The design tracks OpenCV and ORB-SLAM3 semantics closely enough that the hardware keypoint set can be compared against the software one directly: FAST-9 with nonmaxSuppression=true, HARRIS_SCORE and FAST_SCORE as in ORBextractor.h, IC_Angle() moments with the ORB-SLAM3 u_max table, and the EDGE_THRESHOLD and 35-pixel cell grid from ORBextractor.cc.

        AXI4-Stream slave  (8-bit pixels, raster order, from AXI DMA MM2S)
                 |
                 v
          +--------------+
          |  window_NxN  |  31x31 sliding window, 30 line buffers, 1-cycle latency
          +--------------+
             |        |
             |        +--------------------+
             v                             v
      +--------------+             +-----------------+
      |  fast9_core  | latency 3   |     fast_nms    | latency 5
      |  detect only |             |  fast_response  |
      +--------------+             | + 3x3 local max |
             |                     +-----------------+
             |                             |
             +--------> push gate <--------+   corner AND local max AND
                            |                  not suppressed by the cell gate
                            v
                  +---------------------+
                  |   orientation_top   |
                  |  +---------------+  |
                  |  |  corner_fifo  |  |  31x31 patch + coords, URAM + BRAM,
                  |  +---------------+  |  zero-drop via backpressure
                  |  +---------------+  |
                  |  | moment_proc.  |  |  m10 / m01, 23 cycles
                  |  | harris_resp.  |  |  7x7 Sobel Harris, 19 cycles (G_SCORE_TYPE=0)
                  |  +---------------+  |
                  |  +---------------+  |
                  |  | cordic_atan2  |  |  22 iterations, 24-cycle latency
                  |  +---------------+  |
                  +---------------------+
                            |
                            v
                     +-------------+
                     |     TOP     |  AXI4-Lite registers, keypoint FIFO,
                     +-------------+  128-bit packing, EOF sentinel
                            |
                            v
        AXI4-Stream master (128-bit keypoints, to AXI DMA S2MM)

Everything inside extractor_top is aligned to T + ALIGN_LAT (5 cycles after the window output). The sub-module latency constants are cross-checked by static asserts and by a simulation-time self-check against the actual valid chains, so a latency change in one module fails the next testbench run instead of silently misaligning the pipeline.

window_NxN receives the image dimensions on ports rather than generics, so a single instance serves every pyramid level. Only the line-buffer depth is fixed at elaboration, via MAX_WIDTH, sized for level 0.

Two mechanisms keep the corner FIFO from overflowing:

• Backpressure. o_prog_full is wired into the pixel-stream tready gate in TOP, so the input stalls before the FIFO can fill. Tail drops became impossible. The cost is measured in STALLCNT (0x2C), and DROPCNT (0x18) should stay at zero.
• Strict-cell gate. A permissive corner is discarded when the ORB-SLAM3 selection cell it falls into already holds a strict corner, because the per-cell selection on the software side would discard it anyway. The grid math replicates ORBextractor.cc (EDGE_THRESHOLD = 19, nominal cell width 35). Suppressed corners are counted in SUPPCNT (0x28). Writing zero to CELLDIM and CELLNUM leaves the gate inert.

src/            synthesizable RTL
testbenches/    self-checking testbenches and their Python vector generators
vivado/         block design and packaged IP definitions
deploy/         device-tree overlay source and the xmutil app descriptor

32-bit registers, byte offsets from the base address of the block.

Per-frame sequence driven by the PS: write WIDTH and HEIGHT (and the cell grid) for the level, set CTRL[0], wait for STATUS[1], clear CTRL[0], repeat for the next level. The control FSM holds the core in reset while idle, so every level starts from clean counters.

ID is the compatibility contract with the driver, and the keypoint layout below is coupled to it. Bump the build ID whenever the layout changes. THRESH[16] lets the driver verify at probe time that the loaded bitstream was built with the score type it expects.

The dual-threshold scheme mirrors the iniThFAST and minThFAST adaptation in ORB-SLAM3. Detection always runs at the permissive threshold, and each keypoint carries passed_strict, so the software reproduces the two-pass behavior without a second hardware pass. Detection and NMS are FAST-based in both scoring modes, so the corner set never depends on G_SCORE_TYPE. Only the ranking value does.

The testbenches are self-checking and were developed under ModelSim and QuestaSim. They also analyze and run under GHDL. Compile the packages first (feature_pkg, fast_pkg, umax_pkg), then the module under test, then its testbench.

The testbenches open their vector files by paths relative to the repository root, for example testbenches/cordic_vectors.hex, so run the simulator from the repository root and the generators from inside testbenches/.

The stimulus and golden-output files (*.hex, *expected*.txt) are not in the repository. They amount to roughly 78 MB of generated data and are reproduced from the Python generators that are tracked here. Each generator is the golden model for its module, so generator and testbench always belong together.

python -m venv .venv

pip install -r testbenches/0_requirements.txt

The generators are golden models, so 0_requirements.txt pins exact versions rather than leaving them open: a change in numpy integer or rounding behavior would alter the golden data silently instead of failing. The thesis vectors were produced with numpy 2.4.6 and tqdm 4.67.3 on CPython 3.14.5. Only numpy and tqdm are listed, because the remaining imports are standard library modules that pip has nothing to install for. The leading 0 in the filename exists purely to sort it above the files it describes.

cd testbenches && python window_NxN_generator.py && python fast9_core_generator.py && python fast_response_generator.py && python fast_nms_generator.py && python harris_response_generator.py && python moment_processor_generator.py && python cordic_atan2_generator.py && python orientation_top_generator.py && python extractor_top_generator.py && python pyramid_tb_generator.py && python top_tb_generator.py

Every generator takes -h. Image size, seed, thresholds and score type are parameters, for example python extractor_top_generator.py -W 256 -H 224 --score-type harris. Each one also writes a *_test_params_pkg.vhd, which is tracked, so the parameters a given run used remain visible in the repository even without the vectors.

extractor_top_tb and extractor_pyramid_tb take the expected-output file as a generic, so the same testbench covers both scoring modes:

-gG_USE_HARRIS=true -gG_EXPECTED_FILE=testbenches/extractor_expected_harris.txt

fast_score_consistency.py cross-checks the FAST score model against the detection model and is not tied to a single testbench.

The RTL is added to the Vivado block design by path as a module reference, so src/ stays the single source of truth and there is no mirrored copy inside the Vivado project.

When packaging as a custom IP the file types matter. UG1118 requires the designated top-level file to be plain Verilog or VHDL-93/2002, which is why TOP_vivado_wrapper_vhdl_2002.vhd (entity orb_feature_top) exists. Mark that file and feature_pkg.vhd as VHDL, everything else as VHDL 2008.

G_CORE_URAM_COLS = 32 gives 32 URAM plus 52 BRAM36 per instance, sized so two instances fit the K26. G_KP_FIFO_DEPTH must be a power of two, because the pointer wrap relies on it, and G_PROG_FULL_GAP must be at least G_CORE_FIFO_DEPTH plus the pipeline tail. Both constraints are checked by asserts at elaboration.

Only the parts of the Vivado project that a build cannot regenerate. Synthesis runs, implementation checkpoints, the IP cache and the generated output products are excluded.

To rebuild:

vivado -mode batch -source vivado/create_project.tcl

Two environment prerequisites, neither of which is a file in this repository. The project was written by Vivado 2025.2.1 and uses the Vivado Synthesis 2025 and Vivado Implementation 2025 flows, and HW_Acc_bd.tcl refuses to run on anything whose version string does not contain 2025.2. The board part xilinx.com:kr260_som:part0:1.1 requires the Kria board files to be installed.

Nothing else is needed. The constraint set is empty, since the design is entirely PS-clocked with no PL pins of its own, and every IP in the block design comes from the standard Vivado catalog, so no external IP repository has to be registered.

The project references HW_Acc.bd in place rather than copying it, so Vivado treats this folder as its working directory for the block design. After a run you will find:

Only the first one is tracked, so the others stay invisible. Discard the rewritten block design unless you actually changed it, which keeps the committed file the small, readable one rather than a Vivado re-serialization:

git checkout -- vivado/HW_Acc.bd

This matters most after a failed run, which can leave the file half-written.

The block design instantiates orb_feature_top and axi_axcache_override as module references, so Vivado re-infers their IP definitions from src/ when the block design is generated. src/ stays the single source of truth and the project holds no copy of it.

reports_post_route.tcl runs automatically at the end of route_design. To run it standalone against an already routed checkpoint, pass the configuration and the checkpoint:

vivado -mode batch -source vivado/reports_post_route.tcl -tclargs HARRIS /path/to/HW_Acc_wrapper_routed_HARRIS.dcp

Checkpoints live outside this repository, so the script has no default location for them. Either give the .dcp path as the second argument, as above, or set ARCHIVE_ROOT in the environment and the script derives $ARCHIVE_ROOT/$CFG/HW_Acc_wrapper_routed_$CFG.dcp. Verify the configuration afterwards from the DSP count in the hierarchical utilization report: 2 for a FAST build, 60 for Harris.

xmutil loadapp reads one firmware directory, /lib/firmware/xilinx/HW_Acc/, and expects three files in it:

The bitstream itself is a build product and is not in this repository, but the step that converts it is. Copy HW_Acc_wrapper.bit out of the implementation run into deploy/, then run bootgen against the tracked image description HW_Acc_wrapper.bif:

bootgen -image HW_Acc_wrapper.bif -arch zynqmp -process_bitstream bin -w -o HW_Acc_wrapper.bit.bin

The output name matters: it is what the firmware-name property in HW_Acc.dts looks for.

shell.json is the app descriptor. XRT_FLAT with one slot tells xmutil this is a flat (non-DFX) design occupying the whole PL.

HW_Acc.dts is the device-tree overlay. It loads the bitstream named by its firmware-name property, sets the PL clock to 125 MHz, and exposes the memory-mapped register files to userspace as generic-uio nodes. It is written for the dual-accelerator build, and because the second instance is simply appended, the same overlay also serves a single-instance bitstream. The unused instance-1 nodes just sit idle.

dtc -@ -O dtb -o HW_Acc.dtbo HW_Acc.dts

-@ keeps the symbol table the overlay needs to resolve &fpga_full and &amba.

Expected UIO probe order, assuming the base platform occupies uio0 through uio3:

Verify the numbering on the board before relying on it:

for d in /sys/class/uio/uio*; do echo "$d -> $(cat $d/name)"; done

The driver polls DMASR and never opens the interrupt nodes. They exist only so that uio_pdrv_genirq registers and masks the wired PL-to-PS lines, because an unhandled level-high SPI would otherwise storm.

The pixel and keypoint buffers are not device-tree nodes. They are created by loading the out-of-tree u-dma-buf module, one input and one output buffer per instance:

sudo insmod u-dma-buf.ko udmabuf0=1048576 udmabuf1=1048576 udmabuf2=1048576 udmabuf3=1048576