Skip to content

docs: Add import/export architecture concept for project portability - #312

Open
jankapunkt with Copilot wants to merge 4 commits into
mainfrom
copilot/concept-project-import-export
Open

jankapunkt with Copilot wants to merge 4 commits into
mainfrom
copilot/concept-project-import-export

Conversation

Copilot AI commented Mar 20, 2026 •

Copy link
Copy Markdown
Contributor

OpenQDA needs to support five import/export channels (full archive, REFI-QDA, publication data, import full archive, import REFI), each varying significantly in data scope and transformation logic, while sharing infrastructure. This document proposes the architecture before implementation begins.

Architecture Overview

Layered design: HTTP Layer → Queue Job → Pipeline (Pipes) → Format Strategy → Data Access

  • Strategy + Registry — ExporterInterface / ImporterInterface per format; a ExporterRegistry (bound in ImportExportServiceProvider) resolves format strings to concrete handlers. New formats = new class + one registration, no existing code changes (OCP).
  • Pipeline — Each handler composes ordered Pipe classes via Laravel's Pipeline facade (SRP; steps are independently testable and reorderable).
  • Value Objects — Typed ExportContext / ExportResult DTOs instead of raw arrays flowing between layers.
  • Optional Template Method — Abstract BaseExporter for shared prepareTempDir → writeContent → packageOutput → cleanup skeleton, only if that skeleton stabilises.

Resource & Consistency Concerns

  • All export/import dispatched as ShouldQueue jobs on a dedicated exports queue — web processes never block.
  • Atomic locks (Cache::lock) prevent duplicate concurrent exports of the same project+format.
  • Exports wrapped in REPEATABLE READ transactions for point-in-time snapshot consistency.
  • Imports wrapped in DB::transaction() — atomic or nothing.
  • Relationship data read via cursor() / chunk() to keep memory O(1) relative to project size.

REFI-QDA

Mapping logic isolated in a RefiMapper class (not inside the exporter/importer). openqda/refi-tools recommended as the XML schema dependency before writing custom XML handling.

Changes

  • docs/installation/preparations.md — concept document added as a new top-level section (# Import and Export Architecture Concept) covering: architecture diagram, all design patterns, queue setup, DB strategy, proposed directory layout, REFI specifics, publication export, HTTP API sketch, testing strategy, and dependency evaluation.
  • docs/.vitepress/config.mts — sidebar entry added linking to the new section.

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 2 commits March 20, 2026 10:40
Co-authored-by: jankapunkt <1135285+jankapunkt@users.noreply.github.com>
Co-authored-by: jankapunkt <1135285+jankapunkt@users.noreply.github.com>
Copilot AI changed the title [WIP] Draft comprehensive concept for project import and export docs: Add import/export architecture concept for project portability Mar 20, 2026
Copilot AI requested a review from jankapunkt March 20, 2026 10:43
@jankapunkt
jankapunkt marked this pull request as ready for review March 21, 2026 07:01
@jankapunkt jankapunkt added this to the 1.1.0 milestone Apr 23, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants