Skip to content

Persist the Yjs doc state alongside the map rows #1389

Description

@JannikStreek

Depends on #1388, so that the stored Yjs state carries image references instead of image bytes.

Problem

The server keeps the Yjs doc of a map only in memory and persists it as mmp_node rows. When the doc is evicted (30s after the last client leaves) and the map is opened again, the doc is rebuilt from those rows with a new Yjs identity. Clients that kept their old doc across a disconnect (for example a laptop that slept for more than 30s) then merge two unrelated histories. The likely result:

  • nodes deleted by others in the meantime come back
  • concurrent changes to the same node property resolve arbitrarily

Proposal

Store the Yjs state of the map in the database and load from it. The node and map rows stay and keep being updated on every persist, so REST, duplicate and export keep working from them. The rows are only used to build the doc when no Yjs state exists yet.

Schema

New nullable column on mmp_map:

Column Notes
yDoc bytea, null until the map is first opened after this change; not loaded by default map queries

Example

id name yDoc nodes in mmp_node
m1 Old map null 12 rows
m2 Active map \x0103a1… (~4 KB) 12 rows

Opening m1 builds the doc from its rows and writes yDoc right away. Opening m2 loads the doc from yDoc; its rows are ignored for loading but still updated on every persist.

Loading and persisting

  • Load: if yDoc is set, load the doc from it. Otherwise build it from the rows as today and write yDoc immediately, so the doc keeps one identity from then on.
  • Persist: write yDoc and the rows in the same transaction.
  • Direct row writes: code paths that change rows without going through the doc (duplicate, seed job) set yDoc to null, so the next load rebuilds from the rows instead of ignoring the change. A duplicated map starts with yDoc null.

Migration

  1. A TypeORM migration adds the nullable column. No data is moved.
  2. Maps get their yDoc lazily, the first time they are opened.
  3. Maps never opened again keep yDoc null and remain fully usable from their rows.

Out of scope

Acceptance criteria

  • Opening a map without yDoc builds the doc from rows and stores yDoc
  • Opening a map with yDoc loads the doc from it
  • A client reconnecting after the doc was evicted does not bring back deleted nodes
  • Rows stay in sync with the doc after every persist
  • Duplicate and seed job reset yDoc

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions