Depends on #1388, so that the stored Yjs state carries image references instead of image bytes.
Problem
The server keeps the Yjs doc of a map only in memory and persists it as mmp_node rows. When the doc is evicted (30s after the last client leaves) and the map is opened again, the doc is rebuilt from those rows with a new Yjs identity. Clients that kept their old doc across a disconnect (for example a laptop that slept for more than 30s) then merge two unrelated histories. The likely result:
- nodes deleted by others in the meantime come back
- concurrent changes to the same node property resolve arbitrarily
Proposal
Store the Yjs state of the map in the database and load from it. The node and map rows stay and keep being updated on every persist, so REST, duplicate and export keep working from them. The rows are only used to build the doc when no Yjs state exists yet.
Schema
New nullable column on mmp_map:
| Column |
Notes |
yDoc |
bytea, null until the map is first opened after this change; not loaded by default map queries |
Example
| id |
name |
yDoc |
nodes in mmp_node |
m1 |
Old map |
null |
12 rows |
m2 |
Active map |
\x0103a1… (~4 KB) |
12 rows |
Opening m1 builds the doc from its rows and writes yDoc right away. Opening m2 loads the doc from yDoc; its rows are ignored for loading but still updated on every persist.
Loading and persisting
- Load: if
yDoc is set, load the doc from it. Otherwise build it from the rows as today and write yDoc immediately, so the doc keeps one identity from then on.
- Persist: write
yDoc and the rows in the same transaction.
- Direct row writes: code paths that change rows without going through the doc (duplicate, seed job) set
yDoc to null, so the next load rebuilds from the rows instead of ignoring the change. A duplicated map starts with yDoc null.
Migration
- A TypeORM migration adds the nullable column. No data is moved.
- Maps get their
yDoc lazily, the first time they are opened.
- Maps never opened again keep
yDoc null and remain fully usable from their rows.
Out of scope
Acceptance criteria
Depends on #1388, so that the stored Yjs state carries image references instead of image bytes.
Problem
The server keeps the Yjs doc of a map only in memory and persists it as
mmp_noderows. When the doc is evicted (30s after the last client leaves) and the map is opened again, the doc is rebuilt from those rows with a new Yjs identity. Clients that kept their old doc across a disconnect (for example a laptop that slept for more than 30s) then merge two unrelated histories. The likely result:Proposal
Store the Yjs state of the map in the database and load from it. The node and map rows stay and keep being updated on every persist, so REST, duplicate and export keep working from them. The rows are only used to build the doc when no Yjs state exists yet.
Schema
New nullable column on
mmp_map:yDocbytea, null until the map is first opened after this change; not loaded by default map queriesExample
mmp_nodem1nullm2\x0103a1…(~4 KB)Opening
m1builds the doc from its rows and writesyDocright away. Openingm2loads the doc fromyDoc; its rows are ignored for loading but still updated on every persist.Loading and persisting
yDocis set, load the doc from it. Otherwise build it from the rows as today and writeyDocimmediately, so the doc keeps one identity from then on.yDocand the rows in the same transaction.yDocto null, so the next load rebuilds from the rows instead of ignoring the change. A duplicated map starts withyDocnull.Migration
yDoclazily, the first time they are opened.yDocnull and remain fully usable from their rows.Out of scope
yDocAcceptance criteria
yDocbuilds the doc from rows and storesyDocyDocloads the doc from ityDoc