Imported from Xantibody/magical-merchant (
.claude/skills/sync-storage/SKILL.md). Install upstream withnpx skills add Xantibody/magical-merchant --skill sync-storage. Copyright stays with the author.
Storage & Sync
Note storage invariants
- Filename
YYYYMMDD_HHMMSS.mdis an immutable ID stamped at creation. Syncthing, widget deep links (?file=) and list-row identity all point at it. Never rename to match the title - Title = the body's leading
# heading(note-title.tssplits it out for the title field and writes it back on save). Never a frontmatter key — one source of truth, and the file stays readable elsewhere. The list still derives its row title from the first body line - Frontmatter (
time/tags/context) is a record of creation, preserved verbatim on every edit.timemust stay the creation time (list sorts by filename = creation order) updated(optional frontmatter): when the body was last rewritten. Stamped only byNotes::update— metadata edits and view toggles are not rewrites. Absent until the first edit, likeviewview(optional frontmatter): per-note display mode (mindmap,preview). A preference, not a record — omitted unless set so untouched notes stay byte-identical. Unknown values resolve to the editor, so a new value never breaks an older build- Any new frontmatter key must be a typed field on
NoteFrontmatterin Rust core; unknown keys are dropped on the next save - Revision guard:
read_notereturnsRevision::of(body); every body write passes it toupdate_note, which refuses withCoreError::Staleif the body moved. The app parks the typed text in the edit backup and reloads; the CLI keeps it in a scratch file; MCP returns the error. The revision covers the body only, so metadata edits never make a save stale. No writer watches the filesystem — this guard is the only protection - Tags come from the body's
#記法. Identity is the ASCII-lowercased form (#Rust=#rust; Japanese is left alone) and code — fences and spans — is not scanned. The same rule lives twice:core/src/utils/tags.rsandlib/tags.ts; fix both - Place names (
places.json, outsidedata/): derived geocoder cache, display-only. The recordedlocationstays the raw coordinate. Keyed by<locale>:<coordinate>— the OS answers in whatever language it was asked, so the language has to be part of the key
Sync protocol (Workers + R2)
The Worker owns sync state: _sync-state/<user>.json maps every key to a
content hash + server-issued version stamp. One sync = GET /sync-state →
local scan → diff → POST /sync/bulk, repeated until nothing is left over.
| Client sees | Action |
|---|---|
| Hash differs | Upload (new hash, new stamp) |
| Stamp differs | Download |
| Both differ | Conflict — local wins, new stamp |
| Gone locally, stamp matches | Delete remote |
| Gone remotely, hash matches | Delete local |
| No state, hashes match | Nothing to transfer — record only |
- The client never sends its own state (would read undownloaded keys as deletions and erase notes everywhere)
- A bulk is capped at 40 R2 operations (
core/src/sync/round.rs), because Workers Free allows 50 subrequests per invocation and the Worker touches R2 once per file (three times per conflict, once for all remote deletes). The engine loops rounds until nothing is deferred; the Worker refuses more than 45 with 413. A key carried to the next round keeps its record from before the sync — recording the server's version makes the old copy still on disk look like a local edit, and the next round uploads it over the newer remote one.kind: "stalled"means a round sent nothing while work remained - Writes use
expected_etagcompare-and-swap; losing races retry - One sync per data directory:
engine::runtakes an exclusive lock on<base>/.sync.lockat its entry (core/src/sync/lock.rs) and fails withkind: "busy"if another process holds it. The app'sAtomicBoolonly drives the "syncing" indicator; the file lock is the authority. Both the app andmagical-merchant syncstart syncs, so the loser really does getbusy— the CLI says so and exits 1, the app stays quiet - Repair runs under that lock, never outside it:
repair_treeinengine.rs(thedata/timeline/→data/scrawl/move, malformed notes, legacy conflict copies, duplicate IDs) fires right after the lock is taken and before the first scan, and the duplicate-ID pass runs again before a successful sync returns. A caller cannot take the lock on the engine's behalf — it is a flock on the engine's own descriptor, so the engine would then answerbusyto itself data/timeline/is the pre-rename name ofdata/scrawl/.migrate_scrawl_dirmoves it, and every entry that can write a day file calls it before reading: the sync lock, app start, CLI start, the widget's JNI. The remote sees the move asUploadNewon the new keys plusDeleteRemoteon the old ones, so one sync finishes it — but a device still on an older build then loses itsdata/timeline/to that delete and shows an empty Scrawl until it is updated. Same-named days are left in the old directory rather than merged; day files grow by appending- Conflicts keep the loser as
….sync-conflict-<ts>.mdin R2, and on disk asconflicts/<key minus extension>/<ts>.md— outsidedata/, same shape ashistory/, so it neither syncs back nor lands in the notes list. The name is built and read back incore/src/sync/conflict.rsonly.<ts>is precise to the second, so two copies of one key can want the same name: a relocation never overwrites, the second takes<ts>-2.md(rename_without_clobberincore/src/utils/fs.rs) - Auto sync runs a few seconds after any successful write
- Sync on start (
sync_on_start) runs one round at start-up, after the event listeners are in place, and on each return to the foreground at most once a minute (resumeinlib/sync.ts, called fromAppLayout's return hook) data/codex/syncs like everything else underdata/: the Codex file and itscodex/<stem>/*.mdversions are ordinary keys. A build that predates Codex simply never lists that directory. The sync engine runsrelocate_duplicate_idsunder the lock (the app also runs it once at startup, for the note list): promotion on one device plus an offline edit on another can land the same ID in bothnotes/andcodex/; the Codex wins and thenotes/copy goes toconflicts/notes/<stem>/<ts>.mdhistory/(local, machine-taken before an overwrite) anddata/codex/<stem>/(synced, committed by a person) are different things; never move one into the other. A note's filename never changes, but its directory does move once,notes/→codex/, on promotion- JWT: macOS Keychain; Android falls back to app-private file (mode 600) — keyring's in-memory fallback silently loses tokens
- TLS: desktop verifies through the OS trust store (rustls-platform-verifier);
Android uses
webpki-rootsfor the sync client because the platform verifier reports OCSP-less certificates as Revoked (android_tls::sync_tls_config,tauri-app/android-tls/README.md)
Widgets & deep links
- Scrawl capture widget appends via JNI directly into core — the app never starts. Core changes must stay callable from JNI
- "New note" / recent-notes / templates widgets open
magical-merchant://widget/…deep links; handled in AppLayout (onOpenUrlandgetCurrentfor cold start); note rows navigate with?file=<filename>, template rows with?name=<stem> - Widget sources live in
tauri-app/android-widget/, injected byjust tauri_app::android-setup(apply-widget.goregisters the five receivers in the manifest) - Template widgets show
todayTitlefrom core'stemplates_today(same helpers ascreate_note_from_template); Kotlin never resolves a variable. A template button remembers only the template's name perappWidgetId. The app redraws template widgets after writing a template or a note made from one (widget_updates.rs), and can pin a button (pin_template_widget)
