diff --git a/TODO.md b/TODO.md index 6c47de7..8bc472a 100644 --- a/TODO.md +++ b/TODO.md @@ -14,29 +14,31 @@ sync semantics these items implement. ## M1 — bidirectional sync -Core modules: +Functionally complete. Everything below is done except the two automated +integration tests, which are listed with the remaining gaps at the end. + +### Core modules - [x] `state.rs` — JSON sidecar, atomic temp + fsync + rename; records each aggregate's - resolved target endpoint id **and** backend type -- [x] `vdir.rs` — read and write vdir directories + resolved target endpoint id **and** backend type, plus per-Google-endpoint cursors +- [x] `vdir.rs` — read and write vdir directories, keyed by the UID inside each file + rather than by filename, since pimsync names files its own way - [x] `ical.rs` — surgical line-level `.ics` editing (UID rewrite, property injection), respecting RFC 5545 folding; no parse-and-reserialize - [x] `provenance.rs` — deterministic `blake3(aggregate_id, source_id, source_uid)` UIDs - [x] `mirror.rs` — the to-aggregate and to-source transforms (added; not in the original plan, which folded these into `reconcile`) - [x] `reconcile.rs` — the aggregation engine; pure, no I/O -- [x] `sync.rs` — one cycle over the local vdirs, applying what `reconcile` decides +- [x] `sync.rs` — one cycle: pull, reconcile, push +- [x] `retarget.rs` — moving an aggregate to a different target, deliberately - [x] `pimsync.rs` — generate `pimsync.conf` (with `on_empty skip` and `on_delete skip`), drive one-shot `pimsync sync` bracketing the reconcile step -- [x] `doctor` asks `pimsync check` to validate the generated config, since pimsync's - parser does not always match its documentation - [x] `google/auth.rs` — OAuth loopback flow with PKCE, refresh, keyring-sourced secrets -- [x] `google/convert.rs` — JSON to iCalendar, including recurrence and timezones -- [x] `google/api.rs` — incremental pull by syncToken, and push by import / update - / delete with notification suppressed -- [x] Reintroduce `SchedulingSuppression` in `config.rs` (removed in M0 as dead code) +- [x] `google/convert.rs` — iCalendar both ways, including recurrence, timezones and alarms +- [x] `google/api.rs` — incremental pull by syncToken; push by import / update / delete + with notification suppressed -Safety-critical behaviour: +### Behaviour - [x] **`events.import` gate** — settled from Google's own API discovery document: `events.import` accepts no `sendUpdates` parameter at all, while `insert`, @@ -45,46 +47,80 @@ Safety-critical behaviour: Creation goes through `import`; update and delete pass `sendUpdates=none`. - [x] `sync` refuses to run on aggregate target drift, before reconciliation - [x] `aggregate retarget` — flush unrouted creations against the old target, then - re-materialise; keep old orphans by default + re-materialise; keep old orphans by default, purge bounded by the derivation - [x] Mass-deletion guard (`max_delete_fraction`), overridable with `--force`, with an absolute floor so deleting a couple of events is never refused - [x] Echo suppression: derived UIDs are never re-ingested as source events +- [x] Routing: a `@endpoint-id` line in the description, or a matching category, picks + which source a new aggregate event is filed under; a hint naming an invalid sink + is refused rather than redirected to the default +- [x] Alarms map both ways, including Google's minutes-before reminder model +- [x] `webcal` URLs may come from a command, for feeds whose address is itself a + credential — Google's secret iCal address being the case in point +- [x] `doctor` validates the generated pimsync config and each Google authorisation -Tests: +### Tests -- [x] `reconcile` table-driven cases: create/update/delete each direction, both-sides-changed, - routing, echo suppression, mass-delete abort +- [x] `reconcile` table-driven cases: create/update/delete each direction, + both-sides-changed, routing, echo suppression, mass-delete abort - [x] `ical` round-trip fixtures: recurring with overrides, all-day, TZID, unknown `X-` props -- [ ] Integration against Radicale plus a `file://` WebCal fixture; assert idempotence -- [x] Safety (unit level): no live `ATTENDEE`/`ORGANIZER` on a CalDAV-targeted mirror, - `VALARM` intact, `PARTSTAT: DECLINED` maps to `TRANSP: TRANSPARENT`, bulk deletion aborts -- [ ] Safety (integration): the same against a real Radicale instance with an SMTP sink, - proving no mail is emitted -- [x] Retarget: drift makes `sync` exit non-zero having written nothing and losing no source - event; purge is bounded by the derivation; an unrouted creation reaches a sink first +- [x] `convert` against real captured API payloads, including the case where Google's + offset and its `timeZone` disagree +- [x] Topologies: a source feeding several aggregates, several aggregates sharing one + target, a cycle between two aggregates, a delete cascading across aggregates, + and competing edits arriving through two aggregates at once +- [x] Retarget: drift makes `sync` exit non-zero having written nothing and losing no + source event; an unrouted creation reaches a sink first +- [x] Safety (unit): no live `ATTENDEE`/`ORGANIZER` on a CalDAV-targeted mirror, + `VALARM` intact, `PARTSTAT: DECLINED` maps to `TRANSP: TRANSPARENT` ### Verified live, end to end -Two Google calendars aggregating into a Posteo CalDAV calendar, against real accounts: -fan-in from both sources with provenance intact; an edit in the aggregate reaching the -originating Google calendar; an event created in the aggregate routed to a chosen -source by a description marker; deletion propagating from the aggregate through to -Google; and the mass-deletion guard refusing a 100% removal until `--force`. +Against real accounts — two Google calendars aggregating into a Posteo CalDAV +calendar: -### Known gaps carried out of M1 +- Fan-in from both Google sources, each mirror tagged with its origin +- A public iCal feed synced through pimsync and pushed to CalDAV; idempotent across + repeated runs and correctly incremental +- An edit made in the aggregate reaching the originating Google calendar, and the + revert travelling the same way +- An event created in the aggregate routed to a chosen source by its description + marker, with the marker stripped before it arrived +- A deletion in the aggregate propagating through to Google +- The mass-deletion guard refusing a 100% removal until `--force` +- `pimsync check` validating the generated config against a live CalDAV server -- [ ] A recurring series' *exceptions* are not pushed to Google. Google models them - as separate events against an already existing series, so they need - `events.instances` plus a patch per exception. Reported per sync rather than - dropped silently. -- [x] `push` deleting an event remotely — verified live: deleting a mirror in the - CalDAV aggregate removed the origin event from Google. +## Remaining + +### Tests not yet automated + +- [ ] Integration against Radicale plus a WebCal fixture, asserting convergence and + idempotence. Done by hand twice; not yet a test that runs in CI. +- [ ] Safety integration: a real CalDAV server with an SMTP sink, proving no mail is + emitted on mirror writes or mirror deletions. + +### Known gaps + +- [ ] A recurring series' *exceptions* are not pushed to Google. Pulling them works. + Google models them as separate events against an already existing series, so + pushing needs `events.instances` plus a patch per exception. Reported per sync + rather than dropped silently. +- [ ] Each Google endpoint needs its own `google login`, even for the same account and + OAuth client, because tokens are keyed by endpoint id. Two endpoints on one + account should share a credential. - [ ] A failed Google pull aborts the whole cycle, including the CalDAV side. Safe — - reconciling against a stale snapshot could read as mass deletion — but it means - a lapsed token stops everything. + reconciling against a stale snapshot could read as mass deletion — but a lapsed + token stops everything. Skipping only the affected aggregates would be better. +- [ ] Removing an endpoint from the config leaves its vdir behind, holding events + nothing manages any more. +- [ ] `--dry-run` skips the pull entirely, so it reports against whatever the last + real cycle left behind rather than against current remote state. +- [ ] A `TZID` is emitted without an accompanying `VTIMEZONE`. Tolerated by the servers + tested so far, and confined to recurring events, but not strictly conformant. ## M2 — interface and packaging - [ ] axum configuration UI, bound to 127.0.0.1 -- [ ] OAuth loopback redirect handler +- [ ] Trigger `google login` from the UI (the loopback handler itself already exists + in `google/auth.rs`) - [ ] systemd user units: `calcalist.service` (oneshot) and `calcalist.timer`