diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..8a25e60 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,74 @@ +# calcalist + +Aggregates events between CalDAV, Google Calendar and iCal feeds. `README.md` is +the user-facing guide; the design rationale lives in the module doc comments, +beside the code it governs. + +These conventions were `SPECS.md`'s, which has been retired now the roadmap is +finished. Git history holds it, and `TODO.md`, if the record is ever wanted. + +## Development + +Use devbox for dependency management and scripts. Use jj for version control +(colocated with git). + +- `devbox run fmt` — format. +- `devbox run check` — format check, clippy with warnings denied, and tests. + This is the gate; jj has no commit step to hang a hook off, since the working + copy is itself a commit. +- `devbox run test` — tests only. + +Keep `/target` in `.gitignore`: jj snapshots the working copy on every command, +with no staging step. + +The integration tests in `tests/` need `radicale` and `pimsync`, both of which +devbox provides. Outside that shell they report what is missing and pass. + +## Coding style + +- Simple, readable, idiomatic. +- Explicit types for public APIs and important fields. +- Prefer immutable data structures. +- No global mutable state. +- No speculative features or dependencies. +- Keep functions short and single-purpose; extract helpers to avoid nesting. +- File and module naming: `snake_case.rs`. Types and traits: `PascalCase`. + Functions, variables and fields: `snake_case`. Constants: + `SCREAMING_SNAKE_CASE`. +- Run `devbox run check` before describing a change. +- Comment only when intent is not obvious from the code. +- Prefer composition and traits over deep type hierarchies. + +## Decisions already taken + +Settled after real design work. Reopen only with new information, not from +first principles. + +**No configuration web interface.** Designed in full, then dropped. Its audience +would be people who find TOML hard — but every user must first create a Google +Cloud project, configure a consent screen and put a secret in a keyring, which +is a far higher bar than editing the config. Anyone who clears it can edit the +file; anyone who cannot never reaches the file. `doctor`, `status` and +`discover` cover what the interface was actually for. + +**Bring-your-own OAuth client, not a shipped verified one.** Verification for +the Calendar scope needs a domain you own — a `github.io` address is rejected — +and Calendar API quota is per Cloud project, with charges announced for +exceeding it. A shipped client would pool every user's syncing into the +publisher's project, and their bill. Shipping an *unverified* shared client is +worse still: the 100-new-user cap is permanent for the project's lifetime. + +**pimsync cannot reach Google, and never will.** It has no REST storage, and its +config exposes only HTTP Basic auth, which Google's CalDAV endpoint rejected in +March 2025. The Google leg is calcalist's own in every design. + +**The config file is never machine-rewritten.** It is the portable, +hand-editable artefact. Anything that writes it must preserve comments, +ordering and omitted defaults — `toml::to_string` on the `Serialize` derive +does none of those things. + +## Known gap + +A failing `pimsync sync` stops the whole CalDAV leg. Unlike the Google side this +cannot be narrowed: pimsync is one process covering every pair, and a failure +does not say which pair it belongs to. diff --git a/README.md b/README.md index 5e191c8..6947530 100644 --- a/README.md +++ b/README.md @@ -9,20 +9,7 @@ back to the calendar it came from, so the aggregate is somewhere to work rather than a read-only noticeboard. CalDAV and Google are read/write in both directions; iCal feeds are read-only, because the protocol is. -This is pre-1.0. The command line and the configuration format are settled; -there is no web interface yet. `SPECS.md` covers why it is built this way. - -## How it works - -Every endpoint — the sources and the aggregate's target alike — is mirrored to a -directory of `.ics` files on your machine, and the aggregation engine works -purely on those files. One cycle is: pull every remote into its local mirror, -reconcile the mirrors against each other, push the results back out. - -`pimsync` carries the CalDAV and iCal legs. calcalist carries the Google leg -itself, over the Calendar REST API — pimsync cannot reach Google at all, having -no REST storage and only HTTP Basic auth, which Google's CalDAV endpoint has -rejected since March 2025. +Pre-1.0. The command line and the configuration format are settled. ## Install @@ -31,31 +18,24 @@ cargo build --release # target/release/calcalist ``` -One runtime dependency: **`pimsync` 0.5.x on `PATH`**, needed only if any -endpoint is CalDAV or an iCal feed. A Google-only setup needs nothing else. -pimsync is in nixpkgs; this repo's `devbox.json` pins 0.5.11 if you would rather -not install it yourself. +One runtime dependency: **`pimsync` 0.5.x on `PATH`**, which carries the CalDAV +and iCal legs. It is needed only if some endpoint is one of those — a +Google-only setup needs nothing else, because calcalist talks to Google itself. +pimsync is in nixpkgs, and this repo's `devbox.json` pins 0.5.11. -Then check the environment: +Then check the environment, which is worth doing before writing any config: ```sh calcalist doctor ``` -It reports whether pimsync is present and in the supported version series, -whether the state directory is writable, whether your configuration is valid, -whether pimsync accepts the configuration calcalist generates for it, and -whether each Google endpoint still has a usable authorisation. - -## Configuration +## Configure One TOML file at `$XDG_CONFIG_HOME/calcalist/calcalist.toml`, or wherever -`--config` points. - -It holds **no secrets and no sync state**, so it is safe to copy between -machines. Credentials come from `*_command` fields instead: each is run through -`sh -c` and the first line of its output is used. Point them at `secret-tool`, -`pass`, `gpg -d`, or anything else that prints the secret. +`--config` points. It holds **no secrets and no sync state**, so it is safe to +copy between machines. Credentials come from `*_command` fields: each is run +through `sh -c` and the first line of its output is used. Point them at +`secret-tool`, `pass`, `gpg -d`, or anything else that prints the secret. ```toml version = 1 @@ -76,9 +56,15 @@ username = "you@posteo.de" secret_command = "secret-tool lookup service posteo user you@posteo.de" ``` -`url` must name the calendar collection itself, not the server root or your -principal — the last path segment is the calendar. A URL with no collection in -it is rejected, and `calcalist doctor` says so. +`url` must name the calendar collection itself, not the server root. Rather than +digging it out of a provider's web interface, ask the server — this prints a +ready-to-paste block per calendar, with ids taken from the URL for you to +rename: + +```sh +calcalist discover https://posteo.de:8443/calendars/you/ \ + --username you@posteo.de --secret-command "secret-tool lookup service posteo" +``` **Google** @@ -93,8 +79,8 @@ client_secret_command = "secret-tool lookup service google-calcalist" ``` `calendar_id` is `primary`, or the calendar's own address — the "Calendar ID" -under the calendar's settings in Google Calendar. See -[Authorising Google](#authorising-google) below for `client_id` and the secret. +under its settings in Google Calendar. See [Authorising +Google](#authorising-google) for `client_id` and the secret. **iCal feed** @@ -105,14 +91,9 @@ type = "webcal" url = "https://example.org/holidays.ics" ``` -Always read-only. Give exactly one of `url` or `url_command` — the latter for a -feed whose address is itself a credential, Google's "secret address in iCal -format" being the case in point, since anyone holding it can read the whole -calendar: - -```toml -url_command = "secret-tool lookup service google-secret-ics" -``` +Always read-only. Give exactly one of `url` or `url_command`; the latter is for +a feed whose address is itself a credential, such as Google's "secret address in +iCal format", which grants read access to anyone holding it. ### Aggregates @@ -132,11 +113,6 @@ propagate_deletes = true # deleting a mirror deletes the original max_delete_fraction = 0.2 # abort if more of the aggregate than this vanishes ``` -`max_delete_fraction` guards against a cycle that would remove a large part of -your calendars — a mistyped URL, a server answering with an empty collection. It -has an absolute floor of three deletions, so removing one or two events is never -refused. `calcalist sync --force` overrides it. - The rules, all checked before anything is synchronised, with every problem reported in one pass: @@ -144,8 +120,8 @@ reported in one pass: - `target` must not also be one of its own `sources`. - `sources` must be non-empty, and each must exist and be named once. - `default_sink`, if given, must be one of the `sources` and must be writable. - Leaving it out is a deliberate mode, not an omission — see - [Events of the aggregate's own](#events-of-the-aggregates-own). + Leaving it out is a deliberate mode — see [Getting events to the right + calendar](#getting-events-to-the-right-calendar). - No endpoint may be called `local`, which is reserved for the routing marker. ### A complete example @@ -218,25 +194,21 @@ setup, so in full: **One login covers every endpoint on that account.** The authorisation is filed under the account it was granted for, not the endpoint that asked, so a second Google calendar on the same account needs no login of its own. Set `account` on -an endpoint only when calcalist is logged in to more than one account and has to -be told which is meant. +an endpoint only when calcalist is logged in to more than one account. **While the consent screen is in Testing, Google expires refresh tokens after seven days.** An installation that worked last week then stops with nothing -having changed locally; `calcalist google login` again, or publish the consent -screen. `calcalist doctor` reports this rather than letting it surface as an -opaque failure. +having changed locally. Publish the consent screen to production to end this; +`calcalist doctor` reports it either way. -## Routing: getting a new event to the right calendar +## Getting events to the right calendar -Every mirrored event carries where it came from — `X-CALCALIST-SOURCE` and -`X-CALCALIST-ORIGIN-UID` — and a UID derived from its origin, so an edit made in -the aggregate goes back to the calendar that owns the event without any -guesswork. +An event mirrored from a source goes back to that source when you edit it — +calcalist tracks where each one came from. An event you *create* in the +aggregate belongs to nothing yet, and goes to the aggregate's `default_sink`. -An event you *create* in the aggregate belongs to nothing yet. It goes to the -aggregate's `default_sink`. To send it somewhere else, put the endpoint's id -after an `@` **on a line of its own** in the event's notes: +To send one somewhere else, put the endpoint's id after an `@` **on a line of +its own** in the event's notes: ``` Dentist, bring the referral @@ -244,140 +216,47 @@ Dentist, bring the referral @gcal ``` -The marker is removed before the event reaches the calendar. A whole line is -required so that an address or a handle written in prose is not mistaken for an -instruction. +The marker is removed before the event arrives. `calcalist status` prints the +markers that would actually work, so you need not remember what is in the +config; a marker naming anything else is refused and reported rather than +quietly filed under the default. -`calcalist status` prints the markers that would actually work, so you do not -have to remember what is in the configuration file: +**To keep an event in the aggregate itself**, tag it `@local` — it then stays +put even though a `default_sink` would have taken it, and the marker is +deliberately left in place so later cycles decide the same way. Leaving +`default_sink` out of the aggregate entirely makes that the rule for every +untagged event, so routing becomes opt-in. Such events live only in that +calendar and cannot be rebuilt from a source, but they are ordinary events on +that server, visible to every client subscribed to it. -``` -aggregate `unified` - published to posteo - sources gcal, holidays - mirroring 42 event(s) - new events go to `gcal` unless told otherwise - to choose, put one of these on a line of its own in the event's notes: - @gcal - @local (keep the event here, in this calendar only) +A category matching an endpoint id works too. Note that *any* category on a new +event is read as a routing instruction, so the notes marker is the safer form. + +## What it does to your calendars + +- **Deleting an event in the aggregate deletes it at its source**, unless + `propagate_deletes = false`. A cycle that would remove more than + `max_delete_fraction` of the aggregate is refused until you pass `--force` — + with a floor of three, so removing one or two events is never blocked. +- **Edited in both places since the last sync, the source wins.** The conflict + is reported. +- **Alarms are always kept**, with no option to strip them. The tool exists so + you subscribe to one calendar rather than several, so dropping reminders would + disarm every one you have. +- **Mirroring never mails your guests.** On a Google target attendees are kept + and Google is told not to notify; on a CalDAV target, which cannot be told + that, the guest list is carried as inert data instead. Routing an event *you* + created is the exception: that server invites its guests for real, which is + the point of creating it. + +## Run it + +```sh +calcalist doctor # environment and configuration +calcalist sync # one cycle ``` -Read-only feeds are left out, since they cannot take an event. A marker naming -anything else is refused and reported — the event stays where it is rather than -being quietly filed under the default, which would put it somewhere you did not -ask for. - -### Events of the aggregate's own - -An aggregate is also a calendar, and it can hold events that belong to nothing -else. There are two ways to get one: - -- **`@local`** on an event, exactly like any other marker, keeps it where it was - written even though a `default_sink` would otherwise have taken it. Unlike - every other marker it is *not* removed from the event: the others have done - their job once the event reaches its source, whereas this one has to stay - legible so every later cycle reaches the same decision. -- **Leave `default_sink` out of the aggregate entirely.** Then nothing is - configured to file a new event under, so every untagged event stays put and - only `@`-tagged ones are sent anywhere. Routing becomes opt-in rather than - opt-out. - -`local` is a reserved endpoint id for this reason, and configuring an endpoint -with that name is refused. - -One thing to be aware of: every other event in an aggregate is derived from a -source, so the aggregate is disposable — lose the calendar and it rebuilds -itself on the next sync. **An event of the aggregate's own exists in exactly one -place.** Nothing else holds a copy, so it is only as safe as that calendar is. -`aggregate retarget` carries these events to the new target along with -everything else, since there is nothing to re-derive them from. - -A **category** matching an endpoint id works too, that being the field -iCalendar intends for this. Be aware that any category on a newly created event -is read as a routing instruction, so an event carrying an unrelated category -will be refused rather than sent to the default sink. The notes marker is the -safer form, and the one every mobile client exposes. - -### Attendees and alarms - -Writing to an aggregate must never mail anyone: the guests were already invited -from the original calendar. On a Google target, attendees are kept verbatim and -Google is told not to notify. A CalDAV server offers no portable way to be told -that, so the guest list is instead carried as inert data — the live `ATTENDEE` -and `ORGANIZER` properties are dropped, the guests appear in -`X-CALCALIST-ATTENDEES` and in the description, and a meeting you declined is -marked free. - -Routing works the other way round: an event you created and sent to a source is -a meeting you are deliberately organising, so its attendees are preserved and -that server invites them for real. - -Alarms are always kept, with no option to strip them. The whole point of the -tool is that you subscribe to one calendar rather than several, so dropping -reminders would silently disarm every one you have. - -## Commands - -``` -calcalist sync [--dry-run] [--force] -calcalist status -calcalist doctor -calcalist prune [--force] -calcalist discover --username [--secret-command ] -calcalist google login -calcalist aggregate retarget --to [--keep-old | --purge-old] -``` - -- **`sync`** runs one cycle. `--dry-run` pulls from the servers for real, into a - throwaway copy of the local mirrors, so what it reports is measured against - your calendars as they are now — and nothing outside that copy is written. - `--force` overrides the mass-deletion guard. -- **`status`** lists endpoints and aggregates, how many events each aggregate is - mirroring, and the routing markers that would work. -- **`doctor`** checks the environment and configuration. Run it first. -- **`discover`** asks a CalDAV server which calendars it has and prints an - `[[endpoint]]` block for each, ready to paste. Give it the account or - principal URL rather than a single calendar. This is the easy way to get the - `url` field right. -- **`prune`** reports local mirrors belonging to endpoints your configuration no - longer names. It only lists them until you pass `--force`, since an endpoint - may just have been renamed. -- **`aggregate retarget`** moves an aggregate to a different target calendar - deliberately. `sync` refuses to run when an aggregate's `target` has changed - under it and points here instead — the new, empty target would otherwise read - as an aggregate whose every event had been deleted, and with delete - propagation on, that would remove them from every source calendar. Events left - on the old target are kept unless you pass `--purge-old`, which only removes - events calcalist put there. - -Exit codes: `0` all well, `1` something failed or an endpoint could not be -reached. - -If a Google endpoint cannot be reached — a lapsed token, no network — the cycle -does not stop. The endpoint is named, only the aggregates that depend on it -stand down, everything else is synchronised as usual, and the run exits `1` so -the failure cannot pass unnoticed. - -## Where things are kept - -Everything machine-local lives under `$XDG_STATE_HOME/calcalist`: - -``` -state.json event mappings, content hashes, sync cursors -pimsync.conf generated on every sync; edits are overwritten -pimsync-status/ pimsync's own record of what it has seen -vdir// the local mirror of each endpoint -google/accounts/.json refresh tokens, 0600 -``` - -None of this belongs in a backup or a dotfiles repository. The state file is a -cache rather than a single point of failure — aggregate UIDs are derived from -their origin, so a lost state file is rebuilt by re-deriving it. Copy the -configuration between machines; leave this behind. - -## Running it regularly - -Units ship in [systemd/](systemd/): +To run it regularly, units ship in [systemd/](systemd/): ```sh install -Dm644 systemd/calcalist.service ~/.config/systemd/user/calcalist.service @@ -387,27 +266,50 @@ systemctl --user enable --now calcalist.timer ``` The service expects the binary at `~/.local/bin/calcalist`; edit `ExecStart` if -yours is elsewhere. The timer runs every 15 minutes with a randomised delay of -up to two minutes, which spreads requests instead of every installation calling -Google on the quarter hour — its quota is enforced per minute, per project. +yours is elsewhere. Only one cycle runs at a time — a timer firing while you are +running `sync` by hand is refused, naming the process that holds the lock. A +cycle that could not reach an endpoint exits non-zero on purpose, so a lapsed +token shows up as a failed unit rather than passing unnoticed; +`journalctl --user -u calcalist` has the report. If your secrets come from a +keyring needing an unlocked session, the timer only works while you are logged +in. -A cycle that could not reach an endpoint exits non-zero deliberately, so a -lapsed token surfaces as a failed unit in `systemctl --user status calcalist` -rather than passing unnoticed. `journalctl --user -u calcalist` has the report. +## Commands -Only one cycle runs at a time: calcalist takes a lock on its state directory, so -a timer firing while you are running `calcalist sync` by hand is refused with a -message naming the process that holds it, rather than the two interleaving their -writes. If your secrets come from a keyring that needs an unlocked session, the -timer will only work while you are logged in. +`--help` has the flags. -## Not yet +| Command | | +|---|---| +| `sync` | One cycle. `--dry-run` reports against current remote state without writing; `--force` overrides the mass-deletion guard. | +| `status` | Endpoints, aggregates, how much is mirrored, and the routing markers that work. | +| `doctor` | Checks the environment and configuration. Run it first. | +| `discover` | Lists a CalDAV server's calendars as endpoint blocks to paste. | +| `google login` | Authorises an account. | +| `aggregate retarget` | Moves an aggregate to a different target calendar. `sync` refuses on its own when the target changes under it, and points here. | +| `prune` | Reports local mirrors of endpoints no longer configured; removes them under `--force`. | + +Exit codes: `0` all well, `1` something failed or an endpoint could not be +reached. + +## Where things are kept + +Everything machine-local lives under `$XDG_STATE_HOME/calcalist`: + +``` +state.json event mappings, content hashes, sync cursors +lock held for the duration of a cycle +pimsync.conf generated on every sync; edits are overwritten +pimsync-status/ pimsync's own record of what it has seen +vdir// the local mirror of each endpoint +google/accounts/.json refresh tokens, 0600 +``` + +None of it belongs in a backup or a dotfiles repository — it is a cache, and a +lost state file costs a re-materialisation rather than data. Copy the +configuration between machines; leave this behind. + +## Known gaps - **A failing `pimsync sync` stops the whole CalDAV leg.** Google endpoints are isolated from each other, but pimsync is a single process covering every CalDAV and feed pair, and a failure does not say which pair it belongs to. - -There is deliberately no web interface. It was designed and then dropped: the -Google OAuth setup, which no interface can remove, is a far higher bar than -editing this file, so a UI would have served an audience that never gets as far -as the config. `doctor`, `status` and `discover` cover what it was for. diff --git a/SPECS.md b/SPECS.md deleted file mode 100644 index a00697b..0000000 --- a/SPECS.md +++ /dev/null @@ -1,62 +0,0 @@ -# Calcalist - -## Idea - -Small Rust based commandline tool with utility systemd service file that can aggregate and sync events between CalDAV/Google Calendar/iCal. Users who have a Google Calendar can create a new calendar that syncs events from several CalDAV calendars. CalDAV users can sync several Google Calendars into one calendar. This should be doable both ways, so the aggregating calendar needs to be able to distinguish where its events came from and also have a way to create new events that get synced to the correct source. In addition iCal feeds can also be aggregated but they remain one way only. - -Configuration is saved in a portable toml file that can be easily migrated. - -The web interface this originally called for was designed and then dropped. Setting up Google's OAuth client — a Cloud project, a consent screen, a secret in a keyring — is a far higher bar than editing thirty lines of TOML, and no interface can remove it, so a configuration UI would have served an audience that never reaches the configuration. `calcalist doctor` reports every problem in one pass, `calcalist status` names the routing markers an event can carry, and `calcalist discover` prints ready-to-paste endpoint blocks for a server's calendars — which is what the interface was actually for. - -## Architecture - -Every endpoint — sources and aggregate targets alike — is mirrored to a local vdir, so the aggregation engine works purely on local files. - -- `pimsync` is driven as a one-shot subprocess for the CalDAV and WebCal legs. Its daemon mode is unusable here: it would race the reconciler over the same vdir files, so calcalist owns scheduling. -- The Google leg is calcalist's own, via the Calendar REST API. pimsync cannot reach Google — it has no REST storage, and its config exposes only HTTP Basic auth, which Google's CalDAV endpoint has rejected since 2025-03-14. -- Sync state (event mappings, hashes, sync tokens) lives in a JSON sidecar under `$XDG_STATE_HOME/calcalist/`, never in the portable config. -- Secrets are never stored in the config either. Credentials come from `*_command` fields that are executed to fetch them. -- A Google authorisation is filed under the account it was granted for, not the endpoint that asked for it, so every endpoint on that account shares it. An endpoint names its `account` only when calcalist is logged in to more than one. - -## Sync semantics - -| Question | Behaviour | -|---|---| -| Provenance | Aggregate UIDs derived as `blake3(aggregate_id, source_id, source_uid)`; the state file is a cache, not a single point of failure | -| Routing new events | A `@endpoint-id` line in the description, or a matching category, picks the source; otherwise the aggregate's `default_sink`. A hint naming an invalid sink is refused, never redirected to the default | -| Conflicts | Source wins — the origin calendar is authoritative | -| Deletion | Propagates to the source, guarded by a mass-deletion threshold | -| Attendees (mirroring) | Kept verbatim on a Google target; demoted to inert data on a CalDAV target | -| Attendees (routing) | Preserved — the sink server sends real invitations, which is intended | -| Alarms | Always preserved, with no option to strip them | -| Retargeting | `sync` refuses on target drift; `aggregate retarget` performs it deliberately | -| Recurrence | A series and its exceptions share one file. Google addresses an exception through the series, so the master is sent first and each override is matched to its instance by original start time and patched | -| Unreachable endpoints | Named in the report; only the aggregates depending on them stand down, and the run exits non-zero | -| Dry runs | Pull for real, into a throwaway copy of the local mirrors and through a read-only pimsync configuration. Nothing outside the copy is written | - -The governing rule behind the attendee handling: **writes to an aggregate must never emit scheduling mail; writes to a source schedule normally.** Google can be told not to notify, so attendees survive intact there. CalDAV offers no portable way to suppress RFC 6638 scheduling, so inertness is achieved structurally by dropping the live properties instead. - -Alarms are never stripped because the whole point of the tool is that a user subscribes to one calendar rather than several — so the duplicate-notification problem that stripping would guard against does not arise, while stripping would silently destroy every reminder in the one calendar the user actually watches. - -## Development - -Use devbox for dependency management and scripts. Use jj for version control (colocated with git). - -- `devbox run fmt` — format. -- `devbox run check` — format check, clippy with warnings denied, and tests. This is the gate; jj has no commit step to hang a hook off, since the working copy is itself a commit. -- `devbox run test` — tests only. - -Keep `/target` in `.gitignore`: jj snapshots the working copy on every command, with no staging step. - -## Coding style - -- Simple, readable, idiomatic. -- Explicit types for public APIs and important fields. -- Prefer immutable data structures. -- No global mutable state. -- No speculative features or dependencies. -- Keep functions short and single-purpose; extract helpers to avoid nesting. -- File and module naming: `snake_case.rs`. Types and traits: `PascalCase`. Functions, variables and fields: `snake_case`. Constants: `SCREAMING_SNAKE_CASE`. -- Run `devbox run check` before describing a change. -- Comment only when intent is not obvious from the code. -- Prefer composition and traits over deep type hierarchies. diff --git a/TODO.md b/TODO.md deleted file mode 100644 index b1240a5..0000000 --- a/TODO.md +++ /dev/null @@ -1,175 +0,0 @@ -# Roadmap - -Milestones from the implementation plan. See SPECS.md for the architecture and the -sync semantics these items implement. - -## M0 — skeleton (done) - -- [x] Initialise jj colocated with git; `.gitignore` written before the first build -- [x] `devbox.json` pinning Rust 1.97.1, pimsync 0.5.11, jujutsu 0.44.0, radicale 3.7.8 -- [x] Configuration model with referential validation, reporting every problem in one pass -- [x] `calcalist doctor` — pimsync presence and version series, state directory, configuration -- [x] Define the full CLI surface; unimplemented commands exit 2 rather than pretend -- [x] SPECS.md: Rust naming conventions, `devbox run check` gate, recorded sync semantics - -## M1 — bidirectional sync - -Complete. Every item from the plan is built, tested and verified live. - -### Core modules - -- [x] `state.rs` — JSON sidecar, atomic temp + fsync + rename; records each aggregate's - resolved target endpoint id **and** backend type, plus per-Google-endpoint cursors -- [x] `vdir.rs` — read and write vdir directories, keyed by the UID inside each file - rather than by filename, since pimsync names files its own way -- [x] `ical.rs` — surgical line-level `.ics` editing (UID rewrite, property injection), - respecting RFC 5545 folding; no parse-and-reserialize -- [x] `provenance.rs` — deterministic `blake3(aggregate_id, source_id, source_uid)` UIDs -- [x] `mirror.rs` — the to-aggregate and to-source transforms (added; not in the - original plan, which folded these into `reconcile`) -- [x] `reconcile.rs` — the aggregation engine; pure, no I/O -- [x] `sync.rs` — one cycle: pull, reconcile, push -- [x] `retarget.rs` — moving an aggregate to a different target, deliberately -- [x] `pimsync.rs` — generate `pimsync.conf` (with `on_empty skip` and `on_delete skip`), - drive one-shot `pimsync sync` bracketing the reconcile step -- [x] `google/auth.rs` — OAuth loopback flow with PKCE, refresh, keyring-sourced secrets -- [x] `google/convert.rs` — iCalendar both ways, including recurrence, timezones and alarms -- [x] `google/api.rs` — incremental pull by syncToken; push by import / update / delete - with notification suppressed - -### Behaviour - -- [x] **`events.import` gate** — settled from Google's own API discovery document: - `events.import` accepts no `sendUpdates` parameter at all, while `insert`, - `update` and `delete` all do, and it is documented as adding "a private copy of - an existing event". Confirmed live that attendees and alarms survive an import. - Creation goes through `import`; update and delete pass `sendUpdates=none`. -- [x] `sync` refuses to run on aggregate target drift, before reconciliation -- [x] `aggregate retarget` — flush unrouted creations against the old target, then - re-materialise; keep old orphans by default, purge bounded by the derivation -- [x] Mass-deletion guard (`max_delete_fraction`), overridable with `--force`, with an - absolute floor so deleting a couple of events is never refused -- [x] Echo suppression: derived UIDs are never re-ingested as source events -- [x] Routing: a `@endpoint-id` line in the description, or a matching category, picks - which source a new aggregate event is filed under; a hint naming an invalid sink - is refused rather than redirected to the default -- [x] Alarms map both ways, including Google's minutes-before reminder model -- [x] `webcal` URLs may come from a command, for feeds whose address is itself a - credential — Google's secret iCal address being the case in point -- [x] `doctor` validates the generated pimsync config and each Google authorisation -- [x] `calcalist status` — endpoints, aggregates, how many events are mirrored, and the - exact `@marker` names an event can carry, so routing is discoverable without - opening the config file -- [x] Recurrence overrides are pushed to Google. Google addresses an exception - through the series rather than as an event of its own, so the master goes up - first and each override is then matched to its instance by original start - time and patched. An override matching no occurrence is counted, not forced. -- [x] A `TZID` now travels with the `VTIMEZONE` that defines it, derived from the - zone's own transition table as the yearly rule it implies. A zone that does - not shift gets one fixed observance. -- [x] One Google authorisation covers every endpoint on that account. Tokens are - keyed by the account, discovered at login from the primary calendar's id — - which needs no scope beyond the calendar one already granted. An endpoint - names its `account` only when calcalist is logged in to more than one. -- [x] A Google endpoint that cannot be reached no longer ends the cycle. It is - named, only the aggregates depending on it stand down, and the run exits - non-zero so a lapsed token cannot pass for success. -- [x] `calcalist prune` — reports local mirrors of endpoints the configuration no - longer names, and removes them under `--force` -- [x] `--dry-run` pulls for real, into a throwaway copy of the local mirrors and - through a pimsync configuration that only ever reads from a server. What it - reports is measured against the calendars as they are now, and nothing - outside the copy is written. - -### Tests - -- [x] `reconcile` table-driven cases: create/update/delete each direction, - both-sides-changed, routing, echo suppression, mass-delete abort -- [x] `ical` round-trip fixtures: recurring with overrides, all-day, TZID, unknown `X-` props -- [x] `convert` against real captured API payloads, including the case where Google's - offset and its `timeZone` disagree -- [x] Topologies: a source feeding several aggregates, several aggregates sharing one - target, a cycle between two aggregates, a delete cascading across aggregates, - and competing edits arriving through two aggregates at once -- [x] Retarget: drift makes `sync` exit non-zero having written nothing and losing no - source event; an unrouted creation reaches a sink first -- [x] Safety (unit): no live `ATTENDEE`/`ORGANIZER` on a CalDAV-targeted mirror, - `VALARM` intact, `PARTSTAT: DECLINED` maps to `TRANSP: TRANSPARENT` - -### Verified live, end to end - -Against real accounts — two Google calendars aggregating into a Posteo CalDAV -calendar: - -- Fan-in from both Google sources, each mirror tagged with its origin -- A public iCal feed synced through pimsync and pushed to CalDAV; idempotent across - repeated runs and correctly incremental -- An edit made in the aggregate reaching the originating Google calendar, and the - revert travelling the same way -- An event created in the aggregate routed to a chosen source by its description - marker, with the marker stripped before it arrived -- A deletion in the aggregate propagating through to Google -- The mass-deletion guard refusing a 100% removal until `--force` -- `pimsync check` validating the generated config against a live CalDAV server - -### Integration - -- [x] Against a real Radicale server and a real iCal feed, over pimsync: two - sources converge on a CalDAV target, and the next cycle is a no-op. -- [x] Safety, against the same server: no live `ATTENDEE` or `ORGANIZER` reaches - the aggregate, the guest list survives as inert data, the alarm survives - intact, and the source keeps its scheduling properties. - - The plan asked for an SMTP sink here. Radicale implements no RFC 6638 - scheduling, so a quiet SMTP port would have proved nothing about the - transform — no server in reach of a test sends calendar mail at all. What - is asserted instead is the bytes that reached the server, which is the - thing the transform is actually responsible for. Proving the Google half - was settled separately, from the API discovery document. -- [x] `--dry-run` reaches the servers, reports what it found, and leaves both the - target and the local mirrors untouched. -- [x] A retired endpoint's mirror is reported by `prune` and removed under - `--force`. - -## M2 — packaging (done) - -- [x] Run lock (`lock.rs`) — `flock` on the state directory, held by `sync`, - `aggregate retarget` and `prune --force`. Not a UI feature: nothing - previously stopped a timer firing into a hand-run cycle, and two cycles - interleaving writes over the same vdirs is what the design otherwise - avoids. The kernel releases it however the process ends, so a crash - cannot leave a lock to clear by hand; the refusal names the holding pid. -- [x] systemd user units in `systemd/` — `calcalist.service` (oneshot) and - `calcalist.timer`, with `RandomizedDelaySec` so installations do not all - call Google on the quarter hour, and no filesystem or IPC sandboxing - because the secret commands need the session keyring over D-Bus. -- [x] `calcalist discover` — asks a CalDAV server which calendars it has and - prints a ready-to-paste `[[endpoint]]` block for each. The `url` field is - the most error-prone thing in the config, and providers rarely show it. -- [x] `serve` removed from the CLI, and with it the last `unimplemented` - command, so every command the binary advertises now does something. - -### The web interface, dropped - -Specified from the start and designed in full before being dropped. The -reasoning, so it is not rediscovered as an oversight: - -- Its audience would be people who find TOML hard. But with bring-your-own - OAuth client settled, every user must first create a Google Cloud project, - configure a consent screen and put a secret in a keyring — a far higher bar - than editing thirty lines of config. Anyone who clears it can edit the file; - anyone who cannot never reaches the file. -- The cost was three dependencies, five modules, an `auth.rs` refactor and a - security surface (token, `Host` validation, CSRF, secrets through subprocess - stdin) guarding something that reads the config and touches the keyring — - against SPECS.md's own "no speculative features or dependencies". -- `doctor`, `status` and `discover` had already absorbed what it was for. - -The one idea worth keeping from the design was collection discovery, which -needed no web layer at all. - -## Known gaps - -- [ ] A `pimsync sync` that fails takes the whole CalDAV leg with it. Unlike the - Google side this cannot be narrowed: pimsync is one process covering every - pair, so a failure does not say which pair it belongs to.