diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..8a25e60 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,74 @@ +# calcalist + +Aggregates events between CalDAV, Google Calendar and iCal feeds. `README.md` is +the user-facing guide; the design rationale lives in the module doc comments, +beside the code it governs. + +These conventions were `SPECS.md`'s, which has been retired now the roadmap is +finished. Git history holds it, and `TODO.md`, if the record is ever wanted. + +## Development + +Use devbox for dependency management and scripts. Use jj for version control +(colocated with git). + +- `devbox run fmt` — format. +- `devbox run check` — format check, clippy with warnings denied, and tests. + This is the gate; jj has no commit step to hang a hook off, since the working + copy is itself a commit. +- `devbox run test` — tests only. + +Keep `/target` in `.gitignore`: jj snapshots the working copy on every command, +with no staging step. + +The integration tests in `tests/` need `radicale` and `pimsync`, both of which +devbox provides. Outside that shell they report what is missing and pass. + +## Coding style + +- Simple, readable, idiomatic. +- Explicit types for public APIs and important fields. +- Prefer immutable data structures. +- No global mutable state. +- No speculative features or dependencies. +- Keep functions short and single-purpose; extract helpers to avoid nesting. +- File and module naming: `snake_case.rs`. Types and traits: `PascalCase`. + Functions, variables and fields: `snake_case`. Constants: + `SCREAMING_SNAKE_CASE`. +- Run `devbox run check` before describing a change. +- Comment only when intent is not obvious from the code. +- Prefer composition and traits over deep type hierarchies. + +## Decisions already taken + +Settled after real design work. Reopen only with new information, not from +first principles. + +**No configuration web interface.** Designed in full, then dropped. Its audience +would be people who find TOML hard — but every user must first create a Google +Cloud project, configure a consent screen and put a secret in a keyring, which +is a far higher bar than editing the config. Anyone who clears it can edit the +file; anyone who cannot never reaches the file. `doctor`, `status` and +`discover` cover what the interface was actually for. + +**Bring-your-own OAuth client, not a shipped verified one.** Verification for +the Calendar scope needs a domain you own — a `github.io` address is rejected — +and Calendar API quota is per Cloud project, with charges announced for +exceeding it. A shipped client would pool every user's syncing into the +publisher's project, and their bill. Shipping an *unverified* shared client is +worse still: the 100-new-user cap is permanent for the project's lifetime. + +**pimsync cannot reach Google, and never will.** It has no REST storage, and its +config exposes only HTTP Basic auth, which Google's CalDAV endpoint rejected in +March 2025. The Google leg is calcalist's own in every design. + +**The config file is never machine-rewritten.** It is the portable, +hand-editable artefact. Anything that writes it must preserve comments, +ordering and omitted defaults — `toml::to_string` on the `Serialize` derive +does none of those things. + +## Known gap + +A failing `pimsync sync` stops the whole CalDAV leg. Unlike the Google side this +cannot be narrowed: pimsync is one process covering every pair, and a failure +does not say which pair it belongs to. diff --git a/Cargo.lock b/Cargo.lock index 1299f71..2621f6f 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -117,6 +117,7 @@ dependencies = [ "blake3", "clap", "jiff", + "rustix", "serde", "serde_json", "sha2", diff --git a/Cargo.toml b/Cargo.toml index c8cac75..ef7a6d6 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -4,12 +4,14 @@ version = "0.1.0" edition = "2024" rust-version = "1.97" description = "Aggregate and sync events between CalDAV, Google Calendar and iCal feeds" +license = "MIT" publish = false [dependencies] blake3 = "1.8" clap = { version = "4.6", features = ["derive"] } jiff = "0.2" +rustix = { version = "1.1", features = ["fs"] } serde = { version = "1.0", features = ["derive"] } serde_json = "1.0" sha2 = "0.11" diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..2e6cbfe --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 randogoth + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/README.md b/README.md new file mode 100644 index 0000000..aba2467 --- /dev/null +++ b/README.md @@ -0,0 +1,255 @@ +# CalCalist + +Aggregate several calendars into one, and keep them in step. + +CalCalist mirrors any number of CalDAV calendars, Google calendars and iCal feeds into a single calendar you subscribe to. The point is to subscribe to one calendar instead of several: an event created or edited in the aggregate travels back to the calendar it came from, so the aggregate is somewhere to work rather than a read-only noticeboard. CalDAV and Google are read/write in both directions; iCal feeds are read-only, because the protocol is. + +## Install + +```sh +cargo build --release +# target/release/calcalist +``` + +One runtime dependency is needed for CalDAV and iCal sync: **`pimsync` 0.5.x on `PATH`**. + +Then check the environment, which is worth doing before writing any config: + +```sh +calcalist doctor +``` + +## Configure + +One TOML file at `$XDG_CONFIG_HOME/calcalist/calcalist.toml`, or wherever `--config` points. It holds **no secrets and no sync state**, so it is safe to +copy between machines. + +Credentials come from `*_command` fields: each is run through `sh -c` and the first line of its output is used. Point them at `secret-tool`, `pass`, `gpg -d`, or anything else that prints the secret. + +```toml +version = 1 +``` + +Each `[[endpoint]]` is one calendar, with an `id` you refer to it by elsewhere. + +### CalDAV Endpoint + +```toml +[[endpoint]] +id = "posteo" +type = "caldav" +url = "https://posteo.de:8443/calendars/you/abc123/work/" +username = "you@posteo.de" +secret_command = "secret-tool lookup service posteo user you@posteo.de" +``` + +`url` must name the calendar collection itself, not the server root. + +Rather than digging it out of a provider's web interface, ask the server — this prints a ready-to-paste block per calendar, with ids taken from the URL for you to rename: + +```sh +calcalist discover https://posteo.de:8443/calendars/you/ \ + --username you@posteo.de --secret-command "secret-tool lookup service posteo" +``` + +## Google Calendar Endpoint + +```toml +[[endpoint]] +id = "gcal" +type = "google" +calendar_id = "you@gmail.com" +client_id = "1234-abcd.apps.googleusercontent.com" +client_secret_command = "secret-tool lookup service google-calcalist" +# account = "you@gmail.com" # only when logged in to more than one account +``` + +`calendar_id` is `primary`, or the calendar's own address — the "Calendar ID" under its settings in Google Calendar. See [Authorising Google](#authorising-google) for `client_id` and the secret. + +## iCal Feed Endpoint + +```toml +[[endpoint]] +id = "holidays" +type = "webcal" +url = "https://example.org/holidays.ics" +``` + +Always read-only. Give exactly one of `url` or `url_command`; the latter is for a feed whose address is itself a credential, such as Google's "secret address in iCal format". + +## Aggregates + +Each `[[aggregate]]` composes endpoints: several `sources` mirrored into one `target`. + +```toml +[[aggregate]] +id = "unified" +target = "posteo" +sources = ["gcal", "holidays"] +default_sink = "gcal" + +# Defaults, all optional: +conflict = "source_wins" # the only policy; the origin calendar wins +propagate_deletes = true # deleting a mirror deletes the original +max_delete_fraction = 0.2 # abort if more of the aggregate than this vanishes +``` + +The rules, all checked before anything is synchronised, with every problem reported in one pass: + +- `target` must exist and be writable, so never an iCal feed. +- `target` must not also be one of its own `sources`. +- `sources` must be non-empty, and each must exist and be named once. +- `default_sink`, if given, must be one of the `sources` and must be writable. Leaving it out is a deliberate mode — see [Getting events to the right calendar](#getting-events-to-the-right-calendar). +- No endpoint may be called `local`, which is reserved for the routing marker. + +## A complete example + +A Google calendar and a public holiday feed, aggregated into a CalDAV calendar: + +```toml +version = 1 + +[[endpoint]] +id = "gcal" +type = "google" +calendar_id = "primary" +client_id = "1234-abcd.apps.googleusercontent.com" +client_secret_command = "secret-tool lookup service google-calcalist" + +[[endpoint]] +id = "holidays" +type = "webcal" +url = "https://www.officeholidays.com/ics/germany" + +[[endpoint]] +id = "posteo" +type = "caldav" +url = "https://posteo.de:8443/calendars/you/abc123/unified/" +username = "you@posteo.de" +secret_command = "secret-tool lookup service posteo user you@posteo.de" + +[[aggregate]] +id = "unified" +target = "posteo" +sources = ["gcal", "holidays"] +default_sink = "gcal" +``` + +# Authorising Google + +Google requires an OAuth client of your own. This is the fiddliest part of the setup, so in full: + +1. At [console.cloud.google.com](https://console.cloud.google.com), create a + project — any name. +2. **APIs & Services → Library**, find **Google Calendar API**, enable it. +3. **OAuth consent screen**: choose **External**. Fill in the required fields. Under **Test users**, add your own Google address. +4. **Credentials → Create credentials → OAuth client ID**, and for **Application type** choose **Desktop app**. +5. Copy the client ID into `client_id`, and put the client secret somewhere `client_secret_command` can read it: + + ```sh + secret-tool store --label="calcalist Google client secret" service google-calcalist + ``` + +6. Authorise: + + ```sh + calcalist google login gcal + ``` + + A browser opens; approve the request. CalCalist asks only for `https://www.googleapis.com/auth/calendar`. The refresh token is written into the state directory, never into your configuration. + +**One login covers every endpoint on that account.** The authorisation is filed under the account it was granted for, not the endpoint that asked, so a second Google calendar on the same account needs no login of its own. Set `account` on an endpoint only when CalCalist is logged in to more than one account. + +**While the consent screen is in Testing, Google expires refresh tokens after seven days.** An installation that worked last week then stops with nothing +having changed locally. Publish the consent screen to production to end this; `calcalist doctor` reports it either way. + +## Getting events to the right calendar + +An event mirrored from a source goes back to that source when you edit it — CalCalist tracks where each one came from. An event you *create* in the +aggregate belongs to nothing yet, and goes to the aggregate's `default_sink`. + +To send one somewhere else, put the endpoint's id after an `@` **on a line of its own** in the event's notes: + +``` +Dentist, bring the referral + +@gcal +``` + +The marker is removed before the event arrives. `calcalist status` prints the markers that would actually work, so you need not remember what is in the +config; a marker naming anything else is refused and reported rather than quietly filed under the default. + +**To keep an event in the aggregate itself**, tag it `@local` — it then stays put even though a `default_sink` would have taken it, and the marker is +deliberately left in place so later cycles decide the same way. + +Leaving `default_sink` out of the aggregate entirely makes that the rule for every untagged event, so routing becomes opt-in. Such events live only in that calendar and cannot be rebuilt from a source, but they are ordinary events on that server, visible to every client subscribed to it. + +A category matching an endpoint id works too. Note that *any* category on a new event is read as a routing instruction, so the notes marker is the safer form. + +## What it does to your calendars + +- **Deleting an event in the aggregate deletes it at its source**, unless `propagate_deletes = false`. A cycle that would remove more than `max_delete_fraction` of the aggregate is refused until you pass `--force` — with a floor of three, so removing one or two events is never blocked. +- **Edited in both places since the last sync, the source wins.** The conflict is reported. +- **Alarms are always kept**, with no option to strip them. The tool exists so you subscribe to one calendar rather than several, so dropping reminders would disarm every one you have. +- **Mirroring never mails your guests.** On a Google target attendees are kept and Google is told not to notify; on a CalDAV target, which cannot be told + that, the guest list is carried as inert data instead. Routing an event *you* created is the exception: that server invites its guests for real, which is the point of creating it. + +## Run it + +```sh +calcalist doctor # environment and configuration +calcalist sync # one cycle +``` + +To run it regularly, units ship in [systemd/](systemd/): + +```sh +install -Dm644 systemd/calcalist.service ~/.config/systemd/user/calcalist.service +install -Dm644 systemd/calcalist.timer ~/.config/systemd/user/calcalist.timer +systemctl --user daemon-reload +systemctl --user enable --now calcalist.timer +``` + +The service expects the binary at `~/.local/bin/calcalist`; edit `ExecStart` if yours is elsewhere. Only one cycle runs at a time — a timer firing while you are running `sync` by hand is refused, naming the process that holds the lock. A cycle that could not reach an endpoint exits non-zero on purpose, so a lapsed token shows up as a failed unit rather than passing unnoticed; `journalctl --user -u calcalist` has the report. If your secrets come from a +keyring needing an unlocked session, the timer only works while you are logged in. + +## Commands + +`--help` has the flags. + +| Command | | +|---|---| +| `sync` | One cycle. `--dry-run` reports against current remote state without writing; `--force` overrides the mass-deletion guard. | +| `status` | Endpoints, aggregates, how much is mirrored, and the routing markers that work. | +| `doctor` | Checks the environment and configuration. Run it first. | +| `discover` | Lists a CalDAV server's calendars as endpoint blocks to paste. | +| `google login` | Authorises an account. | +| `aggregate retarget` | Moves an aggregate to a different target calendar. `sync` refuses on its own when the target changes under it, and points here. | +| `prune` | Reports local mirrors of endpoints no longer configured; removes them under `--force`. | + +Exit codes: `0` all well, `1` something failed or an endpoint could not be reached. + +## Where things are kept + +Everything machine-local lives under `$XDG_STATE_HOME/calcalist`: + +``` +state.json event mappings, content hashes, sync cursors +lock held for the duration of a cycle +pimsync.conf generated on every sync; edits are overwritten +pimsync-status/ pimsync's own record of what it has seen +vdir// the local mirror of each endpoint +google/accounts/.json refresh tokens, 0600 +``` + +None of it belongs in a backup or a dotfiles repository — it is a cache, and a lost state file costs a re-materialisation rather than data. Copy the +configuration between machines; leave this behind. + +## Known gaps + +- **A failing `pimsync sync` stops the whole CalDAV leg.** Google endpoints are isolated from each other, but pimsync is a single process covering every + CalDAV and feed pair, and a failure does not say which pair it belongs to. + +## License + +MIT. See [LICENSE](LICENSE). diff --git a/SPECS.md b/SPECS.md deleted file mode 100644 index 7300e38..0000000 --- a/SPECS.md +++ /dev/null @@ -1,56 +0,0 @@ -# Calcalist - -## Idea - -Small Rust based commandline tool with utility systemd service file that can aggregate and sync events between CalDAV/Google Calendar/iCal. Users who have a Google Calendar can create a new calendar that syncs events from several CalDAV calendars. CalDAV users can sync several Google Calendars into one calendar. This should be doable both ways, so the aggregating calendar needs to be able to distinguish where its events came from and also have a way to create new events that get synced to the correct source. In addition iCal feeds can also be aggregated but they remain one way only. - -Configuration via a simple web interface. Configuration is saved in a portable toml file that can be easily migrated. - -## Architecture - -Every endpoint — sources and aggregate targets alike — is mirrored to a local vdir, so the aggregation engine works purely on local files. - -- `pimsync` is driven as a one-shot subprocess for the CalDAV and WebCal legs. Its daemon mode is unusable here: it would race the reconciler over the same vdir files, so calcalist owns scheduling. -- The Google leg is calcalist's own, via the Calendar REST API. pimsync cannot reach Google — it has no REST storage, and its config exposes only HTTP Basic auth, which Google's CalDAV endpoint has rejected since 2025-03-14. -- Sync state (event mappings, hashes, sync tokens) lives in a JSON sidecar under `$XDG_STATE_HOME/calcalist/`, never in the portable config. -- Secrets are never stored in the config either. Credentials come from `*_command` fields that are executed to fetch them. - -## Sync semantics - -| Question | Behaviour | -|---|---| -| Provenance | Aggregate UIDs derived as `blake3(aggregate_id, source_id, source_uid)`; the state file is a cache, not a single point of failure | -| Routing new events | A `@endpoint-id` line in the description, or a matching category, picks the source; otherwise the aggregate's `default_sink`. A hint naming an invalid sink is refused, never redirected to the default | -| Conflicts | Source wins — the origin calendar is authoritative | -| Deletion | Propagates to the source, guarded by a mass-deletion threshold | -| Attendees (mirroring) | Kept verbatim on a Google target; demoted to inert data on a CalDAV target | -| Attendees (routing) | Preserved — the sink server sends real invitations, which is intended | -| Alarms | Always preserved, with no option to strip them | -| Retargeting | `sync` refuses on target drift; `aggregate retarget` performs it deliberately | - -The governing rule behind the attendee handling: **writes to an aggregate must never emit scheduling mail; writes to a source schedule normally.** Google can be told not to notify, so attendees survive intact there. CalDAV offers no portable way to suppress RFC 6638 scheduling, so inertness is achieved structurally by dropping the live properties instead. - -Alarms are never stripped because the whole point of the tool is that a user subscribes to one calendar rather than several — so the duplicate-notification problem that stripping would guard against does not arise, while stripping would silently destroy every reminder in the one calendar the user actually watches. - -## Development - -Use devbox for dependency management and scripts. Use jj for version control (colocated with git). - -- `devbox run fmt` — format. -- `devbox run check` — format check, clippy with warnings denied, and tests. This is the gate; jj has no commit step to hang a hook off, since the working copy is itself a commit. -- `devbox run test` — tests only. - -Keep `/target` in `.gitignore`: jj snapshots the working copy on every command, with no staging step. - -## Coding style - -- Simple, readable, idiomatic. -- Explicit types for public APIs and important fields. -- Prefer immutable data structures. -- No global mutable state. -- No speculative features or dependencies. -- Keep functions short and single-purpose; extract helpers to avoid nesting. -- File and module naming: `snake_case.rs`. Types and traits: `PascalCase`. Functions, variables and fields: `snake_case`. Constants: `SCREAMING_SNAKE_CASE`. -- Run `devbox run check` before describing a change. -- Comment only when intent is not obvious from the code. -- Prefer composition and traits over deep type hierarchies. diff --git a/TODO.md b/TODO.md deleted file mode 100644 index 6c47de7..0000000 --- a/TODO.md +++ /dev/null @@ -1,90 +0,0 @@ -# Roadmap - -Milestones from the implementation plan. See SPECS.md for the architecture and the -sync semantics these items implement. - -## M0 — skeleton (done) - -- [x] Initialise jj colocated with git; `.gitignore` written before the first build -- [x] `devbox.json` pinning Rust 1.97.1, pimsync 0.5.11, jujutsu 0.44.0, radicale 3.7.8 -- [x] Configuration model with referential validation, reporting every problem in one pass -- [x] `calcalist doctor` — pimsync presence and version series, state directory, configuration -- [x] Define the full CLI surface; unimplemented commands exit 2 rather than pretend -- [x] SPECS.md: Rust naming conventions, `devbox run check` gate, recorded sync semantics - -## M1 — bidirectional sync - -Core modules: - -- [x] `state.rs` — JSON sidecar, atomic temp + fsync + rename; records each aggregate's - resolved target endpoint id **and** backend type -- [x] `vdir.rs` — read and write vdir directories -- [x] `ical.rs` — surgical line-level `.ics` editing (UID rewrite, property injection), - respecting RFC 5545 folding; no parse-and-reserialize -- [x] `provenance.rs` — deterministic `blake3(aggregate_id, source_id, source_uid)` UIDs -- [x] `mirror.rs` — the to-aggregate and to-source transforms (added; not in the - original plan, which folded these into `reconcile`) -- [x] `reconcile.rs` — the aggregation engine; pure, no I/O -- [x] `sync.rs` — one cycle over the local vdirs, applying what `reconcile` decides -- [x] `pimsync.rs` — generate `pimsync.conf` (with `on_empty skip` and `on_delete skip`), - drive one-shot `pimsync sync` bracketing the reconcile step -- [x] `doctor` asks `pimsync check` to validate the generated config, since pimsync's - parser does not always match its documentation -- [x] `google/auth.rs` — OAuth loopback flow with PKCE, refresh, keyring-sourced secrets -- [x] `google/convert.rs` — JSON to iCalendar, including recurrence and timezones -- [x] `google/api.rs` — incremental pull by syncToken, and push by import / update - / delete with notification suppressed -- [x] Reintroduce `SchedulingSuppression` in `config.rs` (removed in M0 as dead code) - -Safety-critical behaviour: - -- [x] **`events.import` gate** — settled from Google's own API discovery document: - `events.import` accepts no `sendUpdates` parameter at all, while `insert`, - `update` and `delete` all do, and it is documented as adding "a private copy of - an existing event". Confirmed live that attendees and alarms survive an import. - Creation goes through `import`; update and delete pass `sendUpdates=none`. -- [x] `sync` refuses to run on aggregate target drift, before reconciliation -- [x] `aggregate retarget` — flush unrouted creations against the old target, then - re-materialise; keep old orphans by default -- [x] Mass-deletion guard (`max_delete_fraction`), overridable with `--force`, with an - absolute floor so deleting a couple of events is never refused -- [x] Echo suppression: derived UIDs are never re-ingested as source events - -Tests: - -- [x] `reconcile` table-driven cases: create/update/delete each direction, both-sides-changed, - routing, echo suppression, mass-delete abort -- [x] `ical` round-trip fixtures: recurring with overrides, all-day, TZID, unknown `X-` props -- [ ] Integration against Radicale plus a `file://` WebCal fixture; assert idempotence -- [x] Safety (unit level): no live `ATTENDEE`/`ORGANIZER` on a CalDAV-targeted mirror, - `VALARM` intact, `PARTSTAT: DECLINED` maps to `TRANSP: TRANSPARENT`, bulk deletion aborts -- [ ] Safety (integration): the same against a real Radicale instance with an SMTP sink, - proving no mail is emitted -- [x] Retarget: drift makes `sync` exit non-zero having written nothing and losing no source - event; purge is bounded by the derivation; an unrouted creation reaches a sink first - -### Verified live, end to end - -Two Google calendars aggregating into a Posteo CalDAV calendar, against real accounts: -fan-in from both sources with provenance intact; an edit in the aggregate reaching the -originating Google calendar; an event created in the aggregate routed to a chosen -source by a description marker; deletion propagating from the aggregate through to -Google; and the mass-deletion guard refusing a 100% removal until `--force`. - -### Known gaps carried out of M1 - -- [ ] A recurring series' *exceptions* are not pushed to Google. Google models them - as separate events against an already existing series, so they need - `events.instances` plus a patch per exception. Reported per sync rather than - dropped silently. -- [x] `push` deleting an event remotely — verified live: deleting a mirror in the - CalDAV aggregate removed the origin event from Google. -- [ ] A failed Google pull aborts the whole cycle, including the CalDAV side. Safe — - reconciling against a stale snapshot could read as mass deletion — but it means - a lapsed token stops everything. - -## M2 — interface and packaging - -- [ ] axum configuration UI, bound to 127.0.0.1 -- [ ] OAuth loopback redirect handler -- [ ] systemd user units: `calcalist.service` (oneshot) and `calcalist.timer` diff --git a/src/cli.rs b/src/cli.rs index faffd2c..d072276 100644 --- a/src/cli.rs +++ b/src/cli.rs @@ -32,10 +32,17 @@ pub enum Command { }, /// Show endpoints, aggregates and the last sync result Status, - /// Serve the configuration web interface on localhost - Serve { - #[arg(long, default_value_t = 8723)] - port: u16, + /// List the calendars a CalDAV server offers, as endpoint blocks to paste + Discover { + /// The account or principal URL to enumerate, not a single calendar + url: String, + /// Username to authenticate as + #[arg(long)] + username: String, + /// Command printing the password, exactly as it would appear in the + /// configuration. Secrets are never passed as arguments. + #[arg(long, value_name = "COMMAND")] + secret_command: Option, }, /// Google account operations Google { @@ -47,6 +54,12 @@ pub enum Command { #[command(subcommand)] command: AggregateCommand, }, + /// Remove local mirrors of endpoints the configuration no longer names + Prune { + /// Delete them; without this the command only reports what it found + #[arg(long)] + force: bool, + }, /// Check that the environment and configuration are usable Doctor, } diff --git a/src/config.rs b/src/config.rs index 779f039..0617fc3 100644 --- a/src/config.rs +++ b/src/config.rs @@ -43,6 +43,13 @@ pub enum EndpointKind { calendar_id: String, client_id: String, client_secret_command: Option, + /// Which authorised Google account this calendar belongs to. + /// + /// One authorisation covers every calendar an account can see, so + /// several endpoints on one account share it and this can be left out. + /// It is only needed to say which account is meant when calcalist has + /// been logged in to more than one. + account: Option, }, Webcal { /// The feed URL, when it is not sensitive. @@ -68,6 +75,14 @@ impl EndpointKind { } } + /// Which authorised Google account this endpoint names, if it says. + pub fn google_account(&self) -> Option<&str> { + match self { + Self::Google { account, .. } => account.as_deref(), + _ => None, + } + } + pub fn scheduling_suppression(&self) -> SchedulingSuppression { match self { Self::Google { .. } => SchedulingSuppression::Native, @@ -192,10 +207,20 @@ pub enum Problem { NoUrlSource { endpoint: String }, #[error("endpoint `{endpoint}` has both `url` and `url_command`; use one or the other")] AmbiguousUrlSource { endpoint: String }, + #[error( + "endpoint `{endpoint}` uses a reserved id: `@{endpoint}` means \"keep this event in the aggregate\", so an endpoint of that name could never be routed to" + )] + ReservedEndpointId { endpoint: String }, } -/// A webcal feed must name exactly one source for its URL. +/// A webcal feed must name exactly one source for its URL, and no endpoint may +/// take a name the routing markers have already spoken for. fn check_endpoint(endpoint: &Endpoint, problems: &mut Vec) { + if crate::mirror::is_local_sink(&endpoint.id) { + problems.push(Problem::ReservedEndpointId { + endpoint: endpoint.id.clone(), + }); + } let EndpointKind::Webcal { url, url_command } = &endpoint.kind else { return; }; @@ -594,4 +619,25 @@ sources = ["work", "work"] endpoint: "work".into(), })); } + + /// `@local` already means "keep this event here", so an endpoint of that + /// name could never be routed to. + #[test] + fn an_endpoint_may_not_take_the_reserved_name() { + let problems = problems( + r#" +[[endpoint]] +id = "local" +type = "caldav" +url = "https://caldav.example.com/other/" +username = "me@example.com" +"#, + ); + assert!( + problems + .iter() + .any(|problem| matches!(problem, Problem::ReservedEndpointId { .. })), + "{problems:?}" + ); + } } diff --git a/src/doctor.rs b/src/doctor.rs index 228c4ea..348e0be 100644 --- a/src/doctor.rs +++ b/src/doctor.rs @@ -143,7 +143,12 @@ fn check_google_authorisation(config: &Config) -> Vec { let outcome = match auth::credentials_for(config, &endpoint.id) { Err(error) => Outcome::Fail(error.to_string()), Ok(credentials) => { - match auth::access_token(&state_dir, &endpoint.id, &credentials) { + match auth::access_token( + &state_dir, + &endpoint.id, + endpoint.kind.google_account(), + &credentials, + ) { Ok(_) => Outcome::Ok(format!("`{}` is authorised", endpoint.id)), Err(error @ (AuthError::NotAuthorised(_) | AuthError::Expired { .. })) => { Outcome::Warn(error.to_string()) diff --git a/src/google/api.rs b/src/google/api.rs index 29bfa54..03fd12e 100644 --- a/src/google/api.rs +++ b/src/google/api.rs @@ -161,6 +161,44 @@ impl Client { Self::event_id(&body) } + /// The occurrences of a series inside a window. + /// + /// Exceptions are addressed through the series rather than created + /// directly, so applying one means finding the instance it replaces. + pub fn instances( + &self, + master_id: &str, + time_min: &str, + time_max: &str, + ) -> Result, ApiError> { + let path = format!("events/{}/instances", percent_encode(master_id)); + let query = [ + ("maxResults".to_string(), PAGE_SIZE.to_string()), + ("timeMin".to_string(), time_min.to_string()), + ("timeMax".to_string(), time_max.to_string()), + ("showDeleted".to_string(), "false".to_string()), + ]; + let body = self.get(&path, &query)?; + let page: serde_json::Value = + serde_json::from_str(&body).map_err(|error| ApiError::Malformed(error.to_string()))?; + let items = page + .get("items") + .cloned() + .unwrap_or(serde_json::Value::Array(Vec::new())); + serde_json::from_value(items).map_err(|error| ApiError::Malformed(error.to_string())) + } + + /// Merges fields into an existing event, leaving the rest alone. + /// + /// An instance carries properties calcalist does not model — its + /// `recurringEventId` and `originalStartTime` above all, which say which + /// occurrence it replaces — so an override is patched rather than replaced. + pub fn patch(&self, id: &str, event: &serde_json::Value) -> Result<(), ApiError> { + let path = format!("events/{}", percent_encode(id)); + self.patch_json(&path, &SUPPRESS_NOTIFICATION, event)?; + Ok(()) + } + pub fn delete(&self, id: &str) -> Result<(), ApiError> { let path = format!("events/{}", percent_encode(id)); match self.delete_path(&path, &SUPPRESS_NOTIFICATION) { @@ -223,6 +261,26 @@ impl Client { ) } + fn patch_json( + &self, + path: &str, + query: &[(&str, &str)], + body: &serde_json::Value, + ) -> Result { + let mut request = self + .agent + .patch(&self.url(path)) + .header("Authorization", format!("Bearer {}", self.access_token)); + for (key, value) in query { + request = request.query(*key, *value); + } + Self::finish( + request + .send_json(body) + .map_err(|error| ApiError::Request(error.to_string()))?, + ) + } + fn delete_path(&self, path: &str, query: &[(&str, &str)]) -> Result { let mut request = self .agent @@ -358,7 +416,9 @@ pub struct PushReport { pub created: usize, pub updated: usize, pub deleted: usize, - /// Series whose exceptions could not be applied; see `push`. + /// Recurrence overrides applied to their instances. + pub exceptions_applied: usize, + /// Overrides whose instance Google did not offer; see `apply_exceptions`. pub exceptions_skipped: usize, } @@ -388,16 +448,11 @@ pub fn push( } }; // A series' exceptions are separate events against an existing series, - // so they cannot ride along with the master. Reported rather than - // silently dropped. - if item - .calendar - .properties("VEVENT", "RECURRENCE-ID") - .next() - .is_some() - { - report.exceptions_skipped += 1; - } + // so they cannot ride along with the master and are applied once it is + // in place. + let applied = apply_exceptions(client, &id, &item.calendar)?; + report.exceptions_applied += applied.applied; + report.exceptions_skipped += applied.skipped; state.events.insert( uid.clone(), GoogleItem { @@ -423,6 +478,46 @@ pub fn push( Ok(report) } +#[derive(Debug, Default)] +struct ExceptionReport { + applied: usize, + skipped: usize, +} + +/// Applies a stored series' recurrence overrides to the instances they replace. +/// +/// Google has no way to send a series and its exceptions in one request: an +/// exception is a separate event, addressed through the series and identified by +/// the occurrence it stands in for. So the master goes up first, then each +/// override is matched to an instance by original start time and patched. +/// +/// An override with no matching instance is counted rather than forced. That +/// happens when the occurrence falls outside the rule — a stale `RECURRENCE-ID` +/// left behind by an edited `RRULE` — and inventing an event for it would put +/// something in the calendar that the series does not contain. +fn apply_exceptions( + client: &Client, + master_id: &str, + calendar: &Calendar, +) -> Result { + let mut report = ExceptionReport::default(); + for exception in convert::exceptions(calendar) { + let Some(key) = convert::recurrence_key(&exception) else { + report.skipped += 1; + continue; + }; + let (time_min, time_max) = key.window(); + let instances = client.instances(master_id, &time_min, &time_max)?; + let Some(instance) = convert::find_instance(&instances, &key) else { + report.skipped += 1; + continue; + }; + client.patch(&instance.id, &convert::to_exception(&exception)?)?; + report.applied += 1; + } + Ok(report) +} + /// Percent-encodes a path segment; calendar ids contain `@` and event ids may /// contain characters that would otherwise change the URL's meaning. fn percent_encode(value: &str) -> String { diff --git a/src/google/auth.rs b/src/google/auth.rs index ac5045f..b7c92b4 100644 --- a/src/google/auth.rs +++ b/src/google/auth.rs @@ -74,6 +74,21 @@ pub enum AuthError { NoRefreshToken, #[error("endpoint `{0}` has not been authorised; run `calcalist google login {0}`")] NotAuthorised(String), + #[error( + "endpoint `{endpoint}` names account `{account}`, which has not been authorised; \ + run `calcalist google login {endpoint}`" + )] + UnknownAccount { endpoint: String, account: String }, + #[error( + "endpoint `{endpoint}` does not say which Google account it belongs to, and calcalist \ + is logged in to several ({accounts}). Add `account = \"…\"` to the endpoint." + )] + AmbiguousAccount { endpoint: String, accounts: String }, + #[error( + "the authorisation stored for account `{account}` was issued to a different OAuth \ + client than endpoint `{endpoint}` configures; run `calcalist google login {endpoint}`" + )] + ClientMismatch { endpoint: String, account: String }, #[error("could not access {path}: {source}")] Io { path: PathBuf, @@ -98,6 +113,15 @@ pub struct StoredToken { /// Unix seconds at which `access_token` stops being usable. #[serde(default)] pub expires_at: u64, + /// The account this authorisation belongs to. Absent in files written + /// before authorisations were shared between endpoints. + #[serde(default)] + pub account: Option, + /// The OAuth client the refresh token was issued to. A refresh token is + /// only valid for the client that obtained it, so a changed `client_id` + /// has to be caught here rather than as an opaque `invalid_grant`. + #[serde(default)] + pub client_id: Option, } impl StoredToken { @@ -109,11 +133,93 @@ impl StoredToken { } } -/// Where an endpoint's refresh token lives. -pub fn token_path(state_dir: &Path, endpoint_id: &str) -> PathBuf { +/// Where an authorisation lives, keyed by the account it was granted for. +/// +/// An authorisation covers every calendar the account can see, so two endpoints +/// on one account share this file and only one of them ever has to log in. +pub fn account_token_path(state_dir: &Path, account: &str) -> PathBuf { + state_dir + .join("google") + .join("accounts") + .join(format!("{}.json", sanitise(account))) +} + +/// Where an endpoint's refresh token used to live, before authorisations were +/// keyed by account. Still read, so an existing installation keeps working +/// until its next login moves it across. +pub fn legacy_token_path(state_dir: &Path, endpoint_id: &str) -> PathBuf { state_dir.join("google").join(format!("{endpoint_id}.json")) } +/// Keeps an account name to one harmless path segment. +fn sanitise(account: &str) -> String { + account + .chars() + .map(|ch| match ch { + 'a'..='z' | 'A'..='Z' | '0'..='9' | '-' | '_' | '.' | '@' => ch, + _ => '_', + }) + .collect() +} + +/// Every account calcalist currently holds an authorisation for. +pub fn authorised_accounts(state_dir: &Path) -> Vec { + let Ok(entries) = std::fs::read_dir(state_dir.join("google").join("accounts")) else { + return Vec::new(); + }; + let mut accounts: Vec = entries + .flatten() + .filter_map(|entry| load_token(&entry.path()).ok().flatten()) + .filter_map(|token| token.account) + .collect(); + accounts.sort(); + accounts.dedup(); + accounts +} + +/// Picks the stored authorisation an endpoint should use. +/// +/// The endpoint may name its account outright. Failing that, an authorisation +/// left over from the per-endpoint scheme is honoured, and then a single +/// account is taken to be the one meant — which is the ordinary case, and what +/// makes a second endpoint on the same account need no login of its own. +fn resolve_token( + state_dir: &Path, + endpoint_id: &str, + account: Option<&str>, +) -> Result<(PathBuf, StoredToken), AuthError> { + if let Some(account) = account { + let path = account_token_path(state_dir, account); + let stored = load_token(&path)?.ok_or_else(|| AuthError::UnknownAccount { + endpoint: endpoint_id.to_string(), + account: account.to_string(), + })?; + return Ok((path, stored)); + } + + let legacy = legacy_token_path(state_dir, endpoint_id); + if let Some(stored) = load_token(&legacy)? { + return Ok((legacy, stored)); + } + + let accounts = authorised_accounts(state_dir); + match accounts.as_slice() { + [] => Err(AuthError::NotAuthorised(endpoint_id.to_string())), + [only] => { + let path = account_token_path(state_dir, only); + let stored = load_token(&path)?.ok_or_else(|| AuthError::UnknownAccount { + endpoint: endpoint_id.to_string(), + account: only.clone(), + })?; + Ok((path, stored)) + } + several => Err(AuthError::AmbiguousAccount { + endpoint: endpoint_id.to_string(), + accounts: several.join(", "), + }), + } +} + /// Runs an external command and returns its first line of output. /// /// Secrets are fetched this way rather than stored, so the configuration file @@ -181,11 +287,18 @@ pub fn credentials_for( pub fn access_token( state_dir: &Path, endpoint_id: &str, + account: Option<&str>, credentials: &Credentials, ) -> Result { - let path = token_path(state_dir, endpoint_id); - let mut stored = - load_token(&path)?.ok_or_else(|| AuthError::NotAuthorised(endpoint_id.to_string()))?; + let (path, mut stored) = resolve_token(state_dir, endpoint_id, account)?; + if let Some(issued_to) = &stored.client_id + && *issued_to != credentials.client_id + { + return Err(AuthError::ClientMismatch { + endpoint: endpoint_id.to_string(), + account: stored.account.clone().unwrap_or_else(|| "?".to_string()), + }); + } if let Some(token) = stored.usable_access_token() { return Ok(token.to_string()); @@ -248,19 +361,66 @@ pub fn login( ])?; let refresh_token = issued.refresh_token.ok_or(AuthError::NoRefreshToken)?; - let path = token_path(state_dir, endpoint_id); + // Which account consented is not in the token response, but the primary + // calendar's id is the account's own address — and reading it needs nothing + // beyond the calendar scope already granted, unlike the userinfo endpoint. + let account = primary_calendar_id(&issued.access_token); + let path = match &account { + Some(account) => account_token_path(state_dir, account), + None => legacy_token_path(state_dir, endpoint_id), + }; save_token( &path, &StoredToken { refresh_token, access_token: Some(issued.access_token), expires_at: unix_now() + issued.expires_in, + account: account.clone(), + client_id: Some(credentials.client_id.clone()), }, )?; - println!("Authorised. Refresh token stored at {}", path.display()); + // The authorisation now lives under the account, so the copy this endpoint + // used to keep would only shadow it. + let legacy = legacy_token_path(state_dir, endpoint_id); + if account.is_some() && legacy.exists() { + let _ = std::fs::remove_file(&legacy); + } + match &account { + Some(account) => println!( + "Authorised {account}. Every endpoint on this account can use it; \ + the refresh token is stored at {}", + path.display() + ), + None => println!("Authorised. Refresh token stored at {}", path.display()), + } Ok(()) } +/// The address of the authorised account, read from its primary calendar. +/// +/// Returned as an option rather than an error: failing to name the account only +/// costs the sharing, and the authorisation itself is perfectly good without it. +fn primary_calendar_id(access_token: &str) -> Option { + let agent: ureq::Agent = ureq::Agent::config_builder() + .http_status_as_error(false) + .build() + .into(); + let mut response = agent + .get("https://www.googleapis.com/calendar/v3/calendars/primary") + .header("Authorization", format!("Bearer {access_token}")) + .call() + .ok()?; + if !(200..300).contains(&response.status().as_u16()) { + return None; + } + let body = response.body_mut().read_to_string().ok()?; + let value: serde_json::Value = serde_json::from_str(&body).ok()?; + value + .get("id") + .and_then(|id| id.as_str()) + .map(str::to_string) +} + /// Accepts one loopback request and extracts the authorisation code from it. fn wait_for_code(listener: &TcpListener, expected_state: &str) -> Result { let (mut stream, _) = listener.accept().map_err(AuthError::Listen)?; @@ -586,6 +746,8 @@ mod tests { refresh_token: "r".into(), access_token: Some("a".into()), expires_at: unix_now() + 3600, + account: None, + client_id: None, }; assert_eq!(fresh.usable_access_token(), Some("a")); @@ -602,4 +764,101 @@ mod tests { }; assert_eq!(nearly.usable_access_token(), None); } + + fn stored(account: &str, client_id: &str) -> StoredToken { + StoredToken { + refresh_token: format!("refresh-for-{account}"), + access_token: None, + expires_at: 0, + account: Some(account.to_string()), + client_id: Some(client_id.to_string()), + } + } + + /// The gap this closes: two endpoints on one account needed a login each, + /// because the authorisation was filed under the endpoint rather than the + /// account it actually belongs to. + #[test] + fn one_authorisation_serves_every_endpoint_on_that_account() { + let dir = tempfile::tempdir().expect("temp"); + save_token( + &account_token_path(dir.path(), "me@example.com"), + &stored("me@example.com", "client-a"), + ) + .expect("save"); + + for endpoint in ["work", "personal", "never-logged-in"] { + let (_, token) = resolve_token(dir.path(), endpoint, None).expect("resolve"); + assert_eq!(token.refresh_token, "refresh-for-me@example.com"); + } + } + + /// Two accounts cannot be guessed between, so the endpoint has to say. + #[test] + fn several_accounts_have_to_be_told_apart() { + let dir = tempfile::tempdir().expect("temp"); + for account in ["one@example.com", "two@example.com"] { + save_token( + &account_token_path(dir.path(), account), + &stored(account, "client-a"), + ) + .expect("save"); + } + + let error = resolve_token(dir.path(), "work", None).expect_err("ambiguous"); + assert!( + matches!(error, AuthError::AmbiguousAccount { .. }), + "{error:?}" + ); + assert!(error.to_string().contains("one@example.com"), "{error}"); + + let (_, token) = + resolve_token(dir.path(), "work", Some("two@example.com")).expect("resolve"); + assert_eq!(token.refresh_token, "refresh-for-two@example.com"); + } + + /// An authorisation written by an earlier version keeps working untouched. + #[test] + fn an_authorisation_from_the_old_layout_is_still_honoured() { + let dir = tempfile::tempdir().expect("temp"); + save_token( + &legacy_token_path(dir.path(), "work"), + &StoredToken { + refresh_token: "old".into(), + access_token: None, + expires_at: 0, + account: None, + client_id: None, + }, + ) + .expect("save"); + + let (_, token) = resolve_token(dir.path(), "work", None).expect("resolve"); + assert_eq!(token.refresh_token, "old"); + // A different endpoint has nothing to fall back on. + let error = resolve_token(dir.path(), "other", None).expect_err("not authorised"); + assert!(matches!(error, AuthError::NotAuthorised(_)), "{error:?}"); + } + + /// A refresh token only works for the client it was issued to, so a changed + /// client_id has to say so rather than surface as an opaque invalid_grant. + #[test] + fn a_token_from_another_oauth_client_is_refused_by_name() { + let dir = tempfile::tempdir().expect("temp"); + save_token( + &account_token_path(dir.path(), "me@example.com"), + &stored("me@example.com", "client-a"), + ) + .expect("save"); + + let credentials = Credentials { + client_id: "client-b".into(), + client_secret: "secret".into(), + }; + let error = access_token(dir.path(), "work", None, &credentials).expect_err("mismatch"); + assert!( + matches!(error, AuthError::ClientMismatch { .. }), + "{error:?}" + ); + } } diff --git a/src/google/convert.rs b/src/google/convert.rs index e69ec99..4543f32 100644 --- a/src/google/convert.rs +++ b/src/google/convert.rs @@ -14,9 +14,9 @@ //! * An exception that was deleted comes back as an override with //! `status: cancelled`, which is an `EXDATE` on the master rather than an event. -use std::collections::BTreeMap; +use std::collections::{BTreeMap, BTreeSet}; -use jiff::{Timestamp, tz::TimeZone}; +use jiff::{SignedDuration, Timestamp, tz::Dst, tz::TimeZone}; use serde::Deserialize; use thiserror::Error; @@ -157,34 +157,186 @@ pub fn to_ical(uid: &str, group: &Group) -> Result { }); }; - let mut out = String::from("BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:-//calcalist//EN\r\n"); - out.push_str(&render_event(uid, master, None)?); + let mut zones = BTreeSet::new(); + let mut body = render_event(uid, master, None, &mut zones)?; // A deleted occurrence is an absence, not an event: it belongs on the master // as an EXDATE. for cancelled in group.overrides.iter().filter(|event| event.is_cancelled()) { if let Some(start) = &cancelled.original_start_time { - let line = time_property("EXDATE", start, &cancelled.id, master.is_recurring())?; + let line = time_property( + "EXDATE", + start, + &cancelled.id, + master.is_recurring(), + &mut zones, + )?; // Insert before the master's END:VEVENT. - let end = out.rfind("END:VEVENT\r\n").unwrap_or(out.len()); - out.insert_str(end, &format!("{line}\r\n")); + let end = body.rfind("END:VEVENT\r\n").unwrap_or(body.len()); + body.insert_str(end, &format!("{line}\r\n")); } } for moved in group.overrides.iter().filter(|event| !event.is_cancelled()) { - out.push_str(&render_event( + body.push_str(&render_event( uid, moved, moved.original_start_time.as_ref(), + &mut zones, )?); } + + let mut out = String::from("BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:-//calcalist//EN\r\n"); + // A TZID names a zone the reader is expected to resolve, but RFC 5545 wants + // the definition to travel with the calendar: a client with no zone database + // of its own has nothing else to expand the recurrence against. + let reference = master + .start + .as_ref() + .and_then(instant_of) + .unwrap_or_else(Timestamp::now); + for zone in &zones { + if let Some(block) = vtimezone(zone, reference) { + out.push_str(&block); + } + } + out.push_str(&body); out.push_str("END:VCALENDAR\r\n"); Ok(out) } +/// The instant an event time names, whether it is dated or timed. +fn instant_of(time: &EventTime) -> Option { + match (&time.date, &time.date_time) { + (_, Some(value)) => value.parse().ok(), + (Some(date), None) => midnight_utc(date), + (None, None) => None, + } +} + +/// An RFC 5545 `VTIMEZONE` for `name`, describing the rules around `reference`. +/// +/// Two observances describe every zone in current use — one standard, one +/// daylight — each recurring yearly on the same weekday of the same month. A +/// zone that does not shift gets a single fixed observance instead, which is +/// also the fallback when the transition table says nothing about this period. +fn vtimezone(name: &str, reference: Timestamp) -> Option { + let zone = TimeZone::get(name).ok()?; + // Start a little over a year back, so a full cycle of transitions is in + // view however close the event sits to one of them. + let from = reference + .checked_sub(SignedDuration::from_hours(24 * 400)) + .ok()?; + let upcoming: Vec<_> = zone.following(from).take(4).collect(); + + let mut out = format!("BEGIN:VTIMEZONE\r\nTZID:{name}\r\n"); + let daylight = upcoming + .iter() + .find(|transition| transition.dst() == Dst::Yes); + let standard = upcoming + .iter() + .find(|transition| transition.dst() == Dst::No); + + match (daylight, standard) { + (Some(daylight), Some(standard)) => { + out.push_str(&observance(&zone, daylight, "DAYLIGHT")?); + out.push_str(&observance(&zone, standard, "STANDARD")?); + } + _ => { + let offset = zone.to_offset(reference); + let text = format_offset(offset); + out.push_str(&format!( + "BEGIN:STANDARD\r\nDTSTART:19700101T000000\r\nTZOFFSETFROM:{text}\r\n\ + TZOFFSETTO:{text}\r\nEND:STANDARD\r\n" + )); + } + } + out.push_str("END:VTIMEZONE\r\n"); + Some(out) +} + +/// One `STANDARD` or `DAYLIGHT` block, as the yearly rule that produced it. +fn observance( + zone: &TimeZone, + transition: &jiff::tz::TimeZoneTransition<'_>, + component: &str, +) -> Option { + let to = transition.offset(); + // The offset in force a moment earlier is the one the shift moved away from. + let before = transition + .timestamp() + .checked_sub(SignedDuration::from_secs(1)) + .ok()?; + let from = zone.to_offset(before); + // A transition's local start is written in the *old* offset, since that is + // the clock a reader is still on when the rule fires. + let local = transition + .timestamp() + .to_zoned(TimeZone::fixed(from)) + .datetime(); + + let mut block = format!( + "BEGIN:{component}\r\nDTSTART:{}\r\nTZOFFSETFROM:{}\r\nTZOFFSETTO:{}\r\n\ + RRULE:{}\r\n", + local.strftime("%Y%m%dT%H%M%S"), + format_offset(from), + format_offset(to), + yearly_rule(local) + ); + let abbreviation = transition.abbreviation(); + if !abbreviation.is_empty() { + block.push_str(&format!("TZNAME:{abbreviation}\r\n")); + } + block.push_str(&format!("END:{component}\r\n")); + Some(block) +} + +/// The yearly recurrence a transition date implies, as "the nth weekday of the +/// month" — the form every zone's rules are actually written in. A date within +/// seven days of the month's end is the *last* such weekday rather than a fixed +/// ordinal, which is what keeps the rule stable across years of differing length. +fn yearly_rule(local: jiff::civil::DateTime) -> String { + let day = local.day(); + let ordinal = if day + 7 > local.days_in_month() { + -1 + } else { + (day - 1) / 7 + 1 + }; + format!( + "FREQ=YEARLY;BYMONTH={};BYDAY={ordinal}{}", + local.month(), + weekday_code(local.weekday()) + ) +} + +fn weekday_code(weekday: jiff::civil::Weekday) -> &'static str { + match weekday { + jiff::civil::Weekday::Monday => "MO", + jiff::civil::Weekday::Tuesday => "TU", + jiff::civil::Weekday::Wednesday => "WE", + jiff::civil::Weekday::Thursday => "TH", + jiff::civil::Weekday::Friday => "FR", + jiff::civil::Weekday::Saturday => "SA", + jiff::civil::Weekday::Sunday => "SU", + } +} + +/// `+0200` — RFC 5545's UTC offset form, which has no colon. +fn format_offset(offset: jiff::tz::Offset) -> String { + let total = offset.seconds(); + let sign = if total < 0 { '-' } else { '+' }; + let magnitude = total.abs(); + format!( + "{sign}{:02}{:02}", + magnitude / 3600, + (magnitude % 3600) / 60 + ) +} + fn render_event( uid: &str, event: &Event, recurrence_id: Option<&EventTime>, + zones: &mut BTreeSet, ) -> Result { let recurring = event.is_recurring() || recurrence_id.is_some(); let mut out = String::from("BEGIN:VEVENT\r\n"); @@ -197,16 +349,19 @@ fn render_event( original, &event.id, recurring, + zones, )?); out.push_str("\r\n"); } let start = event.start.as_ref().ok_or_else(|| ConvertError::NoStart { id: event.id.clone(), })?; - out.push_str(&time_property("DTSTART", start, &event.id, recurring)?); + out.push_str(&time_property( + "DTSTART", start, &event.id, recurring, zones, + )?); out.push_str("\r\n"); if let Some(end) = &event.end { - out.push_str(&time_property("DTEND", end, &event.id, recurring)?); + out.push_str(&time_property("DTEND", end, &event.id, recurring, zones)?); out.push_str("\r\n"); } for line in event.recurrence.iter().flatten() { @@ -269,6 +424,7 @@ fn time_property( time: &EventTime, id: &str, recurring: bool, + zones: &mut BTreeSet, ) -> Result { if let Some(date) = &time.date { return Ok(format!("{name};VALUE=DATE:{}", date.replace('-', ""))); @@ -292,6 +448,7 @@ fn time_property( match zone { Some((zone_name, zone)) => { + zones.insert(zone_name.to_string()); let local = instant.to_zoned(zone); Ok(format!( "{name};TZID={zone_name}:{}", @@ -319,18 +476,61 @@ fn stamp(event: &Event) -> Result { Ok(instant.strftime("%Y%m%dT%H%M%SZ").to_string()) } +/// Which half of a series a body describes. +/// +/// Google models an exception as a separate event addressed through the series +/// it belongs to, so it carries neither the series' `iCalUID` — which would +/// name the master — nor a recurrence rule of its own. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum Role { + Master, + Exception, +} + /// Renders a stored calendar as the JSON body Google expects. /// /// Only the master is described. A series' exceptions cannot be expressed in the -/// same request — Google models them as separate events against an already -/// existing series — so the caller applies those afterwards. +/// same request, so the caller applies those afterwards with `to_exception`. pub fn to_google(calendar: &Calendar) -> Result { - let uid = calendar.uid().ok_or_else(|| ConvertError::NoUid { + let master = master_of(calendar).ok_or_else(|| ConvertError::NoUid { id: "".to_string(), })?; + to_body(&master, Role::Master) +} +/// The master event of a stored file: the one component without a `RECURRENCE-ID`. +/// +/// Falls back to the file as a whole when it holds no `VEVENT` at all, so a +/// malformed item still reports through the ordinary error path. +pub fn master_of(calendar: &Calendar) -> Option { + calendar + .events() + .into_iter() + .find(|event| event.properties("VEVENT", "RECURRENCE-ID").next().is_none()) +} + +/// Every recurrence override in a stored file, in the order they appear. +pub fn exceptions(calendar: &Calendar) -> Vec { + calendar + .events() + .into_iter() + .filter(|event| event.properties("VEVENT", "RECURRENCE-ID").next().is_some()) + .collect() +} + +/// Renders one recurrence override as a patch against its Google instance. +pub fn to_exception(event: &Calendar) -> Result { + to_body(event, Role::Exception) +} + +fn to_body(calendar: &Calendar, role: Role) -> Result { let mut event = serde_json::Map::new(); - event.insert("iCalUID".into(), uid.into()); + if role == Role::Master { + let uid = calendar.uid().ok_or_else(|| ConvertError::NoUid { + id: "".to_string(), + })?; + event.insert("iCalUID".into(), uid.into()); + } event.insert("status".into(), status_of(calendar).into()); for (property, field) in [ @@ -356,6 +556,7 @@ pub fn to_google(calendar: &Calendar) -> Result // verbatim. Their unfolded form is what Google expects. let recurrence: Vec = ["RRULE", "EXDATE", "RDATE"] .iter() + .filter(|_| role == Role::Master) .flat_map(|name| calendar.property_lines("VEVENT", name)) .map(|line| serde_json::Value::from(line.to_string())) .collect(); @@ -517,6 +718,83 @@ fn google_time(calendar: &Calendar, name: &str) -> Result (String, String) { + let centre = match self { + InstanceKey::Instant(instant) => *instant, + InstanceKey::Date(date) => midnight_utc(date).unwrap_or_else(Timestamp::now), + }; + let margin = jiff::SignedDuration::from_hours(36); + ((centre - margin).to_string(), (centre + margin).to_string()) + } +} + +fn midnight_utc(date: &str) -> Option { + let date: jiff::civil::Date = date.parse().ok()?; + date.to_zoned(TimeZone::UTC) + .ok() + .map(|zoned| zoned.timestamp()) +} + +/// The occurrence a stored recurrence override refers to. +pub fn recurrence_key(event: &Calendar) -> Option { + let property = event.properties("VEVENT", "RECURRENCE-ID").next()?; + let value = property.value; + if property.param("VALUE") == Some("DATE") || value.len() == 8 { + return Some(InstanceKey::Date(hyphenate(value))); + } + let civil = parse_basic(value, "RECURRENCE-ID").ok()?; + let zone = match property.param("TZID") { + Some(name) => TimeZone::get(name).ok()?, + None => TimeZone::UTC, + }; + civil + .to_zoned(zone) + .ok() + .map(|zoned| InstanceKey::Instant(zoned.timestamp())) +} + +/// The occurrence a Google override refers to. +pub fn original_start_key(time: &EventTime) -> Option { + if let Some(date) = &time.date { + return Some(InstanceKey::Date(date.clone())); + } + time.date_time + .as_deref()? + .parse() + .ok() + .map(InstanceKey::Instant) +} + +/// Picks the instance an override belongs to out of a page of them. +pub fn find_instance<'a>(instances: &'a [Event], key: &InstanceKey) -> Option<&'a Event> { + instances.iter().find(|instance| { + instance + .original_start_time + .as_ref() + .and_then(original_start_key) + .is_some_and(|candidate| candidate == *key) + }) +} + /// Parses `YYYYMMDDTHHMMSS`, with or without a trailing Z. fn parse_basic(value: &str, field: &'static str) -> Result { let trimmed = value.trim_end_matches('Z'); @@ -860,4 +1138,154 @@ mod tests { "{ics}" ); } + + /// A zone that shifts gets both observances, each as the yearly rule the + /// transition implies: the EU moves on the last Sunday of March and October. + #[test] + fn a_zoned_recurrence_carries_its_timezone_definition() { + let json = r#"[{ + "id": "berlin", "status": "confirmed", "summary": "Standup", + "start": {"dateTime": "2026-03-03T09:00:00+01:00", "timeZone": "Europe/Berlin"}, + "end": {"dateTime": "2026-03-03T09:30:00+01:00", "timeZone": "Europe/Berlin"}, + "recurrence": ["RRULE:FREQ=WEEKLY;BYDAY=TU"], + "iCalUID": "berlin@google.com", "updated": "2026-01-01T00:00:00.000Z" + }]"#; + let groups = group_by_uid(parse(json)); + let ics = to_ical("berlin@google.com", &groups["berlin@google.com"]).expect("convert"); + + assert!( + ics.contains("BEGIN:VTIMEZONE\r\nTZID:Europe/Berlin\r\n"), + "{ics}" + ); + assert!(ics.contains("BEGIN:DAYLIGHT"), "{ics}"); + assert!( + ics.contains("TZOFFSETFROM:+0100\r\nTZOFFSETTO:+0200"), + "{ics}" + ); + assert!( + ics.contains("RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU"), + "{ics}" + ); + assert!(ics.contains("BEGIN:STANDARD"), "{ics}"); + assert!( + ics.contains("TZOFFSETFROM:+0200\r\nTZOFFSETTO:+0100"), + "{ics}" + ); + assert!( + ics.contains("RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU"), + "{ics}" + ); + // The definition has to precede the event that references it. + assert!( + ics.find("BEGIN:VTIMEZONE") < ics.find("BEGIN:VEVENT"), + "{ics}" + ); + assert!(Calendar::parse(&ics).is_ok(), "{ics}"); + } + + /// Pakistan abandoned daylight saving in 2009, so there is no rule to state — + /// only the offset that has applied ever since. + #[test] + fn a_zone_that_never_shifts_gets_one_fixed_observance() { + let groups = group_by_uid(parse(KARACHI_MASTER)); + let ics = to_ical("master1@google.com", &groups["master1@google.com"]).expect("convert"); + assert!(ics.contains("TZID:Asia/Karachi"), "{ics}"); + assert!( + ics.contains("TZOFFSETFROM:+0500\r\nTZOFFSETTO:+0500"), + "{ics}" + ); + assert!(!ics.contains("BEGIN:DAYLIGHT"), "{ics}"); + } + + /// A one-off is written in UTC, so it names no zone and needs no definition. + #[test] + fn an_unzoned_event_carries_no_timezone_definition() { + let json = r#"[{ + "id": "one", "status": "confirmed", "summary": "Once", + "start": {"dateTime": "2026-04-01T10:00:00+02:00", "timeZone": "Europe/Berlin"}, + "end": {"dateTime": "2026-04-01T11:00:00+02:00", "timeZone": "Europe/Berlin"}, + "iCalUID": "one@google.com", "updated": "2026-01-01T00:00:00.000Z" + }]"#; + let groups = group_by_uid(parse(json)); + let ics = to_ical("one@google.com", &groups["one@google.com"]).expect("convert"); + assert!(!ics.contains("VTIMEZONE"), "{ics}"); + } + + /// The master's rule must not leak into the override's body, or Google would + /// read the exception as a series of its own. + #[test] + fn an_exception_body_carries_no_recurrence_and_no_uid() { + let ics = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\n\ + BEGIN:VEVENT\r\nUID:series@calcalist\r\nDTSTAMP:20260101T000000Z\r\n\ + DTSTART;TZID=Europe/Berlin:20260303T090000\r\nRRULE:FREQ=WEEKLY;BYDAY=TU\r\n\ + SUMMARY:Standup\r\nEND:VEVENT\r\n\ + BEGIN:VEVENT\r\nUID:series@calcalist\r\nDTSTAMP:20260101T000000Z\r\n\ + RECURRENCE-ID;TZID=Europe/Berlin:20260310T090000\r\n\ + DTSTART;TZID=Europe/Berlin:20260310T110000\r\n\ + SUMMARY:Standup, moved\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n"; + let calendar = Calendar::parse(ics).expect("parse"); + + let exceptions = exceptions(&calendar); + assert_eq!(exceptions.len(), 1); + let body = to_exception(&exceptions[0]).expect("convert"); + assert!(body.get("recurrence").is_none(), "{body}"); + assert!(body.get("iCalUID").is_none(), "{body}"); + assert_eq!(body["summary"], "Standup, moved"); + assert_eq!(body["start"]["dateTime"], "2026-03-10T10:00:00Z"); + + // The master keeps its own summary rather than the override's. + let master = to_google(&calendar).expect("convert"); + assert_eq!(master["summary"], "Standup"); + assert_eq!(master["iCalUID"], "series@calcalist"); + assert_eq!(master["recurrence"][0], "RRULE:FREQ=WEEKLY;BYDAY=TU"); + } + + /// RECURRENCE-ID and originalStartTime spell the same instant differently: + /// one as a zoned local time, the other as an absolute offset. + #[test] + fn an_override_matches_its_instance_across_the_two_spellings() { + let ics = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\n\ + BEGIN:VEVENT\r\nUID:s@calcalist\r\nDTSTAMP:20260101T000000Z\r\n\ + RECURRENCE-ID;TZID=Europe/Berlin:20260310T090000\r\n\ + DTSTART;TZID=Europe/Berlin:20260310T110000\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n"; + let calendar = Calendar::parse(ics).expect("parse"); + let key = recurrence_key(&calendar).expect("a recurrence id"); + + let instances = parse( + r#"[ + {"id": "abc_20260303T080000Z", "status": "confirmed", + "start": {"dateTime": "2026-03-03T09:00:00+01:00"}, + "originalStartTime": {"dateTime": "2026-03-03T09:00:00+01:00"}}, + {"id": "abc_20260310T080000Z", "status": "confirmed", + "start": {"dateTime": "2026-03-10T11:00:00+01:00"}, + "originalStartTime": {"dateTime": "2026-03-10T09:00:00+01:00"}} + ]"#, + ); + let found = find_instance(&instances, &key).expect("a matching instance"); + assert_eq!(found.id, "abc_20260310T080000Z"); + + // The window has to contain the occurrence it was built for. + let (min, max) = key.window(); + assert!(min.as_str() < "2026-03-10T08:00:00Z", "{min}"); + assert!(max.as_str() > "2026-03-10T08:00:00Z", "{max}"); + } + + /// An all-day series identifies its occurrences by date on both sides. + #[test] + fn an_all_day_override_matches_by_date() { + let ics = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\n\ + BEGIN:VEVENT\r\nUID:s@calcalist\r\nDTSTAMP:20260101T000000Z\r\n\ + RECURRENCE-ID;VALUE=DATE:20260310\r\n\ + DTSTART;VALUE=DATE:20260311\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n"; + let calendar = Calendar::parse(ics).expect("parse"); + let key = recurrence_key(&calendar).expect("a recurrence id"); + assert_eq!(key, InstanceKey::Date("2026-03-10".into())); + + let instances = parse( + r#"[{"id": "abc_20260310", "status": "confirmed", + "start": {"date": "2026-03-11"}, + "originalStartTime": {"date": "2026-03-10"}}]"#, + ); + assert!(find_instance(&instances, &key).is_some()); + } } diff --git a/src/ical.rs b/src/ical.rs index 78c188b..89524c3 100644 --- a/src/ical.rs +++ b/src/ical.rs @@ -134,15 +134,62 @@ impl Calendar { pub fn to_ics(&self) -> String { let mut out = String::with_capacity(self.source.len() + 64); for line in &self.lines { - match &line.body { - Body::Original(span) => out.push_str(&self.source[span.clone()]), - Body::Generated(text) => out.push_str(&fold(text, &self.newline)), - } + out.push_str(&self.rendered(line)); out.push_str(&self.newline); } out } + /// One line as it will be emitted: verbatim when untouched, folded when built. + fn rendered(&self, line: &Line) -> String { + match &line.body { + Body::Original(span) => self.source[span.clone()].to_string(), + Body::Generated(text) => fold(text, &self.newline), + } + } + + /// Each `VEVENT` as a calendar of its own, nested components included. + /// + /// `properties` deliberately flattens every component of a name together, + /// which is right for the UID a series shares but wrong for a recurrence + /// override: there its `SUMMARY` and the master's are indistinguishable. + /// Splitting gives each component a view of its own. It re-parses, unlike + /// the rest of this module, which is acceptable because the only caller + /// converts to another model rather than round-tripping back to iCalendar. + pub fn events(&self) -> Vec { + let newline = &self.newline; + let mut out = Vec::new(); + let mut block: Option = None; + for line in &self.lines { + let Some(depth) = line + .path + .iter() + .position(|name| name.eq_ignore_ascii_case("VEVENT")) + else { + continue; + }; + let outermost = depth + 1 == line.path.len(); + if outermost && line.has_name("BEGIN") { + block = Some(String::new()); + } + let Some(text) = block.as_mut() else { + continue; + }; + text.push_str(&self.rendered(line)); + text.push_str(newline); + if outermost + && line.has_name("END") + && let Some(text) = block.take() + && let Ok(event) = Calendar::parse(&format!( + "BEGIN:VCALENDAR{newline}VERSION:2.0{newline}{text}END:VCALENDAR{newline}" + )) + { + out.push(event); + } + } + out + } + /// The UID shared by the event and any recurrence overrides. pub fn uid(&self) -> Option<&str> { self.properties("VEVENT", "UID").next().map(|p| p.value) @@ -678,4 +725,47 @@ mod tests { IcalError::DanglingContinuation { line: 1 } ); } + + /// Splitting has to keep each component's nested parts with it, and give a + /// master and its override separate views of the properties they share. + #[test] + fn events_are_split_with_their_nested_components() { + let ics = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\n\ + BEGIN:VEVENT\r\nUID:s@x\r\nSUMMARY:Series\r\n\ + BEGIN:VALARM\r\nACTION:DISPLAY\r\nTRIGGER:-PT10M\r\nEND:VALARM\r\n\ + END:VEVENT\r\n\ + BEGIN:VEVENT\r\nUID:s@x\r\nRECURRENCE-ID:20260310T090000Z\r\n\ + SUMMARY:Moved\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n"; + let calendar = Calendar::parse(ics).expect("parse"); + + // Flattened, the two summaries are indistinguishable. + assert_eq!(calendar.properties("VEVENT", "SUMMARY").count(), 2); + + let events = calendar.events(); + assert_eq!(events.len(), 2); + assert_eq!( + events[0] + .properties("VEVENT", "SUMMARY") + .next() + .unwrap() + .value, + "Series" + ); + assert_eq!(events[0].properties("VALARM", "TRIGGER").count(), 1); + assert!( + events[0] + .properties("VEVENT", "RECURRENCE-ID") + .next() + .is_none() + ); + assert_eq!( + events[1] + .properties("VEVENT", "SUMMARY") + .next() + .unwrap() + .value, + "Moved" + ); + assert_eq!(events[1].properties("VALARM", "TRIGGER").count(), 0); + } } diff --git a/src/lock.rs b/src/lock.rs new file mode 100644 index 0000000..bde4e86 --- /dev/null +++ b/src/lock.rs @@ -0,0 +1,144 @@ +//! A lock held for the duration of anything that writes. +//! +//! A cycle rewrites vdirs and the state file, and two of them running at once +//! would interleave those writes — the timer firing while someone runs `sync` by +//! hand is the ordinary way that happens. `flock` is used rather than a pid file +//! because the kernel releases it when the process ends, however it ends, so a +//! crash cannot leave a lock behind for someone to clear by hand. + +use std::fs::{File, OpenOptions}; +use std::io::{Read, Seek, SeekFrom, Write}; +use std::path::{Path, PathBuf}; + +use rustix::fs::{FlockOperation, flock}; +use thiserror::Error; + +const FILE_NAME: &str = "lock"; + +#[derive(Debug, Error)] +pub enum LockError { + #[error("could not open {path}: {source}")] + Open { + path: PathBuf, + #[source] + source: std::io::Error, + }, + #[error( + "another calcalist is already running (pid {pid}); it holds {path}. \ + Wait for it to finish rather than running two cycles at once." + )] + Held { pid: String, path: PathBuf }, + #[error("could not lock {path}: {source}")] + Failed { + path: PathBuf, + #[source] + source: std::io::Error, + }, +} + +/// An exclusive claim on the state directory, released when dropped. +/// +/// The lock lives in the open file descriptor, so there is nothing to undo: it +/// goes when the `File` closes. The file itself is deliberately left behind — +/// unlinking it would let a waiter take a lock on an inode nobody else can see. +#[derive(Debug)] +pub struct Lock { + _file: File, +} + +pub fn path(state_dir: &Path) -> PathBuf { + state_dir.join(FILE_NAME) +} + +/// Claims the state directory, or reports who already has it. +pub fn acquire(state_dir: &Path) -> Result { + let path = path(state_dir); + if let Some(parent) = path.parent() { + std::fs::create_dir_all(parent).map_err(|source| LockError::Open { + path: path.clone(), + source, + })?; + } + let mut file = OpenOptions::new() + .read(true) + .write(true) + .create(true) + .truncate(false) + .open(&path) + .map_err(|source| LockError::Open { + path: path.clone(), + source, + })?; + + if let Err(error) = flock(&file, FlockOperation::NonBlockingLockExclusive) { + if error.kind() == std::io::ErrorKind::WouldBlock { + // Whoever holds it is alive: the kernel would have released the lock + // otherwise. So the pid in the file is worth reporting. + return Err(LockError::Held { + pid: holder(&mut file), + path, + }); + } + return Err(LockError::Failed { + path, + source: error.into(), + }); + } + + // Record who holds it, for the benefit of whoever is refused next. + let _ = file.set_len(0); + let _ = file.seek(SeekFrom::Start(0)); + let _ = write!(file, "{}", std::process::id()); + let _ = file.flush(); + Ok(Lock { _file: file }) +} + +/// The pid recorded in the lock file, or a placeholder when it says nothing. +fn holder(file: &mut File) -> String { + let mut recorded = String::new(); + if file.seek(SeekFrom::Start(0)).is_ok() + && file.read_to_string(&mut recorded).is_ok() + && !recorded.trim().is_empty() + { + return recorded.trim().to_string(); + } + "unknown".to_string() +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn a_second_claim_is_refused_while_the_first_is_held() { + let dir = tempfile::tempdir().expect("temp"); + let first = acquire(dir.path()).expect("the first claim should succeed"); + + let error = acquire(dir.path()).expect_err("the second should be refused"); + assert!(matches!(error, LockError::Held { .. }), "{error:?}"); + // The refusal has to name the holder, or it is not actionable. + assert!( + error.to_string().contains(&std::process::id().to_string()), + "{error}" + ); + + drop(first); + acquire(dir.path()).expect("the lock should be free again"); + } + + /// The file has to survive, so the next run locks the same inode. + #[test] + fn the_lock_file_outlives_the_lock() { + let dir = tempfile::tempdir().expect("temp"); + drop(acquire(dir.path()).expect("claim")); + assert!(path(dir.path()).exists()); + } + + #[test] + fn a_missing_state_directory_is_created() { + let dir = tempfile::tempdir().expect("temp"); + let nested = dir.path().join("not").join("there"); + acquire(&nested).expect("claim"); + assert!(path(&nested).exists()); + } +} diff --git a/src/main.rs b/src/main.rs index 1aee63a..8a7bd9a 100644 --- a/src/main.rs +++ b/src/main.rs @@ -5,6 +5,7 @@ mod config; mod doctor; mod google; mod ical; +mod lock; mod mirror; mod paths; mod pimsync; @@ -12,6 +13,7 @@ mod provenance; mod reconcile; mod retarget; mod state; +mod status; mod sync; mod vdir; @@ -30,10 +32,27 @@ fn main() -> ExitCode { command: Command::Doctor, .. } => run_doctor(cli), + cli @ Cli { + command: Command::Status, + .. + } => run_status(cli), cli @ Cli { command: Command::Sync { dry_run, force }, .. } => run_sync(cli, *dry_run, *force), + cli @ Cli { + command: Command::Prune { force }, + .. + } => run_prune(cli, *force), + Cli { + command: + Command::Discover { + url, + username, + secret_command, + }, + .. + } => run_discover(url, username, secret_command.as_deref()), cli @ Cli { command: Command::Google { @@ -51,7 +70,6 @@ fn main() -> ExitCode { }, .. } => run_retarget(cli, id, to, *purge_old), - Cli { command, .. } => unimplemented(command), } } @@ -76,15 +94,128 @@ fn run_sync(cli: &Cli, dry_run: bool, force: bool) -> ExitCode { Ok(dir) => dir, Err(error) => return fail(&error), }; + // Held for the whole cycle: a timer firing mid-run would otherwise interleave + // its writes with this one over the same vdirs. + let _lock = match lock::acquire(&state_dir) { + Ok(lock) => lock, + Err(error) => return fail(&error), + }; match sync::run(&config, &state_dir, dry_run, force) { Ok(report) => { print_report(&report); - ExitCode::SUCCESS + // A cycle that could not reach an endpoint did real work, but saying + // so only in passing would let a lapsed token go unnoticed until the + // calendars had drifted a long way apart. + if report.is_complete() { + ExitCode::SUCCESS + } else { + ExitCode::FAILURE + } } Err(error) => fail(&error), } } +/// Lists a server's calendars as configuration ready to paste. +/// +/// The URL of a CalDAV calendar is the most error-prone thing in the config: it +/// must name the collection exactly, and providers rarely show it. Asking the +/// server is more reliable than reading a web interface. +fn run_discover(url: &str, username: &str, secret_command: Option<&str>) -> ExitCode { + let state_dir = match paths::state_dir() { + Ok(dir) => dir, + Err(error) => return fail(&error), + }; + let found = match pimsync::discover(&state_dir, url, username, secret_command) { + Ok(found) => found, + Err(error) => return fail(&error), + }; + if found.is_empty() { + eprintln!("calcalist: {url} offered no calendars"); + return ExitCode::FAILURE; + } + + let base = url.split_once("://").and_then(|(scheme, rest)| { + rest.split_once('/') + .map(|(authority, _)| format!("{scheme}://{authority}")) + }); + println!( + "# {} calendar(s) found. Paste what you want into calcalist.toml.", + found.len() + ); + // Some servers name calendars with opaque ids — Posteo's look like + // `cydhlw` — and the id is what an event carries after `@` to be routed + // here, so it is worth renaming before it ends up in anyone's notes. + println!("# Ids come from the URL. Rename them: an id is what you type after `@`."); + for collection in &found { + let full = match &base { + Some(base) => format!("{base}{}", collection.href), + // pimsync reported something that is not a path; show it as it came + // rather than gluing it onto an origin and inventing a URL. + None => collection.href.clone(), + }; + println!("\n[[endpoint]]"); + println!("id = {}", quoted(&collection.id)); + println!("type = \"caldav\""); + println!("url = {}", quoted(&full)); + println!("username = {}", quoted(username)); + match secret_command { + Some(command) => println!("secret_command = {}", quoted(command)), + None => println!("# secret_command = \"secret-tool lookup ...\""), + } + } + ExitCode::SUCCESS +} + +/// A TOML basic string, so a value containing quotes or backslashes survives. +fn quoted(value: &str) -> String { + format!("\"{}\"", value.replace('\\', "\\\\").replace('"', "\\\"")) +} + +/// Removes local mirrors of endpoints the configuration no longer names. +fn run_prune(cli: &Cli, force: bool) -> ExitCode { + let (config, _) = match Config::load(cli.config.as_deref()) { + Ok(loaded) => loaded, + Err(error) => return fail(&error), + }; + let state_dir = match paths::state_dir() { + Ok(dir) => dir, + Err(error) => return fail(&error), + }; + // Only the destructive half needs the lock; listing reads nothing a cycle + // could be part-way through changing. + let _lock = if force { + match lock::acquire(&state_dir) { + Ok(lock) => Some(lock), + Err(error) => return fail(&error), + } + } else { + None + }; + let orphans = sync::orphan_vdirs(&config, &state_dir); + if orphans.is_empty() { + println!("nothing to prune: every local mirror belongs to a configured endpoint"); + return ExitCode::SUCCESS; + } + for orphan in &orphans { + if !force { + println!("would remove {}", orphan.display()); + continue; + } + match std::fs::remove_dir_all(orphan) { + Ok(()) => println!("removed {}", orphan.display()), + Err(error) => { + eprintln!("calcalist: could not remove {}: {error}", orphan.display()); + return ExitCode::FAILURE; + } + } + } + if !force { + println!("re-run with --force to remove them"); + } + ExitCode::SUCCESS +} + /// Authorises one Google endpoint, storing a refresh token for later cycles. fn run_google_login(cli: &Cli, endpoint_id: &str) -> ExitCode { let (config, _) = match Config::load(cli.config.as_deref()) { @@ -105,6 +236,25 @@ fn run_google_login(cli: &Cli, endpoint_id: &str) -> ExitCode { } } +/// Shows what is configured, and what an event's notes may say to steer it. +fn run_status(cli: &Cli) -> ExitCode { + let (config, _) = match Config::load(cli.config.as_deref()) { + Ok(loaded) => loaded, + Err(error) => return fail(&error), + }; + let state_dir = match paths::state_dir() { + Ok(dir) => dir, + Err(error) => return fail(&error), + }; + match status::render(&config, &state_dir) { + Ok(report) => { + print!("{report}"); + ExitCode::SUCCESS + } + Err(error) => fail(&error), + } +} + /// Moves an aggregate to a different target endpoint, deliberately. fn run_retarget(cli: &Cli, id: &str, to: &str, purge_old: bool) -> ExitCode { let (config, _) = match Config::load(cli.config.as_deref()) { @@ -115,6 +265,10 @@ fn run_retarget(cli: &Cli, id: &str, to: &str, purge_old: bool) -> ExitCode { Ok(dir) => dir, Err(error) => return fail(&error), }; + let _lock = match lock::acquire(&state_dir) { + Ok(lock) => lock, + Err(error) => return fail(&error), + }; match retarget::retarget(&config, &state_dir, id, to, purge_old) { Ok(outcome) => { println!( @@ -126,6 +280,12 @@ fn run_retarget(cli: &Cli, id: &str, to: &str, purge_old: bool) -> ExitCode { outcome.routed_first, outcome.purged, ); + if outcome.carried > 0 { + println!( + " {} event(s) belonging to the aggregate itself were moved across", + outcome.carried + ); + } if outcome.purged == 0 && !purge_old { println!( " the previous target still holds this aggregate's events; re-run with --purge-old to remove them" @@ -138,6 +298,27 @@ fn run_retarget(cli: &Cli, id: &str, to: &str, purge_old: bool) -> ExitCode { } fn print_report(report: &Report) { + for failure in &report.failures { + println!("{}: unavailable — {}", failure.endpoint, failure.reason); + } + for blocked in &report.blocked { + println!( + "{}: left alone; {} could not be reached this cycle", + blocked.id, + blocked + .endpoints + .iter() + .map(|id| format!("`{id}`")) + .collect::>() + .join(", ") + ); + } + for orphan in &report.orphan_vdirs { + println!( + "note: {} mirrors an endpoint that is no longer configured; `calcalist prune` removes it", + orphan.display() + ); + } for google in &report.google { let resync = if google.full_resync { " (Google rejected the sync cursor, so everything was refetched)" @@ -153,9 +334,15 @@ fn print_report(report: &Report) { google.updated_remotely, google.deleted_remotely, ); + if google.exceptions_applied > 0 { + println!( + " {} recurrence override(s) applied to their instances", + google.exceptions_applied + ); + } if google.exceptions_skipped > 0 { println!( - " {} recurring series had exceptions that were not pushed; Google models those as separate events against an existing series", + " {} recurrence override(s) matched no occurrence of their series and were left alone", google.exceptions_skipped ); } @@ -173,6 +360,12 @@ fn print_report(report: &Report) { aggregate.deleted_from_aggregate, aggregate.deleted_from_sources, ); + if aggregate.kept_local > 0 { + println!( + " {} event(s) kept local: they belong to the aggregate itself, not to a source", + aggregate.kept_local + ); + } for conflict in &aggregate.conflicts { println!( " conflict: {} changed in both places; kept the version from `{}`", @@ -184,15 +377,12 @@ fn print_report(report: &Report) { } } if report.dry_run { - println!("(dry run — nothing was written)"); + println!("(dry run — pulled into a throwaway copy; nothing was written)"); } } fn describe(skipped: &reconcile::Skipped) -> String { match skipped { - reconcile::Skipped::NoSink { aggregate_uid } => format!( - "`{aggregate_uid}` was created in the aggregate, but no default_sink is set to route it to" - ), reconcile::Skipped::ReadOnlySource { aggregate_uid, source_id, @@ -202,8 +392,14 @@ fn describe(skipped: &reconcile::Skipped) -> String { reconcile::Skipped::UnknownSink { aggregate_uid, requested, + available, } => format!( - "`{aggregate_uid}` asked to be filed under `{requested}`, which is not a writable source of this aggregate; it was left where it is" + "`{aggregate_uid}` asked to be filed under `@{requested}`, which is not a writable source of this aggregate. It was left where it is. Try one of: {}", + available + .iter() + .map(|name| format!("@{name}")) + .collect::>() + .join(", ") ), } } @@ -218,17 +414,3 @@ fn fail(error: &dyn std::error::Error) -> ExitCode { } ExitCode::FAILURE } - -/// M1 is still landing: `serve`, `status`, `google` and `aggregate` come later. -fn unimplemented(command: &Command) -> ExitCode { - let name = match command { - Command::Sync { .. } => "sync", - Command::Status => "status", - Command::Serve { .. } => "serve", - Command::Google { .. } => "google", - Command::Aggregate { .. } => "aggregate", - Command::Doctor => "doctor", - }; - eprintln!("calcalist: `{name}` is not implemented yet"); - ExitCode::from(2) -} diff --git a/src/mirror.rs b/src/mirror.rs index f846344..1bf3f61 100644 --- a/src/mirror.rs +++ b/src/mirror.rs @@ -30,6 +30,15 @@ const OWN_PROPERTIES: &[&str] = &[SOURCE_PROPERTY, ORIGIN_UID_PROPERTY, ATTENDEE /// Live scheduling properties, whose presence is what makes a server send mail. const SCHEDULING_PROPERTIES: &[&str] = &["ATTENDEE", "ORGANIZER"]; +/// The marker that keeps an event in the aggregate rather than filing it under +/// a source. Reserved: an endpoint may not be given this id. +pub const LOCAL_SINK: &str = "local"; + +/// Whether a routing hint asks for the event to stay where it is. +pub fn is_local_sink(hint: &str) -> bool { + hint.eq_ignore_ascii_case(LOCAL_SINK) +} + /// Builds the aggregate copy of a source event. pub fn to_aggregate( source: &Calendar, @@ -52,6 +61,24 @@ pub fn to_aggregate( mirrored } +/// Applies a target's scheduling rule to an event that is not a mirror. +/// +/// An event living only in the aggregate still has to obey the rule that writes +/// to an aggregate never emit scheduling mail — moving one between calendars is +/// as capable of mailing a guest list as mirroring is. It carries no provenance, +/// having none, so this is the demotion alone. +pub fn make_inert( + calendar: &Calendar, + owner: Option<&str>, + suppression: SchedulingSuppression, +) -> Calendar { + let mut inert = calendar.clone(); + if suppression == SchedulingSuppression::None { + demote_attendees(&mut inert, owner); + } + inert +} + /// Removes the live guest list, keeping its information in inert form. fn demote_attendees(calendar: &mut Calendar, owner: Option<&str>) { let guests: Vec = calendar diff --git a/src/pimsync.rs b/src/pimsync.rs index 3af46f9..1e1809b 100644 --- a/src/pimsync.rs +++ b/src/pimsync.rs @@ -106,16 +106,31 @@ pub fn config_path(state_dir: &Path) -> PathBuf { state_dir.join("pimsync.conf") } -fn status_path(state_dir: &Path) -> PathBuf { +pub(crate) fn status_path(state_dir: &Path) -> PathBuf { state_dir.join("pimsync-status") } +/// Which way a generated configuration lets items travel. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Direction { + /// The ordinary cycle: remote and local converge on each other. + Both, + /// Remote to local only, for `--dry-run`. pimsync writes nothing to any + /// server in this mode — but it does delete local items the remote has + /// never seen, so it is only ever pointed at a throwaway copy of the vdirs. + PullOnly, +} + /// Builds a pimsync configuration covering every CalDAV and WebCal endpoint. /// /// Google endpoints are absent by design: pimsync has no REST storage, and its /// CalDAV storage speaks only HTTP Basic auth, which Google's endpoint has /// rejected since March 2025. calcalist syncs those itself. -pub fn generate(config: &Config, state_dir: &Path) -> Result { +pub fn generate_for( + config: &Config, + state_dir: &Path, + direction: Direction, +) -> Result { let mut out = String::new(); out.push_str("# Generated by calcalist. Edits here are overwritten on every sync.\n\n"); out.push_str(&format!( @@ -129,7 +144,7 @@ pub fn generate(config: &Config, state_dir: &Path) -> Result out.push_str(&caldav_pair(endpoint)?), + EndpointKind::Caldav { .. } => out.push_str(&caldav_pair(endpoint, direction)?), EndpointKind::Webcal { url, url_command } => { out.push_str(&webcal_pair( endpoint, @@ -144,7 +159,7 @@ pub fn generate(config: &Config, state_dir: &Path) -> Result Result { +fn caldav_pair(endpoint: &Endpoint, direction: Direction) -> Result { let EndpointKind::Caldav { url, username, @@ -175,10 +190,15 @@ fn caldav_pair(endpoint: &Endpoint) -> Result { // The remote is storage_a, so a concurrent server-side change wins over a // local one calcalist has not yet reconciled. Nothing is lost: the next // cycle sees the server's version and decides properly. + let resolution = match direction { + // `one_way` never produces a conflict, so there is nothing to resolve. + Direction::PullOnly => "\tone_way\n", + Direction::Both => "\tconflict_resolution keep a\n", + }; block.push_str(&format!( "\npair {id} {{\n\tstorage_a {id}_remote\n\tstorage_b {LOCAL_STORAGE}\n\ \tcollection {{\n\t\talias {id}\n\t\thref_a {}\n\t\tid_b {id}\n\t}}\n\ - \tconflict_resolution keep a\n\ton_empty skip\n\ton_delete skip\n}}\n", + {resolution}\ton_empty skip\n\ton_delete skip\n}}\n", quote(&href) )); Ok(block) @@ -235,8 +255,16 @@ fn quote(value: &str) -> String { /// Writes the generated configuration, returning its path. pub fn write_config(config: &Config, state_dir: &Path) -> Result { + write_config_for(config, state_dir, Direction::Both) +} + +pub fn write_config_for( + config: &Config, + state_dir: &Path, + direction: Direction, +) -> Result { let path = config_path(state_dir); - let text = generate(config, state_dir)?; + let text = generate_for(config, state_dir, direction)?; fs::create_dir_all(status_path(state_dir)).map_err(|source| PimsyncError::Write { path: status_path(state_dir), source, @@ -248,6 +276,172 @@ pub fn write_config(config: &Config, state_dir: &Path) -> Result, +) -> Result, PimsyncError> { + let config_path = state_dir.join("discover.conf"); + let scratch = state_dir.join("discover-vdir"); + fs::create_dir_all(&scratch).map_err(|source| PimsyncError::Write { + path: scratch.clone(), + source, + })?; + fs::write( + &config_path, + probe_config(state_dir, &scratch, url, username, secret_command), + ) + .map_err(|source| PimsyncError::Write { + path: config_path.clone(), + source, + })?; + + let output = Command::new(BINARY) + .arg("-c") + .arg(&config_path) + .arg("discover") + .output() + .map_err(|error| { + if error.kind() == io::ErrorKind::NotFound { + PimsyncError::NotFound + } else { + PimsyncError::Spawn(error) + } + })?; + + // Everything, including the failures, comes back on stdout. + let text = String::from_utf8_lossy(&output.stdout).into_owned(); + let _ = fs::remove_dir_all(&scratch); + if !output.status.success() { + return Err(PimsyncError::Command { + command: "discover".to_string(), + message: strip_ansi(&text).trim().to_string(), + }); + } + Ok(parse_discovery(&text)) +} + +fn probe_config( + state_dir: &Path, + scratch: &Path, + url: &str, + username: &str, + secret_command: Option<&str>, +) -> String { + let password = match secret_command { + Some(command) => format!("\tpassword {{\n\t\tshell {command}\n\t}}\n"), + None => String::new(), + }; + format!( + "# Generated by `calcalist discover`. Rewritten on every run.\n\ + status_path {}\n\n\ + storage local {{\n\ttype vdir/icalendar\n\tpath {}\n\tfileext ics\n}}\n\n\ + storage {PROBE_REMOTE} {{\n\ttype caldav\n\turl {}\n\tdiscovery collections\n\ + \tusername {}\n{password}}}\n\n\ + pair probe {{\n\tstorage_a {PROBE_REMOTE}\n\tstorage_b local\n\ + \tcollections from a\n}}\n", + quote(&state_dir.join("discover-status").display().to_string()), + quote(&scratch.display().to_string()), + quote(url), + quote(username), + ) +} + +/// Reads the collections out of `pimsync discover` output. +/// +/// The format is not documented, so this was established against a real server +/// and is deliberately forgiving. The one thing it must get right: a pair has +/// two storages and pimsync reports both, so `Found` lines are only taken after +/// the heading naming the server. Otherwise the scratch directory's own +/// contents would be offered as if they were the user's calendars. +fn parse_discovery(output: &str) -> Vec { + let mut collections = Vec::new(); + let mut in_remote = false; + for line in output.lines() { + let line = strip_ansi(line); + let line = line.trim(); + if line.starts_with("==>") { + in_remote = line.contains(PROBE_REMOTE); + continue; + } + let Some(href) = line.strip_prefix("Found ") else { + continue; + }; + let href = href.trim(); + if !in_remote || href.is_empty() { + continue; + } + collections.push(Collection { + id: collection_id(href), + href: href.to_string(), + }); + } + collections +} + +/// A plausible endpoint id: the last non-empty path segment, restricted to +/// characters that are unambiguous in a configuration file. +fn collection_id(href: &str) -> String { + let segment = href + .rsplit('/') + .find(|segment| !segment.is_empty()) + .unwrap_or(href); + let id: String = segment + .chars() + .map(|ch| match ch { + 'a'..='z' | '0'..='9' | '-' | '_' => ch, + 'A'..='Z' => ch.to_ascii_lowercase(), + _ => '-', + }) + .collect(); + let id = id.trim_matches('-').to_string(); + if id.is_empty() { "calendar".into() } else { id } +} + +/// Removes ANSI colour, which pimsync emits even when its output is a pipe and +/// even under NO_COLOR. +fn strip_ansi(text: &str) -> String { + let mut out = String::with_capacity(text.len()); + let mut chars = text.chars(); + while let Some(ch) = chars.next() { + if ch != '\u{1b}' { + out.push(ch); + continue; + } + // Skip up to and including the terminating letter of the escape. + for next in chars.by_ref() { + if next.is_ascii_alphabetic() { + break; + } + } + } + out +} + /// Runs one pimsync cycle. /// /// Deliberately one-shot: `pimsync daemon` would write the same vdirs the @@ -327,7 +521,8 @@ url = "https://example.org/holidays.ics" fn generated() -> String { let config: Config = toml::from_str(CONFIG).expect("config should parse"); - generate(&config, Path::new("/var/state/calcalist")).expect("generation should succeed") + generate_for(&config, Path::new("/var/state/calcalist"), Direction::Both) + .expect("generation should succeed") } #[test] @@ -404,7 +599,8 @@ url_command = "secret-tool lookup service calcalist account gcal-ics" "#, ) .expect("config should parse"); - let text = generate(&config, Path::new("/var/state/calcalist")).expect("generate"); + let text = generate_for(&config, Path::new("/var/state/calcalist"), Direction::Both) + .expect("generate"); assert!(text.contains("url {")); assert!(text.contains("shell secret-tool lookup service calcalist account gcal-ics")); @@ -488,4 +684,93 @@ url_command = "secret-tool lookup service calcalist account gcal-ics" assert!(supported.is_supported()); assert!(!next_series.is_supported()); } + + /// Established against a real server: everything arrives on stdout, the + /// headings are colour-coded even through a pipe, and both storages of the + /// pair are reported. + const DISCOVERY: &str = "\u{1b}[1m\u{1b}[32m==>\u{1b}[0m Discovering collections in local\u{2026}\n\ + Found leftover-one\n\ + Found leftover-two\n\ + \u{1b}[1m\u{1b}[32m==>\u{1b}[0m Discovering collections in probe_remote\u{2026}\n\ + Found /calendars/me/work/\n\ + Found /calendars/me/Private Stuff/\n"; + + /// The trap: pimsync reports the scratch directory's contents too, and + /// offering those as the user's calendars would be nonsense. + #[test] + fn only_the_servers_collections_are_taken() { + let found = parse_discovery(DISCOVERY); + + assert_eq!( + found.iter().map(|c| c.href.as_str()).collect::>(), + vec!["/calendars/me/work/", "/calendars/me/Private Stuff/"] + ); + } + + #[test] + fn an_id_is_taken_from_the_last_segment() { + let found = parse_discovery(DISCOVERY); + assert_eq!(found[0].id, "work"); + // Spaces and capitals cannot appear in an id used as a routing marker. + assert_eq!(found[1].id, "private-stuff"); + } + + #[test] + fn a_server_offering_nothing_yields_nothing() { + let empty = "\u{1b}[1m==>\u{1b}[0m Discovering collections in probe_remote\u{2026}\n"; + assert!(parse_discovery(empty).is_empty()); + } + + /// A failure is reported in place of the collections, so nothing is found. + #[test] + fn an_unreachable_server_yields_nothing() { + let failed = "\u{1b}[1m==>\u{1b}[0m Discovering collections in probe_remote\u{2026}\n\ + input/output error: client error executing request: client error (Connect)\n"; + assert!(parse_discovery(failed).is_empty()); + } + + #[test] + fn colour_is_stripped_from_headings() { + assert_eq!( + strip_ansi("\u{1b}[1m\u{1b}[32m==>\u{1b}[0m Discovering"), + "==> Discovering" + ); + } + + /// The probe must never be mistaken for the real configuration, and must + /// name the storage the parser anchors on. + #[test] + fn the_probe_config_is_separate_and_parseable() { + let text = probe_config( + Path::new("/var/state/calcalist"), + Path::new("/var/state/calcalist/discover-vdir"), + "https://dav.example.com/calendars/me/", + "me@example.com", + Some("secret-tool lookup service dav"), + ); + assert!( + text.contains(&format!("storage {PROBE_REMOTE} {{")), + "{text}" + ); + assert!(text.contains("collections from a"), "{text}"); + assert!(text.contains("discovery collections"), "{text}"); + assert!( + text.contains("shell secret-tool lookup service dav"), + "{text}" + ); + // Its status path is its own, so a probe cannot disturb a real sync. + assert!(text.contains("discover-status"), "{text}"); + } + + #[test] + fn a_server_needing_no_password_gets_no_password_directive() { + let text = probe_config( + Path::new("/var/state/calcalist"), + Path::new("/tmp/scratch"), + "https://dav.example.com/calendars/me/", + "me", + None, + ); + assert!(!text.contains("password"), "{text}"); + } } diff --git a/src/reconcile.rs b/src/reconcile.rs index 6ac7a07..0d339ef 100644 --- a/src/reconcile.rs +++ b/src/reconcile.rs @@ -83,9 +83,6 @@ pub struct Conflict { /// Something that could not be done, and why. #[derive(Debug, Clone, PartialEq, Eq)] pub enum Skipped { - /// A user created an event in the aggregate, but no sink is configured to - /// route it to, and guessing one would put it in the wrong calendar. - NoSink { aggregate_uid: String }, /// An edit was made to a mirror of a read-only feed, which cannot accept it. ReadOnlySource { aggregate_uid: String, @@ -93,10 +90,12 @@ pub enum Skipped { }, /// The event asked to be filed under a source that cannot take it. Refused /// rather than sent to the default, since a typo should not quietly put the - /// event in the wrong calendar. + /// event in the wrong calendar. The valid names travel with it, so the report + /// can say what would have worked. UnknownSink { aggregate_uid: String, requested: String, + available: Vec, }, } @@ -107,6 +106,9 @@ pub struct Outcome { pub links: BTreeMap, pub conflicts: Vec, pub skipped: Vec, + /// Events left in the aggregate rather than filed under a source. Not a + /// failure: an aggregate is also a calendar, and these are its own events. + pub kept_local: Vec, } #[derive(Debug, Error, PartialEq)] @@ -138,6 +140,7 @@ pub fn reconcile( links: BTreeMap::new(), conflicts: Vec::new(), skipped: Vec::new(), + kept_local: Vec::new(), }; let mut live: HashSet = HashSet::new(); let mut deletions = 0usize; @@ -358,21 +361,38 @@ fn route_new_events( // An event may name the source it belongs in; otherwise the aggregate's // configured sink takes it. let requested = mirror::routing_hint(&item.calendar); + // `@local` keeps the event where it was written. Unlike every other + // marker it is deliberately not stripped: the others have done their + // job once the event reaches its source, while this one has to survive + // to make the same decision on every later cycle. + if requested.as_deref().is_some_and(mirror::is_local_sink) { + outcome.kept_local.push(uid.clone()); + continue; + } let wanted = requested.as_deref().or(policy.default_sink); + // Case-insensitively: the name is typed into a calendar app by hand, and + // capitalisation is not worth failing over. let sink = wanted - .and_then(|sink| sources.iter().find(|source| source.id == sink)) + .and_then(|sink| { + sources + .iter() + .find(|source| source.id.eq_ignore_ascii_case(sink)) + }) .filter(|sink| sink.writable); let Some(sink) = sink else { - outcome.skipped.push(match requested { - Some(requested) => Skipped::UnknownSink { + match requested { + Some(requested) => outcome.skipped.push(Skipped::UnknownSink { aggregate_uid: uid.clone(), requested, - }, - None => Skipped::NoSink { - aggregate_uid: uid.clone(), - }, - }); + available: writable_sinks(sources), + }), + // Nothing says where this event should go, so it stays where it + // was written. With no default_sink configured that is the + // mode, not an omission — the aggregate is a calendar the user + // can also write in directly. + None => outcome.kept_local.push(uid.clone()), + } continue; }; @@ -431,6 +451,15 @@ fn route_new_events( /// that failed to populate, which shows up as a *bulk* disappearance. const ALWAYS_ALLOWED_DELETIONS: usize = 3; +/// The sources of this aggregate that an event could actually be filed under. +fn writable_sinks(sources: &[SourceView<'_>]) -> Vec { + sources + .iter() + .filter(|source| source.writable) + .map(|source| source.id.to_string()) + .collect() +} + /// Refuses a cycle that would delete an implausible share of the aggregate. /// /// A source vdir that failed to populate looks exactly like one whose events were @@ -826,6 +855,7 @@ mod tests { vec![Skipped::UnknownSink { aggregate_uid: "hand@phone".into(), requested: "nosuchplace".into(), + available: vec![SRC.to_string()], }] ); } @@ -860,7 +890,9 @@ mod tests { ); } - /// Guessing a sink would put the event in the wrong calendar, so refuse. + /// With nowhere configured to file it, the event stays where it was + /// written. Guessing a sink would put it in the wrong calendar, and this is + /// how an aggregate holds events of its own. #[test] fn without_a_sink_a_user_created_event_is_left_alone() { let sources = map(vec![]); @@ -868,12 +900,30 @@ mod tests { let outcome = reconcile(policy(), &[view(&sources, true)], &aggregate, None).expect("ok"); assert!(outcome.actions.is_empty()); - assert_eq!( - outcome.skipped, - vec![Skipped::NoSink { - aggregate_uid: "hand-written@phone".into() - }] - ); + assert!(outcome.skipped.is_empty(), "{:?}", outcome.skipped); + assert_eq!(outcome.kept_local, vec!["hand-written@phone".to_string()]); + } + + /// `@local` opts one event out even where a default sink would take it. + #[test] + fn the_local_marker_keeps_an_event_out_of_every_source() { + let sources = map(vec![]); + let mut tagged = item("hand-written@phone", "Dentist"); + tagged + .calendar + .set_property("VEVENT", "DESCRIPTION", "Bring the referral\\n\\n@local"); + tagged.hash = tagged.calendar.content_hash(); + let aggregate = map(vec![tagged]); + + let policy = Policy { + default_sink: Some(SRC), + ..policy() + }; + let outcome = reconcile(policy, &[view(&sources, true)], &aggregate, None).expect("ok"); + + assert!(outcome.actions.is_empty(), "{:?}", outcome.actions); + assert!(outcome.skipped.is_empty(), "{:?}", outcome.skipped); + assert_eq!(outcome.kept_local, vec!["hand-written@phone".to_string()]); } /// Our own mirror appearing inside a source must not be mirrored again. diff --git a/src/retarget.rs b/src/retarget.rs index 3b1f54c..0cbd347 100644 --- a/src/retarget.rs +++ b/src/retarget.rs @@ -55,6 +55,8 @@ pub struct Outcome { /// Events flushed to a sink before the move, which existed nowhere else. pub routed_first: usize, pub materialised: usize, + /// Events belonging to the aggregate itself, moved rather than re-derived. + pub carried: usize, pub purged: usize, } @@ -117,7 +119,14 @@ pub fn retarget( false, )?; - let materialised = materialise(config, aggregate, new_target, state_dir, &mut state)?; + let (materialised, carried) = materialise( + config, + aggregate, + &old_target, + new_target, + state_dir, + &mut state, + )?; let purged = if purge_old { purge(aggregate_id, &old_target, state_dir, &state)? } else { @@ -138,6 +147,7 @@ pub fn retarget( to: new_target.id.clone(), routed_first: settled.written_back, materialised, + carried, purged, }) } @@ -150,10 +160,11 @@ pub fn retarget( fn materialise( config: &Config, aggregate: &crate::config::Aggregate, + old_target: &Endpoint, new_target: &Endpoint, state_dir: &Path, state: &mut State, -) -> Result { +) -> Result<(usize, usize), RetargetError> { let new_dir = vdir_path(state_dir, &new_target.id); let existing = vdir::read(&new_dir)?; let suppression = new_target.kind.scheduling_suppression(); @@ -188,10 +199,30 @@ fn materialise( } written += 1; } + + // Events created in the aggregate and never filed under a source exist only + // on the old target, so there is nothing to re-derive them from: they are + // moved rather than rebuilt. Without this they would sit on the calendar + // being left behind while everything around them moved on. + // + // The new target's scheduling rule still applies. This is a write to an + // aggregate like any other, and a guest list carried live onto a server + // that schedules would mail everyone on it. + let mut carried = 0; + for (uid, item) in vdir::read(&vdir_path(state_dir, &old_target.id))? { + if provenance::is_derived(&uid) { + continue; + } + let inert = mirror::make_inert(&item.calendar, new_target.kind.owner(), suppression); + let replaces = existing.get(&uid).map(|held| held.path.clone()); + vdir::write(&new_dir, &uid, &inert, replaces.as_deref())?; + carried += 1; + } + state .aggregate_mut(&aggregate.id, target_of(new_target)) .links = updated; - Ok(written) + Ok((written, carried)) } /// Removes this aggregate's mirrors from the calendar it has left. @@ -280,6 +311,53 @@ default_sink = "src" ) } + /// An event belonging to the aggregate itself has to follow the move. + /// + /// There is nothing to re-derive it from, so before this it stayed on the + /// calendar being left behind while every mirrored event moved on — quietly, + /// which is the worst way for a calendar to lose an appointment. + #[test] + fn an_event_belonging_to_the_aggregate_is_carried_to_the_new_target() { + // No default_sink: events written into the aggregate stay there. + let config: Config = toml::from_str(&CONFIG.replace(r#"default_sink = "src""#, "")) + .expect("config should parse"); + let dir = seed(&config); + + let old = vdir_path(dir.path(), "agg1"); + fs::write( + old.join("dentist.ics"), + event("hand-written@phone", "Dentist"), + ) + .expect("write"); + settle(&config, dir.path()); + + let outcome = retarget(&config, dir.path(), "unified", "agg2", false).expect("retarget"); + + assert_eq!(outcome.carried, 1, "{outcome:?}"); + let moved = vdir::read(&vdir_path(dir.path(), "agg2")).expect("read"); + assert!( + moved.contains_key("hand-written@phone"), + "the aggregate's own event did not follow: {:?}", + moved.keys().collect::>() + ); + assert!( + moved["hand-written@phone"] + .calendar + .to_ics() + .contains("Dentist"), + "it arrived without its content" + ); + // And it is still not a mirror of anything, so nothing was invented for it. + let state = State::load(&dir.path().join(crate::state::FILE_NAME)).expect("state"); + assert!( + !state + .aggregate("unified") + .expect("aggregate") + .links + .contains_key("hand-written@phone") + ); + } + /// Reconciles one aggregate and records the result, without running a full /// cycle. A full cycle would contact Google for real, which these tests have /// no business doing — they are about what happens to local files. diff --git a/src/status.rs b/src/status.rs new file mode 100644 index 0000000..6932cb3 --- /dev/null +++ b/src/status.rs @@ -0,0 +1,169 @@ +//! `calcalist status` — what is configured, and what it has done. +//! +//! Its most practical job is naming the markers an event can carry. A routing +//! hint is typed by hand into a calendar app, and having to remember what was +//! written in a TOML file is no way to find out what it should say. + +use std::fmt::Write as _; +use std::path::Path; + +use crate::config::{Aggregate, Config}; +use crate::state::{State, StateError}; + +/// Renders the report shown by `calcalist status`. +pub fn render(config: &Config, state_dir: &Path) -> Result { + let state = State::load(&state_dir.join(crate::state::FILE_NAME))?; + let mut out = String::new(); + + out.push_str("endpoints\n"); + for endpoint in &config.endpoints { + let access = if endpoint.kind.is_writable() { + "read/write" + } else { + "read-only" + }; + let _ = writeln!( + out, + " {:<18} {:<8} {access}", + endpoint.id, + endpoint.kind.kind_name() + ); + } + + for aggregate in &config.aggregates { + let _ = write!(out, "\naggregate `{}`\n", aggregate.id); + let _ = writeln!(out, " published to {}", aggregate.target); + let _ = writeln!(out, " sources {}", aggregate.sources.join(", ")); + + let tracked = state + .aggregate(&aggregate.id) + .map_or(0, |entry| entry.links.len()); + let _ = writeln!(out, " mirroring {tracked} event(s)"); + + describe_routing(&mut out, config, aggregate); + } + + if config.aggregates.is_empty() { + out.push_str("\nno aggregates configured\n"); + } + Ok(out) +} + +/// Explains where a newly created event goes, and how to send it elsewhere. +fn describe_routing(out: &mut String, config: &Config, aggregate: &Aggregate) { + let sinks: Vec<&str> = aggregate + .sources + .iter() + .filter(|id| { + config + .endpoint(id) + .is_some_and(|endpoint| endpoint.kind.is_writable()) + }) + .map(String::as_str) + .collect(); + + match &aggregate.default_sink { + Some(sink) => { + let _ = writeln!(out, " new events go to `{sink}` unless told otherwise"); + } + None => { + let _ = writeln!( + out, + " new events stay here: no default_sink is configured, so the aggregate keeps its own" + ); + } + } + if sinks.is_empty() { + let _ = writeln!(out, " none of its sources can be written to"); + return; + } + let _ = writeln!( + out, + " to choose, put one of these on a line of its own in the event's notes:" + ); + for sink in sinks { + let _ = writeln!(out, " @{sink}"); + } + let _ = writeln!( + out, + " @{} (keep the event here, in this calendar only)", + crate::mirror::LOCAL_SINK + ); +} + +#[cfg(test)] +mod tests { + use super::*; + + const CONFIG: &str = r#" +version = 1 + +[[endpoint]] +id = "gcal" +type = "google" +calendar_id = "a@example.com" +client_id = "x.apps.googleusercontent.com" + +[[endpoint]] +id = "posteo" +type = "caldav" +url = "https://dav.example.com/dav/calendars/me/work/" +username = "me@example.com" + +[[endpoint]] +id = "holidays" +type = "webcal" +url = "https://example.org/h.ics" + +[[aggregate]] +id = "unified" +target = "posteo" +sources = ["gcal", "holidays"] +default_sink = "gcal" +"#; + + fn report() -> String { + let config: Config = toml::from_str(CONFIG).expect("config"); + let dir = tempfile::tempdir().expect("temp dir"); + render(&config, dir.path()).expect("render") + } + + /// The whole point of the command: knowing what to type without opening the + /// configuration file. + #[test] + fn it_names_the_markers_that_would_work() { + let report = report(); + assert!(report.contains("@gcal"), "{report}"); + } + + /// A read-only feed cannot take an event, so offering it would mislead. + #[test] + fn it_does_not_offer_a_read_only_source_as_a_destination() { + let report = report(); + assert!(!report.contains("@holidays"), "{report}"); + assert!( + report.contains("holidays"), + "it should still be listed as a source" + ); + } + + #[test] + fn it_says_where_untagged_events_go() { + assert!(report().contains("new events go to `gcal`")); + } + + /// The escape hatch has to be as findable as the destinations. + #[test] + fn it_names_the_marker_that_keeps_an_event_local() { + assert!(report().contains("@local"), "{}", report()); + } + + #[test] + fn it_says_when_nothing_would_be_routed() { + let sinkless: Config = + toml::from_str(&CONFIG.replace(r#"default_sink = "gcal""#, "")).expect("config"); + let dir = tempfile::tempdir().expect("temp dir"); + let report = render(&sinkless, dir.path()).expect("render"); + assert!(report.contains("the aggregate keeps its own"), "{report}"); + } +} diff --git a/src/sync.rs b/src/sync.rs index b6e1f1d..fe491d4 100644 --- a/src/sync.rs +++ b/src/sync.rs @@ -5,7 +5,8 @@ //! out of those vdirs is the job of pimsync (CalDAV, WebCal) and the Google //! module, which bracket this step. -use std::collections::BTreeMap; +use std::collections::{BTreeMap, BTreeSet}; +use std::fs; use std::path::{Path, PathBuf}; use thiserror::Error; @@ -13,7 +14,7 @@ use thiserror::Error; use crate::config::{Aggregate, Config, Endpoint, EndpointKind}; use crate::google::api::{self, ApiError}; use crate::google::auth::{self, AuthError}; -use crate::pimsync::{self, PimsyncError}; +use crate::pimsync::{self, Direction, PimsyncError}; use crate::reconcile::{self, Action, Conflict, Policy, ReconcileError, Skipped, SourceView}; use crate::state::{State, Target}; use crate::vdir::{self, VdirError}; @@ -69,9 +70,37 @@ pub enum SyncError { pub struct Report { pub aggregates: Vec, pub google: Vec, + pub failures: Vec, + pub blocked: Vec, + pub orphan_vdirs: Vec, pub dry_run: bool, } +impl Report { + /// Whether the cycle covered everything it was asked to. + /// + /// A partial cycle is not a crash — the aggregates that could be reconciled + /// were — but it must not read as success either, or a lapsed token would go + /// unnoticed until the calendars had drifted a long way apart. + pub fn is_complete(&self) -> bool { + self.failures.is_empty() && self.blocked.is_empty() + } +} + +/// An endpoint that could not be reached this cycle. +#[derive(Debug)] +pub struct EndpointFailure { + pub endpoint: String, + pub reason: String, +} + +/// An aggregate left alone because an endpoint it depends on was unreachable. +#[derive(Debug)] +pub struct BlockedAggregate { + pub id: String, + pub endpoints: Vec, +} + #[derive(Debug)] pub struct GoogleReport { pub endpoint: String, @@ -81,6 +110,7 @@ pub struct GoogleReport { pub created_remotely: usize, pub updated_remotely: usize, pub deleted_remotely: usize, + pub exceptions_applied: usize, pub exceptions_skipped: usize, } @@ -93,6 +123,7 @@ pub struct AggregateReport { pub deleted_from_sources: usize, pub conflicts: Vec, pub skipped: Vec, + pub kept_local: usize, } /// Runs the local half of a cycle: reconcile every aggregate and apply the result. @@ -106,133 +137,257 @@ pub fn run( let mut state = State::load(&state_path)?; let mut report = Report { dry_run, + orphan_vdirs: orphan_vdirs(config, state_dir), ..Report::default() }; prepare_vdirs(config, state_dir)?; - // A dry run must leave the vdirs exactly as it found them, and pimsync - // writes to them, so the pulls and pushes are skipped along with everything - // else. What it reports is therefore what the *last* pull left behind. - let pimsync_config = if dry_run || !needs_pimsync(config) { - None + // A dry run pulls for real, so that what it reports is measured against the + // calendars as they are now — but into a throwaway copy of the local + // mirrors, and through a configuration that only ever reads from a server. + // Nothing outside that copy is written, and it is removed at the end. + let workspace = if dry_run { + Some(Workspace::open(state_dir)?) } else { - Some(pimsync::write_config(config, state_dir)?) + None + }; + let work_dir = workspace.as_ref().map_or(state_dir, Workspace::path); + + let pimsync_config = if needs_pimsync(config) { + let direction = if dry_run { + Direction::PullOnly + } else { + Direction::Both + }; + Some(pimsync::write_config_for(config, work_dir, direction)?) + } else { + None }; // Pull first, so the reconciler sees one consistent snapshot of every remote. if let Some(path) = &pimsync_config { pimsync::sync(path)?; } - if !dry_run { - report.google = pull_google(config, state_dir, &mut state)?; - } + let unavailable = pull_google(config, state_dir, work_dir, &mut state, &mut report); for aggregate in &config.aggregates { + // An aggregate whose target or any source could not be pulled is left + // untouched. Reconciling it would compare against a stale snapshot, and + // an endpoint that failed to answer looks exactly like an emptied one. + let endpoints: Vec = std::iter::once(&aggregate.target) + .chain(aggregate.sources.iter()) + .filter(|id| unavailable.contains(*id)) + .cloned() + .collect(); + if !endpoints.is_empty() { + report.blocked.push(BlockedAggregate { + id: aggregate.id.clone(), + endpoints, + }); + continue; + } + let target = resolve(config, aggregate, &aggregate.target)?; check_target_drift(&state, aggregate, target)?; report.aggregates.push(sync_aggregate( - config, aggregate, target, state_dir, &mut state, dry_run, force, + config, aggregate, target, work_dir, &mut state, dry_run, force, )?); } // Push what the reconciler decided out to the remotes. if !dry_run { - push_google(config, state_dir, &mut state, &mut report)?; - } - if let Some(path) = &pimsync_config { - pimsync::sync(path)?; - } - - if !dry_run { + push_google(config, state_dir, &mut state, &mut report, &unavailable); + if let Some(path) = &pimsync_config { + pimsync::sync(path)?; + } state.save(&state_path)?; } + if let Some(workspace) = workspace { + workspace.discard(); + } Ok(report) } -/// Brings every Google endpoint's vdir into step with its remote calendar. +/// A throwaway copy of the local mirrors, used by `--dry-run`. +/// +/// Pulling needs somewhere to put what it reads. Writing into the real mirrors +/// would leave them describing a moment the recorded state knows nothing about, +/// so a dry run gets its own copy and the originals are never opened for writing. +struct Workspace { + root: PathBuf, +} + +impl Workspace { + const DIRECTORY: &'static str = "dry-run"; + + fn open(state_dir: &Path) -> Result { + let root = state_dir.join(Self::DIRECTORY); + // A previous run that was interrupted may have left one behind. + if root.exists() { + fs::remove_dir_all(&root).map_err(|source| SyncError::Prepare { + path: root.clone(), + source, + })?; + } + copy_tree(&vdir_root(state_dir), &root.join("vdir"))?; + // pimsync's own record of what it has seen comes too, or it would treat + // every local item as new and plan to create it on the server. + copy_tree( + &pimsync::status_path(state_dir), + &pimsync::status_path(&root), + )?; + Ok(Workspace { root }) + } + + fn path(&self) -> &Path { + &self.root + } + + fn discard(self) { + let _ = fs::remove_dir_all(&self.root); + } +} + +/// Copies a directory tree, treating a missing source as an empty one. +fn copy_tree(from: &Path, to: &Path) -> Result<(), SyncError> { + fs::create_dir_all(to).map_err(|source| SyncError::Prepare { + path: to.to_path_buf(), + source, + })?; + let Ok(entries) = fs::read_dir(from) else { + return Ok(()); + }; + for entry in entries.flatten() { + let target = to.join(entry.file_name()); + let copied = if entry.path().is_dir() { + copy_tree(&entry.path(), &target) + } else { + fs::copy(entry.path(), &target) + .map(|_| ()) + .map_err(|source| SyncError::Prepare { + path: target, + source, + }) + }; + copied?; + } + Ok(()) +} + +/// Local mirrors belonging to endpoints the configuration no longer names. +/// +/// Removing an endpoint leaves its events sitting in the state directory, where +/// nothing manages them any more. They are reported rather than deleted: the +/// endpoint may have been renamed, or commented out for an afternoon. +pub fn orphan_vdirs(config: &Config, state_dir: &Path) -> Vec { + let Ok(entries) = fs::read_dir(vdir_root(state_dir)) else { + return Vec::new(); + }; + let mut orphans: Vec = entries + .flatten() + .filter(|entry| entry.path().is_dir()) + .filter(|entry| { + let name = entry.file_name().to_string_lossy().into_owned(); + config.endpoint(&name).is_none() + }) + .map(|entry| entry.path()) + .collect(); + orphans.sort(); + orphans +} + +/// Brings every Google endpoint's vdir into step with its remote calendar, +/// returning the endpoints that could not be reached. /// /// This is the Google equivalent of the pimsync pull: pimsync cannot reach /// Google at all, so calcalist does this leg itself. +/// +/// A failure here used to end the cycle. It no longer does: one lapsed token +/// would otherwise stop the CalDAV side too, which has nothing to do with it. +/// The affected endpoints are named, and only the aggregates that depend on +/// them stand down. fn pull_google( config: &Config, state_dir: &Path, + vdir_base: &Path, state: &mut State, -) -> Result, SyncError> { - let mut reports = Vec::new(); + report: &mut Report, +) -> BTreeSet { + let mut unavailable = BTreeSet::new(); for endpoint in &config.endpoints { let EndpointKind::Google { calendar_id, .. } = &endpoint.kind else { continue; }; - let credentials = auth::credentials_for(config, &endpoint.id).map_err(|source| { - SyncError::GoogleAuth { + let pulled = client_for(config, state_dir, endpoint, calendar_id).and_then(|client| { + let dir = vdir_path(vdir_base, &endpoint.id); + let entry = state.google.entry(endpoint.id.clone()).or_default(); + api::pull(&client, &dir, entry).map_err(|source| SyncError::Google { endpoint: endpoint.id.clone(), source, - } - })?; - let token = - auth::access_token(state_dir, &endpoint.id, &credentials).map_err(|source| { - SyncError::GoogleAuth { - endpoint: endpoint.id.clone(), - source, - } - })?; - - let client = api::Client::new(token, calendar_id.clone()); - let dir = vdir_path(state_dir, &endpoint.id); - let entry = state.google.entry(endpoint.id.clone()).or_default(); - let pulled = api::pull(&client, &dir, entry).map_err(|source| SyncError::Google { - endpoint: endpoint.id.clone(), - source, - })?; - reports.push(GoogleReport { - endpoint: endpoint.id.clone(), - written: pulled.written, - deleted: pulled.deleted, - full_resync: pulled.full_resync, - created_remotely: 0, - updated_remotely: 0, - deleted_remotely: 0, - exceptions_skipped: 0, + }) }); + match pulled { + Ok(pulled) => report.google.push(GoogleReport { + endpoint: endpoint.id.clone(), + written: pulled.written, + deleted: pulled.deleted, + full_resync: pulled.full_resync, + created_remotely: 0, + updated_remotely: 0, + deleted_remotely: 0, + exceptions_applied: 0, + exceptions_skipped: 0, + }), + Err(error) => { + unavailable.insert(endpoint.id.clone()); + report.failures.push(EndpointFailure { + endpoint: endpoint.id.clone(), + reason: chain(&error), + }); + } + } } - Ok(reports) + unavailable } /// Sends each Google endpoint's local changes up to its calendar. /// /// Runs after reconciliation, so what goes up is what the reconciler decided. +/// An endpoint whose pull failed is skipped: its vdir describes some earlier +/// moment, and pushing that would undo whatever has happened since. fn push_google( config: &Config, state_dir: &Path, state: &mut State, report: &mut Report, -) -> Result<(), SyncError> { + unavailable: &BTreeSet, +) { for endpoint in &config.endpoints { let EndpointKind::Google { calendar_id, .. } = &endpoint.kind else { continue; }; - let credentials = auth::credentials_for(config, &endpoint.id).map_err(|source| { - SyncError::GoogleAuth { + if unavailable.contains(&endpoint.id) { + continue; + } + let pushed = client_for(config, state_dir, endpoint, calendar_id).and_then(|client| { + let dir = vdir_path(state_dir, &endpoint.id); + let entry = state.google.entry(endpoint.id.clone()).or_default(); + api::push(&client, &dir, entry).map_err(|source| SyncError::Google { endpoint: endpoint.id.clone(), source, - } - })?; - let token = - auth::access_token(state_dir, &endpoint.id, &credentials).map_err(|source| { - SyncError::GoogleAuth { + }) + }); + let pushed = match pushed { + Ok(pushed) => pushed, + Err(error) => { + report.failures.push(EndpointFailure { endpoint: endpoint.id.clone(), - source, - } - })?; - - let client = api::Client::new(token, calendar_id.clone()); - let dir = vdir_path(state_dir, &endpoint.id); - let entry = state.google.entry(endpoint.id.clone()).or_default(); - let pushed = api::push(&client, &dir, entry).map_err(|source| SyncError::Google { - endpoint: endpoint.id.clone(), - source, - })?; + reason: chain(&error), + }); + continue; + } + }; if let Some(existing) = report .google @@ -242,10 +397,46 @@ fn push_google( existing.created_remotely = pushed.created; existing.updated_remotely = pushed.updated; existing.deleted_remotely = pushed.deleted; + existing.exceptions_applied = pushed.exceptions_applied; existing.exceptions_skipped = pushed.exceptions_skipped; } } - Ok(()) +} + +/// An authorised client for one Google endpoint. +fn client_for( + config: &Config, + state_dir: &Path, + endpoint: &Endpoint, + calendar_id: &str, +) -> Result { + let credentials = + auth::credentials_for(config, &endpoint.id).map_err(|source| SyncError::GoogleAuth { + endpoint: endpoint.id.clone(), + source, + })?; + let token = auth::access_token( + state_dir, + &endpoint.id, + endpoint.kind.google_account(), + &credentials, + ) + .map_err(|source| SyncError::GoogleAuth { + endpoint: endpoint.id.clone(), + source, + })?; + Ok(api::Client::new(token, calendar_id.to_string())) +} + +/// An error and its causes on one line, for a report that keeps going. +fn chain(error: &dyn std::error::Error) -> String { + let mut out = error.to_string(); + let mut source = error.source(); + while let Some(cause) = source { + out.push_str(&format!(": {cause}")); + source = cause.source(); + } + out } /// Whether any endpoint is one pimsync handles. A Google-only setup needs none. @@ -374,6 +565,7 @@ pub(crate) fn sync_aggregate( deleted_from_sources: 0, conflicts: outcome.conflicts, skipped: outcome.skipped, + kept_local: outcome.kept_local.len(), }; for action in &outcome.actions { @@ -705,4 +897,73 @@ sources = ["gcal"] uids(dir.path(), "work") ); } + + /// An endpoint calcalist cannot reach must not take the whole cycle with it. + /// Here every endpoint is Google and none is authorised, so all of them fail + /// — and the run still returns, naming what it could not do. + #[test] + fn an_unreachable_endpoint_blocks_only_what_depends_on_it() { + let dir = tempfile::tempdir().expect("temp"); + let config: Config = toml::from_str(&format!( + "{CONFIG}\n[[aggregate]]\nid = \"unified\"\ntarget = \"posteo\"\nsources = [\"work\"]\n" + )) + .expect("config"); + seed(dir.path(), "work", "a@example.com", "Standup"); + + let report = run(&config, dir.path(), false, false).expect("the cycle still returns"); + + assert!(!report.failures.is_empty(), "{report:?}"); + assert_eq!(report.blocked.len(), 1, "{report:?}"); + assert_eq!(report.blocked[0].id, "unified"); + assert!(report.blocked[0].endpoints.contains(&"work".to_string())); + assert!(!report.is_complete(), "a partial cycle is not a success"); + // Nothing was reconciled against the stale snapshot. + assert!(report.aggregates.is_empty(), "{report:?}"); + assert!(uids(dir.path(), "posteo").is_empty()); + } + + /// A mirror whose endpoint has gone from the configuration is reported, not + /// deleted: the endpoint may simply have been renamed. + #[test] + fn a_mirror_with_no_endpoint_is_reported_as_an_orphan() { + let dir = tempfile::tempdir().expect("temp"); + let config: Config = toml::from_str(CONFIG).expect("config"); + seed(dir.path(), "work", "a@example.com", "Standup"); + seed(dir.path(), "retired", "b@example.com", "Old"); + + let orphans = orphan_vdirs(&config, dir.path()); + + assert_eq!(orphans, vec![vdir_path(dir.path(), "retired")]); + } + + /// The dry run's copy is thrown away, and the mirrors it was made from are + /// left exactly as they were. + #[test] + fn a_dry_run_leaves_the_real_mirrors_alone() { + let dir = tempfile::tempdir().expect("temp"); + let config: Config = toml::from_str(&format!( + "{CONFIG}\n[[aggregate]]\nid = \"unified\"\ntarget = \"posteo\"\nsources = [\"work\"]\n" + )) + .expect("config"); + seed(dir.path(), "work", "a@example.com", "Standup"); + let before = fs::read_to_string(vdir_path(dir.path(), "work").join("a@example.com.ics")) + .expect("read"); + + let report = run(&config, dir.path(), true, false).expect("dry run"); + + assert!(report.dry_run); + assert!( + !dir.path().join("dry-run").exists(), + "the workspace should be gone" + ); + assert_eq!( + fs::read_to_string(vdir_path(dir.path(), "work").join("a@example.com.ics")) + .expect("read"), + before + ); + assert!( + !dir.path().join(crate::state::FILE_NAME).exists(), + "a dry run records nothing" + ); + } } diff --git a/systemd/calcalist.service b/systemd/calcalist.service new file mode 100644 index 0000000..caeadf5 --- /dev/null +++ b/systemd/calcalist.service @@ -0,0 +1,15 @@ +[Unit] +Description=Synchronise calendars with calcalist + +[Service] +Type=oneshot +ExecStart=%h/.local/bin/calcalist sync +NoNewPrivileges=true + +# Deliberately no ProtectSystem, PrivateTmp or IPC sandboxing. Credentials come +# from commands like `secret-tool`, which need the session keyring over D-Bus, +# and those options break them in ways that surface as unexplained auth +# failures rather than as anything pointing at the sandbox. + +# A cycle that could not reach an endpoint exits non-zero on purpose, so a +# lapsed token shows up as a failed unit rather than passing unnoticed. diff --git a/systemd/calcalist.timer b/systemd/calcalist.timer new file mode 100644 index 0000000..d88f322 --- /dev/null +++ b/systemd/calcalist.timer @@ -0,0 +1,15 @@ +[Unit] +Description=Synchronise calendars with calcalist every 15 minutes + +[Timer] +OnBootSec=2m +OnUnitActiveSec=15m +Persistent=true + +# Spreads requests instead of every installation calling Google on the quarter +# hour. Google's Calendar API quota is per project and enforced per minute, and +# its own guidance is to randomise timing rather than to burst. +RandomizedDelaySec=120 + +[Install] +WantedBy=timers.target diff --git a/tests/caldav.rs b/tests/caldav.rs new file mode 100644 index 0000000..d770839 --- /dev/null +++ b/tests/caldav.rs @@ -0,0 +1,499 @@ +//! End-to-end tests against a real CalDAV server and a real iCal feed. +//! +//! The unit tests establish that the reconciler decides correctly. These +//! establish that the decisions survive the round trip through pimsync and a +//! server that rewrites what it stores — which is where every bug found by hand +//! during M1 actually lived. +//! +//! Radicale and pimsync both come from devbox, so `devbox run check` has them. +//! Outside that shell the tests report what is missing and pass, rather than +//! failing for a reason that has nothing to do with the code. + +mod support; + +use support::{Calcalist, Feed, Radicale}; + +/// An event with guests, an organiser and an alarm — the combination the +/// scheduling rules are about. +const MEETING: &str = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:-//test//EN\r\n\ + BEGIN:VEVENT\r\nUID:meeting@work\r\nDTSTAMP:20260101T000000Z\r\n\ + DTSTART:20260910T090000Z\r\nDTEND:20260910T100000Z\r\nSUMMARY:Planning\r\n\ + ORGANIZER;CN=Chair:mailto:chair@example.com\r\n\ + ATTENDEE;CN=Guest;PARTSTAT=ACCEPTED:mailto:guest@example.com\r\n\ + BEGIN:VALARM\r\nACTION:DISPLAY\r\nTRIGGER:-PT15M\r\nDESCRIPTION:Soon\r\n\ + END:VALARM\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n"; + +/// An event written straight into the aggregate, belonging to no source. +const HAND_WRITTEN: &str = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:-//phone//EN\r\n\ + BEGIN:VEVENT\r\nUID:hand-written@phone\r\nDTSTAMP:20260101T000000Z\r\n\ + DTSTART:20260911T140000Z\r\nDTEND:20260911T150000Z\r\nSUMMARY:Dentist\r\n\ + END:VEVENT\r\nEND:VCALENDAR\r\n"; + +/// The same event, asking to be left where it was written. +const KEPT_LOCAL: &str = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:-//phone//EN\r\n\ + BEGIN:VEVENT\r\nUID:kept@phone\r\nDTSTAMP:20260101T000000Z\r\n\ + DTSTART:20260912T140000Z\r\nDTEND:20260912T150000Z\r\nSUMMARY:Haircut\r\n\ + DESCRIPTION:Around the corner\\n\\n@local\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n"; + +const HOLIDAY_FEED: &str = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:-//feed//EN\r\n\ + BEGIN:VEVENT\r\nUID:newyear@feed\r\nDTSTAMP:20260101T000000Z\r\n\ + DTSTART;VALUE=DATE:20260101\r\nDTEND;VALUE=DATE:20260102\r\n\ + SUMMARY:New Year\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n"; + +struct Fixture { + _root: tempfile::TempDir, + server: Radicale, + _feed: Feed, + calcalist: Calcalist, +} + +/// A CalDAV source, a read-only feed, and a CalDAV target to aggregate into. +fn fixture() -> Option { + let missing = support::missing_binaries(); + if !missing.is_empty() { + eprintln!( + "skipped: {} not on PATH; run under devbox", + missing.join(", ") + ); + return None; + } + + let root = tempfile::tempdir().expect("temp"); + let server = Radicale::start(root.path()); + server.create_calendar("work"); + server.create_calendar("unified"); + server.put_event("work", "meeting.ics", MEETING); + let feed = Feed::start(HOLIDAY_FEED.to_string()); + + let config = format!( + r#" +version = 1 + +[[endpoint]] +id = "work" +type = "caldav" +url = "{work}" +username = "{user}" +secret_command = "printf password" + +[[endpoint]] +id = "published" +type = "caldav" +url = "{unified}" +username = "{user}" +secret_command = "printf password" + +[[endpoint]] +id = "holidays" +type = "webcal" +url = "{feed}" + +[[aggregate]] +id = "everything" +target = "published" +sources = ["work", "holidays"] +default_sink = "work" +"#, + work = server.url("work"), + unified = server.url("unified"), + user = support::USER, + feed = feed.url(), + ); + + let calcalist = Calcalist::new(root.path(), &config); + Some(Fixture { + _root: root, + server, + _feed: feed, + calcalist, + }) +} + +/// Both sources reach the target, and running again changes nothing. +/// +/// Idempotence is the property that matters most: a cycle that is not a no-op +/// against unchanged input would rewrite every event on every run, and each +/// rewrite is a chance for the server to hand back something slightly different. +#[test] +fn a_cycle_converges_and_the_next_one_does_nothing() { + let Some(fixture) = fixture() else { return }; + + let first = fixture.calcalist.run(&["sync"]); + assert!( + first.succeeded(), + "first sync failed\n{}\n{}", + first.stdout, + first.stderr + ); + + let published = fixture.server.stored("unified"); + assert_eq!( + published.len(), + 2, + "both sources should arrive: {published:?}" + ); + let all = published.join("\n"); + assert!(all.contains("Planning"), "{all}"); + assert!(all.contains("New Year"), "{all}"); + // Provenance travels with each mirror, so the aggregate knows where its + // events came from without consulting the state file. + assert!(all.contains("X-CALCALIST-SOURCE:work"), "{all}"); + assert!(all.contains("X-CALCALIST-SOURCE:holidays"), "{all}"); + + let second = fixture.calcalist.run(&["sync"]); + assert!( + second.succeeded(), + "second sync failed\n{}\n{}", + second.stdout, + second.stderr + ); + assert!( + second.stdout.contains("everything: 0 mirrored"), + "the second cycle should be a no-op:\n{}", + second.stdout + ); + assert_eq!( + fixture.server.stored("unified"), + published, + "the second cycle rewrote the target" + ); +} + +/// The scheduling rule, checked against what actually reached the server. +/// +/// A CalDAV server has no portable way to be told not to send invitations, so +/// inertness is structural: the mirror carries the guest list as data and not as +/// live scheduling properties. The alarm is the deliberate exception — stripping +/// it would destroy every reminder in the one calendar the user subscribes to. +/// +/// Radicale implements no scheduling of its own, so this asserts on the bytes +/// stored rather than on a mail sink: with nothing to send mail, a quiet SMTP +/// port would prove nothing about the transform. +#[test] +fn a_mirror_carries_no_live_scheduling_properties() { + let Some(fixture) = fixture() else { return }; + + let run = fixture.calcalist.run(&["sync"]); + assert!( + run.succeeded(), + "sync failed\n{}\n{}", + run.stdout, + run.stderr + ); + + let mirror = fixture + .server + .stored("unified") + .into_iter() + .find(|item| item.contains("Planning")) + .expect("the meeting should have been mirrored"); + + for property in ["ATTENDEE;", "ATTENDEE:", "ORGANIZER;", "ORGANIZER:"] { + assert!( + !mirror + .lines() + .any(|line| line.trim_start().starts_with(property)), + "a live {property} reached the aggregate:\n{mirror}" + ); + } + // The guests are still there, as something nothing will act on. + assert!( + mirror.contains("X-CALCALIST-ATTENDEES"), + "the guest list was lost:\n{mirror}" + ); + assert!( + mirror.contains("guest@example.com"), + "the guest list was lost:\n{mirror}" + ); + assert!( + mirror.contains("BEGIN:VALARM") && mirror.contains("TRIGGER:-PT15M"), + "the alarm did not survive:\n{mirror}" + ); + + // The source keeps its scheduling properties: only the aggregate is inert. + let source = fixture + .server + .stored("work") + .into_iter() + .find(|item| item.contains("Planning")) + .expect("the meeting should still be in its source"); + assert!(source.contains("ATTENDEE"), "{source}"); + assert!(source.contains("ORGANIZER"), "{source}"); +} + +/// A dry run must reach the servers to be worth anything, and change nothing. +#[test] +fn a_dry_run_reports_without_touching_anything() { + let Some(fixture) = fixture() else { return }; + + let dry = fixture.calcalist.run(&["sync", "--dry-run"]); + assert!( + dry.succeeded(), + "dry run failed\n{}\n{}", + dry.stdout, + dry.stderr + ); + assert!( + dry.stdout.contains("everything: 2 mirrored"), + "the dry run should have seen both sources:\n{}", + dry.stdout + ); + assert!( + fixture.server.stored("unified").is_empty(), + "the dry run published events" + ); + + // And the real cycle that follows is not confused by it. + let real = fixture.calcalist.run(&["sync"]); + assert!( + real.succeeded(), + "sync after a dry run failed\n{}\n{}", + real.stdout, + real.stderr + ); + assert_eq!(fixture.server.stored("unified").len(), 2); +} + +/// Removing an endpoint used to leave its events sitting in the state directory +/// with nothing managing them. They are now reported, and removed on request. +#[test] +fn a_retired_endpoints_mirror_is_reported_and_then_removed() { + let Some(fixture) = fixture() else { return }; + + assert!(fixture.calcalist.run(&["sync"]).succeeded()); + let mirror = fixture.calcalist.state.join("calcalist/vdir/holidays"); + assert!(mirror.is_dir(), "the feed should have been mirrored"); + + // The feed is dropped from the configuration, as a user would drop it. + std::fs::write( + &fixture.calcalist.config, + fixture.calcalist.config_without_feed(), + ) + .expect("rewrite config"); + + let listed = fixture.calcalist.run(&["prune"]); + assert!(listed.succeeded(), "{}", listed.stderr); + assert!( + listed.stdout.contains("would remove") && listed.stdout.contains("holidays"), + "prune should say what it found:\n{}", + listed.stdout + ); + assert!(mirror.is_dir(), "listing must not delete anything"); + + let removed = fixture.calcalist.run(&["prune", "--force"]); + assert!(removed.succeeded(), "{}", removed.stderr); + assert!(!mirror.exists(), "the orphaned mirror should be gone"); +} + +/// An aggregate with no `default_sink` leaves events created in it alone. +/// +/// This is how a target calendar keeps events of its own: with nowhere +/// configured to file a new event, calcalist refuses to guess rather than +/// picking a source, so the event simply stays where it was written. Worth +/// pinning, because "left alone" has to hold on every subsequent cycle too — +/// an event that survived the first one and was swept up by the second would +/// be worse than never having worked. +#[test] +fn an_event_created_in_a_sinkless_aggregate_stays_where_it_is() { + let Some(fixture) = fixture() else { return }; + + // The fixture's aggregate has a default_sink; take it away. + let sinkless = std::fs::read_to_string(&fixture.calcalist.config) + .expect("read config") + .replace("default_sink = \"work\"\n", ""); + std::fs::write(&fixture.calcalist.config, sinkless).expect("rewrite config"); + + assert!(fixture.calcalist.run(&["sync"]).succeeded()); + + // Now add an event by hand, as a calendar app would. + fixture + .server + .put_event("unified", "dentist.ics", HAND_WRITTEN); + + for cycle in 1..=3 { + let run = fixture.calcalist.run(&["sync"]); + assert!( + run.succeeded(), + "cycle {cycle} failed\n{}\n{}", + run.stdout, + run.stderr + ); + + let published = fixture.server.stored("unified"); + assert!( + published.iter().any(|item| item.contains("Dentist")), + "cycle {cycle}: the hand-written event was removed:\n{published:?}" + ); + // It never reaches a source: there is nowhere it was told to go. + assert!( + !fixture + .server + .stored("work") + .iter() + .any(|item| item.contains("Dentist")), + "cycle {cycle}: the hand-written event leaked into the source" + ); + } +} + +/// `@local` opts one event out of a default sink that would otherwise take it. +/// +/// The marker has to survive, unlike every other one: it is stripped markers +/// that have done their job on arrival, whereas this event never leaves, so the +/// next cycle has to be able to reach the same decision. +#[test] +fn the_local_marker_keeps_an_event_out_of_a_configured_sink() { + let Some(fixture) = fixture() else { return }; + + assert!(fixture.calcalist.run(&["sync"]).succeeded()); + fixture + .server + .put_event("unified", "haircut.ics", KEPT_LOCAL); + + for cycle in 1..=3 { + let run = fixture.calcalist.run(&["sync"]); + assert!( + run.succeeded(), + "cycle {cycle} failed\n{}\n{}", + run.stdout, + run.stderr + ); + assert!( + run.stdout.contains("1 event(s) kept local"), + "cycle {cycle} should report it as kept, not skipped:\n{}", + run.stdout + ); + + let kept = fixture + .server + .stored("unified") + .into_iter() + .find(|item| item.contains("Haircut")) + .expect("the event should still be in the aggregate"); + // Stripping it would let the default sink claim the event next cycle. + assert!(kept.contains("@local"), "cycle {cycle}: {kept}"); + + assert!( + !fixture + .server + .stored("work") + .iter() + .any(|item| item.contains("Haircut")), + "cycle {cycle}: it reached the default sink anyway" + ); + } +} + +/// Discovery, against a server that really has two calendars. +/// +/// The command exists because the CalDAV `url` must name the collection exactly +/// and providers rarely show it. What it prints therefore has to be usable +/// as-is, which is what the parse-back below actually checks. +#[test] +fn discover_lists_the_servers_calendars_as_pasteable_config() { + let Some(fixture) = fixture() else { return }; + + let run = fixture.calcalist.run(&[ + "discover", + &format!( + "http://127.0.0.1:{}/{}/", + fixture.server.port, + support::USER + ), + "--username", + support::USER, + "--secret-command", + "printf password", + ]); + assert!( + run.succeeded(), + "discover failed\n{}\n{}", + run.stdout, + run.stderr + ); + + assert!(run.stdout.contains("id = \"work\""), "{}", run.stdout); + assert!(run.stdout.contains("id = \"unified\""), "{}", run.stdout); + // The URL has to come back absolute, not as the bare path pimsync reports. + assert!( + run.stdout + .contains(&format!("http://127.0.0.1:{}/", fixture.server.port)), + "{}", + run.stdout + ); + + // The point of the command: what it prints is valid configuration. + let config = format!("version = 1\n{}", run.stdout); + let parsed: toml::Value = toml::from_str(&config).expect("the output should be valid TOML"); + let endpoints = parsed["endpoint"].as_array().expect("endpoints"); + assert_eq!(endpoints.len(), 2, "{}", run.stdout); + assert_eq!(endpoints[0]["type"].as_str(), Some("caldav")); + assert_eq!( + endpoints[0]["secret_command"].as_str(), + Some("printf password") + ); +} + +/// An unreachable server has to say so rather than print an empty result that +/// looks like "this server has no calendars". +#[test] +fn discover_reports_a_server_it_cannot_reach() { + if !support::missing_binaries().is_empty() { + eprintln!("skipped: run under devbox"); + return; + } + let root = tempfile::tempdir().expect("temp"); + let calcalist = Calcalist::new(root.path(), "version = 1\n"); + + // Port 9 is discard: it refuses or blackholes, and nothing listens for DAV. + let run = calcalist.run(&[ + "discover", + "http://127.0.0.1:9/calendars/me/", + "--username", + "me", + ]); + + assert!( + !run.succeeded(), + "it should not claim success\n{}", + run.stdout + ); + assert!( + !run.stderr.is_empty(), + "the failure should be explained: {run:?}", + run = run.stderr + ); +} + +/// The lock is what stops a timer firing into a hand-run cycle. +/// +/// The lock is taken here rather than by racing two real syncs, which would be +/// timing-dependent and would pass by accident most of the time. +#[test] +fn a_second_cycle_is_refused_while_one_is_running() { + let Some(fixture) = fixture() else { return }; + assert!(fixture.calcalist.run(&["sync"]).succeeded()); + + let lock_path = fixture.calcalist.state.join("calcalist/lock"); + let held = std::fs::OpenOptions::new() + .read(true) + .write(true) + .open(&lock_path) + .expect("the lock file should exist after a cycle"); + rustix::fs::flock(&held, rustix::fs::FlockOperation::NonBlockingLockExclusive) + .expect("the test should be able to take the lock"); + + let refused = fixture.calcalist.run(&["sync"]); + assert!(!refused.succeeded(), "{}", refused.stdout); + assert!( + refused.stderr.contains("already running"), + "the refusal should say why: {}", + refused.stderr + ); + + drop(held); + assert!( + fixture.calcalist.run(&["sync"]).succeeded(), + "the lock should be free again" + ); +} diff --git a/tests/support/mod.rs b/tests/support/mod.rs new file mode 100644 index 0000000..90bf8fc --- /dev/null +++ b/tests/support/mod.rs @@ -0,0 +1,378 @@ +//! Scaffolding for the integration tests: a real CalDAV server, a real feed. +//! +//! Everything here talks HTTP over a plain socket rather than through a client +//! crate. The requests involved are few and mostly unusual — `MKCALENDAR`, `PUT` +//! of an `.ics` — and writing them out makes exactly what the server is asked +//! for visible in the test. + +use std::io::{Read, Write}; +use std::net::{Ipv4Addr, SocketAddr, TcpListener, TcpStream}; +use std::path::{Path, PathBuf}; +use std::process::{Child, Command, Stdio}; +use std::time::{Duration, Instant}; + +/// How long to wait for a server to start answering. +const STARTUP_TIMEOUT: Duration = Duration::from_secs(20); + +/// How many ports to try before giving up on starting a server. +const ATTEMPTS: usize = 8; + +/// The account Radicale files everything under. With authentication disabled it +/// accepts whatever name arrives, and `owner_only` rights then grant that name +/// its own tree — so the URLs below all live beneath it. +pub const USER: &str = "calcalist"; + +/// A binary the integration tests need, and whether it is here. +pub fn missing_binaries() -> Vec<&'static str> { + ["radicale", "pimsync"] + .into_iter() + .filter(|binary| which(binary).is_none()) + .collect() +} + +fn which(binary: &str) -> Option { + std::env::var_os("PATH")? + .to_str()? + .split(':') + .map(|dir| Path::new(dir).join(binary)) + .find(|candidate| candidate.is_file()) +} + +/// A Radicale instance with its own storage, shut down when dropped. +pub struct Radicale { + process: Child, + pub port: u16, + storage: PathBuf, +} + +impl Radicale { + pub fn start(root: &Path) -> Radicale { + let storage = root.join("radicale"); + std::fs::create_dir_all(&storage).expect("create storage"); + + // A free port is chosen by binding one and letting go, so two tests + // running at once can pick the same number. The loser's radicale then + // fails to bind and exits — and, worse, waiting for the port to answer + // succeeds anyway, because the winner is listening on it. Tests would + // quietly share a server. So: confirm the child we spawned is the one + // still alive, and take a different port if it is not. + for _ in 0..ATTEMPTS { + let port = free_port(); + let mut process = Command::new("radicale") + .arg("--config") + .arg("") + .arg("--server-hosts") + .arg(format!("127.0.0.1:{port}")) + .arg("--auth-type") + .arg("none") + .arg("--storage-filesystem-folder") + .arg(&storage) + .stdout(Stdio::null()) + .stderr(Stdio::null()) + .spawn() + .expect("radicale should start"); + + if listening(&mut process, port) { + return Radicale { + process, + port, + storage, + }; + } + let _ = process.kill(); + let _ = process.wait(); + } + panic!("radicale did not come up on a port of its own after {ATTEMPTS} attempts"); + } + + pub fn url(&self, calendar: &str) -> String { + format!("http://127.0.0.1:{}/{USER}/{calendar}/", self.port) + } + + /// Creates a calendar collection. pimsync deliberately never creates one, + /// so the calendars a test syncs have to exist on the server first. + pub fn create_calendar(&self, calendar: &str) { + let body = "\ + \ + calendar\ + "; + let response = self.request( + "MKCALENDAR", + &format!("/{USER}/{calendar}/"), + "application/xml; charset=utf-8", + body, + ); + // Radicale answers HTTP/1.0, so the status is read out of the line + // rather than matched against a whole prefix. + assert!( + matches!(status_of(&response), Some(201 | 405)), + "creating {calendar}: {response}" + ); + } + + /// Stores an event, as a calendar client would. + pub fn put_event(&self, calendar: &str, name: &str, ics: &str) { + let response = self.request( + "PUT", + &format!("/{USER}/{calendar}/{name}"), + "text/calendar; charset=utf-8", + ics, + ); + assert!( + status_of(&response).is_some_and(|status| (200..300).contains(&status)), + "storing {name}: {response}" + ); + } + + /// Everything the server holds in a calendar, as stored. + /// + /// Read from Radicale's own storage rather than fetched back, so what is + /// asserted on is the bytes that reached the server. + pub fn stored(&self, calendar: &str) -> Vec { + let dir = self + .storage + .join("collection-root") + .join(USER) + .join(calendar); + let Ok(entries) = std::fs::read_dir(dir) else { + return Vec::new(); + }; + let mut items: Vec = entries + .flatten() + .filter(|entry| { + entry + .path() + .extension() + .is_some_and(|extension| extension == "ics") + }) + .filter_map(|entry| std::fs::read_to_string(entry.path()).ok()) + .collect(); + items.sort(); + items + } + + fn request(&self, method: &str, path: &str, content_type: &str, body: &str) -> String { + // Retried because a server under parallel test load occasionally resets + // a connection it has accepted; that says nothing about the request. + let mut last = String::new(); + for attempt in 0..ATTEMPTS { + match self.try_request(method, path, content_type, body) { + Ok(response) => return response, + Err(error) => { + last = error; + std::thread::sleep(Duration::from_millis(50 * (attempt as u64 + 1))); + } + } + } + panic!("{method} {path} never got a response: {last}"); + } + + fn try_request( + &self, + method: &str, + path: &str, + content_type: &str, + body: &str, + ) -> Result { + let address = SocketAddr::from((Ipv4Addr::LOCALHOST, self.port)); + let mut stream = TcpStream::connect_timeout(&address, Duration::from_secs(2)) + .map_err(|error| error.to_string())?; + stream + .set_read_timeout(Some(Duration::from_secs(10))) + .map_err(|error| error.to_string())?; + // With authentication disabled the credentials only pick the account. + let authorization = base64(format!("{USER}:password").as_bytes()); + let request = format!( + "{method} {path} HTTP/1.1\r\nHost: 127.0.0.1:{}\r\n\ + Authorization: Basic {authorization}\r\nContent-Type: {content_type}\r\n\ + Content-Length: {}\r\nConnection: close\r\n\r\n{body}", + self.port, + body.len() + ); + stream + .write_all(request.as_bytes()) + .map_err(|error| error.to_string())?; + let mut response = String::new(); + stream + .read_to_string(&mut response) + .map_err(|error| error.to_string())?; + Ok(response) + } +} + +impl Drop for Radicale { + fn drop(&mut self) { + let _ = self.process.kill(); + let _ = self.process.wait(); + } +} + +/// A one-file HTTP server, standing in for a published iCal feed. +pub struct Feed { + pub port: u16, +} + +impl Feed { + /// Serves `ics` at any path, for as long as the test runs. + pub fn start(ics: String) -> Feed { + let listener = TcpListener::bind(SocketAddr::from((Ipv4Addr::LOCALHOST, 0))) + .expect("bind the feed port"); + let port = listener.local_addr().expect("feed address").port(); + std::thread::spawn(move || { + for stream in listener.incoming().flatten() { + serve_once(stream, &ics); + } + }); + Feed { port } + } + + pub fn url(&self) -> String { + format!("http://127.0.0.1:{}/holidays.ics", self.port) + } +} + +fn serve_once(mut stream: TcpStream, ics: &str) { + // Enough of the request to reach the blank line; the path does not matter. + let mut buffer = [0u8; 2048]; + let _ = stream.read(&mut buffer); + let response = format!( + "HTTP/1.1 200 OK\r\nContent-Type: text/calendar; charset=utf-8\r\n\ + Content-Length: {}\r\nConnection: close\r\n\r\n{ics}", + ics.len() + ); + let _ = stream.write_all(response.as_bytes()); +} + +/// Runs `calcalist` against a config and state directory of the test's own. +pub struct Calcalist { + pub config: PathBuf, + pub state: PathBuf, +} + +/// What one `calcalist` invocation printed and returned. +pub struct Run { + pub status: Option, + pub stdout: String, + pub stderr: String, +} + +impl Run { + pub fn succeeded(&self) -> bool { + self.status == Some(0) + } +} + +impl Calcalist { + pub fn new(root: &Path, config: &str) -> Calcalist { + let config_path = root.join("calcalist.toml"); + std::fs::write(&config_path, config).expect("write config"); + let state = root.join("state"); + std::fs::create_dir_all(&state).expect("create state directory"); + Calcalist { + config: config_path, + state, + } + } + + /// The same configuration with the read-only feed taken out, for the test + /// that retires an endpoint. + pub fn config_without_feed(&self) -> String { + let text = std::fs::read_to_string(&self.config).expect("read config"); + text.split("\n\n") + .filter(|block| !block.contains("id = \"holidays\"")) + .map(|block| block.replace(", \"holidays\"", "")) + .collect::>() + .join("\n\n") + } + + pub fn run(&self, arguments: &[&str]) -> Run { + let output = Command::new(env!("CARGO_BIN_EXE_calcalist")) + .arg("--config") + .arg(&self.config) + .args(arguments) + // The state directory is resolved through XDG, so this is what + // keeps the test off the developer's own calendars and state. + .env("XDG_STATE_HOME", &self.state) + .output() + .expect("calcalist should run"); + Run { + status: output.status.code(), + stdout: String::from_utf8_lossy(&output.stdout).into_owned(), + stderr: String::from_utf8_lossy(&output.stderr).into_owned(), + } + } +} + +/// The status code out of a response's first line, whatever HTTP version it +/// claims. +fn status_of(response: &str) -> Option { + response + .lines() + .next()? + .split_whitespace() + .nth(1)? + .parse() + .ok() +} + +fn free_port() -> u16 { + let listener = + TcpListener::bind(SocketAddr::from((Ipv4Addr::LOCALHOST, 0))).expect("bind a free port"); + listener.local_addr().expect("port").port() +} + +/// Whether a real HTTP request gets a real HTTP response. +fn answers(address: SocketAddr) -> bool { + let Ok(mut stream) = TcpStream::connect_timeout(&address, Duration::from_millis(200)) else { + return false; + }; + let _ = stream.set_read_timeout(Some(Duration::from_secs(2))); + let request = format!("OPTIONS / HTTP/1.1\r\nHost: {address}\r\nConnection: close\r\n\r\n"); + if stream.write_all(request.as_bytes()).is_err() { + return false; + } + let mut response = String::new(); + stream.read_to_string(&mut response).is_ok() && response.starts_with("HTTP/") +} + +/// Waits for our own radicale to answer, giving up as soon as it has exited. +/// +/// The liveness check is the point: without it a process that lost a port race +/// looks healthy, because something else is answering on that port. +fn listening(process: &mut Child, port: u16) -> bool { + let address = SocketAddr::from((Ipv4Addr::LOCALHOST, port)); + let deadline = Instant::now() + STARTUP_TIMEOUT; + while Instant::now() < deadline { + match process.try_wait() { + Ok(Some(_)) => return false, + Ok(None) => {} + Err(_) => return false, + } + // An open port is not readiness: the socket listens before the + // application behind it can answer, and under parallel tests that gap + // is wide enough to get a connection reset instead of a response. + if answers(address) { + return true; + } + std::thread::sleep(Duration::from_millis(50)); + } + false +} + +fn base64(input: &[u8]) -> String { + const ALPHABET: &[u8] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"; + let mut out = String::new(); + for chunk in input.chunks(3) { + let mut block = [0u8; 3]; + block[..chunk.len()].copy_from_slice(chunk); + let packed = u32::from(block[0]) << 16 | u32::from(block[1]) << 8 | u32::from(block[2]); + for index in 0..4 { + if index <= chunk.len() { + out.push(ALPHABET[(packed >> (18 - index * 6)) as usize & 0x3f] as char); + } else { + out.push('='); + } + } + } + out +}