From 838e7a18918388cd12d211f5353c28e093cd67f9 Mon Sep 17 00:00:00 2001 From: randogoth Date: Thu, 10 Sep 2026 13:39:54 +0300 Subject: [PATCH 01/10] Bring TODO.md up to date with M1 as built MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit M1 is functionally complete, so the roadmap now separates what is done from what is genuinely outstanding rather than leaving both in one list. Records the work that arrived after the original plan — routing hints, alarm mapping both ways, command-sourced feed URLs, the topology tests — and what was verified against real accounts rather than only in tests. Adds four gaps found while building and testing that were not previously written down: a Google endpoint needs its own login even when it shares an account and client with another; removing an endpoint leaves its vdir behind; --dry-run reports against the last pull rather than current remote state; and a TZID is emitted without an accompanying VTIMEZONE. Co-Authored-By: Claude Opus 5 --- TODO.md | 110 +++++++++++++++++++++++++++++++++++++------------------- 1 file changed, 73 insertions(+), 37 deletions(-) diff --git a/TODO.md b/TODO.md index 6c47de7..8bc472a 100644 --- a/TODO.md +++ b/TODO.md @@ -14,29 +14,31 @@ sync semantics these items implement. ## M1 — bidirectional sync -Core modules: +Functionally complete. Everything below is done except the two automated +integration tests, which are listed with the remaining gaps at the end. + +### Core modules - [x] `state.rs` — JSON sidecar, atomic temp + fsync + rename; records each aggregate's - resolved target endpoint id **and** backend type -- [x] `vdir.rs` — read and write vdir directories + resolved target endpoint id **and** backend type, plus per-Google-endpoint cursors +- [x] `vdir.rs` — read and write vdir directories, keyed by the UID inside each file + rather than by filename, since pimsync names files its own way - [x] `ical.rs` — surgical line-level `.ics` editing (UID rewrite, property injection), respecting RFC 5545 folding; no parse-and-reserialize - [x] `provenance.rs` — deterministic `blake3(aggregate_id, source_id, source_uid)` UIDs - [x] `mirror.rs` — the to-aggregate and to-source transforms (added; not in the original plan, which folded these into `reconcile`) - [x] `reconcile.rs` — the aggregation engine; pure, no I/O -- [x] `sync.rs` — one cycle over the local vdirs, applying what `reconcile` decides +- [x] `sync.rs` — one cycle: pull, reconcile, push +- [x] `retarget.rs` — moving an aggregate to a different target, deliberately - [x] `pimsync.rs` — generate `pimsync.conf` (with `on_empty skip` and `on_delete skip`), drive one-shot `pimsync sync` bracketing the reconcile step -- [x] `doctor` asks `pimsync check` to validate the generated config, since pimsync's - parser does not always match its documentation - [x] `google/auth.rs` — OAuth loopback flow with PKCE, refresh, keyring-sourced secrets -- [x] `google/convert.rs` — JSON to iCalendar, including recurrence and timezones -- [x] `google/api.rs` — incremental pull by syncToken, and push by import / update - / delete with notification suppressed -- [x] Reintroduce `SchedulingSuppression` in `config.rs` (removed in M0 as dead code) +- [x] `google/convert.rs` — iCalendar both ways, including recurrence, timezones and alarms +- [x] `google/api.rs` — incremental pull by syncToken; push by import / update / delete + with notification suppressed -Safety-critical behaviour: +### Behaviour - [x] **`events.import` gate** — settled from Google's own API discovery document: `events.import` accepts no `sendUpdates` parameter at all, while `insert`, @@ -45,46 +47,80 @@ Safety-critical behaviour: Creation goes through `import`; update and delete pass `sendUpdates=none`. - [x] `sync` refuses to run on aggregate target drift, before reconciliation - [x] `aggregate retarget` — flush unrouted creations against the old target, then - re-materialise; keep old orphans by default + re-materialise; keep old orphans by default, purge bounded by the derivation - [x] Mass-deletion guard (`max_delete_fraction`), overridable with `--force`, with an absolute floor so deleting a couple of events is never refused - [x] Echo suppression: derived UIDs are never re-ingested as source events +- [x] Routing: a `@endpoint-id` line in the description, or a matching category, picks + which source a new aggregate event is filed under; a hint naming an invalid sink + is refused rather than redirected to the default +- [x] Alarms map both ways, including Google's minutes-before reminder model +- [x] `webcal` URLs may come from a command, for feeds whose address is itself a + credential — Google's secret iCal address being the case in point +- [x] `doctor` validates the generated pimsync config and each Google authorisation -Tests: +### Tests -- [x] `reconcile` table-driven cases: create/update/delete each direction, both-sides-changed, - routing, echo suppression, mass-delete abort +- [x] `reconcile` table-driven cases: create/update/delete each direction, + both-sides-changed, routing, echo suppression, mass-delete abort - [x] `ical` round-trip fixtures: recurring with overrides, all-day, TZID, unknown `X-` props -- [ ] Integration against Radicale plus a `file://` WebCal fixture; assert idempotence -- [x] Safety (unit level): no live `ATTENDEE`/`ORGANIZER` on a CalDAV-targeted mirror, - `VALARM` intact, `PARTSTAT: DECLINED` maps to `TRANSP: TRANSPARENT`, bulk deletion aborts -- [ ] Safety (integration): the same against a real Radicale instance with an SMTP sink, - proving no mail is emitted -- [x] Retarget: drift makes `sync` exit non-zero having written nothing and losing no source - event; purge is bounded by the derivation; an unrouted creation reaches a sink first +- [x] `convert` against real captured API payloads, including the case where Google's + offset and its `timeZone` disagree +- [x] Topologies: a source feeding several aggregates, several aggregates sharing one + target, a cycle between two aggregates, a delete cascading across aggregates, + and competing edits arriving through two aggregates at once +- [x] Retarget: drift makes `sync` exit non-zero having written nothing and losing no + source event; an unrouted creation reaches a sink first +- [x] Safety (unit): no live `ATTENDEE`/`ORGANIZER` on a CalDAV-targeted mirror, + `VALARM` intact, `PARTSTAT: DECLINED` maps to `TRANSP: TRANSPARENT` ### Verified live, end to end -Two Google calendars aggregating into a Posteo CalDAV calendar, against real accounts: -fan-in from both sources with provenance intact; an edit in the aggregate reaching the -originating Google calendar; an event created in the aggregate routed to a chosen -source by a description marker; deletion propagating from the aggregate through to -Google; and the mass-deletion guard refusing a 100% removal until `--force`. +Against real accounts — two Google calendars aggregating into a Posteo CalDAV +calendar: -### Known gaps carried out of M1 +- Fan-in from both Google sources, each mirror tagged with its origin +- A public iCal feed synced through pimsync and pushed to CalDAV; idempotent across + repeated runs and correctly incremental +- An edit made in the aggregate reaching the originating Google calendar, and the + revert travelling the same way +- An event created in the aggregate routed to a chosen source by its description + marker, with the marker stripped before it arrived +- A deletion in the aggregate propagating through to Google +- The mass-deletion guard refusing a 100% removal until `--force` +- `pimsync check` validating the generated config against a live CalDAV server -- [ ] A recurring series' *exceptions* are not pushed to Google. Google models them - as separate events against an already existing series, so they need - `events.instances` plus a patch per exception. Reported per sync rather than - dropped silently. -- [x] `push` deleting an event remotely — verified live: deleting a mirror in the - CalDAV aggregate removed the origin event from Google. +## Remaining + +### Tests not yet automated + +- [ ] Integration against Radicale plus a WebCal fixture, asserting convergence and + idempotence. Done by hand twice; not yet a test that runs in CI. +- [ ] Safety integration: a real CalDAV server with an SMTP sink, proving no mail is + emitted on mirror writes or mirror deletions. + +### Known gaps + +- [ ] A recurring series' *exceptions* are not pushed to Google. Pulling them works. + Google models them as separate events against an already existing series, so + pushing needs `events.instances` plus a patch per exception. Reported per sync + rather than dropped silently. +- [ ] Each Google endpoint needs its own `google login`, even for the same account and + OAuth client, because tokens are keyed by endpoint id. Two endpoints on one + account should share a credential. - [ ] A failed Google pull aborts the whole cycle, including the CalDAV side. Safe — - reconciling against a stale snapshot could read as mass deletion — but it means - a lapsed token stops everything. + reconciling against a stale snapshot could read as mass deletion — but a lapsed + token stops everything. Skipping only the affected aggregates would be better. +- [ ] Removing an endpoint from the config leaves its vdir behind, holding events + nothing manages any more. +- [ ] `--dry-run` skips the pull entirely, so it reports against whatever the last + real cycle left behind rather than against current remote state. +- [ ] A `TZID` is emitted without an accompanying `VTIMEZONE`. Tolerated by the servers + tested so far, and confined to recurring events, but not strictly conformant. ## M2 — interface and packaging - [ ] axum configuration UI, bound to 127.0.0.1 -- [ ] OAuth loopback redirect handler +- [ ] Trigger `google login` from the UI (the loopback handler itself already exists + in `google/auth.rs`) - [ ] systemd user units: `calcalist.service` (oneshot) and `calcalist.timer` From 06e661b0c41f020144487cc29602541e69a0d225 Mon Sep 17 00:00:00 2001 From: randogoth Date: Thu, 10 Sep 2026 13:40:27 +0300 Subject: [PATCH 02/10] Add calcalist status, and make routing markers discoverable MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The marker that steers a new event is the endpoint's id, which until now meant remembering what was written in a TOML file while typing into a calendar app. status lists the endpoints, what each aggregate publishes and mirrors, where untagged events go, and the exact @markers that would work — read-only sources omitted, since offering one as a destination would mislead. A refused marker now names the alternatives rather than only reporting that it failed, and matching ignores case: the name is typed by hand, and capitalisation is not worth failing over. 133 tests. Co-Authored-By: Claude Opus 5 --- TODO.md | 3 + src/main.rs | 32 +++++++++- src/reconcile.rs | 23 ++++++- src/status.rs | 158 +++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 213 insertions(+), 3 deletions(-) create mode 100644 src/status.rs diff --git a/TODO.md b/TODO.md index 8bc472a..266842f 100644 --- a/TODO.md +++ b/TODO.md @@ -58,6 +58,9 @@ integration tests, which are listed with the remaining gaps at the end. - [x] `webcal` URLs may come from a command, for feeds whose address is itself a credential — Google's secret iCal address being the case in point - [x] `doctor` validates the generated pimsync config and each Google authorisation +- [x] `calcalist status` — endpoints, aggregates, how many events are mirrored, and the + exact `@marker` names an event can carry, so routing is discoverable without + opening the config file ### Tests diff --git a/src/main.rs b/src/main.rs index 1aee63a..3bb5c2f 100644 --- a/src/main.rs +++ b/src/main.rs @@ -12,6 +12,7 @@ mod provenance; mod reconcile; mod retarget; mod state; +mod status; mod sync; mod vdir; @@ -30,6 +31,10 @@ fn main() -> ExitCode { command: Command::Doctor, .. } => run_doctor(cli), + cli @ Cli { + command: Command::Status, + .. + } => run_status(cli), cli @ Cli { command: Command::Sync { dry_run, force }, .. @@ -105,6 +110,25 @@ fn run_google_login(cli: &Cli, endpoint_id: &str) -> ExitCode { } } +/// Shows what is configured, and what an event's notes may say to steer it. +fn run_status(cli: &Cli) -> ExitCode { + let (config, _) = match Config::load(cli.config.as_deref()) { + Ok(loaded) => loaded, + Err(error) => return fail(&error), + }; + let state_dir = match paths::state_dir() { + Ok(dir) => dir, + Err(error) => return fail(&error), + }; + match status::render(&config, &state_dir) { + Ok(report) => { + print!("{report}"); + ExitCode::SUCCESS + } + Err(error) => fail(&error), + } +} + /// Moves an aggregate to a different target endpoint, deliberately. fn run_retarget(cli: &Cli, id: &str, to: &str, purge_old: bool) -> ExitCode { let (config, _) = match Config::load(cli.config.as_deref()) { @@ -202,8 +226,14 @@ fn describe(skipped: &reconcile::Skipped) -> String { reconcile::Skipped::UnknownSink { aggregate_uid, requested, + available, } => format!( - "`{aggregate_uid}` asked to be filed under `{requested}`, which is not a writable source of this aggregate; it was left where it is" + "`{aggregate_uid}` asked to be filed under `@{requested}`, which is not a writable source of this aggregate. It was left where it is. Try one of: {}", + available + .iter() + .map(|name| format!("@{name}")) + .collect::>() + .join(", ") ), } } diff --git a/src/reconcile.rs b/src/reconcile.rs index 6ac7a07..4eb4635 100644 --- a/src/reconcile.rs +++ b/src/reconcile.rs @@ -93,10 +93,12 @@ pub enum Skipped { }, /// The event asked to be filed under a source that cannot take it. Refused /// rather than sent to the default, since a typo should not quietly put the - /// event in the wrong calendar. + /// event in the wrong calendar. The valid names travel with it, so the report + /// can say what would have worked. UnknownSink { aggregate_uid: String, requested: String, + available: Vec, }, } @@ -359,8 +361,14 @@ fn route_new_events( // configured sink takes it. let requested = mirror::routing_hint(&item.calendar); let wanted = requested.as_deref().or(policy.default_sink); + // Case-insensitively: the name is typed into a calendar app by hand, and + // capitalisation is not worth failing over. let sink = wanted - .and_then(|sink| sources.iter().find(|source| source.id == sink)) + .and_then(|sink| { + sources + .iter() + .find(|source| source.id.eq_ignore_ascii_case(sink)) + }) .filter(|sink| sink.writable); let Some(sink) = sink else { @@ -368,6 +376,7 @@ fn route_new_events( Some(requested) => Skipped::UnknownSink { aggregate_uid: uid.clone(), requested, + available: writable_sinks(sources), }, None => Skipped::NoSink { aggregate_uid: uid.clone(), @@ -431,6 +440,15 @@ fn route_new_events( /// that failed to populate, which shows up as a *bulk* disappearance. const ALWAYS_ALLOWED_DELETIONS: usize = 3; +/// The sources of this aggregate that an event could actually be filed under. +fn writable_sinks(sources: &[SourceView<'_>]) -> Vec { + sources + .iter() + .filter(|source| source.writable) + .map(|source| source.id.to_string()) + .collect() +} + /// Refuses a cycle that would delete an implausible share of the aggregate. /// /// A source vdir that failed to populate looks exactly like one whose events were @@ -826,6 +844,7 @@ mod tests { vec![Skipped::UnknownSink { aggregate_uid: "hand@phone".into(), requested: "nosuchplace".into(), + available: vec![SRC.to_string()], }] ); } diff --git a/src/status.rs b/src/status.rs new file mode 100644 index 0000000..e13d427 --- /dev/null +++ b/src/status.rs @@ -0,0 +1,158 @@ +//! `calcalist status` — what is configured, and what it has done. +//! +//! Its most practical job is naming the markers an event can carry. A routing +//! hint is typed by hand into a calendar app, and having to remember what was +//! written in a TOML file is no way to find out what it should say. + +use std::fmt::Write as _; +use std::path::Path; + +use crate::config::{Aggregate, Config}; +use crate::state::{State, StateError}; + +/// Renders the report shown by `calcalist status`. +pub fn render(config: &Config, state_dir: &Path) -> Result { + let state = State::load(&state_dir.join(crate::state::FILE_NAME))?; + let mut out = String::new(); + + out.push_str("endpoints\n"); + for endpoint in &config.endpoints { + let access = if endpoint.kind.is_writable() { + "read/write" + } else { + "read-only" + }; + let _ = writeln!( + out, + " {:<18} {:<8} {access}", + endpoint.id, + endpoint.kind.kind_name() + ); + } + + for aggregate in &config.aggregates { + let _ = write!(out, "\naggregate `{}`\n", aggregate.id); + let _ = writeln!(out, " published to {}", aggregate.target); + let _ = writeln!(out, " sources {}", aggregate.sources.join(", ")); + + let tracked = state + .aggregate(&aggregate.id) + .map_or(0, |entry| entry.links.len()); + let _ = writeln!(out, " mirroring {tracked} event(s)"); + + describe_routing(&mut out, config, aggregate); + } + + if config.aggregates.is_empty() { + out.push_str("\nno aggregates configured\n"); + } + Ok(out) +} + +/// Explains where a newly created event goes, and how to send it elsewhere. +fn describe_routing(out: &mut String, config: &Config, aggregate: &Aggregate) { + let sinks: Vec<&str> = aggregate + .sources + .iter() + .filter(|id| { + config + .endpoint(id) + .is_some_and(|endpoint| endpoint.kind.is_writable()) + }) + .map(String::as_str) + .collect(); + + match &aggregate.default_sink { + Some(sink) => { + let _ = writeln!(out, " new events go to `{sink}` unless told otherwise"); + } + None => { + let _ = writeln!( + out, + " new events are left alone: no default_sink is configured" + ); + } + } + if sinks.is_empty() { + let _ = writeln!(out, " none of its sources can be written to"); + return; + } + let _ = writeln!( + out, + " to choose, put one of these on a line of its own in the event's notes:" + ); + for sink in sinks { + let _ = writeln!(out, " @{sink}"); + } +} + +#[cfg(test)] +mod tests { + use super::*; + + const CONFIG: &str = r#" +version = 1 + +[[endpoint]] +id = "gcal" +type = "google" +calendar_id = "a@example.com" +client_id = "x.apps.googleusercontent.com" + +[[endpoint]] +id = "posteo" +type = "caldav" +url = "https://dav.example.com/dav/calendars/me/work/" +username = "me@example.com" + +[[endpoint]] +id = "holidays" +type = "webcal" +url = "https://example.org/h.ics" + +[[aggregate]] +id = "unified" +target = "posteo" +sources = ["gcal", "holidays"] +default_sink = "gcal" +"#; + + fn report() -> String { + let config: Config = toml::from_str(CONFIG).expect("config"); + let dir = tempfile::tempdir().expect("temp dir"); + render(&config, dir.path()).expect("render") + } + + /// The whole point of the command: knowing what to type without opening the + /// configuration file. + #[test] + fn it_names_the_markers_that_would_work() { + let report = report(); + assert!(report.contains("@gcal"), "{report}"); + } + + /// A read-only feed cannot take an event, so offering it would mislead. + #[test] + fn it_does_not_offer_a_read_only_source_as_a_destination() { + let report = report(); + assert!(!report.contains("@holidays"), "{report}"); + assert!( + report.contains("holidays"), + "it should still be listed as a source" + ); + } + + #[test] + fn it_says_where_untagged_events_go() { + assert!(report().contains("new events go to `gcal`")); + } + + #[test] + fn it_says_when_nothing_would_be_routed() { + let sinkless: Config = + toml::from_str(&CONFIG.replace(r#"default_sink = "gcal""#, "")).expect("config"); + let dir = tempfile::tempdir().expect("temp dir"); + let report = render(&sinkless, dir.path()).expect("render"); + assert!(report.contains("no default_sink is configured"), "{report}"); + } +} From 623505b9c8a6aa02475a09b987bbdebd23d0eea1 Mon Sep 17 00:00:00 2001 From: randogoth Date: Thu, 10 Sep 2026 13:49:38 +0300 Subject: [PATCH 03/10] Close the remaining M1 gaps MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Eight items were still open at the end of M1: two integration tests that had only been run by hand, and six known gaps. Recurrence overrides now reach Google. Google addresses an exception through the series rather than as an event of its own, so the master is sent first and each override is then matched to its instance by original start time and patched. Matching needs the two sides' spellings reduced to one key: iCalendar writes a zoned local time, Google an absolute offset. An override matching no occurrence is counted rather than forced — that means a stale RECURRENCE-ID left behind by an edited RRULE, and inventing an event for it would put something in the calendar the series does not contain. Reading one component apart from another needed a view `properties` cannot give: it flattens every VEVENT together, which is right for the UID a series shares and wrong for an override, whose SUMMARY and the master's are then indistinguishable. `Calendar::events` splits them. A TZID now travels with the VTIMEZONE that defines it, derived from the zone's own transition table as the yearly rule it implies. This changes the content hash of every zoned recurring event, so the first cycle after this re-pushes them. A Google authorisation is filed under the account it was granted for rather than the endpoint that asked for it, so two endpoints on one account no longer need a login each. The account is read from the primary calendar's id, which needs no scope beyond the calendar one already granted. Authorisations written by the previous scheme are still honoured, and move across at the next login. An unreachable Google endpoint no longer ends the cycle — one lapsed token used to stop the CalDAV side too. It is named, only the aggregates depending on it stand down, and the run exits non-zero so a partial cycle cannot pass for success. The CalDAV leg cannot be narrowed the same way: pimsync is one process covering every pair, so a failure does not say which pair it belongs to. `--dry-run` now pulls for real, into a throwaway copy of the local mirrors and through a pimsync configuration that only ever reads from a server. What it reports is measured against the calendars as they are now rather than against whatever the last real cycle left behind. `calcalist prune` reports local mirrors of endpoints the configuration no longer names, and removes them under --force. The integration tests run against a real Radicale server and a real iCal feed: convergence and idempotence, the dry run, prune, and the scheduling rule asserted on the bytes that actually reached the server. The plan asked for an SMTP sink for that last one; Radicale implements no RFC 6638 scheduling, so a quiet SMTP port would have proved nothing about the transform. Co-Authored-By: Claude Opus 5 --- SPECS.md | 4 + TODO.md | 66 +++--- src/cli.rs | 6 + src/config.rs | 15 ++ src/doctor.rs | 7 +- src/google/api.rs | 117 ++++++++++- src/google/auth.rs | 273 ++++++++++++++++++++++++- src/google/convert.rs | 456 ++++++++++++++++++++++++++++++++++++++++-- src/ical.rs | 98 ++++++++- src/main.rs | 79 +++++++- src/pimsync.rs | 46 ++++- src/sync.rs | 401 ++++++++++++++++++++++++++++++------- tests/caldav.rs | 274 +++++++++++++++++++++++++ tests/support/mod.rs | 310 ++++++++++++++++++++++++++++ 14 files changed, 2008 insertions(+), 144 deletions(-) create mode 100644 tests/caldav.rs create mode 100644 tests/support/mod.rs diff --git a/SPECS.md b/SPECS.md index 7300e38..710f76a 100644 --- a/SPECS.md +++ b/SPECS.md @@ -14,6 +14,7 @@ Every endpoint — sources and aggregate targets alike — is mirrored to a loca - The Google leg is calcalist's own, via the Calendar REST API. pimsync cannot reach Google — it has no REST storage, and its config exposes only HTTP Basic auth, which Google's CalDAV endpoint has rejected since 2025-03-14. - Sync state (event mappings, hashes, sync tokens) lives in a JSON sidecar under `$XDG_STATE_HOME/calcalist/`, never in the portable config. - Secrets are never stored in the config either. Credentials come from `*_command` fields that are executed to fetch them. +- A Google authorisation is filed under the account it was granted for, not the endpoint that asked for it, so every endpoint on that account shares it. An endpoint names its `account` only when calcalist is logged in to more than one. ## Sync semantics @@ -27,6 +28,9 @@ Every endpoint — sources and aggregate targets alike — is mirrored to a loca | Attendees (routing) | Preserved — the sink server sends real invitations, which is intended | | Alarms | Always preserved, with no option to strip them | | Retargeting | `sync` refuses on target drift; `aggregate retarget` performs it deliberately | +| Recurrence | A series and its exceptions share one file. Google addresses an exception through the series, so the master is sent first and each override is matched to its instance by original start time and patched | +| Unreachable endpoints | Named in the report; only the aggregates depending on them stand down, and the run exits non-zero | +| Dry runs | Pull for real, into a throwaway copy of the local mirrors and through a read-only pimsync configuration. Nothing outside the copy is written | The governing rule behind the attendee handling: **writes to an aggregate must never emit scheduling mail; writes to a source schedule normally.** Google can be told not to notify, so attendees survive intact there. CalDAV offers no portable way to suppress RFC 6638 scheduling, so inertness is achieved structurally by dropping the live properties instead. diff --git a/TODO.md b/TODO.md index 266842f..34b4537 100644 --- a/TODO.md +++ b/TODO.md @@ -14,8 +14,7 @@ sync semantics these items implement. ## M1 — bidirectional sync -Functionally complete. Everything below is done except the two automated -integration tests, which are listed with the remaining gaps at the end. +Complete. Every item from the plan is built, tested and verified live. ### Core modules @@ -61,6 +60,26 @@ integration tests, which are listed with the remaining gaps at the end. - [x] `calcalist status` — endpoints, aggregates, how many events are mirrored, and the exact `@marker` names an event can carry, so routing is discoverable without opening the config file +- [x] Recurrence overrides are pushed to Google. Google addresses an exception + through the series rather than as an event of its own, so the master goes up + first and each override is then matched to its instance by original start + time and patched. An override matching no occurrence is counted, not forced. +- [x] A `TZID` now travels with the `VTIMEZONE` that defines it, derived from the + zone's own transition table as the yearly rule it implies. A zone that does + not shift gets one fixed observance. +- [x] One Google authorisation covers every endpoint on that account. Tokens are + keyed by the account, discovered at login from the primary calendar's id — + which needs no scope beyond the calendar one already granted. An endpoint + names its `account` only when calcalist is logged in to more than one. +- [x] A Google endpoint that cannot be reached no longer ends the cycle. It is + named, only the aggregates depending on it stand down, and the run exits + non-zero so a lapsed token cannot pass for success. +- [x] `calcalist prune` — reports local mirrors of endpoints the configuration no + longer names, and removes them under `--force` +- [x] `--dry-run` pulls for real, into a throwaway copy of the local mirrors and + through a pimsync configuration that only ever reads from a server. What it + reports is measured against the calendars as they are now, and nothing + outside the copy is written. ### Tests @@ -93,33 +112,30 @@ calendar: - The mass-deletion guard refusing a 100% removal until `--force` - `pimsync check` validating the generated config against a live CalDAV server -## Remaining +### Integration -### Tests not yet automated +- [x] Against a real Radicale server and a real iCal feed, over pimsync: two + sources converge on a CalDAV target, and the next cycle is a no-op. +- [x] Safety, against the same server: no live `ATTENDEE` or `ORGANIZER` reaches + the aggregate, the guest list survives as inert data, the alarm survives + intact, and the source keeps its scheduling properties. -- [ ] Integration against Radicale plus a WebCal fixture, asserting convergence and - idempotence. Done by hand twice; not yet a test that runs in CI. -- [ ] Safety integration: a real CalDAV server with an SMTP sink, proving no mail is - emitted on mirror writes or mirror deletions. + The plan asked for an SMTP sink here. Radicale implements no RFC 6638 + scheduling, so a quiet SMTP port would have proved nothing about the + transform — no server in reach of a test sends calendar mail at all. What + is asserted instead is the bytes that reached the server, which is the + thing the transform is actually responsible for. Proving the Google half + was settled separately, from the API discovery document. +- [x] `--dry-run` reaches the servers, reports what it found, and leaves both the + target and the local mirrors untouched. +- [x] A retired endpoint's mirror is reported by `prune` and removed under + `--force`. -### Known gaps +## Known gaps -- [ ] A recurring series' *exceptions* are not pushed to Google. Pulling them works. - Google models them as separate events against an already existing series, so - pushing needs `events.instances` plus a patch per exception. Reported per sync - rather than dropped silently. -- [ ] Each Google endpoint needs its own `google login`, even for the same account and - OAuth client, because tokens are keyed by endpoint id. Two endpoints on one - account should share a credential. -- [ ] A failed Google pull aborts the whole cycle, including the CalDAV side. Safe — - reconciling against a stale snapshot could read as mass deletion — but a lapsed - token stops everything. Skipping only the affected aggregates would be better. -- [ ] Removing an endpoint from the config leaves its vdir behind, holding events - nothing manages any more. -- [ ] `--dry-run` skips the pull entirely, so it reports against whatever the last - real cycle left behind rather than against current remote state. -- [ ] A `TZID` is emitted without an accompanying `VTIMEZONE`. Tolerated by the servers - tested so far, and confined to recurring events, but not strictly conformant. +- [ ] A `pimsync sync` that fails takes the whole CalDAV leg with it. Unlike the + Google side this cannot be narrowed: pimsync is one process covering every + pair, so a failure does not say which pair it belongs to. ## M2 — interface and packaging diff --git a/src/cli.rs b/src/cli.rs index faffd2c..0dbcd93 100644 --- a/src/cli.rs +++ b/src/cli.rs @@ -47,6 +47,12 @@ pub enum Command { #[command(subcommand)] command: AggregateCommand, }, + /// Remove local mirrors of endpoints the configuration no longer names + Prune { + /// Delete them; without this the command only reports what it found + #[arg(long)] + force: bool, + }, /// Check that the environment and configuration are usable Doctor, } diff --git a/src/config.rs b/src/config.rs index 779f039..4e606d4 100644 --- a/src/config.rs +++ b/src/config.rs @@ -43,6 +43,13 @@ pub enum EndpointKind { calendar_id: String, client_id: String, client_secret_command: Option, + /// Which authorised Google account this calendar belongs to. + /// + /// One authorisation covers every calendar an account can see, so + /// several endpoints on one account share it and this can be left out. + /// It is only needed to say which account is meant when calcalist has + /// been logged in to more than one. + account: Option, }, Webcal { /// The feed URL, when it is not sensitive. @@ -68,6 +75,14 @@ impl EndpointKind { } } + /// Which authorised Google account this endpoint names, if it says. + pub fn google_account(&self) -> Option<&str> { + match self { + Self::Google { account, .. } => account.as_deref(), + _ => None, + } + } + pub fn scheduling_suppression(&self) -> SchedulingSuppression { match self { Self::Google { .. } => SchedulingSuppression::Native, diff --git a/src/doctor.rs b/src/doctor.rs index 228c4ea..348e0be 100644 --- a/src/doctor.rs +++ b/src/doctor.rs @@ -143,7 +143,12 @@ fn check_google_authorisation(config: &Config) -> Vec { let outcome = match auth::credentials_for(config, &endpoint.id) { Err(error) => Outcome::Fail(error.to_string()), Ok(credentials) => { - match auth::access_token(&state_dir, &endpoint.id, &credentials) { + match auth::access_token( + &state_dir, + &endpoint.id, + endpoint.kind.google_account(), + &credentials, + ) { Ok(_) => Outcome::Ok(format!("`{}` is authorised", endpoint.id)), Err(error @ (AuthError::NotAuthorised(_) | AuthError::Expired { .. })) => { Outcome::Warn(error.to_string()) diff --git a/src/google/api.rs b/src/google/api.rs index 29bfa54..03fd12e 100644 --- a/src/google/api.rs +++ b/src/google/api.rs @@ -161,6 +161,44 @@ impl Client { Self::event_id(&body) } + /// The occurrences of a series inside a window. + /// + /// Exceptions are addressed through the series rather than created + /// directly, so applying one means finding the instance it replaces. + pub fn instances( + &self, + master_id: &str, + time_min: &str, + time_max: &str, + ) -> Result, ApiError> { + let path = format!("events/{}/instances", percent_encode(master_id)); + let query = [ + ("maxResults".to_string(), PAGE_SIZE.to_string()), + ("timeMin".to_string(), time_min.to_string()), + ("timeMax".to_string(), time_max.to_string()), + ("showDeleted".to_string(), "false".to_string()), + ]; + let body = self.get(&path, &query)?; + let page: serde_json::Value = + serde_json::from_str(&body).map_err(|error| ApiError::Malformed(error.to_string()))?; + let items = page + .get("items") + .cloned() + .unwrap_or(serde_json::Value::Array(Vec::new())); + serde_json::from_value(items).map_err(|error| ApiError::Malformed(error.to_string())) + } + + /// Merges fields into an existing event, leaving the rest alone. + /// + /// An instance carries properties calcalist does not model — its + /// `recurringEventId` and `originalStartTime` above all, which say which + /// occurrence it replaces — so an override is patched rather than replaced. + pub fn patch(&self, id: &str, event: &serde_json::Value) -> Result<(), ApiError> { + let path = format!("events/{}", percent_encode(id)); + self.patch_json(&path, &SUPPRESS_NOTIFICATION, event)?; + Ok(()) + } + pub fn delete(&self, id: &str) -> Result<(), ApiError> { let path = format!("events/{}", percent_encode(id)); match self.delete_path(&path, &SUPPRESS_NOTIFICATION) { @@ -223,6 +261,26 @@ impl Client { ) } + fn patch_json( + &self, + path: &str, + query: &[(&str, &str)], + body: &serde_json::Value, + ) -> Result { + let mut request = self + .agent + .patch(&self.url(path)) + .header("Authorization", format!("Bearer {}", self.access_token)); + for (key, value) in query { + request = request.query(*key, *value); + } + Self::finish( + request + .send_json(body) + .map_err(|error| ApiError::Request(error.to_string()))?, + ) + } + fn delete_path(&self, path: &str, query: &[(&str, &str)]) -> Result { let mut request = self .agent @@ -358,7 +416,9 @@ pub struct PushReport { pub created: usize, pub updated: usize, pub deleted: usize, - /// Series whose exceptions could not be applied; see `push`. + /// Recurrence overrides applied to their instances. + pub exceptions_applied: usize, + /// Overrides whose instance Google did not offer; see `apply_exceptions`. pub exceptions_skipped: usize, } @@ -388,16 +448,11 @@ pub fn push( } }; // A series' exceptions are separate events against an existing series, - // so they cannot ride along with the master. Reported rather than - // silently dropped. - if item - .calendar - .properties("VEVENT", "RECURRENCE-ID") - .next() - .is_some() - { - report.exceptions_skipped += 1; - } + // so they cannot ride along with the master and are applied once it is + // in place. + let applied = apply_exceptions(client, &id, &item.calendar)?; + report.exceptions_applied += applied.applied; + report.exceptions_skipped += applied.skipped; state.events.insert( uid.clone(), GoogleItem { @@ -423,6 +478,46 @@ pub fn push( Ok(report) } +#[derive(Debug, Default)] +struct ExceptionReport { + applied: usize, + skipped: usize, +} + +/// Applies a stored series' recurrence overrides to the instances they replace. +/// +/// Google has no way to send a series and its exceptions in one request: an +/// exception is a separate event, addressed through the series and identified by +/// the occurrence it stands in for. So the master goes up first, then each +/// override is matched to an instance by original start time and patched. +/// +/// An override with no matching instance is counted rather than forced. That +/// happens when the occurrence falls outside the rule — a stale `RECURRENCE-ID` +/// left behind by an edited `RRULE` — and inventing an event for it would put +/// something in the calendar that the series does not contain. +fn apply_exceptions( + client: &Client, + master_id: &str, + calendar: &Calendar, +) -> Result { + let mut report = ExceptionReport::default(); + for exception in convert::exceptions(calendar) { + let Some(key) = convert::recurrence_key(&exception) else { + report.skipped += 1; + continue; + }; + let (time_min, time_max) = key.window(); + let instances = client.instances(master_id, &time_min, &time_max)?; + let Some(instance) = convert::find_instance(&instances, &key) else { + report.skipped += 1; + continue; + }; + client.patch(&instance.id, &convert::to_exception(&exception)?)?; + report.applied += 1; + } + Ok(report) +} + /// Percent-encodes a path segment; calendar ids contain `@` and event ids may /// contain characters that would otherwise change the URL's meaning. fn percent_encode(value: &str) -> String { diff --git a/src/google/auth.rs b/src/google/auth.rs index ac5045f..b7c92b4 100644 --- a/src/google/auth.rs +++ b/src/google/auth.rs @@ -74,6 +74,21 @@ pub enum AuthError { NoRefreshToken, #[error("endpoint `{0}` has not been authorised; run `calcalist google login {0}`")] NotAuthorised(String), + #[error( + "endpoint `{endpoint}` names account `{account}`, which has not been authorised; \ + run `calcalist google login {endpoint}`" + )] + UnknownAccount { endpoint: String, account: String }, + #[error( + "endpoint `{endpoint}` does not say which Google account it belongs to, and calcalist \ + is logged in to several ({accounts}). Add `account = \"…\"` to the endpoint." + )] + AmbiguousAccount { endpoint: String, accounts: String }, + #[error( + "the authorisation stored for account `{account}` was issued to a different OAuth \ + client than endpoint `{endpoint}` configures; run `calcalist google login {endpoint}`" + )] + ClientMismatch { endpoint: String, account: String }, #[error("could not access {path}: {source}")] Io { path: PathBuf, @@ -98,6 +113,15 @@ pub struct StoredToken { /// Unix seconds at which `access_token` stops being usable. #[serde(default)] pub expires_at: u64, + /// The account this authorisation belongs to. Absent in files written + /// before authorisations were shared between endpoints. + #[serde(default)] + pub account: Option, + /// The OAuth client the refresh token was issued to. A refresh token is + /// only valid for the client that obtained it, so a changed `client_id` + /// has to be caught here rather than as an opaque `invalid_grant`. + #[serde(default)] + pub client_id: Option, } impl StoredToken { @@ -109,11 +133,93 @@ impl StoredToken { } } -/// Where an endpoint's refresh token lives. -pub fn token_path(state_dir: &Path, endpoint_id: &str) -> PathBuf { +/// Where an authorisation lives, keyed by the account it was granted for. +/// +/// An authorisation covers every calendar the account can see, so two endpoints +/// on one account share this file and only one of them ever has to log in. +pub fn account_token_path(state_dir: &Path, account: &str) -> PathBuf { + state_dir + .join("google") + .join("accounts") + .join(format!("{}.json", sanitise(account))) +} + +/// Where an endpoint's refresh token used to live, before authorisations were +/// keyed by account. Still read, so an existing installation keeps working +/// until its next login moves it across. +pub fn legacy_token_path(state_dir: &Path, endpoint_id: &str) -> PathBuf { state_dir.join("google").join(format!("{endpoint_id}.json")) } +/// Keeps an account name to one harmless path segment. +fn sanitise(account: &str) -> String { + account + .chars() + .map(|ch| match ch { + 'a'..='z' | 'A'..='Z' | '0'..='9' | '-' | '_' | '.' | '@' => ch, + _ => '_', + }) + .collect() +} + +/// Every account calcalist currently holds an authorisation for. +pub fn authorised_accounts(state_dir: &Path) -> Vec { + let Ok(entries) = std::fs::read_dir(state_dir.join("google").join("accounts")) else { + return Vec::new(); + }; + let mut accounts: Vec = entries + .flatten() + .filter_map(|entry| load_token(&entry.path()).ok().flatten()) + .filter_map(|token| token.account) + .collect(); + accounts.sort(); + accounts.dedup(); + accounts +} + +/// Picks the stored authorisation an endpoint should use. +/// +/// The endpoint may name its account outright. Failing that, an authorisation +/// left over from the per-endpoint scheme is honoured, and then a single +/// account is taken to be the one meant — which is the ordinary case, and what +/// makes a second endpoint on the same account need no login of its own. +fn resolve_token( + state_dir: &Path, + endpoint_id: &str, + account: Option<&str>, +) -> Result<(PathBuf, StoredToken), AuthError> { + if let Some(account) = account { + let path = account_token_path(state_dir, account); + let stored = load_token(&path)?.ok_or_else(|| AuthError::UnknownAccount { + endpoint: endpoint_id.to_string(), + account: account.to_string(), + })?; + return Ok((path, stored)); + } + + let legacy = legacy_token_path(state_dir, endpoint_id); + if let Some(stored) = load_token(&legacy)? { + return Ok((legacy, stored)); + } + + let accounts = authorised_accounts(state_dir); + match accounts.as_slice() { + [] => Err(AuthError::NotAuthorised(endpoint_id.to_string())), + [only] => { + let path = account_token_path(state_dir, only); + let stored = load_token(&path)?.ok_or_else(|| AuthError::UnknownAccount { + endpoint: endpoint_id.to_string(), + account: only.clone(), + })?; + Ok((path, stored)) + } + several => Err(AuthError::AmbiguousAccount { + endpoint: endpoint_id.to_string(), + accounts: several.join(", "), + }), + } +} + /// Runs an external command and returns its first line of output. /// /// Secrets are fetched this way rather than stored, so the configuration file @@ -181,11 +287,18 @@ pub fn credentials_for( pub fn access_token( state_dir: &Path, endpoint_id: &str, + account: Option<&str>, credentials: &Credentials, ) -> Result { - let path = token_path(state_dir, endpoint_id); - let mut stored = - load_token(&path)?.ok_or_else(|| AuthError::NotAuthorised(endpoint_id.to_string()))?; + let (path, mut stored) = resolve_token(state_dir, endpoint_id, account)?; + if let Some(issued_to) = &stored.client_id + && *issued_to != credentials.client_id + { + return Err(AuthError::ClientMismatch { + endpoint: endpoint_id.to_string(), + account: stored.account.clone().unwrap_or_else(|| "?".to_string()), + }); + } if let Some(token) = stored.usable_access_token() { return Ok(token.to_string()); @@ -248,19 +361,66 @@ pub fn login( ])?; let refresh_token = issued.refresh_token.ok_or(AuthError::NoRefreshToken)?; - let path = token_path(state_dir, endpoint_id); + // Which account consented is not in the token response, but the primary + // calendar's id is the account's own address — and reading it needs nothing + // beyond the calendar scope already granted, unlike the userinfo endpoint. + let account = primary_calendar_id(&issued.access_token); + let path = match &account { + Some(account) => account_token_path(state_dir, account), + None => legacy_token_path(state_dir, endpoint_id), + }; save_token( &path, &StoredToken { refresh_token, access_token: Some(issued.access_token), expires_at: unix_now() + issued.expires_in, + account: account.clone(), + client_id: Some(credentials.client_id.clone()), }, )?; - println!("Authorised. Refresh token stored at {}", path.display()); + // The authorisation now lives under the account, so the copy this endpoint + // used to keep would only shadow it. + let legacy = legacy_token_path(state_dir, endpoint_id); + if account.is_some() && legacy.exists() { + let _ = std::fs::remove_file(&legacy); + } + match &account { + Some(account) => println!( + "Authorised {account}. Every endpoint on this account can use it; \ + the refresh token is stored at {}", + path.display() + ), + None => println!("Authorised. Refresh token stored at {}", path.display()), + } Ok(()) } +/// The address of the authorised account, read from its primary calendar. +/// +/// Returned as an option rather than an error: failing to name the account only +/// costs the sharing, and the authorisation itself is perfectly good without it. +fn primary_calendar_id(access_token: &str) -> Option { + let agent: ureq::Agent = ureq::Agent::config_builder() + .http_status_as_error(false) + .build() + .into(); + let mut response = agent + .get("https://www.googleapis.com/calendar/v3/calendars/primary") + .header("Authorization", format!("Bearer {access_token}")) + .call() + .ok()?; + if !(200..300).contains(&response.status().as_u16()) { + return None; + } + let body = response.body_mut().read_to_string().ok()?; + let value: serde_json::Value = serde_json::from_str(&body).ok()?; + value + .get("id") + .and_then(|id| id.as_str()) + .map(str::to_string) +} + /// Accepts one loopback request and extracts the authorisation code from it. fn wait_for_code(listener: &TcpListener, expected_state: &str) -> Result { let (mut stream, _) = listener.accept().map_err(AuthError::Listen)?; @@ -586,6 +746,8 @@ mod tests { refresh_token: "r".into(), access_token: Some("a".into()), expires_at: unix_now() + 3600, + account: None, + client_id: None, }; assert_eq!(fresh.usable_access_token(), Some("a")); @@ -602,4 +764,101 @@ mod tests { }; assert_eq!(nearly.usable_access_token(), None); } + + fn stored(account: &str, client_id: &str) -> StoredToken { + StoredToken { + refresh_token: format!("refresh-for-{account}"), + access_token: None, + expires_at: 0, + account: Some(account.to_string()), + client_id: Some(client_id.to_string()), + } + } + + /// The gap this closes: two endpoints on one account needed a login each, + /// because the authorisation was filed under the endpoint rather than the + /// account it actually belongs to. + #[test] + fn one_authorisation_serves_every_endpoint_on_that_account() { + let dir = tempfile::tempdir().expect("temp"); + save_token( + &account_token_path(dir.path(), "me@example.com"), + &stored("me@example.com", "client-a"), + ) + .expect("save"); + + for endpoint in ["work", "personal", "never-logged-in"] { + let (_, token) = resolve_token(dir.path(), endpoint, None).expect("resolve"); + assert_eq!(token.refresh_token, "refresh-for-me@example.com"); + } + } + + /// Two accounts cannot be guessed between, so the endpoint has to say. + #[test] + fn several_accounts_have_to_be_told_apart() { + let dir = tempfile::tempdir().expect("temp"); + for account in ["one@example.com", "two@example.com"] { + save_token( + &account_token_path(dir.path(), account), + &stored(account, "client-a"), + ) + .expect("save"); + } + + let error = resolve_token(dir.path(), "work", None).expect_err("ambiguous"); + assert!( + matches!(error, AuthError::AmbiguousAccount { .. }), + "{error:?}" + ); + assert!(error.to_string().contains("one@example.com"), "{error}"); + + let (_, token) = + resolve_token(dir.path(), "work", Some("two@example.com")).expect("resolve"); + assert_eq!(token.refresh_token, "refresh-for-two@example.com"); + } + + /// An authorisation written by an earlier version keeps working untouched. + #[test] + fn an_authorisation_from_the_old_layout_is_still_honoured() { + let dir = tempfile::tempdir().expect("temp"); + save_token( + &legacy_token_path(dir.path(), "work"), + &StoredToken { + refresh_token: "old".into(), + access_token: None, + expires_at: 0, + account: None, + client_id: None, + }, + ) + .expect("save"); + + let (_, token) = resolve_token(dir.path(), "work", None).expect("resolve"); + assert_eq!(token.refresh_token, "old"); + // A different endpoint has nothing to fall back on. + let error = resolve_token(dir.path(), "other", None).expect_err("not authorised"); + assert!(matches!(error, AuthError::NotAuthorised(_)), "{error:?}"); + } + + /// A refresh token only works for the client it was issued to, so a changed + /// client_id has to say so rather than surface as an opaque invalid_grant. + #[test] + fn a_token_from_another_oauth_client_is_refused_by_name() { + let dir = tempfile::tempdir().expect("temp"); + save_token( + &account_token_path(dir.path(), "me@example.com"), + &stored("me@example.com", "client-a"), + ) + .expect("save"); + + let credentials = Credentials { + client_id: "client-b".into(), + client_secret: "secret".into(), + }; + let error = access_token(dir.path(), "work", None, &credentials).expect_err("mismatch"); + assert!( + matches!(error, AuthError::ClientMismatch { .. }), + "{error:?}" + ); + } } diff --git a/src/google/convert.rs b/src/google/convert.rs index e69ec99..4543f32 100644 --- a/src/google/convert.rs +++ b/src/google/convert.rs @@ -14,9 +14,9 @@ //! * An exception that was deleted comes back as an override with //! `status: cancelled`, which is an `EXDATE` on the master rather than an event. -use std::collections::BTreeMap; +use std::collections::{BTreeMap, BTreeSet}; -use jiff::{Timestamp, tz::TimeZone}; +use jiff::{SignedDuration, Timestamp, tz::Dst, tz::TimeZone}; use serde::Deserialize; use thiserror::Error; @@ -157,34 +157,186 @@ pub fn to_ical(uid: &str, group: &Group) -> Result { }); }; - let mut out = String::from("BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:-//calcalist//EN\r\n"); - out.push_str(&render_event(uid, master, None)?); + let mut zones = BTreeSet::new(); + let mut body = render_event(uid, master, None, &mut zones)?; // A deleted occurrence is an absence, not an event: it belongs on the master // as an EXDATE. for cancelled in group.overrides.iter().filter(|event| event.is_cancelled()) { if let Some(start) = &cancelled.original_start_time { - let line = time_property("EXDATE", start, &cancelled.id, master.is_recurring())?; + let line = time_property( + "EXDATE", + start, + &cancelled.id, + master.is_recurring(), + &mut zones, + )?; // Insert before the master's END:VEVENT. - let end = out.rfind("END:VEVENT\r\n").unwrap_or(out.len()); - out.insert_str(end, &format!("{line}\r\n")); + let end = body.rfind("END:VEVENT\r\n").unwrap_or(body.len()); + body.insert_str(end, &format!("{line}\r\n")); } } for moved in group.overrides.iter().filter(|event| !event.is_cancelled()) { - out.push_str(&render_event( + body.push_str(&render_event( uid, moved, moved.original_start_time.as_ref(), + &mut zones, )?); } + + let mut out = String::from("BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:-//calcalist//EN\r\n"); + // A TZID names a zone the reader is expected to resolve, but RFC 5545 wants + // the definition to travel with the calendar: a client with no zone database + // of its own has nothing else to expand the recurrence against. + let reference = master + .start + .as_ref() + .and_then(instant_of) + .unwrap_or_else(Timestamp::now); + for zone in &zones { + if let Some(block) = vtimezone(zone, reference) { + out.push_str(&block); + } + } + out.push_str(&body); out.push_str("END:VCALENDAR\r\n"); Ok(out) } +/// The instant an event time names, whether it is dated or timed. +fn instant_of(time: &EventTime) -> Option { + match (&time.date, &time.date_time) { + (_, Some(value)) => value.parse().ok(), + (Some(date), None) => midnight_utc(date), + (None, None) => None, + } +} + +/// An RFC 5545 `VTIMEZONE` for `name`, describing the rules around `reference`. +/// +/// Two observances describe every zone in current use — one standard, one +/// daylight — each recurring yearly on the same weekday of the same month. A +/// zone that does not shift gets a single fixed observance instead, which is +/// also the fallback when the transition table says nothing about this period. +fn vtimezone(name: &str, reference: Timestamp) -> Option { + let zone = TimeZone::get(name).ok()?; + // Start a little over a year back, so a full cycle of transitions is in + // view however close the event sits to one of them. + let from = reference + .checked_sub(SignedDuration::from_hours(24 * 400)) + .ok()?; + let upcoming: Vec<_> = zone.following(from).take(4).collect(); + + let mut out = format!("BEGIN:VTIMEZONE\r\nTZID:{name}\r\n"); + let daylight = upcoming + .iter() + .find(|transition| transition.dst() == Dst::Yes); + let standard = upcoming + .iter() + .find(|transition| transition.dst() == Dst::No); + + match (daylight, standard) { + (Some(daylight), Some(standard)) => { + out.push_str(&observance(&zone, daylight, "DAYLIGHT")?); + out.push_str(&observance(&zone, standard, "STANDARD")?); + } + _ => { + let offset = zone.to_offset(reference); + let text = format_offset(offset); + out.push_str(&format!( + "BEGIN:STANDARD\r\nDTSTART:19700101T000000\r\nTZOFFSETFROM:{text}\r\n\ + TZOFFSETTO:{text}\r\nEND:STANDARD\r\n" + )); + } + } + out.push_str("END:VTIMEZONE\r\n"); + Some(out) +} + +/// One `STANDARD` or `DAYLIGHT` block, as the yearly rule that produced it. +fn observance( + zone: &TimeZone, + transition: &jiff::tz::TimeZoneTransition<'_>, + component: &str, +) -> Option { + let to = transition.offset(); + // The offset in force a moment earlier is the one the shift moved away from. + let before = transition + .timestamp() + .checked_sub(SignedDuration::from_secs(1)) + .ok()?; + let from = zone.to_offset(before); + // A transition's local start is written in the *old* offset, since that is + // the clock a reader is still on when the rule fires. + let local = transition + .timestamp() + .to_zoned(TimeZone::fixed(from)) + .datetime(); + + let mut block = format!( + "BEGIN:{component}\r\nDTSTART:{}\r\nTZOFFSETFROM:{}\r\nTZOFFSETTO:{}\r\n\ + RRULE:{}\r\n", + local.strftime("%Y%m%dT%H%M%S"), + format_offset(from), + format_offset(to), + yearly_rule(local) + ); + let abbreviation = transition.abbreviation(); + if !abbreviation.is_empty() { + block.push_str(&format!("TZNAME:{abbreviation}\r\n")); + } + block.push_str(&format!("END:{component}\r\n")); + Some(block) +} + +/// The yearly recurrence a transition date implies, as "the nth weekday of the +/// month" — the form every zone's rules are actually written in. A date within +/// seven days of the month's end is the *last* such weekday rather than a fixed +/// ordinal, which is what keeps the rule stable across years of differing length. +fn yearly_rule(local: jiff::civil::DateTime) -> String { + let day = local.day(); + let ordinal = if day + 7 > local.days_in_month() { + -1 + } else { + (day - 1) / 7 + 1 + }; + format!( + "FREQ=YEARLY;BYMONTH={};BYDAY={ordinal}{}", + local.month(), + weekday_code(local.weekday()) + ) +} + +fn weekday_code(weekday: jiff::civil::Weekday) -> &'static str { + match weekday { + jiff::civil::Weekday::Monday => "MO", + jiff::civil::Weekday::Tuesday => "TU", + jiff::civil::Weekday::Wednesday => "WE", + jiff::civil::Weekday::Thursday => "TH", + jiff::civil::Weekday::Friday => "FR", + jiff::civil::Weekday::Saturday => "SA", + jiff::civil::Weekday::Sunday => "SU", + } +} + +/// `+0200` — RFC 5545's UTC offset form, which has no colon. +fn format_offset(offset: jiff::tz::Offset) -> String { + let total = offset.seconds(); + let sign = if total < 0 { '-' } else { '+' }; + let magnitude = total.abs(); + format!( + "{sign}{:02}{:02}", + magnitude / 3600, + (magnitude % 3600) / 60 + ) +} + fn render_event( uid: &str, event: &Event, recurrence_id: Option<&EventTime>, + zones: &mut BTreeSet, ) -> Result { let recurring = event.is_recurring() || recurrence_id.is_some(); let mut out = String::from("BEGIN:VEVENT\r\n"); @@ -197,16 +349,19 @@ fn render_event( original, &event.id, recurring, + zones, )?); out.push_str("\r\n"); } let start = event.start.as_ref().ok_or_else(|| ConvertError::NoStart { id: event.id.clone(), })?; - out.push_str(&time_property("DTSTART", start, &event.id, recurring)?); + out.push_str(&time_property( + "DTSTART", start, &event.id, recurring, zones, + )?); out.push_str("\r\n"); if let Some(end) = &event.end { - out.push_str(&time_property("DTEND", end, &event.id, recurring)?); + out.push_str(&time_property("DTEND", end, &event.id, recurring, zones)?); out.push_str("\r\n"); } for line in event.recurrence.iter().flatten() { @@ -269,6 +424,7 @@ fn time_property( time: &EventTime, id: &str, recurring: bool, + zones: &mut BTreeSet, ) -> Result { if let Some(date) = &time.date { return Ok(format!("{name};VALUE=DATE:{}", date.replace('-', ""))); @@ -292,6 +448,7 @@ fn time_property( match zone { Some((zone_name, zone)) => { + zones.insert(zone_name.to_string()); let local = instant.to_zoned(zone); Ok(format!( "{name};TZID={zone_name}:{}", @@ -319,18 +476,61 @@ fn stamp(event: &Event) -> Result { Ok(instant.strftime("%Y%m%dT%H%M%SZ").to_string()) } +/// Which half of a series a body describes. +/// +/// Google models an exception as a separate event addressed through the series +/// it belongs to, so it carries neither the series' `iCalUID` — which would +/// name the master — nor a recurrence rule of its own. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum Role { + Master, + Exception, +} + /// Renders a stored calendar as the JSON body Google expects. /// /// Only the master is described. A series' exceptions cannot be expressed in the -/// same request — Google models them as separate events against an already -/// existing series — so the caller applies those afterwards. +/// same request, so the caller applies those afterwards with `to_exception`. pub fn to_google(calendar: &Calendar) -> Result { - let uid = calendar.uid().ok_or_else(|| ConvertError::NoUid { + let master = master_of(calendar).ok_or_else(|| ConvertError::NoUid { id: "".to_string(), })?; + to_body(&master, Role::Master) +} +/// The master event of a stored file: the one component without a `RECURRENCE-ID`. +/// +/// Falls back to the file as a whole when it holds no `VEVENT` at all, so a +/// malformed item still reports through the ordinary error path. +pub fn master_of(calendar: &Calendar) -> Option { + calendar + .events() + .into_iter() + .find(|event| event.properties("VEVENT", "RECURRENCE-ID").next().is_none()) +} + +/// Every recurrence override in a stored file, in the order they appear. +pub fn exceptions(calendar: &Calendar) -> Vec { + calendar + .events() + .into_iter() + .filter(|event| event.properties("VEVENT", "RECURRENCE-ID").next().is_some()) + .collect() +} + +/// Renders one recurrence override as a patch against its Google instance. +pub fn to_exception(event: &Calendar) -> Result { + to_body(event, Role::Exception) +} + +fn to_body(calendar: &Calendar, role: Role) -> Result { let mut event = serde_json::Map::new(); - event.insert("iCalUID".into(), uid.into()); + if role == Role::Master { + let uid = calendar.uid().ok_or_else(|| ConvertError::NoUid { + id: "".to_string(), + })?; + event.insert("iCalUID".into(), uid.into()); + } event.insert("status".into(), status_of(calendar).into()); for (property, field) in [ @@ -356,6 +556,7 @@ pub fn to_google(calendar: &Calendar) -> Result // verbatim. Their unfolded form is what Google expects. let recurrence: Vec = ["RRULE", "EXDATE", "RDATE"] .iter() + .filter(|_| role == Role::Master) .flat_map(|name| calendar.property_lines("VEVENT", name)) .map(|line| serde_json::Value::from(line.to_string())) .collect(); @@ -517,6 +718,83 @@ fn google_time(calendar: &Calendar, name: &str) -> Result (String, String) { + let centre = match self { + InstanceKey::Instant(instant) => *instant, + InstanceKey::Date(date) => midnight_utc(date).unwrap_or_else(Timestamp::now), + }; + let margin = jiff::SignedDuration::from_hours(36); + ((centre - margin).to_string(), (centre + margin).to_string()) + } +} + +fn midnight_utc(date: &str) -> Option { + let date: jiff::civil::Date = date.parse().ok()?; + date.to_zoned(TimeZone::UTC) + .ok() + .map(|zoned| zoned.timestamp()) +} + +/// The occurrence a stored recurrence override refers to. +pub fn recurrence_key(event: &Calendar) -> Option { + let property = event.properties("VEVENT", "RECURRENCE-ID").next()?; + let value = property.value; + if property.param("VALUE") == Some("DATE") || value.len() == 8 { + return Some(InstanceKey::Date(hyphenate(value))); + } + let civil = parse_basic(value, "RECURRENCE-ID").ok()?; + let zone = match property.param("TZID") { + Some(name) => TimeZone::get(name).ok()?, + None => TimeZone::UTC, + }; + civil + .to_zoned(zone) + .ok() + .map(|zoned| InstanceKey::Instant(zoned.timestamp())) +} + +/// The occurrence a Google override refers to. +pub fn original_start_key(time: &EventTime) -> Option { + if let Some(date) = &time.date { + return Some(InstanceKey::Date(date.clone())); + } + time.date_time + .as_deref()? + .parse() + .ok() + .map(InstanceKey::Instant) +} + +/// Picks the instance an override belongs to out of a page of them. +pub fn find_instance<'a>(instances: &'a [Event], key: &InstanceKey) -> Option<&'a Event> { + instances.iter().find(|instance| { + instance + .original_start_time + .as_ref() + .and_then(original_start_key) + .is_some_and(|candidate| candidate == *key) + }) +} + /// Parses `YYYYMMDDTHHMMSS`, with or without a trailing Z. fn parse_basic(value: &str, field: &'static str) -> Result { let trimmed = value.trim_end_matches('Z'); @@ -860,4 +1138,154 @@ mod tests { "{ics}" ); } + + /// A zone that shifts gets both observances, each as the yearly rule the + /// transition implies: the EU moves on the last Sunday of March and October. + #[test] + fn a_zoned_recurrence_carries_its_timezone_definition() { + let json = r#"[{ + "id": "berlin", "status": "confirmed", "summary": "Standup", + "start": {"dateTime": "2026-03-03T09:00:00+01:00", "timeZone": "Europe/Berlin"}, + "end": {"dateTime": "2026-03-03T09:30:00+01:00", "timeZone": "Europe/Berlin"}, + "recurrence": ["RRULE:FREQ=WEEKLY;BYDAY=TU"], + "iCalUID": "berlin@google.com", "updated": "2026-01-01T00:00:00.000Z" + }]"#; + let groups = group_by_uid(parse(json)); + let ics = to_ical("berlin@google.com", &groups["berlin@google.com"]).expect("convert"); + + assert!( + ics.contains("BEGIN:VTIMEZONE\r\nTZID:Europe/Berlin\r\n"), + "{ics}" + ); + assert!(ics.contains("BEGIN:DAYLIGHT"), "{ics}"); + assert!( + ics.contains("TZOFFSETFROM:+0100\r\nTZOFFSETTO:+0200"), + "{ics}" + ); + assert!( + ics.contains("RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU"), + "{ics}" + ); + assert!(ics.contains("BEGIN:STANDARD"), "{ics}"); + assert!( + ics.contains("TZOFFSETFROM:+0200\r\nTZOFFSETTO:+0100"), + "{ics}" + ); + assert!( + ics.contains("RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU"), + "{ics}" + ); + // The definition has to precede the event that references it. + assert!( + ics.find("BEGIN:VTIMEZONE") < ics.find("BEGIN:VEVENT"), + "{ics}" + ); + assert!(Calendar::parse(&ics).is_ok(), "{ics}"); + } + + /// Pakistan abandoned daylight saving in 2009, so there is no rule to state — + /// only the offset that has applied ever since. + #[test] + fn a_zone_that_never_shifts_gets_one_fixed_observance() { + let groups = group_by_uid(parse(KARACHI_MASTER)); + let ics = to_ical("master1@google.com", &groups["master1@google.com"]).expect("convert"); + assert!(ics.contains("TZID:Asia/Karachi"), "{ics}"); + assert!( + ics.contains("TZOFFSETFROM:+0500\r\nTZOFFSETTO:+0500"), + "{ics}" + ); + assert!(!ics.contains("BEGIN:DAYLIGHT"), "{ics}"); + } + + /// A one-off is written in UTC, so it names no zone and needs no definition. + #[test] + fn an_unzoned_event_carries_no_timezone_definition() { + let json = r#"[{ + "id": "one", "status": "confirmed", "summary": "Once", + "start": {"dateTime": "2026-04-01T10:00:00+02:00", "timeZone": "Europe/Berlin"}, + "end": {"dateTime": "2026-04-01T11:00:00+02:00", "timeZone": "Europe/Berlin"}, + "iCalUID": "one@google.com", "updated": "2026-01-01T00:00:00.000Z" + }]"#; + let groups = group_by_uid(parse(json)); + let ics = to_ical("one@google.com", &groups["one@google.com"]).expect("convert"); + assert!(!ics.contains("VTIMEZONE"), "{ics}"); + } + + /// The master's rule must not leak into the override's body, or Google would + /// read the exception as a series of its own. + #[test] + fn an_exception_body_carries_no_recurrence_and_no_uid() { + let ics = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\n\ + BEGIN:VEVENT\r\nUID:series@calcalist\r\nDTSTAMP:20260101T000000Z\r\n\ + DTSTART;TZID=Europe/Berlin:20260303T090000\r\nRRULE:FREQ=WEEKLY;BYDAY=TU\r\n\ + SUMMARY:Standup\r\nEND:VEVENT\r\n\ + BEGIN:VEVENT\r\nUID:series@calcalist\r\nDTSTAMP:20260101T000000Z\r\n\ + RECURRENCE-ID;TZID=Europe/Berlin:20260310T090000\r\n\ + DTSTART;TZID=Europe/Berlin:20260310T110000\r\n\ + SUMMARY:Standup, moved\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n"; + let calendar = Calendar::parse(ics).expect("parse"); + + let exceptions = exceptions(&calendar); + assert_eq!(exceptions.len(), 1); + let body = to_exception(&exceptions[0]).expect("convert"); + assert!(body.get("recurrence").is_none(), "{body}"); + assert!(body.get("iCalUID").is_none(), "{body}"); + assert_eq!(body["summary"], "Standup, moved"); + assert_eq!(body["start"]["dateTime"], "2026-03-10T10:00:00Z"); + + // The master keeps its own summary rather than the override's. + let master = to_google(&calendar).expect("convert"); + assert_eq!(master["summary"], "Standup"); + assert_eq!(master["iCalUID"], "series@calcalist"); + assert_eq!(master["recurrence"][0], "RRULE:FREQ=WEEKLY;BYDAY=TU"); + } + + /// RECURRENCE-ID and originalStartTime spell the same instant differently: + /// one as a zoned local time, the other as an absolute offset. + #[test] + fn an_override_matches_its_instance_across_the_two_spellings() { + let ics = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\n\ + BEGIN:VEVENT\r\nUID:s@calcalist\r\nDTSTAMP:20260101T000000Z\r\n\ + RECURRENCE-ID;TZID=Europe/Berlin:20260310T090000\r\n\ + DTSTART;TZID=Europe/Berlin:20260310T110000\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n"; + let calendar = Calendar::parse(ics).expect("parse"); + let key = recurrence_key(&calendar).expect("a recurrence id"); + + let instances = parse( + r#"[ + {"id": "abc_20260303T080000Z", "status": "confirmed", + "start": {"dateTime": "2026-03-03T09:00:00+01:00"}, + "originalStartTime": {"dateTime": "2026-03-03T09:00:00+01:00"}}, + {"id": "abc_20260310T080000Z", "status": "confirmed", + "start": {"dateTime": "2026-03-10T11:00:00+01:00"}, + "originalStartTime": {"dateTime": "2026-03-10T09:00:00+01:00"}} + ]"#, + ); + let found = find_instance(&instances, &key).expect("a matching instance"); + assert_eq!(found.id, "abc_20260310T080000Z"); + + // The window has to contain the occurrence it was built for. + let (min, max) = key.window(); + assert!(min.as_str() < "2026-03-10T08:00:00Z", "{min}"); + assert!(max.as_str() > "2026-03-10T08:00:00Z", "{max}"); + } + + /// An all-day series identifies its occurrences by date on both sides. + #[test] + fn an_all_day_override_matches_by_date() { + let ics = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\n\ + BEGIN:VEVENT\r\nUID:s@calcalist\r\nDTSTAMP:20260101T000000Z\r\n\ + RECURRENCE-ID;VALUE=DATE:20260310\r\n\ + DTSTART;VALUE=DATE:20260311\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n"; + let calendar = Calendar::parse(ics).expect("parse"); + let key = recurrence_key(&calendar).expect("a recurrence id"); + assert_eq!(key, InstanceKey::Date("2026-03-10".into())); + + let instances = parse( + r#"[{"id": "abc_20260310", "status": "confirmed", + "start": {"date": "2026-03-11"}, + "originalStartTime": {"date": "2026-03-10"}}]"#, + ); + assert!(find_instance(&instances, &key).is_some()); + } } diff --git a/src/ical.rs b/src/ical.rs index 78c188b..89524c3 100644 --- a/src/ical.rs +++ b/src/ical.rs @@ -134,15 +134,62 @@ impl Calendar { pub fn to_ics(&self) -> String { let mut out = String::with_capacity(self.source.len() + 64); for line in &self.lines { - match &line.body { - Body::Original(span) => out.push_str(&self.source[span.clone()]), - Body::Generated(text) => out.push_str(&fold(text, &self.newline)), - } + out.push_str(&self.rendered(line)); out.push_str(&self.newline); } out } + /// One line as it will be emitted: verbatim when untouched, folded when built. + fn rendered(&self, line: &Line) -> String { + match &line.body { + Body::Original(span) => self.source[span.clone()].to_string(), + Body::Generated(text) => fold(text, &self.newline), + } + } + + /// Each `VEVENT` as a calendar of its own, nested components included. + /// + /// `properties` deliberately flattens every component of a name together, + /// which is right for the UID a series shares but wrong for a recurrence + /// override: there its `SUMMARY` and the master's are indistinguishable. + /// Splitting gives each component a view of its own. It re-parses, unlike + /// the rest of this module, which is acceptable because the only caller + /// converts to another model rather than round-tripping back to iCalendar. + pub fn events(&self) -> Vec { + let newline = &self.newline; + let mut out = Vec::new(); + let mut block: Option = None; + for line in &self.lines { + let Some(depth) = line + .path + .iter() + .position(|name| name.eq_ignore_ascii_case("VEVENT")) + else { + continue; + }; + let outermost = depth + 1 == line.path.len(); + if outermost && line.has_name("BEGIN") { + block = Some(String::new()); + } + let Some(text) = block.as_mut() else { + continue; + }; + text.push_str(&self.rendered(line)); + text.push_str(newline); + if outermost + && line.has_name("END") + && let Some(text) = block.take() + && let Ok(event) = Calendar::parse(&format!( + "BEGIN:VCALENDAR{newline}VERSION:2.0{newline}{text}END:VCALENDAR{newline}" + )) + { + out.push(event); + } + } + out + } + /// The UID shared by the event and any recurrence overrides. pub fn uid(&self) -> Option<&str> { self.properties("VEVENT", "UID").next().map(|p| p.value) @@ -678,4 +725,47 @@ mod tests { IcalError::DanglingContinuation { line: 1 } ); } + + /// Splitting has to keep each component's nested parts with it, and give a + /// master and its override separate views of the properties they share. + #[test] + fn events_are_split_with_their_nested_components() { + let ics = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\n\ + BEGIN:VEVENT\r\nUID:s@x\r\nSUMMARY:Series\r\n\ + BEGIN:VALARM\r\nACTION:DISPLAY\r\nTRIGGER:-PT10M\r\nEND:VALARM\r\n\ + END:VEVENT\r\n\ + BEGIN:VEVENT\r\nUID:s@x\r\nRECURRENCE-ID:20260310T090000Z\r\n\ + SUMMARY:Moved\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n"; + let calendar = Calendar::parse(ics).expect("parse"); + + // Flattened, the two summaries are indistinguishable. + assert_eq!(calendar.properties("VEVENT", "SUMMARY").count(), 2); + + let events = calendar.events(); + assert_eq!(events.len(), 2); + assert_eq!( + events[0] + .properties("VEVENT", "SUMMARY") + .next() + .unwrap() + .value, + "Series" + ); + assert_eq!(events[0].properties("VALARM", "TRIGGER").count(), 1); + assert!( + events[0] + .properties("VEVENT", "RECURRENCE-ID") + .next() + .is_none() + ); + assert_eq!( + events[1] + .properties("VEVENT", "SUMMARY") + .next() + .unwrap() + .value, + "Moved" + ); + assert_eq!(events[1].properties("VALARM", "TRIGGER").count(), 0); + } } diff --git a/src/main.rs b/src/main.rs index 3bb5c2f..bb23631 100644 --- a/src/main.rs +++ b/src/main.rs @@ -39,6 +39,10 @@ fn main() -> ExitCode { command: Command::Sync { dry_run, force }, .. } => run_sync(cli, *dry_run, *force), + cli @ Cli { + command: Command::Prune { force }, + .. + } => run_prune(cli, *force), cli @ Cli { command: Command::Google { @@ -84,12 +88,53 @@ fn run_sync(cli: &Cli, dry_run: bool, force: bool) -> ExitCode { match sync::run(&config, &state_dir, dry_run, force) { Ok(report) => { print_report(&report); - ExitCode::SUCCESS + // A cycle that could not reach an endpoint did real work, but saying + // so only in passing would let a lapsed token go unnoticed until the + // calendars had drifted a long way apart. + if report.is_complete() { + ExitCode::SUCCESS + } else { + ExitCode::FAILURE + } } Err(error) => fail(&error), } } +/// Removes local mirrors of endpoints the configuration no longer names. +fn run_prune(cli: &Cli, force: bool) -> ExitCode { + let (config, _) = match Config::load(cli.config.as_deref()) { + Ok(loaded) => loaded, + Err(error) => return fail(&error), + }; + let state_dir = match paths::state_dir() { + Ok(dir) => dir, + Err(error) => return fail(&error), + }; + let orphans = sync::orphan_vdirs(&config, &state_dir); + if orphans.is_empty() { + println!("nothing to prune: every local mirror belongs to a configured endpoint"); + return ExitCode::SUCCESS; + } + for orphan in &orphans { + if !force { + println!("would remove {}", orphan.display()); + continue; + } + match std::fs::remove_dir_all(orphan) { + Ok(()) => println!("removed {}", orphan.display()), + Err(error) => { + eprintln!("calcalist: could not remove {}: {error}", orphan.display()); + return ExitCode::FAILURE; + } + } + } + if !force { + println!("re-run with --force to remove them"); + } + ExitCode::SUCCESS +} + /// Authorises one Google endpoint, storing a refresh token for later cycles. fn run_google_login(cli: &Cli, endpoint_id: &str) -> ExitCode { let (config, _) = match Config::load(cli.config.as_deref()) { @@ -162,6 +207,27 @@ fn run_retarget(cli: &Cli, id: &str, to: &str, purge_old: bool) -> ExitCode { } fn print_report(report: &Report) { + for failure in &report.failures { + println!("{}: unavailable — {}", failure.endpoint, failure.reason); + } + for blocked in &report.blocked { + println!( + "{}: left alone; {} could not be reached this cycle", + blocked.id, + blocked + .endpoints + .iter() + .map(|id| format!("`{id}`")) + .collect::>() + .join(", ") + ); + } + for orphan in &report.orphan_vdirs { + println!( + "note: {} mirrors an endpoint that is no longer configured; `calcalist prune` removes it", + orphan.display() + ); + } for google in &report.google { let resync = if google.full_resync { " (Google rejected the sync cursor, so everything was refetched)" @@ -177,9 +243,15 @@ fn print_report(report: &Report) { google.updated_remotely, google.deleted_remotely, ); + if google.exceptions_applied > 0 { + println!( + " {} recurrence override(s) applied to their instances", + google.exceptions_applied + ); + } if google.exceptions_skipped > 0 { println!( - " {} recurring series had exceptions that were not pushed; Google models those as separate events against an existing series", + " {} recurrence override(s) matched no occurrence of their series and were left alone", google.exceptions_skipped ); } @@ -208,7 +280,7 @@ fn print_report(report: &Report) { } } if report.dry_run { - println!("(dry run — nothing was written)"); + println!("(dry run — pulled into a throwaway copy; nothing was written)"); } } @@ -257,6 +329,7 @@ fn unimplemented(command: &Command) -> ExitCode { Command::Serve { .. } => "serve", Command::Google { .. } => "google", Command::Aggregate { .. } => "aggregate", + Command::Prune { .. } => "prune", Command::Doctor => "doctor", }; eprintln!("calcalist: `{name}` is not implemented yet"); diff --git a/src/pimsync.rs b/src/pimsync.rs index 3af46f9..5731345 100644 --- a/src/pimsync.rs +++ b/src/pimsync.rs @@ -106,16 +106,31 @@ pub fn config_path(state_dir: &Path) -> PathBuf { state_dir.join("pimsync.conf") } -fn status_path(state_dir: &Path) -> PathBuf { +pub(crate) fn status_path(state_dir: &Path) -> PathBuf { state_dir.join("pimsync-status") } +/// Which way a generated configuration lets items travel. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Direction { + /// The ordinary cycle: remote and local converge on each other. + Both, + /// Remote to local only, for `--dry-run`. pimsync writes nothing to any + /// server in this mode — but it does delete local items the remote has + /// never seen, so it is only ever pointed at a throwaway copy of the vdirs. + PullOnly, +} + /// Builds a pimsync configuration covering every CalDAV and WebCal endpoint. /// /// Google endpoints are absent by design: pimsync has no REST storage, and its /// CalDAV storage speaks only HTTP Basic auth, which Google's endpoint has /// rejected since March 2025. calcalist syncs those itself. -pub fn generate(config: &Config, state_dir: &Path) -> Result { +pub fn generate_for( + config: &Config, + state_dir: &Path, + direction: Direction, +) -> Result { let mut out = String::new(); out.push_str("# Generated by calcalist. Edits here are overwritten on every sync.\n\n"); out.push_str(&format!( @@ -129,7 +144,7 @@ pub fn generate(config: &Config, state_dir: &Path) -> Result out.push_str(&caldav_pair(endpoint)?), + EndpointKind::Caldav { .. } => out.push_str(&caldav_pair(endpoint, direction)?), EndpointKind::Webcal { url, url_command } => { out.push_str(&webcal_pair( endpoint, @@ -144,7 +159,7 @@ pub fn generate(config: &Config, state_dir: &Path) -> Result Result { +fn caldav_pair(endpoint: &Endpoint, direction: Direction) -> Result { let EndpointKind::Caldav { url, username, @@ -175,10 +190,15 @@ fn caldav_pair(endpoint: &Endpoint) -> Result { // The remote is storage_a, so a concurrent server-side change wins over a // local one calcalist has not yet reconciled. Nothing is lost: the next // cycle sees the server's version and decides properly. + let resolution = match direction { + // `one_way` never produces a conflict, so there is nothing to resolve. + Direction::PullOnly => "\tone_way\n", + Direction::Both => "\tconflict_resolution keep a\n", + }; block.push_str(&format!( "\npair {id} {{\n\tstorage_a {id}_remote\n\tstorage_b {LOCAL_STORAGE}\n\ \tcollection {{\n\t\talias {id}\n\t\thref_a {}\n\t\tid_b {id}\n\t}}\n\ - \tconflict_resolution keep a\n\ton_empty skip\n\ton_delete skip\n}}\n", + {resolution}\ton_empty skip\n\ton_delete skip\n}}\n", quote(&href) )); Ok(block) @@ -235,8 +255,16 @@ fn quote(value: &str) -> String { /// Writes the generated configuration, returning its path. pub fn write_config(config: &Config, state_dir: &Path) -> Result { + write_config_for(config, state_dir, Direction::Both) +} + +pub fn write_config_for( + config: &Config, + state_dir: &Path, + direction: Direction, +) -> Result { let path = config_path(state_dir); - let text = generate(config, state_dir)?; + let text = generate_for(config, state_dir, direction)?; fs::create_dir_all(status_path(state_dir)).map_err(|source| PimsyncError::Write { path: status_path(state_dir), source, @@ -327,7 +355,8 @@ url = "https://example.org/holidays.ics" fn generated() -> String { let config: Config = toml::from_str(CONFIG).expect("config should parse"); - generate(&config, Path::new("/var/state/calcalist")).expect("generation should succeed") + generate_for(&config, Path::new("/var/state/calcalist"), Direction::Both) + .expect("generation should succeed") } #[test] @@ -404,7 +433,8 @@ url_command = "secret-tool lookup service calcalist account gcal-ics" "#, ) .expect("config should parse"); - let text = generate(&config, Path::new("/var/state/calcalist")).expect("generate"); + let text = generate_for(&config, Path::new("/var/state/calcalist"), Direction::Both) + .expect("generate"); assert!(text.contains("url {")); assert!(text.contains("shell secret-tool lookup service calcalist account gcal-ics")); diff --git a/src/sync.rs b/src/sync.rs index b6e1f1d..1494c70 100644 --- a/src/sync.rs +++ b/src/sync.rs @@ -5,7 +5,8 @@ //! out of those vdirs is the job of pimsync (CalDAV, WebCal) and the Google //! module, which bracket this step. -use std::collections::BTreeMap; +use std::collections::{BTreeMap, BTreeSet}; +use std::fs; use std::path::{Path, PathBuf}; use thiserror::Error; @@ -13,7 +14,7 @@ use thiserror::Error; use crate::config::{Aggregate, Config, Endpoint, EndpointKind}; use crate::google::api::{self, ApiError}; use crate::google::auth::{self, AuthError}; -use crate::pimsync::{self, PimsyncError}; +use crate::pimsync::{self, Direction, PimsyncError}; use crate::reconcile::{self, Action, Conflict, Policy, ReconcileError, Skipped, SourceView}; use crate::state::{State, Target}; use crate::vdir::{self, VdirError}; @@ -69,9 +70,37 @@ pub enum SyncError { pub struct Report { pub aggregates: Vec, pub google: Vec, + pub failures: Vec, + pub blocked: Vec, + pub orphan_vdirs: Vec, pub dry_run: bool, } +impl Report { + /// Whether the cycle covered everything it was asked to. + /// + /// A partial cycle is not a crash — the aggregates that could be reconciled + /// were — but it must not read as success either, or a lapsed token would go + /// unnoticed until the calendars had drifted a long way apart. + pub fn is_complete(&self) -> bool { + self.failures.is_empty() && self.blocked.is_empty() + } +} + +/// An endpoint that could not be reached this cycle. +#[derive(Debug)] +pub struct EndpointFailure { + pub endpoint: String, + pub reason: String, +} + +/// An aggregate left alone because an endpoint it depends on was unreachable. +#[derive(Debug)] +pub struct BlockedAggregate { + pub id: String, + pub endpoints: Vec, +} + #[derive(Debug)] pub struct GoogleReport { pub endpoint: String, @@ -81,6 +110,7 @@ pub struct GoogleReport { pub created_remotely: usize, pub updated_remotely: usize, pub deleted_remotely: usize, + pub exceptions_applied: usize, pub exceptions_skipped: usize, } @@ -106,133 +136,257 @@ pub fn run( let mut state = State::load(&state_path)?; let mut report = Report { dry_run, + orphan_vdirs: orphan_vdirs(config, state_dir), ..Report::default() }; prepare_vdirs(config, state_dir)?; - // A dry run must leave the vdirs exactly as it found them, and pimsync - // writes to them, so the pulls and pushes are skipped along with everything - // else. What it reports is therefore what the *last* pull left behind. - let pimsync_config = if dry_run || !needs_pimsync(config) { - None + // A dry run pulls for real, so that what it reports is measured against the + // calendars as they are now — but into a throwaway copy of the local + // mirrors, and through a configuration that only ever reads from a server. + // Nothing outside that copy is written, and it is removed at the end. + let workspace = if dry_run { + Some(Workspace::open(state_dir)?) } else { - Some(pimsync::write_config(config, state_dir)?) + None + }; + let work_dir = workspace.as_ref().map_or(state_dir, Workspace::path); + + let pimsync_config = if needs_pimsync(config) { + let direction = if dry_run { + Direction::PullOnly + } else { + Direction::Both + }; + Some(pimsync::write_config_for(config, work_dir, direction)?) + } else { + None }; // Pull first, so the reconciler sees one consistent snapshot of every remote. if let Some(path) = &pimsync_config { pimsync::sync(path)?; } - if !dry_run { - report.google = pull_google(config, state_dir, &mut state)?; - } + let unavailable = pull_google(config, state_dir, work_dir, &mut state, &mut report); for aggregate in &config.aggregates { + // An aggregate whose target or any source could not be pulled is left + // untouched. Reconciling it would compare against a stale snapshot, and + // an endpoint that failed to answer looks exactly like an emptied one. + let endpoints: Vec = std::iter::once(&aggregate.target) + .chain(aggregate.sources.iter()) + .filter(|id| unavailable.contains(*id)) + .cloned() + .collect(); + if !endpoints.is_empty() { + report.blocked.push(BlockedAggregate { + id: aggregate.id.clone(), + endpoints, + }); + continue; + } + let target = resolve(config, aggregate, &aggregate.target)?; check_target_drift(&state, aggregate, target)?; report.aggregates.push(sync_aggregate( - config, aggregate, target, state_dir, &mut state, dry_run, force, + config, aggregate, target, work_dir, &mut state, dry_run, force, )?); } // Push what the reconciler decided out to the remotes. if !dry_run { - push_google(config, state_dir, &mut state, &mut report)?; - } - if let Some(path) = &pimsync_config { - pimsync::sync(path)?; - } - - if !dry_run { + push_google(config, state_dir, &mut state, &mut report, &unavailable); + if let Some(path) = &pimsync_config { + pimsync::sync(path)?; + } state.save(&state_path)?; } + if let Some(workspace) = workspace { + workspace.discard(); + } Ok(report) } -/// Brings every Google endpoint's vdir into step with its remote calendar. +/// A throwaway copy of the local mirrors, used by `--dry-run`. +/// +/// Pulling needs somewhere to put what it reads. Writing into the real mirrors +/// would leave them describing a moment the recorded state knows nothing about, +/// so a dry run gets its own copy and the originals are never opened for writing. +struct Workspace { + root: PathBuf, +} + +impl Workspace { + const DIRECTORY: &'static str = "dry-run"; + + fn open(state_dir: &Path) -> Result { + let root = state_dir.join(Self::DIRECTORY); + // A previous run that was interrupted may have left one behind. + if root.exists() { + fs::remove_dir_all(&root).map_err(|source| SyncError::Prepare { + path: root.clone(), + source, + })?; + } + copy_tree(&vdir_root(state_dir), &root.join("vdir"))?; + // pimsync's own record of what it has seen comes too, or it would treat + // every local item as new and plan to create it on the server. + copy_tree( + &pimsync::status_path(state_dir), + &pimsync::status_path(&root), + )?; + Ok(Workspace { root }) + } + + fn path(&self) -> &Path { + &self.root + } + + fn discard(self) { + let _ = fs::remove_dir_all(&self.root); + } +} + +/// Copies a directory tree, treating a missing source as an empty one. +fn copy_tree(from: &Path, to: &Path) -> Result<(), SyncError> { + fs::create_dir_all(to).map_err(|source| SyncError::Prepare { + path: to.to_path_buf(), + source, + })?; + let Ok(entries) = fs::read_dir(from) else { + return Ok(()); + }; + for entry in entries.flatten() { + let target = to.join(entry.file_name()); + let copied = if entry.path().is_dir() { + copy_tree(&entry.path(), &target) + } else { + fs::copy(entry.path(), &target) + .map(|_| ()) + .map_err(|source| SyncError::Prepare { + path: target, + source, + }) + }; + copied?; + } + Ok(()) +} + +/// Local mirrors belonging to endpoints the configuration no longer names. +/// +/// Removing an endpoint leaves its events sitting in the state directory, where +/// nothing manages them any more. They are reported rather than deleted: the +/// endpoint may have been renamed, or commented out for an afternoon. +pub fn orphan_vdirs(config: &Config, state_dir: &Path) -> Vec { + let Ok(entries) = fs::read_dir(vdir_root(state_dir)) else { + return Vec::new(); + }; + let mut orphans: Vec = entries + .flatten() + .filter(|entry| entry.path().is_dir()) + .filter(|entry| { + let name = entry.file_name().to_string_lossy().into_owned(); + config.endpoint(&name).is_none() + }) + .map(|entry| entry.path()) + .collect(); + orphans.sort(); + orphans +} + +/// Brings every Google endpoint's vdir into step with its remote calendar, +/// returning the endpoints that could not be reached. /// /// This is the Google equivalent of the pimsync pull: pimsync cannot reach /// Google at all, so calcalist does this leg itself. +/// +/// A failure here used to end the cycle. It no longer does: one lapsed token +/// would otherwise stop the CalDAV side too, which has nothing to do with it. +/// The affected endpoints are named, and only the aggregates that depend on +/// them stand down. fn pull_google( config: &Config, state_dir: &Path, + vdir_base: &Path, state: &mut State, -) -> Result, SyncError> { - let mut reports = Vec::new(); + report: &mut Report, +) -> BTreeSet { + let mut unavailable = BTreeSet::new(); for endpoint in &config.endpoints { let EndpointKind::Google { calendar_id, .. } = &endpoint.kind else { continue; }; - let credentials = auth::credentials_for(config, &endpoint.id).map_err(|source| { - SyncError::GoogleAuth { + let pulled = client_for(config, state_dir, endpoint, calendar_id).and_then(|client| { + let dir = vdir_path(vdir_base, &endpoint.id); + let entry = state.google.entry(endpoint.id.clone()).or_default(); + api::pull(&client, &dir, entry).map_err(|source| SyncError::Google { endpoint: endpoint.id.clone(), source, - } - })?; - let token = - auth::access_token(state_dir, &endpoint.id, &credentials).map_err(|source| { - SyncError::GoogleAuth { - endpoint: endpoint.id.clone(), - source, - } - })?; - - let client = api::Client::new(token, calendar_id.clone()); - let dir = vdir_path(state_dir, &endpoint.id); - let entry = state.google.entry(endpoint.id.clone()).or_default(); - let pulled = api::pull(&client, &dir, entry).map_err(|source| SyncError::Google { - endpoint: endpoint.id.clone(), - source, - })?; - reports.push(GoogleReport { - endpoint: endpoint.id.clone(), - written: pulled.written, - deleted: pulled.deleted, - full_resync: pulled.full_resync, - created_remotely: 0, - updated_remotely: 0, - deleted_remotely: 0, - exceptions_skipped: 0, + }) }); + match pulled { + Ok(pulled) => report.google.push(GoogleReport { + endpoint: endpoint.id.clone(), + written: pulled.written, + deleted: pulled.deleted, + full_resync: pulled.full_resync, + created_remotely: 0, + updated_remotely: 0, + deleted_remotely: 0, + exceptions_applied: 0, + exceptions_skipped: 0, + }), + Err(error) => { + unavailable.insert(endpoint.id.clone()); + report.failures.push(EndpointFailure { + endpoint: endpoint.id.clone(), + reason: chain(&error), + }); + } + } } - Ok(reports) + unavailable } /// Sends each Google endpoint's local changes up to its calendar. /// /// Runs after reconciliation, so what goes up is what the reconciler decided. +/// An endpoint whose pull failed is skipped: its vdir describes some earlier +/// moment, and pushing that would undo whatever has happened since. fn push_google( config: &Config, state_dir: &Path, state: &mut State, report: &mut Report, -) -> Result<(), SyncError> { + unavailable: &BTreeSet, +) { for endpoint in &config.endpoints { let EndpointKind::Google { calendar_id, .. } = &endpoint.kind else { continue; }; - let credentials = auth::credentials_for(config, &endpoint.id).map_err(|source| { - SyncError::GoogleAuth { + if unavailable.contains(&endpoint.id) { + continue; + } + let pushed = client_for(config, state_dir, endpoint, calendar_id).and_then(|client| { + let dir = vdir_path(state_dir, &endpoint.id); + let entry = state.google.entry(endpoint.id.clone()).or_default(); + api::push(&client, &dir, entry).map_err(|source| SyncError::Google { endpoint: endpoint.id.clone(), source, - } - })?; - let token = - auth::access_token(state_dir, &endpoint.id, &credentials).map_err(|source| { - SyncError::GoogleAuth { + }) + }); + let pushed = match pushed { + Ok(pushed) => pushed, + Err(error) => { + report.failures.push(EndpointFailure { endpoint: endpoint.id.clone(), - source, - } - })?; - - let client = api::Client::new(token, calendar_id.clone()); - let dir = vdir_path(state_dir, &endpoint.id); - let entry = state.google.entry(endpoint.id.clone()).or_default(); - let pushed = api::push(&client, &dir, entry).map_err(|source| SyncError::Google { - endpoint: endpoint.id.clone(), - source, - })?; + reason: chain(&error), + }); + continue; + } + }; if let Some(existing) = report .google @@ -242,10 +396,46 @@ fn push_google( existing.created_remotely = pushed.created; existing.updated_remotely = pushed.updated; existing.deleted_remotely = pushed.deleted; + existing.exceptions_applied = pushed.exceptions_applied; existing.exceptions_skipped = pushed.exceptions_skipped; } } - Ok(()) +} + +/// An authorised client for one Google endpoint. +fn client_for( + config: &Config, + state_dir: &Path, + endpoint: &Endpoint, + calendar_id: &str, +) -> Result { + let credentials = + auth::credentials_for(config, &endpoint.id).map_err(|source| SyncError::GoogleAuth { + endpoint: endpoint.id.clone(), + source, + })?; + let token = auth::access_token( + state_dir, + &endpoint.id, + endpoint.kind.google_account(), + &credentials, + ) + .map_err(|source| SyncError::GoogleAuth { + endpoint: endpoint.id.clone(), + source, + })?; + Ok(api::Client::new(token, calendar_id.to_string())) +} + +/// An error and its causes on one line, for a report that keeps going. +fn chain(error: &dyn std::error::Error) -> String { + let mut out = error.to_string(); + let mut source = error.source(); + while let Some(cause) = source { + out.push_str(&format!(": {cause}")); + source = cause.source(); + } + out } /// Whether any endpoint is one pimsync handles. A Google-only setup needs none. @@ -705,4 +895,73 @@ sources = ["gcal"] uids(dir.path(), "work") ); } + + /// An endpoint calcalist cannot reach must not take the whole cycle with it. + /// Here every endpoint is Google and none is authorised, so all of them fail + /// — and the run still returns, naming what it could not do. + #[test] + fn an_unreachable_endpoint_blocks_only_what_depends_on_it() { + let dir = tempfile::tempdir().expect("temp"); + let config: Config = toml::from_str(&format!( + "{CONFIG}\n[[aggregate]]\nid = \"unified\"\ntarget = \"posteo\"\nsources = [\"work\"]\n" + )) + .expect("config"); + seed(dir.path(), "work", "a@example.com", "Standup"); + + let report = run(&config, dir.path(), false, false).expect("the cycle still returns"); + + assert!(!report.failures.is_empty(), "{report:?}"); + assert_eq!(report.blocked.len(), 1, "{report:?}"); + assert_eq!(report.blocked[0].id, "unified"); + assert!(report.blocked[0].endpoints.contains(&"work".to_string())); + assert!(!report.is_complete(), "a partial cycle is not a success"); + // Nothing was reconciled against the stale snapshot. + assert!(report.aggregates.is_empty(), "{report:?}"); + assert!(uids(dir.path(), "posteo").is_empty()); + } + + /// A mirror whose endpoint has gone from the configuration is reported, not + /// deleted: the endpoint may simply have been renamed. + #[test] + fn a_mirror_with_no_endpoint_is_reported_as_an_orphan() { + let dir = tempfile::tempdir().expect("temp"); + let config: Config = toml::from_str(CONFIG).expect("config"); + seed(dir.path(), "work", "a@example.com", "Standup"); + seed(dir.path(), "retired", "b@example.com", "Old"); + + let orphans = orphan_vdirs(&config, dir.path()); + + assert_eq!(orphans, vec![vdir_path(dir.path(), "retired")]); + } + + /// The dry run's copy is thrown away, and the mirrors it was made from are + /// left exactly as they were. + #[test] + fn a_dry_run_leaves_the_real_mirrors_alone() { + let dir = tempfile::tempdir().expect("temp"); + let config: Config = toml::from_str(&format!( + "{CONFIG}\n[[aggregate]]\nid = \"unified\"\ntarget = \"posteo\"\nsources = [\"work\"]\n" + )) + .expect("config"); + seed(dir.path(), "work", "a@example.com", "Standup"); + let before = fs::read_to_string(vdir_path(dir.path(), "work").join("a@example.com.ics")) + .expect("read"); + + let report = run(&config, dir.path(), true, false).expect("dry run"); + + assert!(report.dry_run); + assert!( + !dir.path().join("dry-run").exists(), + "the workspace should be gone" + ); + assert_eq!( + fs::read_to_string(vdir_path(dir.path(), "work").join("a@example.com.ics")) + .expect("read"), + before + ); + assert!( + !dir.path().join(crate::state::FILE_NAME).exists(), + "a dry run records nothing" + ); + } } diff --git a/tests/caldav.rs b/tests/caldav.rs new file mode 100644 index 0000000..68bdb1a --- /dev/null +++ b/tests/caldav.rs @@ -0,0 +1,274 @@ +//! End-to-end tests against a real CalDAV server and a real iCal feed. +//! +//! The unit tests establish that the reconciler decides correctly. These +//! establish that the decisions survive the round trip through pimsync and a +//! server that rewrites what it stores — which is where every bug found by hand +//! during M1 actually lived. +//! +//! Radicale and pimsync both come from devbox, so `devbox run check` has them. +//! Outside that shell the tests report what is missing and pass, rather than +//! failing for a reason that has nothing to do with the code. + +mod support; + +use support::{Calcalist, Feed, Radicale}; + +/// An event with guests, an organiser and an alarm — the combination the +/// scheduling rules are about. +const MEETING: &str = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:-//test//EN\r\n\ + BEGIN:VEVENT\r\nUID:meeting@work\r\nDTSTAMP:20260101T000000Z\r\n\ + DTSTART:20260910T090000Z\r\nDTEND:20260910T100000Z\r\nSUMMARY:Planning\r\n\ + ORGANIZER;CN=Chair:mailto:chair@example.com\r\n\ + ATTENDEE;CN=Guest;PARTSTAT=ACCEPTED:mailto:guest@example.com\r\n\ + BEGIN:VALARM\r\nACTION:DISPLAY\r\nTRIGGER:-PT15M\r\nDESCRIPTION:Soon\r\n\ + END:VALARM\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n"; + +const HOLIDAY_FEED: &str = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:-//feed//EN\r\n\ + BEGIN:VEVENT\r\nUID:newyear@feed\r\nDTSTAMP:20260101T000000Z\r\n\ + DTSTART;VALUE=DATE:20260101\r\nDTEND;VALUE=DATE:20260102\r\n\ + SUMMARY:New Year\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n"; + +struct Fixture { + _root: tempfile::TempDir, + server: Radicale, + _feed: Feed, + calcalist: Calcalist, +} + +/// A CalDAV source, a read-only feed, and a CalDAV target to aggregate into. +fn fixture() -> Option { + let missing = support::missing_binaries(); + if !missing.is_empty() { + eprintln!( + "skipped: {} not on PATH; run under devbox", + missing.join(", ") + ); + return None; + } + + let root = tempfile::tempdir().expect("temp"); + let server = Radicale::start(root.path()); + server.create_calendar("work"); + server.create_calendar("unified"); + server.put_event("work", "meeting.ics", MEETING); + let feed = Feed::start(HOLIDAY_FEED.to_string()); + + let config = format!( + r#" +version = 1 + +[[endpoint]] +id = "work" +type = "caldav" +url = "{work}" +username = "{user}" +secret_command = "printf password" + +[[endpoint]] +id = "published" +type = "caldav" +url = "{unified}" +username = "{user}" +secret_command = "printf password" + +[[endpoint]] +id = "holidays" +type = "webcal" +url = "{feed}" + +[[aggregate]] +id = "everything" +target = "published" +sources = ["work", "holidays"] +default_sink = "work" +"#, + work = server.url("work"), + unified = server.url("unified"), + user = support::USER, + feed = feed.url(), + ); + + let calcalist = Calcalist::new(root.path(), &config); + Some(Fixture { + _root: root, + server, + _feed: feed, + calcalist, + }) +} + +/// Both sources reach the target, and running again changes nothing. +/// +/// Idempotence is the property that matters most: a cycle that is not a no-op +/// against unchanged input would rewrite every event on every run, and each +/// rewrite is a chance for the server to hand back something slightly different. +#[test] +fn a_cycle_converges_and_the_next_one_does_nothing() { + let Some(fixture) = fixture() else { return }; + + let first = fixture.calcalist.run(&["sync"]); + assert!( + first.succeeded(), + "first sync failed\n{}\n{}", + first.stdout, + first.stderr + ); + + let published = fixture.server.stored("unified"); + assert_eq!( + published.len(), + 2, + "both sources should arrive: {published:?}" + ); + let all = published.join("\n"); + assert!(all.contains("Planning"), "{all}"); + assert!(all.contains("New Year"), "{all}"); + // Provenance travels with each mirror, so the aggregate knows where its + // events came from without consulting the state file. + assert!(all.contains("X-CALCALIST-SOURCE:work"), "{all}"); + assert!(all.contains("X-CALCALIST-SOURCE:holidays"), "{all}"); + + let second = fixture.calcalist.run(&["sync"]); + assert!( + second.succeeded(), + "second sync failed\n{}\n{}", + second.stdout, + second.stderr + ); + assert!( + second.stdout.contains("everything: 0 mirrored"), + "the second cycle should be a no-op:\n{}", + second.stdout + ); + assert_eq!( + fixture.server.stored("unified"), + published, + "the second cycle rewrote the target" + ); +} + +/// The scheduling rule, checked against what actually reached the server. +/// +/// A CalDAV server has no portable way to be told not to send invitations, so +/// inertness is structural: the mirror carries the guest list as data and not as +/// live scheduling properties. The alarm is the deliberate exception — stripping +/// it would destroy every reminder in the one calendar the user subscribes to. +/// +/// Radicale implements no scheduling of its own, so this asserts on the bytes +/// stored rather than on a mail sink: with nothing to send mail, a quiet SMTP +/// port would prove nothing about the transform. +#[test] +fn a_mirror_carries_no_live_scheduling_properties() { + let Some(fixture) = fixture() else { return }; + + let run = fixture.calcalist.run(&["sync"]); + assert!( + run.succeeded(), + "sync failed\n{}\n{}", + run.stdout, + run.stderr + ); + + let mirror = fixture + .server + .stored("unified") + .into_iter() + .find(|item| item.contains("Planning")) + .expect("the meeting should have been mirrored"); + + for property in ["ATTENDEE;", "ATTENDEE:", "ORGANIZER;", "ORGANIZER:"] { + assert!( + !mirror + .lines() + .any(|line| line.trim_start().starts_with(property)), + "a live {property} reached the aggregate:\n{mirror}" + ); + } + // The guests are still there, as something nothing will act on. + assert!( + mirror.contains("X-CALCALIST-ATTENDEES"), + "the guest list was lost:\n{mirror}" + ); + assert!( + mirror.contains("guest@example.com"), + "the guest list was lost:\n{mirror}" + ); + assert!( + mirror.contains("BEGIN:VALARM") && mirror.contains("TRIGGER:-PT15M"), + "the alarm did not survive:\n{mirror}" + ); + + // The source keeps its scheduling properties: only the aggregate is inert. + let source = fixture + .server + .stored("work") + .into_iter() + .find(|item| item.contains("Planning")) + .expect("the meeting should still be in its source"); + assert!(source.contains("ATTENDEE"), "{source}"); + assert!(source.contains("ORGANIZER"), "{source}"); +} + +/// A dry run must reach the servers to be worth anything, and change nothing. +#[test] +fn a_dry_run_reports_without_touching_anything() { + let Some(fixture) = fixture() else { return }; + + let dry = fixture.calcalist.run(&["sync", "--dry-run"]); + assert!( + dry.succeeded(), + "dry run failed\n{}\n{}", + dry.stdout, + dry.stderr + ); + assert!( + dry.stdout.contains("everything: 2 mirrored"), + "the dry run should have seen both sources:\n{}", + dry.stdout + ); + assert!( + fixture.server.stored("unified").is_empty(), + "the dry run published events" + ); + + // And the real cycle that follows is not confused by it. + let real = fixture.calcalist.run(&["sync"]); + assert!( + real.succeeded(), + "sync after a dry run failed\n{}\n{}", + real.stdout, + real.stderr + ); + assert_eq!(fixture.server.stored("unified").len(), 2); +} + +/// Removing an endpoint used to leave its events sitting in the state directory +/// with nothing managing them. They are now reported, and removed on request. +#[test] +fn a_retired_endpoints_mirror_is_reported_and_then_removed() { + let Some(fixture) = fixture() else { return }; + + assert!(fixture.calcalist.run(&["sync"]).succeeded()); + let mirror = fixture.calcalist.state.join("calcalist/vdir/holidays"); + assert!(mirror.is_dir(), "the feed should have been mirrored"); + + // The feed is dropped from the configuration, as a user would drop it. + std::fs::write( + &fixture.calcalist.config, + fixture.calcalist.config_without_feed(), + ) + .expect("rewrite config"); + + let listed = fixture.calcalist.run(&["prune"]); + assert!(listed.succeeded(), "{}", listed.stderr); + assert!( + listed.stdout.contains("would remove") && listed.stdout.contains("holidays"), + "prune should say what it found:\n{}", + listed.stdout + ); + assert!(mirror.is_dir(), "listing must not delete anything"); + + let removed = fixture.calcalist.run(&["prune", "--force"]); + assert!(removed.succeeded(), "{}", removed.stderr); + assert!(!mirror.exists(), "the orphaned mirror should be gone"); +} diff --git a/tests/support/mod.rs b/tests/support/mod.rs new file mode 100644 index 0000000..605818c --- /dev/null +++ b/tests/support/mod.rs @@ -0,0 +1,310 @@ +//! Scaffolding for the integration tests: a real CalDAV server, a real feed. +//! +//! Everything here talks HTTP over a plain socket rather than through a client +//! crate. The requests involved are few and mostly unusual — `MKCALENDAR`, `PUT` +//! of an `.ics` — and writing them out makes exactly what the server is asked +//! for visible in the test. + +use std::io::{Read, Write}; +use std::net::{Ipv4Addr, SocketAddr, TcpListener, TcpStream}; +use std::path::{Path, PathBuf}; +use std::process::{Child, Command, Stdio}; +use std::time::{Duration, Instant}; + +/// How long to wait for a server to start answering. +const STARTUP_TIMEOUT: Duration = Duration::from_secs(20); + +/// The account Radicale files everything under. With authentication disabled it +/// accepts whatever name arrives, and `owner_only` rights then grant that name +/// its own tree — so the URLs below all live beneath it. +pub const USER: &str = "calcalist"; + +/// A binary the integration tests need, and whether it is here. +pub fn missing_binaries() -> Vec<&'static str> { + ["radicale", "pimsync"] + .into_iter() + .filter(|binary| which(binary).is_none()) + .collect() +} + +fn which(binary: &str) -> Option { + std::env::var_os("PATH")? + .to_str()? + .split(':') + .map(|dir| Path::new(dir).join(binary)) + .find(|candidate| candidate.is_file()) +} + +/// A Radicale instance with its own storage, shut down when dropped. +pub struct Radicale { + process: Child, + pub port: u16, + storage: PathBuf, +} + +impl Radicale { + pub fn start(root: &Path) -> Radicale { + let storage = root.join("radicale"); + std::fs::create_dir_all(&storage).expect("create storage"); + let port = free_port(); + + let process = Command::new("radicale") + .arg("--config") + .arg("") + .arg("--server-hosts") + .arg(format!("127.0.0.1:{port}")) + .arg("--auth-type") + .arg("none") + .arg("--storage-filesystem-folder") + .arg(&storage) + .stdout(Stdio::null()) + .stderr(Stdio::null()) + .spawn() + .expect("radicale should start"); + + let server = Radicale { + process, + port, + storage, + }; + wait_until_listening(port, "radicale"); + server + } + + pub fn url(&self, calendar: &str) -> String { + format!("http://127.0.0.1:{}/{USER}/{calendar}/", self.port) + } + + /// Creates a calendar collection. pimsync deliberately never creates one, + /// so the calendars a test syncs have to exist on the server first. + pub fn create_calendar(&self, calendar: &str) { + let body = "\ + \ + calendar\ + "; + let response = self.request( + "MKCALENDAR", + &format!("/{USER}/{calendar}/"), + "application/xml; charset=utf-8", + body, + ); + // Radicale answers HTTP/1.0, so the status is read out of the line + // rather than matched against a whole prefix. + assert!( + matches!(status_of(&response), Some(201 | 405)), + "creating {calendar}: {response}" + ); + } + + /// Stores an event, as a calendar client would. + pub fn put_event(&self, calendar: &str, name: &str, ics: &str) { + let response = self.request( + "PUT", + &format!("/{USER}/{calendar}/{name}"), + "text/calendar; charset=utf-8", + ics, + ); + assert!( + status_of(&response).is_some_and(|status| (200..300).contains(&status)), + "storing {name}: {response}" + ); + } + + /// Everything the server holds in a calendar, as stored. + /// + /// Read from Radicale's own storage rather than fetched back, so what is + /// asserted on is the bytes that reached the server. + pub fn stored(&self, calendar: &str) -> Vec { + let dir = self + .storage + .join("collection-root") + .join(USER) + .join(calendar); + let Ok(entries) = std::fs::read_dir(dir) else { + return Vec::new(); + }; + let mut items: Vec = entries + .flatten() + .filter(|entry| { + entry + .path() + .extension() + .is_some_and(|extension| extension == "ics") + }) + .filter_map(|entry| std::fs::read_to_string(entry.path()).ok()) + .collect(); + items.sort(); + items + } + + fn request(&self, method: &str, path: &str, content_type: &str, body: &str) -> String { + let address = SocketAddr::from((Ipv4Addr::LOCALHOST, self.port)); + let mut stream = TcpStream::connect(address).expect("connect to radicale"); + // With authentication disabled the credentials only pick the account. + let authorization = base64(format!("{USER}:password").as_bytes()); + let request = format!( + "{method} {path} HTTP/1.1\r\nHost: 127.0.0.1:{}\r\n\ + Authorization: Basic {authorization}\r\nContent-Type: {content_type}\r\n\ + Content-Length: {}\r\nConnection: close\r\n\r\n{body}", + self.port, + body.len() + ); + stream.write_all(request.as_bytes()).expect("send request"); + let mut response = String::new(); + stream + .read_to_string(&mut response) + .expect("read the response"); + response + } +} + +impl Drop for Radicale { + fn drop(&mut self) { + let _ = self.process.kill(); + let _ = self.process.wait(); + } +} + +/// A one-file HTTP server, standing in for a published iCal feed. +pub struct Feed { + pub port: u16, +} + +impl Feed { + /// Serves `ics` at any path, for as long as the test runs. + pub fn start(ics: String) -> Feed { + let listener = TcpListener::bind(SocketAddr::from((Ipv4Addr::LOCALHOST, 0))) + .expect("bind the feed port"); + let port = listener.local_addr().expect("feed address").port(); + std::thread::spawn(move || { + for stream in listener.incoming().flatten() { + serve_once(stream, &ics); + } + }); + Feed { port } + } + + pub fn url(&self) -> String { + format!("http://127.0.0.1:{}/holidays.ics", self.port) + } +} + +fn serve_once(mut stream: TcpStream, ics: &str) { + // Enough of the request to reach the blank line; the path does not matter. + let mut buffer = [0u8; 2048]; + let _ = stream.read(&mut buffer); + let response = format!( + "HTTP/1.1 200 OK\r\nContent-Type: text/calendar; charset=utf-8\r\n\ + Content-Length: {}\r\nConnection: close\r\n\r\n{ics}", + ics.len() + ); + let _ = stream.write_all(response.as_bytes()); +} + +/// Runs `calcalist` against a config and state directory of the test's own. +pub struct Calcalist { + pub config: PathBuf, + pub state: PathBuf, +} + +/// What one `calcalist` invocation printed and returned. +pub struct Run { + pub status: Option, + pub stdout: String, + pub stderr: String, +} + +impl Run { + pub fn succeeded(&self) -> bool { + self.status == Some(0) + } +} + +impl Calcalist { + pub fn new(root: &Path, config: &str) -> Calcalist { + let config_path = root.join("calcalist.toml"); + std::fs::write(&config_path, config).expect("write config"); + let state = root.join("state"); + std::fs::create_dir_all(&state).expect("create state directory"); + Calcalist { + config: config_path, + state, + } + } + + /// The same configuration with the read-only feed taken out, for the test + /// that retires an endpoint. + pub fn config_without_feed(&self) -> String { + let text = std::fs::read_to_string(&self.config).expect("read config"); + text.split("\n\n") + .filter(|block| !block.contains("id = \"holidays\"")) + .map(|block| block.replace(", \"holidays\"", "")) + .collect::>() + .join("\n\n") + } + + pub fn run(&self, arguments: &[&str]) -> Run { + let output = Command::new(env!("CARGO_BIN_EXE_calcalist")) + .arg("--config") + .arg(&self.config) + .args(arguments) + // The state directory is resolved through XDG, so this is what + // keeps the test off the developer's own calendars and state. + .env("XDG_STATE_HOME", &self.state) + .output() + .expect("calcalist should run"); + Run { + status: output.status.code(), + stdout: String::from_utf8_lossy(&output.stdout).into_owned(), + stderr: String::from_utf8_lossy(&output.stderr).into_owned(), + } + } +} + +/// The status code out of a response's first line, whatever HTTP version it +/// claims. +fn status_of(response: &str) -> Option { + response + .lines() + .next()? + .split_whitespace() + .nth(1)? + .parse() + .ok() +} + +fn free_port() -> u16 { + let listener = + TcpListener::bind(SocketAddr::from((Ipv4Addr::LOCALHOST, 0))).expect("bind a free port"); + listener.local_addr().expect("port").port() +} + +fn wait_until_listening(port: u16, what: &str) { + let address = SocketAddr::from((Ipv4Addr::LOCALHOST, port)); + let deadline = Instant::now() + STARTUP_TIMEOUT; + while Instant::now() < deadline { + if TcpStream::connect_timeout(&address, Duration::from_millis(200)).is_ok() { + return; + } + std::thread::sleep(Duration::from_millis(100)); + } + panic!("{what} did not start listening on {port} within {STARTUP_TIMEOUT:?}"); +} + +fn base64(input: &[u8]) -> String { + const ALPHABET: &[u8] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"; + let mut out = String::new(); + for chunk in input.chunks(3) { + let mut block = [0u8; 3]; + block[..chunk.len()].copy_from_slice(chunk); + let packed = u32::from(block[0]) << 16 | u32::from(block[1]) << 8 | u32::from(block[2]); + for index in 0..4 { + if index <= chunk.len() { + out.push(ALPHABET[(packed >> (18 - index * 6)) as usize & 0x3f] as char); + } else { + out.push('='); + } + } + } + out +} From 25d181108e8bbe741ae8b319cdf0268738a2ae11 Mon Sep 17 00:00:00 2001 From: randogoth Date: Thu, 10 Sep 2026 14:16:45 +0300 Subject: [PATCH 04/10] Add a README covering use SPECS.md says why the tool is built the way it is and TODO.md tracks what is left; neither tells anyone how to use it. This covers install, the whole configuration format, routing, the commands, and what the state directory holds. The Google section is walked through step by step rather than left to Google's own documentation, because the parts that actually cost time are the ones it does not cover: the client has to be a Desktop app, since calcalist redirects to a loopback port it picks at run time and a Web application client demands a redirect URI and a verified domain it cannot be given; you have to add yourself under Test users or authorisation fails with access_denied against your own account; and a consent screen left in Testing has its refresh tokens expired after seven days, so a working installation stops with nothing having changed. Checked against the binary rather than by reading: every documented command and flag against `--help`, every field against config.rs, and the worked example run through `doctor` and `status` to confirm it validates and prints the routing markers the README says it does. Co-Authored-By: Claude Opus 5 --- README.md | 390 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 390 insertions(+) create mode 100644 README.md diff --git a/README.md b/README.md new file mode 100644 index 0000000..60deca9 --- /dev/null +++ b/README.md @@ -0,0 +1,390 @@ +# calcalist + +Aggregate several calendars into one, and keep them in step. + +calcalist mirrors any number of CalDAV calendars, Google calendars and iCal +feeds into a single calendar you subscribe to. The point is to subscribe to one +calendar instead of several: an event created or edited in the aggregate travels +back to the calendar it came from, so the aggregate is somewhere to work rather +than a read-only noticeboard. CalDAV and Google are read/write in both +directions; iCal feeds are read-only, because the protocol is. + +This is pre-1.0. The command line and the configuration format are settled; +there is no web interface yet. `SPECS.md` covers why it is built this way. + +## How it works + +Every endpoint — the sources and the aggregate's target alike — is mirrored to a +directory of `.ics` files on your machine, and the aggregation engine works +purely on those files. One cycle is: pull every remote into its local mirror, +reconcile the mirrors against each other, push the results back out. + +`pimsync` carries the CalDAV and iCal legs. calcalist carries the Google leg +itself, over the Calendar REST API — pimsync cannot reach Google at all, having +no REST storage and only HTTP Basic auth, which Google's CalDAV endpoint has +rejected since March 2025. + +## Install + +```sh +cargo build --release +# target/release/calcalist +``` + +One runtime dependency: **`pimsync` 0.5.x on `PATH`**, needed only if any +endpoint is CalDAV or an iCal feed. A Google-only setup needs nothing else. +pimsync is in nixpkgs; this repo's `devbox.json` pins 0.5.11 if you would rather +not install it yourself. + +Then check the environment: + +```sh +calcalist doctor +``` + +It reports whether pimsync is present and in the supported version series, +whether the state directory is writable, whether your configuration is valid, +whether pimsync accepts the configuration calcalist generates for it, and +whether each Google endpoint still has a usable authorisation. + +## Configuration + +One TOML file at `$XDG_CONFIG_HOME/calcalist/calcalist.toml`, or wherever +`--config` points. + +It holds **no secrets and no sync state**, so it is safe to copy between +machines. Credentials come from `*_command` fields instead: each is run through +`sh -c` and the first line of its output is used. Point them at `secret-tool`, +`pass`, `gpg -d`, or anything else that prints the secret. + +```toml +version = 1 +``` + +### Endpoints + +Each `[[endpoint]]` is one calendar, with an `id` you refer to it by elsewhere. + +**CalDAV** + +```toml +[[endpoint]] +id = "posteo" +type = "caldav" +url = "https://posteo.de:8443/calendars/you/abc123/work/" +username = "you@posteo.de" +secret_command = "secret-tool lookup service posteo user you@posteo.de" +``` + +`url` must name the calendar collection itself, not the server root or your +principal — the last path segment is the calendar. A URL with no collection in +it is rejected, and `calcalist doctor` says so. + +**Google** + +```toml +[[endpoint]] +id = "gcal" +type = "google" +calendar_id = "you@gmail.com" +client_id = "1234-abcd.apps.googleusercontent.com" +client_secret_command = "secret-tool lookup service google-calcalist" +# account = "you@gmail.com" # only when logged in to more than one account +``` + +`calendar_id` is `primary`, or the calendar's own address — the "Calendar ID" +under the calendar's settings in Google Calendar. See +[Authorising Google](#authorising-google) below for `client_id` and the secret. + +**iCal feed** + +```toml +[[endpoint]] +id = "holidays" +type = "webcal" +url = "https://example.org/holidays.ics" +``` + +Always read-only. Give exactly one of `url` or `url_command` — the latter for a +feed whose address is itself a credential, Google's "secret address in iCal +format" being the case in point, since anyone holding it can read the whole +calendar: + +```toml +url_command = "secret-tool lookup service google-secret-ics" +``` + +### Aggregates + +Each `[[aggregate]]` composes endpoints: several `sources` mirrored into one +`target`. + +```toml +[[aggregate]] +id = "unified" +target = "posteo" +sources = ["gcal", "holidays"] +default_sink = "gcal" + +# Defaults, all optional: +conflict = "source_wins" # the only policy; the origin calendar wins +propagate_deletes = true # deleting a mirror deletes the original +max_delete_fraction = 0.2 # abort if more of the aggregate than this vanishes +``` + +`max_delete_fraction` guards against a cycle that would remove a large part of +your calendars — a mistyped URL, a server answering with an empty collection. It +has an absolute floor of three deletions, so removing one or two events is never +refused. `calcalist sync --force` overrides it. + +The rules, all checked before anything is synchronised, with every problem +reported in one pass: + +- `target` must exist and be writable, so never an iCal feed. +- `target` must not also be one of its own `sources`. +- `sources` must be non-empty, and each must exist and be named once. +- `default_sink`, if given, must be one of the `sources` and must be writable. + +### A complete example + +A Google calendar and a public holiday feed, aggregated into a CalDAV calendar: + +```toml +version = 1 + +[[endpoint]] +id = "gcal" +type = "google" +calendar_id = "primary" +client_id = "1234-abcd.apps.googleusercontent.com" +client_secret_command = "secret-tool lookup service google-calcalist" + +[[endpoint]] +id = "holidays" +type = "webcal" +url = "https://www.officeholidays.com/ics/germany" + +[[endpoint]] +id = "posteo" +type = "caldav" +url = "https://posteo.de:8443/calendars/you/abc123/unified/" +username = "you@posteo.de" +secret_command = "secret-tool lookup service posteo user you@posteo.de" + +[[aggregate]] +id = "unified" +target = "posteo" +sources = ["gcal", "holidays"] +default_sink = "gcal" +``` + +## Authorising Google + +Google requires an OAuth client of your own. This is the fiddliest part of the +setup, so in full: + +1. At [console.cloud.google.com](https://console.cloud.google.com), create a + project — any name. +2. **APIs & Services → Library**, find **Google Calendar API**, enable it. +3. **OAuth consent screen**: choose **External**. Fill in the required fields. + Under **Test users**, add your own Google address. Without this, authorising + fails with `403: access_denied` even though the account is your own. +4. **Credentials → Create credentials → OAuth client ID**, and for + **Application type** choose **Desktop app**. + + Desktop app matters. calcalist listens on a loopback port it picks at run + time and hands Google that address as the redirect, so there is no redirect + URI to register and no domain to verify. A Web application client asks for + both and cannot be made to work here. +5. Copy the client ID into `client_id`, and put the client secret somewhere + `client_secret_command` can read it: + + ```sh + secret-tool store --label="calcalist Google client secret" service google-calcalist + ``` +6. Authorise: + + ```sh + calcalist google login gcal + ``` + + A browser opens; approve the request. calcalist asks only for + `https://www.googleapis.com/auth/calendar`. The refresh token is written + `0600` into the state directory, never into your configuration. + +**One login covers every endpoint on that account.** The authorisation is filed +under the account it was granted for, not the endpoint that asked, so a second +Google calendar on the same account needs no login of its own. Set `account` on +an endpoint only when calcalist is logged in to more than one account and has to +be told which is meant. + +**While the consent screen is in Testing, Google expires refresh tokens after +seven days.** An installation that worked last week then stops with nothing +having changed locally; `calcalist google login` again, or publish the consent +screen. `calcalist doctor` reports this rather than letting it surface as an +opaque failure. + +## Routing: getting a new event to the right calendar + +Every mirrored event carries where it came from — `X-CALCALIST-SOURCE` and +`X-CALCALIST-ORIGIN-UID` — and a UID derived from its origin, so an edit made in +the aggregate goes back to the calendar that owns the event without any +guesswork. + +An event you *create* in the aggregate belongs to nothing yet. It goes to the +aggregate's `default_sink`. To send it somewhere else, put the endpoint's id +after an `@` **on a line of its own** in the event's notes: + +``` +Dentist, bring the referral + +@gcal +``` + +The marker is removed before the event reaches the calendar. A whole line is +required so that an address or a handle written in prose is not mistaken for an +instruction. + +`calcalist status` prints the markers that would actually work, so you do not +have to remember what is in the configuration file: + +``` +aggregate `unified` + published to posteo + sources gcal, holidays + mirroring 42 event(s) + new events go to `gcal` unless told otherwise + to choose, put one of these on a line of its own in the event's notes: + @gcal +``` + +Read-only feeds are left out, since they cannot take an event. A marker naming +anything else is refused and reported — the event stays where it is rather than +being quietly filed under the default, which would put it somewhere you did not +ask for. + +A **category** matching an endpoint id works too, that being the field +iCalendar intends for this. Be aware that any category on a newly created event +is read as a routing instruction, so an event carrying an unrelated category +will be refused rather than sent to the default sink. The notes marker is the +safer form, and the one every mobile client exposes. + +### Attendees and alarms + +Writing to an aggregate must never mail anyone: the guests were already invited +from the original calendar. On a Google target, attendees are kept verbatim and +Google is told not to notify. A CalDAV server offers no portable way to be told +that, so the guest list is instead carried as inert data — the live `ATTENDEE` +and `ORGANIZER` properties are dropped, the guests appear in +`X-CALCALIST-ATTENDEES` and in the description, and a meeting you declined is +marked free. + +Routing works the other way round: an event you created and sent to a source is +a meeting you are deliberately organising, so its attendees are preserved and +that server invites them for real. + +Alarms are always kept, with no option to strip them. The whole point of the +tool is that you subscribe to one calendar rather than several, so dropping +reminders would silently disarm every one you have. + +## Commands + +``` +calcalist sync [--dry-run] [--force] +calcalist status +calcalist doctor +calcalist prune [--force] +calcalist google login +calcalist aggregate retarget --to [--keep-old | --purge-old] +``` + +- **`sync`** runs one cycle. `--dry-run` pulls from the servers for real, into a + throwaway copy of the local mirrors, so what it reports is measured against + your calendars as they are now — and nothing outside that copy is written. + `--force` overrides the mass-deletion guard. +- **`status`** lists endpoints and aggregates, how many events each aggregate is + mirroring, and the routing markers that would work. +- **`doctor`** checks the environment and configuration. Run it first. +- **`prune`** reports local mirrors belonging to endpoints your configuration no + longer names. It only lists them until you pass `--force`, since an endpoint + may just have been renamed. +- **`aggregate retarget`** moves an aggregate to a different target calendar + deliberately. `sync` refuses to run when an aggregate's `target` has changed + under it and points here instead — the new, empty target would otherwise read + as an aggregate whose every event had been deleted, and with delete + propagation on, that would remove them from every source calendar. Events left + on the old target are kept unless you pass `--purge-old`, which only removes + events calcalist put there. + +Exit codes: `0` all well, `1` something failed or an endpoint could not be +reached, `2` not implemented. + +If a Google endpoint cannot be reached — a lapsed token, no network — the cycle +does not stop. The endpoint is named, only the aggregates that depend on it +stand down, everything else is synchronised as usual, and the run exits `1` so +the failure cannot pass unnoticed. + +## Where things are kept + +Everything machine-local lives under `$XDG_STATE_HOME/calcalist`: + +``` +state.json event mappings, content hashes, sync cursors +pimsync.conf generated on every sync; edits are overwritten +pimsync-status/ pimsync's own record of what it has seen +vdir// the local mirror of each endpoint +google/accounts/.json refresh tokens, 0600 +``` + +None of this belongs in a backup or a dotfiles repository. The state file is a +cache rather than a single point of failure — aggregate UIDs are derived from +their origin, so a lost state file is rebuilt by re-deriving it. Copy the +configuration between machines; leave this behind. + +## Running it regularly + +There are no packaged units yet. A systemd user timer does the job: + +`~/.config/systemd/user/calcalist.service` + +```ini +[Unit] +Description=Synchronise calendars with calcalist + +[Service] +Type=oneshot +ExecStart=%h/.local/bin/calcalist sync +``` + +`~/.config/systemd/user/calcalist.timer` + +```ini +[Unit] +Description=Synchronise calendars every 15 minutes + +[Timer] +OnBootSec=2m +OnUnitActiveSec=15m +Persistent=true + +[Install] +WantedBy=timers.target +``` + +```sh +systemctl --user enable --now calcalist.timer +``` + +A cycle is one-shot, and systemd will not start the service again while a run is +still going, so a timer is all the scheduling needed. calcalist takes no lock of +its own, though, so avoid running `calcalist sync` by hand while the timer might +fire. If your secrets come from a keyring that needs an unlocked session, the +timer will only work while you are logged in. + +## Not yet + +- **The web configuration interface.** `calcalist serve` exits 2. +- **Packaged systemd units**, as above. +- **A failing `pimsync sync` stops the whole CalDAV leg.** Google endpoints are + isolated from each other, but pimsync is a single process covering every + CalDAV and feed pair, and a failure does not say which pair it belongs to. From 7607078394ae20f7c57043e30682bce89c72b762 Mon Sep 17 00:00:00 2001 From: randogoth Date: Thu, 10 Sep 2026 14:22:07 +0300 Subject: [PATCH 05/10] Pin the behaviour of an aggregate with no sink MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An aggregate without a default_sink leaves events created in it alone, which is how a target calendar keeps events of its own. The reconciler unit tests cover the decision; this covers it surviving the round trip, and specifically that it holds on every subsequent cycle — an event that survived the first cycle and was swept up by the second would be worse than never having worked. Co-Authored-By: Claude Opus 5 --- tests/caldav.rs | 57 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 57 insertions(+) diff --git a/tests/caldav.rs b/tests/caldav.rs index 68bdb1a..49ecc63 100644 --- a/tests/caldav.rs +++ b/tests/caldav.rs @@ -23,6 +23,12 @@ const MEETING: &str = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:-//test//EN\r\n\ BEGIN:VALARM\r\nACTION:DISPLAY\r\nTRIGGER:-PT15M\r\nDESCRIPTION:Soon\r\n\ END:VALARM\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n"; +/// An event written straight into the aggregate, belonging to no source. +const HAND_WRITTEN: &str = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:-//phone//EN\r\n\ + BEGIN:VEVENT\r\nUID:hand-written@phone\r\nDTSTAMP:20260101T000000Z\r\n\ + DTSTART:20260911T140000Z\r\nDTEND:20260911T150000Z\r\nSUMMARY:Dentist\r\n\ + END:VEVENT\r\nEND:VCALENDAR\r\n"; + const HOLIDAY_FEED: &str = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:-//feed//EN\r\n\ BEGIN:VEVENT\r\nUID:newyear@feed\r\nDTSTAMP:20260101T000000Z\r\n\ DTSTART;VALUE=DATE:20260101\r\nDTEND;VALUE=DATE:20260102\r\n\ @@ -272,3 +278,54 @@ fn a_retired_endpoints_mirror_is_reported_and_then_removed() { assert!(removed.succeeded(), "{}", removed.stderr); assert!(!mirror.exists(), "the orphaned mirror should be gone"); } + +/// An aggregate with no `default_sink` leaves events created in it alone. +/// +/// This is how a target calendar keeps events of its own: with nowhere +/// configured to file a new event, calcalist refuses to guess rather than +/// picking a source, so the event simply stays where it was written. Worth +/// pinning, because "left alone" has to hold on every subsequent cycle too — +/// an event that survived the first one and was swept up by the second would +/// be worse than never having worked. +#[test] +fn an_event_created_in_a_sinkless_aggregate_stays_where_it_is() { + let Some(fixture) = fixture() else { return }; + + // The fixture's aggregate has a default_sink; take it away. + let sinkless = std::fs::read_to_string(&fixture.calcalist.config) + .expect("read config") + .replace("default_sink = \"work\"\n", ""); + std::fs::write(&fixture.calcalist.config, sinkless).expect("rewrite config"); + + assert!(fixture.calcalist.run(&["sync"]).succeeded()); + + // Now add an event by hand, as a calendar app would. + fixture + .server + .put_event("unified", "dentist.ics", HAND_WRITTEN); + + for cycle in 1..=3 { + let run = fixture.calcalist.run(&["sync"]); + assert!( + run.succeeded(), + "cycle {cycle} failed\n{}\n{}", + run.stdout, + run.stderr + ); + + let published = fixture.server.stored("unified"); + assert!( + published.iter().any(|item| item.contains("Dentist")), + "cycle {cycle}: the hand-written event was removed:\n{published:?}" + ); + // It never reaches a source: there is nowhere it was told to go. + assert!( + !fixture + .server + .stored("work") + .iter() + .any(|item| item.contains("Dentist")), + "cycle {cycle}: the hand-written event leaked into the source" + ); + } +} From 47ad8b4c470b4aab6feb04160eca787e55fb8e36 Mon Sep 17 00:00:00 2001 From: randogoth Date: Thu, 10 Sep 2026 14:30:25 +0300 Subject: [PATCH 06/10] Make an aggregate's own events a first-class mode MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An aggregate with no default_sink already left events created in it alone, but treated doing so as a failure: it reported a skip per event per cycle saying no sink was configured, as though something had gone wrong. Nothing had. An aggregate is also a calendar, and holding events of its own is a legitimate way to use one. Skipped::NoSink is replaced by a kept_local count, reported plainly. `@local` joins the routing markers, so the mode also works per-event where a default_sink is configured — which was not previously expressible. Unlike every other marker it is deliberately not stripped: the others have done their job once the event reaches its source, whereas this one never leaves, so it has to stay legible for the next cycle to reach the same decision. `local` is therefore a reserved endpoint id, and configuring one is refused. This also fixes a real defect. `retarget` rebuilds the new target from the recorded links, which cover derived events only, so an event belonging to the aggregate itself did not follow the move — it stayed on the calendar being left behind while everything around it moved on, quietly. It is now carried across, since there is nothing to re-derive it from, and the new target's scheduling rule is applied on the way: this is a write to an aggregate like any other, and a guest list carried live onto a server that schedules would mail everyone on it. Co-Authored-By: Claude Opus 5 --- README.md | 29 +++++++++++++++++ src/config.rs | 33 ++++++++++++++++++- src/main.rs | 15 +++++++-- src/mirror.rs | 27 ++++++++++++++++ src/reconcile.rs | 65 +++++++++++++++++++++++++++---------- src/retarget.rs | 84 ++++++++++++++++++++++++++++++++++++++++++++++-- src/status.rs | 15 +++++++-- src/sync.rs | 2 ++ tests/caldav.rs | 54 +++++++++++++++++++++++++++++++ 9 files changed, 298 insertions(+), 26 deletions(-) diff --git a/README.md b/README.md index 60deca9..f0034bf 100644 --- a/README.md +++ b/README.md @@ -144,6 +144,9 @@ reported in one pass: - `target` must not also be one of its own `sources`. - `sources` must be non-empty, and each must exist and be named once. - `default_sink`, if given, must be one of the `sources` and must be writable. + Leaving it out is a deliberate mode, not an omission — see + [Events of the aggregate's own](#events-of-the-aggregates-own). +- No endpoint may be called `local`, which is reserved for the routing marker. ### A complete example @@ -256,6 +259,7 @@ aggregate `unified` new events go to `gcal` unless told otherwise to choose, put one of these on a line of its own in the event's notes: @gcal + @local (keep the event here, in this calendar only) ``` Read-only feeds are left out, since they cannot take an event. A marker naming @@ -263,6 +267,31 @@ anything else is refused and reported — the event stays where it is rather tha being quietly filed under the default, which would put it somewhere you did not ask for. +### Events of the aggregate's own + +An aggregate is also a calendar, and it can hold events that belong to nothing +else. There are two ways to get one: + +- **`@local`** on an event, exactly like any other marker, keeps it where it was + written even though a `default_sink` would otherwise have taken it. Unlike + every other marker it is *not* removed from the event: the others have done + their job once the event reaches its source, whereas this one has to stay + legible so every later cycle reaches the same decision. +- **Leave `default_sink` out of the aggregate entirely.** Then nothing is + configured to file a new event under, so every untagged event stays put and + only `@`-tagged ones are sent anywhere. Routing becomes opt-in rather than + opt-out. + +`local` is a reserved endpoint id for this reason, and configuring an endpoint +with that name is refused. + +One thing to be aware of: every other event in an aggregate is derived from a +source, so the aggregate is disposable — lose the calendar and it rebuilds +itself on the next sync. **An event of the aggregate's own exists in exactly one +place.** Nothing else holds a copy, so it is only as safe as that calendar is. +`aggregate retarget` carries these events to the new target along with +everything else, since there is nothing to re-derive them from. + A **category** matching an endpoint id works too, that being the field iCalendar intends for this. Be aware that any category on a newly created event is read as a routing instruction, so an event carrying an unrelated category diff --git a/src/config.rs b/src/config.rs index 4e606d4..0617fc3 100644 --- a/src/config.rs +++ b/src/config.rs @@ -207,10 +207,20 @@ pub enum Problem { NoUrlSource { endpoint: String }, #[error("endpoint `{endpoint}` has both `url` and `url_command`; use one or the other")] AmbiguousUrlSource { endpoint: String }, + #[error( + "endpoint `{endpoint}` uses a reserved id: `@{endpoint}` means \"keep this event in the aggregate\", so an endpoint of that name could never be routed to" + )] + ReservedEndpointId { endpoint: String }, } -/// A webcal feed must name exactly one source for its URL. +/// A webcal feed must name exactly one source for its URL, and no endpoint may +/// take a name the routing markers have already spoken for. fn check_endpoint(endpoint: &Endpoint, problems: &mut Vec) { + if crate::mirror::is_local_sink(&endpoint.id) { + problems.push(Problem::ReservedEndpointId { + endpoint: endpoint.id.clone(), + }); + } let EndpointKind::Webcal { url, url_command } = &endpoint.kind else { return; }; @@ -609,4 +619,25 @@ sources = ["work", "work"] endpoint: "work".into(), })); } + + /// `@local` already means "keep this event here", so an endpoint of that + /// name could never be routed to. + #[test] + fn an_endpoint_may_not_take_the_reserved_name() { + let problems = problems( + r#" +[[endpoint]] +id = "local" +type = "caldav" +url = "https://caldav.example.com/other/" +username = "me@example.com" +"#, + ); + assert!( + problems + .iter() + .any(|problem| matches!(problem, Problem::ReservedEndpointId { .. })), + "{problems:?}" + ); + } } diff --git a/src/main.rs b/src/main.rs index bb23631..d30f7b9 100644 --- a/src/main.rs +++ b/src/main.rs @@ -195,6 +195,12 @@ fn run_retarget(cli: &Cli, id: &str, to: &str, purge_old: bool) -> ExitCode { outcome.routed_first, outcome.purged, ); + if outcome.carried > 0 { + println!( + " {} event(s) belonging to the aggregate itself were moved across", + outcome.carried + ); + } if outcome.purged == 0 && !purge_old { println!( " the previous target still holds this aggregate's events; re-run with --purge-old to remove them" @@ -269,6 +275,12 @@ fn print_report(report: &Report) { aggregate.deleted_from_aggregate, aggregate.deleted_from_sources, ); + if aggregate.kept_local > 0 { + println!( + " {} event(s) kept local: they belong to the aggregate itself, not to a source", + aggregate.kept_local + ); + } for conflict in &aggregate.conflicts { println!( " conflict: {} changed in both places; kept the version from `{}`", @@ -286,9 +298,6 @@ fn print_report(report: &Report) { fn describe(skipped: &reconcile::Skipped) -> String { match skipped { - reconcile::Skipped::NoSink { aggregate_uid } => format!( - "`{aggregate_uid}` was created in the aggregate, but no default_sink is set to route it to" - ), reconcile::Skipped::ReadOnlySource { aggregate_uid, source_id, diff --git a/src/mirror.rs b/src/mirror.rs index f846344..1bf3f61 100644 --- a/src/mirror.rs +++ b/src/mirror.rs @@ -30,6 +30,15 @@ const OWN_PROPERTIES: &[&str] = &[SOURCE_PROPERTY, ORIGIN_UID_PROPERTY, ATTENDEE /// Live scheduling properties, whose presence is what makes a server send mail. const SCHEDULING_PROPERTIES: &[&str] = &["ATTENDEE", "ORGANIZER"]; +/// The marker that keeps an event in the aggregate rather than filing it under +/// a source. Reserved: an endpoint may not be given this id. +pub const LOCAL_SINK: &str = "local"; + +/// Whether a routing hint asks for the event to stay where it is. +pub fn is_local_sink(hint: &str) -> bool { + hint.eq_ignore_ascii_case(LOCAL_SINK) +} + /// Builds the aggregate copy of a source event. pub fn to_aggregate( source: &Calendar, @@ -52,6 +61,24 @@ pub fn to_aggregate( mirrored } +/// Applies a target's scheduling rule to an event that is not a mirror. +/// +/// An event living only in the aggregate still has to obey the rule that writes +/// to an aggregate never emit scheduling mail — moving one between calendars is +/// as capable of mailing a guest list as mirroring is. It carries no provenance, +/// having none, so this is the demotion alone. +pub fn make_inert( + calendar: &Calendar, + owner: Option<&str>, + suppression: SchedulingSuppression, +) -> Calendar { + let mut inert = calendar.clone(); + if suppression == SchedulingSuppression::None { + demote_attendees(&mut inert, owner); + } + inert +} + /// Removes the live guest list, keeping its information in inert form. fn demote_attendees(calendar: &mut Calendar, owner: Option<&str>) { let guests: Vec = calendar diff --git a/src/reconcile.rs b/src/reconcile.rs index 4eb4635..0d339ef 100644 --- a/src/reconcile.rs +++ b/src/reconcile.rs @@ -83,9 +83,6 @@ pub struct Conflict { /// Something that could not be done, and why. #[derive(Debug, Clone, PartialEq, Eq)] pub enum Skipped { - /// A user created an event in the aggregate, but no sink is configured to - /// route it to, and guessing one would put it in the wrong calendar. - NoSink { aggregate_uid: String }, /// An edit was made to a mirror of a read-only feed, which cannot accept it. ReadOnlySource { aggregate_uid: String, @@ -109,6 +106,9 @@ pub struct Outcome { pub links: BTreeMap, pub conflicts: Vec, pub skipped: Vec, + /// Events left in the aggregate rather than filed under a source. Not a + /// failure: an aggregate is also a calendar, and these are its own events. + pub kept_local: Vec, } #[derive(Debug, Error, PartialEq)] @@ -140,6 +140,7 @@ pub fn reconcile( links: BTreeMap::new(), conflicts: Vec::new(), skipped: Vec::new(), + kept_local: Vec::new(), }; let mut live: HashSet = HashSet::new(); let mut deletions = 0usize; @@ -360,6 +361,14 @@ fn route_new_events( // An event may name the source it belongs in; otherwise the aggregate's // configured sink takes it. let requested = mirror::routing_hint(&item.calendar); + // `@local` keeps the event where it was written. Unlike every other + // marker it is deliberately not stripped: the others have done their + // job once the event reaches its source, while this one has to survive + // to make the same decision on every later cycle. + if requested.as_deref().is_some_and(mirror::is_local_sink) { + outcome.kept_local.push(uid.clone()); + continue; + } let wanted = requested.as_deref().or(policy.default_sink); // Case-insensitively: the name is typed into a calendar app by hand, and // capitalisation is not worth failing over. @@ -372,16 +381,18 @@ fn route_new_events( .filter(|sink| sink.writable); let Some(sink) = sink else { - outcome.skipped.push(match requested { - Some(requested) => Skipped::UnknownSink { + match requested { + Some(requested) => outcome.skipped.push(Skipped::UnknownSink { aggregate_uid: uid.clone(), requested, available: writable_sinks(sources), - }, - None => Skipped::NoSink { - aggregate_uid: uid.clone(), - }, - }); + }), + // Nothing says where this event should go, so it stays where it + // was written. With no default_sink configured that is the + // mode, not an omission — the aggregate is a calendar the user + // can also write in directly. + None => outcome.kept_local.push(uid.clone()), + } continue; }; @@ -879,7 +890,9 @@ mod tests { ); } - /// Guessing a sink would put the event in the wrong calendar, so refuse. + /// With nowhere configured to file it, the event stays where it was + /// written. Guessing a sink would put it in the wrong calendar, and this is + /// how an aggregate holds events of its own. #[test] fn without_a_sink_a_user_created_event_is_left_alone() { let sources = map(vec![]); @@ -887,12 +900,30 @@ mod tests { let outcome = reconcile(policy(), &[view(&sources, true)], &aggregate, None).expect("ok"); assert!(outcome.actions.is_empty()); - assert_eq!( - outcome.skipped, - vec![Skipped::NoSink { - aggregate_uid: "hand-written@phone".into() - }] - ); + assert!(outcome.skipped.is_empty(), "{:?}", outcome.skipped); + assert_eq!(outcome.kept_local, vec!["hand-written@phone".to_string()]); + } + + /// `@local` opts one event out even where a default sink would take it. + #[test] + fn the_local_marker_keeps_an_event_out_of_every_source() { + let sources = map(vec![]); + let mut tagged = item("hand-written@phone", "Dentist"); + tagged + .calendar + .set_property("VEVENT", "DESCRIPTION", "Bring the referral\\n\\n@local"); + tagged.hash = tagged.calendar.content_hash(); + let aggregate = map(vec![tagged]); + + let policy = Policy { + default_sink: Some(SRC), + ..policy() + }; + let outcome = reconcile(policy, &[view(&sources, true)], &aggregate, None).expect("ok"); + + assert!(outcome.actions.is_empty(), "{:?}", outcome.actions); + assert!(outcome.skipped.is_empty(), "{:?}", outcome.skipped); + assert_eq!(outcome.kept_local, vec!["hand-written@phone".to_string()]); } /// Our own mirror appearing inside a source must not be mirrored again. diff --git a/src/retarget.rs b/src/retarget.rs index 3b1f54c..0cbd347 100644 --- a/src/retarget.rs +++ b/src/retarget.rs @@ -55,6 +55,8 @@ pub struct Outcome { /// Events flushed to a sink before the move, which existed nowhere else. pub routed_first: usize, pub materialised: usize, + /// Events belonging to the aggregate itself, moved rather than re-derived. + pub carried: usize, pub purged: usize, } @@ -117,7 +119,14 @@ pub fn retarget( false, )?; - let materialised = materialise(config, aggregate, new_target, state_dir, &mut state)?; + let (materialised, carried) = materialise( + config, + aggregate, + &old_target, + new_target, + state_dir, + &mut state, + )?; let purged = if purge_old { purge(aggregate_id, &old_target, state_dir, &state)? } else { @@ -138,6 +147,7 @@ pub fn retarget( to: new_target.id.clone(), routed_first: settled.written_back, materialised, + carried, purged, }) } @@ -150,10 +160,11 @@ pub fn retarget( fn materialise( config: &Config, aggregate: &crate::config::Aggregate, + old_target: &Endpoint, new_target: &Endpoint, state_dir: &Path, state: &mut State, -) -> Result { +) -> Result<(usize, usize), RetargetError> { let new_dir = vdir_path(state_dir, &new_target.id); let existing = vdir::read(&new_dir)?; let suppression = new_target.kind.scheduling_suppression(); @@ -188,10 +199,30 @@ fn materialise( } written += 1; } + + // Events created in the aggregate and never filed under a source exist only + // on the old target, so there is nothing to re-derive them from: they are + // moved rather than rebuilt. Without this they would sit on the calendar + // being left behind while everything around them moved on. + // + // The new target's scheduling rule still applies. This is a write to an + // aggregate like any other, and a guest list carried live onto a server + // that schedules would mail everyone on it. + let mut carried = 0; + for (uid, item) in vdir::read(&vdir_path(state_dir, &old_target.id))? { + if provenance::is_derived(&uid) { + continue; + } + let inert = mirror::make_inert(&item.calendar, new_target.kind.owner(), suppression); + let replaces = existing.get(&uid).map(|held| held.path.clone()); + vdir::write(&new_dir, &uid, &inert, replaces.as_deref())?; + carried += 1; + } + state .aggregate_mut(&aggregate.id, target_of(new_target)) .links = updated; - Ok(written) + Ok((written, carried)) } /// Removes this aggregate's mirrors from the calendar it has left. @@ -280,6 +311,53 @@ default_sink = "src" ) } + /// An event belonging to the aggregate itself has to follow the move. + /// + /// There is nothing to re-derive it from, so before this it stayed on the + /// calendar being left behind while every mirrored event moved on — quietly, + /// which is the worst way for a calendar to lose an appointment. + #[test] + fn an_event_belonging_to_the_aggregate_is_carried_to_the_new_target() { + // No default_sink: events written into the aggregate stay there. + let config: Config = toml::from_str(&CONFIG.replace(r#"default_sink = "src""#, "")) + .expect("config should parse"); + let dir = seed(&config); + + let old = vdir_path(dir.path(), "agg1"); + fs::write( + old.join("dentist.ics"), + event("hand-written@phone", "Dentist"), + ) + .expect("write"); + settle(&config, dir.path()); + + let outcome = retarget(&config, dir.path(), "unified", "agg2", false).expect("retarget"); + + assert_eq!(outcome.carried, 1, "{outcome:?}"); + let moved = vdir::read(&vdir_path(dir.path(), "agg2")).expect("read"); + assert!( + moved.contains_key("hand-written@phone"), + "the aggregate's own event did not follow: {:?}", + moved.keys().collect::>() + ); + assert!( + moved["hand-written@phone"] + .calendar + .to_ics() + .contains("Dentist"), + "it arrived without its content" + ); + // And it is still not a mirror of anything, so nothing was invented for it. + let state = State::load(&dir.path().join(crate::state::FILE_NAME)).expect("state"); + assert!( + !state + .aggregate("unified") + .expect("aggregate") + .links + .contains_key("hand-written@phone") + ); + } + /// Reconciles one aggregate and records the result, without running a full /// cycle. A full cycle would contact Google for real, which these tests have /// no business doing — they are about what happens to local files. diff --git a/src/status.rs b/src/status.rs index e13d427..6932cb3 100644 --- a/src/status.rs +++ b/src/status.rs @@ -69,7 +69,7 @@ fn describe_routing(out: &mut String, config: &Config, aggregate: &Aggregate) { None => { let _ = writeln!( out, - " new events are left alone: no default_sink is configured" + " new events stay here: no default_sink is configured, so the aggregate keeps its own" ); } } @@ -84,6 +84,11 @@ fn describe_routing(out: &mut String, config: &Config, aggregate: &Aggregate) { for sink in sinks { let _ = writeln!(out, " @{sink}"); } + let _ = writeln!( + out, + " @{} (keep the event here, in this calendar only)", + crate::mirror::LOCAL_SINK + ); } #[cfg(test)] @@ -147,12 +152,18 @@ default_sink = "gcal" assert!(report().contains("new events go to `gcal`")); } + /// The escape hatch has to be as findable as the destinations. + #[test] + fn it_names_the_marker_that_keeps_an_event_local() { + assert!(report().contains("@local"), "{}", report()); + } + #[test] fn it_says_when_nothing_would_be_routed() { let sinkless: Config = toml::from_str(&CONFIG.replace(r#"default_sink = "gcal""#, "")).expect("config"); let dir = tempfile::tempdir().expect("temp dir"); let report = render(&sinkless, dir.path()).expect("render"); - assert!(report.contains("no default_sink is configured"), "{report}"); + assert!(report.contains("the aggregate keeps its own"), "{report}"); } } diff --git a/src/sync.rs b/src/sync.rs index 1494c70..fe491d4 100644 --- a/src/sync.rs +++ b/src/sync.rs @@ -123,6 +123,7 @@ pub struct AggregateReport { pub deleted_from_sources: usize, pub conflicts: Vec, pub skipped: Vec, + pub kept_local: usize, } /// Runs the local half of a cycle: reconcile every aggregate and apply the result. @@ -564,6 +565,7 @@ pub(crate) fn sync_aggregate( deleted_from_sources: 0, conflicts: outcome.conflicts, skipped: outcome.skipped, + kept_local: outcome.kept_local.len(), }; for action in &outcome.actions { diff --git a/tests/caldav.rs b/tests/caldav.rs index 49ecc63..88eac3c 100644 --- a/tests/caldav.rs +++ b/tests/caldav.rs @@ -29,6 +29,12 @@ const HAND_WRITTEN: &str = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:-//phone//E DTSTART:20260911T140000Z\r\nDTEND:20260911T150000Z\r\nSUMMARY:Dentist\r\n\ END:VEVENT\r\nEND:VCALENDAR\r\n"; +/// The same event, asking to be left where it was written. +const KEPT_LOCAL: &str = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:-//phone//EN\r\n\ + BEGIN:VEVENT\r\nUID:kept@phone\r\nDTSTAMP:20260101T000000Z\r\n\ + DTSTART:20260912T140000Z\r\nDTEND:20260912T150000Z\r\nSUMMARY:Haircut\r\n\ + DESCRIPTION:Around the corner\\n\\n@local\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n"; + const HOLIDAY_FEED: &str = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:-//feed//EN\r\n\ BEGIN:VEVENT\r\nUID:newyear@feed\r\nDTSTAMP:20260101T000000Z\r\n\ DTSTART;VALUE=DATE:20260101\r\nDTEND;VALUE=DATE:20260102\r\n\ @@ -329,3 +335,51 @@ fn an_event_created_in_a_sinkless_aggregate_stays_where_it_is() { ); } } + +/// `@local` opts one event out of a default sink that would otherwise take it. +/// +/// The marker has to survive, unlike every other one: it is stripped markers +/// that have done their job on arrival, whereas this event never leaves, so the +/// next cycle has to be able to reach the same decision. +#[test] +fn the_local_marker_keeps_an_event_out_of_a_configured_sink() { + let Some(fixture) = fixture() else { return }; + + assert!(fixture.calcalist.run(&["sync"]).succeeded()); + fixture + .server + .put_event("unified", "haircut.ics", KEPT_LOCAL); + + for cycle in 1..=3 { + let run = fixture.calcalist.run(&["sync"]); + assert!( + run.succeeded(), + "cycle {cycle} failed\n{}\n{}", + run.stdout, + run.stderr + ); + assert!( + run.stdout.contains("1 event(s) kept local"), + "cycle {cycle} should report it as kept, not skipped:\n{}", + run.stdout + ); + + let kept = fixture + .server + .stored("unified") + .into_iter() + .find(|item| item.contains("Haircut")) + .expect("the event should still be in the aggregate"); + // Stripping it would let the default sink claim the event next cycle. + assert!(kept.contains("@local"), "cycle {cycle}: {kept}"); + + assert!( + !fixture + .server + .stored("work") + .iter() + .any(|item| item.contains("Haircut")), + "cycle {cycle}: it reached the default sink anyway" + ); + } +} From ef6482ed62613319fde1ada6671fc1089589610e Mon Sep 17 00:00:00 2001 From: randogoth Date: Thu, 10 Sep 2026 16:17:34 +0300 Subject: [PATCH 07/10] Finish M2: run lock, systemd units, discover; drop the web UI MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Designing the configuration UI in full made the case against building it. Its audience would be people who find TOML hard, but with bring-your-own OAuth client settled, every user must first create a Google Cloud project, configure a consent screen and put a secret in a keyring — a far higher bar than editing thirty lines of config. Anyone who clears it can edit the file; anyone who cannot never reaches the file. Against that stood three dependencies, five modules, an auth.rs refactor and a security surface guarding something that reads the config and touches the keyring. `doctor` and `status` had already absorbed most of what it was for. The reasoning is recorded in TODO.md and SPECS.md rather than left as an apparent oversight. The run lock is not a UI feature and closes a gap that already existed: nothing stopped a timer firing into a hand-run cycle, and two cycles interleaving writes over the same vdirs is what the design otherwise avoids. flock is used rather than a pid file because the kernel releases it however the process ends, so a crash cannot leave a lock to clear by hand — which also means a lock we failed to take is held by a live process, so the pid in it is worth reporting. The one idea worth keeping from the UI design was collection discovery, which needed no web layer. `calcalist discover` prints a ready-to-paste endpoint block per calendar a server offers, removing the most error-prone field in the config. pimsync's discovery output is undocumented, so the format was established against a real server first. Two things it teaches: everything arrives on stdout including failures, and a pair has two storages, so pimsync reports the scratch vdir's contents too — parsing anchors on the heading naming the server, or a probe directory's leftovers would be offered as the user's calendars. Verified against Posteo as well as Radicale: all four calendars found, the first matching the URL already configured. Also fixes a real defect in the test harness rather than its symptom. Ports were chosen by binding one and letting go, so two tests could pick the same number — and the loser's readiness check then succeeded against the winner's server, silently sharing it. Startup now confirms the child we spawned is the one alive, retries on another port if not, and waits for a real HTTP response rather than an open socket. Co-Authored-By: Claude Opus 5 --- Cargo.lock | 1 + Cargo.toml | 1 + README.md | 62 +++++---- SPECS.md | 4 +- TODO.md | 44 +++++-- src/cli.rs | 15 ++- src/lock.rs | 144 +++++++++++++++++++++ src/main.rs | 102 ++++++++++++--- src/pimsync.rs | 255 ++++++++++++++++++++++++++++++++++++++ systemd/calcalist.service | 15 +++ systemd/calcalist.timer | 15 +++ tests/caldav.rs | 114 +++++++++++++++++ tests/support/mod.rs | 128 ++++++++++++++----- 13 files changed, 808 insertions(+), 92 deletions(-) create mode 100644 src/lock.rs create mode 100644 systemd/calcalist.service create mode 100644 systemd/calcalist.timer diff --git a/Cargo.lock b/Cargo.lock index 1299f71..2621f6f 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -117,6 +117,7 @@ dependencies = [ "blake3", "clap", "jiff", + "rustix", "serde", "serde_json", "sha2", diff --git a/Cargo.toml b/Cargo.toml index c8cac75..7f41e14 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -10,6 +10,7 @@ publish = false blake3 = "1.8" clap = { version = "4.6", features = ["derive"] } jiff = "0.2" +rustix = { version = "1.1", features = ["fs"] } serde = { version = "1.0", features = ["derive"] } serde_json = "1.0" sha2 = "0.11" diff --git a/README.md b/README.md index f0034bf..5e191c8 100644 --- a/README.md +++ b/README.md @@ -323,6 +323,7 @@ calcalist sync [--dry-run] [--force] calcalist status calcalist doctor calcalist prune [--force] +calcalist discover --username [--secret-command ] calcalist google login calcalist aggregate retarget --to [--keep-old | --purge-old] ``` @@ -334,6 +335,10 @@ calcalist aggregate retarget --to [--keep-old | --purge-old] - **`status`** lists endpoints and aggregates, how many events each aggregate is mirroring, and the routing markers that would work. - **`doctor`** checks the environment and configuration. Run it first. +- **`discover`** asks a CalDAV server which calendars it has and prints an + `[[endpoint]]` block for each, ready to paste. Give it the account or + principal URL rather than a single calendar. This is the easy way to get the + `url` field right. - **`prune`** reports local mirrors belonging to endpoints your configuration no longer names. It only lists them until you pass `--force`, since an endpoint may just have been renamed. @@ -346,7 +351,7 @@ calcalist aggregate retarget --to [--keep-old | --purge-old] events calcalist put there. Exit codes: `0` all well, `1` something failed or an endpoint could not be -reached, `2` not implemented. +reached. If a Google endpoint cannot be reached — a lapsed token, no network — the cycle does not stop. The endpoint is named, only the aggregates that depend on it @@ -372,48 +377,37 @@ configuration between machines; leave this behind. ## Running it regularly -There are no packaged units yet. A systemd user timer does the job: - -`~/.config/systemd/user/calcalist.service` - -```ini -[Unit] -Description=Synchronise calendars with calcalist - -[Service] -Type=oneshot -ExecStart=%h/.local/bin/calcalist sync -``` - -`~/.config/systemd/user/calcalist.timer` - -```ini -[Unit] -Description=Synchronise calendars every 15 minutes - -[Timer] -OnBootSec=2m -OnUnitActiveSec=15m -Persistent=true - -[Install] -WantedBy=timers.target -``` +Units ship in [systemd/](systemd/): ```sh +install -Dm644 systemd/calcalist.service ~/.config/systemd/user/calcalist.service +install -Dm644 systemd/calcalist.timer ~/.config/systemd/user/calcalist.timer +systemctl --user daemon-reload systemctl --user enable --now calcalist.timer ``` -A cycle is one-shot, and systemd will not start the service again while a run is -still going, so a timer is all the scheduling needed. calcalist takes no lock of -its own, though, so avoid running `calcalist sync` by hand while the timer might -fire. If your secrets come from a keyring that needs an unlocked session, the +The service expects the binary at `~/.local/bin/calcalist`; edit `ExecStart` if +yours is elsewhere. The timer runs every 15 minutes with a randomised delay of +up to two minutes, which spreads requests instead of every installation calling +Google on the quarter hour — its quota is enforced per minute, per project. + +A cycle that could not reach an endpoint exits non-zero deliberately, so a +lapsed token surfaces as a failed unit in `systemctl --user status calcalist` +rather than passing unnoticed. `journalctl --user -u calcalist` has the report. + +Only one cycle runs at a time: calcalist takes a lock on its state directory, so +a timer firing while you are running `calcalist sync` by hand is refused with a +message naming the process that holds it, rather than the two interleaving their +writes. If your secrets come from a keyring that needs an unlocked session, the timer will only work while you are logged in. ## Not yet -- **The web configuration interface.** `calcalist serve` exits 2. -- **Packaged systemd units**, as above. - **A failing `pimsync sync` stops the whole CalDAV leg.** Google endpoints are isolated from each other, but pimsync is a single process covering every CalDAV and feed pair, and a failure does not say which pair it belongs to. + +There is deliberately no web interface. It was designed and then dropped: the +Google OAuth setup, which no interface can remove, is a far higher bar than +editing this file, so a UI would have served an audience that never gets as far +as the config. `doctor`, `status` and `discover` cover what it was for. diff --git a/SPECS.md b/SPECS.md index 710f76a..a00697b 100644 --- a/SPECS.md +++ b/SPECS.md @@ -4,7 +4,9 @@ Small Rust based commandline tool with utility systemd service file that can aggregate and sync events between CalDAV/Google Calendar/iCal. Users who have a Google Calendar can create a new calendar that syncs events from several CalDAV calendars. CalDAV users can sync several Google Calendars into one calendar. This should be doable both ways, so the aggregating calendar needs to be able to distinguish where its events came from and also have a way to create new events that get synced to the correct source. In addition iCal feeds can also be aggregated but they remain one way only. -Configuration via a simple web interface. Configuration is saved in a portable toml file that can be easily migrated. +Configuration is saved in a portable toml file that can be easily migrated. + +The web interface this originally called for was designed and then dropped. Setting up Google's OAuth client — a Cloud project, a consent screen, a secret in a keyring — is a far higher bar than editing thirty lines of TOML, and no interface can remove it, so a configuration UI would have served an audience that never reaches the configuration. `calcalist doctor` reports every problem in one pass, `calcalist status` names the routing markers an event can carry, and `calcalist discover` prints ready-to-paste endpoint blocks for a server's calendars — which is what the interface was actually for. ## Architecture diff --git a/TODO.md b/TODO.md index 34b4537..b1240a5 100644 --- a/TODO.md +++ b/TODO.md @@ -131,15 +131,45 @@ calendar: - [x] A retired endpoint's mirror is reported by `prune` and removed under `--force`. +## M2 — packaging (done) + +- [x] Run lock (`lock.rs`) — `flock` on the state directory, held by `sync`, + `aggregate retarget` and `prune --force`. Not a UI feature: nothing + previously stopped a timer firing into a hand-run cycle, and two cycles + interleaving writes over the same vdirs is what the design otherwise + avoids. The kernel releases it however the process ends, so a crash + cannot leave a lock to clear by hand; the refusal names the holding pid. +- [x] systemd user units in `systemd/` — `calcalist.service` (oneshot) and + `calcalist.timer`, with `RandomizedDelaySec` so installations do not all + call Google on the quarter hour, and no filesystem or IPC sandboxing + because the secret commands need the session keyring over D-Bus. +- [x] `calcalist discover` — asks a CalDAV server which calendars it has and + prints a ready-to-paste `[[endpoint]]` block for each. The `url` field is + the most error-prone thing in the config, and providers rarely show it. +- [x] `serve` removed from the CLI, and with it the last `unimplemented` + command, so every command the binary advertises now does something. + +### The web interface, dropped + +Specified from the start and designed in full before being dropped. The +reasoning, so it is not rediscovered as an oversight: + +- Its audience would be people who find TOML hard. But with bring-your-own + OAuth client settled, every user must first create a Google Cloud project, + configure a consent screen and put a secret in a keyring — a far higher bar + than editing thirty lines of config. Anyone who clears it can edit the file; + anyone who cannot never reaches the file. +- The cost was three dependencies, five modules, an `auth.rs` refactor and a + security surface (token, `Host` validation, CSRF, secrets through subprocess + stdin) guarding something that reads the config and touches the keyring — + against SPECS.md's own "no speculative features or dependencies". +- `doctor`, `status` and `discover` had already absorbed what it was for. + +The one idea worth keeping from the design was collection discovery, which +needed no web layer at all. + ## Known gaps - [ ] A `pimsync sync` that fails takes the whole CalDAV leg with it. Unlike the Google side this cannot be narrowed: pimsync is one process covering every pair, so a failure does not say which pair it belongs to. - -## M2 — interface and packaging - -- [ ] axum configuration UI, bound to 127.0.0.1 -- [ ] Trigger `google login` from the UI (the loopback handler itself already exists - in `google/auth.rs`) -- [ ] systemd user units: `calcalist.service` (oneshot) and `calcalist.timer` diff --git a/src/cli.rs b/src/cli.rs index 0dbcd93..d072276 100644 --- a/src/cli.rs +++ b/src/cli.rs @@ -32,10 +32,17 @@ pub enum Command { }, /// Show endpoints, aggregates and the last sync result Status, - /// Serve the configuration web interface on localhost - Serve { - #[arg(long, default_value_t = 8723)] - port: u16, + /// List the calendars a CalDAV server offers, as endpoint blocks to paste + Discover { + /// The account or principal URL to enumerate, not a single calendar + url: String, + /// Username to authenticate as + #[arg(long)] + username: String, + /// Command printing the password, exactly as it would appear in the + /// configuration. Secrets are never passed as arguments. + #[arg(long, value_name = "COMMAND")] + secret_command: Option, }, /// Google account operations Google { diff --git a/src/lock.rs b/src/lock.rs new file mode 100644 index 0000000..bde4e86 --- /dev/null +++ b/src/lock.rs @@ -0,0 +1,144 @@ +//! A lock held for the duration of anything that writes. +//! +//! A cycle rewrites vdirs and the state file, and two of them running at once +//! would interleave those writes — the timer firing while someone runs `sync` by +//! hand is the ordinary way that happens. `flock` is used rather than a pid file +//! because the kernel releases it when the process ends, however it ends, so a +//! crash cannot leave a lock behind for someone to clear by hand. + +use std::fs::{File, OpenOptions}; +use std::io::{Read, Seek, SeekFrom, Write}; +use std::path::{Path, PathBuf}; + +use rustix::fs::{FlockOperation, flock}; +use thiserror::Error; + +const FILE_NAME: &str = "lock"; + +#[derive(Debug, Error)] +pub enum LockError { + #[error("could not open {path}: {source}")] + Open { + path: PathBuf, + #[source] + source: std::io::Error, + }, + #[error( + "another calcalist is already running (pid {pid}); it holds {path}. \ + Wait for it to finish rather than running two cycles at once." + )] + Held { pid: String, path: PathBuf }, + #[error("could not lock {path}: {source}")] + Failed { + path: PathBuf, + #[source] + source: std::io::Error, + }, +} + +/// An exclusive claim on the state directory, released when dropped. +/// +/// The lock lives in the open file descriptor, so there is nothing to undo: it +/// goes when the `File` closes. The file itself is deliberately left behind — +/// unlinking it would let a waiter take a lock on an inode nobody else can see. +#[derive(Debug)] +pub struct Lock { + _file: File, +} + +pub fn path(state_dir: &Path) -> PathBuf { + state_dir.join(FILE_NAME) +} + +/// Claims the state directory, or reports who already has it. +pub fn acquire(state_dir: &Path) -> Result { + let path = path(state_dir); + if let Some(parent) = path.parent() { + std::fs::create_dir_all(parent).map_err(|source| LockError::Open { + path: path.clone(), + source, + })?; + } + let mut file = OpenOptions::new() + .read(true) + .write(true) + .create(true) + .truncate(false) + .open(&path) + .map_err(|source| LockError::Open { + path: path.clone(), + source, + })?; + + if let Err(error) = flock(&file, FlockOperation::NonBlockingLockExclusive) { + if error.kind() == std::io::ErrorKind::WouldBlock { + // Whoever holds it is alive: the kernel would have released the lock + // otherwise. So the pid in the file is worth reporting. + return Err(LockError::Held { + pid: holder(&mut file), + path, + }); + } + return Err(LockError::Failed { + path, + source: error.into(), + }); + } + + // Record who holds it, for the benefit of whoever is refused next. + let _ = file.set_len(0); + let _ = file.seek(SeekFrom::Start(0)); + let _ = write!(file, "{}", std::process::id()); + let _ = file.flush(); + Ok(Lock { _file: file }) +} + +/// The pid recorded in the lock file, or a placeholder when it says nothing. +fn holder(file: &mut File) -> String { + let mut recorded = String::new(); + if file.seek(SeekFrom::Start(0)).is_ok() + && file.read_to_string(&mut recorded).is_ok() + && !recorded.trim().is_empty() + { + return recorded.trim().to_string(); + } + "unknown".to_string() +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn a_second_claim_is_refused_while_the_first_is_held() { + let dir = tempfile::tempdir().expect("temp"); + let first = acquire(dir.path()).expect("the first claim should succeed"); + + let error = acquire(dir.path()).expect_err("the second should be refused"); + assert!(matches!(error, LockError::Held { .. }), "{error:?}"); + // The refusal has to name the holder, or it is not actionable. + assert!( + error.to_string().contains(&std::process::id().to_string()), + "{error}" + ); + + drop(first); + acquire(dir.path()).expect("the lock should be free again"); + } + + /// The file has to survive, so the next run locks the same inode. + #[test] + fn the_lock_file_outlives_the_lock() { + let dir = tempfile::tempdir().expect("temp"); + drop(acquire(dir.path()).expect("claim")); + assert!(path(dir.path()).exists()); + } + + #[test] + fn a_missing_state_directory_is_created() { + let dir = tempfile::tempdir().expect("temp"); + let nested = dir.path().join("not").join("there"); + acquire(&nested).expect("claim"); + assert!(path(&nested).exists()); + } +} diff --git a/src/main.rs b/src/main.rs index d30f7b9..8a7bd9a 100644 --- a/src/main.rs +++ b/src/main.rs @@ -5,6 +5,7 @@ mod config; mod doctor; mod google; mod ical; +mod lock; mod mirror; mod paths; mod pimsync; @@ -43,6 +44,15 @@ fn main() -> ExitCode { command: Command::Prune { force }, .. } => run_prune(cli, *force), + Cli { + command: + Command::Discover { + url, + username, + secret_command, + }, + .. + } => run_discover(url, username, secret_command.as_deref()), cli @ Cli { command: Command::Google { @@ -60,7 +70,6 @@ fn main() -> ExitCode { }, .. } => run_retarget(cli, id, to, *purge_old), - Cli { command, .. } => unimplemented(command), } } @@ -85,6 +94,12 @@ fn run_sync(cli: &Cli, dry_run: bool, force: bool) -> ExitCode { Ok(dir) => dir, Err(error) => return fail(&error), }; + // Held for the whole cycle: a timer firing mid-run would otherwise interleave + // its writes with this one over the same vdirs. + let _lock = match lock::acquire(&state_dir) { + Ok(lock) => lock, + Err(error) => return fail(&error), + }; match sync::run(&config, &state_dir, dry_run, force) { Ok(report) => { print_report(&report); @@ -101,6 +116,62 @@ fn run_sync(cli: &Cli, dry_run: bool, force: bool) -> ExitCode { } } +/// Lists a server's calendars as configuration ready to paste. +/// +/// The URL of a CalDAV calendar is the most error-prone thing in the config: it +/// must name the collection exactly, and providers rarely show it. Asking the +/// server is more reliable than reading a web interface. +fn run_discover(url: &str, username: &str, secret_command: Option<&str>) -> ExitCode { + let state_dir = match paths::state_dir() { + Ok(dir) => dir, + Err(error) => return fail(&error), + }; + let found = match pimsync::discover(&state_dir, url, username, secret_command) { + Ok(found) => found, + Err(error) => return fail(&error), + }; + if found.is_empty() { + eprintln!("calcalist: {url} offered no calendars"); + return ExitCode::FAILURE; + } + + let base = url.split_once("://").and_then(|(scheme, rest)| { + rest.split_once('/') + .map(|(authority, _)| format!("{scheme}://{authority}")) + }); + println!( + "# {} calendar(s) found. Paste what you want into calcalist.toml.", + found.len() + ); + // Some servers name calendars with opaque ids — Posteo's look like + // `cydhlw` — and the id is what an event carries after `@` to be routed + // here, so it is worth renaming before it ends up in anyone's notes. + println!("# Ids come from the URL. Rename them: an id is what you type after `@`."); + for collection in &found { + let full = match &base { + Some(base) => format!("{base}{}", collection.href), + // pimsync reported something that is not a path; show it as it came + // rather than gluing it onto an origin and inventing a URL. + None => collection.href.clone(), + }; + println!("\n[[endpoint]]"); + println!("id = {}", quoted(&collection.id)); + println!("type = \"caldav\""); + println!("url = {}", quoted(&full)); + println!("username = {}", quoted(username)); + match secret_command { + Some(command) => println!("secret_command = {}", quoted(command)), + None => println!("# secret_command = \"secret-tool lookup ...\""), + } + } + ExitCode::SUCCESS +} + +/// A TOML basic string, so a value containing quotes or backslashes survives. +fn quoted(value: &str) -> String { + format!("\"{}\"", value.replace('\\', "\\\\").replace('"', "\\\"")) +} + /// Removes local mirrors of endpoints the configuration no longer names. fn run_prune(cli: &Cli, force: bool) -> ExitCode { let (config, _) = match Config::load(cli.config.as_deref()) { @@ -111,6 +182,16 @@ fn run_prune(cli: &Cli, force: bool) -> ExitCode { Ok(dir) => dir, Err(error) => return fail(&error), }; + // Only the destructive half needs the lock; listing reads nothing a cycle + // could be part-way through changing. + let _lock = if force { + match lock::acquire(&state_dir) { + Ok(lock) => Some(lock), + Err(error) => return fail(&error), + } + } else { + None + }; let orphans = sync::orphan_vdirs(&config, &state_dir); if orphans.is_empty() { println!("nothing to prune: every local mirror belongs to a configured endpoint"); @@ -184,6 +265,10 @@ fn run_retarget(cli: &Cli, id: &str, to: &str, purge_old: bool) -> ExitCode { Ok(dir) => dir, Err(error) => return fail(&error), }; + let _lock = match lock::acquire(&state_dir) { + Ok(lock) => lock, + Err(error) => return fail(&error), + }; match retarget::retarget(&config, &state_dir, id, to, purge_old) { Ok(outcome) => { println!( @@ -329,18 +414,3 @@ fn fail(error: &dyn std::error::Error) -> ExitCode { } ExitCode::FAILURE } - -/// M1 is still landing: `serve`, `status`, `google` and `aggregate` come later. -fn unimplemented(command: &Command) -> ExitCode { - let name = match command { - Command::Sync { .. } => "sync", - Command::Status => "status", - Command::Serve { .. } => "serve", - Command::Google { .. } => "google", - Command::Aggregate { .. } => "aggregate", - Command::Prune { .. } => "prune", - Command::Doctor => "doctor", - }; - eprintln!("calcalist: `{name}` is not implemented yet"); - ExitCode::from(2) -} diff --git a/src/pimsync.rs b/src/pimsync.rs index 5731345..1e1809b 100644 --- a/src/pimsync.rs +++ b/src/pimsync.rs @@ -276,6 +276,172 @@ pub fn write_config_for( Ok(path) } +/// The storage name the probe config gives the server being examined. Parsing +/// anchors on it, because pimsync discovers *both* storages in a pair and prints +/// their collections under one heading each. +const PROBE_REMOTE: &str = "probe_remote"; + +/// One collection found on a server. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Collection { + /// The path pimsync reported, e.g. `/calendars/me/work/`. + pub href: String, + /// A usable endpoint id, taken from the last meaningful path segment. + pub id: String, +} + +/// Asks pimsync which calendars a server offers. +/// +/// `url` is a container to enumerate — an account or principal path — rather +/// than a single calendar, so `split_collection_url` does not apply here. +/// +/// `pimsync discover` is documented as informational and as not altering +/// pimsync's state, so pointing it at a server the user has not configured yet +/// is safe. The probe configuration it needs is written under a name of its own +/// so it can never be mistaken for, or overwrite, the generated `pimsync.conf`. +pub fn discover( + state_dir: &Path, + url: &str, + username: &str, + secret_command: Option<&str>, +) -> Result, PimsyncError> { + let config_path = state_dir.join("discover.conf"); + let scratch = state_dir.join("discover-vdir"); + fs::create_dir_all(&scratch).map_err(|source| PimsyncError::Write { + path: scratch.clone(), + source, + })?; + fs::write( + &config_path, + probe_config(state_dir, &scratch, url, username, secret_command), + ) + .map_err(|source| PimsyncError::Write { + path: config_path.clone(), + source, + })?; + + let output = Command::new(BINARY) + .arg("-c") + .arg(&config_path) + .arg("discover") + .output() + .map_err(|error| { + if error.kind() == io::ErrorKind::NotFound { + PimsyncError::NotFound + } else { + PimsyncError::Spawn(error) + } + })?; + + // Everything, including the failures, comes back on stdout. + let text = String::from_utf8_lossy(&output.stdout).into_owned(); + let _ = fs::remove_dir_all(&scratch); + if !output.status.success() { + return Err(PimsyncError::Command { + command: "discover".to_string(), + message: strip_ansi(&text).trim().to_string(), + }); + } + Ok(parse_discovery(&text)) +} + +fn probe_config( + state_dir: &Path, + scratch: &Path, + url: &str, + username: &str, + secret_command: Option<&str>, +) -> String { + let password = match secret_command { + Some(command) => format!("\tpassword {{\n\t\tshell {command}\n\t}}\n"), + None => String::new(), + }; + format!( + "# Generated by `calcalist discover`. Rewritten on every run.\n\ + status_path {}\n\n\ + storage local {{\n\ttype vdir/icalendar\n\tpath {}\n\tfileext ics\n}}\n\n\ + storage {PROBE_REMOTE} {{\n\ttype caldav\n\turl {}\n\tdiscovery collections\n\ + \tusername {}\n{password}}}\n\n\ + pair probe {{\n\tstorage_a {PROBE_REMOTE}\n\tstorage_b local\n\ + \tcollections from a\n}}\n", + quote(&state_dir.join("discover-status").display().to_string()), + quote(&scratch.display().to_string()), + quote(url), + quote(username), + ) +} + +/// Reads the collections out of `pimsync discover` output. +/// +/// The format is not documented, so this was established against a real server +/// and is deliberately forgiving. The one thing it must get right: a pair has +/// two storages and pimsync reports both, so `Found` lines are only taken after +/// the heading naming the server. Otherwise the scratch directory's own +/// contents would be offered as if they were the user's calendars. +fn parse_discovery(output: &str) -> Vec { + let mut collections = Vec::new(); + let mut in_remote = false; + for line in output.lines() { + let line = strip_ansi(line); + let line = line.trim(); + if line.starts_with("==>") { + in_remote = line.contains(PROBE_REMOTE); + continue; + } + let Some(href) = line.strip_prefix("Found ") else { + continue; + }; + let href = href.trim(); + if !in_remote || href.is_empty() { + continue; + } + collections.push(Collection { + id: collection_id(href), + href: href.to_string(), + }); + } + collections +} + +/// A plausible endpoint id: the last non-empty path segment, restricted to +/// characters that are unambiguous in a configuration file. +fn collection_id(href: &str) -> String { + let segment = href + .rsplit('/') + .find(|segment| !segment.is_empty()) + .unwrap_or(href); + let id: String = segment + .chars() + .map(|ch| match ch { + 'a'..='z' | '0'..='9' | '-' | '_' => ch, + 'A'..='Z' => ch.to_ascii_lowercase(), + _ => '-', + }) + .collect(); + let id = id.trim_matches('-').to_string(); + if id.is_empty() { "calendar".into() } else { id } +} + +/// Removes ANSI colour, which pimsync emits even when its output is a pipe and +/// even under NO_COLOR. +fn strip_ansi(text: &str) -> String { + let mut out = String::with_capacity(text.len()); + let mut chars = text.chars(); + while let Some(ch) = chars.next() { + if ch != '\u{1b}' { + out.push(ch); + continue; + } + // Skip up to and including the terminating letter of the escape. + for next in chars.by_ref() { + if next.is_ascii_alphabetic() { + break; + } + } + } + out +} + /// Runs one pimsync cycle. /// /// Deliberately one-shot: `pimsync daemon` would write the same vdirs the @@ -518,4 +684,93 @@ url_command = "secret-tool lookup service calcalist account gcal-ics" assert!(supported.is_supported()); assert!(!next_series.is_supported()); } + + /// Established against a real server: everything arrives on stdout, the + /// headings are colour-coded even through a pipe, and both storages of the + /// pair are reported. + const DISCOVERY: &str = "\u{1b}[1m\u{1b}[32m==>\u{1b}[0m Discovering collections in local\u{2026}\n\ + Found leftover-one\n\ + Found leftover-two\n\ + \u{1b}[1m\u{1b}[32m==>\u{1b}[0m Discovering collections in probe_remote\u{2026}\n\ + Found /calendars/me/work/\n\ + Found /calendars/me/Private Stuff/\n"; + + /// The trap: pimsync reports the scratch directory's contents too, and + /// offering those as the user's calendars would be nonsense. + #[test] + fn only_the_servers_collections_are_taken() { + let found = parse_discovery(DISCOVERY); + + assert_eq!( + found.iter().map(|c| c.href.as_str()).collect::>(), + vec!["/calendars/me/work/", "/calendars/me/Private Stuff/"] + ); + } + + #[test] + fn an_id_is_taken_from_the_last_segment() { + let found = parse_discovery(DISCOVERY); + assert_eq!(found[0].id, "work"); + // Spaces and capitals cannot appear in an id used as a routing marker. + assert_eq!(found[1].id, "private-stuff"); + } + + #[test] + fn a_server_offering_nothing_yields_nothing() { + let empty = "\u{1b}[1m==>\u{1b}[0m Discovering collections in probe_remote\u{2026}\n"; + assert!(parse_discovery(empty).is_empty()); + } + + /// A failure is reported in place of the collections, so nothing is found. + #[test] + fn an_unreachable_server_yields_nothing() { + let failed = "\u{1b}[1m==>\u{1b}[0m Discovering collections in probe_remote\u{2026}\n\ + input/output error: client error executing request: client error (Connect)\n"; + assert!(parse_discovery(failed).is_empty()); + } + + #[test] + fn colour_is_stripped_from_headings() { + assert_eq!( + strip_ansi("\u{1b}[1m\u{1b}[32m==>\u{1b}[0m Discovering"), + "==> Discovering" + ); + } + + /// The probe must never be mistaken for the real configuration, and must + /// name the storage the parser anchors on. + #[test] + fn the_probe_config_is_separate_and_parseable() { + let text = probe_config( + Path::new("/var/state/calcalist"), + Path::new("/var/state/calcalist/discover-vdir"), + "https://dav.example.com/calendars/me/", + "me@example.com", + Some("secret-tool lookup service dav"), + ); + assert!( + text.contains(&format!("storage {PROBE_REMOTE} {{")), + "{text}" + ); + assert!(text.contains("collections from a"), "{text}"); + assert!(text.contains("discovery collections"), "{text}"); + assert!( + text.contains("shell secret-tool lookup service dav"), + "{text}" + ); + // Its status path is its own, so a probe cannot disturb a real sync. + assert!(text.contains("discover-status"), "{text}"); + } + + #[test] + fn a_server_needing_no_password_gets_no_password_directive() { + let text = probe_config( + Path::new("/var/state/calcalist"), + Path::new("/tmp/scratch"), + "https://dav.example.com/calendars/me/", + "me", + None, + ); + assert!(!text.contains("password"), "{text}"); + } } diff --git a/systemd/calcalist.service b/systemd/calcalist.service new file mode 100644 index 0000000..caeadf5 --- /dev/null +++ b/systemd/calcalist.service @@ -0,0 +1,15 @@ +[Unit] +Description=Synchronise calendars with calcalist + +[Service] +Type=oneshot +ExecStart=%h/.local/bin/calcalist sync +NoNewPrivileges=true + +# Deliberately no ProtectSystem, PrivateTmp or IPC sandboxing. Credentials come +# from commands like `secret-tool`, which need the session keyring over D-Bus, +# and those options break them in ways that surface as unexplained auth +# failures rather than as anything pointing at the sandbox. + +# A cycle that could not reach an endpoint exits non-zero on purpose, so a +# lapsed token shows up as a failed unit rather than passing unnoticed. diff --git a/systemd/calcalist.timer b/systemd/calcalist.timer new file mode 100644 index 0000000..d88f322 --- /dev/null +++ b/systemd/calcalist.timer @@ -0,0 +1,15 @@ +[Unit] +Description=Synchronise calendars with calcalist every 15 minutes + +[Timer] +OnBootSec=2m +OnUnitActiveSec=15m +Persistent=true + +# Spreads requests instead of every installation calling Google on the quarter +# hour. Google's Calendar API quota is per project and enforced per minute, and +# its own guidance is to randomise timing rather than to burst. +RandomizedDelaySec=120 + +[Install] +WantedBy=timers.target diff --git a/tests/caldav.rs b/tests/caldav.rs index 88eac3c..d770839 100644 --- a/tests/caldav.rs +++ b/tests/caldav.rs @@ -383,3 +383,117 @@ fn the_local_marker_keeps_an_event_out_of_a_configured_sink() { ); } } + +/// Discovery, against a server that really has two calendars. +/// +/// The command exists because the CalDAV `url` must name the collection exactly +/// and providers rarely show it. What it prints therefore has to be usable +/// as-is, which is what the parse-back below actually checks. +#[test] +fn discover_lists_the_servers_calendars_as_pasteable_config() { + let Some(fixture) = fixture() else { return }; + + let run = fixture.calcalist.run(&[ + "discover", + &format!( + "http://127.0.0.1:{}/{}/", + fixture.server.port, + support::USER + ), + "--username", + support::USER, + "--secret-command", + "printf password", + ]); + assert!( + run.succeeded(), + "discover failed\n{}\n{}", + run.stdout, + run.stderr + ); + + assert!(run.stdout.contains("id = \"work\""), "{}", run.stdout); + assert!(run.stdout.contains("id = \"unified\""), "{}", run.stdout); + // The URL has to come back absolute, not as the bare path pimsync reports. + assert!( + run.stdout + .contains(&format!("http://127.0.0.1:{}/", fixture.server.port)), + "{}", + run.stdout + ); + + // The point of the command: what it prints is valid configuration. + let config = format!("version = 1\n{}", run.stdout); + let parsed: toml::Value = toml::from_str(&config).expect("the output should be valid TOML"); + let endpoints = parsed["endpoint"].as_array().expect("endpoints"); + assert_eq!(endpoints.len(), 2, "{}", run.stdout); + assert_eq!(endpoints[0]["type"].as_str(), Some("caldav")); + assert_eq!( + endpoints[0]["secret_command"].as_str(), + Some("printf password") + ); +} + +/// An unreachable server has to say so rather than print an empty result that +/// looks like "this server has no calendars". +#[test] +fn discover_reports_a_server_it_cannot_reach() { + if !support::missing_binaries().is_empty() { + eprintln!("skipped: run under devbox"); + return; + } + let root = tempfile::tempdir().expect("temp"); + let calcalist = Calcalist::new(root.path(), "version = 1\n"); + + // Port 9 is discard: it refuses or blackholes, and nothing listens for DAV. + let run = calcalist.run(&[ + "discover", + "http://127.0.0.1:9/calendars/me/", + "--username", + "me", + ]); + + assert!( + !run.succeeded(), + "it should not claim success\n{}", + run.stdout + ); + assert!( + !run.stderr.is_empty(), + "the failure should be explained: {run:?}", + run = run.stderr + ); +} + +/// The lock is what stops a timer firing into a hand-run cycle. +/// +/// The lock is taken here rather than by racing two real syncs, which would be +/// timing-dependent and would pass by accident most of the time. +#[test] +fn a_second_cycle_is_refused_while_one_is_running() { + let Some(fixture) = fixture() else { return }; + assert!(fixture.calcalist.run(&["sync"]).succeeded()); + + let lock_path = fixture.calcalist.state.join("calcalist/lock"); + let held = std::fs::OpenOptions::new() + .read(true) + .write(true) + .open(&lock_path) + .expect("the lock file should exist after a cycle"); + rustix::fs::flock(&held, rustix::fs::FlockOperation::NonBlockingLockExclusive) + .expect("the test should be able to take the lock"); + + let refused = fixture.calcalist.run(&["sync"]); + assert!(!refused.succeeded(), "{}", refused.stdout); + assert!( + refused.stderr.contains("already running"), + "the refusal should say why: {}", + refused.stderr + ); + + drop(held); + assert!( + fixture.calcalist.run(&["sync"]).succeeded(), + "the lock should be free again" + ); +} diff --git a/tests/support/mod.rs b/tests/support/mod.rs index 605818c..90bf8fc 100644 --- a/tests/support/mod.rs +++ b/tests/support/mod.rs @@ -14,6 +14,9 @@ use std::time::{Duration, Instant}; /// How long to wait for a server to start answering. const STARTUP_TIMEOUT: Duration = Duration::from_secs(20); +/// How many ports to try before giving up on starting a server. +const ATTEMPTS: usize = 8; + /// The account Radicale files everything under. With authentication disabled it /// accepts whatever name arrives, and `owner_only` rights then grant that name /// its own tree — so the URLs below all live beneath it. @@ -46,29 +49,40 @@ impl Radicale { pub fn start(root: &Path) -> Radicale { let storage = root.join("radicale"); std::fs::create_dir_all(&storage).expect("create storage"); - let port = free_port(); - let process = Command::new("radicale") - .arg("--config") - .arg("") - .arg("--server-hosts") - .arg(format!("127.0.0.1:{port}")) - .arg("--auth-type") - .arg("none") - .arg("--storage-filesystem-folder") - .arg(&storage) - .stdout(Stdio::null()) - .stderr(Stdio::null()) - .spawn() - .expect("radicale should start"); + // A free port is chosen by binding one and letting go, so two tests + // running at once can pick the same number. The loser's radicale then + // fails to bind and exits — and, worse, waiting for the port to answer + // succeeds anyway, because the winner is listening on it. Tests would + // quietly share a server. So: confirm the child we spawned is the one + // still alive, and take a different port if it is not. + for _ in 0..ATTEMPTS { + let port = free_port(); + let mut process = Command::new("radicale") + .arg("--config") + .arg("") + .arg("--server-hosts") + .arg(format!("127.0.0.1:{port}")) + .arg("--auth-type") + .arg("none") + .arg("--storage-filesystem-folder") + .arg(&storage) + .stdout(Stdio::null()) + .stderr(Stdio::null()) + .spawn() + .expect("radicale should start"); - let server = Radicale { - process, - port, - storage, - }; - wait_until_listening(port, "radicale"); - server + if listening(&mut process, port) { + return Radicale { + process, + port, + storage, + }; + } + let _ = process.kill(); + let _ = process.wait(); + } + panic!("radicale did not come up on a port of its own after {ATTEMPTS} attempts"); } pub fn url(&self, calendar: &str) -> String { @@ -138,8 +152,34 @@ impl Radicale { } fn request(&self, method: &str, path: &str, content_type: &str, body: &str) -> String { + // Retried because a server under parallel test load occasionally resets + // a connection it has accepted; that says nothing about the request. + let mut last = String::new(); + for attempt in 0..ATTEMPTS { + match self.try_request(method, path, content_type, body) { + Ok(response) => return response, + Err(error) => { + last = error; + std::thread::sleep(Duration::from_millis(50 * (attempt as u64 + 1))); + } + } + } + panic!("{method} {path} never got a response: {last}"); + } + + fn try_request( + &self, + method: &str, + path: &str, + content_type: &str, + body: &str, + ) -> Result { let address = SocketAddr::from((Ipv4Addr::LOCALHOST, self.port)); - let mut stream = TcpStream::connect(address).expect("connect to radicale"); + let mut stream = TcpStream::connect_timeout(&address, Duration::from_secs(2)) + .map_err(|error| error.to_string())?; + stream + .set_read_timeout(Some(Duration::from_secs(10))) + .map_err(|error| error.to_string())?; // With authentication disabled the credentials only pick the account. let authorization = base64(format!("{USER}:password").as_bytes()); let request = format!( @@ -149,12 +189,14 @@ impl Radicale { self.port, body.len() ); - stream.write_all(request.as_bytes()).expect("send request"); + stream + .write_all(request.as_bytes()) + .map_err(|error| error.to_string())?; let mut response = String::new(); stream .read_to_string(&mut response) - .expect("read the response"); - response + .map_err(|error| error.to_string())?; + Ok(response) } } @@ -279,16 +321,42 @@ fn free_port() -> u16 { listener.local_addr().expect("port").port() } -fn wait_until_listening(port: u16, what: &str) { +/// Whether a real HTTP request gets a real HTTP response. +fn answers(address: SocketAddr) -> bool { + let Ok(mut stream) = TcpStream::connect_timeout(&address, Duration::from_millis(200)) else { + return false; + }; + let _ = stream.set_read_timeout(Some(Duration::from_secs(2))); + let request = format!("OPTIONS / HTTP/1.1\r\nHost: {address}\r\nConnection: close\r\n\r\n"); + if stream.write_all(request.as_bytes()).is_err() { + return false; + } + let mut response = String::new(); + stream.read_to_string(&mut response).is_ok() && response.starts_with("HTTP/") +} + +/// Waits for our own radicale to answer, giving up as soon as it has exited. +/// +/// The liveness check is the point: without it a process that lost a port race +/// looks healthy, because something else is answering on that port. +fn listening(process: &mut Child, port: u16) -> bool { let address = SocketAddr::from((Ipv4Addr::LOCALHOST, port)); let deadline = Instant::now() + STARTUP_TIMEOUT; while Instant::now() < deadline { - if TcpStream::connect_timeout(&address, Duration::from_millis(200)).is_ok() { - return; + match process.try_wait() { + Ok(Some(_)) => return false, + Ok(None) => {} + Err(_) => return false, } - std::thread::sleep(Duration::from_millis(100)); + // An open port is not readiness: the socket listens before the + // application behind it can answer, and under parallel tests that gap + // is wide enough to get a connection reset instead of a response. + if answers(address) { + return true; + } + std::thread::sleep(Duration::from_millis(50)); } - panic!("{what} did not start listening on {port} within {STARTUP_TIMEOUT:?}"); + false } fn base64(input: &[u8]) -> String { From 4854844a72a9d9093fd125fb2dc7b26f27841d17 Mon Sep 17 00:00:00 2001 From: randogoth Date: Thu, 10 Sep 2026 16:38:53 +0300 Subject: [PATCH 08/10] Cut the README to a getting-started guide; retire SPECS.md and TODO.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The README had grown to 413 lines and explained as much as it instructed: someone trying to get calcalist running read past the architecture, the scheduling-inertness rule and the reasoning behind retarget before reaching anything they needed to type. Most of the excess was duplication of SPECS.md, which is now retired along with TODO.md since the roadmap is finished. That inverted the usual answer — the rationale could not simply move to the design document, because there would not be one. Checking what would actually be orphaned: the architecture rationale is already in the module doc comments, beside the code it governs and where it cannot drift; the M0-M2 record is history and git keeps it. Two things had no home: the working agreement that has shaped every commit, and two decisions that cost real design work to decline and would otherwise be re-proposed from first principles. Both are now in CLAUDE.md, which is read at the start of every session rather than filed and forgotten. What the README keeps is the whole setup path, because none of it exists anywhere else now: every configuration field, the Google OAuth walkthrough in full, routing, and the systemd units. What it gains is a short section on what the tool does to your calendars — deletion propagating to the source is not something to discover by accident. What it loses is the design commentary. Also moves `discover` out of the command list and into the CalDAV endpoint section, since getting the url field right is its entire purpose. Verified rather than eyeballed: the worked example was extracted and run through doctor and status, the command table diffed against --help, and every internal anchor resolved. Co-Authored-By: Claude Opus 5 --- CLAUDE.md | 74 +++++++++++++ README.md | 324 +++++++++++++++++++----------------------------------- SPECS.md | 62 ----------- TODO.md | 175 ----------------------------- 4 files changed, 187 insertions(+), 448 deletions(-) create mode 100644 CLAUDE.md delete mode 100644 SPECS.md delete mode 100644 TODO.md diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..8a25e60 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,74 @@ +# calcalist + +Aggregates events between CalDAV, Google Calendar and iCal feeds. `README.md` is +the user-facing guide; the design rationale lives in the module doc comments, +beside the code it governs. + +These conventions were `SPECS.md`'s, which has been retired now the roadmap is +finished. Git history holds it, and `TODO.md`, if the record is ever wanted. + +## Development + +Use devbox for dependency management and scripts. Use jj for version control +(colocated with git). + +- `devbox run fmt` — format. +- `devbox run check` — format check, clippy with warnings denied, and tests. + This is the gate; jj has no commit step to hang a hook off, since the working + copy is itself a commit. +- `devbox run test` — tests only. + +Keep `/target` in `.gitignore`: jj snapshots the working copy on every command, +with no staging step. + +The integration tests in `tests/` need `radicale` and `pimsync`, both of which +devbox provides. Outside that shell they report what is missing and pass. + +## Coding style + +- Simple, readable, idiomatic. +- Explicit types for public APIs and important fields. +- Prefer immutable data structures. +- No global mutable state. +- No speculative features or dependencies. +- Keep functions short and single-purpose; extract helpers to avoid nesting. +- File and module naming: `snake_case.rs`. Types and traits: `PascalCase`. + Functions, variables and fields: `snake_case`. Constants: + `SCREAMING_SNAKE_CASE`. +- Run `devbox run check` before describing a change. +- Comment only when intent is not obvious from the code. +- Prefer composition and traits over deep type hierarchies. + +## Decisions already taken + +Settled after real design work. Reopen only with new information, not from +first principles. + +**No configuration web interface.** Designed in full, then dropped. Its audience +would be people who find TOML hard — but every user must first create a Google +Cloud project, configure a consent screen and put a secret in a keyring, which +is a far higher bar than editing the config. Anyone who clears it can edit the +file; anyone who cannot never reaches the file. `doctor`, `status` and +`discover` cover what the interface was actually for. + +**Bring-your-own OAuth client, not a shipped verified one.** Verification for +the Calendar scope needs a domain you own — a `github.io` address is rejected — +and Calendar API quota is per Cloud project, with charges announced for +exceeding it. A shipped client would pool every user's syncing into the +publisher's project, and their bill. Shipping an *unverified* shared client is +worse still: the 100-new-user cap is permanent for the project's lifetime. + +**pimsync cannot reach Google, and never will.** It has no REST storage, and its +config exposes only HTTP Basic auth, which Google's CalDAV endpoint rejected in +March 2025. The Google leg is calcalist's own in every design. + +**The config file is never machine-rewritten.** It is the portable, +hand-editable artefact. Anything that writes it must preserve comments, +ordering and omitted defaults — `toml::to_string` on the `Serialize` derive +does none of those things. + +## Known gap + +A failing `pimsync sync` stops the whole CalDAV leg. Unlike the Google side this +cannot be narrowed: pimsync is one process covering every pair, and a failure +does not say which pair it belongs to. diff --git a/README.md b/README.md index 5e191c8..6947530 100644 --- a/README.md +++ b/README.md @@ -9,20 +9,7 @@ back to the calendar it came from, so the aggregate is somewhere to work rather than a read-only noticeboard. CalDAV and Google are read/write in both directions; iCal feeds are read-only, because the protocol is. -This is pre-1.0. The command line and the configuration format are settled; -there is no web interface yet. `SPECS.md` covers why it is built this way. - -## How it works - -Every endpoint — the sources and the aggregate's target alike — is mirrored to a -directory of `.ics` files on your machine, and the aggregation engine works -purely on those files. One cycle is: pull every remote into its local mirror, -reconcile the mirrors against each other, push the results back out. - -`pimsync` carries the CalDAV and iCal legs. calcalist carries the Google leg -itself, over the Calendar REST API — pimsync cannot reach Google at all, having -no REST storage and only HTTP Basic auth, which Google's CalDAV endpoint has -rejected since March 2025. +Pre-1.0. The command line and the configuration format are settled. ## Install @@ -31,31 +18,24 @@ cargo build --release # target/release/calcalist ``` -One runtime dependency: **`pimsync` 0.5.x on `PATH`**, needed only if any -endpoint is CalDAV or an iCal feed. A Google-only setup needs nothing else. -pimsync is in nixpkgs; this repo's `devbox.json` pins 0.5.11 if you would rather -not install it yourself. +One runtime dependency: **`pimsync` 0.5.x on `PATH`**, which carries the CalDAV +and iCal legs. It is needed only if some endpoint is one of those — a +Google-only setup needs nothing else, because calcalist talks to Google itself. +pimsync is in nixpkgs, and this repo's `devbox.json` pins 0.5.11. -Then check the environment: +Then check the environment, which is worth doing before writing any config: ```sh calcalist doctor ``` -It reports whether pimsync is present and in the supported version series, -whether the state directory is writable, whether your configuration is valid, -whether pimsync accepts the configuration calcalist generates for it, and -whether each Google endpoint still has a usable authorisation. - -## Configuration +## Configure One TOML file at `$XDG_CONFIG_HOME/calcalist/calcalist.toml`, or wherever -`--config` points. - -It holds **no secrets and no sync state**, so it is safe to copy between -machines. Credentials come from `*_command` fields instead: each is run through -`sh -c` and the first line of its output is used. Point them at `secret-tool`, -`pass`, `gpg -d`, or anything else that prints the secret. +`--config` points. It holds **no secrets and no sync state**, so it is safe to +copy between machines. Credentials come from `*_command` fields: each is run +through `sh -c` and the first line of its output is used. Point them at +`secret-tool`, `pass`, `gpg -d`, or anything else that prints the secret. ```toml version = 1 @@ -76,9 +56,15 @@ username = "you@posteo.de" secret_command = "secret-tool lookup service posteo user you@posteo.de" ``` -`url` must name the calendar collection itself, not the server root or your -principal — the last path segment is the calendar. A URL with no collection in -it is rejected, and `calcalist doctor` says so. +`url` must name the calendar collection itself, not the server root. Rather than +digging it out of a provider's web interface, ask the server — this prints a +ready-to-paste block per calendar, with ids taken from the URL for you to +rename: + +```sh +calcalist discover https://posteo.de:8443/calendars/you/ \ + --username you@posteo.de --secret-command "secret-tool lookup service posteo" +``` **Google** @@ -93,8 +79,8 @@ client_secret_command = "secret-tool lookup service google-calcalist" ``` `calendar_id` is `primary`, or the calendar's own address — the "Calendar ID" -under the calendar's settings in Google Calendar. See -[Authorising Google](#authorising-google) below for `client_id` and the secret. +under its settings in Google Calendar. See [Authorising +Google](#authorising-google) for `client_id` and the secret. **iCal feed** @@ -105,14 +91,9 @@ type = "webcal" url = "https://example.org/holidays.ics" ``` -Always read-only. Give exactly one of `url` or `url_command` — the latter for a -feed whose address is itself a credential, Google's "secret address in iCal -format" being the case in point, since anyone holding it can read the whole -calendar: - -```toml -url_command = "secret-tool lookup service google-secret-ics" -``` +Always read-only. Give exactly one of `url` or `url_command`; the latter is for +a feed whose address is itself a credential, such as Google's "secret address in +iCal format", which grants read access to anyone holding it. ### Aggregates @@ -132,11 +113,6 @@ propagate_deletes = true # deleting a mirror deletes the original max_delete_fraction = 0.2 # abort if more of the aggregate than this vanishes ``` -`max_delete_fraction` guards against a cycle that would remove a large part of -your calendars — a mistyped URL, a server answering with an empty collection. It -has an absolute floor of three deletions, so removing one or two events is never -refused. `calcalist sync --force` overrides it. - The rules, all checked before anything is synchronised, with every problem reported in one pass: @@ -144,8 +120,8 @@ reported in one pass: - `target` must not also be one of its own `sources`. - `sources` must be non-empty, and each must exist and be named once. - `default_sink`, if given, must be one of the `sources` and must be writable. - Leaving it out is a deliberate mode, not an omission — see - [Events of the aggregate's own](#events-of-the-aggregates-own). + Leaving it out is a deliberate mode — see [Getting events to the right + calendar](#getting-events-to-the-right-calendar). - No endpoint may be called `local`, which is reserved for the routing marker. ### A complete example @@ -218,25 +194,21 @@ setup, so in full: **One login covers every endpoint on that account.** The authorisation is filed under the account it was granted for, not the endpoint that asked, so a second Google calendar on the same account needs no login of its own. Set `account` on -an endpoint only when calcalist is logged in to more than one account and has to -be told which is meant. +an endpoint only when calcalist is logged in to more than one account. **While the consent screen is in Testing, Google expires refresh tokens after seven days.** An installation that worked last week then stops with nothing -having changed locally; `calcalist google login` again, or publish the consent -screen. `calcalist doctor` reports this rather than letting it surface as an -opaque failure. +having changed locally. Publish the consent screen to production to end this; +`calcalist doctor` reports it either way. -## Routing: getting a new event to the right calendar +## Getting events to the right calendar -Every mirrored event carries where it came from — `X-CALCALIST-SOURCE` and -`X-CALCALIST-ORIGIN-UID` — and a UID derived from its origin, so an edit made in -the aggregate goes back to the calendar that owns the event without any -guesswork. +An event mirrored from a source goes back to that source when you edit it — +calcalist tracks where each one came from. An event you *create* in the +aggregate belongs to nothing yet, and goes to the aggregate's `default_sink`. -An event you *create* in the aggregate belongs to nothing yet. It goes to the -aggregate's `default_sink`. To send it somewhere else, put the endpoint's id -after an `@` **on a line of its own** in the event's notes: +To send one somewhere else, put the endpoint's id after an `@` **on a line of +its own** in the event's notes: ``` Dentist, bring the referral @@ -244,140 +216,47 @@ Dentist, bring the referral @gcal ``` -The marker is removed before the event reaches the calendar. A whole line is -required so that an address or a handle written in prose is not mistaken for an -instruction. +The marker is removed before the event arrives. `calcalist status` prints the +markers that would actually work, so you need not remember what is in the +config; a marker naming anything else is refused and reported rather than +quietly filed under the default. -`calcalist status` prints the markers that would actually work, so you do not -have to remember what is in the configuration file: +**To keep an event in the aggregate itself**, tag it `@local` — it then stays +put even though a `default_sink` would have taken it, and the marker is +deliberately left in place so later cycles decide the same way. Leaving +`default_sink` out of the aggregate entirely makes that the rule for every +untagged event, so routing becomes opt-in. Such events live only in that +calendar and cannot be rebuilt from a source, but they are ordinary events on +that server, visible to every client subscribed to it. -``` -aggregate `unified` - published to posteo - sources gcal, holidays - mirroring 42 event(s) - new events go to `gcal` unless told otherwise - to choose, put one of these on a line of its own in the event's notes: - @gcal - @local (keep the event here, in this calendar only) +A category matching an endpoint id works too. Note that *any* category on a new +event is read as a routing instruction, so the notes marker is the safer form. + +## What it does to your calendars + +- **Deleting an event in the aggregate deletes it at its source**, unless + `propagate_deletes = false`. A cycle that would remove more than + `max_delete_fraction` of the aggregate is refused until you pass `--force` — + with a floor of three, so removing one or two events is never blocked. +- **Edited in both places since the last sync, the source wins.** The conflict + is reported. +- **Alarms are always kept**, with no option to strip them. The tool exists so + you subscribe to one calendar rather than several, so dropping reminders would + disarm every one you have. +- **Mirroring never mails your guests.** On a Google target attendees are kept + and Google is told not to notify; on a CalDAV target, which cannot be told + that, the guest list is carried as inert data instead. Routing an event *you* + created is the exception: that server invites its guests for real, which is + the point of creating it. + +## Run it + +```sh +calcalist doctor # environment and configuration +calcalist sync # one cycle ``` -Read-only feeds are left out, since they cannot take an event. A marker naming -anything else is refused and reported — the event stays where it is rather than -being quietly filed under the default, which would put it somewhere you did not -ask for. - -### Events of the aggregate's own - -An aggregate is also a calendar, and it can hold events that belong to nothing -else. There are two ways to get one: - -- **`@local`** on an event, exactly like any other marker, keeps it where it was - written even though a `default_sink` would otherwise have taken it. Unlike - every other marker it is *not* removed from the event: the others have done - their job once the event reaches its source, whereas this one has to stay - legible so every later cycle reaches the same decision. -- **Leave `default_sink` out of the aggregate entirely.** Then nothing is - configured to file a new event under, so every untagged event stays put and - only `@`-tagged ones are sent anywhere. Routing becomes opt-in rather than - opt-out. - -`local` is a reserved endpoint id for this reason, and configuring an endpoint -with that name is refused. - -One thing to be aware of: every other event in an aggregate is derived from a -source, so the aggregate is disposable — lose the calendar and it rebuilds -itself on the next sync. **An event of the aggregate's own exists in exactly one -place.** Nothing else holds a copy, so it is only as safe as that calendar is. -`aggregate retarget` carries these events to the new target along with -everything else, since there is nothing to re-derive them from. - -A **category** matching an endpoint id works too, that being the field -iCalendar intends for this. Be aware that any category on a newly created event -is read as a routing instruction, so an event carrying an unrelated category -will be refused rather than sent to the default sink. The notes marker is the -safer form, and the one every mobile client exposes. - -### Attendees and alarms - -Writing to an aggregate must never mail anyone: the guests were already invited -from the original calendar. On a Google target, attendees are kept verbatim and -Google is told not to notify. A CalDAV server offers no portable way to be told -that, so the guest list is instead carried as inert data — the live `ATTENDEE` -and `ORGANIZER` properties are dropped, the guests appear in -`X-CALCALIST-ATTENDEES` and in the description, and a meeting you declined is -marked free. - -Routing works the other way round: an event you created and sent to a source is -a meeting you are deliberately organising, so its attendees are preserved and -that server invites them for real. - -Alarms are always kept, with no option to strip them. The whole point of the -tool is that you subscribe to one calendar rather than several, so dropping -reminders would silently disarm every one you have. - -## Commands - -``` -calcalist sync [--dry-run] [--force] -calcalist status -calcalist doctor -calcalist prune [--force] -calcalist discover --username [--secret-command ] -calcalist google login -calcalist aggregate retarget --to [--keep-old | --purge-old] -``` - -- **`sync`** runs one cycle. `--dry-run` pulls from the servers for real, into a - throwaway copy of the local mirrors, so what it reports is measured against - your calendars as they are now — and nothing outside that copy is written. - `--force` overrides the mass-deletion guard. -- **`status`** lists endpoints and aggregates, how many events each aggregate is - mirroring, and the routing markers that would work. -- **`doctor`** checks the environment and configuration. Run it first. -- **`discover`** asks a CalDAV server which calendars it has and prints an - `[[endpoint]]` block for each, ready to paste. Give it the account or - principal URL rather than a single calendar. This is the easy way to get the - `url` field right. -- **`prune`** reports local mirrors belonging to endpoints your configuration no - longer names. It only lists them until you pass `--force`, since an endpoint - may just have been renamed. -- **`aggregate retarget`** moves an aggregate to a different target calendar - deliberately. `sync` refuses to run when an aggregate's `target` has changed - under it and points here instead — the new, empty target would otherwise read - as an aggregate whose every event had been deleted, and with delete - propagation on, that would remove them from every source calendar. Events left - on the old target are kept unless you pass `--purge-old`, which only removes - events calcalist put there. - -Exit codes: `0` all well, `1` something failed or an endpoint could not be -reached. - -If a Google endpoint cannot be reached — a lapsed token, no network — the cycle -does not stop. The endpoint is named, only the aggregates that depend on it -stand down, everything else is synchronised as usual, and the run exits `1` so -the failure cannot pass unnoticed. - -## Where things are kept - -Everything machine-local lives under `$XDG_STATE_HOME/calcalist`: - -``` -state.json event mappings, content hashes, sync cursors -pimsync.conf generated on every sync; edits are overwritten -pimsync-status/ pimsync's own record of what it has seen -vdir// the local mirror of each endpoint -google/accounts/.json refresh tokens, 0600 -``` - -None of this belongs in a backup or a dotfiles repository. The state file is a -cache rather than a single point of failure — aggregate UIDs are derived from -their origin, so a lost state file is rebuilt by re-deriving it. Copy the -configuration between machines; leave this behind. - -## Running it regularly - -Units ship in [systemd/](systemd/): +To run it regularly, units ship in [systemd/](systemd/): ```sh install -Dm644 systemd/calcalist.service ~/.config/systemd/user/calcalist.service @@ -387,27 +266,50 @@ systemctl --user enable --now calcalist.timer ``` The service expects the binary at `~/.local/bin/calcalist`; edit `ExecStart` if -yours is elsewhere. The timer runs every 15 minutes with a randomised delay of -up to two minutes, which spreads requests instead of every installation calling -Google on the quarter hour — its quota is enforced per minute, per project. +yours is elsewhere. Only one cycle runs at a time — a timer firing while you are +running `sync` by hand is refused, naming the process that holds the lock. A +cycle that could not reach an endpoint exits non-zero on purpose, so a lapsed +token shows up as a failed unit rather than passing unnoticed; +`journalctl --user -u calcalist` has the report. If your secrets come from a +keyring needing an unlocked session, the timer only works while you are logged +in. -A cycle that could not reach an endpoint exits non-zero deliberately, so a -lapsed token surfaces as a failed unit in `systemctl --user status calcalist` -rather than passing unnoticed. `journalctl --user -u calcalist` has the report. +## Commands -Only one cycle runs at a time: calcalist takes a lock on its state directory, so -a timer firing while you are running `calcalist sync` by hand is refused with a -message naming the process that holds it, rather than the two interleaving their -writes. If your secrets come from a keyring that needs an unlocked session, the -timer will only work while you are logged in. +`--help` has the flags. -## Not yet +| Command | | +|---|---| +| `sync` | One cycle. `--dry-run` reports against current remote state without writing; `--force` overrides the mass-deletion guard. | +| `status` | Endpoints, aggregates, how much is mirrored, and the routing markers that work. | +| `doctor` | Checks the environment and configuration. Run it first. | +| `discover` | Lists a CalDAV server's calendars as endpoint blocks to paste. | +| `google login` | Authorises an account. | +| `aggregate retarget` | Moves an aggregate to a different target calendar. `sync` refuses on its own when the target changes under it, and points here. | +| `prune` | Reports local mirrors of endpoints no longer configured; removes them under `--force`. | + +Exit codes: `0` all well, `1` something failed or an endpoint could not be +reached. + +## Where things are kept + +Everything machine-local lives under `$XDG_STATE_HOME/calcalist`: + +``` +state.json event mappings, content hashes, sync cursors +lock held for the duration of a cycle +pimsync.conf generated on every sync; edits are overwritten +pimsync-status/ pimsync's own record of what it has seen +vdir// the local mirror of each endpoint +google/accounts/.json refresh tokens, 0600 +``` + +None of it belongs in a backup or a dotfiles repository — it is a cache, and a +lost state file costs a re-materialisation rather than data. Copy the +configuration between machines; leave this behind. + +## Known gaps - **A failing `pimsync sync` stops the whole CalDAV leg.** Google endpoints are isolated from each other, but pimsync is a single process covering every CalDAV and feed pair, and a failure does not say which pair it belongs to. - -There is deliberately no web interface. It was designed and then dropped: the -Google OAuth setup, which no interface can remove, is a far higher bar than -editing this file, so a UI would have served an audience that never gets as far -as the config. `doctor`, `status` and `discover` cover what it was for. diff --git a/SPECS.md b/SPECS.md deleted file mode 100644 index a00697b..0000000 --- a/SPECS.md +++ /dev/null @@ -1,62 +0,0 @@ -# Calcalist - -## Idea - -Small Rust based commandline tool with utility systemd service file that can aggregate and sync events between CalDAV/Google Calendar/iCal. Users who have a Google Calendar can create a new calendar that syncs events from several CalDAV calendars. CalDAV users can sync several Google Calendars into one calendar. This should be doable both ways, so the aggregating calendar needs to be able to distinguish where its events came from and also have a way to create new events that get synced to the correct source. In addition iCal feeds can also be aggregated but they remain one way only. - -Configuration is saved in a portable toml file that can be easily migrated. - -The web interface this originally called for was designed and then dropped. Setting up Google's OAuth client — a Cloud project, a consent screen, a secret in a keyring — is a far higher bar than editing thirty lines of TOML, and no interface can remove it, so a configuration UI would have served an audience that never reaches the configuration. `calcalist doctor` reports every problem in one pass, `calcalist status` names the routing markers an event can carry, and `calcalist discover` prints ready-to-paste endpoint blocks for a server's calendars — which is what the interface was actually for. - -## Architecture - -Every endpoint — sources and aggregate targets alike — is mirrored to a local vdir, so the aggregation engine works purely on local files. - -- `pimsync` is driven as a one-shot subprocess for the CalDAV and WebCal legs. Its daemon mode is unusable here: it would race the reconciler over the same vdir files, so calcalist owns scheduling. -- The Google leg is calcalist's own, via the Calendar REST API. pimsync cannot reach Google — it has no REST storage, and its config exposes only HTTP Basic auth, which Google's CalDAV endpoint has rejected since 2025-03-14. -- Sync state (event mappings, hashes, sync tokens) lives in a JSON sidecar under `$XDG_STATE_HOME/calcalist/`, never in the portable config. -- Secrets are never stored in the config either. Credentials come from `*_command` fields that are executed to fetch them. -- A Google authorisation is filed under the account it was granted for, not the endpoint that asked for it, so every endpoint on that account shares it. An endpoint names its `account` only when calcalist is logged in to more than one. - -## Sync semantics - -| Question | Behaviour | -|---|---| -| Provenance | Aggregate UIDs derived as `blake3(aggregate_id, source_id, source_uid)`; the state file is a cache, not a single point of failure | -| Routing new events | A `@endpoint-id` line in the description, or a matching category, picks the source; otherwise the aggregate's `default_sink`. A hint naming an invalid sink is refused, never redirected to the default | -| Conflicts | Source wins — the origin calendar is authoritative | -| Deletion | Propagates to the source, guarded by a mass-deletion threshold | -| Attendees (mirroring) | Kept verbatim on a Google target; demoted to inert data on a CalDAV target | -| Attendees (routing) | Preserved — the sink server sends real invitations, which is intended | -| Alarms | Always preserved, with no option to strip them | -| Retargeting | `sync` refuses on target drift; `aggregate retarget` performs it deliberately | -| Recurrence | A series and its exceptions share one file. Google addresses an exception through the series, so the master is sent first and each override is matched to its instance by original start time and patched | -| Unreachable endpoints | Named in the report; only the aggregates depending on them stand down, and the run exits non-zero | -| Dry runs | Pull for real, into a throwaway copy of the local mirrors and through a read-only pimsync configuration. Nothing outside the copy is written | - -The governing rule behind the attendee handling: **writes to an aggregate must never emit scheduling mail; writes to a source schedule normally.** Google can be told not to notify, so attendees survive intact there. CalDAV offers no portable way to suppress RFC 6638 scheduling, so inertness is achieved structurally by dropping the live properties instead. - -Alarms are never stripped because the whole point of the tool is that a user subscribes to one calendar rather than several — so the duplicate-notification problem that stripping would guard against does not arise, while stripping would silently destroy every reminder in the one calendar the user actually watches. - -## Development - -Use devbox for dependency management and scripts. Use jj for version control (colocated with git). - -- `devbox run fmt` — format. -- `devbox run check` — format check, clippy with warnings denied, and tests. This is the gate; jj has no commit step to hang a hook off, since the working copy is itself a commit. -- `devbox run test` — tests only. - -Keep `/target` in `.gitignore`: jj snapshots the working copy on every command, with no staging step. - -## Coding style - -- Simple, readable, idiomatic. -- Explicit types for public APIs and important fields. -- Prefer immutable data structures. -- No global mutable state. -- No speculative features or dependencies. -- Keep functions short and single-purpose; extract helpers to avoid nesting. -- File and module naming: `snake_case.rs`. Types and traits: `PascalCase`. Functions, variables and fields: `snake_case`. Constants: `SCREAMING_SNAKE_CASE`. -- Run `devbox run check` before describing a change. -- Comment only when intent is not obvious from the code. -- Prefer composition and traits over deep type hierarchies. diff --git a/TODO.md b/TODO.md deleted file mode 100644 index b1240a5..0000000 --- a/TODO.md +++ /dev/null @@ -1,175 +0,0 @@ -# Roadmap - -Milestones from the implementation plan. See SPECS.md for the architecture and the -sync semantics these items implement. - -## M0 — skeleton (done) - -- [x] Initialise jj colocated with git; `.gitignore` written before the first build -- [x] `devbox.json` pinning Rust 1.97.1, pimsync 0.5.11, jujutsu 0.44.0, radicale 3.7.8 -- [x] Configuration model with referential validation, reporting every problem in one pass -- [x] `calcalist doctor` — pimsync presence and version series, state directory, configuration -- [x] Define the full CLI surface; unimplemented commands exit 2 rather than pretend -- [x] SPECS.md: Rust naming conventions, `devbox run check` gate, recorded sync semantics - -## M1 — bidirectional sync - -Complete. Every item from the plan is built, tested and verified live. - -### Core modules - -- [x] `state.rs` — JSON sidecar, atomic temp + fsync + rename; records each aggregate's - resolved target endpoint id **and** backend type, plus per-Google-endpoint cursors -- [x] `vdir.rs` — read and write vdir directories, keyed by the UID inside each file - rather than by filename, since pimsync names files its own way -- [x] `ical.rs` — surgical line-level `.ics` editing (UID rewrite, property injection), - respecting RFC 5545 folding; no parse-and-reserialize -- [x] `provenance.rs` — deterministic `blake3(aggregate_id, source_id, source_uid)` UIDs -- [x] `mirror.rs` — the to-aggregate and to-source transforms (added; not in the - original plan, which folded these into `reconcile`) -- [x] `reconcile.rs` — the aggregation engine; pure, no I/O -- [x] `sync.rs` — one cycle: pull, reconcile, push -- [x] `retarget.rs` — moving an aggregate to a different target, deliberately -- [x] `pimsync.rs` — generate `pimsync.conf` (with `on_empty skip` and `on_delete skip`), - drive one-shot `pimsync sync` bracketing the reconcile step -- [x] `google/auth.rs` — OAuth loopback flow with PKCE, refresh, keyring-sourced secrets -- [x] `google/convert.rs` — iCalendar both ways, including recurrence, timezones and alarms -- [x] `google/api.rs` — incremental pull by syncToken; push by import / update / delete - with notification suppressed - -### Behaviour - -- [x] **`events.import` gate** — settled from Google's own API discovery document: - `events.import` accepts no `sendUpdates` parameter at all, while `insert`, - `update` and `delete` all do, and it is documented as adding "a private copy of - an existing event". Confirmed live that attendees and alarms survive an import. - Creation goes through `import`; update and delete pass `sendUpdates=none`. -- [x] `sync` refuses to run on aggregate target drift, before reconciliation -- [x] `aggregate retarget` — flush unrouted creations against the old target, then - re-materialise; keep old orphans by default, purge bounded by the derivation -- [x] Mass-deletion guard (`max_delete_fraction`), overridable with `--force`, with an - absolute floor so deleting a couple of events is never refused -- [x] Echo suppression: derived UIDs are never re-ingested as source events -- [x] Routing: a `@endpoint-id` line in the description, or a matching category, picks - which source a new aggregate event is filed under; a hint naming an invalid sink - is refused rather than redirected to the default -- [x] Alarms map both ways, including Google's minutes-before reminder model -- [x] `webcal` URLs may come from a command, for feeds whose address is itself a - credential — Google's secret iCal address being the case in point -- [x] `doctor` validates the generated pimsync config and each Google authorisation -- [x] `calcalist status` — endpoints, aggregates, how many events are mirrored, and the - exact `@marker` names an event can carry, so routing is discoverable without - opening the config file -- [x] Recurrence overrides are pushed to Google. Google addresses an exception - through the series rather than as an event of its own, so the master goes up - first and each override is then matched to its instance by original start - time and patched. An override matching no occurrence is counted, not forced. -- [x] A `TZID` now travels with the `VTIMEZONE` that defines it, derived from the - zone's own transition table as the yearly rule it implies. A zone that does - not shift gets one fixed observance. -- [x] One Google authorisation covers every endpoint on that account. Tokens are - keyed by the account, discovered at login from the primary calendar's id — - which needs no scope beyond the calendar one already granted. An endpoint - names its `account` only when calcalist is logged in to more than one. -- [x] A Google endpoint that cannot be reached no longer ends the cycle. It is - named, only the aggregates depending on it stand down, and the run exits - non-zero so a lapsed token cannot pass for success. -- [x] `calcalist prune` — reports local mirrors of endpoints the configuration no - longer names, and removes them under `--force` -- [x] `--dry-run` pulls for real, into a throwaway copy of the local mirrors and - through a pimsync configuration that only ever reads from a server. What it - reports is measured against the calendars as they are now, and nothing - outside the copy is written. - -### Tests - -- [x] `reconcile` table-driven cases: create/update/delete each direction, - both-sides-changed, routing, echo suppression, mass-delete abort -- [x] `ical` round-trip fixtures: recurring with overrides, all-day, TZID, unknown `X-` props -- [x] `convert` against real captured API payloads, including the case where Google's - offset and its `timeZone` disagree -- [x] Topologies: a source feeding several aggregates, several aggregates sharing one - target, a cycle between two aggregates, a delete cascading across aggregates, - and competing edits arriving through two aggregates at once -- [x] Retarget: drift makes `sync` exit non-zero having written nothing and losing no - source event; an unrouted creation reaches a sink first -- [x] Safety (unit): no live `ATTENDEE`/`ORGANIZER` on a CalDAV-targeted mirror, - `VALARM` intact, `PARTSTAT: DECLINED` maps to `TRANSP: TRANSPARENT` - -### Verified live, end to end - -Against real accounts — two Google calendars aggregating into a Posteo CalDAV -calendar: - -- Fan-in from both Google sources, each mirror tagged with its origin -- A public iCal feed synced through pimsync and pushed to CalDAV; idempotent across - repeated runs and correctly incremental -- An edit made in the aggregate reaching the originating Google calendar, and the - revert travelling the same way -- An event created in the aggregate routed to a chosen source by its description - marker, with the marker stripped before it arrived -- A deletion in the aggregate propagating through to Google -- The mass-deletion guard refusing a 100% removal until `--force` -- `pimsync check` validating the generated config against a live CalDAV server - -### Integration - -- [x] Against a real Radicale server and a real iCal feed, over pimsync: two - sources converge on a CalDAV target, and the next cycle is a no-op. -- [x] Safety, against the same server: no live `ATTENDEE` or `ORGANIZER` reaches - the aggregate, the guest list survives as inert data, the alarm survives - intact, and the source keeps its scheduling properties. - - The plan asked for an SMTP sink here. Radicale implements no RFC 6638 - scheduling, so a quiet SMTP port would have proved nothing about the - transform — no server in reach of a test sends calendar mail at all. What - is asserted instead is the bytes that reached the server, which is the - thing the transform is actually responsible for. Proving the Google half - was settled separately, from the API discovery document. -- [x] `--dry-run` reaches the servers, reports what it found, and leaves both the - target and the local mirrors untouched. -- [x] A retired endpoint's mirror is reported by `prune` and removed under - `--force`. - -## M2 — packaging (done) - -- [x] Run lock (`lock.rs`) — `flock` on the state directory, held by `sync`, - `aggregate retarget` and `prune --force`. Not a UI feature: nothing - previously stopped a timer firing into a hand-run cycle, and two cycles - interleaving writes over the same vdirs is what the design otherwise - avoids. The kernel releases it however the process ends, so a crash - cannot leave a lock to clear by hand; the refusal names the holding pid. -- [x] systemd user units in `systemd/` — `calcalist.service` (oneshot) and - `calcalist.timer`, with `RandomizedDelaySec` so installations do not all - call Google on the quarter hour, and no filesystem or IPC sandboxing - because the secret commands need the session keyring over D-Bus. -- [x] `calcalist discover` — asks a CalDAV server which calendars it has and - prints a ready-to-paste `[[endpoint]]` block for each. The `url` field is - the most error-prone thing in the config, and providers rarely show it. -- [x] `serve` removed from the CLI, and with it the last `unimplemented` - command, so every command the binary advertises now does something. - -### The web interface, dropped - -Specified from the start and designed in full before being dropped. The -reasoning, so it is not rediscovered as an oversight: - -- Its audience would be people who find TOML hard. But with bring-your-own - OAuth client settled, every user must first create a Google Cloud project, - configure a consent screen and put a secret in a keyring — a far higher bar - than editing thirty lines of config. Anyone who clears it can edit the file; - anyone who cannot never reaches the file. -- The cost was three dependencies, five modules, an `auth.rs` refactor and a - security surface (token, `Host` validation, CSRF, secrets through subprocess - stdin) guarding something that reads the config and touches the keyring — - against SPECS.md's own "no speculative features or dependencies". -- `doctor`, `status` and `discover` had already absorbed what it was for. - -The one idea worth keeping from the design was collection discovery, which -needed no web layer at all. - -## Known gaps - -- [ ] A `pimsync sync` that fails takes the whole CalDAV leg with it. Unlike the - Google side this cannot be narrowed: pimsync is one process covering every - pair, so a failure does not say which pair it belongs to. From 22a1236f791e6d26a935977fb02d05868af95cd1 Mon Sep 17 00:00:00 2001 From: randogoth Date: Thu, 10 Sep 2026 16:42:31 +0300 Subject: [PATCH 09/10] Add the MIT license LICENSE with the standard text, `license = "MIT"` in Cargo.toml so the metadata matches, and a line in the README pointing at it. Copyright is attributed to `randogoth`, taken from the repository's author setting, and dated 2026. Co-Authored-By: Claude Opus 5 --- Cargo.toml | 1 + LICENSE | 21 +++++++++++++++++++++ README.md | 4 ++++ 3 files changed, 26 insertions(+) create mode 100644 LICENSE diff --git a/Cargo.toml b/Cargo.toml index 7f41e14..ef7a6d6 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -4,6 +4,7 @@ version = "0.1.0" edition = "2024" rust-version = "1.97" description = "Aggregate and sync events between CalDAV, Google Calendar and iCal feeds" +license = "MIT" publish = false [dependencies] diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..2e6cbfe --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 randogoth + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/README.md b/README.md index 6947530..b2fb169 100644 --- a/README.md +++ b/README.md @@ -313,3 +313,7 @@ configuration between machines; leave this behind. - **A failing `pimsync sync` stops the whole CalDAV leg.** Google endpoints are isolated from each other, but pimsync is a single process covering every CalDAV and feed pair, and a failure does not say which pair it belongs to. + +## License + +MIT. See [LICENSE](LICENSE). From 16f5d7351f53c9f7057d453f0f7679bb383fa97c Mon Sep 17 00:00:00 2001 From: randogoth Date: Thu, 10 Sep 2026 16:57:51 +0300 Subject: [PATCH 10/10] readme update --- README.md | 162 +++++++++++++++++------------------------------------- 1 file changed, 49 insertions(+), 113 deletions(-) diff --git a/README.md b/README.md index b2fb169..aba2467 100644 --- a/README.md +++ b/README.md @@ -1,15 +1,8 @@ -# calcalist +# CalCalist Aggregate several calendars into one, and keep them in step. -calcalist mirrors any number of CalDAV calendars, Google calendars and iCal -feeds into a single calendar you subscribe to. The point is to subscribe to one -calendar instead of several: an event created or edited in the aggregate travels -back to the calendar it came from, so the aggregate is somewhere to work rather -than a read-only noticeboard. CalDAV and Google are read/write in both -directions; iCal feeds are read-only, because the protocol is. - -Pre-1.0. The command line and the configuration format are settled. +CalCalist mirrors any number of CalDAV calendars, Google calendars and iCal feeds into a single calendar you subscribe to. The point is to subscribe to one calendar instead of several: an event created or edited in the aggregate travels back to the calendar it came from, so the aggregate is somewhere to work rather than a read-only noticeboard. CalDAV and Google are read/write in both directions; iCal feeds are read-only, because the protocol is. ## Install @@ -18,10 +11,7 @@ cargo build --release # target/release/calcalist ``` -One runtime dependency: **`pimsync` 0.5.x on `PATH`**, which carries the CalDAV -and iCal legs. It is needed only if some endpoint is one of those — a -Google-only setup needs nothing else, because calcalist talks to Google itself. -pimsync is in nixpkgs, and this repo's `devbox.json` pins 0.5.11. +One runtime dependency is needed for CalDAV and iCal sync: **`pimsync` 0.5.x on `PATH`**. Then check the environment, which is worth doing before writing any config: @@ -31,21 +21,18 @@ calcalist doctor ## Configure -One TOML file at `$XDG_CONFIG_HOME/calcalist/calcalist.toml`, or wherever -`--config` points. It holds **no secrets and no sync state**, so it is safe to -copy between machines. Credentials come from `*_command` fields: each is run -through `sh -c` and the first line of its output is used. Point them at -`secret-tool`, `pass`, `gpg -d`, or anything else that prints the secret. +One TOML file at `$XDG_CONFIG_HOME/calcalist/calcalist.toml`, or wherever `--config` points. It holds **no secrets and no sync state**, so it is safe to +copy between machines. + +Credentials come from `*_command` fields: each is run through `sh -c` and the first line of its output is used. Point them at `secret-tool`, `pass`, `gpg -d`, or anything else that prints the secret. ```toml version = 1 ``` -### Endpoints - Each `[[endpoint]]` is one calendar, with an `id` you refer to it by elsewhere. -**CalDAV** +### CalDAV Endpoint ```toml [[endpoint]] @@ -56,17 +43,16 @@ username = "you@posteo.de" secret_command = "secret-tool lookup service posteo user you@posteo.de" ``` -`url` must name the calendar collection itself, not the server root. Rather than -digging it out of a provider's web interface, ask the server — this prints a -ready-to-paste block per calendar, with ids taken from the URL for you to -rename: +`url` must name the calendar collection itself, not the server root. + +Rather than digging it out of a provider's web interface, ask the server — this prints a ready-to-paste block per calendar, with ids taken from the URL for you to rename: ```sh calcalist discover https://posteo.de:8443/calendars/you/ \ --username you@posteo.de --secret-command "secret-tool lookup service posteo" ``` -**Google** +## Google Calendar Endpoint ```toml [[endpoint]] @@ -78,11 +64,9 @@ client_secret_command = "secret-tool lookup service google-calcalist" # account = "you@gmail.com" # only when logged in to more than one account ``` -`calendar_id` is `primary`, or the calendar's own address — the "Calendar ID" -under its settings in Google Calendar. See [Authorising -Google](#authorising-google) for `client_id` and the secret. +`calendar_id` is `primary`, or the calendar's own address — the "Calendar ID" under its settings in Google Calendar. See [Authorising Google](#authorising-google) for `client_id` and the secret. -**iCal feed** +## iCal Feed Endpoint ```toml [[endpoint]] @@ -91,14 +75,11 @@ type = "webcal" url = "https://example.org/holidays.ics" ``` -Always read-only. Give exactly one of `url` or `url_command`; the latter is for -a feed whose address is itself a credential, such as Google's "secret address in -iCal format", which grants read access to anyone holding it. +Always read-only. Give exactly one of `url` or `url_command`; the latter is for a feed whose address is itself a credential, such as Google's "secret address in iCal format". -### Aggregates +## Aggregates -Each `[[aggregate]]` composes endpoints: several `sources` mirrored into one -`target`. +Each `[[aggregate]]` composes endpoints: several `sources` mirrored into one `target`. ```toml [[aggregate]] @@ -113,18 +94,15 @@ propagate_deletes = true # deleting a mirror deletes the original max_delete_fraction = 0.2 # abort if more of the aggregate than this vanishes ``` -The rules, all checked before anything is synchronised, with every problem -reported in one pass: +The rules, all checked before anything is synchronised, with every problem reported in one pass: - `target` must exist and be writable, so never an iCal feed. - `target` must not also be one of its own `sources`. - `sources` must be non-empty, and each must exist and be named once. -- `default_sink`, if given, must be one of the `sources` and must be writable. - Leaving it out is a deliberate mode — see [Getting events to the right - calendar](#getting-events-to-the-right-calendar). +- `default_sink`, if given, must be one of the `sources` and must be writable. Leaving it out is a deliberate mode — see [Getting events to the right calendar](#getting-events-to-the-right-calendar). - No endpoint may be called `local`, which is reserved for the routing marker. -### A complete example +## A complete example A Google calendar and a public holiday feed, aggregated into a CalDAV calendar: @@ -157,58 +135,40 @@ sources = ["gcal", "holidays"] default_sink = "gcal" ``` -## Authorising Google +# Authorising Google -Google requires an OAuth client of your own. This is the fiddliest part of the -setup, so in full: +Google requires an OAuth client of your own. This is the fiddliest part of the setup, so in full: 1. At [console.cloud.google.com](https://console.cloud.google.com), create a project — any name. 2. **APIs & Services → Library**, find **Google Calendar API**, enable it. -3. **OAuth consent screen**: choose **External**. Fill in the required fields. - Under **Test users**, add your own Google address. Without this, authorising - fails with `403: access_denied` even though the account is your own. -4. **Credentials → Create credentials → OAuth client ID**, and for - **Application type** choose **Desktop app**. - - Desktop app matters. calcalist listens on a loopback port it picks at run - time and hands Google that address as the redirect, so there is no redirect - URI to register and no domain to verify. A Web application client asks for - both and cannot be made to work here. -5. Copy the client ID into `client_id`, and put the client secret somewhere - `client_secret_command` can read it: +3. **OAuth consent screen**: choose **External**. Fill in the required fields. Under **Test users**, add your own Google address. +4. **Credentials → Create credentials → OAuth client ID**, and for **Application type** choose **Desktop app**. +5. Copy the client ID into `client_id`, and put the client secret somewhere `client_secret_command` can read it: ```sh secret-tool store --label="calcalist Google client secret" service google-calcalist ``` + 6. Authorise: ```sh calcalist google login gcal ``` - A browser opens; approve the request. calcalist asks only for - `https://www.googleapis.com/auth/calendar`. The refresh token is written - `0600` into the state directory, never into your configuration. + A browser opens; approve the request. CalCalist asks only for `https://www.googleapis.com/auth/calendar`. The refresh token is written into the state directory, never into your configuration. -**One login covers every endpoint on that account.** The authorisation is filed -under the account it was granted for, not the endpoint that asked, so a second -Google calendar on the same account needs no login of its own. Set `account` on -an endpoint only when calcalist is logged in to more than one account. +**One login covers every endpoint on that account.** The authorisation is filed under the account it was granted for, not the endpoint that asked, so a second Google calendar on the same account needs no login of its own. Set `account` on an endpoint only when CalCalist is logged in to more than one account. -**While the consent screen is in Testing, Google expires refresh tokens after -seven days.** An installation that worked last week then stops with nothing -having changed locally. Publish the consent screen to production to end this; -`calcalist doctor` reports it either way. +**While the consent screen is in Testing, Google expires refresh tokens after seven days.** An installation that worked last week then stops with nothing +having changed locally. Publish the consent screen to production to end this; `calcalist doctor` reports it either way. ## Getting events to the right calendar -An event mirrored from a source goes back to that source when you edit it — -calcalist tracks where each one came from. An event you *create* in the +An event mirrored from a source goes back to that source when you edit it — CalCalist tracks where each one came from. An event you *create* in the aggregate belongs to nothing yet, and goes to the aggregate's `default_sink`. -To send one somewhere else, put the endpoint's id after an `@` **on a line of -its own** in the event's notes: +To send one somewhere else, put the endpoint's id after an `@` **on a line of its own** in the event's notes: ``` Dentist, bring the referral @@ -216,38 +176,23 @@ Dentist, bring the referral @gcal ``` -The marker is removed before the event arrives. `calcalist status` prints the -markers that would actually work, so you need not remember what is in the -config; a marker naming anything else is refused and reported rather than -quietly filed under the default. +The marker is removed before the event arrives. `calcalist status` prints the markers that would actually work, so you need not remember what is in the +config; a marker naming anything else is refused and reported rather than quietly filed under the default. -**To keep an event in the aggregate itself**, tag it `@local` — it then stays -put even though a `default_sink` would have taken it, and the marker is -deliberately left in place so later cycles decide the same way. Leaving -`default_sink` out of the aggregate entirely makes that the rule for every -untagged event, so routing becomes opt-in. Such events live only in that -calendar and cannot be rebuilt from a source, but they are ordinary events on -that server, visible to every client subscribed to it. +**To keep an event in the aggregate itself**, tag it `@local` — it then stays put even though a `default_sink` would have taken it, and the marker is +deliberately left in place so later cycles decide the same way. -A category matching an endpoint id works too. Note that *any* category on a new -event is read as a routing instruction, so the notes marker is the safer form. +Leaving `default_sink` out of the aggregate entirely makes that the rule for every untagged event, so routing becomes opt-in. Such events live only in that calendar and cannot be rebuilt from a source, but they are ordinary events on that server, visible to every client subscribed to it. + +A category matching an endpoint id works too. Note that *any* category on a new event is read as a routing instruction, so the notes marker is the safer form. ## What it does to your calendars -- **Deleting an event in the aggregate deletes it at its source**, unless - `propagate_deletes = false`. A cycle that would remove more than - `max_delete_fraction` of the aggregate is refused until you pass `--force` — - with a floor of three, so removing one or two events is never blocked. -- **Edited in both places since the last sync, the source wins.** The conflict - is reported. -- **Alarms are always kept**, with no option to strip them. The tool exists so - you subscribe to one calendar rather than several, so dropping reminders would - disarm every one you have. -- **Mirroring never mails your guests.** On a Google target attendees are kept - and Google is told not to notify; on a CalDAV target, which cannot be told - that, the guest list is carried as inert data instead. Routing an event *you* - created is the exception: that server invites its guests for real, which is - the point of creating it. +- **Deleting an event in the aggregate deletes it at its source**, unless `propagate_deletes = false`. A cycle that would remove more than `max_delete_fraction` of the aggregate is refused until you pass `--force` — with a floor of three, so removing one or two events is never blocked. +- **Edited in both places since the last sync, the source wins.** The conflict is reported. +- **Alarms are always kept**, with no option to strip them. The tool exists so you subscribe to one calendar rather than several, so dropping reminders would disarm every one you have. +- **Mirroring never mails your guests.** On a Google target attendees are kept and Google is told not to notify; on a CalDAV target, which cannot be told + that, the guest list is carried as inert data instead. Routing an event *you* created is the exception: that server invites its guests for real, which is the point of creating it. ## Run it @@ -265,14 +210,8 @@ systemctl --user daemon-reload systemctl --user enable --now calcalist.timer ``` -The service expects the binary at `~/.local/bin/calcalist`; edit `ExecStart` if -yours is elsewhere. Only one cycle runs at a time — a timer firing while you are -running `sync` by hand is refused, naming the process that holds the lock. A -cycle that could not reach an endpoint exits non-zero on purpose, so a lapsed -token shows up as a failed unit rather than passing unnoticed; -`journalctl --user -u calcalist` has the report. If your secrets come from a -keyring needing an unlocked session, the timer only works while you are logged -in. +The service expects the binary at `~/.local/bin/calcalist`; edit `ExecStart` if yours is elsewhere. Only one cycle runs at a time — a timer firing while you are running `sync` by hand is refused, naming the process that holds the lock. A cycle that could not reach an endpoint exits non-zero on purpose, so a lapsed token shows up as a failed unit rather than passing unnoticed; `journalctl --user -u calcalist` has the report. If your secrets come from a +keyring needing an unlocked session, the timer only works while you are logged in. ## Commands @@ -288,8 +227,7 @@ in. | `aggregate retarget` | Moves an aggregate to a different target calendar. `sync` refuses on its own when the target changes under it, and points here. | | `prune` | Reports local mirrors of endpoints no longer configured; removes them under `--force`. | -Exit codes: `0` all well, `1` something failed or an endpoint could not be -reached. +Exit codes: `0` all well, `1` something failed or an endpoint could not be reached. ## Where things are kept @@ -304,14 +242,12 @@ vdir// the local mirror of each endpoint google/accounts/.json refresh tokens, 0600 ``` -None of it belongs in a backup or a dotfiles repository — it is a cache, and a -lost state file costs a re-materialisation rather than data. Copy the +None of it belongs in a backup or a dotfiles repository — it is a cache, and a lost state file costs a re-materialisation rather than data. Copy the configuration between machines; leave this behind. ## Known gaps -- **A failing `pimsync sync` stops the whole CalDAV leg.** Google endpoints are - isolated from each other, but pimsync is a single process covering every +- **A failing `pimsync sync` stops the whole CalDAV leg.** Google endpoints are isolated from each other, but pimsync is a single process covering every CalDAV and feed pair, and a failure does not say which pair it belongs to. ## License