CalCalist/src/pimsync.rs

873 lines
30 KiB
Rust
Raw Normal View History

//! Driving the `pimsync` binary.
//!
//! pimsync handles the CalDAV and WebCal legs, mirroring each remote into a local
//! vdir. It is invoked one-shot: its own daemon would race the reconciler over the
//! same files, so scheduling belongs to calcalist.
use std::fmt;
Drive pimsync for the CalDAV and WebCal legs Generates pimsync's configuration from calcalist.toml and invokes it one-shot either side of the reconcile step: pull first so the reconciler sees one consistent snapshot, push afterwards to publish what it decided. Its daemon mode stays unused because it would write the same vdirs the reconciler reads. - Endpoints are mapped with pimsync's collection blocks, matching a local vdir subdirectory to an exact remote path. Matching by collection id instead would have tied our directory names to whatever the server's URL segments happen to be, and two calendars ending in the same segment would collide. - The remote is storage_a with conflict_resolution keep a, so a concurrent server-side change wins over a local one calcalist has not yet reconciled. Nothing is lost: the next cycle sees the server's version and decides properly. - on_empty skip and on_delete skip on every pair, so a failed pull is not mirrored as a mass deletion. - Secrets stay out of the portable config: pimsync is given the same command calcalist would run to fetch them. - Google endpoints are omitted entirely, since pimsync can reach neither its REST API nor its CalDAV endpoint. doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and its parser does not always match its documentation, so asking it is the only reliable check — a parse rejection is reported as a failure, an unreachable server only as a warning. Found while testing against the real binary: pimsync 0.5.11 documents read_only as applying to every storage type but rejects it on a webcal storage, with only "Could not parse file" to go on. Dropped; it was redundant beside one_way. Verified end to end against a live Radicale server and an HTTP-served feed: the feed synced in, mirrored with a derived UID and provenance, and pushed to the server; repeated runs are no-ops and adding one feed event mirrors exactly one. 87 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:25:38 +03:00
use std::fs;
use std::io;
use std::os::fd::AsFd;
Drive pimsync for the CalDAV and WebCal legs Generates pimsync's configuration from calcalist.toml and invokes it one-shot either side of the reconcile step: pull first so the reconciler sees one consistent snapshot, push afterwards to publish what it decided. Its daemon mode stays unused because it would write the same vdirs the reconciler reads. - Endpoints are mapped with pimsync's collection blocks, matching a local vdir subdirectory to an exact remote path. Matching by collection id instead would have tied our directory names to whatever the server's URL segments happen to be, and two calendars ending in the same segment would collide. - The remote is storage_a with conflict_resolution keep a, so a concurrent server-side change wins over a local one calcalist has not yet reconciled. Nothing is lost: the next cycle sees the server's version and decides properly. - on_empty skip and on_delete skip on every pair, so a failed pull is not mirrored as a mass deletion. - Secrets stay out of the portable config: pimsync is given the same command calcalist would run to fetch them. - Google endpoints are omitted entirely, since pimsync can reach neither its REST API nor its CalDAV endpoint. doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and its parser does not always match its documentation, so asking it is the only reliable check — a parse rejection is reported as a failure, an unreachable server only as a warning. Found while testing against the real binary: pimsync 0.5.11 documents read_only as applying to every storage type but rejects it on a webcal storage, with only "Could not parse file" to go on. Dropped; it was redundant beside one_way. Verified end to end against a live Radicale server and an HTTP-served feed: the feed synced in, mirrored with a derived UID and provenance, and pushed to the server; repeated runs are no-ops and adding one feed event mirrors exactly one. 87 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:25:38 +03:00
use std::path::{Path, PathBuf};
use std::process::{Command, Output, Stdio};
use thiserror::Error;
Drive pimsync for the CalDAV and WebCal legs Generates pimsync's configuration from calcalist.toml and invokes it one-shot either side of the reconcile step: pull first so the reconciler sees one consistent snapshot, push afterwards to publish what it decided. Its daemon mode stays unused because it would write the same vdirs the reconciler reads. - Endpoints are mapped with pimsync's collection blocks, matching a local vdir subdirectory to an exact remote path. Matching by collection id instead would have tied our directory names to whatever the server's URL segments happen to be, and two calendars ending in the same segment would collide. - The remote is storage_a with conflict_resolution keep a, so a concurrent server-side change wins over a local one calcalist has not yet reconciled. Nothing is lost: the next cycle sees the server's version and decides properly. - on_empty skip and on_delete skip on every pair, so a failed pull is not mirrored as a mass deletion. - Secrets stay out of the portable config: pimsync is given the same command calcalist would run to fetch them. - Google endpoints are omitted entirely, since pimsync can reach neither its REST API nor its CalDAV endpoint. doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and its parser does not always match its documentation, so asking it is the only reliable check — a parse rejection is reported as a failure, an unreachable server only as a warning. Found while testing against the real binary: pimsync 0.5.11 documents read_only as applying to every storage type but rejects it on a webcal storage, with only "Could not parse file" to go on. Dropped; it was redundant beside one_way. Verified end to end against a live Radicale server and an HTTP-served feed: the feed synced in, mirrored with a derived UID and provenance, and pushed to the server; repeated runs are no-ops and adding one feed event mirrors exactly one. 87 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:25:38 +03:00
use crate::config::{Config, Endpoint, EndpointKind};
pub const BINARY: &str = "pimsync";
/// The oldest pimsync this build generates configuration for. `discover`'s
/// output format is undocumented and changed between 0.5.9 and 0.5.11, and
/// `parse_discovery` only understands the newer shape.
pub const MIN_VERSION: Version = Version {
major: 0,
minor: 5,
patch: 11,
};
/// The first release this build makes no claim about. pimsync is pre-1.0 and
/// its config syntax may change between minor releases, so a series is
/// admitted only once read: 0.6.0 renamed `auth` to `auth_method`, which costs
/// nothing here — generated storages carry `username` and `password`, never
/// `auth` — and left `discover`'s output word for word as it was.
pub const NEXT_UNKNOWN_VERSION: Version = Version {
major: 0,
minor: 7,
patch: 0,
};
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
pub struct Version {
pub major: u64,
pub minor: u64,
pub patch: u64,
}
impl fmt::Display for Version {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{}.{}.{}", self.major, self.minor, self.patch)
}
}
impl Version {
pub fn is_supported(&self) -> bool {
*self >= MIN_VERSION && *self < NEXT_UNKNOWN_VERSION
}
/// Parses the `pimsync 0.5.11` line printed by `pimsync version`.
fn parse(output: &str) -> Option<Self> {
let field = output.split_whitespace().nth(1)?;
let mut parts = field.split('.');
Some(Version {
major: number(parts.next()?)?,
minor: number(parts.next()?)?,
patch: number(parts.next().unwrap_or("0"))?,
})
}
}
/// The leading digits of one version component. pimsync takes its version from
/// `git describe`, so a build off a tag carries a suffix — `0.6.0-1-ga08ee3c`
/// — which belongs to no number.
fn number(part: &str) -> Option<u64> {
part.split(|c: char| !c.is_ascii_digit())
.next()?
.parse()
.ok()
}
#[derive(Debug, Error)]
pub enum PimsyncError {
#[error("`{BINARY}` was not found on PATH; install pimsync {MIN_VERSION} or newer")]
NotFound,
#[error("could not run `{BINARY}`: {0}")]
Spawn(#[source] io::Error),
#[error("`{BINARY} version` failed: {0}")]
Failed(String),
#[error("could not read a version out of `{0}`")]
Unparseable(String),
Drive pimsync for the CalDAV and WebCal legs Generates pimsync's configuration from calcalist.toml and invokes it one-shot either side of the reconcile step: pull first so the reconciler sees one consistent snapshot, push afterwards to publish what it decided. Its daemon mode stays unused because it would write the same vdirs the reconciler reads. - Endpoints are mapped with pimsync's collection blocks, matching a local vdir subdirectory to an exact remote path. Matching by collection id instead would have tied our directory names to whatever the server's URL segments happen to be, and two calendars ending in the same segment would collide. - The remote is storage_a with conflict_resolution keep a, so a concurrent server-side change wins over a local one calcalist has not yet reconciled. Nothing is lost: the next cycle sees the server's version and decides properly. - on_empty skip and on_delete skip on every pair, so a failed pull is not mirrored as a mass deletion. - Secrets stay out of the portable config: pimsync is given the same command calcalist would run to fetch them. - Google endpoints are omitted entirely, since pimsync can reach neither its REST API nor its CalDAV endpoint. doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and its parser does not always match its documentation, so asking it is the only reliable check — a parse rejection is reported as a failure, an unreachable server only as a warning. Found while testing against the real binary: pimsync 0.5.11 documents read_only as applying to every storage type but rejects it on a webcal storage, with only "Could not parse file" to go on. Dropped; it was redundant beside one_way. Verified end to end against a live Radicale server and an HTTP-served feed: the feed synced in, mirrored with a derived UID and provenance, and pushed to the server; repeated runs are no-ops and adding one feed event mirrors exactly one. 87 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:25:38 +03:00
#[error("could not write {path}: {source}")]
Write {
path: std::path::PathBuf,
#[source]
source: io::Error,
},
#[error("`{BINARY} {command}` failed: {message}")]
Command { command: String, message: String },
#[error(
"endpoint `{endpoint}` has url `{url}`, which names no calendar collection; it must point at the calendar itself, not at the server root"
)]
UrlHasNoCollection { endpoint: String, url: String },
}
/// Asks the installed pimsync for its version.
pub fn probe() -> Result<Version, PimsyncError> {
let output = Command::new(BINARY)
.arg("version")
.output()
.map_err(spawn_error)?;
if !output.status.success() {
return Err(PimsyncError::Failed(
String::from_utf8_lossy(&output.stderr).trim().to_string(),
));
}
let text = String::from_utf8_lossy(&output.stdout);
let line = text.lines().next().unwrap_or_default().trim();
Version::parse(line).ok_or_else(|| PimsyncError::Unparseable(line.to_string()))
}
Drive pimsync for the CalDAV and WebCal legs Generates pimsync's configuration from calcalist.toml and invokes it one-shot either side of the reconcile step: pull first so the reconciler sees one consistent snapshot, push afterwards to publish what it decided. Its daemon mode stays unused because it would write the same vdirs the reconciler reads. - Endpoints are mapped with pimsync's collection blocks, matching a local vdir subdirectory to an exact remote path. Matching by collection id instead would have tied our directory names to whatever the server's URL segments happen to be, and two calendars ending in the same segment would collide. - The remote is storage_a with conflict_resolution keep a, so a concurrent server-side change wins over a local one calcalist has not yet reconciled. Nothing is lost: the next cycle sees the server's version and decides properly. - on_empty skip and on_delete skip on every pair, so a failed pull is not mirrored as a mass deletion. - Secrets stay out of the portable config: pimsync is given the same command calcalist would run to fetch them. - Google endpoints are omitted entirely, since pimsync can reach neither its REST API nor its CalDAV endpoint. doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and its parser does not always match its documentation, so asking it is the only reliable check — a parse rejection is reported as a failure, an unreachable server only as a warning. Found while testing against the real binary: pimsync 0.5.11 documents read_only as applying to every storage type but rejects it on a webcal storage, with only "Could not parse file" to go on. Dropped; it was redundant beside one_way. Verified end to end against a live Radicale server and an HTTP-served feed: the feed synced in, mirrored with a derived UID and provenance, and pushed to the server; repeated runs are no-ops and adding one feed event mirrors exactly one. 87 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:25:38 +03:00
/// Name of the single local storage every pair mirrors into. pimsync allows one
/// storage to take part in several pairs, so one declaration serves all of them.
const LOCAL_STORAGE: &str = "local";
pub fn config_path(state_dir: &Path) -> PathBuf {
state_dir.join("pimsync.conf")
}
Close the remaining M1 gaps Eight items were still open at the end of M1: two integration tests that had only been run by hand, and six known gaps. Recurrence overrides now reach Google. Google addresses an exception through the series rather than as an event of its own, so the master is sent first and each override is then matched to its instance by original start time and patched. Matching needs the two sides' spellings reduced to one key: iCalendar writes a zoned local time, Google an absolute offset. An override matching no occurrence is counted rather than forced — that means a stale RECURRENCE-ID left behind by an edited RRULE, and inventing an event for it would put something in the calendar the series does not contain. Reading one component apart from another needed a view `properties` cannot give: it flattens every VEVENT together, which is right for the UID a series shares and wrong for an override, whose SUMMARY and the master's are then indistinguishable. `Calendar::events` splits them. A TZID now travels with the VTIMEZONE that defines it, derived from the zone's own transition table as the yearly rule it implies. This changes the content hash of every zoned recurring event, so the first cycle after this re-pushes them. A Google authorisation is filed under the account it was granted for rather than the endpoint that asked for it, so two endpoints on one account no longer need a login each. The account is read from the primary calendar's id, which needs no scope beyond the calendar one already granted. Authorisations written by the previous scheme are still honoured, and move across at the next login. An unreachable Google endpoint no longer ends the cycle — one lapsed token used to stop the CalDAV side too. It is named, only the aggregates depending on it stand down, and the run exits non-zero so a partial cycle cannot pass for success. The CalDAV leg cannot be narrowed the same way: pimsync is one process covering every pair, so a failure does not say which pair it belongs to. `--dry-run` now pulls for real, into a throwaway copy of the local mirrors and through a pimsync configuration that only ever reads from a server. What it reports is measured against the calendars as they are now rather than against whatever the last real cycle left behind. `calcalist prune` reports local mirrors of endpoints the configuration no longer names, and removes them under --force. The integration tests run against a real Radicale server and a real iCal feed: convergence and idempotence, the dry run, prune, and the scheduling rule asserted on the bytes that actually reached the server. The plan asked for an SMTP sink for that last one; Radicale implements no RFC 6638 scheduling, so a quiet SMTP port would have proved nothing about the transform. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 13:49:38 +03:00
pub(crate) fn status_path(state_dir: &Path) -> PathBuf {
Drive pimsync for the CalDAV and WebCal legs Generates pimsync's configuration from calcalist.toml and invokes it one-shot either side of the reconcile step: pull first so the reconciler sees one consistent snapshot, push afterwards to publish what it decided. Its daemon mode stays unused because it would write the same vdirs the reconciler reads. - Endpoints are mapped with pimsync's collection blocks, matching a local vdir subdirectory to an exact remote path. Matching by collection id instead would have tied our directory names to whatever the server's URL segments happen to be, and two calendars ending in the same segment would collide. - The remote is storage_a with conflict_resolution keep a, so a concurrent server-side change wins over a local one calcalist has not yet reconciled. Nothing is lost: the next cycle sees the server's version and decides properly. - on_empty skip and on_delete skip on every pair, so a failed pull is not mirrored as a mass deletion. - Secrets stay out of the portable config: pimsync is given the same command calcalist would run to fetch them. - Google endpoints are omitted entirely, since pimsync can reach neither its REST API nor its CalDAV endpoint. doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and its parser does not always match its documentation, so asking it is the only reliable check — a parse rejection is reported as a failure, an unreachable server only as a warning. Found while testing against the real binary: pimsync 0.5.11 documents read_only as applying to every storage type but rejects it on a webcal storage, with only "Could not parse file" to go on. Dropped; it was redundant beside one_way. Verified end to end against a live Radicale server and an HTTP-served feed: the feed synced in, mirrored with a derived UID and provenance, and pushed to the server; repeated runs are no-ops and adding one feed event mirrors exactly one. 87 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:25:38 +03:00
state_dir.join("pimsync-status")
}
Close the remaining M1 gaps Eight items were still open at the end of M1: two integration tests that had only been run by hand, and six known gaps. Recurrence overrides now reach Google. Google addresses an exception through the series rather than as an event of its own, so the master is sent first and each override is then matched to its instance by original start time and patched. Matching needs the two sides' spellings reduced to one key: iCalendar writes a zoned local time, Google an absolute offset. An override matching no occurrence is counted rather than forced — that means a stale RECURRENCE-ID left behind by an edited RRULE, and inventing an event for it would put something in the calendar the series does not contain. Reading one component apart from another needed a view `properties` cannot give: it flattens every VEVENT together, which is right for the UID a series shares and wrong for an override, whose SUMMARY and the master's are then indistinguishable. `Calendar::events` splits them. A TZID now travels with the VTIMEZONE that defines it, derived from the zone's own transition table as the yearly rule it implies. This changes the content hash of every zoned recurring event, so the first cycle after this re-pushes them. A Google authorisation is filed under the account it was granted for rather than the endpoint that asked for it, so two endpoints on one account no longer need a login each. The account is read from the primary calendar's id, which needs no scope beyond the calendar one already granted. Authorisations written by the previous scheme are still honoured, and move across at the next login. An unreachable Google endpoint no longer ends the cycle — one lapsed token used to stop the CalDAV side too. It is named, only the aggregates depending on it stand down, and the run exits non-zero so a partial cycle cannot pass for success. The CalDAV leg cannot be narrowed the same way: pimsync is one process covering every pair, so a failure does not say which pair it belongs to. `--dry-run` now pulls for real, into a throwaway copy of the local mirrors and through a pimsync configuration that only ever reads from a server. What it reports is measured against the calendars as they are now rather than against whatever the last real cycle left behind. `calcalist prune` reports local mirrors of endpoints the configuration no longer names, and removes them under --force. The integration tests run against a real Radicale server and a real iCal feed: convergence and idempotence, the dry run, prune, and the scheduling rule asserted on the bytes that actually reached the server. The plan asked for an SMTP sink for that last one; Radicale implements no RFC 6638 scheduling, so a quiet SMTP port would have proved nothing about the transform. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 13:49:38 +03:00
/// Which way a generated configuration lets items travel.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Direction {
/// The ordinary cycle: remote and local converge on each other.
Both,
/// Remote to local only, for `--dry-run`. pimsync writes nothing to any
/// server in this mode — but it does delete local items the remote has
/// never seen, so it is only ever pointed at a throwaway copy of the vdirs.
PullOnly,
}
Drive pimsync for the CalDAV and WebCal legs Generates pimsync's configuration from calcalist.toml and invokes it one-shot either side of the reconcile step: pull first so the reconciler sees one consistent snapshot, push afterwards to publish what it decided. Its daemon mode stays unused because it would write the same vdirs the reconciler reads. - Endpoints are mapped with pimsync's collection blocks, matching a local vdir subdirectory to an exact remote path. Matching by collection id instead would have tied our directory names to whatever the server's URL segments happen to be, and two calendars ending in the same segment would collide. - The remote is storage_a with conflict_resolution keep a, so a concurrent server-side change wins over a local one calcalist has not yet reconciled. Nothing is lost: the next cycle sees the server's version and decides properly. - on_empty skip and on_delete skip on every pair, so a failed pull is not mirrored as a mass deletion. - Secrets stay out of the portable config: pimsync is given the same command calcalist would run to fetch them. - Google endpoints are omitted entirely, since pimsync can reach neither its REST API nor its CalDAV endpoint. doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and its parser does not always match its documentation, so asking it is the only reliable check — a parse rejection is reported as a failure, an unreachable server only as a warning. Found while testing against the real binary: pimsync 0.5.11 documents read_only as applying to every storage type but rejects it on a webcal storage, with only "Could not parse file" to go on. Dropped; it was redundant beside one_way. Verified end to end against a live Radicale server and an HTTP-served feed: the feed synced in, mirrored with a derived UID and provenance, and pushed to the server; repeated runs are no-ops and adding one feed event mirrors exactly one. 87 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:25:38 +03:00
/// Builds a pimsync configuration covering every CalDAV and WebCal endpoint.
///
/// Google endpoints are absent by design: pimsync has no REST storage, and its
/// CalDAV storage speaks only HTTP Basic auth, which Google's endpoint has
/// rejected since March 2025. calcalist syncs those itself.
Close the remaining M1 gaps Eight items were still open at the end of M1: two integration tests that had only been run by hand, and six known gaps. Recurrence overrides now reach Google. Google addresses an exception through the series rather than as an event of its own, so the master is sent first and each override is then matched to its instance by original start time and patched. Matching needs the two sides' spellings reduced to one key: iCalendar writes a zoned local time, Google an absolute offset. An override matching no occurrence is counted rather than forced — that means a stale RECURRENCE-ID left behind by an edited RRULE, and inventing an event for it would put something in the calendar the series does not contain. Reading one component apart from another needed a view `properties` cannot give: it flattens every VEVENT together, which is right for the UID a series shares and wrong for an override, whose SUMMARY and the master's are then indistinguishable. `Calendar::events` splits them. A TZID now travels with the VTIMEZONE that defines it, derived from the zone's own transition table as the yearly rule it implies. This changes the content hash of every zoned recurring event, so the first cycle after this re-pushes them. A Google authorisation is filed under the account it was granted for rather than the endpoint that asked for it, so two endpoints on one account no longer need a login each. The account is read from the primary calendar's id, which needs no scope beyond the calendar one already granted. Authorisations written by the previous scheme are still honoured, and move across at the next login. An unreachable Google endpoint no longer ends the cycle — one lapsed token used to stop the CalDAV side too. It is named, only the aggregates depending on it stand down, and the run exits non-zero so a partial cycle cannot pass for success. The CalDAV leg cannot be narrowed the same way: pimsync is one process covering every pair, so a failure does not say which pair it belongs to. `--dry-run` now pulls for real, into a throwaway copy of the local mirrors and through a pimsync configuration that only ever reads from a server. What it reports is measured against the calendars as they are now rather than against whatever the last real cycle left behind. `calcalist prune` reports local mirrors of endpoints the configuration no longer names, and removes them under --force. The integration tests run against a real Radicale server and a real iCal feed: convergence and idempotence, the dry run, prune, and the scheduling rule asserted on the bytes that actually reached the server. The plan asked for an SMTP sink for that last one; Radicale implements no RFC 6638 scheduling, so a quiet SMTP port would have proved nothing about the transform. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 13:49:38 +03:00
pub fn generate_for(
config: &Config,
state_dir: &Path,
direction: Direction,
) -> Result<String, PimsyncError> {
Drive pimsync for the CalDAV and WebCal legs Generates pimsync's configuration from calcalist.toml and invokes it one-shot either side of the reconcile step: pull first so the reconciler sees one consistent snapshot, push afterwards to publish what it decided. Its daemon mode stays unused because it would write the same vdirs the reconciler reads. - Endpoints are mapped with pimsync's collection blocks, matching a local vdir subdirectory to an exact remote path. Matching by collection id instead would have tied our directory names to whatever the server's URL segments happen to be, and two calendars ending in the same segment would collide. - The remote is storage_a with conflict_resolution keep a, so a concurrent server-side change wins over a local one calcalist has not yet reconciled. Nothing is lost: the next cycle sees the server's version and decides properly. - on_empty skip and on_delete skip on every pair, so a failed pull is not mirrored as a mass deletion. - Secrets stay out of the portable config: pimsync is given the same command calcalist would run to fetch them. - Google endpoints are omitted entirely, since pimsync can reach neither its REST API nor its CalDAV endpoint. doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and its parser does not always match its documentation, so asking it is the only reliable check — a parse rejection is reported as a failure, an unreachable server only as a warning. Found while testing against the real binary: pimsync 0.5.11 documents read_only as applying to every storage type but rejects it on a webcal storage, with only "Could not parse file" to go on. Dropped; it was redundant beside one_way. Verified end to end against a live Radicale server and an HTTP-served feed: the feed synced in, mirrored with a derived UID and provenance, and pushed to the server; repeated runs are no-ops and adding one feed event mirrors exactly one. 87 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:25:38 +03:00
let mut out = String::new();
out.push_str("# Generated by calcalist. Edits here are overwritten on every sync.\n\n");
out.push_str(&format!(
"status_path {}\n\n",
quote(&status_path(state_dir).display().to_string())
));
out.push_str(&format!(
"storage {LOCAL_STORAGE} {{\n\ttype vdir/icalendar\n\tpath {}\n\tfileext ics\n}}\n",
quote(&crate::sync::vdir_root(state_dir).display().to_string())
));
for endpoint in &config.endpoints {
match &endpoint.kind {
Close the remaining M1 gaps Eight items were still open at the end of M1: two integration tests that had only been run by hand, and six known gaps. Recurrence overrides now reach Google. Google addresses an exception through the series rather than as an event of its own, so the master is sent first and each override is then matched to its instance by original start time and patched. Matching needs the two sides' spellings reduced to one key: iCalendar writes a zoned local time, Google an absolute offset. An override matching no occurrence is counted rather than forced — that means a stale RECURRENCE-ID left behind by an edited RRULE, and inventing an event for it would put something in the calendar the series does not contain. Reading one component apart from another needed a view `properties` cannot give: it flattens every VEVENT together, which is right for the UID a series shares and wrong for an override, whose SUMMARY and the master's are then indistinguishable. `Calendar::events` splits them. A TZID now travels with the VTIMEZONE that defines it, derived from the zone's own transition table as the yearly rule it implies. This changes the content hash of every zoned recurring event, so the first cycle after this re-pushes them. A Google authorisation is filed under the account it was granted for rather than the endpoint that asked for it, so two endpoints on one account no longer need a login each. The account is read from the primary calendar's id, which needs no scope beyond the calendar one already granted. Authorisations written by the previous scheme are still honoured, and move across at the next login. An unreachable Google endpoint no longer ends the cycle — one lapsed token used to stop the CalDAV side too. It is named, only the aggregates depending on it stand down, and the run exits non-zero so a partial cycle cannot pass for success. The CalDAV leg cannot be narrowed the same way: pimsync is one process covering every pair, so a failure does not say which pair it belongs to. `--dry-run` now pulls for real, into a throwaway copy of the local mirrors and through a pimsync configuration that only ever reads from a server. What it reports is measured against the calendars as they are now rather than against whatever the last real cycle left behind. `calcalist prune` reports local mirrors of endpoints the configuration no longer names, and removes them under --force. The integration tests run against a real Radicale server and a real iCal feed: convergence and idempotence, the dry run, prune, and the scheduling rule asserted on the bytes that actually reached the server. The plan asked for an SMTP sink for that last one; Radicale implements no RFC 6638 scheduling, so a quiet SMTP port would have proved nothing about the transform. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 13:49:38 +03:00
EndpointKind::Caldav { .. } => out.push_str(&caldav_pair(endpoint, direction)?),
Add Google OAuth, and let feed URLs come from a secret command Google requires OAuth for calendar access; app passwords stopped working for CalDAV, CardDAV and IMAP in March 2025, so there is no simpler path to offer. - Authorisation code flow over a loopback redirect, which is what Google supports for desktop clients now the copy-paste flow is gone, with PKCE so an intercepted code is useless without the verifier. Only the refresh token is persisted, 0600, in the state directory. - An expired grant is reported as itself: a consent screen still in Testing has its refresh tokens expired after 7 days, and "run calcalist google login" is more use than Google's bare invalid_grant. - doctor reports whether each Google endpoint is still authorised, since an installation that worked last week can stop with nothing having changed here. A webcal URL may now come from a command instead of the config. Google's secret iCal address grants read access to a whole calendar to anyone holding it, so writing it into a file described as portable and secret-free was a contradiction. Fixed a serious defect in the first draft of this module: random_token used fs::read on /dev/urandom, which reads to end of file. /dev/urandom has no end, so it allocated until the machine ran out of memory — it took the editor down with it. It now reads exactly 32 bytes, and a randomness failure is fatal rather than falling back to the clock, since a guessable state or PKCE verifier defeats the point of having them. Verified end to end against a live Posteo CalDAV calendar: pimsync validated the generated config against the real server, 58 events from a public feed were mirrored and pushed, and a second run was a no-op. 97 tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 12:40:47 +03:00
EndpointKind::Webcal { url, url_command } => {
out.push_str(&webcal_pair(
endpoint,
url.as_deref(),
url_command.as_deref(),
));
}
Drive pimsync for the CalDAV and WebCal legs Generates pimsync's configuration from calcalist.toml and invokes it one-shot either side of the reconcile step: pull first so the reconciler sees one consistent snapshot, push afterwards to publish what it decided. Its daemon mode stays unused because it would write the same vdirs the reconciler reads. - Endpoints are mapped with pimsync's collection blocks, matching a local vdir subdirectory to an exact remote path. Matching by collection id instead would have tied our directory names to whatever the server's URL segments happen to be, and two calendars ending in the same segment would collide. - The remote is storage_a with conflict_resolution keep a, so a concurrent server-side change wins over a local one calcalist has not yet reconciled. Nothing is lost: the next cycle sees the server's version and decides properly. - on_empty skip and on_delete skip on every pair, so a failed pull is not mirrored as a mass deletion. - Secrets stay out of the portable config: pimsync is given the same command calcalist would run to fetch them. - Google endpoints are omitted entirely, since pimsync can reach neither its REST API nor its CalDAV endpoint. doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and its parser does not always match its documentation, so asking it is the only reliable check — a parse rejection is reported as a failure, an unreachable server only as a warning. Found while testing against the real binary: pimsync 0.5.11 documents read_only as applying to every storage type but rejects it on a webcal storage, with only "Could not parse file" to go on. Dropped; it was redundant beside one_way. Verified end to end against a live Radicale server and an HTTP-served feed: the feed synced in, mirrored with a derived UID and provenance, and pushed to the server; repeated runs are no-ops and adding one feed event mirrors exactly one. 87 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:25:38 +03:00
// Handled by calcalist's own Google module.
EndpointKind::Google { .. } => {}
}
}
Ok(out)
}
Close the remaining M1 gaps Eight items were still open at the end of M1: two integration tests that had only been run by hand, and six known gaps. Recurrence overrides now reach Google. Google addresses an exception through the series rather than as an event of its own, so the master is sent first and each override is then matched to its instance by original start time and patched. Matching needs the two sides' spellings reduced to one key: iCalendar writes a zoned local time, Google an absolute offset. An override matching no occurrence is counted rather than forced — that means a stale RECURRENCE-ID left behind by an edited RRULE, and inventing an event for it would put something in the calendar the series does not contain. Reading one component apart from another needed a view `properties` cannot give: it flattens every VEVENT together, which is right for the UID a series shares and wrong for an override, whose SUMMARY and the master's are then indistinguishable. `Calendar::events` splits them. A TZID now travels with the VTIMEZONE that defines it, derived from the zone's own transition table as the yearly rule it implies. This changes the content hash of every zoned recurring event, so the first cycle after this re-pushes them. A Google authorisation is filed under the account it was granted for rather than the endpoint that asked for it, so two endpoints on one account no longer need a login each. The account is read from the primary calendar's id, which needs no scope beyond the calendar one already granted. Authorisations written by the previous scheme are still honoured, and move across at the next login. An unreachable Google endpoint no longer ends the cycle — one lapsed token used to stop the CalDAV side too. It is named, only the aggregates depending on it stand down, and the run exits non-zero so a partial cycle cannot pass for success. The CalDAV leg cannot be narrowed the same way: pimsync is one process covering every pair, so a failure does not say which pair it belongs to. `--dry-run` now pulls for real, into a throwaway copy of the local mirrors and through a pimsync configuration that only ever reads from a server. What it reports is measured against the calendars as they are now rather than against whatever the last real cycle left behind. `calcalist prune` reports local mirrors of endpoints the configuration no longer names, and removes them under --force. The integration tests run against a real Radicale server and a real iCal feed: convergence and idempotence, the dry run, prune, and the scheduling rule asserted on the bytes that actually reached the server. The plan asked for an SMTP sink for that last one; Radicale implements no RFC 6638 scheduling, so a quiet SMTP port would have proved nothing about the transform. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 13:49:38 +03:00
fn caldav_pair(endpoint: &Endpoint, direction: Direction) -> Result<String, PimsyncError> {
Drive pimsync for the CalDAV and WebCal legs Generates pimsync's configuration from calcalist.toml and invokes it one-shot either side of the reconcile step: pull first so the reconciler sees one consistent snapshot, push afterwards to publish what it decided. Its daemon mode stays unused because it would write the same vdirs the reconciler reads. - Endpoints are mapped with pimsync's collection blocks, matching a local vdir subdirectory to an exact remote path. Matching by collection id instead would have tied our directory names to whatever the server's URL segments happen to be, and two calendars ending in the same segment would collide. - The remote is storage_a with conflict_resolution keep a, so a concurrent server-side change wins over a local one calcalist has not yet reconciled. Nothing is lost: the next cycle sees the server's version and decides properly. - on_empty skip and on_delete skip on every pair, so a failed pull is not mirrored as a mass deletion. - Secrets stay out of the portable config: pimsync is given the same command calcalist would run to fetch them. - Google endpoints are omitted entirely, since pimsync can reach neither its REST API nor its CalDAV endpoint. doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and its parser does not always match its documentation, so asking it is the only reliable check — a parse rejection is reported as a failure, an unreachable server only as a warning. Found while testing against the real binary: pimsync 0.5.11 documents read_only as applying to every storage type but rejects it on a webcal storage, with only "Could not parse file" to go on. Dropped; it was redundant beside one_way. Verified end to end against a live Radicale server and an HTTP-served feed: the feed synced in, mirrored with a derived UID and provenance, and pushed to the server; repeated runs are no-ops and adding one feed event mirrors exactly one. 87 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:25:38 +03:00
let EndpointKind::Caldav {
url,
username,
secret_command,
} = &endpoint.kind
else {
unreachable!("caller matched on Caldav")
};
let (base, href) =
split_collection_url(url).ok_or_else(|| PimsyncError::UrlHasNoCollection {
endpoint: endpoint.id.clone(),
url: url.clone(),
})?;
let id = &endpoint.id;
let mut block = format!(
"\nstorage {id}_remote {{\n\ttype caldav\n\turl {}\n\tdiscovery collections\n\tusername {}\n",
quote(&base),
quote(username)
);
if let Some(command) = secret_command {
// Secrets never live in the portable config, so pimsync runs the same
// command calcalist would to fetch them.
block.push_str(&format!("\tpassword {{\n\t\tshell {command}\n\t}}\n"));
}
block.push_str("}\n");
// The remote is storage_a, so a concurrent server-side change wins over a
// local one calcalist has not yet reconciled. Nothing is lost: the next
// cycle sees the server's version and decides properly.
Close the remaining M1 gaps Eight items were still open at the end of M1: two integration tests that had only been run by hand, and six known gaps. Recurrence overrides now reach Google. Google addresses an exception through the series rather than as an event of its own, so the master is sent first and each override is then matched to its instance by original start time and patched. Matching needs the two sides' spellings reduced to one key: iCalendar writes a zoned local time, Google an absolute offset. An override matching no occurrence is counted rather than forced — that means a stale RECURRENCE-ID left behind by an edited RRULE, and inventing an event for it would put something in the calendar the series does not contain. Reading one component apart from another needed a view `properties` cannot give: it flattens every VEVENT together, which is right for the UID a series shares and wrong for an override, whose SUMMARY and the master's are then indistinguishable. `Calendar::events` splits them. A TZID now travels with the VTIMEZONE that defines it, derived from the zone's own transition table as the yearly rule it implies. This changes the content hash of every zoned recurring event, so the first cycle after this re-pushes them. A Google authorisation is filed under the account it was granted for rather than the endpoint that asked for it, so two endpoints on one account no longer need a login each. The account is read from the primary calendar's id, which needs no scope beyond the calendar one already granted. Authorisations written by the previous scheme are still honoured, and move across at the next login. An unreachable Google endpoint no longer ends the cycle — one lapsed token used to stop the CalDAV side too. It is named, only the aggregates depending on it stand down, and the run exits non-zero so a partial cycle cannot pass for success. The CalDAV leg cannot be narrowed the same way: pimsync is one process covering every pair, so a failure does not say which pair it belongs to. `--dry-run` now pulls for real, into a throwaway copy of the local mirrors and through a pimsync configuration that only ever reads from a server. What it reports is measured against the calendars as they are now rather than against whatever the last real cycle left behind. `calcalist prune` reports local mirrors of endpoints the configuration no longer names, and removes them under --force. The integration tests run against a real Radicale server and a real iCal feed: convergence and idempotence, the dry run, prune, and the scheduling rule asserted on the bytes that actually reached the server. The plan asked for an SMTP sink for that last one; Radicale implements no RFC 6638 scheduling, so a quiet SMTP port would have proved nothing about the transform. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 13:49:38 +03:00
let resolution = match direction {
// `one_way` never produces a conflict, so there is nothing to resolve.
Direction::PullOnly => "\tone_way\n",
Direction::Both => "\tconflict_resolution keep a\n",
};
Drive pimsync for the CalDAV and WebCal legs Generates pimsync's configuration from calcalist.toml and invokes it one-shot either side of the reconcile step: pull first so the reconciler sees one consistent snapshot, push afterwards to publish what it decided. Its daemon mode stays unused because it would write the same vdirs the reconciler reads. - Endpoints are mapped with pimsync's collection blocks, matching a local vdir subdirectory to an exact remote path. Matching by collection id instead would have tied our directory names to whatever the server's URL segments happen to be, and two calendars ending in the same segment would collide. - The remote is storage_a with conflict_resolution keep a, so a concurrent server-side change wins over a local one calcalist has not yet reconciled. Nothing is lost: the next cycle sees the server's version and decides properly. - on_empty skip and on_delete skip on every pair, so a failed pull is not mirrored as a mass deletion. - Secrets stay out of the portable config: pimsync is given the same command calcalist would run to fetch them. - Google endpoints are omitted entirely, since pimsync can reach neither its REST API nor its CalDAV endpoint. doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and its parser does not always match its documentation, so asking it is the only reliable check — a parse rejection is reported as a failure, an unreachable server only as a warning. Found while testing against the real binary: pimsync 0.5.11 documents read_only as applying to every storage type but rejects it on a webcal storage, with only "Could not parse file" to go on. Dropped; it was redundant beside one_way. Verified end to end against a live Radicale server and an HTTP-served feed: the feed synced in, mirrored with a derived UID and provenance, and pushed to the server; repeated runs are no-ops and adding one feed event mirrors exactly one. 87 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:25:38 +03:00
block.push_str(&format!(
"\npair {id} {{\n\tstorage_a {id}_remote\n\tstorage_b {LOCAL_STORAGE}\n\
\tcollection {{\n\t\talias {id}\n\t\thref_a {}\n\t\tid_b {id}\n\t}}\n\
Close the remaining M1 gaps Eight items were still open at the end of M1: two integration tests that had only been run by hand, and six known gaps. Recurrence overrides now reach Google. Google addresses an exception through the series rather than as an event of its own, so the master is sent first and each override is then matched to its instance by original start time and patched. Matching needs the two sides' spellings reduced to one key: iCalendar writes a zoned local time, Google an absolute offset. An override matching no occurrence is counted rather than forced — that means a stale RECURRENCE-ID left behind by an edited RRULE, and inventing an event for it would put something in the calendar the series does not contain. Reading one component apart from another needed a view `properties` cannot give: it flattens every VEVENT together, which is right for the UID a series shares and wrong for an override, whose SUMMARY and the master's are then indistinguishable. `Calendar::events` splits them. A TZID now travels with the VTIMEZONE that defines it, derived from the zone's own transition table as the yearly rule it implies. This changes the content hash of every zoned recurring event, so the first cycle after this re-pushes them. A Google authorisation is filed under the account it was granted for rather than the endpoint that asked for it, so two endpoints on one account no longer need a login each. The account is read from the primary calendar's id, which needs no scope beyond the calendar one already granted. Authorisations written by the previous scheme are still honoured, and move across at the next login. An unreachable Google endpoint no longer ends the cycle — one lapsed token used to stop the CalDAV side too. It is named, only the aggregates depending on it stand down, and the run exits non-zero so a partial cycle cannot pass for success. The CalDAV leg cannot be narrowed the same way: pimsync is one process covering every pair, so a failure does not say which pair it belongs to. `--dry-run` now pulls for real, into a throwaway copy of the local mirrors and through a pimsync configuration that only ever reads from a server. What it reports is measured against the calendars as they are now rather than against whatever the last real cycle left behind. `calcalist prune` reports local mirrors of endpoints the configuration no longer names, and removes them under --force. The integration tests run against a real Radicale server and a real iCal feed: convergence and idempotence, the dry run, prune, and the scheduling rule asserted on the bytes that actually reached the server. The plan asked for an SMTP sink for that last one; Radicale implements no RFC 6638 scheduling, so a quiet SMTP port would have proved nothing about the transform. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 13:49:38 +03:00
{resolution}\ton_empty skip\n\ton_delete skip\n}}\n",
Drive pimsync for the CalDAV and WebCal legs Generates pimsync's configuration from calcalist.toml and invokes it one-shot either side of the reconcile step: pull first so the reconciler sees one consistent snapshot, push afterwards to publish what it decided. Its daemon mode stays unused because it would write the same vdirs the reconciler reads. - Endpoints are mapped with pimsync's collection blocks, matching a local vdir subdirectory to an exact remote path. Matching by collection id instead would have tied our directory names to whatever the server's URL segments happen to be, and two calendars ending in the same segment would collide. - The remote is storage_a with conflict_resolution keep a, so a concurrent server-side change wins over a local one calcalist has not yet reconciled. Nothing is lost: the next cycle sees the server's version and decides properly. - on_empty skip and on_delete skip on every pair, so a failed pull is not mirrored as a mass deletion. - Secrets stay out of the portable config: pimsync is given the same command calcalist would run to fetch them. - Google endpoints are omitted entirely, since pimsync can reach neither its REST API nor its CalDAV endpoint. doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and its parser does not always match its documentation, so asking it is the only reliable check — a parse rejection is reported as a failure, an unreachable server only as a warning. Found while testing against the real binary: pimsync 0.5.11 documents read_only as applying to every storage type but rejects it on a webcal storage, with only "Could not parse file" to go on. Dropped; it was redundant beside one_way. Verified end to end against a live Radicale server and an HTTP-served feed: the feed synced in, mirrored with a derived UID and provenance, and pushed to the server; repeated runs are no-ops and adding one feed event mirrors exactly one. 87 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:25:38 +03:00
quote(&href)
));
Ok(block)
}
Add Google OAuth, and let feed URLs come from a secret command Google requires OAuth for calendar access; app passwords stopped working for CalDAV, CardDAV and IMAP in March 2025, so there is no simpler path to offer. - Authorisation code flow over a loopback redirect, which is what Google supports for desktop clients now the copy-paste flow is gone, with PKCE so an intercepted code is useless without the verifier. Only the refresh token is persisted, 0600, in the state directory. - An expired grant is reported as itself: a consent screen still in Testing has its refresh tokens expired after 7 days, and "run calcalist google login" is more use than Google's bare invalid_grant. - doctor reports whether each Google endpoint is still authorised, since an installation that worked last week can stop with nothing having changed here. A webcal URL may now come from a command instead of the config. Google's secret iCal address grants read access to a whole calendar to anyone holding it, so writing it into a file described as portable and secret-free was a contradiction. Fixed a serious defect in the first draft of this module: random_token used fs::read on /dev/urandom, which reads to end of file. /dev/urandom has no end, so it allocated until the machine ran out of memory — it took the editor down with it. It now reads exactly 32 bytes, and a randomness failure is fatal rather than falling back to the clock, since a guessable state or PKCE verifier defeats the point of having them. Verified end to end against a live Posteo CalDAV calendar: pimsync validated the generated config against the real server, 58 events from a public feed were mirrored and pushed, and a second run was a no-op. 97 tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 12:40:47 +03:00
fn webcal_pair(endpoint: &Endpoint, url: Option<&str>, url_command: Option<&str>) -> String {
Drive pimsync for the CalDAV and WebCal legs Generates pimsync's configuration from calcalist.toml and invokes it one-shot either side of the reconcile step: pull first so the reconciler sees one consistent snapshot, push afterwards to publish what it decided. Its daemon mode stays unused because it would write the same vdirs the reconciler reads. - Endpoints are mapped with pimsync's collection blocks, matching a local vdir subdirectory to an exact remote path. Matching by collection id instead would have tied our directory names to whatever the server's URL segments happen to be, and two calendars ending in the same segment would collide. - The remote is storage_a with conflict_resolution keep a, so a concurrent server-side change wins over a local one calcalist has not yet reconciled. Nothing is lost: the next cycle sees the server's version and decides properly. - on_empty skip and on_delete skip on every pair, so a failed pull is not mirrored as a mass deletion. - Secrets stay out of the portable config: pimsync is given the same command calcalist would run to fetch them. - Google endpoints are omitted entirely, since pimsync can reach neither its REST API nor its CalDAV endpoint. doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and its parser does not always match its documentation, so asking it is the only reliable check — a parse rejection is reported as a failure, an unreachable server only as a warning. Found while testing against the real binary: pimsync 0.5.11 documents read_only as applying to every storage type but rejects it on a webcal storage, with only "Could not parse file" to go on. Dropped; it was redundant beside one_way. Verified end to end against a live Radicale server and an HTTP-served feed: the feed synced in, mirrored with a derived UID and provenance, and pushed to the server; repeated runs are no-ops and adding one feed event mirrors exactly one. 87 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:25:38 +03:00
let id = &endpoint.id;
Add Google OAuth, and let feed URLs come from a secret command Google requires OAuth for calendar access; app passwords stopped working for CalDAV, CardDAV and IMAP in March 2025, so there is no simpler path to offer. - Authorisation code flow over a loopback redirect, which is what Google supports for desktop clients now the copy-paste flow is gone, with PKCE so an intercepted code is useless without the verifier. Only the refresh token is persisted, 0600, in the state directory. - An expired grant is reported as itself: a consent screen still in Testing has its refresh tokens expired after 7 days, and "run calcalist google login" is more use than Google's bare invalid_grant. - doctor reports whether each Google endpoint is still authorised, since an installation that worked last week can stop with nothing having changed here. A webcal URL may now come from a command instead of the config. Google's secret iCal address grants read access to a whole calendar to anyone holding it, so writing it into a file described as portable and secret-free was a contradiction. Fixed a serious defect in the first draft of this module: random_token used fs::read on /dev/urandom, which reads to end of file. /dev/urandom has no end, so it allocated until the machine ran out of memory — it took the editor down with it. It now reads exactly 32 bytes, and a randomness failure is fatal rather than falling back to the clock, since a guessable state or PKCE verifier defeats the point of having them. Verified end to end against a live Posteo CalDAV calendar: pimsync validated the generated config against the real server, 58 events from a public feed were mirrored and pushed, and a second run was a no-op. 97 tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 12:40:47 +03:00
// A feed URL that is itself a credential is fetched by command, the same way
// passwords are, so it never appears in either configuration file.
let url_directive = match (url, url_command) {
(_, Some(command)) => format!("\turl {{\n\t\tshell {command}\n\t}}\n"),
(Some(url), None) => format!("\turl {}\n", quote(url)),
(None, None) => String::new(),
};
Drive pimsync for the CalDAV and WebCal legs Generates pimsync's configuration from calcalist.toml and invokes it one-shot either side of the reconcile step: pull first so the reconciler sees one consistent snapshot, push afterwards to publish what it decided. Its daemon mode stays unused because it would write the same vdirs the reconciler reads. - Endpoints are mapped with pimsync's collection blocks, matching a local vdir subdirectory to an exact remote path. Matching by collection id instead would have tied our directory names to whatever the server's URL segments happen to be, and two calendars ending in the same segment would collide. - The remote is storage_a with conflict_resolution keep a, so a concurrent server-side change wins over a local one calcalist has not yet reconciled. Nothing is lost: the next cycle sees the server's version and decides properly. - on_empty skip and on_delete skip on every pair, so a failed pull is not mirrored as a mass deletion. - Secrets stay out of the portable config: pimsync is given the same command calcalist would run to fetch them. - Google endpoints are omitted entirely, since pimsync can reach neither its REST API nor its CalDAV endpoint. doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and its parser does not always match its documentation, so asking it is the only reliable check — a parse rejection is reported as a failure, an unreachable server only as a warning. Found while testing against the real binary: pimsync 0.5.11 documents read_only as applying to every storage type but rejects it on a webcal storage, with only "Could not parse file" to go on. Dropped; it was redundant beside one_way. Verified end to end against a live Radicale server and an HTTP-served feed: the feed synced in, mirrored with a derived UID and provenance, and pushed to the server; repeated runs are no-ops and adding one feed event mirrors exactly one. 87 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:25:38 +03:00
// No `read_only` directive here: pimsync 0.5.11 documents it as applying to
// every storage type, but rejects it on a webcal storage with a bare "Could
// not parse file". It is redundant regardless — webcal is read-only by type,
// and `one_way` already fixes the direction.
format!(
Add Google OAuth, and let feed URLs come from a secret command Google requires OAuth for calendar access; app passwords stopped working for CalDAV, CardDAV and IMAP in March 2025, so there is no simpler path to offer. - Authorisation code flow over a loopback redirect, which is what Google supports for desktop clients now the copy-paste flow is gone, with PKCE so an intercepted code is useless without the verifier. Only the refresh token is persisted, 0600, in the state directory. - An expired grant is reported as itself: a consent screen still in Testing has its refresh tokens expired after 7 days, and "run calcalist google login" is more use than Google's bare invalid_grant. - doctor reports whether each Google endpoint is still authorised, since an installation that worked last week can stop with nothing having changed here. A webcal URL may now come from a command instead of the config. Google's secret iCal address grants read access to a whole calendar to anyone holding it, so writing it into a file described as portable and secret-free was a contradiction. Fixed a serious defect in the first draft of this module: random_token used fs::read on /dev/urandom, which reads to end of file. /dev/urandom has no end, so it allocated until the machine ran out of memory — it took the editor down with it. It now reads exactly 32 bytes, and a randomness failure is fatal rather than falling back to the clock, since a guessable state or PKCE verifier defeats the point of having them. Verified end to end against a live Posteo CalDAV calendar: pimsync validated the generated config against the real server, 58 events from a public feed were mirrored and pushed, and a second run was a no-op. 97 tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 12:40:47 +03:00
"\nstorage {id}_remote {{\n\ttype webcal\n{url_directive}\tcollection_id {id}\n}}\n\
Drive pimsync for the CalDAV and WebCal legs Generates pimsync's configuration from calcalist.toml and invokes it one-shot either side of the reconcile step: pull first so the reconciler sees one consistent snapshot, push afterwards to publish what it decided. Its daemon mode stays unused because it would write the same vdirs the reconciler reads. - Endpoints are mapped with pimsync's collection blocks, matching a local vdir subdirectory to an exact remote path. Matching by collection id instead would have tied our directory names to whatever the server's URL segments happen to be, and two calendars ending in the same segment would collide. - The remote is storage_a with conflict_resolution keep a, so a concurrent server-side change wins over a local one calcalist has not yet reconciled. Nothing is lost: the next cycle sees the server's version and decides properly. - on_empty skip and on_delete skip on every pair, so a failed pull is not mirrored as a mass deletion. - Secrets stay out of the portable config: pimsync is given the same command calcalist would run to fetch them. - Google endpoints are omitted entirely, since pimsync can reach neither its REST API nor its CalDAV endpoint. doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and its parser does not always match its documentation, so asking it is the only reliable check — a parse rejection is reported as a failure, an unreachable server only as a warning. Found while testing against the real binary: pimsync 0.5.11 documents read_only as applying to every storage type but rejects it on a webcal storage, with only "Could not parse file" to go on. Dropped; it was redundant beside one_way. Verified end to end against a live Radicale server and an HTTP-served feed: the feed synced in, mirrored with a derived UID and provenance, and pushed to the server; repeated runs are no-ops and adding one feed event mirrors exactly one. 87 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:25:38 +03:00
\npair {id} {{\n\tstorage_a {id}_remote\n\tstorage_b {LOCAL_STORAGE}\n\
\tcollection {{\n\t\talias {id}\n\t\tid_a {id}\n\t\tid_b {id}\n\t}}\n\
Add Google OAuth, and let feed URLs come from a secret command Google requires OAuth for calendar access; app passwords stopped working for CalDAV, CardDAV and IMAP in March 2025, so there is no simpler path to offer. - Authorisation code flow over a loopback redirect, which is what Google supports for desktop clients now the copy-paste flow is gone, with PKCE so an intercepted code is useless without the verifier. Only the refresh token is persisted, 0600, in the state directory. - An expired grant is reported as itself: a consent screen still in Testing has its refresh tokens expired after 7 days, and "run calcalist google login" is more use than Google's bare invalid_grant. - doctor reports whether each Google endpoint is still authorised, since an installation that worked last week can stop with nothing having changed here. A webcal URL may now come from a command instead of the config. Google's secret iCal address grants read access to a whole calendar to anyone holding it, so writing it into a file described as portable and secret-free was a contradiction. Fixed a serious defect in the first draft of this module: random_token used fs::read on /dev/urandom, which reads to end of file. /dev/urandom has no end, so it allocated until the machine ran out of memory — it took the editor down with it. It now reads exactly 32 bytes, and a randomness failure is fatal rather than falling back to the clock, since a guessable state or PKCE verifier defeats the point of having them. Verified end to end against a live Posteo CalDAV calendar: pimsync validated the generated config against the real server, 58 events from a public feed were mirrored and pushed, and a second run was a no-op. 97 tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 12:40:47 +03:00
\tone_way\n\ton_empty skip\n\ton_delete skip\n}}\n"
Drive pimsync for the CalDAV and WebCal legs Generates pimsync's configuration from calcalist.toml and invokes it one-shot either side of the reconcile step: pull first so the reconciler sees one consistent snapshot, push afterwards to publish what it decided. Its daemon mode stays unused because it would write the same vdirs the reconciler reads. - Endpoints are mapped with pimsync's collection blocks, matching a local vdir subdirectory to an exact remote path. Matching by collection id instead would have tied our directory names to whatever the server's URL segments happen to be, and two calendars ending in the same segment would collide. - The remote is storage_a with conflict_resolution keep a, so a concurrent server-side change wins over a local one calcalist has not yet reconciled. Nothing is lost: the next cycle sees the server's version and decides properly. - on_empty skip and on_delete skip on every pair, so a failed pull is not mirrored as a mass deletion. - Secrets stay out of the portable config: pimsync is given the same command calcalist would run to fetch them. - Google endpoints are omitted entirely, since pimsync can reach neither its REST API nor its CalDAV endpoint. doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and its parser does not always match its documentation, so asking it is the only reliable check — a parse rejection is reported as a failure, an unreachable server only as a warning. Found while testing against the real binary: pimsync 0.5.11 documents read_only as applying to every storage type but rejects it on a webcal storage, with only "Could not parse file" to go on. Dropped; it was redundant beside one_way. Verified end to end against a live Radicale server and an HTTP-served feed: the feed synced in, mirrored with a derived UID and provenance, and pushed to the server; repeated runs are no-ops and adding one feed event mirrors exactly one. 87 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:25:38 +03:00
)
}
/// Splits a calendar URL into the container to enumerate and the collection's
/// own path.
///
/// pimsync identifies a collection either by an id taken from a URL segment or
/// by its exact path. The path is used here, so a calendar whose last segment
/// happens to match another's cannot be confused for it.
fn split_collection_url(url: &str) -> Option<(String, String)> {
let authority_start = url.find("://")? + 3;
let path_start = authority_start + url[authority_start..].find('/')?;
let segments: Vec<&str> = url[path_start..]
.split('/')
.filter(|segment| !segment.is_empty())
.collect();
let (_, parents) = segments.split_last()?;
let href = format!("/{}/", segments.join("/"));
let base = match parents {
[] => format!("{}/", &url[..path_start]),
parents => format!("{}/{}/", &url[..path_start], parents.join("/")),
};
Some((base, href))
}
/// pimsync accepts quoted values, which keeps paths with spaces intact.
fn quote(value: &str) -> String {
format!("\"{}\"", value.replace('\\', "\\\\").replace('"', "\\\""))
}
/// Writes the generated configuration, returning its path.
pub fn write_config(config: &Config, state_dir: &Path) -> Result<PathBuf, PimsyncError> {
Close the remaining M1 gaps Eight items were still open at the end of M1: two integration tests that had only been run by hand, and six known gaps. Recurrence overrides now reach Google. Google addresses an exception through the series rather than as an event of its own, so the master is sent first and each override is then matched to its instance by original start time and patched. Matching needs the two sides' spellings reduced to one key: iCalendar writes a zoned local time, Google an absolute offset. An override matching no occurrence is counted rather than forced — that means a stale RECURRENCE-ID left behind by an edited RRULE, and inventing an event for it would put something in the calendar the series does not contain. Reading one component apart from another needed a view `properties` cannot give: it flattens every VEVENT together, which is right for the UID a series shares and wrong for an override, whose SUMMARY and the master's are then indistinguishable. `Calendar::events` splits them. A TZID now travels with the VTIMEZONE that defines it, derived from the zone's own transition table as the yearly rule it implies. This changes the content hash of every zoned recurring event, so the first cycle after this re-pushes them. A Google authorisation is filed under the account it was granted for rather than the endpoint that asked for it, so two endpoints on one account no longer need a login each. The account is read from the primary calendar's id, which needs no scope beyond the calendar one already granted. Authorisations written by the previous scheme are still honoured, and move across at the next login. An unreachable Google endpoint no longer ends the cycle — one lapsed token used to stop the CalDAV side too. It is named, only the aggregates depending on it stand down, and the run exits non-zero so a partial cycle cannot pass for success. The CalDAV leg cannot be narrowed the same way: pimsync is one process covering every pair, so a failure does not say which pair it belongs to. `--dry-run` now pulls for real, into a throwaway copy of the local mirrors and through a pimsync configuration that only ever reads from a server. What it reports is measured against the calendars as they are now rather than against whatever the last real cycle left behind. `calcalist prune` reports local mirrors of endpoints the configuration no longer names, and removes them under --force. The integration tests run against a real Radicale server and a real iCal feed: convergence and idempotence, the dry run, prune, and the scheduling rule asserted on the bytes that actually reached the server. The plan asked for an SMTP sink for that last one; Radicale implements no RFC 6638 scheduling, so a quiet SMTP port would have proved nothing about the transform. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 13:49:38 +03:00
write_config_for(config, state_dir, Direction::Both)
}
pub fn write_config_for(
config: &Config,
state_dir: &Path,
direction: Direction,
) -> Result<PathBuf, PimsyncError> {
Drive pimsync for the CalDAV and WebCal legs Generates pimsync's configuration from calcalist.toml and invokes it one-shot either side of the reconcile step: pull first so the reconciler sees one consistent snapshot, push afterwards to publish what it decided. Its daemon mode stays unused because it would write the same vdirs the reconciler reads. - Endpoints are mapped with pimsync's collection blocks, matching a local vdir subdirectory to an exact remote path. Matching by collection id instead would have tied our directory names to whatever the server's URL segments happen to be, and two calendars ending in the same segment would collide. - The remote is storage_a with conflict_resolution keep a, so a concurrent server-side change wins over a local one calcalist has not yet reconciled. Nothing is lost: the next cycle sees the server's version and decides properly. - on_empty skip and on_delete skip on every pair, so a failed pull is not mirrored as a mass deletion. - Secrets stay out of the portable config: pimsync is given the same command calcalist would run to fetch them. - Google endpoints are omitted entirely, since pimsync can reach neither its REST API nor its CalDAV endpoint. doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and its parser does not always match its documentation, so asking it is the only reliable check — a parse rejection is reported as a failure, an unreachable server only as a warning. Found while testing against the real binary: pimsync 0.5.11 documents read_only as applying to every storage type but rejects it on a webcal storage, with only "Could not parse file" to go on. Dropped; it was redundant beside one_way. Verified end to end against a live Radicale server and an HTTP-served feed: the feed synced in, mirrored with a derived UID and provenance, and pushed to the server; repeated runs are no-ops and adding one feed event mirrors exactly one. 87 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:25:38 +03:00
let path = config_path(state_dir);
Close the remaining M1 gaps Eight items were still open at the end of M1: two integration tests that had only been run by hand, and six known gaps. Recurrence overrides now reach Google. Google addresses an exception through the series rather than as an event of its own, so the master is sent first and each override is then matched to its instance by original start time and patched. Matching needs the two sides' spellings reduced to one key: iCalendar writes a zoned local time, Google an absolute offset. An override matching no occurrence is counted rather than forced — that means a stale RECURRENCE-ID left behind by an edited RRULE, and inventing an event for it would put something in the calendar the series does not contain. Reading one component apart from another needed a view `properties` cannot give: it flattens every VEVENT together, which is right for the UID a series shares and wrong for an override, whose SUMMARY and the master's are then indistinguishable. `Calendar::events` splits them. A TZID now travels with the VTIMEZONE that defines it, derived from the zone's own transition table as the yearly rule it implies. This changes the content hash of every zoned recurring event, so the first cycle after this re-pushes them. A Google authorisation is filed under the account it was granted for rather than the endpoint that asked for it, so two endpoints on one account no longer need a login each. The account is read from the primary calendar's id, which needs no scope beyond the calendar one already granted. Authorisations written by the previous scheme are still honoured, and move across at the next login. An unreachable Google endpoint no longer ends the cycle — one lapsed token used to stop the CalDAV side too. It is named, only the aggregates depending on it stand down, and the run exits non-zero so a partial cycle cannot pass for success. The CalDAV leg cannot be narrowed the same way: pimsync is one process covering every pair, so a failure does not say which pair it belongs to. `--dry-run` now pulls for real, into a throwaway copy of the local mirrors and through a pimsync configuration that only ever reads from a server. What it reports is measured against the calendars as they are now rather than against whatever the last real cycle left behind. `calcalist prune` reports local mirrors of endpoints the configuration no longer names, and removes them under --force. The integration tests run against a real Radicale server and a real iCal feed: convergence and idempotence, the dry run, prune, and the scheduling rule asserted on the bytes that actually reached the server. The plan asked for an SMTP sink for that last one; Radicale implements no RFC 6638 scheduling, so a quiet SMTP port would have proved nothing about the transform. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 13:49:38 +03:00
let text = generate_for(config, state_dir, direction)?;
Drive pimsync for the CalDAV and WebCal legs Generates pimsync's configuration from calcalist.toml and invokes it one-shot either side of the reconcile step: pull first so the reconciler sees one consistent snapshot, push afterwards to publish what it decided. Its daemon mode stays unused because it would write the same vdirs the reconciler reads. - Endpoints are mapped with pimsync's collection blocks, matching a local vdir subdirectory to an exact remote path. Matching by collection id instead would have tied our directory names to whatever the server's URL segments happen to be, and two calendars ending in the same segment would collide. - The remote is storage_a with conflict_resolution keep a, so a concurrent server-side change wins over a local one calcalist has not yet reconciled. Nothing is lost: the next cycle sees the server's version and decides properly. - on_empty skip and on_delete skip on every pair, so a failed pull is not mirrored as a mass deletion. - Secrets stay out of the portable config: pimsync is given the same command calcalist would run to fetch them. - Google endpoints are omitted entirely, since pimsync can reach neither its REST API nor its CalDAV endpoint. doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and its parser does not always match its documentation, so asking it is the only reliable check — a parse rejection is reported as a failure, an unreachable server only as a warning. Found while testing against the real binary: pimsync 0.5.11 documents read_only as applying to every storage type but rejects it on a webcal storage, with only "Could not parse file" to go on. Dropped; it was redundant beside one_way. Verified end to end against a live Radicale server and an HTTP-served feed: the feed synced in, mirrored with a derived UID and provenance, and pushed to the server; repeated runs are no-ops and adding one feed event mirrors exactly one. 87 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:25:38 +03:00
fs::create_dir_all(status_path(state_dir)).map_err(|source| PimsyncError::Write {
path: status_path(state_dir),
source,
})?;
fs::write(&path, text).map_err(|source| PimsyncError::Write {
path: path.clone(),
source,
})?;
Ok(path)
}
Finish M2: run lock, systemd units, discover; drop the web UI Designing the configuration UI in full made the case against building it. Its audience would be people who find TOML hard, but with bring-your-own OAuth client settled, every user must first create a Google Cloud project, configure a consent screen and put a secret in a keyring — a far higher bar than editing thirty lines of config. Anyone who clears it can edit the file; anyone who cannot never reaches the file. Against that stood three dependencies, five modules, an auth.rs refactor and a security surface guarding something that reads the config and touches the keyring. `doctor` and `status` had already absorbed most of what it was for. The reasoning is recorded in TODO.md and SPECS.md rather than left as an apparent oversight. The run lock is not a UI feature and closes a gap that already existed: nothing stopped a timer firing into a hand-run cycle, and two cycles interleaving writes over the same vdirs is what the design otherwise avoids. flock is used rather than a pid file because the kernel releases it however the process ends, so a crash cannot leave a lock to clear by hand — which also means a lock we failed to take is held by a live process, so the pid in it is worth reporting. The one idea worth keeping from the UI design was collection discovery, which needed no web layer. `calcalist discover` prints a ready-to-paste endpoint block per calendar a server offers, removing the most error-prone field in the config. pimsync's discovery output is undocumented, so the format was established against a real server first. Two things it teaches: everything arrives on stdout including failures, and a pair has two storages, so pimsync reports the scratch vdir's contents too — parsing anchors on the heading naming the server, or a probe directory's leftovers would be offered as the user's calendars. Verified against Posteo as well as Radicale: all four calendars found, the first matching the URL already configured. Also fixes a real defect in the test harness rather than its symptom. Ports were chosen by binding one and letting go, so two tests could pick the same number — and the loser's readiness check then succeeded against the winner's server, silently sharing it. Startup now confirms the child we spawned is the one alive, retries on another port if not, and waits for a real HTTP response rather than an open socket. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 16:17:34 +03:00
/// The storage name the probe config gives the server being examined. Parsing
/// anchors on it, because pimsync discovers *both* storages in a pair and prints
/// their collections under one heading each.
const PROBE_REMOTE: &str = "probe_remote";
/// One collection found on a server.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Collection {
/// The path pimsync reported, e.g. `/calendars/me/work/`.
pub href: String,
/// A usable endpoint id, taken from the last meaningful path segment.
pub id: String,
}
/// Asks pimsync which calendars a server offers.
///
/// `url` is a container to enumerate — an account or principal path — rather
/// than a single calendar, so `split_collection_url` does not apply here.
///
/// `pimsync discover` is documented as informational and as not altering
/// pimsync's state, so pointing it at a server the user has not configured yet
/// is safe. The probe configuration it needs is written under a name of its own
/// so it can never be mistaken for, or overwrite, the generated `pimsync.conf`.
pub fn discover(
state_dir: &Path,
url: &str,
username: &str,
secret_command: Option<&str>,
) -> Result<Vec<Collection>, PimsyncError> {
let config_path = state_dir.join("discover.conf");
let scratch = state_dir.join("discover-vdir");
fs::create_dir_all(&scratch).map_err(|source| PimsyncError::Write {
path: scratch.clone(),
source,
})?;
fs::write(
&config_path,
probe_config(state_dir, &scratch, url, username, secret_command),
)
.map_err(|source| PimsyncError::Write {
path: config_path.clone(),
source,
})?;
let output = Command::new(BINARY)
.arg("-c")
.arg(&config_path)
.arg("discover")
.output()
.map_err(spawn_error)?;
Finish M2: run lock, systemd units, discover; drop the web UI Designing the configuration UI in full made the case against building it. Its audience would be people who find TOML hard, but with bring-your-own OAuth client settled, every user must first create a Google Cloud project, configure a consent screen and put a secret in a keyring — a far higher bar than editing thirty lines of config. Anyone who clears it can edit the file; anyone who cannot never reaches the file. Against that stood three dependencies, five modules, an auth.rs refactor and a security surface guarding something that reads the config and touches the keyring. `doctor` and `status` had already absorbed most of what it was for. The reasoning is recorded in TODO.md and SPECS.md rather than left as an apparent oversight. The run lock is not a UI feature and closes a gap that already existed: nothing stopped a timer firing into a hand-run cycle, and two cycles interleaving writes over the same vdirs is what the design otherwise avoids. flock is used rather than a pid file because the kernel releases it however the process ends, so a crash cannot leave a lock to clear by hand — which also means a lock we failed to take is held by a live process, so the pid in it is worth reporting. The one idea worth keeping from the UI design was collection discovery, which needed no web layer. `calcalist discover` prints a ready-to-paste endpoint block per calendar a server offers, removing the most error-prone field in the config. pimsync's discovery output is undocumented, so the format was established against a real server first. Two things it teaches: everything arrives on stdout including failures, and a pair has two storages, so pimsync reports the scratch vdir's contents too — parsing anchors on the heading naming the server, or a probe directory's leftovers would be offered as the user's calendars. Verified against Posteo as well as Radicale: all four calendars found, the first matching the URL already configured. Also fixes a real defect in the test harness rather than its symptom. Ports were chosen by binding one and letting go, so two tests could pick the same number — and the loser's readiness check then succeeded against the winner's server, silently sharing it. Startup now confirms the child we spawned is the one alive, retries on another port if not, and waits for a real HTTP response rather than an open socket. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 16:17:34 +03:00
// Everything, including the failures, comes back on stdout.
let text = String::from_utf8_lossy(&output.stdout).into_owned();
let _ = fs::remove_dir_all(&scratch);
if !output.status.success() {
return Err(PimsyncError::Command {
command: "discover".to_string(),
message: strip_ansi(&text).trim().to_string(),
});
}
Ok(parse_discovery(&text))
}
fn probe_config(
state_dir: &Path,
scratch: &Path,
url: &str,
username: &str,
secret_command: Option<&str>,
) -> String {
let password = match secret_command {
Some(command) => format!("\tpassword {{\n\t\tshell {command}\n\t}}\n"),
None => String::new(),
};
format!(
"# Generated by `calcalist discover`. Rewritten on every run.\n\
status_path {}\n\n\
storage local {{\n\ttype vdir/icalendar\n\tpath {}\n\tfileext ics\n}}\n\n\
storage {PROBE_REMOTE} {{\n\ttype caldav\n\turl {}\n\tdiscovery collections\n\
\tusername {}\n{password}}}\n\n\
pair probe {{\n\tstorage_a {PROBE_REMOTE}\n\tstorage_b local\n\
\tcollections from a\n}}\n",
quote(&state_dir.join("discover-status").display().to_string()),
quote(&scratch.display().to_string()),
quote(url),
quote(username),
)
}
/// Reads the collections out of `pimsync discover` output.
///
/// The format is not documented, so this was established against a real server
/// and is deliberately forgiving. The one thing it must get right: a pair has
/// two storages and pimsync reports both, so `Found` lines are only taken after
/// the heading naming the server. Otherwise the scratch directory's own
/// contents would be offered as if they were the user's calendars.
fn parse_discovery(output: &str) -> Vec<Collection> {
let mut collections = Vec::new();
let mut in_remote = false;
for line in output.lines() {
let line = strip_ansi(line);
let line = line.trim();
if line.starts_with("==>") {
in_remote = line.contains(PROBE_REMOTE);
continue;
}
let Some(href) = line.strip_prefix("Found ") else {
continue;
};
let href = href.trim();
if !in_remote || href.is_empty() {
continue;
}
collections.push(Collection {
id: collection_id(href),
href: href.to_string(),
});
}
collections
}
/// A plausible endpoint id: the last non-empty path segment, restricted to
/// characters that are unambiguous in a configuration file.
fn collection_id(href: &str) -> String {
let segment = href
.rsplit('/')
.find(|segment| !segment.is_empty())
.unwrap_or(href);
let id: String = segment
.chars()
.map(|ch| match ch {
'a'..='z' | '0'..='9' | '-' | '_' => ch,
'A'..='Z' => ch.to_ascii_lowercase(),
_ => '-',
})
.collect();
let id = id.trim_matches('-').to_string();
if id.is_empty() { "calendar".into() } else { id }
}
/// Removes ANSI colour, which pimsync emits even when its output is a pipe and
/// even under NO_COLOR.
fn strip_ansi(text: &str) -> String {
let mut out = String::with_capacity(text.len());
let mut chars = text.chars();
while let Some(ch) = chars.next() {
if ch != '\u{1b}' {
out.push(ch);
continue;
}
// Skip up to and including the terminating letter of the escape.
for next in chars.by_ref() {
if next.is_ascii_alphabetic() {
break;
}
}
}
out
}
Drive pimsync for the CalDAV and WebCal legs Generates pimsync's configuration from calcalist.toml and invokes it one-shot either side of the reconcile step: pull first so the reconciler sees one consistent snapshot, push afterwards to publish what it decided. Its daemon mode stays unused because it would write the same vdirs the reconciler reads. - Endpoints are mapped with pimsync's collection blocks, matching a local vdir subdirectory to an exact remote path. Matching by collection id instead would have tied our directory names to whatever the server's URL segments happen to be, and two calendars ending in the same segment would collide. - The remote is storage_a with conflict_resolution keep a, so a concurrent server-side change wins over a local one calcalist has not yet reconciled. Nothing is lost: the next cycle sees the server's version and decides properly. - on_empty skip and on_delete skip on every pair, so a failed pull is not mirrored as a mass deletion. - Secrets stay out of the portable config: pimsync is given the same command calcalist would run to fetch them. - Google endpoints are omitted entirely, since pimsync can reach neither its REST API nor its CalDAV endpoint. doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and its parser does not always match its documentation, so asking it is the only reliable check — a parse rejection is reported as a failure, an unreachable server only as a warning. Found while testing against the real binary: pimsync 0.5.11 documents read_only as applying to every storage type but rejects it on a webcal storage, with only "Could not parse file" to go on. Dropped; it was redundant beside one_way. Verified end to end against a live Radicale server and an HTTP-served feed: the feed synced in, mirrored with a derived UID and provenance, and pushed to the server; repeated runs are no-ops and adding one feed event mirrors exactly one. 87 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:25:38 +03:00
/// Runs one pimsync cycle.
///
/// Deliberately one-shot: `pimsync daemon` would write the same vdirs the
/// reconciler is reading, so calcalist owns the scheduling instead.
/// A cycle is minutes of network work, and pimsync's per-item progress is the
/// only sign it is alive. Capturing that output left `sync` looking wedged for
/// the whole run and then printing nothing, so it goes straight to the terminal.
Drive pimsync for the CalDAV and WebCal legs Generates pimsync's configuration from calcalist.toml and invokes it one-shot either side of the reconcile step: pull first so the reconciler sees one consistent snapshot, push afterwards to publish what it decided. Its daemon mode stays unused because it would write the same vdirs the reconciler reads. - Endpoints are mapped with pimsync's collection blocks, matching a local vdir subdirectory to an exact remote path. Matching by collection id instead would have tied our directory names to whatever the server's URL segments happen to be, and two calendars ending in the same segment would collide. - The remote is storage_a with conflict_resolution keep a, so a concurrent server-side change wins over a local one calcalist has not yet reconciled. Nothing is lost: the next cycle sees the server's version and decides properly. - on_empty skip and on_delete skip on every pair, so a failed pull is not mirrored as a mass deletion. - Secrets stay out of the portable config: pimsync is given the same command calcalist would run to fetch them. - Google endpoints are omitted entirely, since pimsync can reach neither its REST API nor its CalDAV endpoint. doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and its parser does not always match its documentation, so asking it is the only reliable check — a parse rejection is reported as a failure, an unreachable server only as a warning. Found while testing against the real binary: pimsync 0.5.11 documents read_only as applying to every storage type but rejects it on a webcal storage, with only "Could not parse file" to go on. Dropped; it was redundant beside one_way. Verified end to end against a live Radicale server and an HTTP-served feed: the feed synced in, mirrored with a derived UID and provenance, and pushed to the server; repeated runs are no-ops and adding one feed event mirrors exactly one. 87 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:25:38 +03:00
pub fn sync(config_path: &Path) -> Result<(), PimsyncError> {
let status = invocation(config_path, "sync")
.stdout(progress_stream()?)
.stderr(Stdio::inherit())
.status()
.map_err(spawn_error)?;
if status.success() {
return Ok(());
}
// The detail has already been printed as it happened; repeating it here
// would only put a second copy under the report.
Err(PimsyncError::Command {
command: "sync".to_string(),
message: format!("exited with {status}"),
})
Drive pimsync for the CalDAV and WebCal legs Generates pimsync's configuration from calcalist.toml and invokes it one-shot either side of the reconcile step: pull first so the reconciler sees one consistent snapshot, push afterwards to publish what it decided. Its daemon mode stays unused because it would write the same vdirs the reconciler reads. - Endpoints are mapped with pimsync's collection blocks, matching a local vdir subdirectory to an exact remote path. Matching by collection id instead would have tied our directory names to whatever the server's URL segments happen to be, and two calendars ending in the same segment would collide. - The remote is storage_a with conflict_resolution keep a, so a concurrent server-side change wins over a local one calcalist has not yet reconciled. Nothing is lost: the next cycle sees the server's version and decides properly. - on_empty skip and on_delete skip on every pair, so a failed pull is not mirrored as a mass deletion. - Secrets stay out of the portable config: pimsync is given the same command calcalist would run to fetch them. - Google endpoints are omitted entirely, since pimsync can reach neither its REST API nor its CalDAV endpoint. doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and its parser does not always match its documentation, so asking it is the only reliable check — a parse rejection is reported as a failure, an unreachable server only as a warning. Found while testing against the real binary: pimsync 0.5.11 documents read_only as applying to every storage type but rejects it on a webcal storage, with only "Could not parse file" to go on. Dropped; it was redundant beside one_way. Verified end to end against a live Radicale server and an HTTP-served feed: the feed synced in, mirrored with a derived UID and provenance, and pushed to the server; repeated runs are no-ops and adding one feed event mirrors exactly one. 87 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:25:38 +03:00
}
/// Validates the generated configuration and the storages it names.
///
/// Captured rather than streamed: `doctor` reads the message to tell a config
/// it must fix from a server it cannot reach, and prints its own summary.
Drive pimsync for the CalDAV and WebCal legs Generates pimsync's configuration from calcalist.toml and invokes it one-shot either side of the reconcile step: pull first so the reconciler sees one consistent snapshot, push afterwards to publish what it decided. Its daemon mode stays unused because it would write the same vdirs the reconciler reads. - Endpoints are mapped with pimsync's collection blocks, matching a local vdir subdirectory to an exact remote path. Matching by collection id instead would have tied our directory names to whatever the server's URL segments happen to be, and two calendars ending in the same segment would collide. - The remote is storage_a with conflict_resolution keep a, so a concurrent server-side change wins over a local one calcalist has not yet reconciled. Nothing is lost: the next cycle sees the server's version and decides properly. - on_empty skip and on_delete skip on every pair, so a failed pull is not mirrored as a mass deletion. - Secrets stay out of the portable config: pimsync is given the same command calcalist would run to fetch them. - Google endpoints are omitted entirely, since pimsync can reach neither its REST API nor its CalDAV endpoint. doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and its parser does not always match its documentation, so asking it is the only reliable check — a parse rejection is reported as a failure, an unreachable server only as a warning. Found while testing against the real binary: pimsync 0.5.11 documents read_only as applying to every storage type but rejects it on a webcal storage, with only "Could not parse file" to go on. Dropped; it was redundant beside one_way. Verified end to end against a live Radicale server and an HTTP-served feed: the feed synced in, mirrored with a derived UID and provenance, and pushed to the server; repeated runs are no-ops and adding one feed event mirrors exactly one. 87 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:25:38 +03:00
pub fn check(config_path: &Path) -> Result<(), PimsyncError> {
let output = invocation(config_path, "check")
Drive pimsync for the CalDAV and WebCal legs Generates pimsync's configuration from calcalist.toml and invokes it one-shot either side of the reconcile step: pull first so the reconciler sees one consistent snapshot, push afterwards to publish what it decided. Its daemon mode stays unused because it would write the same vdirs the reconciler reads. - Endpoints are mapped with pimsync's collection blocks, matching a local vdir subdirectory to an exact remote path. Matching by collection id instead would have tied our directory names to whatever the server's URL segments happen to be, and two calendars ending in the same segment would collide. - The remote is storage_a with conflict_resolution keep a, so a concurrent server-side change wins over a local one calcalist has not yet reconciled. Nothing is lost: the next cycle sees the server's version and decides properly. - on_empty skip and on_delete skip on every pair, so a failed pull is not mirrored as a mass deletion. - Secrets stay out of the portable config: pimsync is given the same command calcalist would run to fetch them. - Google endpoints are omitted entirely, since pimsync can reach neither its REST API nor its CalDAV endpoint. doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and its parser does not always match its documentation, so asking it is the only reliable check — a parse rejection is reported as a failure, an unreachable server only as a warning. Found while testing against the real binary: pimsync 0.5.11 documents read_only as applying to every storage type but rejects it on a webcal storage, with only "Could not parse file" to go on. Dropped; it was redundant beside one_way. Verified end to end against a live Radicale server and an HTTP-served feed: the feed synced in, mirrored with a derived UID and provenance, and pushed to the server; repeated runs are no-ops and adding one feed event mirrors exactly one. 87 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:25:38 +03:00
.output()
.map_err(spawn_error)?;
Drive pimsync for the CalDAV and WebCal legs Generates pimsync's configuration from calcalist.toml and invokes it one-shot either side of the reconcile step: pull first so the reconciler sees one consistent snapshot, push afterwards to publish what it decided. Its daemon mode stays unused because it would write the same vdirs the reconciler reads. - Endpoints are mapped with pimsync's collection blocks, matching a local vdir subdirectory to an exact remote path. Matching by collection id instead would have tied our directory names to whatever the server's URL segments happen to be, and two calendars ending in the same segment would collide. - The remote is storage_a with conflict_resolution keep a, so a concurrent server-side change wins over a local one calcalist has not yet reconciled. Nothing is lost: the next cycle sees the server's version and decides properly. - on_empty skip and on_delete skip on every pair, so a failed pull is not mirrored as a mass deletion. - Secrets stay out of the portable config: pimsync is given the same command calcalist would run to fetch them. - Google endpoints are omitted entirely, since pimsync can reach neither its REST API nor its CalDAV endpoint. doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and its parser does not always match its documentation, so asking it is the only reliable check — a parse rejection is reported as a failure, an unreachable server only as a warning. Found while testing against the real binary: pimsync 0.5.11 documents read_only as applying to every storage type but rejects it on a webcal storage, with only "Could not parse file" to go on. Dropped; it was redundant beside one_way. Verified end to end against a live Radicale server and an HTTP-served feed: the feed synced in, mirrored with a derived UID and provenance, and pushed to the server; repeated runs are no-ops and adding one feed event mirrors exactly one. 87 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:25:38 +03:00
if output.status.success() {
return Ok(());
}
let message = diagnostics(&output);
Drive pimsync for the CalDAV and WebCal legs Generates pimsync's configuration from calcalist.toml and invokes it one-shot either side of the reconcile step: pull first so the reconciler sees one consistent snapshot, push afterwards to publish what it decided. Its daemon mode stays unused because it would write the same vdirs the reconciler reads. - Endpoints are mapped with pimsync's collection blocks, matching a local vdir subdirectory to an exact remote path. Matching by collection id instead would have tied our directory names to whatever the server's URL segments happen to be, and two calendars ending in the same segment would collide. - The remote is storage_a with conflict_resolution keep a, so a concurrent server-side change wins over a local one calcalist has not yet reconciled. Nothing is lost: the next cycle sees the server's version and decides properly. - on_empty skip and on_delete skip on every pair, so a failed pull is not mirrored as a mass deletion. - Secrets stay out of the portable config: pimsync is given the same command calcalist would run to fetch them. - Google endpoints are omitted entirely, since pimsync can reach neither its REST API nor its CalDAV endpoint. doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and its parser does not always match its documentation, so asking it is the only reliable check — a parse rejection is reported as a failure, an unreachable server only as a warning. Found while testing against the real binary: pimsync 0.5.11 documents read_only as applying to every storage type but rejects it on a webcal storage, with only "Could not parse file" to go on. Dropped; it was redundant beside one_way. Verified end to end against a live Radicale server and an HTTP-served feed: the feed synced in, mirrored with a derived UID and provenance, and pushed to the server; repeated runs are no-ops and adding one feed event mirrors exactly one. 87 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:25:38 +03:00
Err(PimsyncError::Command {
command: "check".to_string(),
Drive pimsync for the CalDAV and WebCal legs Generates pimsync's configuration from calcalist.toml and invokes it one-shot either side of the reconcile step: pull first so the reconciler sees one consistent snapshot, push afterwards to publish what it decided. Its daemon mode stays unused because it would write the same vdirs the reconciler reads. - Endpoints are mapped with pimsync's collection blocks, matching a local vdir subdirectory to an exact remote path. Matching by collection id instead would have tied our directory names to whatever the server's URL segments happen to be, and two calendars ending in the same segment would collide. - The remote is storage_a with conflict_resolution keep a, so a concurrent server-side change wins over a local one calcalist has not yet reconciled. Nothing is lost: the next cycle sees the server's version and decides properly. - on_empty skip and on_delete skip on every pair, so a failed pull is not mirrored as a mass deletion. - Secrets stay out of the portable config: pimsync is given the same command calcalist would run to fetch them. - Google endpoints are omitted entirely, since pimsync can reach neither its REST API nor its CalDAV endpoint. doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and its parser does not always match its documentation, so asking it is the only reliable check — a parse rejection is reported as a failure, an unreachable server only as a warning. Found while testing against the real binary: pimsync 0.5.11 documents read_only as applying to every storage type but rejects it on a webcal storage, with only "Could not parse file" to go on. Dropped; it was redundant beside one_way. Verified end to end against a live Radicale server and an HTTP-served feed: the feed synced in, mirrored with a derived UID and provenance, and pushed to the server; repeated runs are no-ops and adding one feed event mirrors exactly one. 87 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:25:38 +03:00
message: if message.is_empty() {
format!("exited with {}", output.status)
} else {
message
},
})
}
/// `pimsync -c <config> <command>`, with stdin closed so that a secret helper
/// reaching for a terminal fails outright rather than waiting on one forever.
fn invocation(config_path: &Path, command: &str) -> Command {
let mut process = Command::new(BINARY);
process
.arg("-c")
.arg(config_path)
.arg(command)
.stdin(Stdio::null());
process
}
/// pimsync's progress, routed to our stderr so that stdout carries only the
/// report and stays worth piping.
fn progress_stream() -> Result<Stdio, PimsyncError> {
io::stderr()
.as_fd()
.try_clone_to_owned()
.map(Stdio::from)
.map_err(PimsyncError::Spawn)
}
fn spawn_error(error: io::Error) -> PimsyncError {
if error.kind() == io::ErrorKind::NotFound {
PimsyncError::NotFound
} else {
PimsyncError::Spawn(error)
}
}
/// What a failed run had to say for itself.
///
/// Both streams are read: pimsync reports execution errors on stderr but puts
/// parse failures on stdout, and which one carries the reason is not knowable
/// from here.
fn diagnostics(output: &Output) -> String {
let streams = [&output.stderr, &output.stdout];
let said: Vec<String> = streams
.iter()
.map(|stream| String::from_utf8_lossy(stream).trim().to_string())
.filter(|text| !text.is_empty())
.collect();
said.join("\n")
}
#[cfg(test)]
mod tests {
use super::*;
Drive pimsync for the CalDAV and WebCal legs Generates pimsync's configuration from calcalist.toml and invokes it one-shot either side of the reconcile step: pull first so the reconciler sees one consistent snapshot, push afterwards to publish what it decided. Its daemon mode stays unused because it would write the same vdirs the reconciler reads. - Endpoints are mapped with pimsync's collection blocks, matching a local vdir subdirectory to an exact remote path. Matching by collection id instead would have tied our directory names to whatever the server's URL segments happen to be, and two calendars ending in the same segment would collide. - The remote is storage_a with conflict_resolution keep a, so a concurrent server-side change wins over a local one calcalist has not yet reconciled. Nothing is lost: the next cycle sees the server's version and decides properly. - on_empty skip and on_delete skip on every pair, so a failed pull is not mirrored as a mass deletion. - Secrets stay out of the portable config: pimsync is given the same command calcalist would run to fetch them. - Google endpoints are omitted entirely, since pimsync can reach neither its REST API nor its CalDAV endpoint. doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and its parser does not always match its documentation, so asking it is the only reliable check — a parse rejection is reported as a failure, an unreachable server only as a warning. Found while testing against the real binary: pimsync 0.5.11 documents read_only as applying to every storage type but rejects it on a webcal storage, with only "Could not parse file" to go on. Dropped; it was redundant beside one_way. Verified end to end against a live Radicale server and an HTTP-served feed: the feed synced in, mirrored with a derived UID and provenance, and pushed to the server; repeated runs are no-ops and adding one feed event mirrors exactly one. 87 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:25:38 +03:00
use crate::config::Config;
use std::path::Path;
const CONFIG: &str = r#"
version = 1
[[endpoint]]
id = "work"
type = "caldav"
url = "https://dav.example.com/dav/calendars/user/me/work/"
username = "me@example.com"
secret_command = "pass show example/caldav"
[[endpoint]]
id = "gcal"
type = "google"
calendar_id = "me@example.com"
client_id = "abc.apps.googleusercontent.com"
[[endpoint]]
id = "holidays"
type = "webcal"
url = "https://example.org/holidays.ics"
"#;
/// The text of one `storage`/`pair` block, so assertions do not accidentally
/// match a directive belonging to a different block.
fn block<'a>(text: &'a str, opening: &str) -> &'a str {
let start = text.find(opening).expect("block should be present");
let rest = &text[start..];
rest.find("\n}\n").map_or(rest, |end| &rest[..end])
}
fn generated() -> String {
let config: Config = toml::from_str(CONFIG).expect("config should parse");
Close the remaining M1 gaps Eight items were still open at the end of M1: two integration tests that had only been run by hand, and six known gaps. Recurrence overrides now reach Google. Google addresses an exception through the series rather than as an event of its own, so the master is sent first and each override is then matched to its instance by original start time and patched. Matching needs the two sides' spellings reduced to one key: iCalendar writes a zoned local time, Google an absolute offset. An override matching no occurrence is counted rather than forced — that means a stale RECURRENCE-ID left behind by an edited RRULE, and inventing an event for it would put something in the calendar the series does not contain. Reading one component apart from another needed a view `properties` cannot give: it flattens every VEVENT together, which is right for the UID a series shares and wrong for an override, whose SUMMARY and the master's are then indistinguishable. `Calendar::events` splits them. A TZID now travels with the VTIMEZONE that defines it, derived from the zone's own transition table as the yearly rule it implies. This changes the content hash of every zoned recurring event, so the first cycle after this re-pushes them. A Google authorisation is filed under the account it was granted for rather than the endpoint that asked for it, so two endpoints on one account no longer need a login each. The account is read from the primary calendar's id, which needs no scope beyond the calendar one already granted. Authorisations written by the previous scheme are still honoured, and move across at the next login. An unreachable Google endpoint no longer ends the cycle — one lapsed token used to stop the CalDAV side too. It is named, only the aggregates depending on it stand down, and the run exits non-zero so a partial cycle cannot pass for success. The CalDAV leg cannot be narrowed the same way: pimsync is one process covering every pair, so a failure does not say which pair it belongs to. `--dry-run` now pulls for real, into a throwaway copy of the local mirrors and through a pimsync configuration that only ever reads from a server. What it reports is measured against the calendars as they are now rather than against whatever the last real cycle left behind. `calcalist prune` reports local mirrors of endpoints the configuration no longer names, and removes them under --force. The integration tests run against a real Radicale server and a real iCal feed: convergence and idempotence, the dry run, prune, and the scheduling rule asserted on the bytes that actually reached the server. The plan asked for an SMTP sink for that last one; Radicale implements no RFC 6638 scheduling, so a quiet SMTP port would have proved nothing about the transform. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 13:49:38 +03:00
generate_for(&config, Path::new("/var/state/calcalist"), Direction::Both)
.expect("generation should succeed")
Drive pimsync for the CalDAV and WebCal legs Generates pimsync's configuration from calcalist.toml and invokes it one-shot either side of the reconcile step: pull first so the reconciler sees one consistent snapshot, push afterwards to publish what it decided. Its daemon mode stays unused because it would write the same vdirs the reconciler reads. - Endpoints are mapped with pimsync's collection blocks, matching a local vdir subdirectory to an exact remote path. Matching by collection id instead would have tied our directory names to whatever the server's URL segments happen to be, and two calendars ending in the same segment would collide. - The remote is storage_a with conflict_resolution keep a, so a concurrent server-side change wins over a local one calcalist has not yet reconciled. Nothing is lost: the next cycle sees the server's version and decides properly. - on_empty skip and on_delete skip on every pair, so a failed pull is not mirrored as a mass deletion. - Secrets stay out of the portable config: pimsync is given the same command calcalist would run to fetch them. - Google endpoints are omitted entirely, since pimsync can reach neither its REST API nor its CalDAV endpoint. doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and its parser does not always match its documentation, so asking it is the only reliable check — a parse rejection is reported as a failure, an unreachable server only as a warning. Found while testing against the real binary: pimsync 0.5.11 documents read_only as applying to every storage type but rejects it on a webcal storage, with only "Could not parse file" to go on. Dropped; it was redundant beside one_way. Verified end to end against a live Radicale server and an HTTP-served feed: the feed synced in, mirrored with a derived UID and provenance, and pushed to the server; repeated runs are no-ops and adding one feed event mirrors exactly one. 87 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:25:38 +03:00
}
#[test]
fn declares_one_local_storage_for_every_pair() {
let text = generated();
assert_eq!(text.matches("storage local {").count(), 1);
assert!(text.contains("path \"/var/state/calcalist/vdir\""));
assert!(text.contains("status_path \"/var/state/calcalist/pimsync-status\""));
}
/// pimsync cannot authenticate to Google at all, so it must not be asked to.
#[test]
fn google_endpoints_are_left_out_entirely() {
let text = generated();
assert!(!text.contains("gcal"), "google endpoint leaked into {text}");
}
#[test]
fn a_caldav_endpoint_becomes_a_bidirectional_pair() {
let text = generated();
assert!(text.contains("storage work_remote {"));
assert!(text.contains("type caldav"));
// The container is enumerated, and the calendar picked out by exact path.
assert!(text.contains("url \"https://dav.example.com/dav/calendars/user/me/\""));
assert!(text.contains("href_a \"/dav/calendars/user/me/work/\""));
assert!(text.contains("id_b work"));
assert!(
!block(&text, "pair work {").contains("one_way"),
"caldav pairs must stay bidirectional"
);
}
/// Secrets stay out of the portable config; pimsync fetches them the same way.
#[test]
fn secrets_are_fetched_by_command_not_embedded() {
let text = generated();
assert!(text.contains("password {"));
assert!(text.contains("shell pass show example/caldav"));
}
#[test]
fn a_feed_becomes_a_one_way_read_only_pair() {
let text = generated();
assert!(text.contains("type webcal"));
assert!(text.contains("one_way"));
// pimsync 0.5.11 rejects `read_only` on a webcal storage despite
// documenting it for all storage types.
assert!(!block(&text, "storage holidays_remote {").contains("read_only"));
assert!(text.contains("collection_id holidays"));
}
/// Both guards against a failed pull being mirrored as a mass deletion.
#[test]
fn every_pair_carries_the_deletion_guards() {
let text = generated();
let pairs = text.matches("\npair ").count();
assert_eq!(pairs, 2);
assert_eq!(text.matches("on_empty skip").count(), pairs);
assert_eq!(text.matches("on_delete skip").count(), pairs);
}
Add Google OAuth, and let feed URLs come from a secret command Google requires OAuth for calendar access; app passwords stopped working for CalDAV, CardDAV and IMAP in March 2025, so there is no simpler path to offer. - Authorisation code flow over a loopback redirect, which is what Google supports for desktop clients now the copy-paste flow is gone, with PKCE so an intercepted code is useless without the verifier. Only the refresh token is persisted, 0600, in the state directory. - An expired grant is reported as itself: a consent screen still in Testing has its refresh tokens expired after 7 days, and "run calcalist google login" is more use than Google's bare invalid_grant. - doctor reports whether each Google endpoint is still authorised, since an installation that worked last week can stop with nothing having changed here. A webcal URL may now come from a command instead of the config. Google's secret iCal address grants read access to a whole calendar to anyone holding it, so writing it into a file described as portable and secret-free was a contradiction. Fixed a serious defect in the first draft of this module: random_token used fs::read on /dev/urandom, which reads to end of file. /dev/urandom has no end, so it allocated until the machine ran out of memory — it took the editor down with it. It now reads exactly 32 bytes, and a randomness failure is fatal rather than falling back to the clock, since a guessable state or PKCE verifier defeats the point of having them. Verified end to end against a live Posteo CalDAV calendar: pimsync validated the generated config against the real server, 58 events from a public feed were mirrored and pushed, and a second run was a no-op. 97 tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 12:40:47 +03:00
/// A feed whose URL is itself a credential — Google's secret iCal address —
/// must be fetched by command, never written into either config file.
#[test]
fn a_secret_feed_url_is_fetched_by_command() {
let config: Config = toml::from_str(
r#"
version = 1
[[endpoint]]
id = "gcal-feed"
type = "webcal"
url_command = "secret-tool lookup service calcalist account gcal-ics"
"#,
)
.expect("config should parse");
Close the remaining M1 gaps Eight items were still open at the end of M1: two integration tests that had only been run by hand, and six known gaps. Recurrence overrides now reach Google. Google addresses an exception through the series rather than as an event of its own, so the master is sent first and each override is then matched to its instance by original start time and patched. Matching needs the two sides' spellings reduced to one key: iCalendar writes a zoned local time, Google an absolute offset. An override matching no occurrence is counted rather than forced — that means a stale RECURRENCE-ID left behind by an edited RRULE, and inventing an event for it would put something in the calendar the series does not contain. Reading one component apart from another needed a view `properties` cannot give: it flattens every VEVENT together, which is right for the UID a series shares and wrong for an override, whose SUMMARY and the master's are then indistinguishable. `Calendar::events` splits them. A TZID now travels with the VTIMEZONE that defines it, derived from the zone's own transition table as the yearly rule it implies. This changes the content hash of every zoned recurring event, so the first cycle after this re-pushes them. A Google authorisation is filed under the account it was granted for rather than the endpoint that asked for it, so two endpoints on one account no longer need a login each. The account is read from the primary calendar's id, which needs no scope beyond the calendar one already granted. Authorisations written by the previous scheme are still honoured, and move across at the next login. An unreachable Google endpoint no longer ends the cycle — one lapsed token used to stop the CalDAV side too. It is named, only the aggregates depending on it stand down, and the run exits non-zero so a partial cycle cannot pass for success. The CalDAV leg cannot be narrowed the same way: pimsync is one process covering every pair, so a failure does not say which pair it belongs to. `--dry-run` now pulls for real, into a throwaway copy of the local mirrors and through a pimsync configuration that only ever reads from a server. What it reports is measured against the calendars as they are now rather than against whatever the last real cycle left behind. `calcalist prune` reports local mirrors of endpoints the configuration no longer names, and removes them under --force. The integration tests run against a real Radicale server and a real iCal feed: convergence and idempotence, the dry run, prune, and the scheduling rule asserted on the bytes that actually reached the server. The plan asked for an SMTP sink for that last one; Radicale implements no RFC 6638 scheduling, so a quiet SMTP port would have proved nothing about the transform. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 13:49:38 +03:00
let text = generate_for(&config, Path::new("/var/state/calcalist"), Direction::Both)
.expect("generate");
Add Google OAuth, and let feed URLs come from a secret command Google requires OAuth for calendar access; app passwords stopped working for CalDAV, CardDAV and IMAP in March 2025, so there is no simpler path to offer. - Authorisation code flow over a loopback redirect, which is what Google supports for desktop clients now the copy-paste flow is gone, with PKCE so an intercepted code is useless without the verifier. Only the refresh token is persisted, 0600, in the state directory. - An expired grant is reported as itself: a consent screen still in Testing has its refresh tokens expired after 7 days, and "run calcalist google login" is more use than Google's bare invalid_grant. - doctor reports whether each Google endpoint is still authorised, since an installation that worked last week can stop with nothing having changed here. A webcal URL may now come from a command instead of the config. Google's secret iCal address grants read access to a whole calendar to anyone holding it, so writing it into a file described as portable and secret-free was a contradiction. Fixed a serious defect in the first draft of this module: random_token used fs::read on /dev/urandom, which reads to end of file. /dev/urandom has no end, so it allocated until the machine ran out of memory — it took the editor down with it. It now reads exactly 32 bytes, and a randomness failure is fatal rather than falling back to the clock, since a guessable state or PKCE verifier defeats the point of having them. Verified end to end against a live Posteo CalDAV calendar: pimsync validated the generated config against the real server, 58 events from a public feed were mirrored and pushed, and a second run was a no-op. 97 tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 12:40:47 +03:00
assert!(text.contains("url {"));
assert!(text.contains("shell secret-tool lookup service calcalist account gcal-ics"));
assert!(!text.contains("https://"), "no literal URL should appear");
}
Drive pimsync for the CalDAV and WebCal legs Generates pimsync's configuration from calcalist.toml and invokes it one-shot either side of the reconcile step: pull first so the reconciler sees one consistent snapshot, push afterwards to publish what it decided. Its daemon mode stays unused because it would write the same vdirs the reconciler reads. - Endpoints are mapped with pimsync's collection blocks, matching a local vdir subdirectory to an exact remote path. Matching by collection id instead would have tied our directory names to whatever the server's URL segments happen to be, and two calendars ending in the same segment would collide. - The remote is storage_a with conflict_resolution keep a, so a concurrent server-side change wins over a local one calcalist has not yet reconciled. Nothing is lost: the next cycle sees the server's version and decides properly. - on_empty skip and on_delete skip on every pair, so a failed pull is not mirrored as a mass deletion. - Secrets stay out of the portable config: pimsync is given the same command calcalist would run to fetch them. - Google endpoints are omitted entirely, since pimsync can reach neither its REST API nor its CalDAV endpoint. doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and its parser does not always match its documentation, so asking it is the only reliable check — a parse rejection is reported as a failure, an unreachable server only as a warning. Found while testing against the real binary: pimsync 0.5.11 documents read_only as applying to every storage type but rejects it on a webcal storage, with only "Could not parse file" to go on. Dropped; it was redundant beside one_way. Verified end to end against a live Radicale server and an HTTP-served feed: the feed synced in, mirrored with a derived UID and provenance, and pushed to the server; repeated runs are no-ops and adding one feed event mirrors exactly one. 87 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:25:38 +03:00
#[test]
fn splits_a_calendar_url_into_container_and_path() {
assert_eq!(
split_collection_url("https://dav.example.com/dav/calendars/user/me/work/"),
Some((
"https://dav.example.com/dav/calendars/user/me/".into(),
"/dav/calendars/user/me/work/".into()
))
);
// A trailing slash is optional.
assert_eq!(
split_collection_url("https://dav.example.com/calendars/work"),
Some((
"https://dav.example.com/calendars/".into(),
"/calendars/work/".into()
))
);
// A single segment leaves the root as the container.
assert_eq!(
split_collection_url("https://dav.example.com/work/"),
Some(("https://dav.example.com/".into(), "/work/".into()))
);
}
/// A bare server root names no calendar, so it must be rejected rather than
/// guessed at.
#[test]
fn a_url_without_a_collection_is_refused() {
assert_eq!(split_collection_url("https://dav.example.com/"), None);
assert_eq!(split_collection_url("https://dav.example.com"), None);
}
#[test]
fn parses_the_version_line() {
assert_eq!(
Version::parse("pimsync 0.5.11"),
Some(Version {
major: 0,
minor: 5,
patch: 11
})
);
}
#[test]
fn tolerates_a_two_component_version() {
assert_eq!(
Version::parse("pimsync 1.2"),
Some(Version {
major: 1,
minor: 2,
patch: 0
})
);
}
/// pimsync built from a checkout rather than a tag reports `git describe`'s
/// suffix, which must not swallow the patch number.
#[test]
fn tolerates_a_describe_suffix() {
assert_eq!(
Version::parse("pimsync 0.6.0-1-ga08ee3c"),
Some(Version {
major: 0,
minor: 6,
patch: 0
})
);
}
#[test]
fn rejects_unparseable_output() {
assert_eq!(Version::parse("pimsync"), None);
assert_eq!(Version::parse("pimsync unreleased"), None);
}
#[test]
fn the_read_series_are_accepted() {
let floor = Version {
major: 0,
minor: 5,
patch: 11,
};
let read = Version {
major: 0,
minor: 6,
patch: 0,
};
let unread = Version {
major: 0,
minor: 7,
patch: 0,
};
assert!(floor.is_supported());
assert!(read.is_supported());
assert!(!unread.is_supported());
}
Finish M2: run lock, systemd units, discover; drop the web UI Designing the configuration UI in full made the case against building it. Its audience would be people who find TOML hard, but with bring-your-own OAuth client settled, every user must first create a Google Cloud project, configure a consent screen and put a secret in a keyring — a far higher bar than editing thirty lines of config. Anyone who clears it can edit the file; anyone who cannot never reaches the file. Against that stood three dependencies, five modules, an auth.rs refactor and a security surface guarding something that reads the config and touches the keyring. `doctor` and `status` had already absorbed most of what it was for. The reasoning is recorded in TODO.md and SPECS.md rather than left as an apparent oversight. The run lock is not a UI feature and closes a gap that already existed: nothing stopped a timer firing into a hand-run cycle, and two cycles interleaving writes over the same vdirs is what the design otherwise avoids. flock is used rather than a pid file because the kernel releases it however the process ends, so a crash cannot leave a lock to clear by hand — which also means a lock we failed to take is held by a live process, so the pid in it is worth reporting. The one idea worth keeping from the UI design was collection discovery, which needed no web layer. `calcalist discover` prints a ready-to-paste endpoint block per calendar a server offers, removing the most error-prone field in the config. pimsync's discovery output is undocumented, so the format was established against a real server first. Two things it teaches: everything arrives on stdout including failures, and a pair has two storages, so pimsync reports the scratch vdir's contents too — parsing anchors on the heading naming the server, or a probe directory's leftovers would be offered as the user's calendars. Verified against Posteo as well as Radicale: all four calendars found, the first matching the URL already configured. Also fixes a real defect in the test harness rather than its symptom. Ports were chosen by binding one and letting go, so two tests could pick the same number — and the loser's readiness check then succeeded against the winner's server, silently sharing it. Startup now confirms the child we spawned is the one alive, retries on another port if not, and waits for a real HTTP response rather than an open socket. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 16:17:34 +03:00
#[test]
fn a_patch_older_than_the_floor_is_rejected() {
// discover's output format changed between 0.5.9 and 0.5.11; older
// patches within the otherwise-supported minor series are refused.
let too_old = Version {
major: 0,
minor: 5,
patch: 9,
};
assert!(!too_old.is_supported());
}
Finish M2: run lock, systemd units, discover; drop the web UI Designing the configuration UI in full made the case against building it. Its audience would be people who find TOML hard, but with bring-your-own OAuth client settled, every user must first create a Google Cloud project, configure a consent screen and put a secret in a keyring — a far higher bar than editing thirty lines of config. Anyone who clears it can edit the file; anyone who cannot never reaches the file. Against that stood three dependencies, five modules, an auth.rs refactor and a security surface guarding something that reads the config and touches the keyring. `doctor` and `status` had already absorbed most of what it was for. The reasoning is recorded in TODO.md and SPECS.md rather than left as an apparent oversight. The run lock is not a UI feature and closes a gap that already existed: nothing stopped a timer firing into a hand-run cycle, and two cycles interleaving writes over the same vdirs is what the design otherwise avoids. flock is used rather than a pid file because the kernel releases it however the process ends, so a crash cannot leave a lock to clear by hand — which also means a lock we failed to take is held by a live process, so the pid in it is worth reporting. The one idea worth keeping from the UI design was collection discovery, which needed no web layer. `calcalist discover` prints a ready-to-paste endpoint block per calendar a server offers, removing the most error-prone field in the config. pimsync's discovery output is undocumented, so the format was established against a real server first. Two things it teaches: everything arrives on stdout including failures, and a pair has two storages, so pimsync reports the scratch vdir's contents too — parsing anchors on the heading naming the server, or a probe directory's leftovers would be offered as the user's calendars. Verified against Posteo as well as Radicale: all four calendars found, the first matching the URL already configured. Also fixes a real defect in the test harness rather than its symptom. Ports were chosen by binding one and letting go, so two tests could pick the same number — and the loser's readiness check then succeeded against the winner's server, silently sharing it. Startup now confirms the child we spawned is the one alive, retries on another port if not, and waits for a real HTTP response rather than an open socket. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 16:17:34 +03:00
/// Established against a real server: everything arrives on stdout, the
/// headings are colour-coded even through a pipe, and both storages of the
/// pair are reported.
const DISCOVERY: &str = "\u{1b}[1m\u{1b}[32m==>\u{1b}[0m Discovering collections in local\u{2026}\n\
Found leftover-one\n\
Found leftover-two\n\
\u{1b}[1m\u{1b}[32m==>\u{1b}[0m Discovering collections in probe_remote\u{2026}\n\
Found /calendars/me/work/\n\
Found /calendars/me/Private Stuff/\n";
/// The trap: pimsync reports the scratch directory's contents too, and
/// offering those as the user's calendars would be nonsense.
#[test]
fn only_the_servers_collections_are_taken() {
let found = parse_discovery(DISCOVERY);
assert_eq!(
found.iter().map(|c| c.href.as_str()).collect::<Vec<_>>(),
vec!["/calendars/me/work/", "/calendars/me/Private Stuff/"]
);
}
#[test]
fn an_id_is_taken_from_the_last_segment() {
let found = parse_discovery(DISCOVERY);
assert_eq!(found[0].id, "work");
// Spaces and capitals cannot appear in an id used as a routing marker.
assert_eq!(found[1].id, "private-stuff");
}
#[test]
fn a_server_offering_nothing_yields_nothing() {
let empty = "\u{1b}[1m==>\u{1b}[0m Discovering collections in probe_remote\u{2026}\n";
assert!(parse_discovery(empty).is_empty());
}
/// A failure is reported in place of the collections, so nothing is found.
#[test]
fn an_unreachable_server_yields_nothing() {
let failed = "\u{1b}[1m==>\u{1b}[0m Discovering collections in probe_remote\u{2026}\n\
input/output error: client error executing request: client error (Connect)\n";
assert!(parse_discovery(failed).is_empty());
}
#[test]
fn colour_is_stripped_from_headings() {
assert_eq!(
strip_ansi("\u{1b}[1m\u{1b}[32m==>\u{1b}[0m Discovering"),
"==> Discovering"
);
}
/// The probe must never be mistaken for the real configuration, and must
/// name the storage the parser anchors on.
#[test]
fn the_probe_config_is_separate_and_parseable() {
let text = probe_config(
Path::new("/var/state/calcalist"),
Path::new("/var/state/calcalist/discover-vdir"),
"https://dav.example.com/calendars/me/",
"me@example.com",
Some("secret-tool lookup service dav"),
);
assert!(
text.contains(&format!("storage {PROBE_REMOTE} {{")),
"{text}"
);
assert!(text.contains("collections from a"), "{text}");
assert!(text.contains("discovery collections"), "{text}");
assert!(
text.contains("shell secret-tool lookup service dav"),
"{text}"
);
// Its status path is its own, so a probe cannot disturb a real sync.
assert!(text.contains("discover-status"), "{text}");
}
#[test]
fn a_server_needing_no_password_gets_no_password_directive() {
let text = probe_config(
Path::new("/var/state/calcalist"),
Path::new("/tmp/scratch"),
"https://dav.example.com/calendars/me/",
"me",
None,
);
assert!(!text.contains("password"), "{text}");
}
}