Drive pimsync for the CalDAV and WebCal legs

Generates pimsync's configuration from calcalist.toml and invokes it one-shot
either side of the reconcile step: pull first so the reconciler sees one
consistent snapshot, push afterwards to publish what it decided. Its daemon
mode stays unused because it would write the same vdirs the reconciler reads.

- Endpoints are mapped with pimsync's collection blocks, matching a local vdir
  subdirectory to an exact remote path. Matching by collection id instead would
  have tied our directory names to whatever the server's URL segments happen to
  be, and two calendars ending in the same segment would collide.
- The remote is storage_a with conflict_resolution keep a, so a concurrent
  server-side change wins over a local one calcalist has not yet reconciled.
  Nothing is lost: the next cycle sees the server's version and decides properly.
- on_empty skip and on_delete skip on every pair, so a failed pull is not
  mirrored as a mass deletion.
- Secrets stay out of the portable config: pimsync is given the same command
  calcalist would run to fetch them.
- Google endpoints are omitted entirely, since pimsync can reach neither its REST
  API nor its CalDAV endpoint.

doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and
its parser does not always match its documentation, so asking it is the only
reliable check — a parse rejection is reported as a failure, an unreachable
server only as a warning.

Found while testing against the real binary: pimsync 0.5.11 documents read_only
as applying to every storage type but rejects it on a webcal storage, with only
"Could not parse file" to go on. Dropped; it was redundant beside one_way.

Verified end to end against a live Radicale server and an HTTP-served feed: the
feed synced in, mirrored with a derived UID and provenance, and pushed to the
server; repeated runs are no-ops and adding one feed event mirrors exactly one.

87 tests.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

TODO.md

@@ -26,8 +26,10 @@ Core modules:
       original plan, which folded these into `reconcile`)
 - [x] `reconcile.rs` — the aggregation engine; pure, no I/O
 - [x] `sync.rs` — one cycle over the local vdirs, applying what `reconcile` decides
-- [ ] `pimsync.rs` — generate `pimsync.conf` (with `on_empty skip` and `on_delete skip`),
-      drive one-shot `pimsync sync`
+- [x] `pimsync.rs` — generate `pimsync.conf` (with `on_empty skip` and `on_delete skip`),
+      drive one-shot `pimsync sync` bracketing the reconcile step
+- [x] `doctor` asks `pimsync check` to validate the generated config, since pimsync's
+      parser does not always match its documentation
 - [ ] `google/auth.rs`, `google/api.rs`, `google/convert.rs`
 - [x] Reintroduce `SchedulingSuppression` in `config.rs` (removed in M0 as dead code)
 

src/doctor.rs

@@ -6,7 +6,7 @@ use std::path::Path;
 
 use crate::config::{Config, ConfigError};
 use crate::paths;
-use crate::pimsync;
+use crate::pimsync::{self, PimsyncError};
 
 #[derive(Debug, PartialEq, Eq)]
 pub enum Outcome {
@@ -51,11 +51,41 @@ impl fmt::Display for Check {
 
 /// Runs every check. Returns the checks in the order they should be printed.
 pub fn run(config_path: Option<&Path>) -> Vec<Check> {
-    vec![
-        check_pimsync(),
-        check_state_dir(),
-        check_config(config_path),
-    ]
+    let loaded = Config::load(config_path);
+    let mut checks = vec![check_pimsync(), check_state_dir()];
+    if let Ok((config, _)) = &loaded {
+        checks.push(check_generated_config(config));
+    }
+    checks.push(check_config(loaded));
+    checks
+}
+
+/// Asks pimsync to validate the configuration calcalist generates for it.
+///
+/// pimsync is pre-1.0 and its parser does not always match its documentation, so
+/// the only reliable way to know it accepts what we write is to ask it.
+fn check_generated_config(config: &Config) -> Check {
+    let outcome = match paths::state_dir() {
+        Err(error) => Outcome::Fail(error.to_string()),
+        Ok(state_dir) => match pimsync::write_config(config, &state_dir) {
+            Err(error) => Outcome::Fail(error.to_string()),
+            Ok(path) => match pimsync::check(&path) {
+                Ok(()) => Outcome::Ok(path.display().to_string()),
+                // A parse failure is ours to fix. Anything else is usually the
+                // server being unreachable, which says nothing about the config.
+                Err(PimsyncError::Command { message, .. })
+                    if message.contains("Could not parse") =>
+                {
+                    Outcome::Fail(format!("pimsync rejected {}: {message}", path.display()))
+                }
+                Err(error) => Outcome::Warn(format!("could not be verified: {error}")),
+            },
+        },
+    };
+    Check {
+        name: "pimsync config",
+        outcome,
+    }
 }
 
 pub fn any_failed(checks: &[Check]) -> bool {
@@ -94,8 +124,8 @@ fn check_state_dir() -> Check {
     }
 }
 
-fn check_config(path: Option<&Path>) -> Check {
-    let outcome = match Config::load(path) {
+fn check_config(loaded: Result<(Config, std::path::PathBuf), ConfigError>) -> Check {
+    let outcome = match loaded {
         Ok((config, path)) => Outcome::Ok(format!(
             "{} ({} endpoint(s), {} aggregate(s))",
             path.display(),

src/pimsync.rs

@@ -5,11 +5,15 @@
 //! same files, so scheduling belongs to calcalist.
 
 use std::fmt;
+use std::fs;
 use std::io;
+use std::path::{Path, PathBuf};
 use std::process::Command;
 
 use thiserror::Error;
 
+use crate::config::{Config, Endpoint, EndpointKind};
+
 pub const BINARY: &str = "pimsync";
 
 /// The `0.5.x` series this build generates configuration for. pimsync is
@@ -58,6 +62,18 @@ pub enum PimsyncError {
     Failed(String),
     #[error("could not read a version out of `{0}`")]
     Unparseable(String),
+    #[error("could not write {path}: {source}")]
+    Write {
+        path: std::path::PathBuf,
+        #[source]
+        source: io::Error,
+    },
+    #[error("`{BINARY} {command}` failed: {message}")]
+    Command { command: String, message: String },
+    #[error(
+        "endpoint `{endpoint}` has url `{url}`, which names no calendar collection; it must point at the calendar itself, not at the server root"
+    )]
+    UrlHasNoCollection { endpoint: String, url: String },
 }
 
 /// Asks the installed pimsync for its version.
@@ -82,10 +98,317 @@ pub fn probe() -> Result<Version, PimsyncError> {
     Version::parse(line).ok_or_else(|| PimsyncError::Unparseable(line.to_string()))
 }
 
+/// Name of the single local storage every pair mirrors into. pimsync allows one
+/// storage to take part in several pairs, so one declaration serves all of them.
+const LOCAL_STORAGE: &str = "local";
+
+pub fn config_path(state_dir: &Path) -> PathBuf {
+    state_dir.join("pimsync.conf")
+}
+
+fn status_path(state_dir: &Path) -> PathBuf {
+    state_dir.join("pimsync-status")
+}
+
+/// Builds a pimsync configuration covering every CalDAV and WebCal endpoint.
+///
+/// Google endpoints are absent by design: pimsync has no REST storage, and its
+/// CalDAV storage speaks only HTTP Basic auth, which Google's endpoint has
+/// rejected since March 2025. calcalist syncs those itself.
+pub fn generate(config: &Config, state_dir: &Path) -> Result<String, PimsyncError> {
+    let mut out = String::new();
+    out.push_str("# Generated by calcalist. Edits here are overwritten on every sync.\n\n");
+    out.push_str(&format!(
+        "status_path {}\n\n",
+        quote(&status_path(state_dir).display().to_string())
+    ));
+    out.push_str(&format!(
+        "storage {LOCAL_STORAGE} {{\n\ttype vdir/icalendar\n\tpath {}\n\tfileext ics\n}}\n",
+        quote(&crate::sync::vdir_root(state_dir).display().to_string())
+    ));
+
+    for endpoint in &config.endpoints {
+        match &endpoint.kind {
+            EndpointKind::Caldav { .. } => out.push_str(&caldav_pair(endpoint)?),
+            EndpointKind::Webcal { url } => out.push_str(&webcal_pair(endpoint, url)),
+            // Handled by calcalist's own Google module.
+            EndpointKind::Google { .. } => {}
+        }
+    }
+    Ok(out)
+}
+
+fn caldav_pair(endpoint: &Endpoint) -> Result<String, PimsyncError> {
+    let EndpointKind::Caldav {
+        url,
+        username,
+        secret_command,
+    } = &endpoint.kind
+    else {
+        unreachable!("caller matched on Caldav")
+    };
+    let (base, href) =
+        split_collection_url(url).ok_or_else(|| PimsyncError::UrlHasNoCollection {
+            endpoint: endpoint.id.clone(),
+            url: url.clone(),
+        })?;
+
+    let id = &endpoint.id;
+    let mut block = format!(
+        "\nstorage {id}_remote {{\n\ttype caldav\n\turl {}\n\tdiscovery collections\n\tusername {}\n",
+        quote(&base),
+        quote(username)
+    );
+    if let Some(command) = secret_command {
+        // Secrets never live in the portable config, so pimsync runs the same
+        // command calcalist would to fetch them.
+        block.push_str(&format!("\tpassword {{\n\t\tshell {command}\n\t}}\n"));
+    }
+    block.push_str("}\n");
+
+    // The remote is storage_a, so a concurrent server-side change wins over a
+    // local one calcalist has not yet reconciled. Nothing is lost: the next
+    // cycle sees the server's version and decides properly.
+    block.push_str(&format!(
+        "\npair {id} {{\n\tstorage_a {id}_remote\n\tstorage_b {LOCAL_STORAGE}\n\
+         \tcollection {{\n\t\talias {id}\n\t\thref_a {}\n\t\tid_b {id}\n\t}}\n\
+         \tconflict_resolution keep a\n\ton_empty skip\n\ton_delete skip\n}}\n",
+        quote(&href)
+    ));
+    Ok(block)
+}
+
+fn webcal_pair(endpoint: &Endpoint, url: &str) -> String {
+    let id = &endpoint.id;
+    // No `read_only` directive here: pimsync 0.5.11 documents it as applying to
+    // every storage type, but rejects it on a webcal storage with a bare "Could
+    // not parse file". It is redundant regardless — webcal is read-only by type,
+    // and `one_way` already fixes the direction.
+    format!(
+        "\nstorage {id}_remote {{\n\ttype webcal\n\turl {}\n\tcollection_id {id}\n}}\n\
+         \npair {id} {{\n\tstorage_a {id}_remote\n\tstorage_b {LOCAL_STORAGE}\n\
+         \tcollection {{\n\t\talias {id}\n\t\tid_a {id}\n\t\tid_b {id}\n\t}}\n\
+         \tone_way\n\ton_empty skip\n\ton_delete skip\n}}\n",
+        quote(url)
+    )
+}
+
+/// Splits a calendar URL into the container to enumerate and the collection's
+/// own path.
+///
+/// pimsync identifies a collection either by an id taken from a URL segment or
+/// by its exact path. The path is used here, so a calendar whose last segment
+/// happens to match another's cannot be confused for it.
+fn split_collection_url(url: &str) -> Option<(String, String)> {
+    let authority_start = url.find("://")? + 3;
+    let path_start = authority_start + url[authority_start..].find('/')?;
+    let segments: Vec<&str> = url[path_start..]
+        .split('/')
+        .filter(|segment| !segment.is_empty())
+        .collect();
+    let (_, parents) = segments.split_last()?;
+
+    let href = format!("/{}/", segments.join("/"));
+    let base = match parents {
+        [] => format!("{}/", &url[..path_start]),
+        parents => format!("{}/{}/", &url[..path_start], parents.join("/")),
+    };
+    Some((base, href))
+}
+
+/// pimsync accepts quoted values, which keeps paths with spaces intact.
+fn quote(value: &str) -> String {
+    format!("\"{}\"", value.replace('\\', "\\\\").replace('"', "\\\""))
+}
+
+/// Writes the generated configuration, returning its path.
+pub fn write_config(config: &Config, state_dir: &Path) -> Result<PathBuf, PimsyncError> {
+    let path = config_path(state_dir);
+    let text = generate(config, state_dir)?;
+    fs::create_dir_all(status_path(state_dir)).map_err(|source| PimsyncError::Write {
+        path: status_path(state_dir),
+        source,
+    })?;
+    fs::write(&path, text).map_err(|source| PimsyncError::Write {
+        path: path.clone(),
+        source,
+    })?;
+    Ok(path)
+}
+
+/// Runs one pimsync cycle.
+///
+/// Deliberately one-shot: `pimsync daemon` would write the same vdirs the
+/// reconciler is reading, so calcalist owns the scheduling instead.
+pub fn sync(config_path: &Path) -> Result<(), PimsyncError> {
+    run(config_path, "sync")
+}
+
+/// Validates the generated configuration and the storages it names.
+pub fn check(config_path: &Path) -> Result<(), PimsyncError> {
+    run(config_path, "check")
+}
+
+fn run(config_path: &Path, command: &str) -> Result<(), PimsyncError> {
+    let output = Command::new(BINARY)
+        .arg("-c")
+        .arg(config_path)
+        .arg(command)
+        .output()
+        .map_err(|error| {
+            if error.kind() == io::ErrorKind::NotFound {
+                PimsyncError::NotFound
+            } else {
+                PimsyncError::Spawn(error)
+            }
+        })?;
+    if output.status.success() {
+        return Ok(());
+    }
+    let message = String::from_utf8_lossy(&output.stderr).trim().to_string();
+    Err(PimsyncError::Command {
+        command: command.to_string(),
+        message: if message.is_empty() {
+            format!("exited with {}", output.status)
+        } else {
+            message
+        },
+    })
+}
+
 #[cfg(test)]
 mod tests {
     use super::*;
 
+    use crate::config::Config;
+    use std::path::Path;
+
+    const CONFIG: &str = r#"
+version = 1
+
+[[endpoint]]
+id = "work"
+type = "caldav"
+url = "https://dav.example.com/dav/calendars/user/me/work/"
+username = "me@example.com"
+secret_command = "pass show example/caldav"
+
+[[endpoint]]
+id = "gcal"
+type = "google"
+calendar_id = "me@example.com"
+client_id = "abc.apps.googleusercontent.com"
+
+[[endpoint]]
+id = "holidays"
+type = "webcal"
+url = "https://example.org/holidays.ics"
+"#;
+
+    /// The text of one `storage`/`pair` block, so assertions do not accidentally
+    /// match a directive belonging to a different block.
+    fn block<'a>(text: &'a str, opening: &str) -> &'a str {
+        let start = text.find(opening).expect("block should be present");
+        let rest = &text[start..];
+        rest.find("\n}\n").map_or(rest, |end| &rest[..end])
+    }
+
+    fn generated() -> String {
+        let config: Config = toml::from_str(CONFIG).expect("config should parse");
+        generate(&config, Path::new("/var/state/calcalist")).expect("generation should succeed")
+    }
+
+    #[test]
+    fn declares_one_local_storage_for_every_pair() {
+        let text = generated();
+        assert_eq!(text.matches("storage local {").count(), 1);
+        assert!(text.contains("path \"/var/state/calcalist/vdir\""));
+        assert!(text.contains("status_path \"/var/state/calcalist/pimsync-status\""));
+    }
+
+    /// pimsync cannot authenticate to Google at all, so it must not be asked to.
+    #[test]
+    fn google_endpoints_are_left_out_entirely() {
+        let text = generated();
+        assert!(!text.contains("gcal"), "google endpoint leaked into {text}");
+    }
+
+    #[test]
+    fn a_caldav_endpoint_becomes_a_bidirectional_pair() {
+        let text = generated();
+        assert!(text.contains("storage work_remote {"));
+        assert!(text.contains("type caldav"));
+        // The container is enumerated, and the calendar picked out by exact path.
+        assert!(text.contains("url \"https://dav.example.com/dav/calendars/user/me/\""));
+        assert!(text.contains("href_a \"/dav/calendars/user/me/work/\""));
+        assert!(text.contains("id_b work"));
+        assert!(
+            !block(&text, "pair work {").contains("one_way"),
+            "caldav pairs must stay bidirectional"
+        );
+    }
+
+    /// Secrets stay out of the portable config; pimsync fetches them the same way.
+    #[test]
+    fn secrets_are_fetched_by_command_not_embedded() {
+        let text = generated();
+        assert!(text.contains("password {"));
+        assert!(text.contains("shell pass show example/caldav"));
+    }
+
+    #[test]
+    fn a_feed_becomes_a_one_way_read_only_pair() {
+        let text = generated();
+        assert!(text.contains("type webcal"));
+        assert!(text.contains("one_way"));
+        // pimsync 0.5.11 rejects `read_only` on a webcal storage despite
+        // documenting it for all storage types.
+        assert!(!block(&text, "storage holidays_remote {").contains("read_only"));
+        assert!(text.contains("collection_id holidays"));
+    }
+
+    /// Both guards against a failed pull being mirrored as a mass deletion.
+    #[test]
+    fn every_pair_carries_the_deletion_guards() {
+        let text = generated();
+        let pairs = text.matches("\npair ").count();
+        assert_eq!(pairs, 2);
+        assert_eq!(text.matches("on_empty skip").count(), pairs);
+        assert_eq!(text.matches("on_delete skip").count(), pairs);
+    }
+
+    #[test]
+    fn splits_a_calendar_url_into_container_and_path() {
+        assert_eq!(
+            split_collection_url("https://dav.example.com/dav/calendars/user/me/work/"),
+            Some((
+                "https://dav.example.com/dav/calendars/user/me/".into(),
+                "/dav/calendars/user/me/work/".into()
+            ))
+        );
+        // A trailing slash is optional.
+        assert_eq!(
+            split_collection_url("https://dav.example.com/calendars/work"),
+            Some((
+                "https://dav.example.com/calendars/".into(),
+                "/calendars/work/".into()
+            ))
+        );
+        // A single segment leaves the root as the container.
+        assert_eq!(
+            split_collection_url("https://dav.example.com/work/"),
+            Some(("https://dav.example.com/".into(), "/work/".into()))
+        );
+    }
+
+    /// A bare server root names no calendar, so it must be rejected rather than
+    /// guessed at.
+    #[test]
+    fn a_url_without_a_collection_is_refused() {
+        assert_eq!(split_collection_url("https://dav.example.com/"), None);
+        assert_eq!(split_collection_url("https://dav.example.com"), None);
+    }
+
     #[test]
     fn parses_the_version_line() {
         assert_eq!(

src/sync.rs

@@ -10,7 +10,8 @@ use std::path::{Path, PathBuf};
 
 use thiserror::Error;
 
-use crate::config::{Aggregate, Config, Endpoint};
+use crate::config::{Aggregate, Config, Endpoint, EndpointKind};
+use crate::pimsync::{self, PimsyncError};
 use crate::reconcile::{self, Action, Conflict, Policy, ReconcileError, Skipped, SourceView};
 use crate::state::{State, Target};
 use crate::vdir::{self, VdirError};
@@ -21,6 +22,14 @@ pub enum SyncError {
     Vdir(#[from] VdirError),
     #[error(transparent)]
     State(#[from] crate::state::StateError),
+    #[error(transparent)]
+    Pimsync(#[from] PimsyncError),
+    #[error("could not create {path}: {source}")]
+    Prepare {
+        path: PathBuf,
+        #[source]
+        source: std::io::Error,
+    },
     #[error("aggregate `{aggregate}`: {source}")]
     Reconcile {
         aggregate: String,
@@ -73,6 +82,22 @@ pub fn run(
         ..Report::default()
     };
 
+    prepare_vdirs(config, state_dir)?;
+
+    // A dry run must leave the vdirs exactly as it found them, and pimsync
+    // writes to them, so the pulls and pushes are skipped along with everything
+    // else. What it reports is therefore what the *last* pull left behind.
+    let pimsync_config = if dry_run || !needs_pimsync(config) {
+        None
+    } else {
+        Some(pimsync::write_config(config, state_dir)?)
+    };
+
+    // Pull first, so the reconciler sees one consistent snapshot of every remote.
+    if let Some(path) = &pimsync_config {
+        pimsync::sync(path)?;
+    }
+
     for aggregate in &config.aggregates {
         let target = resolve(config, aggregate, &aggregate.target)?;
         check_target_drift(&state, aggregate, target)?;
@@ -81,12 +106,37 @@ pub fn run(
         )?);
     }
 
+    // Push what the reconciler decided out to the remotes.
+    if let Some(path) = &pimsync_config {
+        pimsync::sync(path)?;
+    }
+
     if !dry_run {
         state.save(&state_path)?;
     }
     Ok(report)
 }
 
+/// Whether any endpoint is one pimsync handles. A Google-only setup needs none.
+fn needs_pimsync(config: &Config) -> bool {
+    config.endpoints.iter().any(|endpoint| {
+        matches!(
+            endpoint.kind,
+            EndpointKind::Caldav { .. } | EndpointKind::Webcal { .. }
+        )
+    })
+}
+
+/// pimsync requires every vdir collection to exist before it will run, and an
+/// endpoint that has never synced has no directory yet.
+fn prepare_vdirs(config: &Config, state_dir: &Path) -> Result<(), SyncError> {
+    for endpoint in &config.endpoints {
+        let dir = vdir_path(state_dir, &endpoint.id);
+        std::fs::create_dir_all(&dir).map_err(|source| SyncError::Prepare { path: dir, source })?;
+    }
+    Ok(())
+}
+
 fn resolve<'a>(
     config: &'a Config,
     aggregate: &Aggregate,
@@ -244,7 +294,13 @@ fn sync_aggregate(
     Ok(report)
 }
 
-/// Where an endpoint's local mirror lives.
-pub fn vdir_path(state_dir: &Path, endpoint_id: &str) -> PathBuf {
-    state_dir.join("vdir").join(endpoint_id)
+/// The directory holding every endpoint's local mirror. This is the vdir storage
+/// pimsync is pointed at; each endpoint is one collection inside it.
+pub fn vdir_root(state_dir: &Path) -> PathBuf {
+    state_dir.join("vdir")
+}
+
+/// Where a single endpoint's local mirror lives.
+pub fn vdir_path(state_dir: &Path, endpoint_id: &str) -> PathBuf {
+    vdir_root(state_dir).join(endpoint_id)
 }