Close the remaining M1 gaps

Eight items were still open at the end of M1: two integration tests that had
only been run by hand, and six known gaps.

Recurrence overrides now reach Google. Google addresses an exception through
the series rather than as an event of its own, so the master is sent first and
each override is then matched to its instance by original start time and
patched. Matching needs the two sides' spellings reduced to one key: iCalendar
writes a zoned local time, Google an absolute offset. An override matching no
occurrence is counted rather than forced — that means a stale RECURRENCE-ID
left behind by an edited RRULE, and inventing an event for it would put
something in the calendar the series does not contain.

Reading one component apart from another needed a view `properties` cannot
give: it flattens every VEVENT together, which is right for the UID a series
shares and wrong for an override, whose SUMMARY and the master's are then
indistinguishable. `Calendar::events` splits them.

A TZID now travels with the VTIMEZONE that defines it, derived from the zone's
own transition table as the yearly rule it implies. This changes the content
hash of every zoned recurring event, so the first cycle after this re-pushes
them.

A Google authorisation is filed under the account it was granted for rather
than the endpoint that asked for it, so two endpoints on one account no longer
need a login each. The account is read from the primary calendar's id, which
needs no scope beyond the calendar one already granted. Authorisations written
by the previous scheme are still honoured, and move across at the next login.

An unreachable Google endpoint no longer ends the cycle — one lapsed token used
to stop the CalDAV side too. It is named, only the aggregates depending on it
stand down, and the run exits non-zero so a partial cycle cannot pass for
success. The CalDAV leg cannot be narrowed the same way: pimsync is one process
covering every pair, so a failure does not say which pair it belongs to.

`--dry-run` now pulls for real, into a throwaway copy of the local mirrors and
through a pimsync configuration that only ever reads from a server. What it
reports is measured against the calendars as they are now rather than against
whatever the last real cycle left behind.

`calcalist prune` reports local mirrors of endpoints the configuration no
longer names, and removes them under --force.

The integration tests run against a real Radicale server and a real iCal feed:
convergence and idempotence, the dry run, prune, and the scheduling rule
asserted on the bytes that actually reached the server. The plan asked for an
SMTP sink for that last one; Radicale implements no RFC 6638 scheduling, so a
quiet SMTP port would have proved nothing about the transform.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
randogoth 2026-09-10 13:49:38 +03:00
parent 06e661b0c4
commit 623505b9c8
14 changed files with 2008 additions and 144 deletions

274
tests/caldav.rs Normal file
View file

@ -0,0 +1,274 @@
//! End-to-end tests against a real CalDAV server and a real iCal feed.
//!
//! The unit tests establish that the reconciler decides correctly. These
//! establish that the decisions survive the round trip through pimsync and a
//! server that rewrites what it stores — which is where every bug found by hand
//! during M1 actually lived.
//!
//! Radicale and pimsync both come from devbox, so `devbox run check` has them.
//! Outside that shell the tests report what is missing and pass, rather than
//! failing for a reason that has nothing to do with the code.
mod support;
use support::{Calcalist, Feed, Radicale};
/// An event with guests, an organiser and an alarm — the combination the
/// scheduling rules are about.
const MEETING: &str = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:-//test//EN\r\n\
BEGIN:VEVENT\r\nUID:meeting@work\r\nDTSTAMP:20260101T000000Z\r\n\
DTSTART:20260910T090000Z\r\nDTEND:20260910T100000Z\r\nSUMMARY:Planning\r\n\
ORGANIZER;CN=Chair:mailto:chair@example.com\r\n\
ATTENDEE;CN=Guest;PARTSTAT=ACCEPTED:mailto:guest@example.com\r\n\
BEGIN:VALARM\r\nACTION:DISPLAY\r\nTRIGGER:-PT15M\r\nDESCRIPTION:Soon\r\n\
END:VALARM\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n";
const HOLIDAY_FEED: &str = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:-//feed//EN\r\n\
BEGIN:VEVENT\r\nUID:newyear@feed\r\nDTSTAMP:20260101T000000Z\r\n\
DTSTART;VALUE=DATE:20260101\r\nDTEND;VALUE=DATE:20260102\r\n\
SUMMARY:New Year\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n";
struct Fixture {
_root: tempfile::TempDir,
server: Radicale,
_feed: Feed,
calcalist: Calcalist,
}
/// A CalDAV source, a read-only feed, and a CalDAV target to aggregate into.
fn fixture() -> Option<Fixture> {
let missing = support::missing_binaries();
if !missing.is_empty() {
eprintln!(
"skipped: {} not on PATH; run under devbox",
missing.join(", ")
);
return None;
}
let root = tempfile::tempdir().expect("temp");
let server = Radicale::start(root.path());
server.create_calendar("work");
server.create_calendar("unified");
server.put_event("work", "meeting.ics", MEETING);
let feed = Feed::start(HOLIDAY_FEED.to_string());
let config = format!(
r#"
version = 1
[[endpoint]]
id = "work"
type = "caldav"
url = "{work}"
username = "{user}"
secret_command = "printf password"
[[endpoint]]
id = "published"
type = "caldav"
url = "{unified}"
username = "{user}"
secret_command = "printf password"
[[endpoint]]
id = "holidays"
type = "webcal"
url = "{feed}"
[[aggregate]]
id = "everything"
target = "published"
sources = ["work", "holidays"]
default_sink = "work"
"#,
work = server.url("work"),
unified = server.url("unified"),
user = support::USER,
feed = feed.url(),
);
let calcalist = Calcalist::new(root.path(), &config);
Some(Fixture {
_root: root,
server,
_feed: feed,
calcalist,
})
}
/// Both sources reach the target, and running again changes nothing.
///
/// Idempotence is the property that matters most: a cycle that is not a no-op
/// against unchanged input would rewrite every event on every run, and each
/// rewrite is a chance for the server to hand back something slightly different.
#[test]
fn a_cycle_converges_and_the_next_one_does_nothing() {
let Some(fixture) = fixture() else { return };
let first = fixture.calcalist.run(&["sync"]);
assert!(
first.succeeded(),
"first sync failed\n{}\n{}",
first.stdout,
first.stderr
);
let published = fixture.server.stored("unified");
assert_eq!(
published.len(),
2,
"both sources should arrive: {published:?}"
);
let all = published.join("\n");
assert!(all.contains("Planning"), "{all}");
assert!(all.contains("New Year"), "{all}");
// Provenance travels with each mirror, so the aggregate knows where its
// events came from without consulting the state file.
assert!(all.contains("X-CALCALIST-SOURCE:work"), "{all}");
assert!(all.contains("X-CALCALIST-SOURCE:holidays"), "{all}");
let second = fixture.calcalist.run(&["sync"]);
assert!(
second.succeeded(),
"second sync failed\n{}\n{}",
second.stdout,
second.stderr
);
assert!(
second.stdout.contains("everything: 0 mirrored"),
"the second cycle should be a no-op:\n{}",
second.stdout
);
assert_eq!(
fixture.server.stored("unified"),
published,
"the second cycle rewrote the target"
);
}
/// The scheduling rule, checked against what actually reached the server.
///
/// A CalDAV server has no portable way to be told not to send invitations, so
/// inertness is structural: the mirror carries the guest list as data and not as
/// live scheduling properties. The alarm is the deliberate exception — stripping
/// it would destroy every reminder in the one calendar the user subscribes to.
///
/// Radicale implements no scheduling of its own, so this asserts on the bytes
/// stored rather than on a mail sink: with nothing to send mail, a quiet SMTP
/// port would prove nothing about the transform.
#[test]
fn a_mirror_carries_no_live_scheduling_properties() {
let Some(fixture) = fixture() else { return };
let run = fixture.calcalist.run(&["sync"]);
assert!(
run.succeeded(),
"sync failed\n{}\n{}",
run.stdout,
run.stderr
);
let mirror = fixture
.server
.stored("unified")
.into_iter()
.find(|item| item.contains("Planning"))
.expect("the meeting should have been mirrored");
for property in ["ATTENDEE;", "ATTENDEE:", "ORGANIZER;", "ORGANIZER:"] {
assert!(
!mirror
.lines()
.any(|line| line.trim_start().starts_with(property)),
"a live {property} reached the aggregate:\n{mirror}"
);
}
// The guests are still there, as something nothing will act on.
assert!(
mirror.contains("X-CALCALIST-ATTENDEES"),
"the guest list was lost:\n{mirror}"
);
assert!(
mirror.contains("guest@example.com"),
"the guest list was lost:\n{mirror}"
);
assert!(
mirror.contains("BEGIN:VALARM") && mirror.contains("TRIGGER:-PT15M"),
"the alarm did not survive:\n{mirror}"
);
// The source keeps its scheduling properties: only the aggregate is inert.
let source = fixture
.server
.stored("work")
.into_iter()
.find(|item| item.contains("Planning"))
.expect("the meeting should still be in its source");
assert!(source.contains("ATTENDEE"), "{source}");
assert!(source.contains("ORGANIZER"), "{source}");
}
/// A dry run must reach the servers to be worth anything, and change nothing.
#[test]
fn a_dry_run_reports_without_touching_anything() {
let Some(fixture) = fixture() else { return };
let dry = fixture.calcalist.run(&["sync", "--dry-run"]);
assert!(
dry.succeeded(),
"dry run failed\n{}\n{}",
dry.stdout,
dry.stderr
);
assert!(
dry.stdout.contains("everything: 2 mirrored"),
"the dry run should have seen both sources:\n{}",
dry.stdout
);
assert!(
fixture.server.stored("unified").is_empty(),
"the dry run published events"
);
// And the real cycle that follows is not confused by it.
let real = fixture.calcalist.run(&["sync"]);
assert!(
real.succeeded(),
"sync after a dry run failed\n{}\n{}",
real.stdout,
real.stderr
);
assert_eq!(fixture.server.stored("unified").len(), 2);
}
/// Removing an endpoint used to leave its events sitting in the state directory
/// with nothing managing them. They are now reported, and removed on request.
#[test]
fn a_retired_endpoints_mirror_is_reported_and_then_removed() {
let Some(fixture) = fixture() else { return };
assert!(fixture.calcalist.run(&["sync"]).succeeded());
let mirror = fixture.calcalist.state.join("calcalist/vdir/holidays");
assert!(mirror.is_dir(), "the feed should have been mirrored");
// The feed is dropped from the configuration, as a user would drop it.
std::fs::write(
&fixture.calcalist.config,
fixture.calcalist.config_without_feed(),
)
.expect("rewrite config");
let listed = fixture.calcalist.run(&["prune"]);
assert!(listed.succeeded(), "{}", listed.stderr);
assert!(
listed.stdout.contains("would remove") && listed.stdout.contains("holidays"),
"prune should say what it found:\n{}",
listed.stdout
);
assert!(mirror.is_dir(), "listing must not delete anything");
let removed = fixture.calcalist.run(&["prune", "--force"]);
assert!(removed.succeeded(), "{}", removed.stderr);
assert!(!mirror.exists(), "the orphaned mirror should be gone");
}