An aggregate with no default_sink already left events created in it alone, but
treated doing so as a failure: it reported a skip per event per cycle saying no
sink was configured, as though something had gone wrong. Nothing had. An
aggregate is also a calendar, and holding events of its own is a legitimate way
to use one.
Skipped::NoSink is replaced by a kept_local count, reported plainly. `@local`
joins the routing markers, so the mode also works per-event where a
default_sink is configured — which was not previously expressible. Unlike every
other marker it is deliberately not stripped: the others have done their job
once the event reaches its source, whereas this one never leaves, so it has to
stay legible for the next cycle to reach the same decision. `local` is
therefore a reserved endpoint id, and configuring one is refused.
This also fixes a real defect. `retarget` rebuilds the new target from the
recorded links, which cover derived events only, so an event belonging to the
aggregate itself did not follow the move — it stayed on the calendar being left
behind while everything around it moved on, quietly. It is now carried across,
since there is nothing to re-derive it from, and the new target's scheduling
rule is applied on the way: this is a write to an aggregate like any other, and
a guest list carried live onto a server that schedules would mail everyone on
it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An aggregate without a default_sink leaves events created in it alone, which is
how a target calendar keeps events of its own. The reconciler unit tests cover
the decision; this covers it surviving the round trip, and specifically that it
holds on every subsequent cycle — an event that survived the first cycle and
was swept up by the second would be worse than never having worked.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
SPECS.md says why the tool is built the way it is and TODO.md tracks what is
left; neither tells anyone how to use it. This covers install, the whole
configuration format, routing, the commands, and what the state directory
holds.
The Google section is walked through step by step rather than left to Google's
own documentation, because the parts that actually cost time are the ones it
does not cover: the client has to be a Desktop app, since calcalist redirects
to a loopback port it picks at run time and a Web application client demands a
redirect URI and a verified domain it cannot be given; you have to add yourself
under Test users or authorisation fails with access_denied against your own
account; and a consent screen left in Testing has its refresh tokens expired
after seven days, so a working installation stops with nothing having changed.
Checked against the binary rather than by reading: every documented command and
flag against `--help`, every field against config.rs, and the worked example
run through `doctor` and `status` to confirm it validates and prints the
routing markers the README says it does.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Eight items were still open at the end of M1: two integration tests that had
only been run by hand, and six known gaps.
Recurrence overrides now reach Google. Google addresses an exception through
the series rather than as an event of its own, so the master is sent first and
each override is then matched to its instance by original start time and
patched. Matching needs the two sides' spellings reduced to one key: iCalendar
writes a zoned local time, Google an absolute offset. An override matching no
occurrence is counted rather than forced — that means a stale RECURRENCE-ID
left behind by an edited RRULE, and inventing an event for it would put
something in the calendar the series does not contain.
Reading one component apart from another needed a view `properties` cannot
give: it flattens every VEVENT together, which is right for the UID a series
shares and wrong for an override, whose SUMMARY and the master's are then
indistinguishable. `Calendar::events` splits them.
A TZID now travels with the VTIMEZONE that defines it, derived from the zone's
own transition table as the yearly rule it implies. This changes the content
hash of every zoned recurring event, so the first cycle after this re-pushes
them.
A Google authorisation is filed under the account it was granted for rather
than the endpoint that asked for it, so two endpoints on one account no longer
need a login each. The account is read from the primary calendar's id, which
needs no scope beyond the calendar one already granted. Authorisations written
by the previous scheme are still honoured, and move across at the next login.
An unreachable Google endpoint no longer ends the cycle — one lapsed token used
to stop the CalDAV side too. It is named, only the aggregates depending on it
stand down, and the run exits non-zero so a partial cycle cannot pass for
success. The CalDAV leg cannot be narrowed the same way: pimsync is one process
covering every pair, so a failure does not say which pair it belongs to.
`--dry-run` now pulls for real, into a throwaway copy of the local mirrors and
through a pimsync configuration that only ever reads from a server. What it
reports is measured against the calendars as they are now rather than against
whatever the last real cycle left behind.
`calcalist prune` reports local mirrors of endpoints the configuration no
longer names, and removes them under --force.
The integration tests run against a real Radicale server and a real iCal feed:
convergence and idempotence, the dry run, prune, and the scheduling rule
asserted on the bytes that actually reached the server. The plan asked for an
SMTP sink for that last one; Radicale implements no RFC 6638 scheduling, so a
quiet SMTP port would have proved nothing about the transform.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The marker that steers a new event is the endpoint's id, which until now meant
remembering what was written in a TOML file while typing into a calendar app.
status lists the endpoints, what each aggregate publishes and mirrors, where
untagged events go, and the exact @markers that would work — read-only sources
omitted, since offering one as a destination would mislead.
A refused marker now names the alternatives rather than only reporting that it
failed, and matching ignores case: the name is typed by hand, and capitalisation
is not worth failing over.
133 tests.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
M1 is functionally complete, so the roadmap now separates what is done from what
is genuinely outstanding rather than leaving both in one list.
Records the work that arrived after the original plan — routing hints, alarm
mapping both ways, command-sourced feed URLs, the topology tests — and what was
verified against real accounts rather than only in tests.
Adds four gaps found while building and testing that were not previously written
down: a Google endpoint needs its own login even when it shares an account and
client with another; removing an endpoint leaves its vdir behind; --dry-run
reports against the last pull rather than current remote state; and a TZID is
emitted without an accompanying VTIMEZONE.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An event created in an aggregate went to the configured default_sink and nowhere
else, so with several writable sources there was no way to say which calendar a
new event belonged in.
A line reading @endpoint-id in the description now picks the source, and a
matching CATEGORIES value does too. The description rather than the title because
every calendar client exposes a notes field and editing it does not disfigure the
event's name; CATEGORIES as well because that is the field iCalendar intends,
even though many mobile clients hide it. The marker is stripped before the event
reaches the calendar, being calcalist's bookkeeping rather than content.
A marker naming something that is not a writable source of that aggregate is
refused and reported, not redirected to the default: a typo should not quietly
file an event in the wrong calendar. A bare address in prose is not a marker
either, since a marker must be a line of its own.
Also covers the shapes beyond many-into-one: a source feeding several aggregates,
several aggregates sharing one target, a cycle between two aggregates, a delete
cascading across aggregates, and competing edits arriving through two aggregates
at once — the last being caught by the existing conflict detection rather than
silently overwriting.
Error display no longer repeats itself; thiserror already prints the cause chain.
Verified live against real accounts: two Google calendars aggregating into a
Posteo calendar, an edit in the aggregate reaching the originating Google
calendar, an event routed to a chosen source by its description marker, and a
deletion propagating from the aggregate through to Google.
129 tests.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Mirrors are created with events.import, updated and deleted with
sendUpdates=none, so a write to an aggregate cannot mail anyone about a meeting
that was already invited from its source.
The import question the plan flagged is settled from Google's own API discovery
document rather than by guesswork: events.import accepts no sendUpdates parameter
at all, while insert, update and delete all do, and it is documented as adding "a
private copy of an existing event". A method with notification behaviour would
need that control. Confirmed live that attendees survive an import with their
response statuses intact.
Testing against the real API caught a defect that unit tests could not: the first
push dropped VALARM entirely and Google substituted the calendar's default
reminders. Alarms are never stripped by decision, so they now map to and from
Google's reminder overrides in both directions, with relative TRIGGER durations
converted to whole minutes. A trigger Google cannot express — absolute, or after
the start — is dropped rather than guessed at.
Known gaps recorded in TODO.md rather than papered over: a series' exceptions are
not pushed, since Google models those as separate events against an existing
series; and a failed Google pull still aborts the cycle, which is safe but stops
the CalDAV side too.
121 tests.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The Google equivalent of the pimsync pull, which pimsync cannot do: incremental
fetch by syncToken, converted to iCalendar and written into the endpoint's vdir.
A cursor Google no longer accepts comes back as 410, which means start again
rather than something broke, so that case refetches instead of failing.
The conversion was written against what the API actually returns, having probed
a real calendar first. Three findings shaped it:
- A recurring event's exceptions carry the same iCalUID as their master, so a
series belongs in one file, which is exactly the vdir convention.
- start.dateTime is an absolute instant while start.timeZone names the zone the
recurrence expands in, and the two need not agree: a real event reads
2023-10-31T13:00:00+02:00 with timeZone Asia/Karachi, which is +05:00.
Emitting the instant under that TZID unconverted would move it three hours, so
the instant is converted into its zone. That needs a timezone database, hence
jiff.
- A deleted occurrence arrives as an override with status cancelled. That is an
absence rather than an event, so it becomes an EXDATE on the master; moved
occurrences become RECURRENCE-ID events.
Timed values are written in UTC unless the event recurs. Only a recurrence needs
a zone to expand in, and confining TZID to those events limits how far we depend
on clients tolerating a TZID with no VTIMEZONE alongside it.
Verified against a live calendar: 30 real events, including a weekly series with
both a cancelled and several moved occurrences, converted and re-parsed intact.
The retarget tests no longer run a full cycle. They had used Google endpoints as
inert local stand-ins, which stopped being true the moment sync learned to
contact Google; they now reconcile directly.
115 tests.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Moving an aggregate to a different endpoint is the sharpest failure mode in the
design, so it is a command rather than something a config edit triggers: sync
already refuses on target drift and points here.
The move settles against the old target first. An event a user created in the
aggregate and that has not yet reached a sink exists only there, and would be
lost the moment we stop looking at that calendar. Only then is every mapped event
rebuilt on the new target — a rewrite rather than a copy, since the rendered
content differs between backends that can and cannot suppress scheduling.
Orphans on the old target are kept unless --purge-old, and a purge is bounded by
the derivation, so events the user keeps in that calendar themselves survive.
Also fixed prepare_vdirs, which created a directory per endpoint. That told
pimsync a local collection existed before its remote counterpart had been seen,
so it tried to create the counterpart: unsupported for a read-only feed, and it
would have invented calendars on a CalDAV server. Only the vdir root is created
now; collections are pimsync's to make from what it discovers.
One test premise was wrong rather than the code: a foreign event in an aggregate
that has a sink is a user-created event and gets adopted into that sink, so it
never reaches a purge as an outsider. Both behaviours are now covered.
Verified against the live Posteo calendar after the change: still a clean no-op.
106 tests.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Google requires OAuth for calendar access; app passwords stopped working for
CalDAV, CardDAV and IMAP in March 2025, so there is no simpler path to offer.
- Authorisation code flow over a loopback redirect, which is what Google
supports for desktop clients now the copy-paste flow is gone, with PKCE so an
intercepted code is useless without the verifier. Only the refresh token is
persisted, 0600, in the state directory.
- An expired grant is reported as itself: a consent screen still in Testing has
its refresh tokens expired after 7 days, and "run calcalist google login" is
more use than Google's bare invalid_grant.
- doctor reports whether each Google endpoint is still authorised, since an
installation that worked last week can stop with nothing having changed here.
A webcal URL may now come from a command instead of the config. Google's secret
iCal address grants read access to a whole calendar to anyone holding it, so
writing it into a file described as portable and secret-free was a contradiction.
Fixed a serious defect in the first draft of this module: random_token used
fs::read on /dev/urandom, which reads to end of file. /dev/urandom has no end,
so it allocated until the machine ran out of memory — it took the editor down
with it. It now reads exactly 32 bytes, and a randomness failure is fatal rather
than falling back to the clock, since a guessable state or PKCE verifier defeats
the point of having them.
Verified end to end against a live Posteo CalDAV calendar: pimsync validated the
generated config against the real server, 58 events from a public feed were
mirrored and pushed, and a second run was a no-op.
97 tests.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Generates pimsync's configuration from calcalist.toml and invokes it one-shot
either side of the reconcile step: pull first so the reconciler sees one
consistent snapshot, push afterwards to publish what it decided. Its daemon
mode stays unused because it would write the same vdirs the reconciler reads.
- Endpoints are mapped with pimsync's collection blocks, matching a local vdir
subdirectory to an exact remote path. Matching by collection id instead would
have tied our directory names to whatever the server's URL segments happen to
be, and two calendars ending in the same segment would collide.
- The remote is storage_a with conflict_resolution keep a, so a concurrent
server-side change wins over a local one calcalist has not yet reconciled.
Nothing is lost: the next cycle sees the server's version and decides properly.
- on_empty skip and on_delete skip on every pair, so a failed pull is not
mirrored as a mass deletion.
- Secrets stay out of the portable config: pimsync is given the same command
calcalist would run to fetch them.
- Google endpoints are omitted entirely, since pimsync can reach neither its REST
API nor its CalDAV endpoint.
doctor now asks pimsync to validate the generated config. pimsync is pre-1.0 and
its parser does not always match its documentation, so asking it is the only
reliable check — a parse rejection is reported as a failure, an unreachable
server only as a warning.
Found while testing against the real binary: pimsync 0.5.11 documents read_only
as applying to every storage type but rejects it on a webcal storage, with only
"Could not parse file" to go on. Dropped; it was redundant beside one_way.
Verified end to end against a live Radicale server and an HTTP-served feed: the
feed synced in, mirrored with a derived UID and provenance, and pushed to the
server; repeated runs are no-ops and adding one feed event mirrors exactly one.
87 tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The core of M1: everything needed to reconcile source calendars against an
aggregate, short of getting events in and out over the network.
- ical: surgical line-level editing. Each logical line keeps a byte range into
the original, so untouched lines are emitted verbatim and only edited ones are
rebuilt. Parsing and re-serialising would drop every property we do not model.
- ical: content hashing excludes DTSTAMP and LAST-MODIFIED. Servers rewrite them
on every store, so hashing them would report a change on every cycle forever.
- provenance: aggregate UIDs derived as blake3(aggregate, source, source_uid),
length-prefixed so field boundaries cannot collide. Deriving rather than
recording makes the state file a cache, and makes our own mirrors recognisable,
which is what stops writes echoing back around.
- mirror: the transforms. An aggregate copy must be scheduling inert, so writing
it never mails invitations for a meeting already invited from its source.
Google can suppress notification and keeps real attendees; CalDAV cannot, so
the guest list is demoted to inert data and a declined meeting is marked
TRANSP:TRANSPARENT. Writing an edit back uses the source as donor for what the
demotion removed, so editing a time cannot silently drop the guests.
- reconcile: pure decision engine. Only the source changed updates the mirror,
only the aggregate changed writes back, both changed keeps the source and logs
a conflict.
- Mass-deletion guard takes an absolute floor as well as a fraction: a share
alone is meaningless at small counts, where deleting the only event is 100%.
- sync refuses to run when an aggregate's configured target differs from the
recorded one, before reconciling. Otherwise the new empty target would read as
an aggregate whose every event was deleted, and delete propagation would then
remove them from every source.
Found by end-to-end testing: writing an item already present under a different
filename created a duplicate rather than replacing it, because filenames are
derived from the UID while pimsync picks its own. Writes now carry the path they
supersede. Covered by a regression test.
79 tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The implementation plan lives outside the repo, so it neither travels with the
project nor appears in history. Record the milestones here instead, with M0
checked off and the safety-critical M1 items called out separately from the
module work.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Initialise the project per the approved implementation plan (M0).
- Pin the toolchain with devbox: Rust 1.97.1, pimsync 0.5.11, jujutsu 0.44.0,
and radicale 3.7.8 for later integration tests.
- Add the configuration model with full referential validation, reporting
every problem in one pass rather than short-circuiting on the first.
- Add `calcalist doctor`, verifying pimsync's presence and version series,
the state directory, and the configuration.
- Define the whole CLI surface; only `doctor` acts, the rest exit 2 rather
than pretending to work.
- Rewrite SPECS.md: Rust naming conventions in place of the JS/TS style
lines, a `devbox run check` gate instead of a pre-commit hook (jj runs no
git hooks), and the sync semantics the spec previously left unstated.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
