Compare commits

...

10 commits

Author SHA1 Message Date
randogoth
16f5d7351f readme update 2026-09-10 16:57:51 +03:00
randogoth
22a1236f79 Add the MIT license
LICENSE with the standard text, `license = "MIT"` in Cargo.toml so the metadata
matches, and a line in the README pointing at it.

Copyright is attributed to `randogoth`, taken from the repository's author
setting, and dated 2026.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 16:42:41 +03:00
randogoth
4854844a72 Cut the README to a getting-started guide; retire SPECS.md and TODO.md
The README had grown to 413 lines and explained as much as it instructed:
someone trying to get calcalist running read past the architecture, the
scheduling-inertness rule and the reasoning behind retarget before reaching
anything they needed to type.

Most of the excess was duplication of SPECS.md, which is now retired along with
TODO.md since the roadmap is finished. That inverted the usual answer — the
rationale could not simply move to the design document, because there would not
be one. Checking what would actually be orphaned: the architecture rationale is
already in the module doc comments, beside the code it governs and where it
cannot drift; the M0-M2 record is history and git keeps it. Two things had no
home: the working agreement that has shaped every commit, and two decisions
that cost real design work to decline and would otherwise be re-proposed from
first principles. Both are now in CLAUDE.md, which is read at the start of every
session rather than filed and forgotten.

What the README keeps is the whole setup path, because none of it exists
anywhere else now: every configuration field, the Google OAuth walkthrough in
full, routing, and the systemd units. What it gains is a short section on what
the tool does to your calendars — deletion propagating to the source is not
something to discover by accident. What it loses is the design commentary.

Also moves `discover` out of the command list and into the CalDAV endpoint
section, since getting the url field right is its entire purpose.

Verified rather than eyeballed: the worked example was extracted and run through
doctor and status, the command table diffed against --help, and every internal
anchor resolved.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 16:41:34 +03:00
randogoth
ef6482ed62 Finish M2: run lock, systemd units, discover; drop the web UI
Designing the configuration UI in full made the case against building it. Its
audience would be people who find TOML hard, but with bring-your-own OAuth
client settled, every user must first create a Google Cloud project, configure
a consent screen and put a secret in a keyring — a far higher bar than editing
thirty lines of config. Anyone who clears it can edit the file; anyone who
cannot never reaches the file. Against that stood three dependencies, five
modules, an auth.rs refactor and a security surface guarding something that
reads the config and touches the keyring. `doctor` and `status` had already
absorbed most of what it was for. The reasoning is recorded in TODO.md and
SPECS.md rather than left as an apparent oversight.

The run lock is not a UI feature and closes a gap that already existed: nothing
stopped a timer firing into a hand-run cycle, and two cycles interleaving
writes over the same vdirs is what the design otherwise avoids. flock is used
rather than a pid file because the kernel releases it however the process ends,
so a crash cannot leave a lock to clear by hand — which also means a lock we
failed to take is held by a live process, so the pid in it is worth reporting.

The one idea worth keeping from the UI design was collection discovery, which
needed no web layer. `calcalist discover` prints a ready-to-paste endpoint
block per calendar a server offers, removing the most error-prone field in the
config. pimsync's discovery output is undocumented, so the format was
established against a real server first. Two things it teaches: everything
arrives on stdout including failures, and a pair has two storages, so pimsync
reports the scratch vdir's contents too — parsing anchors on the heading naming
the server, or a probe directory's leftovers would be offered as the user's
calendars.

Verified against Posteo as well as Radicale: all four calendars found, the
first matching the URL already configured.

Also fixes a real defect in the test harness rather than its symptom. Ports were
chosen by binding one and letting go, so two tests could pick the same number —
and the loser's readiness check then succeeded against the winner's server,
silently sharing it. Startup now confirms the child we spawned is the one alive,
retries on another port if not, and waits for a real HTTP response rather than
an open socket.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 16:23:49 +03:00
randogoth
47ad8b4c47 Make an aggregate's own events a first-class mode
An aggregate with no default_sink already left events created in it alone, but
treated doing so as a failure: it reported a skip per event per cycle saying no
sink was configured, as though something had gone wrong. Nothing had. An
aggregate is also a calendar, and holding events of its own is a legitimate way
to use one.

Skipped::NoSink is replaced by a kept_local count, reported plainly. `@local`
joins the routing markers, so the mode also works per-event where a
default_sink is configured — which was not previously expressible. Unlike every
other marker it is deliberately not stripped: the others have done their job
once the event reaches its source, whereas this one never leaves, so it has to
stay legible for the next cycle to reach the same decision. `local` is
therefore a reserved endpoint id, and configuring one is refused.

This also fixes a real defect. `retarget` rebuilds the new target from the
recorded links, which cover derived events only, so an event belonging to the
aggregate itself did not follow the move — it stayed on the calendar being left
behind while everything around it moved on, quietly. It is now carried across,
since there is nothing to re-derive it from, and the new target's scheduling
rule is applied on the way: this is a write to an aggregate like any other, and
a guest list carried live onto a server that schedules would mail everyone on
it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 14:34:15 +03:00
randogoth
7607078394 Pin the behaviour of an aggregate with no sink
An aggregate without a default_sink leaves events created in it alone, which is
how a target calendar keeps events of its own. The reconciler unit tests cover
the decision; this covers it surviving the round trip, and specifically that it
holds on every subsequent cycle — an event that survived the first cycle and
was swept up by the second would be worse than never having worked.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 14:23:05 +03:00
randogoth
25d181108e Add a README covering use
SPECS.md says why the tool is built the way it is and TODO.md tracks what is
left; neither tells anyone how to use it. This covers install, the whole
configuration format, routing, the commands, and what the state directory
holds.

The Google section is walked through step by step rather than left to Google's
own documentation, because the parts that actually cost time are the ones it
does not cover: the client has to be a Desktop app, since calcalist redirects
to a loopback port it picks at run time and a Web application client demands a
redirect URI and a verified domain it cannot be given; you have to add yourself
under Test users or authorisation fails with access_denied against your own
account; and a consent screen left in Testing has its refresh tokens expired
after seven days, so a working installation stops with nothing having changed.

Checked against the binary rather than by reading: every documented command and
flag against `--help`, every field against config.rs, and the worked example
run through `doctor` and `status` to confirm it validates and prints the
routing markers the README says it does.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 14:18:49 +03:00
randogoth
623505b9c8 Close the remaining M1 gaps
Eight items were still open at the end of M1: two integration tests that had
only been run by hand, and six known gaps.

Recurrence overrides now reach Google. Google addresses an exception through
the series rather than as an event of its own, so the master is sent first and
each override is then matched to its instance by original start time and
patched. Matching needs the two sides' spellings reduced to one key: iCalendar
writes a zoned local time, Google an absolute offset. An override matching no
occurrence is counted rather than forced — that means a stale RECURRENCE-ID
left behind by an edited RRULE, and inventing an event for it would put
something in the calendar the series does not contain.

Reading one component apart from another needed a view `properties` cannot
give: it flattens every VEVENT together, which is right for the UID a series
shares and wrong for an override, whose SUMMARY and the master's are then
indistinguishable. `Calendar::events` splits them.

A TZID now travels with the VTIMEZONE that defines it, derived from the zone's
own transition table as the yearly rule it implies. This changes the content
hash of every zoned recurring event, so the first cycle after this re-pushes
them.

A Google authorisation is filed under the account it was granted for rather
than the endpoint that asked for it, so two endpoints on one account no longer
need a login each. The account is read from the primary calendar's id, which
needs no scope beyond the calendar one already granted. Authorisations written
by the previous scheme are still honoured, and move across at the next login.

An unreachable Google endpoint no longer ends the cycle — one lapsed token used
to stop the CalDAV side too. It is named, only the aggregates depending on it
stand down, and the run exits non-zero so a partial cycle cannot pass for
success. The CalDAV leg cannot be narrowed the same way: pimsync is one process
covering every pair, so a failure does not say which pair it belongs to.

`--dry-run` now pulls for real, into a throwaway copy of the local mirrors and
through a pimsync configuration that only ever reads from a server. What it
reports is measured against the calendars as they are now rather than against
whatever the last real cycle left behind.

`calcalist prune` reports local mirrors of endpoints the configuration no
longer names, and removes them under --force.

The integration tests run against a real Radicale server and a real iCal feed:
convergence and idempotence, the dry run, prune, and the scheduling rule
asserted on the bytes that actually reached the server. The plan asked for an
SMTP sink for that last one; Radicale implements no RFC 6638 scheduling, so a
quiet SMTP port would have proved nothing about the transform.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 14:03:44 +03:00
randogoth
06e661b0c4 Add calcalist status, and make routing markers discoverable
The marker that steers a new event is the endpoint's id, which until now meant
remembering what was written in a TOML file while typing into a calendar app.

status lists the endpoints, what each aggregate publishes and mirrors, where
untagged events go, and the exact @markers that would work — read-only sources
omitted, since offering one as a destination would mislead.

A refused marker now names the alternatives rather than only reporting that it
failed, and matching ignores case: the name is typed by hand, and capitalisation
is not worth failing over.

133 tests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 13:41:26 +03:00
randogoth
838e7a1891 Bring TODO.md up to date with M1 as built
M1 is functionally complete, so the roadmap now separates what is done from what
is genuinely outstanding rather than leaving both in one list.

Records the work that arrived after the original plan — routing hints, alarm
mapping both ways, command-sourced feed URLs, the topology tests — and what was
verified against real accounts rather than only in tests.

Adds four gaps found while building and testing that were not previously written
down: a Google endpoint needs its own login even when it shares an account and
client with another; removing an endpoint leaves its vdir behind; --dry-run
reports against the last pull rather than current remote state; and a TZID is
emitted without an accompanying VTIMEZONE.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 13:39:54 +03:00
26 changed files with 3557 additions and 311 deletions

74
CLAUDE.md Normal file
View file

@ -0,0 +1,74 @@
# calcalist
Aggregates events between CalDAV, Google Calendar and iCal feeds. `README.md` is
the user-facing guide; the design rationale lives in the module doc comments,
beside the code it governs.
These conventions were `SPECS.md`'s, which has been retired now the roadmap is
finished. Git history holds it, and `TODO.md`, if the record is ever wanted.
## Development
Use devbox for dependency management and scripts. Use jj for version control
(colocated with git).
- `devbox run fmt` — format.
- `devbox run check` — format check, clippy with warnings denied, and tests.
This is the gate; jj has no commit step to hang a hook off, since the working
copy is itself a commit.
- `devbox run test` — tests only.
Keep `/target` in `.gitignore`: jj snapshots the working copy on every command,
with no staging step.
The integration tests in `tests/` need `radicale` and `pimsync`, both of which
devbox provides. Outside that shell they report what is missing and pass.
## Coding style
- Simple, readable, idiomatic.
- Explicit types for public APIs and important fields.
- Prefer immutable data structures.
- No global mutable state.
- No speculative features or dependencies.
- Keep functions short and single-purpose; extract helpers to avoid nesting.
- File and module naming: `snake_case.rs`. Types and traits: `PascalCase`.
Functions, variables and fields: `snake_case`. Constants:
`SCREAMING_SNAKE_CASE`.
- Run `devbox run check` before describing a change.
- Comment only when intent is not obvious from the code.
- Prefer composition and traits over deep type hierarchies.
## Decisions already taken
Settled after real design work. Reopen only with new information, not from
first principles.
**No configuration web interface.** Designed in full, then dropped. Its audience
would be people who find TOML hard — but every user must first create a Google
Cloud project, configure a consent screen and put a secret in a keyring, which
is a far higher bar than editing the config. Anyone who clears it can edit the
file; anyone who cannot never reaches the file. `doctor`, `status` and
`discover` cover what the interface was actually for.
**Bring-your-own OAuth client, not a shipped verified one.** Verification for
the Calendar scope needs a domain you own — a `github.io` address is rejected —
and Calendar API quota is per Cloud project, with charges announced for
exceeding it. A shipped client would pool every user's syncing into the
publisher's project, and their bill. Shipping an *unverified* shared client is
worse still: the 100-new-user cap is permanent for the project's lifetime.
**pimsync cannot reach Google, and never will.** It has no REST storage, and its
config exposes only HTTP Basic auth, which Google's CalDAV endpoint rejected in
March 2025. The Google leg is calcalist's own in every design.
**The config file is never machine-rewritten.** It is the portable,
hand-editable artefact. Anything that writes it must preserve comments,
ordering and omitted defaults — `toml::to_string` on the `Serialize` derive
does none of those things.
## Known gap
A failing `pimsync sync` stops the whole CalDAV leg. Unlike the Google side this
cannot be narrowed: pimsync is one process covering every pair, and a failure
does not say which pair it belongs to.

1
Cargo.lock generated
View file

@ -117,6 +117,7 @@ dependencies = [
"blake3",
"clap",
"jiff",
"rustix",
"serde",
"serde_json",
"sha2",

View file

@ -4,12 +4,14 @@ version = "0.1.0"
edition = "2024"
rust-version = "1.97"
description = "Aggregate and sync events between CalDAV, Google Calendar and iCal feeds"
license = "MIT"
publish = false
[dependencies]
blake3 = "1.8"
clap = { version = "4.6", features = ["derive"] }
jiff = "0.2"
rustix = { version = "1.1", features = ["fs"] }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
sha2 = "0.11"

21
LICENSE Normal file
View file

@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 randogoth
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

255
README.md Normal file
View file

@ -0,0 +1,255 @@
# CalCalist
Aggregate several calendars into one, and keep them in step.
CalCalist mirrors any number of CalDAV calendars, Google calendars and iCal feeds into a single calendar you subscribe to. The point is to subscribe to one calendar instead of several: an event created or edited in the aggregate travels back to the calendar it came from, so the aggregate is somewhere to work rather than a read-only noticeboard. CalDAV and Google are read/write in both directions; iCal feeds are read-only, because the protocol is.
## Install
```sh
cargo build --release
# target/release/calcalist
```
One runtime dependency is needed for CalDAV and iCal sync: **`pimsync` 0.5.x on `PATH`**.
Then check the environment, which is worth doing before writing any config:
```sh
calcalist doctor
```
## Configure
One TOML file at `$XDG_CONFIG_HOME/calcalist/calcalist.toml`, or wherever `--config` points. It holds **no secrets and no sync state**, so it is safe to
copy between machines.
Credentials come from `*_command` fields: each is run through `sh -c` and the first line of its output is used. Point them at `secret-tool`, `pass`, `gpg -d`, or anything else that prints the secret.
```toml
version = 1
```
Each `[[endpoint]]` is one calendar, with an `id` you refer to it by elsewhere.
### CalDAV Endpoint
```toml
[[endpoint]]
id = "posteo"
type = "caldav"
url = "https://posteo.de:8443/calendars/you/abc123/work/"
username = "you@posteo.de"
secret_command = "secret-tool lookup service posteo user you@posteo.de"
```
`url` must name the calendar collection itself, not the server root.
Rather than digging it out of a provider's web interface, ask the server — this prints a ready-to-paste block per calendar, with ids taken from the URL for you to rename:
```sh
calcalist discover https://posteo.de:8443/calendars/you/ \
--username you@posteo.de --secret-command "secret-tool lookup service posteo"
```
## Google Calendar Endpoint
```toml
[[endpoint]]
id = "gcal"
type = "google"
calendar_id = "you@gmail.com"
client_id = "1234-abcd.apps.googleusercontent.com"
client_secret_command = "secret-tool lookup service google-calcalist"
# account = "you@gmail.com" # only when logged in to more than one account
```
`calendar_id` is `primary`, or the calendar's own address — the "Calendar ID" under its settings in Google Calendar. See [Authorising Google](#authorising-google) for `client_id` and the secret.
## iCal Feed Endpoint
```toml
[[endpoint]]
id = "holidays"
type = "webcal"
url = "https://example.org/holidays.ics"
```
Always read-only. Give exactly one of `url` or `url_command`; the latter is for a feed whose address is itself a credential, such as Google's "secret address in iCal format".
## Aggregates
Each `[[aggregate]]` composes endpoints: several `sources` mirrored into one `target`.
```toml
[[aggregate]]
id = "unified"
target = "posteo"
sources = ["gcal", "holidays"]
default_sink = "gcal"
# Defaults, all optional:
conflict = "source_wins" # the only policy; the origin calendar wins
propagate_deletes = true # deleting a mirror deletes the original
max_delete_fraction = 0.2 # abort if more of the aggregate than this vanishes
```
The rules, all checked before anything is synchronised, with every problem reported in one pass:
- `target` must exist and be writable, so never an iCal feed.
- `target` must not also be one of its own `sources`.
- `sources` must be non-empty, and each must exist and be named once.
- `default_sink`, if given, must be one of the `sources` and must be writable. Leaving it out is a deliberate mode — see [Getting events to the right calendar](#getting-events-to-the-right-calendar).
- No endpoint may be called `local`, which is reserved for the routing marker.
## A complete example
A Google calendar and a public holiday feed, aggregated into a CalDAV calendar:
```toml
version = 1
[[endpoint]]
id = "gcal"
type = "google"
calendar_id = "primary"
client_id = "1234-abcd.apps.googleusercontent.com"
client_secret_command = "secret-tool lookup service google-calcalist"
[[endpoint]]
id = "holidays"
type = "webcal"
url = "https://www.officeholidays.com/ics/germany"
[[endpoint]]
id = "posteo"
type = "caldav"
url = "https://posteo.de:8443/calendars/you/abc123/unified/"
username = "you@posteo.de"
secret_command = "secret-tool lookup service posteo user you@posteo.de"
[[aggregate]]
id = "unified"
target = "posteo"
sources = ["gcal", "holidays"]
default_sink = "gcal"
```
# Authorising Google
Google requires an OAuth client of your own. This is the fiddliest part of the setup, so in full:
1. At [console.cloud.google.com](https://console.cloud.google.com), create a
project — any name.
2. **APIs & Services → Library**, find **Google Calendar API**, enable it.
3. **OAuth consent screen**: choose **External**. Fill in the required fields. Under **Test users**, add your own Google address.
4. **Credentials → Create credentials → OAuth client ID**, and for **Application type** choose **Desktop app**.
5. Copy the client ID into `client_id`, and put the client secret somewhere `client_secret_command` can read it:
```sh
secret-tool store --label="calcalist Google client secret" service google-calcalist
```
6. Authorise:
```sh
calcalist google login gcal
```
A browser opens; approve the request. CalCalist asks only for `https://www.googleapis.com/auth/calendar`. The refresh token is written into the state directory, never into your configuration.
**One login covers every endpoint on that account.** The authorisation is filed under the account it was granted for, not the endpoint that asked, so a second Google calendar on the same account needs no login of its own. Set `account` on an endpoint only when CalCalist is logged in to more than one account.
**While the consent screen is in Testing, Google expires refresh tokens after seven days.** An installation that worked last week then stops with nothing
having changed locally. Publish the consent screen to production to end this; `calcalist doctor` reports it either way.
## Getting events to the right calendar
An event mirrored from a source goes back to that source when you edit it — CalCalist tracks where each one came from. An event you *create* in the
aggregate belongs to nothing yet, and goes to the aggregate's `default_sink`.
To send one somewhere else, put the endpoint's id after an `@` **on a line of its own** in the event's notes:
```
Dentist, bring the referral
@gcal
```
The marker is removed before the event arrives. `calcalist status` prints the markers that would actually work, so you need not remember what is in the
config; a marker naming anything else is refused and reported rather than quietly filed under the default.
**To keep an event in the aggregate itself**, tag it `@local` — it then stays put even though a `default_sink` would have taken it, and the marker is
deliberately left in place so later cycles decide the same way.
Leaving `default_sink` out of the aggregate entirely makes that the rule for every untagged event, so routing becomes opt-in. Such events live only in that calendar and cannot be rebuilt from a source, but they are ordinary events on that server, visible to every client subscribed to it.
A category matching an endpoint id works too. Note that *any* category on a new event is read as a routing instruction, so the notes marker is the safer form.
## What it does to your calendars
- **Deleting an event in the aggregate deletes it at its source**, unless `propagate_deletes = false`. A cycle that would remove more than `max_delete_fraction` of the aggregate is refused until you pass `--force` — with a floor of three, so removing one or two events is never blocked.
- **Edited in both places since the last sync, the source wins.** The conflict is reported.
- **Alarms are always kept**, with no option to strip them. The tool exists so you subscribe to one calendar rather than several, so dropping reminders would disarm every one you have.
- **Mirroring never mails your guests.** On a Google target attendees are kept and Google is told not to notify; on a CalDAV target, which cannot be told
that, the guest list is carried as inert data instead. Routing an event *you* created is the exception: that server invites its guests for real, which is the point of creating it.
## Run it
```sh
calcalist doctor # environment and configuration
calcalist sync # one cycle
```
To run it regularly, units ship in [systemd/](systemd/):
```sh
install -Dm644 systemd/calcalist.service ~/.config/systemd/user/calcalist.service
install -Dm644 systemd/calcalist.timer ~/.config/systemd/user/calcalist.timer
systemctl --user daemon-reload
systemctl --user enable --now calcalist.timer
```
The service expects the binary at `~/.local/bin/calcalist`; edit `ExecStart` if yours is elsewhere. Only one cycle runs at a time — a timer firing while you are running `sync` by hand is refused, naming the process that holds the lock. A cycle that could not reach an endpoint exits non-zero on purpose, so a lapsed token shows up as a failed unit rather than passing unnoticed; `journalctl --user -u calcalist` has the report. If your secrets come from a
keyring needing an unlocked session, the timer only works while you are logged in.
## Commands
`--help` has the flags.
| Command | |
|---|---|
| `sync` | One cycle. `--dry-run` reports against current remote state without writing; `--force` overrides the mass-deletion guard. |
| `status` | Endpoints, aggregates, how much is mirrored, and the routing markers that work. |
| `doctor` | Checks the environment and configuration. Run it first. |
| `discover` | Lists a CalDAV server's calendars as endpoint blocks to paste. |
| `google login` | Authorises an account. |
| `aggregate retarget` | Moves an aggregate to a different target calendar. `sync` refuses on its own when the target changes under it, and points here. |
| `prune` | Reports local mirrors of endpoints no longer configured; removes them under `--force`. |
Exit codes: `0` all well, `1` something failed or an endpoint could not be reached.
## Where things are kept
Everything machine-local lives under `$XDG_STATE_HOME/calcalist`:
```
state.json event mappings, content hashes, sync cursors
lock held for the duration of a cycle
pimsync.conf generated on every sync; edits are overwritten
pimsync-status/ pimsync's own record of what it has seen
vdir/<endpoint-id>/ the local mirror of each endpoint
google/accounts/<account>.json refresh tokens, 0600
```
None of it belongs in a backup or a dotfiles repository — it is a cache, and a lost state file costs a re-materialisation rather than data. Copy the
configuration between machines; leave this behind.
## Known gaps
- **A failing `pimsync sync` stops the whole CalDAV leg.** Google endpoints are isolated from each other, but pimsync is a single process covering every
CalDAV and feed pair, and a failure does not say which pair it belongs to.
## License
MIT. See [LICENSE](LICENSE).

View file

@ -1,56 +0,0 @@
# Calcalist
## Idea
Small Rust based commandline tool with utility systemd service file that can aggregate and sync events between CalDAV/Google Calendar/iCal. Users who have a Google Calendar can create a new calendar that syncs events from several CalDAV calendars. CalDAV users can sync several Google Calendars into one calendar. This should be doable both ways, so the aggregating calendar needs to be able to distinguish where its events came from and also have a way to create new events that get synced to the correct source. In addition iCal feeds can also be aggregated but they remain one way only.
Configuration via a simple web interface. Configuration is saved in a portable toml file that can be easily migrated.
## Architecture
Every endpoint — sources and aggregate targets alike — is mirrored to a local vdir, so the aggregation engine works purely on local files.
- `pimsync` is driven as a one-shot subprocess for the CalDAV and WebCal legs. Its daemon mode is unusable here: it would race the reconciler over the same vdir files, so calcalist owns scheduling.
- The Google leg is calcalist's own, via the Calendar REST API. pimsync cannot reach Google — it has no REST storage, and its config exposes only HTTP Basic auth, which Google's CalDAV endpoint has rejected since 2025-03-14.
- Sync state (event mappings, hashes, sync tokens) lives in a JSON sidecar under `$XDG_STATE_HOME/calcalist/`, never in the portable config.
- Secrets are never stored in the config either. Credentials come from `*_command` fields that are executed to fetch them.
## Sync semantics
| Question | Behaviour |
|---|---|
| Provenance | Aggregate UIDs derived as `blake3(aggregate_id, source_id, source_uid)`; the state file is a cache, not a single point of failure |
| Routing new events | A `@endpoint-id` line in the description, or a matching category, picks the source; otherwise the aggregate's `default_sink`. A hint naming an invalid sink is refused, never redirected to the default |
| Conflicts | Source wins — the origin calendar is authoritative |
| Deletion | Propagates to the source, guarded by a mass-deletion threshold |
| Attendees (mirroring) | Kept verbatim on a Google target; demoted to inert data on a CalDAV target |
| Attendees (routing) | Preserved — the sink server sends real invitations, which is intended |
| Alarms | Always preserved, with no option to strip them |
| Retargeting | `sync` refuses on target drift; `aggregate retarget` performs it deliberately |
The governing rule behind the attendee handling: **writes to an aggregate must never emit scheduling mail; writes to a source schedule normally.** Google can be told not to notify, so attendees survive intact there. CalDAV offers no portable way to suppress RFC 6638 scheduling, so inertness is achieved structurally by dropping the live properties instead.
Alarms are never stripped because the whole point of the tool is that a user subscribes to one calendar rather than several — so the duplicate-notification problem that stripping would guard against does not arise, while stripping would silently destroy every reminder in the one calendar the user actually watches.
## Development
Use devbox for dependency management and scripts. Use jj for version control (colocated with git).
- `devbox run fmt` — format.
- `devbox run check` — format check, clippy with warnings denied, and tests. This is the gate; jj has no commit step to hang a hook off, since the working copy is itself a commit.
- `devbox run test` — tests only.
Keep `/target` in `.gitignore`: jj snapshots the working copy on every command, with no staging step.
## Coding style
- Simple, readable, idiomatic.
- Explicit types for public APIs and important fields.
- Prefer immutable data structures.
- No global mutable state.
- No speculative features or dependencies.
- Keep functions short and single-purpose; extract helpers to avoid nesting.
- File and module naming: `snake_case.rs`. Types and traits: `PascalCase`. Functions, variables and fields: `snake_case`. Constants: `SCREAMING_SNAKE_CASE`.
- Run `devbox run check` before describing a change.
- Comment only when intent is not obvious from the code.
- Prefer composition and traits over deep type hierarchies.

90
TODO.md
View file

@ -1,90 +0,0 @@
# Roadmap
Milestones from the implementation plan. See SPECS.md for the architecture and the
sync semantics these items implement.
## M0 — skeleton (done)
- [x] Initialise jj colocated with git; `.gitignore` written before the first build
- [x] `devbox.json` pinning Rust 1.97.1, pimsync 0.5.11, jujutsu 0.44.0, radicale 3.7.8
- [x] Configuration model with referential validation, reporting every problem in one pass
- [x] `calcalist doctor` — pimsync presence and version series, state directory, configuration
- [x] Define the full CLI surface; unimplemented commands exit 2 rather than pretend
- [x] SPECS.md: Rust naming conventions, `devbox run check` gate, recorded sync semantics
## M1 — bidirectional sync
Core modules:
- [x] `state.rs` — JSON sidecar, atomic temp + fsync + rename; records each aggregate's
resolved target endpoint id **and** backend type
- [x] `vdir.rs` — read and write vdir directories
- [x] `ical.rs` — surgical line-level `.ics` editing (UID rewrite, property injection),
respecting RFC 5545 folding; no parse-and-reserialize
- [x] `provenance.rs` — deterministic `blake3(aggregate_id, source_id, source_uid)` UIDs
- [x] `mirror.rs` — the to-aggregate and to-source transforms (added; not in the
original plan, which folded these into `reconcile`)
- [x] `reconcile.rs` — the aggregation engine; pure, no I/O
- [x] `sync.rs` — one cycle over the local vdirs, applying what `reconcile` decides
- [x] `pimsync.rs` — generate `pimsync.conf` (with `on_empty skip` and `on_delete skip`),
drive one-shot `pimsync sync` bracketing the reconcile step
- [x] `doctor` asks `pimsync check` to validate the generated config, since pimsync's
parser does not always match its documentation
- [x] `google/auth.rs` — OAuth loopback flow with PKCE, refresh, keyring-sourced secrets
- [x] `google/convert.rs` — JSON to iCalendar, including recurrence and timezones
- [x] `google/api.rs` — incremental pull by syncToken, and push by import / update
/ delete with notification suppressed
- [x] Reintroduce `SchedulingSuppression` in `config.rs` (removed in M0 as dead code)
Safety-critical behaviour:
- [x] **`events.import` gate** — settled from Google's own API discovery document:
`events.import` accepts no `sendUpdates` parameter at all, while `insert`,
`update` and `delete` all do, and it is documented as adding "a private copy of
an existing event". Confirmed live that attendees and alarms survive an import.
Creation goes through `import`; update and delete pass `sendUpdates=none`.
- [x] `sync` refuses to run on aggregate target drift, before reconciliation
- [x] `aggregate retarget` — flush unrouted creations against the old target, then
re-materialise; keep old orphans by default
- [x] Mass-deletion guard (`max_delete_fraction`), overridable with `--force`, with an
absolute floor so deleting a couple of events is never refused
- [x] Echo suppression: derived UIDs are never re-ingested as source events
Tests:
- [x] `reconcile` table-driven cases: create/update/delete each direction, both-sides-changed,
routing, echo suppression, mass-delete abort
- [x] `ical` round-trip fixtures: recurring with overrides, all-day, TZID, unknown `X-` props
- [ ] Integration against Radicale plus a `file://` WebCal fixture; assert idempotence
- [x] Safety (unit level): no live `ATTENDEE`/`ORGANIZER` on a CalDAV-targeted mirror,
`VALARM` intact, `PARTSTAT: DECLINED` maps to `TRANSP: TRANSPARENT`, bulk deletion aborts
- [ ] Safety (integration): the same against a real Radicale instance with an SMTP sink,
proving no mail is emitted
- [x] Retarget: drift makes `sync` exit non-zero having written nothing and losing no source
event; purge is bounded by the derivation; an unrouted creation reaches a sink first
### Verified live, end to end
Two Google calendars aggregating into a Posteo CalDAV calendar, against real accounts:
fan-in from both sources with provenance intact; an edit in the aggregate reaching the
originating Google calendar; an event created in the aggregate routed to a chosen
source by a description marker; deletion propagating from the aggregate through to
Google; and the mass-deletion guard refusing a 100% removal until `--force`.
### Known gaps carried out of M1
- [ ] A recurring series' *exceptions* are not pushed to Google. Google models them
as separate events against an already existing series, so they need
`events.instances` plus a patch per exception. Reported per sync rather than
dropped silently.
- [x] `push` deleting an event remotely — verified live: deleting a mirror in the
CalDAV aggregate removed the origin event from Google.
- [ ] A failed Google pull aborts the whole cycle, including the CalDAV side. Safe —
reconciling against a stale snapshot could read as mass deletion — but it means
a lapsed token stops everything.
## M2 — interface and packaging
- [ ] axum configuration UI, bound to 127.0.0.1
- [ ] OAuth loopback redirect handler
- [ ] systemd user units: `calcalist.service` (oneshot) and `calcalist.timer`

View file

@ -32,10 +32,17 @@ pub enum Command {
},
/// Show endpoints, aggregates and the last sync result
Status,
/// Serve the configuration web interface on localhost
Serve {
#[arg(long, default_value_t = 8723)]
port: u16,
/// List the calendars a CalDAV server offers, as endpoint blocks to paste
Discover {
/// The account or principal URL to enumerate, not a single calendar
url: String,
/// Username to authenticate as
#[arg(long)]
username: String,
/// Command printing the password, exactly as it would appear in the
/// configuration. Secrets are never passed as arguments.
#[arg(long, value_name = "COMMAND")]
secret_command: Option<String>,
},
/// Google account operations
Google {
@ -47,6 +54,12 @@ pub enum Command {
#[command(subcommand)]
command: AggregateCommand,
},
/// Remove local mirrors of endpoints the configuration no longer names
Prune {
/// Delete them; without this the command only reports what it found
#[arg(long)]
force: bool,
},
/// Check that the environment and configuration are usable
Doctor,
}

View file

@ -43,6 +43,13 @@ pub enum EndpointKind {
calendar_id: String,
client_id: String,
client_secret_command: Option<String>,
/// Which authorised Google account this calendar belongs to.
///
/// One authorisation covers every calendar an account can see, so
/// several endpoints on one account share it and this can be left out.
/// It is only needed to say which account is meant when calcalist has
/// been logged in to more than one.
account: Option<String>,
},
Webcal {
/// The feed URL, when it is not sensitive.
@ -68,6 +75,14 @@ impl EndpointKind {
}
}
/// Which authorised Google account this endpoint names, if it says.
pub fn google_account(&self) -> Option<&str> {
match self {
Self::Google { account, .. } => account.as_deref(),
_ => None,
}
}
pub fn scheduling_suppression(&self) -> SchedulingSuppression {
match self {
Self::Google { .. } => SchedulingSuppression::Native,
@ -192,10 +207,20 @@ pub enum Problem {
NoUrlSource { endpoint: String },
#[error("endpoint `{endpoint}` has both `url` and `url_command`; use one or the other")]
AmbiguousUrlSource { endpoint: String },
#[error(
"endpoint `{endpoint}` uses a reserved id: `@{endpoint}` means \"keep this event in the aggregate\", so an endpoint of that name could never be routed to"
)]
ReservedEndpointId { endpoint: String },
}
/// A webcal feed must name exactly one source for its URL.
/// A webcal feed must name exactly one source for its URL, and no endpoint may
/// take a name the routing markers have already spoken for.
fn check_endpoint(endpoint: &Endpoint, problems: &mut Vec<Problem>) {
if crate::mirror::is_local_sink(&endpoint.id) {
problems.push(Problem::ReservedEndpointId {
endpoint: endpoint.id.clone(),
});
}
let EndpointKind::Webcal { url, url_command } = &endpoint.kind else {
return;
};
@ -594,4 +619,25 @@ sources = ["work", "work"]
endpoint: "work".into(),
}));
}
/// `@local` already means "keep this event here", so an endpoint of that
/// name could never be routed to.
#[test]
fn an_endpoint_may_not_take_the_reserved_name() {
let problems = problems(
r#"
[[endpoint]]
id = "local"
type = "caldav"
url = "https://caldav.example.com/other/"
username = "me@example.com"
"#,
);
assert!(
problems
.iter()
.any(|problem| matches!(problem, Problem::ReservedEndpointId { .. })),
"{problems:?}"
);
}
}

View file

@ -143,7 +143,12 @@ fn check_google_authorisation(config: &Config) -> Vec<Check> {
let outcome = match auth::credentials_for(config, &endpoint.id) {
Err(error) => Outcome::Fail(error.to_string()),
Ok(credentials) => {
match auth::access_token(&state_dir, &endpoint.id, &credentials) {
match auth::access_token(
&state_dir,
&endpoint.id,
endpoint.kind.google_account(),
&credentials,
) {
Ok(_) => Outcome::Ok(format!("`{}` is authorised", endpoint.id)),
Err(error @ (AuthError::NotAuthorised(_) | AuthError::Expired { .. })) => {
Outcome::Warn(error.to_string())

View file

@ -161,6 +161,44 @@ impl Client {
Self::event_id(&body)
}
/// The occurrences of a series inside a window.
///
/// Exceptions are addressed through the series rather than created
/// directly, so applying one means finding the instance it replaces.
pub fn instances(
&self,
master_id: &str,
time_min: &str,
time_max: &str,
) -> Result<Vec<Event>, ApiError> {
let path = format!("events/{}/instances", percent_encode(master_id));
let query = [
("maxResults".to_string(), PAGE_SIZE.to_string()),
("timeMin".to_string(), time_min.to_string()),
("timeMax".to_string(), time_max.to_string()),
("showDeleted".to_string(), "false".to_string()),
];
let body = self.get(&path, &query)?;
let page: serde_json::Value =
serde_json::from_str(&body).map_err(|error| ApiError::Malformed(error.to_string()))?;
let items = page
.get("items")
.cloned()
.unwrap_or(serde_json::Value::Array(Vec::new()));
serde_json::from_value(items).map_err(|error| ApiError::Malformed(error.to_string()))
}
/// Merges fields into an existing event, leaving the rest alone.
///
/// An instance carries properties calcalist does not model — its
/// `recurringEventId` and `originalStartTime` above all, which say which
/// occurrence it replaces — so an override is patched rather than replaced.
pub fn patch(&self, id: &str, event: &serde_json::Value) -> Result<(), ApiError> {
let path = format!("events/{}", percent_encode(id));
self.patch_json(&path, &SUPPRESS_NOTIFICATION, event)?;
Ok(())
}
pub fn delete(&self, id: &str) -> Result<(), ApiError> {
let path = format!("events/{}", percent_encode(id));
match self.delete_path(&path, &SUPPRESS_NOTIFICATION) {
@ -223,6 +261,26 @@ impl Client {
)
}
fn patch_json(
&self,
path: &str,
query: &[(&str, &str)],
body: &serde_json::Value,
) -> Result<String, ApiError> {
let mut request = self
.agent
.patch(&self.url(path))
.header("Authorization", format!("Bearer {}", self.access_token));
for (key, value) in query {
request = request.query(*key, *value);
}
Self::finish(
request
.send_json(body)
.map_err(|error| ApiError::Request(error.to_string()))?,
)
}
fn delete_path(&self, path: &str, query: &[(&str, &str)]) -> Result<String, ApiError> {
let mut request = self
.agent
@ -358,7 +416,9 @@ pub struct PushReport {
pub created: usize,
pub updated: usize,
pub deleted: usize,
/// Series whose exceptions could not be applied; see `push`.
/// Recurrence overrides applied to their instances.
pub exceptions_applied: usize,
/// Overrides whose instance Google did not offer; see `apply_exceptions`.
pub exceptions_skipped: usize,
}
@ -388,16 +448,11 @@ pub fn push(
}
};
// A series' exceptions are separate events against an existing series,
// so they cannot ride along with the master. Reported rather than
// silently dropped.
if item
.calendar
.properties("VEVENT", "RECURRENCE-ID")
.next()
.is_some()
{
report.exceptions_skipped += 1;
}
// so they cannot ride along with the master and are applied once it is
// in place.
let applied = apply_exceptions(client, &id, &item.calendar)?;
report.exceptions_applied += applied.applied;
report.exceptions_skipped += applied.skipped;
state.events.insert(
uid.clone(),
GoogleItem {
@ -423,6 +478,46 @@ pub fn push(
Ok(report)
}
#[derive(Debug, Default)]
struct ExceptionReport {
applied: usize,
skipped: usize,
}
/// Applies a stored series' recurrence overrides to the instances they replace.
///
/// Google has no way to send a series and its exceptions in one request: an
/// exception is a separate event, addressed through the series and identified by
/// the occurrence it stands in for. So the master goes up first, then each
/// override is matched to an instance by original start time and patched.
///
/// An override with no matching instance is counted rather than forced. That
/// happens when the occurrence falls outside the rule — a stale `RECURRENCE-ID`
/// left behind by an edited `RRULE` — and inventing an event for it would put
/// something in the calendar that the series does not contain.
fn apply_exceptions(
client: &Client,
master_id: &str,
calendar: &Calendar,
) -> Result<ExceptionReport, ApiError> {
let mut report = ExceptionReport::default();
for exception in convert::exceptions(calendar) {
let Some(key) = convert::recurrence_key(&exception) else {
report.skipped += 1;
continue;
};
let (time_min, time_max) = key.window();
let instances = client.instances(master_id, &time_min, &time_max)?;
let Some(instance) = convert::find_instance(&instances, &key) else {
report.skipped += 1;
continue;
};
client.patch(&instance.id, &convert::to_exception(&exception)?)?;
report.applied += 1;
}
Ok(report)
}
/// Percent-encodes a path segment; calendar ids contain `@` and event ids may
/// contain characters that would otherwise change the URL's meaning.
fn percent_encode(value: &str) -> String {

View file

@ -74,6 +74,21 @@ pub enum AuthError {
NoRefreshToken,
#[error("endpoint `{0}` has not been authorised; run `calcalist google login {0}`")]
NotAuthorised(String),
#[error(
"endpoint `{endpoint}` names account `{account}`, which has not been authorised; \
run `calcalist google login {endpoint}`"
)]
UnknownAccount { endpoint: String, account: String },
#[error(
"endpoint `{endpoint}` does not say which Google account it belongs to, and calcalist \
is logged in to several ({accounts}). Add `account = \"\"` to the endpoint."
)]
AmbiguousAccount { endpoint: String, accounts: String },
#[error(
"the authorisation stored for account `{account}` was issued to a different OAuth \
client than endpoint `{endpoint}` configures; run `calcalist google login {endpoint}`"
)]
ClientMismatch { endpoint: String, account: String },
#[error("could not access {path}: {source}")]
Io {
path: PathBuf,
@ -98,6 +113,15 @@ pub struct StoredToken {
/// Unix seconds at which `access_token` stops being usable.
#[serde(default)]
pub expires_at: u64,
/// The account this authorisation belongs to. Absent in files written
/// before authorisations were shared between endpoints.
#[serde(default)]
pub account: Option<String>,
/// The OAuth client the refresh token was issued to. A refresh token is
/// only valid for the client that obtained it, so a changed `client_id`
/// has to be caught here rather than as an opaque `invalid_grant`.
#[serde(default)]
pub client_id: Option<String>,
}
impl StoredToken {
@ -109,11 +133,93 @@ impl StoredToken {
}
}
/// Where an endpoint's refresh token lives.
pub fn token_path(state_dir: &Path, endpoint_id: &str) -> PathBuf {
/// Where an authorisation lives, keyed by the account it was granted for.
///
/// An authorisation covers every calendar the account can see, so two endpoints
/// on one account share this file and only one of them ever has to log in.
pub fn account_token_path(state_dir: &Path, account: &str) -> PathBuf {
state_dir
.join("google")
.join("accounts")
.join(format!("{}.json", sanitise(account)))
}
/// Where an endpoint's refresh token used to live, before authorisations were
/// keyed by account. Still read, so an existing installation keeps working
/// until its next login moves it across.
pub fn legacy_token_path(state_dir: &Path, endpoint_id: &str) -> PathBuf {
state_dir.join("google").join(format!("{endpoint_id}.json"))
}
/// Keeps an account name to one harmless path segment.
fn sanitise(account: &str) -> String {
account
.chars()
.map(|ch| match ch {
'a'..='z' | 'A'..='Z' | '0'..='9' | '-' | '_' | '.' | '@' => ch,
_ => '_',
})
.collect()
}
/// Every account calcalist currently holds an authorisation for.
pub fn authorised_accounts(state_dir: &Path) -> Vec<String> {
let Ok(entries) = std::fs::read_dir(state_dir.join("google").join("accounts")) else {
return Vec::new();
};
let mut accounts: Vec<String> = entries
.flatten()
.filter_map(|entry| load_token(&entry.path()).ok().flatten())
.filter_map(|token| token.account)
.collect();
accounts.sort();
accounts.dedup();
accounts
}
/// Picks the stored authorisation an endpoint should use.
///
/// The endpoint may name its account outright. Failing that, an authorisation
/// left over from the per-endpoint scheme is honoured, and then a single
/// account is taken to be the one meant — which is the ordinary case, and what
/// makes a second endpoint on the same account need no login of its own.
fn resolve_token(
state_dir: &Path,
endpoint_id: &str,
account: Option<&str>,
) -> Result<(PathBuf, StoredToken), AuthError> {
if let Some(account) = account {
let path = account_token_path(state_dir, account);
let stored = load_token(&path)?.ok_or_else(|| AuthError::UnknownAccount {
endpoint: endpoint_id.to_string(),
account: account.to_string(),
})?;
return Ok((path, stored));
}
let legacy = legacy_token_path(state_dir, endpoint_id);
if let Some(stored) = load_token(&legacy)? {
return Ok((legacy, stored));
}
let accounts = authorised_accounts(state_dir);
match accounts.as_slice() {
[] => Err(AuthError::NotAuthorised(endpoint_id.to_string())),
[only] => {
let path = account_token_path(state_dir, only);
let stored = load_token(&path)?.ok_or_else(|| AuthError::UnknownAccount {
endpoint: endpoint_id.to_string(),
account: only.clone(),
})?;
Ok((path, stored))
}
several => Err(AuthError::AmbiguousAccount {
endpoint: endpoint_id.to_string(),
accounts: several.join(", "),
}),
}
}
/// Runs an external command and returns its first line of output.
///
/// Secrets are fetched this way rather than stored, so the configuration file
@ -181,11 +287,18 @@ pub fn credentials_for(
pub fn access_token(
state_dir: &Path,
endpoint_id: &str,
account: Option<&str>,
credentials: &Credentials,
) -> Result<String, AuthError> {
let path = token_path(state_dir, endpoint_id);
let mut stored =
load_token(&path)?.ok_or_else(|| AuthError::NotAuthorised(endpoint_id.to_string()))?;
let (path, mut stored) = resolve_token(state_dir, endpoint_id, account)?;
if let Some(issued_to) = &stored.client_id
&& *issued_to != credentials.client_id
{
return Err(AuthError::ClientMismatch {
endpoint: endpoint_id.to_string(),
account: stored.account.clone().unwrap_or_else(|| "?".to_string()),
});
}
if let Some(token) = stored.usable_access_token() {
return Ok(token.to_string());
@ -248,19 +361,66 @@ pub fn login(
])?;
let refresh_token = issued.refresh_token.ok_or(AuthError::NoRefreshToken)?;
let path = token_path(state_dir, endpoint_id);
// Which account consented is not in the token response, but the primary
// calendar's id is the account's own address — and reading it needs nothing
// beyond the calendar scope already granted, unlike the userinfo endpoint.
let account = primary_calendar_id(&issued.access_token);
let path = match &account {
Some(account) => account_token_path(state_dir, account),
None => legacy_token_path(state_dir, endpoint_id),
};
save_token(
&path,
&StoredToken {
refresh_token,
access_token: Some(issued.access_token),
expires_at: unix_now() + issued.expires_in,
account: account.clone(),
client_id: Some(credentials.client_id.clone()),
},
)?;
println!("Authorised. Refresh token stored at {}", path.display());
// The authorisation now lives under the account, so the copy this endpoint
// used to keep would only shadow it.
let legacy = legacy_token_path(state_dir, endpoint_id);
if account.is_some() && legacy.exists() {
let _ = std::fs::remove_file(&legacy);
}
match &account {
Some(account) => println!(
"Authorised {account}. Every endpoint on this account can use it; \
the refresh token is stored at {}",
path.display()
),
None => println!("Authorised. Refresh token stored at {}", path.display()),
}
Ok(())
}
/// The address of the authorised account, read from its primary calendar.
///
/// Returned as an option rather than an error: failing to name the account only
/// costs the sharing, and the authorisation itself is perfectly good without it.
fn primary_calendar_id(access_token: &str) -> Option<String> {
let agent: ureq::Agent = ureq::Agent::config_builder()
.http_status_as_error(false)
.build()
.into();
let mut response = agent
.get("https://www.googleapis.com/calendar/v3/calendars/primary")
.header("Authorization", format!("Bearer {access_token}"))
.call()
.ok()?;
if !(200..300).contains(&response.status().as_u16()) {
return None;
}
let body = response.body_mut().read_to_string().ok()?;
let value: serde_json::Value = serde_json::from_str(&body).ok()?;
value
.get("id")
.and_then(|id| id.as_str())
.map(str::to_string)
}
/// Accepts one loopback request and extracts the authorisation code from it.
fn wait_for_code(listener: &TcpListener, expected_state: &str) -> Result<String, AuthError> {
let (mut stream, _) = listener.accept().map_err(AuthError::Listen)?;
@ -586,6 +746,8 @@ mod tests {
refresh_token: "r".into(),
access_token: Some("a".into()),
expires_at: unix_now() + 3600,
account: None,
client_id: None,
};
assert_eq!(fresh.usable_access_token(), Some("a"));
@ -602,4 +764,101 @@ mod tests {
};
assert_eq!(nearly.usable_access_token(), None);
}
fn stored(account: &str, client_id: &str) -> StoredToken {
StoredToken {
refresh_token: format!("refresh-for-{account}"),
access_token: None,
expires_at: 0,
account: Some(account.to_string()),
client_id: Some(client_id.to_string()),
}
}
/// The gap this closes: two endpoints on one account needed a login each,
/// because the authorisation was filed under the endpoint rather than the
/// account it actually belongs to.
#[test]
fn one_authorisation_serves_every_endpoint_on_that_account() {
let dir = tempfile::tempdir().expect("temp");
save_token(
&account_token_path(dir.path(), "me@example.com"),
&stored("me@example.com", "client-a"),
)
.expect("save");
for endpoint in ["work", "personal", "never-logged-in"] {
let (_, token) = resolve_token(dir.path(), endpoint, None).expect("resolve");
assert_eq!(token.refresh_token, "refresh-for-me@example.com");
}
}
/// Two accounts cannot be guessed between, so the endpoint has to say.
#[test]
fn several_accounts_have_to_be_told_apart() {
let dir = tempfile::tempdir().expect("temp");
for account in ["one@example.com", "two@example.com"] {
save_token(
&account_token_path(dir.path(), account),
&stored(account, "client-a"),
)
.expect("save");
}
let error = resolve_token(dir.path(), "work", None).expect_err("ambiguous");
assert!(
matches!(error, AuthError::AmbiguousAccount { .. }),
"{error:?}"
);
assert!(error.to_string().contains("one@example.com"), "{error}");
let (_, token) =
resolve_token(dir.path(), "work", Some("two@example.com")).expect("resolve");
assert_eq!(token.refresh_token, "refresh-for-two@example.com");
}
/// An authorisation written by an earlier version keeps working untouched.
#[test]
fn an_authorisation_from_the_old_layout_is_still_honoured() {
let dir = tempfile::tempdir().expect("temp");
save_token(
&legacy_token_path(dir.path(), "work"),
&StoredToken {
refresh_token: "old".into(),
access_token: None,
expires_at: 0,
account: None,
client_id: None,
},
)
.expect("save");
let (_, token) = resolve_token(dir.path(), "work", None).expect("resolve");
assert_eq!(token.refresh_token, "old");
// A different endpoint has nothing to fall back on.
let error = resolve_token(dir.path(), "other", None).expect_err("not authorised");
assert!(matches!(error, AuthError::NotAuthorised(_)), "{error:?}");
}
/// A refresh token only works for the client it was issued to, so a changed
/// client_id has to say so rather than surface as an opaque invalid_grant.
#[test]
fn a_token_from_another_oauth_client_is_refused_by_name() {
let dir = tempfile::tempdir().expect("temp");
save_token(
&account_token_path(dir.path(), "me@example.com"),
&stored("me@example.com", "client-a"),
)
.expect("save");
let credentials = Credentials {
client_id: "client-b".into(),
client_secret: "secret".into(),
};
let error = access_token(dir.path(), "work", None, &credentials).expect_err("mismatch");
assert!(
matches!(error, AuthError::ClientMismatch { .. }),
"{error:?}"
);
}
}

View file

@ -14,9 +14,9 @@
//! * An exception that was deleted comes back as an override with
//! `status: cancelled`, which is an `EXDATE` on the master rather than an event.
use std::collections::BTreeMap;
use std::collections::{BTreeMap, BTreeSet};
use jiff::{Timestamp, tz::TimeZone};
use jiff::{SignedDuration, Timestamp, tz::Dst, tz::TimeZone};
use serde::Deserialize;
use thiserror::Error;
@ -157,34 +157,186 @@ pub fn to_ical(uid: &str, group: &Group) -> Result<String, ConvertError> {
});
};
let mut out = String::from("BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:-//calcalist//EN\r\n");
out.push_str(&render_event(uid, master, None)?);
let mut zones = BTreeSet::new();
let mut body = render_event(uid, master, None, &mut zones)?;
// A deleted occurrence is an absence, not an event: it belongs on the master
// as an EXDATE.
for cancelled in group.overrides.iter().filter(|event| event.is_cancelled()) {
if let Some(start) = &cancelled.original_start_time {
let line = time_property("EXDATE", start, &cancelled.id, master.is_recurring())?;
let line = time_property(
"EXDATE",
start,
&cancelled.id,
master.is_recurring(),
&mut zones,
)?;
// Insert before the master's END:VEVENT.
let end = out.rfind("END:VEVENT\r\n").unwrap_or(out.len());
out.insert_str(end, &format!("{line}\r\n"));
let end = body.rfind("END:VEVENT\r\n").unwrap_or(body.len());
body.insert_str(end, &format!("{line}\r\n"));
}
}
for moved in group.overrides.iter().filter(|event| !event.is_cancelled()) {
out.push_str(&render_event(
body.push_str(&render_event(
uid,
moved,
moved.original_start_time.as_ref(),
&mut zones,
)?);
}
let mut out = String::from("BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:-//calcalist//EN\r\n");
// A TZID names a zone the reader is expected to resolve, but RFC 5545 wants
// the definition to travel with the calendar: a client with no zone database
// of its own has nothing else to expand the recurrence against.
let reference = master
.start
.as_ref()
.and_then(instant_of)
.unwrap_or_else(Timestamp::now);
for zone in &zones {
if let Some(block) = vtimezone(zone, reference) {
out.push_str(&block);
}
}
out.push_str(&body);
out.push_str("END:VCALENDAR\r\n");
Ok(out)
}
/// The instant an event time names, whether it is dated or timed.
fn instant_of(time: &EventTime) -> Option<Timestamp> {
match (&time.date, &time.date_time) {
(_, Some(value)) => value.parse().ok(),
(Some(date), None) => midnight_utc(date),
(None, None) => None,
}
}
/// An RFC 5545 `VTIMEZONE` for `name`, describing the rules around `reference`.
///
/// Two observances describe every zone in current use — one standard, one
/// daylight — each recurring yearly on the same weekday of the same month. A
/// zone that does not shift gets a single fixed observance instead, which is
/// also the fallback when the transition table says nothing about this period.
fn vtimezone(name: &str, reference: Timestamp) -> Option<String> {
let zone = TimeZone::get(name).ok()?;
// Start a little over a year back, so a full cycle of transitions is in
// view however close the event sits to one of them.
let from = reference
.checked_sub(SignedDuration::from_hours(24 * 400))
.ok()?;
let upcoming: Vec<_> = zone.following(from).take(4).collect();
let mut out = format!("BEGIN:VTIMEZONE\r\nTZID:{name}\r\n");
let daylight = upcoming
.iter()
.find(|transition| transition.dst() == Dst::Yes);
let standard = upcoming
.iter()
.find(|transition| transition.dst() == Dst::No);
match (daylight, standard) {
(Some(daylight), Some(standard)) => {
out.push_str(&observance(&zone, daylight, "DAYLIGHT")?);
out.push_str(&observance(&zone, standard, "STANDARD")?);
}
_ => {
let offset = zone.to_offset(reference);
let text = format_offset(offset);
out.push_str(&format!(
"BEGIN:STANDARD\r\nDTSTART:19700101T000000\r\nTZOFFSETFROM:{text}\r\n\
TZOFFSETTO:{text}\r\nEND:STANDARD\r\n"
));
}
}
out.push_str("END:VTIMEZONE\r\n");
Some(out)
}
/// One `STANDARD` or `DAYLIGHT` block, as the yearly rule that produced it.
fn observance(
zone: &TimeZone,
transition: &jiff::tz::TimeZoneTransition<'_>,
component: &str,
) -> Option<String> {
let to = transition.offset();
// The offset in force a moment earlier is the one the shift moved away from.
let before = transition
.timestamp()
.checked_sub(SignedDuration::from_secs(1))
.ok()?;
let from = zone.to_offset(before);
// A transition's local start is written in the *old* offset, since that is
// the clock a reader is still on when the rule fires.
let local = transition
.timestamp()
.to_zoned(TimeZone::fixed(from))
.datetime();
let mut block = format!(
"BEGIN:{component}\r\nDTSTART:{}\r\nTZOFFSETFROM:{}\r\nTZOFFSETTO:{}\r\n\
RRULE:{}\r\n",
local.strftime("%Y%m%dT%H%M%S"),
format_offset(from),
format_offset(to),
yearly_rule(local)
);
let abbreviation = transition.abbreviation();
if !abbreviation.is_empty() {
block.push_str(&format!("TZNAME:{abbreviation}\r\n"));
}
block.push_str(&format!("END:{component}\r\n"));
Some(block)
}
/// The yearly recurrence a transition date implies, as "the nth weekday of the
/// month" — the form every zone's rules are actually written in. A date within
/// seven days of the month's end is the *last* such weekday rather than a fixed
/// ordinal, which is what keeps the rule stable across years of differing length.
fn yearly_rule(local: jiff::civil::DateTime) -> String {
let day = local.day();
let ordinal = if day + 7 > local.days_in_month() {
-1
} else {
(day - 1) / 7 + 1
};
format!(
"FREQ=YEARLY;BYMONTH={};BYDAY={ordinal}{}",
local.month(),
weekday_code(local.weekday())
)
}
fn weekday_code(weekday: jiff::civil::Weekday) -> &'static str {
match weekday {
jiff::civil::Weekday::Monday => "MO",
jiff::civil::Weekday::Tuesday => "TU",
jiff::civil::Weekday::Wednesday => "WE",
jiff::civil::Weekday::Thursday => "TH",
jiff::civil::Weekday::Friday => "FR",
jiff::civil::Weekday::Saturday => "SA",
jiff::civil::Weekday::Sunday => "SU",
}
}
/// `+0200` — RFC 5545's UTC offset form, which has no colon.
fn format_offset(offset: jiff::tz::Offset) -> String {
let total = offset.seconds();
let sign = if total < 0 { '-' } else { '+' };
let magnitude = total.abs();
format!(
"{sign}{:02}{:02}",
magnitude / 3600,
(magnitude % 3600) / 60
)
}
fn render_event(
uid: &str,
event: &Event,
recurrence_id: Option<&EventTime>,
zones: &mut BTreeSet<String>,
) -> Result<String, ConvertError> {
let recurring = event.is_recurring() || recurrence_id.is_some();
let mut out = String::from("BEGIN:VEVENT\r\n");
@ -197,16 +349,19 @@ fn render_event(
original,
&event.id,
recurring,
zones,
)?);
out.push_str("\r\n");
}
let start = event.start.as_ref().ok_or_else(|| ConvertError::NoStart {
id: event.id.clone(),
})?;
out.push_str(&time_property("DTSTART", start, &event.id, recurring)?);
out.push_str(&time_property(
"DTSTART", start, &event.id, recurring, zones,
)?);
out.push_str("\r\n");
if let Some(end) = &event.end {
out.push_str(&time_property("DTEND", end, &event.id, recurring)?);
out.push_str(&time_property("DTEND", end, &event.id, recurring, zones)?);
out.push_str("\r\n");
}
for line in event.recurrence.iter().flatten() {
@ -269,6 +424,7 @@ fn time_property(
time: &EventTime,
id: &str,
recurring: bool,
zones: &mut BTreeSet<String>,
) -> Result<String, ConvertError> {
if let Some(date) = &time.date {
return Ok(format!("{name};VALUE=DATE:{}", date.replace('-', "")));
@ -292,6 +448,7 @@ fn time_property(
match zone {
Some((zone_name, zone)) => {
zones.insert(zone_name.to_string());
let local = instant.to_zoned(zone);
Ok(format!(
"{name};TZID={zone_name}:{}",
@ -319,18 +476,61 @@ fn stamp(event: &Event) -> Result<String, ConvertError> {
Ok(instant.strftime("%Y%m%dT%H%M%SZ").to_string())
}
/// Which half of a series a body describes.
///
/// Google models an exception as a separate event addressed through the series
/// it belongs to, so it carries neither the series' `iCalUID` — which would
/// name the master — nor a recurrence rule of its own.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum Role {
Master,
Exception,
}
/// Renders a stored calendar as the JSON body Google expects.
///
/// Only the master is described. A series' exceptions cannot be expressed in the
/// same request — Google models them as separate events against an already
/// existing series — so the caller applies those afterwards.
/// same request, so the caller applies those afterwards with `to_exception`.
pub fn to_google(calendar: &Calendar) -> Result<serde_json::Value, ConvertError> {
let uid = calendar.uid().ok_or_else(|| ConvertError::NoUid {
let master = master_of(calendar).ok_or_else(|| ConvertError::NoUid {
id: "<local>".to_string(),
})?;
to_body(&master, Role::Master)
}
/// The master event of a stored file: the one component without a `RECURRENCE-ID`.
///
/// Falls back to the file as a whole when it holds no `VEVENT` at all, so a
/// malformed item still reports through the ordinary error path.
pub fn master_of(calendar: &Calendar) -> Option<Calendar> {
calendar
.events()
.into_iter()
.find(|event| event.properties("VEVENT", "RECURRENCE-ID").next().is_none())
}
/// Every recurrence override in a stored file, in the order they appear.
pub fn exceptions(calendar: &Calendar) -> Vec<Calendar> {
calendar
.events()
.into_iter()
.filter(|event| event.properties("VEVENT", "RECURRENCE-ID").next().is_some())
.collect()
}
/// Renders one recurrence override as a patch against its Google instance.
pub fn to_exception(event: &Calendar) -> Result<serde_json::Value, ConvertError> {
to_body(event, Role::Exception)
}
fn to_body(calendar: &Calendar, role: Role) -> Result<serde_json::Value, ConvertError> {
let mut event = serde_json::Map::new();
event.insert("iCalUID".into(), uid.into());
if role == Role::Master {
let uid = calendar.uid().ok_or_else(|| ConvertError::NoUid {
id: "<local>".to_string(),
})?;
event.insert("iCalUID".into(), uid.into());
}
event.insert("status".into(), status_of(calendar).into());
for (property, field) in [
@ -356,6 +556,7 @@ pub fn to_google(calendar: &Calendar) -> Result<serde_json::Value, ConvertError>
// verbatim. Their unfolded form is what Google expects.
let recurrence: Vec<serde_json::Value> = ["RRULE", "EXDATE", "RDATE"]
.iter()
.filter(|_| role == Role::Master)
.flat_map(|name| calendar.property_lines("VEVENT", name))
.map(|line| serde_json::Value::from(line.to_string()))
.collect();
@ -517,6 +718,83 @@ fn google_time(calendar: &Calendar, name: &str) -> Result<serde_json::Value, Con
Ok(serde_json::Value::Object(time))
}
/// How one occurrence of a series is identified.
///
/// iCalendar names it with `RECURRENCE-ID` and Google with `originalStartTime`.
/// Reducing both to the same key is what lets a stored override be applied to
/// the right instance: the two sides agree on the instant but not on how they
/// spell it, one carrying a zoned local time and the other an absolute offset.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum InstanceKey {
/// An occurrence of an all-day series, as `YYYY-MM-DD`.
Date(String),
Instant(Timestamp),
}
impl InstanceKey {
/// A window around the occurrence, to ask Google for the instances in it.
///
/// Wide enough that an override which moved the event by most of a day, or a
/// series expanding in a zone far from UTC, still falls inside — the match
/// itself is exact, so a generous window costs nothing but a few more rows.
pub fn window(&self) -> (String, String) {
let centre = match self {
InstanceKey::Instant(instant) => *instant,
InstanceKey::Date(date) => midnight_utc(date).unwrap_or_else(Timestamp::now),
};
let margin = jiff::SignedDuration::from_hours(36);
((centre - margin).to_string(), (centre + margin).to_string())
}
}
fn midnight_utc(date: &str) -> Option<Timestamp> {
let date: jiff::civil::Date = date.parse().ok()?;
date.to_zoned(TimeZone::UTC)
.ok()
.map(|zoned| zoned.timestamp())
}
/// The occurrence a stored recurrence override refers to.
pub fn recurrence_key(event: &Calendar) -> Option<InstanceKey> {
let property = event.properties("VEVENT", "RECURRENCE-ID").next()?;
let value = property.value;
if property.param("VALUE") == Some("DATE") || value.len() == 8 {
return Some(InstanceKey::Date(hyphenate(value)));
}
let civil = parse_basic(value, "RECURRENCE-ID").ok()?;
let zone = match property.param("TZID") {
Some(name) => TimeZone::get(name).ok()?,
None => TimeZone::UTC,
};
civil
.to_zoned(zone)
.ok()
.map(|zoned| InstanceKey::Instant(zoned.timestamp()))
}
/// The occurrence a Google override refers to.
pub fn original_start_key(time: &EventTime) -> Option<InstanceKey> {
if let Some(date) = &time.date {
return Some(InstanceKey::Date(date.clone()));
}
time.date_time
.as_deref()?
.parse()
.ok()
.map(InstanceKey::Instant)
}
/// Picks the instance an override belongs to out of a page of them.
pub fn find_instance<'a>(instances: &'a [Event], key: &InstanceKey) -> Option<&'a Event> {
instances.iter().find(|instance| {
instance
.original_start_time
.as_ref()
.and_then(original_start_key)
.is_some_and(|candidate| candidate == *key)
})
}
/// Parses `YYYYMMDDTHHMMSS`, with or without a trailing Z.
fn parse_basic(value: &str, field: &'static str) -> Result<jiff::civil::DateTime, ConvertError> {
let trimmed = value.trim_end_matches('Z');
@ -860,4 +1138,154 @@ mod tests {
"{ics}"
);
}
/// A zone that shifts gets both observances, each as the yearly rule the
/// transition implies: the EU moves on the last Sunday of March and October.
#[test]
fn a_zoned_recurrence_carries_its_timezone_definition() {
let json = r#"[{
"id": "berlin", "status": "confirmed", "summary": "Standup",
"start": {"dateTime": "2026-03-03T09:00:00+01:00", "timeZone": "Europe/Berlin"},
"end": {"dateTime": "2026-03-03T09:30:00+01:00", "timeZone": "Europe/Berlin"},
"recurrence": ["RRULE:FREQ=WEEKLY;BYDAY=TU"],
"iCalUID": "berlin@google.com", "updated": "2026-01-01T00:00:00.000Z"
}]"#;
let groups = group_by_uid(parse(json));
let ics = to_ical("berlin@google.com", &groups["berlin@google.com"]).expect("convert");
assert!(
ics.contains("BEGIN:VTIMEZONE\r\nTZID:Europe/Berlin\r\n"),
"{ics}"
);
assert!(ics.contains("BEGIN:DAYLIGHT"), "{ics}");
assert!(
ics.contains("TZOFFSETFROM:+0100\r\nTZOFFSETTO:+0200"),
"{ics}"
);
assert!(
ics.contains("RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU"),
"{ics}"
);
assert!(ics.contains("BEGIN:STANDARD"), "{ics}");
assert!(
ics.contains("TZOFFSETFROM:+0200\r\nTZOFFSETTO:+0100"),
"{ics}"
);
assert!(
ics.contains("RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU"),
"{ics}"
);
// The definition has to precede the event that references it.
assert!(
ics.find("BEGIN:VTIMEZONE") < ics.find("BEGIN:VEVENT"),
"{ics}"
);
assert!(Calendar::parse(&ics).is_ok(), "{ics}");
}
/// Pakistan abandoned daylight saving in 2009, so there is no rule to state —
/// only the offset that has applied ever since.
#[test]
fn a_zone_that_never_shifts_gets_one_fixed_observance() {
let groups = group_by_uid(parse(KARACHI_MASTER));
let ics = to_ical("master1@google.com", &groups["master1@google.com"]).expect("convert");
assert!(ics.contains("TZID:Asia/Karachi"), "{ics}");
assert!(
ics.contains("TZOFFSETFROM:+0500\r\nTZOFFSETTO:+0500"),
"{ics}"
);
assert!(!ics.contains("BEGIN:DAYLIGHT"), "{ics}");
}
/// A one-off is written in UTC, so it names no zone and needs no definition.
#[test]
fn an_unzoned_event_carries_no_timezone_definition() {
let json = r#"[{
"id": "one", "status": "confirmed", "summary": "Once",
"start": {"dateTime": "2026-04-01T10:00:00+02:00", "timeZone": "Europe/Berlin"},
"end": {"dateTime": "2026-04-01T11:00:00+02:00", "timeZone": "Europe/Berlin"},
"iCalUID": "one@google.com", "updated": "2026-01-01T00:00:00.000Z"
}]"#;
let groups = group_by_uid(parse(json));
let ics = to_ical("one@google.com", &groups["one@google.com"]).expect("convert");
assert!(!ics.contains("VTIMEZONE"), "{ics}");
}
/// The master's rule must not leak into the override's body, or Google would
/// read the exception as a series of its own.
#[test]
fn an_exception_body_carries_no_recurrence_and_no_uid() {
let ics = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\n\
BEGIN:VEVENT\r\nUID:series@calcalist\r\nDTSTAMP:20260101T000000Z\r\n\
DTSTART;TZID=Europe/Berlin:20260303T090000\r\nRRULE:FREQ=WEEKLY;BYDAY=TU\r\n\
SUMMARY:Standup\r\nEND:VEVENT\r\n\
BEGIN:VEVENT\r\nUID:series@calcalist\r\nDTSTAMP:20260101T000000Z\r\n\
RECURRENCE-ID;TZID=Europe/Berlin:20260310T090000\r\n\
DTSTART;TZID=Europe/Berlin:20260310T110000\r\n\
SUMMARY:Standup, moved\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n";
let calendar = Calendar::parse(ics).expect("parse");
let exceptions = exceptions(&calendar);
assert_eq!(exceptions.len(), 1);
let body = to_exception(&exceptions[0]).expect("convert");
assert!(body.get("recurrence").is_none(), "{body}");
assert!(body.get("iCalUID").is_none(), "{body}");
assert_eq!(body["summary"], "Standup, moved");
assert_eq!(body["start"]["dateTime"], "2026-03-10T10:00:00Z");
// The master keeps its own summary rather than the override's.
let master = to_google(&calendar).expect("convert");
assert_eq!(master["summary"], "Standup");
assert_eq!(master["iCalUID"], "series@calcalist");
assert_eq!(master["recurrence"][0], "RRULE:FREQ=WEEKLY;BYDAY=TU");
}
/// RECURRENCE-ID and originalStartTime spell the same instant differently:
/// one as a zoned local time, the other as an absolute offset.
#[test]
fn an_override_matches_its_instance_across_the_two_spellings() {
let ics = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\n\
BEGIN:VEVENT\r\nUID:s@calcalist\r\nDTSTAMP:20260101T000000Z\r\n\
RECURRENCE-ID;TZID=Europe/Berlin:20260310T090000\r\n\
DTSTART;TZID=Europe/Berlin:20260310T110000\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n";
let calendar = Calendar::parse(ics).expect("parse");
let key = recurrence_key(&calendar).expect("a recurrence id");
let instances = parse(
r#"[
{"id": "abc_20260303T080000Z", "status": "confirmed",
"start": {"dateTime": "2026-03-03T09:00:00+01:00"},
"originalStartTime": {"dateTime": "2026-03-03T09:00:00+01:00"}},
{"id": "abc_20260310T080000Z", "status": "confirmed",
"start": {"dateTime": "2026-03-10T11:00:00+01:00"},
"originalStartTime": {"dateTime": "2026-03-10T09:00:00+01:00"}}
]"#,
);
let found = find_instance(&instances, &key).expect("a matching instance");
assert_eq!(found.id, "abc_20260310T080000Z");
// The window has to contain the occurrence it was built for.
let (min, max) = key.window();
assert!(min.as_str() < "2026-03-10T08:00:00Z", "{min}");
assert!(max.as_str() > "2026-03-10T08:00:00Z", "{max}");
}
/// An all-day series identifies its occurrences by date on both sides.
#[test]
fn an_all_day_override_matches_by_date() {
let ics = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\n\
BEGIN:VEVENT\r\nUID:s@calcalist\r\nDTSTAMP:20260101T000000Z\r\n\
RECURRENCE-ID;VALUE=DATE:20260310\r\n\
DTSTART;VALUE=DATE:20260311\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n";
let calendar = Calendar::parse(ics).expect("parse");
let key = recurrence_key(&calendar).expect("a recurrence id");
assert_eq!(key, InstanceKey::Date("2026-03-10".into()));
let instances = parse(
r#"[{"id": "abc_20260310", "status": "confirmed",
"start": {"date": "2026-03-11"},
"originalStartTime": {"date": "2026-03-10"}}]"#,
);
assert!(find_instance(&instances, &key).is_some());
}
}

View file

@ -134,15 +134,62 @@ impl Calendar {
pub fn to_ics(&self) -> String {
let mut out = String::with_capacity(self.source.len() + 64);
for line in &self.lines {
match &line.body {
Body::Original(span) => out.push_str(&self.source[span.clone()]),
Body::Generated(text) => out.push_str(&fold(text, &self.newline)),
}
out.push_str(&self.rendered(line));
out.push_str(&self.newline);
}
out
}
/// One line as it will be emitted: verbatim when untouched, folded when built.
fn rendered(&self, line: &Line) -> String {
match &line.body {
Body::Original(span) => self.source[span.clone()].to_string(),
Body::Generated(text) => fold(text, &self.newline),
}
}
/// Each `VEVENT` as a calendar of its own, nested components included.
///
/// `properties` deliberately flattens every component of a name together,
/// which is right for the UID a series shares but wrong for a recurrence
/// override: there its `SUMMARY` and the master's are indistinguishable.
/// Splitting gives each component a view of its own. It re-parses, unlike
/// the rest of this module, which is acceptable because the only caller
/// converts to another model rather than round-tripping back to iCalendar.
pub fn events(&self) -> Vec<Calendar> {
let newline = &self.newline;
let mut out = Vec::new();
let mut block: Option<String> = None;
for line in &self.lines {
let Some(depth) = line
.path
.iter()
.position(|name| name.eq_ignore_ascii_case("VEVENT"))
else {
continue;
};
let outermost = depth + 1 == line.path.len();
if outermost && line.has_name("BEGIN") {
block = Some(String::new());
}
let Some(text) = block.as_mut() else {
continue;
};
text.push_str(&self.rendered(line));
text.push_str(newline);
if outermost
&& line.has_name("END")
&& let Some(text) = block.take()
&& let Ok(event) = Calendar::parse(&format!(
"BEGIN:VCALENDAR{newline}VERSION:2.0{newline}{text}END:VCALENDAR{newline}"
))
{
out.push(event);
}
}
out
}
/// The UID shared by the event and any recurrence overrides.
pub fn uid(&self) -> Option<&str> {
self.properties("VEVENT", "UID").next().map(|p| p.value)
@ -678,4 +725,47 @@ mod tests {
IcalError::DanglingContinuation { line: 1 }
);
}
/// Splitting has to keep each component's nested parts with it, and give a
/// master and its override separate views of the properties they share.
#[test]
fn events_are_split_with_their_nested_components() {
let ics = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\n\
BEGIN:VEVENT\r\nUID:s@x\r\nSUMMARY:Series\r\n\
BEGIN:VALARM\r\nACTION:DISPLAY\r\nTRIGGER:-PT10M\r\nEND:VALARM\r\n\
END:VEVENT\r\n\
BEGIN:VEVENT\r\nUID:s@x\r\nRECURRENCE-ID:20260310T090000Z\r\n\
SUMMARY:Moved\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n";
let calendar = Calendar::parse(ics).expect("parse");
// Flattened, the two summaries are indistinguishable.
assert_eq!(calendar.properties("VEVENT", "SUMMARY").count(), 2);
let events = calendar.events();
assert_eq!(events.len(), 2);
assert_eq!(
events[0]
.properties("VEVENT", "SUMMARY")
.next()
.unwrap()
.value,
"Series"
);
assert_eq!(events[0].properties("VALARM", "TRIGGER").count(), 1);
assert!(
events[0]
.properties("VEVENT", "RECURRENCE-ID")
.next()
.is_none()
);
assert_eq!(
events[1]
.properties("VEVENT", "SUMMARY")
.next()
.unwrap()
.value,
"Moved"
);
assert_eq!(events[1].properties("VALARM", "TRIGGER").count(), 0);
}
}

144
src/lock.rs Normal file
View file

@ -0,0 +1,144 @@
//! A lock held for the duration of anything that writes.
//!
//! A cycle rewrites vdirs and the state file, and two of them running at once
//! would interleave those writes — the timer firing while someone runs `sync` by
//! hand is the ordinary way that happens. `flock` is used rather than a pid file
//! because the kernel releases it when the process ends, however it ends, so a
//! crash cannot leave a lock behind for someone to clear by hand.
use std::fs::{File, OpenOptions};
use std::io::{Read, Seek, SeekFrom, Write};
use std::path::{Path, PathBuf};
use rustix::fs::{FlockOperation, flock};
use thiserror::Error;
const FILE_NAME: &str = "lock";
#[derive(Debug, Error)]
pub enum LockError {
#[error("could not open {path}: {source}")]
Open {
path: PathBuf,
#[source]
source: std::io::Error,
},
#[error(
"another calcalist is already running (pid {pid}); it holds {path}. \
Wait for it to finish rather than running two cycles at once."
)]
Held { pid: String, path: PathBuf },
#[error("could not lock {path}: {source}")]
Failed {
path: PathBuf,
#[source]
source: std::io::Error,
},
}
/// An exclusive claim on the state directory, released when dropped.
///
/// The lock lives in the open file descriptor, so there is nothing to undo: it
/// goes when the `File` closes. The file itself is deliberately left behind —
/// unlinking it would let a waiter take a lock on an inode nobody else can see.
#[derive(Debug)]
pub struct Lock {
_file: File,
}
pub fn path(state_dir: &Path) -> PathBuf {
state_dir.join(FILE_NAME)
}
/// Claims the state directory, or reports who already has it.
pub fn acquire(state_dir: &Path) -> Result<Lock, LockError> {
let path = path(state_dir);
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent).map_err(|source| LockError::Open {
path: path.clone(),
source,
})?;
}
let mut file = OpenOptions::new()
.read(true)
.write(true)
.create(true)
.truncate(false)
.open(&path)
.map_err(|source| LockError::Open {
path: path.clone(),
source,
})?;
if let Err(error) = flock(&file, FlockOperation::NonBlockingLockExclusive) {
if error.kind() == std::io::ErrorKind::WouldBlock {
// Whoever holds it is alive: the kernel would have released the lock
// otherwise. So the pid in the file is worth reporting.
return Err(LockError::Held {
pid: holder(&mut file),
path,
});
}
return Err(LockError::Failed {
path,
source: error.into(),
});
}
// Record who holds it, for the benefit of whoever is refused next.
let _ = file.set_len(0);
let _ = file.seek(SeekFrom::Start(0));
let _ = write!(file, "{}", std::process::id());
let _ = file.flush();
Ok(Lock { _file: file })
}
/// The pid recorded in the lock file, or a placeholder when it says nothing.
fn holder(file: &mut File) -> String {
let mut recorded = String::new();
if file.seek(SeekFrom::Start(0)).is_ok()
&& file.read_to_string(&mut recorded).is_ok()
&& !recorded.trim().is_empty()
{
return recorded.trim().to_string();
}
"unknown".to_string()
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_second_claim_is_refused_while_the_first_is_held() {
let dir = tempfile::tempdir().expect("temp");
let first = acquire(dir.path()).expect("the first claim should succeed");
let error = acquire(dir.path()).expect_err("the second should be refused");
assert!(matches!(error, LockError::Held { .. }), "{error:?}");
// The refusal has to name the holder, or it is not actionable.
assert!(
error.to_string().contains(&std::process::id().to_string()),
"{error}"
);
drop(first);
acquire(dir.path()).expect("the lock should be free again");
}
/// The file has to survive, so the next run locks the same inode.
#[test]
fn the_lock_file_outlives_the_lock() {
let dir = tempfile::tempdir().expect("temp");
drop(acquire(dir.path()).expect("claim"));
assert!(path(dir.path()).exists());
}
#[test]
fn a_missing_state_directory_is_created() {
let dir = tempfile::tempdir().expect("temp");
let nested = dir.path().join("not").join("there");
acquire(&nested).expect("claim");
assert!(path(&nested).exists());
}
}

View file

@ -5,6 +5,7 @@ mod config;
mod doctor;
mod google;
mod ical;
mod lock;
mod mirror;
mod paths;
mod pimsync;
@ -12,6 +13,7 @@ mod provenance;
mod reconcile;
mod retarget;
mod state;
mod status;
mod sync;
mod vdir;
@ -30,10 +32,27 @@ fn main() -> ExitCode {
command: Command::Doctor,
..
} => run_doctor(cli),
cli @ Cli {
command: Command::Status,
..
} => run_status(cli),
cli @ Cli {
command: Command::Sync { dry_run, force },
..
} => run_sync(cli, *dry_run, *force),
cli @ Cli {
command: Command::Prune { force },
..
} => run_prune(cli, *force),
Cli {
command:
Command::Discover {
url,
username,
secret_command,
},
..
} => run_discover(url, username, secret_command.as_deref()),
cli @ Cli {
command:
Command::Google {
@ -51,7 +70,6 @@ fn main() -> ExitCode {
},
..
} => run_retarget(cli, id, to, *purge_old),
Cli { command, .. } => unimplemented(command),
}
}
@ -76,15 +94,128 @@ fn run_sync(cli: &Cli, dry_run: bool, force: bool) -> ExitCode {
Ok(dir) => dir,
Err(error) => return fail(&error),
};
// Held for the whole cycle: a timer firing mid-run would otherwise interleave
// its writes with this one over the same vdirs.
let _lock = match lock::acquire(&state_dir) {
Ok(lock) => lock,
Err(error) => return fail(&error),
};
match sync::run(&config, &state_dir, dry_run, force) {
Ok(report) => {
print_report(&report);
ExitCode::SUCCESS
// A cycle that could not reach an endpoint did real work, but saying
// so only in passing would let a lapsed token go unnoticed until the
// calendars had drifted a long way apart.
if report.is_complete() {
ExitCode::SUCCESS
} else {
ExitCode::FAILURE
}
}
Err(error) => fail(&error),
}
}
/// Lists a server's calendars as configuration ready to paste.
///
/// The URL of a CalDAV calendar is the most error-prone thing in the config: it
/// must name the collection exactly, and providers rarely show it. Asking the
/// server is more reliable than reading a web interface.
fn run_discover(url: &str, username: &str, secret_command: Option<&str>) -> ExitCode {
let state_dir = match paths::state_dir() {
Ok(dir) => dir,
Err(error) => return fail(&error),
};
let found = match pimsync::discover(&state_dir, url, username, secret_command) {
Ok(found) => found,
Err(error) => return fail(&error),
};
if found.is_empty() {
eprintln!("calcalist: {url} offered no calendars");
return ExitCode::FAILURE;
}
let base = url.split_once("://").and_then(|(scheme, rest)| {
rest.split_once('/')
.map(|(authority, _)| format!("{scheme}://{authority}"))
});
println!(
"# {} calendar(s) found. Paste what you want into calcalist.toml.",
found.len()
);
// Some servers name calendars with opaque ids — Posteo's look like
// `cydhlw` — and the id is what an event carries after `@` to be routed
// here, so it is worth renaming before it ends up in anyone's notes.
println!("# Ids come from the URL. Rename them: an id is what you type after `@`.");
for collection in &found {
let full = match &base {
Some(base) => format!("{base}{}", collection.href),
// pimsync reported something that is not a path; show it as it came
// rather than gluing it onto an origin and inventing a URL.
None => collection.href.clone(),
};
println!("\n[[endpoint]]");
println!("id = {}", quoted(&collection.id));
println!("type = \"caldav\"");
println!("url = {}", quoted(&full));
println!("username = {}", quoted(username));
match secret_command {
Some(command) => println!("secret_command = {}", quoted(command)),
None => println!("# secret_command = \"secret-tool lookup ...\""),
}
}
ExitCode::SUCCESS
}
/// A TOML basic string, so a value containing quotes or backslashes survives.
fn quoted(value: &str) -> String {
format!("\"{}\"", value.replace('\\', "\\\\").replace('"', "\\\""))
}
/// Removes local mirrors of endpoints the configuration no longer names.
fn run_prune(cli: &Cli, force: bool) -> ExitCode {
let (config, _) = match Config::load(cli.config.as_deref()) {
Ok(loaded) => loaded,
Err(error) => return fail(&error),
};
let state_dir = match paths::state_dir() {
Ok(dir) => dir,
Err(error) => return fail(&error),
};
// Only the destructive half needs the lock; listing reads nothing a cycle
// could be part-way through changing.
let _lock = if force {
match lock::acquire(&state_dir) {
Ok(lock) => Some(lock),
Err(error) => return fail(&error),
}
} else {
None
};
let orphans = sync::orphan_vdirs(&config, &state_dir);
if orphans.is_empty() {
println!("nothing to prune: every local mirror belongs to a configured endpoint");
return ExitCode::SUCCESS;
}
for orphan in &orphans {
if !force {
println!("would remove {}", orphan.display());
continue;
}
match std::fs::remove_dir_all(orphan) {
Ok(()) => println!("removed {}", orphan.display()),
Err(error) => {
eprintln!("calcalist: could not remove {}: {error}", orphan.display());
return ExitCode::FAILURE;
}
}
}
if !force {
println!("re-run with --force to remove them");
}
ExitCode::SUCCESS
}
/// Authorises one Google endpoint, storing a refresh token for later cycles.
fn run_google_login(cli: &Cli, endpoint_id: &str) -> ExitCode {
let (config, _) = match Config::load(cli.config.as_deref()) {
@ -105,6 +236,25 @@ fn run_google_login(cli: &Cli, endpoint_id: &str) -> ExitCode {
}
}
/// Shows what is configured, and what an event's notes may say to steer it.
fn run_status(cli: &Cli) -> ExitCode {
let (config, _) = match Config::load(cli.config.as_deref()) {
Ok(loaded) => loaded,
Err(error) => return fail(&error),
};
let state_dir = match paths::state_dir() {
Ok(dir) => dir,
Err(error) => return fail(&error),
};
match status::render(&config, &state_dir) {
Ok(report) => {
print!("{report}");
ExitCode::SUCCESS
}
Err(error) => fail(&error),
}
}
/// Moves an aggregate to a different target endpoint, deliberately.
fn run_retarget(cli: &Cli, id: &str, to: &str, purge_old: bool) -> ExitCode {
let (config, _) = match Config::load(cli.config.as_deref()) {
@ -115,6 +265,10 @@ fn run_retarget(cli: &Cli, id: &str, to: &str, purge_old: bool) -> ExitCode {
Ok(dir) => dir,
Err(error) => return fail(&error),
};
let _lock = match lock::acquire(&state_dir) {
Ok(lock) => lock,
Err(error) => return fail(&error),
};
match retarget::retarget(&config, &state_dir, id, to, purge_old) {
Ok(outcome) => {
println!(
@ -126,6 +280,12 @@ fn run_retarget(cli: &Cli, id: &str, to: &str, purge_old: bool) -> ExitCode {
outcome.routed_first,
outcome.purged,
);
if outcome.carried > 0 {
println!(
" {} event(s) belonging to the aggregate itself were moved across",
outcome.carried
);
}
if outcome.purged == 0 && !purge_old {
println!(
" the previous target still holds this aggregate's events; re-run with --purge-old to remove them"
@ -138,6 +298,27 @@ fn run_retarget(cli: &Cli, id: &str, to: &str, purge_old: bool) -> ExitCode {
}
fn print_report(report: &Report) {
for failure in &report.failures {
println!("{}: unavailable — {}", failure.endpoint, failure.reason);
}
for blocked in &report.blocked {
println!(
"{}: left alone; {} could not be reached this cycle",
blocked.id,
blocked
.endpoints
.iter()
.map(|id| format!("`{id}`"))
.collect::<Vec<_>>()
.join(", ")
);
}
for orphan in &report.orphan_vdirs {
println!(
"note: {} mirrors an endpoint that is no longer configured; `calcalist prune` removes it",
orphan.display()
);
}
for google in &report.google {
let resync = if google.full_resync {
" (Google rejected the sync cursor, so everything was refetched)"
@ -153,9 +334,15 @@ fn print_report(report: &Report) {
google.updated_remotely,
google.deleted_remotely,
);
if google.exceptions_applied > 0 {
println!(
" {} recurrence override(s) applied to their instances",
google.exceptions_applied
);
}
if google.exceptions_skipped > 0 {
println!(
" {} recurring series had exceptions that were not pushed; Google models those as separate events against an existing series",
" {} recurrence override(s) matched no occurrence of their series and were left alone",
google.exceptions_skipped
);
}
@ -173,6 +360,12 @@ fn print_report(report: &Report) {
aggregate.deleted_from_aggregate,
aggregate.deleted_from_sources,
);
if aggregate.kept_local > 0 {
println!(
" {} event(s) kept local: they belong to the aggregate itself, not to a source",
aggregate.kept_local
);
}
for conflict in &aggregate.conflicts {
println!(
" conflict: {} changed in both places; kept the version from `{}`",
@ -184,15 +377,12 @@ fn print_report(report: &Report) {
}
}
if report.dry_run {
println!("(dry run — nothing was written)");
println!("(dry run — pulled into a throwaway copy; nothing was written)");
}
}
fn describe(skipped: &reconcile::Skipped) -> String {
match skipped {
reconcile::Skipped::NoSink { aggregate_uid } => format!(
"`{aggregate_uid}` was created in the aggregate, but no default_sink is set to route it to"
),
reconcile::Skipped::ReadOnlySource {
aggregate_uid,
source_id,
@ -202,8 +392,14 @@ fn describe(skipped: &reconcile::Skipped) -> String {
reconcile::Skipped::UnknownSink {
aggregate_uid,
requested,
available,
} => format!(
"`{aggregate_uid}` asked to be filed under `{requested}`, which is not a writable source of this aggregate; it was left where it is"
"`{aggregate_uid}` asked to be filed under `@{requested}`, which is not a writable source of this aggregate. It was left where it is. Try one of: {}",
available
.iter()
.map(|name| format!("@{name}"))
.collect::<Vec<_>>()
.join(", ")
),
}
}
@ -218,17 +414,3 @@ fn fail(error: &dyn std::error::Error) -> ExitCode {
}
ExitCode::FAILURE
}
/// M1 is still landing: `serve`, `status`, `google` and `aggregate` come later.
fn unimplemented(command: &Command) -> ExitCode {
let name = match command {
Command::Sync { .. } => "sync",
Command::Status => "status",
Command::Serve { .. } => "serve",
Command::Google { .. } => "google",
Command::Aggregate { .. } => "aggregate",
Command::Doctor => "doctor",
};
eprintln!("calcalist: `{name}` is not implemented yet");
ExitCode::from(2)
}

View file

@ -30,6 +30,15 @@ const OWN_PROPERTIES: &[&str] = &[SOURCE_PROPERTY, ORIGIN_UID_PROPERTY, ATTENDEE
/// Live scheduling properties, whose presence is what makes a server send mail.
const SCHEDULING_PROPERTIES: &[&str] = &["ATTENDEE", "ORGANIZER"];
/// The marker that keeps an event in the aggregate rather than filing it under
/// a source. Reserved: an endpoint may not be given this id.
pub const LOCAL_SINK: &str = "local";
/// Whether a routing hint asks for the event to stay where it is.
pub fn is_local_sink(hint: &str) -> bool {
hint.eq_ignore_ascii_case(LOCAL_SINK)
}
/// Builds the aggregate copy of a source event.
pub fn to_aggregate(
source: &Calendar,
@ -52,6 +61,24 @@ pub fn to_aggregate(
mirrored
}
/// Applies a target's scheduling rule to an event that is not a mirror.
///
/// An event living only in the aggregate still has to obey the rule that writes
/// to an aggregate never emit scheduling mail — moving one between calendars is
/// as capable of mailing a guest list as mirroring is. It carries no provenance,
/// having none, so this is the demotion alone.
pub fn make_inert(
calendar: &Calendar,
owner: Option<&str>,
suppression: SchedulingSuppression,
) -> Calendar {
let mut inert = calendar.clone();
if suppression == SchedulingSuppression::None {
demote_attendees(&mut inert, owner);
}
inert
}
/// Removes the live guest list, keeping its information in inert form.
fn demote_attendees(calendar: &mut Calendar, owner: Option<&str>) {
let guests: Vec<String> = calendar

View file

@ -106,16 +106,31 @@ pub fn config_path(state_dir: &Path) -> PathBuf {
state_dir.join("pimsync.conf")
}
fn status_path(state_dir: &Path) -> PathBuf {
pub(crate) fn status_path(state_dir: &Path) -> PathBuf {
state_dir.join("pimsync-status")
}
/// Which way a generated configuration lets items travel.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Direction {
/// The ordinary cycle: remote and local converge on each other.
Both,
/// Remote to local only, for `--dry-run`. pimsync writes nothing to any
/// server in this mode — but it does delete local items the remote has
/// never seen, so it is only ever pointed at a throwaway copy of the vdirs.
PullOnly,
}
/// Builds a pimsync configuration covering every CalDAV and WebCal endpoint.
///
/// Google endpoints are absent by design: pimsync has no REST storage, and its
/// CalDAV storage speaks only HTTP Basic auth, which Google's endpoint has
/// rejected since March 2025. calcalist syncs those itself.
pub fn generate(config: &Config, state_dir: &Path) -> Result<String, PimsyncError> {
pub fn generate_for(
config: &Config,
state_dir: &Path,
direction: Direction,
) -> Result<String, PimsyncError> {
let mut out = String::new();
out.push_str("# Generated by calcalist. Edits here are overwritten on every sync.\n\n");
out.push_str(&format!(
@ -129,7 +144,7 @@ pub fn generate(config: &Config, state_dir: &Path) -> Result<String, PimsyncErro
for endpoint in &config.endpoints {
match &endpoint.kind {
EndpointKind::Caldav { .. } => out.push_str(&caldav_pair(endpoint)?),
EndpointKind::Caldav { .. } => out.push_str(&caldav_pair(endpoint, direction)?),
EndpointKind::Webcal { url, url_command } => {
out.push_str(&webcal_pair(
endpoint,
@ -144,7 +159,7 @@ pub fn generate(config: &Config, state_dir: &Path) -> Result<String, PimsyncErro
Ok(out)
}
fn caldav_pair(endpoint: &Endpoint) -> Result<String, PimsyncError> {
fn caldav_pair(endpoint: &Endpoint, direction: Direction) -> Result<String, PimsyncError> {
let EndpointKind::Caldav {
url,
username,
@ -175,10 +190,15 @@ fn caldav_pair(endpoint: &Endpoint) -> Result<String, PimsyncError> {
// The remote is storage_a, so a concurrent server-side change wins over a
// local one calcalist has not yet reconciled. Nothing is lost: the next
// cycle sees the server's version and decides properly.
let resolution = match direction {
// `one_way` never produces a conflict, so there is nothing to resolve.
Direction::PullOnly => "\tone_way\n",
Direction::Both => "\tconflict_resolution keep a\n",
};
block.push_str(&format!(
"\npair {id} {{\n\tstorage_a {id}_remote\n\tstorage_b {LOCAL_STORAGE}\n\
\tcollection {{\n\t\talias {id}\n\t\thref_a {}\n\t\tid_b {id}\n\t}}\n\
\tconflict_resolution keep a\n\ton_empty skip\n\ton_delete skip\n}}\n",
{resolution}\ton_empty skip\n\ton_delete skip\n}}\n",
quote(&href)
));
Ok(block)
@ -235,8 +255,16 @@ fn quote(value: &str) -> String {
/// Writes the generated configuration, returning its path.
pub fn write_config(config: &Config, state_dir: &Path) -> Result<PathBuf, PimsyncError> {
write_config_for(config, state_dir, Direction::Both)
}
pub fn write_config_for(
config: &Config,
state_dir: &Path,
direction: Direction,
) -> Result<PathBuf, PimsyncError> {
let path = config_path(state_dir);
let text = generate(config, state_dir)?;
let text = generate_for(config, state_dir, direction)?;
fs::create_dir_all(status_path(state_dir)).map_err(|source| PimsyncError::Write {
path: status_path(state_dir),
source,
@ -248,6 +276,172 @@ pub fn write_config(config: &Config, state_dir: &Path) -> Result<PathBuf, Pimsyn
Ok(path)
}
/// The storage name the probe config gives the server being examined. Parsing
/// anchors on it, because pimsync discovers *both* storages in a pair and prints
/// their collections under one heading each.
const PROBE_REMOTE: &str = "probe_remote";
/// One collection found on a server.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Collection {
/// The path pimsync reported, e.g. `/calendars/me/work/`.
pub href: String,
/// A usable endpoint id, taken from the last meaningful path segment.
pub id: String,
}
/// Asks pimsync which calendars a server offers.
///
/// `url` is a container to enumerate — an account or principal path — rather
/// than a single calendar, so `split_collection_url` does not apply here.
///
/// `pimsync discover` is documented as informational and as not altering
/// pimsync's state, so pointing it at a server the user has not configured yet
/// is safe. The probe configuration it needs is written under a name of its own
/// so it can never be mistaken for, or overwrite, the generated `pimsync.conf`.
pub fn discover(
state_dir: &Path,
url: &str,
username: &str,
secret_command: Option<&str>,
) -> Result<Vec<Collection>, PimsyncError> {
let config_path = state_dir.join("discover.conf");
let scratch = state_dir.join("discover-vdir");
fs::create_dir_all(&scratch).map_err(|source| PimsyncError::Write {
path: scratch.clone(),
source,
})?;
fs::write(
&config_path,
probe_config(state_dir, &scratch, url, username, secret_command),
)
.map_err(|source| PimsyncError::Write {
path: config_path.clone(),
source,
})?;
let output = Command::new(BINARY)
.arg("-c")
.arg(&config_path)
.arg("discover")
.output()
.map_err(|error| {
if error.kind() == io::ErrorKind::NotFound {
PimsyncError::NotFound
} else {
PimsyncError::Spawn(error)
}
})?;
// Everything, including the failures, comes back on stdout.
let text = String::from_utf8_lossy(&output.stdout).into_owned();
let _ = fs::remove_dir_all(&scratch);
if !output.status.success() {
return Err(PimsyncError::Command {
command: "discover".to_string(),
message: strip_ansi(&text).trim().to_string(),
});
}
Ok(parse_discovery(&text))
}
fn probe_config(
state_dir: &Path,
scratch: &Path,
url: &str,
username: &str,
secret_command: Option<&str>,
) -> String {
let password = match secret_command {
Some(command) => format!("\tpassword {{\n\t\tshell {command}\n\t}}\n"),
None => String::new(),
};
format!(
"# Generated by `calcalist discover`. Rewritten on every run.\n\
status_path {}\n\n\
storage local {{\n\ttype vdir/icalendar\n\tpath {}\n\tfileext ics\n}}\n\n\
storage {PROBE_REMOTE} {{\n\ttype caldav\n\turl {}\n\tdiscovery collections\n\
\tusername {}\n{password}}}\n\n\
pair probe {{\n\tstorage_a {PROBE_REMOTE}\n\tstorage_b local\n\
\tcollections from a\n}}\n",
quote(&state_dir.join("discover-status").display().to_string()),
quote(&scratch.display().to_string()),
quote(url),
quote(username),
)
}
/// Reads the collections out of `pimsync discover` output.
///
/// The format is not documented, so this was established against a real server
/// and is deliberately forgiving. The one thing it must get right: a pair has
/// two storages and pimsync reports both, so `Found` lines are only taken after
/// the heading naming the server. Otherwise the scratch directory's own
/// contents would be offered as if they were the user's calendars.
fn parse_discovery(output: &str) -> Vec<Collection> {
let mut collections = Vec::new();
let mut in_remote = false;
for line in output.lines() {
let line = strip_ansi(line);
let line = line.trim();
if line.starts_with("==>") {
in_remote = line.contains(PROBE_REMOTE);
continue;
}
let Some(href) = line.strip_prefix("Found ") else {
continue;
};
let href = href.trim();
if !in_remote || href.is_empty() {
continue;
}
collections.push(Collection {
id: collection_id(href),
href: href.to_string(),
});
}
collections
}
/// A plausible endpoint id: the last non-empty path segment, restricted to
/// characters that are unambiguous in a configuration file.
fn collection_id(href: &str) -> String {
let segment = href
.rsplit('/')
.find(|segment| !segment.is_empty())
.unwrap_or(href);
let id: String = segment
.chars()
.map(|ch| match ch {
'a'..='z' | '0'..='9' | '-' | '_' => ch,
'A'..='Z' => ch.to_ascii_lowercase(),
_ => '-',
})
.collect();
let id = id.trim_matches('-').to_string();
if id.is_empty() { "calendar".into() } else { id }
}
/// Removes ANSI colour, which pimsync emits even when its output is a pipe and
/// even under NO_COLOR.
fn strip_ansi(text: &str) -> String {
let mut out = String::with_capacity(text.len());
let mut chars = text.chars();
while let Some(ch) = chars.next() {
if ch != '\u{1b}' {
out.push(ch);
continue;
}
// Skip up to and including the terminating letter of the escape.
for next in chars.by_ref() {
if next.is_ascii_alphabetic() {
break;
}
}
}
out
}
/// Runs one pimsync cycle.
///
/// Deliberately one-shot: `pimsync daemon` would write the same vdirs the
@ -327,7 +521,8 @@ url = "https://example.org/holidays.ics"
fn generated() -> String {
let config: Config = toml::from_str(CONFIG).expect("config should parse");
generate(&config, Path::new("/var/state/calcalist")).expect("generation should succeed")
generate_for(&config, Path::new("/var/state/calcalist"), Direction::Both)
.expect("generation should succeed")
}
#[test]
@ -404,7 +599,8 @@ url_command = "secret-tool lookup service calcalist account gcal-ics"
"#,
)
.expect("config should parse");
let text = generate(&config, Path::new("/var/state/calcalist")).expect("generate");
let text = generate_for(&config, Path::new("/var/state/calcalist"), Direction::Both)
.expect("generate");
assert!(text.contains("url {"));
assert!(text.contains("shell secret-tool lookup service calcalist account gcal-ics"));
@ -488,4 +684,93 @@ url_command = "secret-tool lookup service calcalist account gcal-ics"
assert!(supported.is_supported());
assert!(!next_series.is_supported());
}
/// Established against a real server: everything arrives on stdout, the
/// headings are colour-coded even through a pipe, and both storages of the
/// pair are reported.
const DISCOVERY: &str = "\u{1b}[1m\u{1b}[32m==>\u{1b}[0m Discovering collections in local\u{2026}\n\
Found leftover-one\n\
Found leftover-two\n\
\u{1b}[1m\u{1b}[32m==>\u{1b}[0m Discovering collections in probe_remote\u{2026}\n\
Found /calendars/me/work/\n\
Found /calendars/me/Private Stuff/\n";
/// The trap: pimsync reports the scratch directory's contents too, and
/// offering those as the user's calendars would be nonsense.
#[test]
fn only_the_servers_collections_are_taken() {
let found = parse_discovery(DISCOVERY);
assert_eq!(
found.iter().map(|c| c.href.as_str()).collect::<Vec<_>>(),
vec!["/calendars/me/work/", "/calendars/me/Private Stuff/"]
);
}
#[test]
fn an_id_is_taken_from_the_last_segment() {
let found = parse_discovery(DISCOVERY);
assert_eq!(found[0].id, "work");
// Spaces and capitals cannot appear in an id used as a routing marker.
assert_eq!(found[1].id, "private-stuff");
}
#[test]
fn a_server_offering_nothing_yields_nothing() {
let empty = "\u{1b}[1m==>\u{1b}[0m Discovering collections in probe_remote\u{2026}\n";
assert!(parse_discovery(empty).is_empty());
}
/// A failure is reported in place of the collections, so nothing is found.
#[test]
fn an_unreachable_server_yields_nothing() {
let failed = "\u{1b}[1m==>\u{1b}[0m Discovering collections in probe_remote\u{2026}\n\
input/output error: client error executing request: client error (Connect)\n";
assert!(parse_discovery(failed).is_empty());
}
#[test]
fn colour_is_stripped_from_headings() {
assert_eq!(
strip_ansi("\u{1b}[1m\u{1b}[32m==>\u{1b}[0m Discovering"),
"==> Discovering"
);
}
/// The probe must never be mistaken for the real configuration, and must
/// name the storage the parser anchors on.
#[test]
fn the_probe_config_is_separate_and_parseable() {
let text = probe_config(
Path::new("/var/state/calcalist"),
Path::new("/var/state/calcalist/discover-vdir"),
"https://dav.example.com/calendars/me/",
"me@example.com",
Some("secret-tool lookup service dav"),
);
assert!(
text.contains(&format!("storage {PROBE_REMOTE} {{")),
"{text}"
);
assert!(text.contains("collections from a"), "{text}");
assert!(text.contains("discovery collections"), "{text}");
assert!(
text.contains("shell secret-tool lookup service dav"),
"{text}"
);
// Its status path is its own, so a probe cannot disturb a real sync.
assert!(text.contains("discover-status"), "{text}");
}
#[test]
fn a_server_needing_no_password_gets_no_password_directive() {
let text = probe_config(
Path::new("/var/state/calcalist"),
Path::new("/tmp/scratch"),
"https://dav.example.com/calendars/me/",
"me",
None,
);
assert!(!text.contains("password"), "{text}");
}
}

View file

@ -83,9 +83,6 @@ pub struct Conflict {
/// Something that could not be done, and why.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum Skipped {
/// A user created an event in the aggregate, but no sink is configured to
/// route it to, and guessing one would put it in the wrong calendar.
NoSink { aggregate_uid: String },
/// An edit was made to a mirror of a read-only feed, which cannot accept it.
ReadOnlySource {
aggregate_uid: String,
@ -93,10 +90,12 @@ pub enum Skipped {
},
/// The event asked to be filed under a source that cannot take it. Refused
/// rather than sent to the default, since a typo should not quietly put the
/// event in the wrong calendar.
/// event in the wrong calendar. The valid names travel with it, so the report
/// can say what would have worked.
UnknownSink {
aggregate_uid: String,
requested: String,
available: Vec<String>,
},
}
@ -107,6 +106,9 @@ pub struct Outcome {
pub links: BTreeMap<String, Link>,
pub conflicts: Vec<Conflict>,
pub skipped: Vec<Skipped>,
/// Events left in the aggregate rather than filed under a source. Not a
/// failure: an aggregate is also a calendar, and these are its own events.
pub kept_local: Vec<String>,
}
#[derive(Debug, Error, PartialEq)]
@ -138,6 +140,7 @@ pub fn reconcile(
links: BTreeMap::new(),
conflicts: Vec::new(),
skipped: Vec::new(),
kept_local: Vec::new(),
};
let mut live: HashSet<String> = HashSet::new();
let mut deletions = 0usize;
@ -358,21 +361,38 @@ fn route_new_events(
// An event may name the source it belongs in; otherwise the aggregate's
// configured sink takes it.
let requested = mirror::routing_hint(&item.calendar);
// `@local` keeps the event where it was written. Unlike every other
// marker it is deliberately not stripped: the others have done their
// job once the event reaches its source, while this one has to survive
// to make the same decision on every later cycle.
if requested.as_deref().is_some_and(mirror::is_local_sink) {
outcome.kept_local.push(uid.clone());
continue;
}
let wanted = requested.as_deref().or(policy.default_sink);
// Case-insensitively: the name is typed into a calendar app by hand, and
// capitalisation is not worth failing over.
let sink = wanted
.and_then(|sink| sources.iter().find(|source| source.id == sink))
.and_then(|sink| {
sources
.iter()
.find(|source| source.id.eq_ignore_ascii_case(sink))
})
.filter(|sink| sink.writable);
let Some(sink) = sink else {
outcome.skipped.push(match requested {
Some(requested) => Skipped::UnknownSink {
match requested {
Some(requested) => outcome.skipped.push(Skipped::UnknownSink {
aggregate_uid: uid.clone(),
requested,
},
None => Skipped::NoSink {
aggregate_uid: uid.clone(),
},
});
available: writable_sinks(sources),
}),
// Nothing says where this event should go, so it stays where it
// was written. With no default_sink configured that is the
// mode, not an omission — the aggregate is a calendar the user
// can also write in directly.
None => outcome.kept_local.push(uid.clone()),
}
continue;
};
@ -431,6 +451,15 @@ fn route_new_events(
/// that failed to populate, which shows up as a *bulk* disappearance.
const ALWAYS_ALLOWED_DELETIONS: usize = 3;
/// The sources of this aggregate that an event could actually be filed under.
fn writable_sinks(sources: &[SourceView<'_>]) -> Vec<String> {
sources
.iter()
.filter(|source| source.writable)
.map(|source| source.id.to_string())
.collect()
}
/// Refuses a cycle that would delete an implausible share of the aggregate.
///
/// A source vdir that failed to populate looks exactly like one whose events were
@ -826,6 +855,7 @@ mod tests {
vec![Skipped::UnknownSink {
aggregate_uid: "hand@phone".into(),
requested: "nosuchplace".into(),
available: vec![SRC.to_string()],
}]
);
}
@ -860,7 +890,9 @@ mod tests {
);
}
/// Guessing a sink would put the event in the wrong calendar, so refuse.
/// With nowhere configured to file it, the event stays where it was
/// written. Guessing a sink would put it in the wrong calendar, and this is
/// how an aggregate holds events of its own.
#[test]
fn without_a_sink_a_user_created_event_is_left_alone() {
let sources = map(vec![]);
@ -868,12 +900,30 @@ mod tests {
let outcome = reconcile(policy(), &[view(&sources, true)], &aggregate, None).expect("ok");
assert!(outcome.actions.is_empty());
assert_eq!(
outcome.skipped,
vec![Skipped::NoSink {
aggregate_uid: "hand-written@phone".into()
}]
);
assert!(outcome.skipped.is_empty(), "{:?}", outcome.skipped);
assert_eq!(outcome.kept_local, vec!["hand-written@phone".to_string()]);
}
/// `@local` opts one event out even where a default sink would take it.
#[test]
fn the_local_marker_keeps_an_event_out_of_every_source() {
let sources = map(vec![]);
let mut tagged = item("hand-written@phone", "Dentist");
tagged
.calendar
.set_property("VEVENT", "DESCRIPTION", "Bring the referral\\n\\n@local");
tagged.hash = tagged.calendar.content_hash();
let aggregate = map(vec![tagged]);
let policy = Policy {
default_sink: Some(SRC),
..policy()
};
let outcome = reconcile(policy, &[view(&sources, true)], &aggregate, None).expect("ok");
assert!(outcome.actions.is_empty(), "{:?}", outcome.actions);
assert!(outcome.skipped.is_empty(), "{:?}", outcome.skipped);
assert_eq!(outcome.kept_local, vec!["hand-written@phone".to_string()]);
}
/// Our own mirror appearing inside a source must not be mirrored again.

View file

@ -55,6 +55,8 @@ pub struct Outcome {
/// Events flushed to a sink before the move, which existed nowhere else.
pub routed_first: usize,
pub materialised: usize,
/// Events belonging to the aggregate itself, moved rather than re-derived.
pub carried: usize,
pub purged: usize,
}
@ -117,7 +119,14 @@ pub fn retarget(
false,
)?;
let materialised = materialise(config, aggregate, new_target, state_dir, &mut state)?;
let (materialised, carried) = materialise(
config,
aggregate,
&old_target,
new_target,
state_dir,
&mut state,
)?;
let purged = if purge_old {
purge(aggregate_id, &old_target, state_dir, &state)?
} else {
@ -138,6 +147,7 @@ pub fn retarget(
to: new_target.id.clone(),
routed_first: settled.written_back,
materialised,
carried,
purged,
})
}
@ -150,10 +160,11 @@ pub fn retarget(
fn materialise(
config: &Config,
aggregate: &crate::config::Aggregate,
old_target: &Endpoint,
new_target: &Endpoint,
state_dir: &Path,
state: &mut State,
) -> Result<usize, RetargetError> {
) -> Result<(usize, usize), RetargetError> {
let new_dir = vdir_path(state_dir, &new_target.id);
let existing = vdir::read(&new_dir)?;
let suppression = new_target.kind.scheduling_suppression();
@ -188,10 +199,30 @@ fn materialise(
}
written += 1;
}
// Events created in the aggregate and never filed under a source exist only
// on the old target, so there is nothing to re-derive them from: they are
// moved rather than rebuilt. Without this they would sit on the calendar
// being left behind while everything around them moved on.
//
// The new target's scheduling rule still applies. This is a write to an
// aggregate like any other, and a guest list carried live onto a server
// that schedules would mail everyone on it.
let mut carried = 0;
for (uid, item) in vdir::read(&vdir_path(state_dir, &old_target.id))? {
if provenance::is_derived(&uid) {
continue;
}
let inert = mirror::make_inert(&item.calendar, new_target.kind.owner(), suppression);
let replaces = existing.get(&uid).map(|held| held.path.clone());
vdir::write(&new_dir, &uid, &inert, replaces.as_deref())?;
carried += 1;
}
state
.aggregate_mut(&aggregate.id, target_of(new_target))
.links = updated;
Ok(written)
Ok((written, carried))
}
/// Removes this aggregate's mirrors from the calendar it has left.
@ -280,6 +311,53 @@ default_sink = "src"
)
}
/// An event belonging to the aggregate itself has to follow the move.
///
/// There is nothing to re-derive it from, so before this it stayed on the
/// calendar being left behind while every mirrored event moved on — quietly,
/// which is the worst way for a calendar to lose an appointment.
#[test]
fn an_event_belonging_to_the_aggregate_is_carried_to_the_new_target() {
// No default_sink: events written into the aggregate stay there.
let config: Config = toml::from_str(&CONFIG.replace(r#"default_sink = "src""#, ""))
.expect("config should parse");
let dir = seed(&config);
let old = vdir_path(dir.path(), "agg1");
fs::write(
old.join("dentist.ics"),
event("hand-written@phone", "Dentist"),
)
.expect("write");
settle(&config, dir.path());
let outcome = retarget(&config, dir.path(), "unified", "agg2", false).expect("retarget");
assert_eq!(outcome.carried, 1, "{outcome:?}");
let moved = vdir::read(&vdir_path(dir.path(), "agg2")).expect("read");
assert!(
moved.contains_key("hand-written@phone"),
"the aggregate's own event did not follow: {:?}",
moved.keys().collect::<Vec<_>>()
);
assert!(
moved["hand-written@phone"]
.calendar
.to_ics()
.contains("Dentist"),
"it arrived without its content"
);
// And it is still not a mirror of anything, so nothing was invented for it.
let state = State::load(&dir.path().join(crate::state::FILE_NAME)).expect("state");
assert!(
!state
.aggregate("unified")
.expect("aggregate")
.links
.contains_key("hand-written@phone")
);
}
/// Reconciles one aggregate and records the result, without running a full
/// cycle. A full cycle would contact Google for real, which these tests have
/// no business doing — they are about what happens to local files.

169
src/status.rs Normal file
View file

@ -0,0 +1,169 @@
//! `calcalist status` — what is configured, and what it has done.
//!
//! Its most practical job is naming the markers an event can carry. A routing
//! hint is typed by hand into a calendar app, and having to remember what was
//! written in a TOML file is no way to find out what it should say.
use std::fmt::Write as _;
use std::path::Path;
use crate::config::{Aggregate, Config};
use crate::state::{State, StateError};
/// Renders the report shown by `calcalist status`.
pub fn render(config: &Config, state_dir: &Path) -> Result<String, StateError> {
let state = State::load(&state_dir.join(crate::state::FILE_NAME))?;
let mut out = String::new();
out.push_str("endpoints\n");
for endpoint in &config.endpoints {
let access = if endpoint.kind.is_writable() {
"read/write"
} else {
"read-only"
};
let _ = writeln!(
out,
" {:<18} {:<8} {access}",
endpoint.id,
endpoint.kind.kind_name()
);
}
for aggregate in &config.aggregates {
let _ = write!(out, "\naggregate `{}`\n", aggregate.id);
let _ = writeln!(out, " published to {}", aggregate.target);
let _ = writeln!(out, " sources {}", aggregate.sources.join(", "));
let tracked = state
.aggregate(&aggregate.id)
.map_or(0, |entry| entry.links.len());
let _ = writeln!(out, " mirroring {tracked} event(s)");
describe_routing(&mut out, config, aggregate);
}
if config.aggregates.is_empty() {
out.push_str("\nno aggregates configured\n");
}
Ok(out)
}
/// Explains where a newly created event goes, and how to send it elsewhere.
fn describe_routing(out: &mut String, config: &Config, aggregate: &Aggregate) {
let sinks: Vec<&str> = aggregate
.sources
.iter()
.filter(|id| {
config
.endpoint(id)
.is_some_and(|endpoint| endpoint.kind.is_writable())
})
.map(String::as_str)
.collect();
match &aggregate.default_sink {
Some(sink) => {
let _ = writeln!(out, " new events go to `{sink}` unless told otherwise");
}
None => {
let _ = writeln!(
out,
" new events stay here: no default_sink is configured, so the aggregate keeps its own"
);
}
}
if sinks.is_empty() {
let _ = writeln!(out, " none of its sources can be written to");
return;
}
let _ = writeln!(
out,
" to choose, put one of these on a line of its own in the event's notes:"
);
for sink in sinks {
let _ = writeln!(out, " @{sink}");
}
let _ = writeln!(
out,
" @{} (keep the event here, in this calendar only)",
crate::mirror::LOCAL_SINK
);
}
#[cfg(test)]
mod tests {
use super::*;
const CONFIG: &str = r#"
version = 1
[[endpoint]]
id = "gcal"
type = "google"
calendar_id = "a@example.com"
client_id = "x.apps.googleusercontent.com"
[[endpoint]]
id = "posteo"
type = "caldav"
url = "https://dav.example.com/dav/calendars/me/work/"
username = "me@example.com"
[[endpoint]]
id = "holidays"
type = "webcal"
url = "https://example.org/h.ics"
[[aggregate]]
id = "unified"
target = "posteo"
sources = ["gcal", "holidays"]
default_sink = "gcal"
"#;
fn report() -> String {
let config: Config = toml::from_str(CONFIG).expect("config");
let dir = tempfile::tempdir().expect("temp dir");
render(&config, dir.path()).expect("render")
}
/// The whole point of the command: knowing what to type without opening the
/// configuration file.
#[test]
fn it_names_the_markers_that_would_work() {
let report = report();
assert!(report.contains("@gcal"), "{report}");
}
/// A read-only feed cannot take an event, so offering it would mislead.
#[test]
fn it_does_not_offer_a_read_only_source_as_a_destination() {
let report = report();
assert!(!report.contains("@holidays"), "{report}");
assert!(
report.contains("holidays"),
"it should still be listed as a source"
);
}
#[test]
fn it_says_where_untagged_events_go() {
assert!(report().contains("new events go to `gcal`"));
}
/// The escape hatch has to be as findable as the destinations.
#[test]
fn it_names_the_marker_that_keeps_an_event_local() {
assert!(report().contains("@local"), "{}", report());
}
#[test]
fn it_says_when_nothing_would_be_routed() {
let sinkless: Config =
toml::from_str(&CONFIG.replace(r#"default_sink = "gcal""#, "")).expect("config");
let dir = tempfile::tempdir().expect("temp dir");
let report = render(&sinkless, dir.path()).expect("render");
assert!(report.contains("the aggregate keeps its own"), "{report}");
}
}

View file

@ -5,7 +5,8 @@
//! out of those vdirs is the job of pimsync (CalDAV, WebCal) and the Google
//! module, which bracket this step.
use std::collections::BTreeMap;
use std::collections::{BTreeMap, BTreeSet};
use std::fs;
use std::path::{Path, PathBuf};
use thiserror::Error;
@ -13,7 +14,7 @@ use thiserror::Error;
use crate::config::{Aggregate, Config, Endpoint, EndpointKind};
use crate::google::api::{self, ApiError};
use crate::google::auth::{self, AuthError};
use crate::pimsync::{self, PimsyncError};
use crate::pimsync::{self, Direction, PimsyncError};
use crate::reconcile::{self, Action, Conflict, Policy, ReconcileError, Skipped, SourceView};
use crate::state::{State, Target};
use crate::vdir::{self, VdirError};
@ -69,9 +70,37 @@ pub enum SyncError {
pub struct Report {
pub aggregates: Vec<AggregateReport>,
pub google: Vec<GoogleReport>,
pub failures: Vec<EndpointFailure>,
pub blocked: Vec<BlockedAggregate>,
pub orphan_vdirs: Vec<PathBuf>,
pub dry_run: bool,
}
impl Report {
/// Whether the cycle covered everything it was asked to.
///
/// A partial cycle is not a crash — the aggregates that could be reconciled
/// were — but it must not read as success either, or a lapsed token would go
/// unnoticed until the calendars had drifted a long way apart.
pub fn is_complete(&self) -> bool {
self.failures.is_empty() && self.blocked.is_empty()
}
}
/// An endpoint that could not be reached this cycle.
#[derive(Debug)]
pub struct EndpointFailure {
pub endpoint: String,
pub reason: String,
}
/// An aggregate left alone because an endpoint it depends on was unreachable.
#[derive(Debug)]
pub struct BlockedAggregate {
pub id: String,
pub endpoints: Vec<String>,
}
#[derive(Debug)]
pub struct GoogleReport {
pub endpoint: String,
@ -81,6 +110,7 @@ pub struct GoogleReport {
pub created_remotely: usize,
pub updated_remotely: usize,
pub deleted_remotely: usize,
pub exceptions_applied: usize,
pub exceptions_skipped: usize,
}
@ -93,6 +123,7 @@ pub struct AggregateReport {
pub deleted_from_sources: usize,
pub conflicts: Vec<Conflict>,
pub skipped: Vec<Skipped>,
pub kept_local: usize,
}
/// Runs the local half of a cycle: reconcile every aggregate and apply the result.
@ -106,133 +137,257 @@ pub fn run(
let mut state = State::load(&state_path)?;
let mut report = Report {
dry_run,
orphan_vdirs: orphan_vdirs(config, state_dir),
..Report::default()
};
prepare_vdirs(config, state_dir)?;
// A dry run must leave the vdirs exactly as it found them, and pimsync
// writes to them, so the pulls and pushes are skipped along with everything
// else. What it reports is therefore what the *last* pull left behind.
let pimsync_config = if dry_run || !needs_pimsync(config) {
None
// A dry run pulls for real, so that what it reports is measured against the
// calendars as they are now — but into a throwaway copy of the local
// mirrors, and through a configuration that only ever reads from a server.
// Nothing outside that copy is written, and it is removed at the end.
let workspace = if dry_run {
Some(Workspace::open(state_dir)?)
} else {
Some(pimsync::write_config(config, state_dir)?)
None
};
let work_dir = workspace.as_ref().map_or(state_dir, Workspace::path);
let pimsync_config = if needs_pimsync(config) {
let direction = if dry_run {
Direction::PullOnly
} else {
Direction::Both
};
Some(pimsync::write_config_for(config, work_dir, direction)?)
} else {
None
};
// Pull first, so the reconciler sees one consistent snapshot of every remote.
if let Some(path) = &pimsync_config {
pimsync::sync(path)?;
}
if !dry_run {
report.google = pull_google(config, state_dir, &mut state)?;
}
let unavailable = pull_google(config, state_dir, work_dir, &mut state, &mut report);
for aggregate in &config.aggregates {
// An aggregate whose target or any source could not be pulled is left
// untouched. Reconciling it would compare against a stale snapshot, and
// an endpoint that failed to answer looks exactly like an emptied one.
let endpoints: Vec<String> = std::iter::once(&aggregate.target)
.chain(aggregate.sources.iter())
.filter(|id| unavailable.contains(*id))
.cloned()
.collect();
if !endpoints.is_empty() {
report.blocked.push(BlockedAggregate {
id: aggregate.id.clone(),
endpoints,
});
continue;
}
let target = resolve(config, aggregate, &aggregate.target)?;
check_target_drift(&state, aggregate, target)?;
report.aggregates.push(sync_aggregate(
config, aggregate, target, state_dir, &mut state, dry_run, force,
config, aggregate, target, work_dir, &mut state, dry_run, force,
)?);
}
// Push what the reconciler decided out to the remotes.
if !dry_run {
push_google(config, state_dir, &mut state, &mut report)?;
}
if let Some(path) = &pimsync_config {
pimsync::sync(path)?;
}
if !dry_run {
push_google(config, state_dir, &mut state, &mut report, &unavailable);
if let Some(path) = &pimsync_config {
pimsync::sync(path)?;
}
state.save(&state_path)?;
}
if let Some(workspace) = workspace {
workspace.discard();
}
Ok(report)
}
/// Brings every Google endpoint's vdir into step with its remote calendar.
/// A throwaway copy of the local mirrors, used by `--dry-run`.
///
/// Pulling needs somewhere to put what it reads. Writing into the real mirrors
/// would leave them describing a moment the recorded state knows nothing about,
/// so a dry run gets its own copy and the originals are never opened for writing.
struct Workspace {
root: PathBuf,
}
impl Workspace {
const DIRECTORY: &'static str = "dry-run";
fn open(state_dir: &Path) -> Result<Self, SyncError> {
let root = state_dir.join(Self::DIRECTORY);
// A previous run that was interrupted may have left one behind.
if root.exists() {
fs::remove_dir_all(&root).map_err(|source| SyncError::Prepare {
path: root.clone(),
source,
})?;
}
copy_tree(&vdir_root(state_dir), &root.join("vdir"))?;
// pimsync's own record of what it has seen comes too, or it would treat
// every local item as new and plan to create it on the server.
copy_tree(
&pimsync::status_path(state_dir),
&pimsync::status_path(&root),
)?;
Ok(Workspace { root })
}
fn path(&self) -> &Path {
&self.root
}
fn discard(self) {
let _ = fs::remove_dir_all(&self.root);
}
}
/// Copies a directory tree, treating a missing source as an empty one.
fn copy_tree(from: &Path, to: &Path) -> Result<(), SyncError> {
fs::create_dir_all(to).map_err(|source| SyncError::Prepare {
path: to.to_path_buf(),
source,
})?;
let Ok(entries) = fs::read_dir(from) else {
return Ok(());
};
for entry in entries.flatten() {
let target = to.join(entry.file_name());
let copied = if entry.path().is_dir() {
copy_tree(&entry.path(), &target)
} else {
fs::copy(entry.path(), &target)
.map(|_| ())
.map_err(|source| SyncError::Prepare {
path: target,
source,
})
};
copied?;
}
Ok(())
}
/// Local mirrors belonging to endpoints the configuration no longer names.
///
/// Removing an endpoint leaves its events sitting in the state directory, where
/// nothing manages them any more. They are reported rather than deleted: the
/// endpoint may have been renamed, or commented out for an afternoon.
pub fn orphan_vdirs(config: &Config, state_dir: &Path) -> Vec<PathBuf> {
let Ok(entries) = fs::read_dir(vdir_root(state_dir)) else {
return Vec::new();
};
let mut orphans: Vec<PathBuf> = entries
.flatten()
.filter(|entry| entry.path().is_dir())
.filter(|entry| {
let name = entry.file_name().to_string_lossy().into_owned();
config.endpoint(&name).is_none()
})
.map(|entry| entry.path())
.collect();
orphans.sort();
orphans
}
/// Brings every Google endpoint's vdir into step with its remote calendar,
/// returning the endpoints that could not be reached.
///
/// This is the Google equivalent of the pimsync pull: pimsync cannot reach
/// Google at all, so calcalist does this leg itself.
///
/// A failure here used to end the cycle. It no longer does: one lapsed token
/// would otherwise stop the CalDAV side too, which has nothing to do with it.
/// The affected endpoints are named, and only the aggregates that depend on
/// them stand down.
fn pull_google(
config: &Config,
state_dir: &Path,
vdir_base: &Path,
state: &mut State,
) -> Result<Vec<GoogleReport>, SyncError> {
let mut reports = Vec::new();
report: &mut Report,
) -> BTreeSet<String> {
let mut unavailable = BTreeSet::new();
for endpoint in &config.endpoints {
let EndpointKind::Google { calendar_id, .. } = &endpoint.kind else {
continue;
};
let credentials = auth::credentials_for(config, &endpoint.id).map_err(|source| {
SyncError::GoogleAuth {
let pulled = client_for(config, state_dir, endpoint, calendar_id).and_then(|client| {
let dir = vdir_path(vdir_base, &endpoint.id);
let entry = state.google.entry(endpoint.id.clone()).or_default();
api::pull(&client, &dir, entry).map_err(|source| SyncError::Google {
endpoint: endpoint.id.clone(),
source,
}
})?;
let token =
auth::access_token(state_dir, &endpoint.id, &credentials).map_err(|source| {
SyncError::GoogleAuth {
endpoint: endpoint.id.clone(),
source,
}
})?;
let client = api::Client::new(token, calendar_id.clone());
let dir = vdir_path(state_dir, &endpoint.id);
let entry = state.google.entry(endpoint.id.clone()).or_default();
let pulled = api::pull(&client, &dir, entry).map_err(|source| SyncError::Google {
endpoint: endpoint.id.clone(),
source,
})?;
reports.push(GoogleReport {
endpoint: endpoint.id.clone(),
written: pulled.written,
deleted: pulled.deleted,
full_resync: pulled.full_resync,
created_remotely: 0,
updated_remotely: 0,
deleted_remotely: 0,
exceptions_skipped: 0,
})
});
match pulled {
Ok(pulled) => report.google.push(GoogleReport {
endpoint: endpoint.id.clone(),
written: pulled.written,
deleted: pulled.deleted,
full_resync: pulled.full_resync,
created_remotely: 0,
updated_remotely: 0,
deleted_remotely: 0,
exceptions_applied: 0,
exceptions_skipped: 0,
}),
Err(error) => {
unavailable.insert(endpoint.id.clone());
report.failures.push(EndpointFailure {
endpoint: endpoint.id.clone(),
reason: chain(&error),
});
}
}
}
Ok(reports)
unavailable
}
/// Sends each Google endpoint's local changes up to its calendar.
///
/// Runs after reconciliation, so what goes up is what the reconciler decided.
/// An endpoint whose pull failed is skipped: its vdir describes some earlier
/// moment, and pushing that would undo whatever has happened since.
fn push_google(
config: &Config,
state_dir: &Path,
state: &mut State,
report: &mut Report,
) -> Result<(), SyncError> {
unavailable: &BTreeSet<String>,
) {
for endpoint in &config.endpoints {
let EndpointKind::Google { calendar_id, .. } = &endpoint.kind else {
continue;
};
let credentials = auth::credentials_for(config, &endpoint.id).map_err(|source| {
SyncError::GoogleAuth {
if unavailable.contains(&endpoint.id) {
continue;
}
let pushed = client_for(config, state_dir, endpoint, calendar_id).and_then(|client| {
let dir = vdir_path(state_dir, &endpoint.id);
let entry = state.google.entry(endpoint.id.clone()).or_default();
api::push(&client, &dir, entry).map_err(|source| SyncError::Google {
endpoint: endpoint.id.clone(),
source,
}
})?;
let token =
auth::access_token(state_dir, &endpoint.id, &credentials).map_err(|source| {
SyncError::GoogleAuth {
})
});
let pushed = match pushed {
Ok(pushed) => pushed,
Err(error) => {
report.failures.push(EndpointFailure {
endpoint: endpoint.id.clone(),
source,
}
})?;
let client = api::Client::new(token, calendar_id.clone());
let dir = vdir_path(state_dir, &endpoint.id);
let entry = state.google.entry(endpoint.id.clone()).or_default();
let pushed = api::push(&client, &dir, entry).map_err(|source| SyncError::Google {
endpoint: endpoint.id.clone(),
source,
})?;
reason: chain(&error),
});
continue;
}
};
if let Some(existing) = report
.google
@ -242,10 +397,46 @@ fn push_google(
existing.created_remotely = pushed.created;
existing.updated_remotely = pushed.updated;
existing.deleted_remotely = pushed.deleted;
existing.exceptions_applied = pushed.exceptions_applied;
existing.exceptions_skipped = pushed.exceptions_skipped;
}
}
Ok(())
}
/// An authorised client for one Google endpoint.
fn client_for(
config: &Config,
state_dir: &Path,
endpoint: &Endpoint,
calendar_id: &str,
) -> Result<api::Client, SyncError> {
let credentials =
auth::credentials_for(config, &endpoint.id).map_err(|source| SyncError::GoogleAuth {
endpoint: endpoint.id.clone(),
source,
})?;
let token = auth::access_token(
state_dir,
&endpoint.id,
endpoint.kind.google_account(),
&credentials,
)
.map_err(|source| SyncError::GoogleAuth {
endpoint: endpoint.id.clone(),
source,
})?;
Ok(api::Client::new(token, calendar_id.to_string()))
}
/// An error and its causes on one line, for a report that keeps going.
fn chain(error: &dyn std::error::Error) -> String {
let mut out = error.to_string();
let mut source = error.source();
while let Some(cause) = source {
out.push_str(&format!(": {cause}"));
source = cause.source();
}
out
}
/// Whether any endpoint is one pimsync handles. A Google-only setup needs none.
@ -374,6 +565,7 @@ pub(crate) fn sync_aggregate(
deleted_from_sources: 0,
conflicts: outcome.conflicts,
skipped: outcome.skipped,
kept_local: outcome.kept_local.len(),
};
for action in &outcome.actions {
@ -705,4 +897,73 @@ sources = ["gcal"]
uids(dir.path(), "work")
);
}
/// An endpoint calcalist cannot reach must not take the whole cycle with it.
/// Here every endpoint is Google and none is authorised, so all of them fail
/// — and the run still returns, naming what it could not do.
#[test]
fn an_unreachable_endpoint_blocks_only_what_depends_on_it() {
let dir = tempfile::tempdir().expect("temp");
let config: Config = toml::from_str(&format!(
"{CONFIG}\n[[aggregate]]\nid = \"unified\"\ntarget = \"posteo\"\nsources = [\"work\"]\n"
))
.expect("config");
seed(dir.path(), "work", "a@example.com", "Standup");
let report = run(&config, dir.path(), false, false).expect("the cycle still returns");
assert!(!report.failures.is_empty(), "{report:?}");
assert_eq!(report.blocked.len(), 1, "{report:?}");
assert_eq!(report.blocked[0].id, "unified");
assert!(report.blocked[0].endpoints.contains(&"work".to_string()));
assert!(!report.is_complete(), "a partial cycle is not a success");
// Nothing was reconciled against the stale snapshot.
assert!(report.aggregates.is_empty(), "{report:?}");
assert!(uids(dir.path(), "posteo").is_empty());
}
/// A mirror whose endpoint has gone from the configuration is reported, not
/// deleted: the endpoint may simply have been renamed.
#[test]
fn a_mirror_with_no_endpoint_is_reported_as_an_orphan() {
let dir = tempfile::tempdir().expect("temp");
let config: Config = toml::from_str(CONFIG).expect("config");
seed(dir.path(), "work", "a@example.com", "Standup");
seed(dir.path(), "retired", "b@example.com", "Old");
let orphans = orphan_vdirs(&config, dir.path());
assert_eq!(orphans, vec![vdir_path(dir.path(), "retired")]);
}
/// The dry run's copy is thrown away, and the mirrors it was made from are
/// left exactly as they were.
#[test]
fn a_dry_run_leaves_the_real_mirrors_alone() {
let dir = tempfile::tempdir().expect("temp");
let config: Config = toml::from_str(&format!(
"{CONFIG}\n[[aggregate]]\nid = \"unified\"\ntarget = \"posteo\"\nsources = [\"work\"]\n"
))
.expect("config");
seed(dir.path(), "work", "a@example.com", "Standup");
let before = fs::read_to_string(vdir_path(dir.path(), "work").join("a@example.com.ics"))
.expect("read");
let report = run(&config, dir.path(), true, false).expect("dry run");
assert!(report.dry_run);
assert!(
!dir.path().join("dry-run").exists(),
"the workspace should be gone"
);
assert_eq!(
fs::read_to_string(vdir_path(dir.path(), "work").join("a@example.com.ics"))
.expect("read"),
before
);
assert!(
!dir.path().join(crate::state::FILE_NAME).exists(),
"a dry run records nothing"
);
}
}

15
systemd/calcalist.service Normal file
View file

@ -0,0 +1,15 @@
[Unit]
Description=Synchronise calendars with calcalist
[Service]
Type=oneshot
ExecStart=%h/.local/bin/calcalist sync
NoNewPrivileges=true
# Deliberately no ProtectSystem, PrivateTmp or IPC sandboxing. Credentials come
# from commands like `secret-tool`, which need the session keyring over D-Bus,
# and those options break them in ways that surface as unexplained auth
# failures rather than as anything pointing at the sandbox.
# A cycle that could not reach an endpoint exits non-zero on purpose, so a
# lapsed token shows up as a failed unit rather than passing unnoticed.

15
systemd/calcalist.timer Normal file
View file

@ -0,0 +1,15 @@
[Unit]
Description=Synchronise calendars with calcalist every 15 minutes
[Timer]
OnBootSec=2m
OnUnitActiveSec=15m
Persistent=true
# Spreads requests instead of every installation calling Google on the quarter
# hour. Google's Calendar API quota is per project and enforced per minute, and
# its own guidance is to randomise timing rather than to burst.
RandomizedDelaySec=120
[Install]
WantedBy=timers.target

499
tests/caldav.rs Normal file
View file

@ -0,0 +1,499 @@
//! End-to-end tests against a real CalDAV server and a real iCal feed.
//!
//! The unit tests establish that the reconciler decides correctly. These
//! establish that the decisions survive the round trip through pimsync and a
//! server that rewrites what it stores — which is where every bug found by hand
//! during M1 actually lived.
//!
//! Radicale and pimsync both come from devbox, so `devbox run check` has them.
//! Outside that shell the tests report what is missing and pass, rather than
//! failing for a reason that has nothing to do with the code.
mod support;
use support::{Calcalist, Feed, Radicale};
/// An event with guests, an organiser and an alarm — the combination the
/// scheduling rules are about.
const MEETING: &str = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:-//test//EN\r\n\
BEGIN:VEVENT\r\nUID:meeting@work\r\nDTSTAMP:20260101T000000Z\r\n\
DTSTART:20260910T090000Z\r\nDTEND:20260910T100000Z\r\nSUMMARY:Planning\r\n\
ORGANIZER;CN=Chair:mailto:chair@example.com\r\n\
ATTENDEE;CN=Guest;PARTSTAT=ACCEPTED:mailto:guest@example.com\r\n\
BEGIN:VALARM\r\nACTION:DISPLAY\r\nTRIGGER:-PT15M\r\nDESCRIPTION:Soon\r\n\
END:VALARM\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n";
/// An event written straight into the aggregate, belonging to no source.
const HAND_WRITTEN: &str = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:-//phone//EN\r\n\
BEGIN:VEVENT\r\nUID:hand-written@phone\r\nDTSTAMP:20260101T000000Z\r\n\
DTSTART:20260911T140000Z\r\nDTEND:20260911T150000Z\r\nSUMMARY:Dentist\r\n\
END:VEVENT\r\nEND:VCALENDAR\r\n";
/// The same event, asking to be left where it was written.
const KEPT_LOCAL: &str = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:-//phone//EN\r\n\
BEGIN:VEVENT\r\nUID:kept@phone\r\nDTSTAMP:20260101T000000Z\r\n\
DTSTART:20260912T140000Z\r\nDTEND:20260912T150000Z\r\nSUMMARY:Haircut\r\n\
DESCRIPTION:Around the corner\\n\\n@local\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n";
const HOLIDAY_FEED: &str = "BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:-//feed//EN\r\n\
BEGIN:VEVENT\r\nUID:newyear@feed\r\nDTSTAMP:20260101T000000Z\r\n\
DTSTART;VALUE=DATE:20260101\r\nDTEND;VALUE=DATE:20260102\r\n\
SUMMARY:New Year\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n";
struct Fixture {
_root: tempfile::TempDir,
server: Radicale,
_feed: Feed,
calcalist: Calcalist,
}
/// A CalDAV source, a read-only feed, and a CalDAV target to aggregate into.
fn fixture() -> Option<Fixture> {
let missing = support::missing_binaries();
if !missing.is_empty() {
eprintln!(
"skipped: {} not on PATH; run under devbox",
missing.join(", ")
);
return None;
}
let root = tempfile::tempdir().expect("temp");
let server = Radicale::start(root.path());
server.create_calendar("work");
server.create_calendar("unified");
server.put_event("work", "meeting.ics", MEETING);
let feed = Feed::start(HOLIDAY_FEED.to_string());
let config = format!(
r#"
version = 1
[[endpoint]]
id = "work"
type = "caldav"
url = "{work}"
username = "{user}"
secret_command = "printf password"
[[endpoint]]
id = "published"
type = "caldav"
url = "{unified}"
username = "{user}"
secret_command = "printf password"
[[endpoint]]
id = "holidays"
type = "webcal"
url = "{feed}"
[[aggregate]]
id = "everything"
target = "published"
sources = ["work", "holidays"]
default_sink = "work"
"#,
work = server.url("work"),
unified = server.url("unified"),
user = support::USER,
feed = feed.url(),
);
let calcalist = Calcalist::new(root.path(), &config);
Some(Fixture {
_root: root,
server,
_feed: feed,
calcalist,
})
}
/// Both sources reach the target, and running again changes nothing.
///
/// Idempotence is the property that matters most: a cycle that is not a no-op
/// against unchanged input would rewrite every event on every run, and each
/// rewrite is a chance for the server to hand back something slightly different.
#[test]
fn a_cycle_converges_and_the_next_one_does_nothing() {
let Some(fixture) = fixture() else { return };
let first = fixture.calcalist.run(&["sync"]);
assert!(
first.succeeded(),
"first sync failed\n{}\n{}",
first.stdout,
first.stderr
);
let published = fixture.server.stored("unified");
assert_eq!(
published.len(),
2,
"both sources should arrive: {published:?}"
);
let all = published.join("\n");
assert!(all.contains("Planning"), "{all}");
assert!(all.contains("New Year"), "{all}");
// Provenance travels with each mirror, so the aggregate knows where its
// events came from without consulting the state file.
assert!(all.contains("X-CALCALIST-SOURCE:work"), "{all}");
assert!(all.contains("X-CALCALIST-SOURCE:holidays"), "{all}");
let second = fixture.calcalist.run(&["sync"]);
assert!(
second.succeeded(),
"second sync failed\n{}\n{}",
second.stdout,
second.stderr
);
assert!(
second.stdout.contains("everything: 0 mirrored"),
"the second cycle should be a no-op:\n{}",
second.stdout
);
assert_eq!(
fixture.server.stored("unified"),
published,
"the second cycle rewrote the target"
);
}
/// The scheduling rule, checked against what actually reached the server.
///
/// A CalDAV server has no portable way to be told not to send invitations, so
/// inertness is structural: the mirror carries the guest list as data and not as
/// live scheduling properties. The alarm is the deliberate exception — stripping
/// it would destroy every reminder in the one calendar the user subscribes to.
///
/// Radicale implements no scheduling of its own, so this asserts on the bytes
/// stored rather than on a mail sink: with nothing to send mail, a quiet SMTP
/// port would prove nothing about the transform.
#[test]
fn a_mirror_carries_no_live_scheduling_properties() {
let Some(fixture) = fixture() else { return };
let run = fixture.calcalist.run(&["sync"]);
assert!(
run.succeeded(),
"sync failed\n{}\n{}",
run.stdout,
run.stderr
);
let mirror = fixture
.server
.stored("unified")
.into_iter()
.find(|item| item.contains("Planning"))
.expect("the meeting should have been mirrored");
for property in ["ATTENDEE;", "ATTENDEE:", "ORGANIZER;", "ORGANIZER:"] {
assert!(
!mirror
.lines()
.any(|line| line.trim_start().starts_with(property)),
"a live {property} reached the aggregate:\n{mirror}"
);
}
// The guests are still there, as something nothing will act on.
assert!(
mirror.contains("X-CALCALIST-ATTENDEES"),
"the guest list was lost:\n{mirror}"
);
assert!(
mirror.contains("guest@example.com"),
"the guest list was lost:\n{mirror}"
);
assert!(
mirror.contains("BEGIN:VALARM") && mirror.contains("TRIGGER:-PT15M"),
"the alarm did not survive:\n{mirror}"
);
// The source keeps its scheduling properties: only the aggregate is inert.
let source = fixture
.server
.stored("work")
.into_iter()
.find(|item| item.contains("Planning"))
.expect("the meeting should still be in its source");
assert!(source.contains("ATTENDEE"), "{source}");
assert!(source.contains("ORGANIZER"), "{source}");
}
/// A dry run must reach the servers to be worth anything, and change nothing.
#[test]
fn a_dry_run_reports_without_touching_anything() {
let Some(fixture) = fixture() else { return };
let dry = fixture.calcalist.run(&["sync", "--dry-run"]);
assert!(
dry.succeeded(),
"dry run failed\n{}\n{}",
dry.stdout,
dry.stderr
);
assert!(
dry.stdout.contains("everything: 2 mirrored"),
"the dry run should have seen both sources:\n{}",
dry.stdout
);
assert!(
fixture.server.stored("unified").is_empty(),
"the dry run published events"
);
// And the real cycle that follows is not confused by it.
let real = fixture.calcalist.run(&["sync"]);
assert!(
real.succeeded(),
"sync after a dry run failed\n{}\n{}",
real.stdout,
real.stderr
);
assert_eq!(fixture.server.stored("unified").len(), 2);
}
/// Removing an endpoint used to leave its events sitting in the state directory
/// with nothing managing them. They are now reported, and removed on request.
#[test]
fn a_retired_endpoints_mirror_is_reported_and_then_removed() {
let Some(fixture) = fixture() else { return };
assert!(fixture.calcalist.run(&["sync"]).succeeded());
let mirror = fixture.calcalist.state.join("calcalist/vdir/holidays");
assert!(mirror.is_dir(), "the feed should have been mirrored");
// The feed is dropped from the configuration, as a user would drop it.
std::fs::write(
&fixture.calcalist.config,
fixture.calcalist.config_without_feed(),
)
.expect("rewrite config");
let listed = fixture.calcalist.run(&["prune"]);
assert!(listed.succeeded(), "{}", listed.stderr);
assert!(
listed.stdout.contains("would remove") && listed.stdout.contains("holidays"),
"prune should say what it found:\n{}",
listed.stdout
);
assert!(mirror.is_dir(), "listing must not delete anything");
let removed = fixture.calcalist.run(&["prune", "--force"]);
assert!(removed.succeeded(), "{}", removed.stderr);
assert!(!mirror.exists(), "the orphaned mirror should be gone");
}
/// An aggregate with no `default_sink` leaves events created in it alone.
///
/// This is how a target calendar keeps events of its own: with nowhere
/// configured to file a new event, calcalist refuses to guess rather than
/// picking a source, so the event simply stays where it was written. Worth
/// pinning, because "left alone" has to hold on every subsequent cycle too —
/// an event that survived the first one and was swept up by the second would
/// be worse than never having worked.
#[test]
fn an_event_created_in_a_sinkless_aggregate_stays_where_it_is() {
let Some(fixture) = fixture() else { return };
// The fixture's aggregate has a default_sink; take it away.
let sinkless = std::fs::read_to_string(&fixture.calcalist.config)
.expect("read config")
.replace("default_sink = \"work\"\n", "");
std::fs::write(&fixture.calcalist.config, sinkless).expect("rewrite config");
assert!(fixture.calcalist.run(&["sync"]).succeeded());
// Now add an event by hand, as a calendar app would.
fixture
.server
.put_event("unified", "dentist.ics", HAND_WRITTEN);
for cycle in 1..=3 {
let run = fixture.calcalist.run(&["sync"]);
assert!(
run.succeeded(),
"cycle {cycle} failed\n{}\n{}",
run.stdout,
run.stderr
);
let published = fixture.server.stored("unified");
assert!(
published.iter().any(|item| item.contains("Dentist")),
"cycle {cycle}: the hand-written event was removed:\n{published:?}"
);
// It never reaches a source: there is nowhere it was told to go.
assert!(
!fixture
.server
.stored("work")
.iter()
.any(|item| item.contains("Dentist")),
"cycle {cycle}: the hand-written event leaked into the source"
);
}
}
/// `@local` opts one event out of a default sink that would otherwise take it.
///
/// The marker has to survive, unlike every other one: it is stripped markers
/// that have done their job on arrival, whereas this event never leaves, so the
/// next cycle has to be able to reach the same decision.
#[test]
fn the_local_marker_keeps_an_event_out_of_a_configured_sink() {
let Some(fixture) = fixture() else { return };
assert!(fixture.calcalist.run(&["sync"]).succeeded());
fixture
.server
.put_event("unified", "haircut.ics", KEPT_LOCAL);
for cycle in 1..=3 {
let run = fixture.calcalist.run(&["sync"]);
assert!(
run.succeeded(),
"cycle {cycle} failed\n{}\n{}",
run.stdout,
run.stderr
);
assert!(
run.stdout.contains("1 event(s) kept local"),
"cycle {cycle} should report it as kept, not skipped:\n{}",
run.stdout
);
let kept = fixture
.server
.stored("unified")
.into_iter()
.find(|item| item.contains("Haircut"))
.expect("the event should still be in the aggregate");
// Stripping it would let the default sink claim the event next cycle.
assert!(kept.contains("@local"), "cycle {cycle}: {kept}");
assert!(
!fixture
.server
.stored("work")
.iter()
.any(|item| item.contains("Haircut")),
"cycle {cycle}: it reached the default sink anyway"
);
}
}
/// Discovery, against a server that really has two calendars.
///
/// The command exists because the CalDAV `url` must name the collection exactly
/// and providers rarely show it. What it prints therefore has to be usable
/// as-is, which is what the parse-back below actually checks.
#[test]
fn discover_lists_the_servers_calendars_as_pasteable_config() {
let Some(fixture) = fixture() else { return };
let run = fixture.calcalist.run(&[
"discover",
&format!(
"http://127.0.0.1:{}/{}/",
fixture.server.port,
support::USER
),
"--username",
support::USER,
"--secret-command",
"printf password",
]);
assert!(
run.succeeded(),
"discover failed\n{}\n{}",
run.stdout,
run.stderr
);
assert!(run.stdout.contains("id = \"work\""), "{}", run.stdout);
assert!(run.stdout.contains("id = \"unified\""), "{}", run.stdout);
// The URL has to come back absolute, not as the bare path pimsync reports.
assert!(
run.stdout
.contains(&format!("http://127.0.0.1:{}/", fixture.server.port)),
"{}",
run.stdout
);
// The point of the command: what it prints is valid configuration.
let config = format!("version = 1\n{}", run.stdout);
let parsed: toml::Value = toml::from_str(&config).expect("the output should be valid TOML");
let endpoints = parsed["endpoint"].as_array().expect("endpoints");
assert_eq!(endpoints.len(), 2, "{}", run.stdout);
assert_eq!(endpoints[0]["type"].as_str(), Some("caldav"));
assert_eq!(
endpoints[0]["secret_command"].as_str(),
Some("printf password")
);
}
/// An unreachable server has to say so rather than print an empty result that
/// looks like "this server has no calendars".
#[test]
fn discover_reports_a_server_it_cannot_reach() {
if !support::missing_binaries().is_empty() {
eprintln!("skipped: run under devbox");
return;
}
let root = tempfile::tempdir().expect("temp");
let calcalist = Calcalist::new(root.path(), "version = 1\n");
// Port 9 is discard: it refuses or blackholes, and nothing listens for DAV.
let run = calcalist.run(&[
"discover",
"http://127.0.0.1:9/calendars/me/",
"--username",
"me",
]);
assert!(
!run.succeeded(),
"it should not claim success\n{}",
run.stdout
);
assert!(
!run.stderr.is_empty(),
"the failure should be explained: {run:?}",
run = run.stderr
);
}
/// The lock is what stops a timer firing into a hand-run cycle.
///
/// The lock is taken here rather than by racing two real syncs, which would be
/// timing-dependent and would pass by accident most of the time.
#[test]
fn a_second_cycle_is_refused_while_one_is_running() {
let Some(fixture) = fixture() else { return };
assert!(fixture.calcalist.run(&["sync"]).succeeded());
let lock_path = fixture.calcalist.state.join("calcalist/lock");
let held = std::fs::OpenOptions::new()
.read(true)
.write(true)
.open(&lock_path)
.expect("the lock file should exist after a cycle");
rustix::fs::flock(&held, rustix::fs::FlockOperation::NonBlockingLockExclusive)
.expect("the test should be able to take the lock");
let refused = fixture.calcalist.run(&["sync"]);
assert!(!refused.succeeded(), "{}", refused.stdout);
assert!(
refused.stderr.contains("already running"),
"the refusal should say why: {}",
refused.stderr
);
drop(held);
assert!(
fixture.calcalist.run(&["sync"]).succeeded(),
"the lock should be free again"
);
}

378
tests/support/mod.rs Normal file
View file

@ -0,0 +1,378 @@
//! Scaffolding for the integration tests: a real CalDAV server, a real feed.
//!
//! Everything here talks HTTP over a plain socket rather than through a client
//! crate. The requests involved are few and mostly unusual — `MKCALENDAR`, `PUT`
//! of an `.ics` — and writing them out makes exactly what the server is asked
//! for visible in the test.
use std::io::{Read, Write};
use std::net::{Ipv4Addr, SocketAddr, TcpListener, TcpStream};
use std::path::{Path, PathBuf};
use std::process::{Child, Command, Stdio};
use std::time::{Duration, Instant};
/// How long to wait for a server to start answering.
const STARTUP_TIMEOUT: Duration = Duration::from_secs(20);
/// How many ports to try before giving up on starting a server.
const ATTEMPTS: usize = 8;
/// The account Radicale files everything under. With authentication disabled it
/// accepts whatever name arrives, and `owner_only` rights then grant that name
/// its own tree — so the URLs below all live beneath it.
pub const USER: &str = "calcalist";
/// A binary the integration tests need, and whether it is here.
pub fn missing_binaries() -> Vec<&'static str> {
["radicale", "pimsync"]
.into_iter()
.filter(|binary| which(binary).is_none())
.collect()
}
fn which(binary: &str) -> Option<PathBuf> {
std::env::var_os("PATH")?
.to_str()?
.split(':')
.map(|dir| Path::new(dir).join(binary))
.find(|candidate| candidate.is_file())
}
/// A Radicale instance with its own storage, shut down when dropped.
pub struct Radicale {
process: Child,
pub port: u16,
storage: PathBuf,
}
impl Radicale {
pub fn start(root: &Path) -> Radicale {
let storage = root.join("radicale");
std::fs::create_dir_all(&storage).expect("create storage");
// A free port is chosen by binding one and letting go, so two tests
// running at once can pick the same number. The loser's radicale then
// fails to bind and exits — and, worse, waiting for the port to answer
// succeeds anyway, because the winner is listening on it. Tests would
// quietly share a server. So: confirm the child we spawned is the one
// still alive, and take a different port if it is not.
for _ in 0..ATTEMPTS {
let port = free_port();
let mut process = Command::new("radicale")
.arg("--config")
.arg("")
.arg("--server-hosts")
.arg(format!("127.0.0.1:{port}"))
.arg("--auth-type")
.arg("none")
.arg("--storage-filesystem-folder")
.arg(&storage)
.stdout(Stdio::null())
.stderr(Stdio::null())
.spawn()
.expect("radicale should start");
if listening(&mut process, port) {
return Radicale {
process,
port,
storage,
};
}
let _ = process.kill();
let _ = process.wait();
}
panic!("radicale did not come up on a port of its own after {ATTEMPTS} attempts");
}
pub fn url(&self, calendar: &str) -> String {
format!("http://127.0.0.1:{}/{USER}/{calendar}/", self.port)
}
/// Creates a calendar collection. pimsync deliberately never creates one,
/// so the calendars a test syncs have to exist on the server first.
pub fn create_calendar(&self, calendar: &str) {
let body = "<?xml version=\"1.0\" encoding=\"utf-8\"?>\
<C:mkcalendar xmlns:D=\"DAV:\" xmlns:C=\"urn:ietf:params:xml:ns:caldav\">\
<D:set><D:prop><D:displayname>calendar</D:displayname></D:prop></D:set>\
</C:mkcalendar>";
let response = self.request(
"MKCALENDAR",
&format!("/{USER}/{calendar}/"),
"application/xml; charset=utf-8",
body,
);
// Radicale answers HTTP/1.0, so the status is read out of the line
// rather than matched against a whole prefix.
assert!(
matches!(status_of(&response), Some(201 | 405)),
"creating {calendar}: {response}"
);
}
/// Stores an event, as a calendar client would.
pub fn put_event(&self, calendar: &str, name: &str, ics: &str) {
let response = self.request(
"PUT",
&format!("/{USER}/{calendar}/{name}"),
"text/calendar; charset=utf-8",
ics,
);
assert!(
status_of(&response).is_some_and(|status| (200..300).contains(&status)),
"storing {name}: {response}"
);
}
/// Everything the server holds in a calendar, as stored.
///
/// Read from Radicale's own storage rather than fetched back, so what is
/// asserted on is the bytes that reached the server.
pub fn stored(&self, calendar: &str) -> Vec<String> {
let dir = self
.storage
.join("collection-root")
.join(USER)
.join(calendar);
let Ok(entries) = std::fs::read_dir(dir) else {
return Vec::new();
};
let mut items: Vec<String> = entries
.flatten()
.filter(|entry| {
entry
.path()
.extension()
.is_some_and(|extension| extension == "ics")
})
.filter_map(|entry| std::fs::read_to_string(entry.path()).ok())
.collect();
items.sort();
items
}
fn request(&self, method: &str, path: &str, content_type: &str, body: &str) -> String {
// Retried because a server under parallel test load occasionally resets
// a connection it has accepted; that says nothing about the request.
let mut last = String::new();
for attempt in 0..ATTEMPTS {
match self.try_request(method, path, content_type, body) {
Ok(response) => return response,
Err(error) => {
last = error;
std::thread::sleep(Duration::from_millis(50 * (attempt as u64 + 1)));
}
}
}
panic!("{method} {path} never got a response: {last}");
}
fn try_request(
&self,
method: &str,
path: &str,
content_type: &str,
body: &str,
) -> Result<String, String> {
let address = SocketAddr::from((Ipv4Addr::LOCALHOST, self.port));
let mut stream = TcpStream::connect_timeout(&address, Duration::from_secs(2))
.map_err(|error| error.to_string())?;
stream
.set_read_timeout(Some(Duration::from_secs(10)))
.map_err(|error| error.to_string())?;
// With authentication disabled the credentials only pick the account.
let authorization = base64(format!("{USER}:password").as_bytes());
let request = format!(
"{method} {path} HTTP/1.1\r\nHost: 127.0.0.1:{}\r\n\
Authorization: Basic {authorization}\r\nContent-Type: {content_type}\r\n\
Content-Length: {}\r\nConnection: close\r\n\r\n{body}",
self.port,
body.len()
);
stream
.write_all(request.as_bytes())
.map_err(|error| error.to_string())?;
let mut response = String::new();
stream
.read_to_string(&mut response)
.map_err(|error| error.to_string())?;
Ok(response)
}
}
impl Drop for Radicale {
fn drop(&mut self) {
let _ = self.process.kill();
let _ = self.process.wait();
}
}
/// A one-file HTTP server, standing in for a published iCal feed.
pub struct Feed {
pub port: u16,
}
impl Feed {
/// Serves `ics` at any path, for as long as the test runs.
pub fn start(ics: String) -> Feed {
let listener = TcpListener::bind(SocketAddr::from((Ipv4Addr::LOCALHOST, 0)))
.expect("bind the feed port");
let port = listener.local_addr().expect("feed address").port();
std::thread::spawn(move || {
for stream in listener.incoming().flatten() {
serve_once(stream, &ics);
}
});
Feed { port }
}
pub fn url(&self) -> String {
format!("http://127.0.0.1:{}/holidays.ics", self.port)
}
}
fn serve_once(mut stream: TcpStream, ics: &str) {
// Enough of the request to reach the blank line; the path does not matter.
let mut buffer = [0u8; 2048];
let _ = stream.read(&mut buffer);
let response = format!(
"HTTP/1.1 200 OK\r\nContent-Type: text/calendar; charset=utf-8\r\n\
Content-Length: {}\r\nConnection: close\r\n\r\n{ics}",
ics.len()
);
let _ = stream.write_all(response.as_bytes());
}
/// Runs `calcalist` against a config and state directory of the test's own.
pub struct Calcalist {
pub config: PathBuf,
pub state: PathBuf,
}
/// What one `calcalist` invocation printed and returned.
pub struct Run {
pub status: Option<i32>,
pub stdout: String,
pub stderr: String,
}
impl Run {
pub fn succeeded(&self) -> bool {
self.status == Some(0)
}
}
impl Calcalist {
pub fn new(root: &Path, config: &str) -> Calcalist {
let config_path = root.join("calcalist.toml");
std::fs::write(&config_path, config).expect("write config");
let state = root.join("state");
std::fs::create_dir_all(&state).expect("create state directory");
Calcalist {
config: config_path,
state,
}
}
/// The same configuration with the read-only feed taken out, for the test
/// that retires an endpoint.
pub fn config_without_feed(&self) -> String {
let text = std::fs::read_to_string(&self.config).expect("read config");
text.split("\n\n")
.filter(|block| !block.contains("id = \"holidays\""))
.map(|block| block.replace(", \"holidays\"", ""))
.collect::<Vec<_>>()
.join("\n\n")
}
pub fn run(&self, arguments: &[&str]) -> Run {
let output = Command::new(env!("CARGO_BIN_EXE_calcalist"))
.arg("--config")
.arg(&self.config)
.args(arguments)
// The state directory is resolved through XDG, so this is what
// keeps the test off the developer's own calendars and state.
.env("XDG_STATE_HOME", &self.state)
.output()
.expect("calcalist should run");
Run {
status: output.status.code(),
stdout: String::from_utf8_lossy(&output.stdout).into_owned(),
stderr: String::from_utf8_lossy(&output.stderr).into_owned(),
}
}
}
/// The status code out of a response's first line, whatever HTTP version it
/// claims.
fn status_of(response: &str) -> Option<u16> {
response
.lines()
.next()?
.split_whitespace()
.nth(1)?
.parse()
.ok()
}
fn free_port() -> u16 {
let listener =
TcpListener::bind(SocketAddr::from((Ipv4Addr::LOCALHOST, 0))).expect("bind a free port");
listener.local_addr().expect("port").port()
}
/// Whether a real HTTP request gets a real HTTP response.
fn answers(address: SocketAddr) -> bool {
let Ok(mut stream) = TcpStream::connect_timeout(&address, Duration::from_millis(200)) else {
return false;
};
let _ = stream.set_read_timeout(Some(Duration::from_secs(2)));
let request = format!("OPTIONS / HTTP/1.1\r\nHost: {address}\r\nConnection: close\r\n\r\n");
if stream.write_all(request.as_bytes()).is_err() {
return false;
}
let mut response = String::new();
stream.read_to_string(&mut response).is_ok() && response.starts_with("HTTP/")
}
/// Waits for our own radicale to answer, giving up as soon as it has exited.
///
/// The liveness check is the point: without it a process that lost a port race
/// looks healthy, because something else is answering on that port.
fn listening(process: &mut Child, port: u16) -> bool {
let address = SocketAddr::from((Ipv4Addr::LOCALHOST, port));
let deadline = Instant::now() + STARTUP_TIMEOUT;
while Instant::now() < deadline {
match process.try_wait() {
Ok(Some(_)) => return false,
Ok(None) => {}
Err(_) => return false,
}
// An open port is not readiness: the socket listens before the
// application behind it can answer, and under parallel tests that gap
// is wide enough to get a connection reset instead of a response.
if answers(address) {
return true;
}
std::thread::sleep(Duration::from_millis(50));
}
false
}
fn base64(input: &[u8]) -> String {
const ALPHABET: &[u8] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
let mut out = String::new();
for chunk in input.chunks(3) {
let mut block = [0u8; 3];
block[..chunk.len()].copy_from_slice(chunk);
let packed = u32::from(block[0]) << 16 | u32::from(block[1]) << 8 | u32::from(block[2]);
for index in 0..4 {
if index <= chunk.len() {
out.push(ALPHABET[(packed >> (18 - index * 6)) as usize & 0x3f] as char);
} else {
out.push('=');
}
}
}
out
}