diff --git a/README.md b/README.md new file mode 100644 index 0000000..60deca9 --- /dev/null +++ b/README.md @@ -0,0 +1,390 @@ +# calcalist + +Aggregate several calendars into one, and keep them in step. + +calcalist mirrors any number of CalDAV calendars, Google calendars and iCal +feeds into a single calendar you subscribe to. The point is to subscribe to one +calendar instead of several: an event created or edited in the aggregate travels +back to the calendar it came from, so the aggregate is somewhere to work rather +than a read-only noticeboard. CalDAV and Google are read/write in both +directions; iCal feeds are read-only, because the protocol is. + +This is pre-1.0. The command line and the configuration format are settled; +there is no web interface yet. `SPECS.md` covers why it is built this way. + +## How it works + +Every endpoint — the sources and the aggregate's target alike — is mirrored to a +directory of `.ics` files on your machine, and the aggregation engine works +purely on those files. One cycle is: pull every remote into its local mirror, +reconcile the mirrors against each other, push the results back out. + +`pimsync` carries the CalDAV and iCal legs. calcalist carries the Google leg +itself, over the Calendar REST API — pimsync cannot reach Google at all, having +no REST storage and only HTTP Basic auth, which Google's CalDAV endpoint has +rejected since March 2025. + +## Install + +```sh +cargo build --release +# target/release/calcalist +``` + +One runtime dependency: **`pimsync` 0.5.x on `PATH`**, needed only if any +endpoint is CalDAV or an iCal feed. A Google-only setup needs nothing else. +pimsync is in nixpkgs; this repo's `devbox.json` pins 0.5.11 if you would rather +not install it yourself. + +Then check the environment: + +```sh +calcalist doctor +``` + +It reports whether pimsync is present and in the supported version series, +whether the state directory is writable, whether your configuration is valid, +whether pimsync accepts the configuration calcalist generates for it, and +whether each Google endpoint still has a usable authorisation. + +## Configuration + +One TOML file at `$XDG_CONFIG_HOME/calcalist/calcalist.toml`, or wherever +`--config` points. + +It holds **no secrets and no sync state**, so it is safe to copy between +machines. Credentials come from `*_command` fields instead: each is run through +`sh -c` and the first line of its output is used. Point them at `secret-tool`, +`pass`, `gpg -d`, or anything else that prints the secret. + +```toml +version = 1 +``` + +### Endpoints + +Each `[[endpoint]]` is one calendar, with an `id` you refer to it by elsewhere. + +**CalDAV** + +```toml +[[endpoint]] +id = "posteo" +type = "caldav" +url = "https://posteo.de:8443/calendars/you/abc123/work/" +username = "you@posteo.de" +secret_command = "secret-tool lookup service posteo user you@posteo.de" +``` + +`url` must name the calendar collection itself, not the server root or your +principal — the last path segment is the calendar. A URL with no collection in +it is rejected, and `calcalist doctor` says so. + +**Google** + +```toml +[[endpoint]] +id = "gcal" +type = "google" +calendar_id = "you@gmail.com" +client_id = "1234-abcd.apps.googleusercontent.com" +client_secret_command = "secret-tool lookup service google-calcalist" +# account = "you@gmail.com" # only when logged in to more than one account +``` + +`calendar_id` is `primary`, or the calendar's own address — the "Calendar ID" +under the calendar's settings in Google Calendar. See +[Authorising Google](#authorising-google) below for `client_id` and the secret. + +**iCal feed** + +```toml +[[endpoint]] +id = "holidays" +type = "webcal" +url = "https://example.org/holidays.ics" +``` + +Always read-only. Give exactly one of `url` or `url_command` — the latter for a +feed whose address is itself a credential, Google's "secret address in iCal +format" being the case in point, since anyone holding it can read the whole +calendar: + +```toml +url_command = "secret-tool lookup service google-secret-ics" +``` + +### Aggregates + +Each `[[aggregate]]` composes endpoints: several `sources` mirrored into one +`target`. + +```toml +[[aggregate]] +id = "unified" +target = "posteo" +sources = ["gcal", "holidays"] +default_sink = "gcal" + +# Defaults, all optional: +conflict = "source_wins" # the only policy; the origin calendar wins +propagate_deletes = true # deleting a mirror deletes the original +max_delete_fraction = 0.2 # abort if more of the aggregate than this vanishes +``` + +`max_delete_fraction` guards against a cycle that would remove a large part of +your calendars — a mistyped URL, a server answering with an empty collection. It +has an absolute floor of three deletions, so removing one or two events is never +refused. `calcalist sync --force` overrides it. + +The rules, all checked before anything is synchronised, with every problem +reported in one pass: + +- `target` must exist and be writable, so never an iCal feed. +- `target` must not also be one of its own `sources`. +- `sources` must be non-empty, and each must exist and be named once. +- `default_sink`, if given, must be one of the `sources` and must be writable. + +### A complete example + +A Google calendar and a public holiday feed, aggregated into a CalDAV calendar: + +```toml +version = 1 + +[[endpoint]] +id = "gcal" +type = "google" +calendar_id = "primary" +client_id = "1234-abcd.apps.googleusercontent.com" +client_secret_command = "secret-tool lookup service google-calcalist" + +[[endpoint]] +id = "holidays" +type = "webcal" +url = "https://www.officeholidays.com/ics/germany" + +[[endpoint]] +id = "posteo" +type = "caldav" +url = "https://posteo.de:8443/calendars/you/abc123/unified/" +username = "you@posteo.de" +secret_command = "secret-tool lookup service posteo user you@posteo.de" + +[[aggregate]] +id = "unified" +target = "posteo" +sources = ["gcal", "holidays"] +default_sink = "gcal" +``` + +## Authorising Google + +Google requires an OAuth client of your own. This is the fiddliest part of the +setup, so in full: + +1. At [console.cloud.google.com](https://console.cloud.google.com), create a + project — any name. +2. **APIs & Services → Library**, find **Google Calendar API**, enable it. +3. **OAuth consent screen**: choose **External**. Fill in the required fields. + Under **Test users**, add your own Google address. Without this, authorising + fails with `403: access_denied` even though the account is your own. +4. **Credentials → Create credentials → OAuth client ID**, and for + **Application type** choose **Desktop app**. + + Desktop app matters. calcalist listens on a loopback port it picks at run + time and hands Google that address as the redirect, so there is no redirect + URI to register and no domain to verify. A Web application client asks for + both and cannot be made to work here. +5. Copy the client ID into `client_id`, and put the client secret somewhere + `client_secret_command` can read it: + + ```sh + secret-tool store --label="calcalist Google client secret" service google-calcalist + ``` +6. Authorise: + + ```sh + calcalist google login gcal + ``` + + A browser opens; approve the request. calcalist asks only for + `https://www.googleapis.com/auth/calendar`. The refresh token is written + `0600` into the state directory, never into your configuration. + +**One login covers every endpoint on that account.** The authorisation is filed +under the account it was granted for, not the endpoint that asked, so a second +Google calendar on the same account needs no login of its own. Set `account` on +an endpoint only when calcalist is logged in to more than one account and has to +be told which is meant. + +**While the consent screen is in Testing, Google expires refresh tokens after +seven days.** An installation that worked last week then stops with nothing +having changed locally; `calcalist google login` again, or publish the consent +screen. `calcalist doctor` reports this rather than letting it surface as an +opaque failure. + +## Routing: getting a new event to the right calendar + +Every mirrored event carries where it came from — `X-CALCALIST-SOURCE` and +`X-CALCALIST-ORIGIN-UID` — and a UID derived from its origin, so an edit made in +the aggregate goes back to the calendar that owns the event without any +guesswork. + +An event you *create* in the aggregate belongs to nothing yet. It goes to the +aggregate's `default_sink`. To send it somewhere else, put the endpoint's id +after an `@` **on a line of its own** in the event's notes: + +``` +Dentist, bring the referral + +@gcal +``` + +The marker is removed before the event reaches the calendar. A whole line is +required so that an address or a handle written in prose is not mistaken for an +instruction. + +`calcalist status` prints the markers that would actually work, so you do not +have to remember what is in the configuration file: + +``` +aggregate `unified` + published to posteo + sources gcal, holidays + mirroring 42 event(s) + new events go to `gcal` unless told otherwise + to choose, put one of these on a line of its own in the event's notes: + @gcal +``` + +Read-only feeds are left out, since they cannot take an event. A marker naming +anything else is refused and reported — the event stays where it is rather than +being quietly filed under the default, which would put it somewhere you did not +ask for. + +A **category** matching an endpoint id works too, that being the field +iCalendar intends for this. Be aware that any category on a newly created event +is read as a routing instruction, so an event carrying an unrelated category +will be refused rather than sent to the default sink. The notes marker is the +safer form, and the one every mobile client exposes. + +### Attendees and alarms + +Writing to an aggregate must never mail anyone: the guests were already invited +from the original calendar. On a Google target, attendees are kept verbatim and +Google is told not to notify. A CalDAV server offers no portable way to be told +that, so the guest list is instead carried as inert data — the live `ATTENDEE` +and `ORGANIZER` properties are dropped, the guests appear in +`X-CALCALIST-ATTENDEES` and in the description, and a meeting you declined is +marked free. + +Routing works the other way round: an event you created and sent to a source is +a meeting you are deliberately organising, so its attendees are preserved and +that server invites them for real. + +Alarms are always kept, with no option to strip them. The whole point of the +tool is that you subscribe to one calendar rather than several, so dropping +reminders would silently disarm every one you have. + +## Commands + +``` +calcalist sync [--dry-run] [--force] +calcalist status +calcalist doctor +calcalist prune [--force] +calcalist google login +calcalist aggregate retarget --to [--keep-old | --purge-old] +``` + +- **`sync`** runs one cycle. `--dry-run` pulls from the servers for real, into a + throwaway copy of the local mirrors, so what it reports is measured against + your calendars as they are now — and nothing outside that copy is written. + `--force` overrides the mass-deletion guard. +- **`status`** lists endpoints and aggregates, how many events each aggregate is + mirroring, and the routing markers that would work. +- **`doctor`** checks the environment and configuration. Run it first. +- **`prune`** reports local mirrors belonging to endpoints your configuration no + longer names. It only lists them until you pass `--force`, since an endpoint + may just have been renamed. +- **`aggregate retarget`** moves an aggregate to a different target calendar + deliberately. `sync` refuses to run when an aggregate's `target` has changed + under it and points here instead — the new, empty target would otherwise read + as an aggregate whose every event had been deleted, and with delete + propagation on, that would remove them from every source calendar. Events left + on the old target are kept unless you pass `--purge-old`, which only removes + events calcalist put there. + +Exit codes: `0` all well, `1` something failed or an endpoint could not be +reached, `2` not implemented. + +If a Google endpoint cannot be reached — a lapsed token, no network — the cycle +does not stop. The endpoint is named, only the aggregates that depend on it +stand down, everything else is synchronised as usual, and the run exits `1` so +the failure cannot pass unnoticed. + +## Where things are kept + +Everything machine-local lives under `$XDG_STATE_HOME/calcalist`: + +``` +state.json event mappings, content hashes, sync cursors +pimsync.conf generated on every sync; edits are overwritten +pimsync-status/ pimsync's own record of what it has seen +vdir// the local mirror of each endpoint +google/accounts/.json refresh tokens, 0600 +``` + +None of this belongs in a backup or a dotfiles repository. The state file is a +cache rather than a single point of failure — aggregate UIDs are derived from +their origin, so a lost state file is rebuilt by re-deriving it. Copy the +configuration between machines; leave this behind. + +## Running it regularly + +There are no packaged units yet. A systemd user timer does the job: + +`~/.config/systemd/user/calcalist.service` + +```ini +[Unit] +Description=Synchronise calendars with calcalist + +[Service] +Type=oneshot +ExecStart=%h/.local/bin/calcalist sync +``` + +`~/.config/systemd/user/calcalist.timer` + +```ini +[Unit] +Description=Synchronise calendars every 15 minutes + +[Timer] +OnBootSec=2m +OnUnitActiveSec=15m +Persistent=true + +[Install] +WantedBy=timers.target +``` + +```sh +systemctl --user enable --now calcalist.timer +``` + +A cycle is one-shot, and systemd will not start the service again while a run is +still going, so a timer is all the scheduling needed. calcalist takes no lock of +its own, though, so avoid running `calcalist sync` by hand while the timer might +fire. If your secrets come from a keyring that needs an unlocked session, the +timer will only work while you are logged in. + +## Not yet + +- **The web configuration interface.** `calcalist serve` exits 2. +- **Packaged systemd units**, as above. +- **A failing `pimsync sync` stops the whole CalDAV leg.** Google endpoints are + isolated from each other, but pimsync is a single process covering every + CalDAV and feed pair, and a failure does not say which pair it belongs to.