smolboid/README.md

198 lines
13 KiB
Markdown
Raw Normal View History

2026-10-05 11:12:23 +03:00
# smolboid: signed twtxt over smolmail
smolboid is a small daemon that serves [twtxt](https://twtxt.readthedocs.io/) feeds for many users and lets them publish and manage their feed by sending ordinary [smolmail](https://code.randogoth.com/randogoth/smolmail). A post is a mail body; management rides custom frontmatter fields. Every published record carries the author's own smolmail payload signature, so any reader can verify authorship offline against the author's key alone — the host can withhold records but cannot forge one. No extra client tool is needed: any conformant smolmail client can post, and legacy twtxt and Yarn clients read the feeds unchanged.
The workspace is `core/`, the feed format and verification library with no I/O, and `daemon/`, the server binary. [audit.md](audit.md) records the security audit, its findings and the test suites that prove them.
## Build and test
devbox manages the toolchain; every script runs in it.
```
devbox run build # compile the workspace
devbox run test # unit, property and vector tests
devbox run vectors # regenerate and pin core/vectors.json
devbox run rig # end-to-end against a local bunshin, the fumi CLI as author
devbox run stress # mail flood, quotas, reply budget, HTTP load
devbox run pentest # adversarial probes, raw sockets and real mail
devbox run audit-deps # cargo audit over the dependency tree
```
## Running an instance
The daemon holds one smolmail identity — a 32-byte master key — and one mailbox, e.g. `twtxt@smol.place`. Bootstrap its mailbox once with the fumi CLI against the same key and store, because the first server pin and the registration are operator actions:
```
smolboid keygen /srv/smolboid/master.key
fumi --key /srv/smolboid/master.key --db /srv/smolboid/data/mail.db trust smol.place <server-key-b32>
fumi --key /srv/smolboid/master.key --db /srv/smolboid/data/mail.db register twtxt@smol.place
```
Then configure and serve. The config file is one flat `key = value` list; anything unset keeps its default:
```
# smolboid.toml
listen = 127.0.0.1:8080 # HTTP listener for the feeds
base_url = https://smol.place # the origin the feeds' # url metadata carries
smolmail_host = smol.place # the host serving the daemon's mailbox
mailbox = twtxt # local part of the mailbox, for display
master = /srv/smolboid/master.key
data = /srv/smolboid/data # state.json, mail.db and feeds/
poll_secs = 60 # fetch interval
timeout = 30 # smolmail network timeout, seconds
posts_per_day = 50 # per-feed daily quota
reposts_per_day = 50
max_post_chars = 288 # longest post, in characters (code points)
2026-10-05 11:12:23 +03:00
cap_bytes = 1048576 # per-feed size cap, default 1 MiB
registration = silent # open | closed | silent — see Admission
registrations_per_hour = 10 # global cap on attempts that reach the network
info_replies_per_hour = 30 # global cap on replies to unknown senders
admins = # comma-separated addresses allowed admin commands
admin_notify = on # mail every admin when a feed is created
register_hosts = # host allowlist; empty means any host
2026-10-05 11:12:23 +03:00
```
```
smolboid serve --config smolboid.toml
```
## Admission
Posting needs an accept token (§5.8), and a feed exists only once its owner's address has been verified. There are two ways to get there.
**By mail, if `registration` is `open`.** An author writes to the mailbox; the daemon replies telling them how to register; they send `Twt-Register` and get a feed, a token and a confirmation. No operator action at all. See [The author's side](#the-authors-side).
Before the daemon answers an unknown sender *at all*, it resolves the address their signed `Reply-To` claims, on that address's own server, and requires the resolved key to equal the key that signed the mail. The claim alone proves only that the sender signed that URI — not that they hold the mailbox it names — so without this check a signed `Reply-To` naming someone else's address would be enough to take their feed. Replying to the claimed `smol://` URI would be worse still: it would pin the sender's key as verified under that address, and the real owner could never register afterwards.
`registration` is `silent` by default, which drops unknown senders without a reply exactly as every version before self-service did. `closed` tells them registration is shut. An admin can change it by mail, so the mode lives in `state.json` and the config key is only its initial value.
**By the operator**, whatever the mode:
2026-10-05 11:12:23 +03:00
```
smolboid admit --config smolboid.toml tobias@smol.place # verify, accept, create the feed
smolboid block --config smolboid.toml tobias@smol.place # withdraw the token
smolboid check /srv/smolboid/data/feeds/tobias.txt # verify a feed's records, nonzero on forged ones
```
`admit` passes no expected key, because the target has not written in — it prints the resolved key and fingerprint for an out-of-band check instead. A mail-driven registration always passes one.
## Admin commands
Listed addresses in `admins` may run these by mail, from their own address, so an instance can be operated without shell access:
| Field | Value | Effect |
| --- | --- | --- |
| `Twt-Admit` | an address | Verify, accept and create that author's feed |
| `Twt-Block` | an address | Withdraw their token; the feed keeps being served |
| `Twt-Registration` | `open` \| `closed` \| `silent` | Change who may register |
An admin's own mail reaches the main tier only once the daemon holds a token for them, so admit your admin identity once with the CLI; until then the acks arrive in your requests tier. An admin with no feed still has to prove they hold their address, by the same resolve as any stranger: a signed `Reply-To` naming an admin proves nothing on its own.
A non-admin sending one of these gets the ordinary answer for their situation and is never told that admin commands exist.
## Deploying with Nix
The flake exposes `packages.default`, `packages.static` (a static musl binary,
for deploy targets that should not compile Rust) and `nixosModules.default`:
```nix
{
inputs.smolboid.url = "git+https://code.randogoth.com/randogoth/smolboid.git";
# in your nixosSystem modules:
imports = [ inputs.smolboid.nixosModules.default ];
services.smolboid = {
enable = true;
listen = "127.0.0.1:8098";
baseUrl = "https://smol.place";
smolmailHost = "smol.place";
masterFile = "/run/smolboid/master.key"; # 32 raw bytes, provisioned out of band
};
}
```
The module runs the daemon as a real `smolboid` user rather than under
`DynamicUser`, because `admit`, `block` and the fumi bootstrap are operator
commands against the same master key and store and need a stable uid to
`sudo -u` into. It does not generate or manage the master.
`nix run .#release-static` builds the static binary and publishes it as a
Forgejo release on this repo, tagged from `daemon/Cargo.toml`'s version; it
refuses unless HEAD carries that tag. Needs `FORGEJO_TOKEN` with
`write:repository`.
`fumi-core` is a path dependency on a sibling `../fumi` checkout, which escapes
this repo and so cannot be reached from a flake `src`. The flake vendors fumi
as its own input and re-lays it beside the source tree in the store, the way
bunshin vendors microReticulum. One consequence: `nix build` compiles against
the **locked** fumi rev while `devbox run build` uses the sibling worktree, so
the two can disagree — the lock is the one that ships.
2026-10-05 11:12:23 +03:00
## The author's side
Ask for a feed, if the instance is open:
2026-10-05 11:12:23 +03:00
```
fumi send twtxt@smol.place --header "Twt-Register: yournick"
```
Then fetch the reply — it carries the accept token that moves your posts out of the requests tier — and post by sending the twt as the mail body, from any smolmail client:
```
fumi fetch
2026-10-05 11:12:23 +03:00
fumi send twtxt@smol.place --body "hello smolverse"
```
**Do not accept the daemon's address in your client.** Replies arrive in your requests tier (`fumi list --requests`), which is where to look for them. If you accept the address, your client attaches an accept token to everything you send afterwards — and a published record is verified by rebuilding the exact bytes you signed, which cannot include a token without publishing it. Every post you sent would be refused, with a reply saying so.
2026-10-05 11:12:23 +03:00
Management travels as frontmatter fields, which fumi emits with `--header`:
| Field | Value | Effect |
| --- | --- | --- |
| `Twt-Nick` | 1-30 bytes of `[a-z0-9-]` | Set the display nick; the feed path never changes |
| `Twt-Delete` | a record timestamp | Remove that post or repost |
| `Twt-Repost` | `<mention> <#hash>` | Publish a repost |
| `Twt-Prune` | `fifo` or `off` | Choose what a full feed does: prune its oldest records, or reject new ones (the default) |
| `Twt-Register` | empty, or a nick | Ask for a feed. Only for a sender who has none |
| `Twt-Help` | empty | Mail back this list and your feed's address |
One command per mail. Two of them together, or one mixed with another, is refused rather than half-applied.
2026-10-05 11:12:23 +03:00
```
fumi send twtxt@smol.place --header Twt-Delete:2026-10-01T12:47:00Z --body ""
```
The rules the daemon enforces: text is at most `max_post_chars` characters (288 by default; twtxt itself sets no hard limit) and may not contain control characters such as TAB, LF or CR, as twtxt requires, while U+2028 is allowed as the Multiline extension's line break; the timestamp is the mail's own time, pinned by the signature, so a same-second resend is rejected with a reply — wait a second and resend; at 90% of the size cap the daemon mails a warning naming the cap and both prune modes. Error replies are subject to the address's rules: at most ten per sender per hour, never twice for the same rejected payload.
2026-10-05 11:12:23 +03:00
## What a feed looks like
A standard twtxt file plus `# smol:` comment lines. Legacy clients ignore them; smol-aware clients verify.
```
# nick = tobias
# url = https://smol.place/twtxt/tobias.txt
# smol:author = tobias@smol.place
# smol:host_key = <52-char base32>
2026-10-01T12:47:00Z hello smolverse
# smol:mail <sender b32> <epk b32> <signature b32>
```
Each record's signature covers the smolmail payload bytes: the author's key, the record's timestamp, and the text in one of the two canonical forms a real client emits (bare, or with the ordinary send's `Reply-To` block). The verifier rebuilds those bytes from the record itself plus the feed's metadata, tries each `smol:host_key` as the recipient, and marks the record verified, unsigned or invalid. `smolboid check` reports exactly that.
Feeds are served at `GET /twtxt/<username>.txt` (`HEAD` answers with the same headers, no body) with a strong SHA-256 ETag, `Last-Modified`, `Cache-Control: max-age=300` and `Access-Control-Allow-Origin: *`; `If-None-Match` takes precedence over `If-Modified-Since`, and both answer 304. Unknown and missing feeds are 404; the listener sheds concurrent-connection floods with 503 and closes stalled connections after ten seconds.
## The deployment trust contract
Read this before running an instance on the public internet.
**TLS is your job.** The listener is plain HTTP. The `base_url` above assumes TLS termination at a reverse proxy in front of the daemon; pointed at the internet directly, it serves feeds — public data — in cleartext, and a network attacker could serve stale or substituted feeds to readers. Terminate TLS upstream, or front the daemon with whatever serves the rest of your domain.
**The first pin is on you.** Bootstrap is trust-on-first-use: `fumi trust` pins your smolmail server's Noise static key and `smolboid admit` resolves each author on their own server with the same semantics. A lying DNS path fails closed at the pin mismatch, but the *first* pin you enter is taken on faith. Verify the server key and your authors' keys out of band before admitting them; `smolboid admit` prints each resolved key.
**The master key is the whole identity.** The 32-byte master at 0600 is the only secret on disk — everything else under `data/` is public keys, counters and ledgers. Back it up, because a lost master means a new identity and a new `smol:host_key` in every feed; a stolen one means an attacker can register the daemon's mailbox elsewhere and post as it, though they cannot forge the authors' records already published.
**One domain per instance.** The feed path is fixed to the author's smolmail username, and discovery resolves `user@domain` to `https://domain/twtxt/<username>.txt`, so one instance serves one domain; multiple domains run multiple daemons.
**Authors own their words, not your storage.** An admitted author can fill their quota daily and switch to FIFO pruning; `cap_bytes` is the bound. An unknown sender publishes nothing and gets no token; with `registration = open` they draw at most one reply telling them how to register, bounded by `info_replies_per_hour` globally and the per-sender hourly budget. With the default `silent` they draw nothing at all.