From d6fa0976266e09ae102d97ed1e8fd1eab80ff074 Mon Sep 17 00:00:00 2001 From: randogoth Date: Sun, 27 Sep 2026 11:14:25 +0300 Subject: [PATCH] feat: adopt smolmail protocol 1.1 with accept tokens and leave-mail-on-server --- README.md | 23 +-- test/client.mjs | 183 ++++++++++++++++---- test/e2e.py | 38 ++-- test/gen_vectors.py | 20 ++- test/vectors.json | 12 +- test/vectors.test.mjs | 45 +++-- web/index.html | 32 +++- web/js/app.js | 393 +++++++++++++++++++++++++++++++++++------- web/js/proto.js | 123 +++++++++---- web/js/store.js | 266 +++++++++++++++++++++------- web/style.css | 8 + 11 files changed, 911 insertions(+), 232 deletions(-) diff --git a/README.md b/README.md index 7333a5b..eb4cfe4 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # gsmol -An HTML5 browser client for [Smol Mail](../smolmail), the minimalist end-to-end encrypted mail protocol: one Ed25519 identity, five operations, Noise_NX transport, sealed and signed messages. Everything the reference CLI client does, in a browser tab. +An HTML5 browser client for [Smol Mail](../smolmail), the minimalist end-to-end encrypted mail protocol: one 32-byte master secret, five operations, Noise_NX transport, sealed and signed messages, and accept tokens that sort correspondence from cold mail. Everything the reference CLI client does, in a browser tab. ## Why there is a bridge @@ -18,7 +18,7 @@ browser (all crypto, all keys) ⇄ bridge.py (dumb byte pipe) ⇄ smolmaild devbox run serve ``` -Then open http://127.0.0.1:8096. First use: create an identity, back up the seed, pin your server's public key (from the operator, through a trusted channel), register an address, and fetch. +Then open http://127.0.0.1:8096. First use: create an identity, back up the master, pin your server's public key (from the operator, through a trusted channel), register an address, and fetch. For a server on a non-default port, start the bridge with `uv run bridge.py --allow-port `. @@ -64,11 +64,13 @@ The service runs under `DynamicUser`, restarts on failure, and — like the stan ## What it implements -- **Identity** (§2): one 32-byte seed; create or restore; the X25519 agreement keys are derived from the Ed25519 keypair. The seed lives in localStorage, mail stays sealed in IndexedDB — the browser profile is the trust boundary. +- **Identity** (§2): one 32-byte master secret; create or restore. Every rotation index's signing seed and the accept key are derived from it with HKDF, so the X25519 agreement keys and every superseded identity are re-derivable rather than archived. The master lives in localStorage, mail stays sealed in IndexedDB — the browser profile is the trust boundary. - **Addressing** (§3): short `user@host` and self-certifying `smol://` addresses, base32 fingerprints. -- **Trust** (§4, §8): server keys pinned on first trust, mismatch aborts the handshake; `RESOLVE` over an unpinned session is marked unverified; registration and fetching require a pinned server. -- **Mail** (§5): sealed and signed envelopes, 1 KiB padding, frontmatter bodies, sent copies sealed to self, fetch verifies id and signature before acknowledging — anything unreadable stays on the server. Outgoing mail carries a signed `Reply-To` frontmatter field with the sender's full `smol://` address (an "anonymous" checkbox omits it), so a first-time recipient can name the sender and answer; on receipt such a claim binds only when its key matches the message's signer, and never over an address already pinned to a different key. -- **Rotation** (§7): rotate with a signed certificate; contacts accept the change from the chain; superseded seeds are kept, since mail sealed to them is readable with nothing else; key changes without a chain are surfaced, never applied silently. +- **Trust** (§4, §8): server keys pinned on first trust, mismatch aborts the handshake; `RESOLVE` over an unpinned session is marked unverified; registration and fetching require a pinned server; `REGISTER` carries a proof-of-possession signature bound to the server's own key, so a binding cannot be replayed elsewhere. +- **Mail** (§5): sealed and signed envelopes, 1 KiB padding, frontmatter bodies, sent copies sealed to self, fetch pages forward by cursor and verifies id and signature before acknowledging — anything unreadable stays on the server. Outgoing mail carries a signed `Reply-To` frontmatter field with the sender's full `smol://` address (an "anonymous" checkbox omits it), so a first-time recipient can name the sender and answer; on receipt such a claim binds only when its key matches the message's signer, and never over an address already pinned to a different key. +- **Accept tokens** (§5.8): mail from an accepted correspondent lands in the inbox's main tier; everything else lands in a separate "requests" tier. Accepting or blocking a contact pushes the changed token set to the server immediately; a learned token is attached automatically to outgoing mail once a correspondent's own token has been received. +- **Leave mail on server**: a settings toggle that has fetch page forward by cursor without acknowledging, so mail stays server-side until deleted from the reader — off by default, which acknowledges (and so deletes) fetched mail immediately, as before. +- **Rotation** (§7): rotate to the next index's derived key with a certificate both the old and new keys sign, binding the change to this username; contacts accept it from the chain; every superseded key stays derivable from the master, since mail sealed to it is readable with nothing else; key changes without a chain are surfaced, never applied silently. ## Layout @@ -91,14 +93,15 @@ The crypto is pure JavaScript rather than WebCrypto so the exact same modules ru devbox run test ``` -1. `test/gen_vectors.py` regenerates `test/vectors.json` from the reference stack (hashlib/hmac, PyNaCl, noiseprotocol): hashes, AEAD, key agreement, signatures, the §2 conversions, a fixed-ephemeral envelope, and a deterministic Noise_NX transcript with internal states. -2. `test/vectors.test.mjs` checks the JS implementation byte for byte against those vectors, plus protocol cases: frontmatter, addresses, rotation chains (accepted, broken, missing link, oversize). -3. `test/e2e.py` runs a live `smolmaild.py` and drives the reference client (Alice) against this client's JS stack (Bob, headless via `test/client.mjs`): registration, both send directions, fetch with acknowledgment, and a rotation whose chain the reference client walks. Bob's reply travels through the bridge's WebSocket relay — the exact byte path the browser takes. +1. `test/gen_vectors.py` regenerates `test/vectors.json` from the reference stack (hashlib/hmac, PyNaCl, noiseprotocol): hashes, AEAD, key agreement, signatures, the §2 conversions, master-derived rotation seeds and accept tokens, a fixed-ephemeral envelope, and a deterministic Noise_NX transcript with internal states. +2. `test/vectors.test.mjs` checks the JS implementation byte for byte against those vectors, plus protocol cases: frontmatter (case-insensitive keys), addresses, and rotation chains bound to a username (accepted, broken, missing link, oversize, wrong username). +3. `test/e2e.py` runs a live `smolmaild.py` and drives the reference client (Alice) against this client's JS stack (Bob, headless via `test/client.mjs`): registration, unsolicited mail landing in the requests tier, accepting a correspondent and the accept-token round trip that follows, fetch with acknowledgment, and a rotation whose chain the reference client walks. Bob's reply travels through the bridge's WebSocket relay — the exact byte path the browser takes. What is not verified here: clicking through the actual browser UI. `devbox run serve` and a manual pass remain the last step; the logic under every button is what the tests exercise. ## Notes and limits -- The seed sits in localStorage: an XSS in the page or a compromised browser profile is game over, same as a stolen `identity.key` for the CLI client. The bridge serves a strict Content-Security-Policy (`default-src 'self'`, no inline script or style, no foreign origin) to raise that bar, but it is a mitigation and not a fix. No server push or notifications in v1 (SPEC.md §13); click fetch. +- The master sits in localStorage: an XSS in the page or a compromised browser profile is game over, same as a stolen `identity.key` for the CLI client. The bridge serves a strict Content-Security-Policy (`default-src 'self'`, no inline script or style, no foreign origin) to raise that bar, but it is a mitigation and not a fix. No server push or notifications; click fetch. - Rotation is not revocation (§7): a stolen key can rotate onward and the chain validates. The UI surfaces rotations instead of applying them invisibly; out-of-band re-verification is the only defence. +- The requests tier is a quota boundary the server enforces, not a filter: it does not refuse anyone's mail outright, which stays the reader's decision after decryption (§5.8). - The bridge binds to 127.0.0.1 by default and has no TLS of its own. Serving it beyond localhost directly is possible but unadvised: the relay is plaintext-agnostic (the traffic is already end-to-end encrypted, but the bridge would relay anyone's bytes to allowlisted ports). The NixOS module's `nginx.enable` puts a TLS-terminating reverse proxy in front instead, which is the deployment path "Deploy" above documents. diff --git a/test/client.mjs b/test/client.mjs index c443a91..87d5a74 100644 --- a/test/client.mjs +++ b/test/client.mjs @@ -5,8 +5,8 @@ // node test/client.mjs --state bob.json [args] // // Commands mirror the reference smolmail.py: keygen, trust, register, resolve, -// import, send, fetch, list, read, rotate. State lives in a JSON file; mail is -// stored sealed, like the browser store does. +// import, accept, block, send, fetch, list, read, rotate. State lives in a +// JSON file; mail is stored sealed, like the browser store does. import { existsSync, readFileSync, writeFileSync } from "node:fs"; import net from "node:net"; @@ -83,10 +83,43 @@ async function open(host, port, { requirePin }) { return opened; } +// §2: the identity at the state's current rotation index, plus every earlier +// one — mail sealed to a superseded key is readable with nothing else. +function currentIdentity(state) { + return proto.identityFromSeed(proto.identitySeed(unhex(state.master), state.rotations ?? 0)); +} + function identities(state) { - const current = proto.identityFromSeed(unhex(state.seed)); - const retired = (state.retired || []).map(s => proto.identityFromSeed(unhex(s))); - return [current, ...retired]; + const master = unhex(state.master); + const out = []; + for (let n = state.rotations ?? 0; n >= 0; n--) out.push(proto.identityFromSeed(proto.identitySeed(master, n))); + return out; +} + +function cursor(state) { + return [state.afterTime ?? 0, state.afterId ? unhex(state.afterId) : new Uint8Array(proto.ID_LEN)]; +} + +// §4/§5.8: the accept tokens to push with AUTH, and whether to push at all — +// a client that cannot vouch for its own set (freshly restored) must not +// replace the server's with an incomplete one. +function tokenSet(state) { + if (state.syncOk === false) return { sync: 0, tokens: [] }; + const accepted = Object.entries(state.accepted || {}).filter(([, a]) => a.active) + .sort(([a], [b]) => a.localeCompare(b)); + const master = unhex(state.master); + return { sync: 1, tokens: accepted.map(([, a]) => proto.tokenFor(master, b32decode(a.identity))) }; +} + +function addressOfSigner(state, sender, replyToField) { + const known = Object.entries(state.contacts || {}).find(([, c]) => c.key === b32encode(sender))?.[0]; + if (known) return known; + if (!replyToField) return null; + try { + const parsed = proto.parseAddress(replyToField); + if (parsed.identity && timingSafeEqual(parsed.identity, sender)) return parsed.short; + } catch { /* malformed claim */ } + return null; } // --- commands ------------------------------------------------------------------- @@ -94,9 +127,11 @@ function identities(state) { const commands = { keygen() { const state = load(); - state.seed = hex(cryptoRandom(32)); + state.master = hex(cryptoRandom(32)); + state.rotations = 0; + state.syncOk = true; save(state); - const me = proto.identityFromSeed(unhex(state.seed)); + const me = currentIdentity(state); console.log(`public key: ${b32encode(me.publicKey)}`); console.log(`fingerprint: ${fingerprint(me.publicKey)}`); }, @@ -112,11 +147,11 @@ const commands = { async register() { const state = load(); - const me = proto.identityFromSeed(unhex(state.seed)); + const me = currentIdentity(state); const addr = proto.parseAddress(rest[0]); const opened = await open(addr.host, addr.port, { requirePin: true }); try { - await proto.registerOp(opened.session, { + await proto.registerOp(opened.session, opened.serverStatic, { username: addr.user, identity: me, token: parseOption("--token") || "", }); } finally { @@ -147,7 +182,7 @@ const commands = { return; } if (timingSafeEqual(b32decode(known.key), current)) return console.log(`${addr.short}: unchanged`); - if (proto.walkChain(b32decode(known.key), current, chain)) { + if (proto.walkChain(addr.user, b32decode(known.key), current, chain)) { state.contacts[addr.short] = { key: b32encode(current), verified: known.verified }; save(state); console.log(`warning: ${addr.short} rotated its key; a signed chain confirms it`); @@ -170,13 +205,44 @@ const commands = { contacts() { const state = load(); - for (const [address, c] of Object.entries(state.contacts || {})) - console.log(`${address} ${c.key} ${c.verified ? "verified" : "tofu"}`); + for (const [address, c] of Object.entries(state.contacts || {})) { + const accepted = state.accepted?.[address]; + const tag = accepted ? (accepted.active ? "accepted" : "blocked") : ""; + console.log(`${address} ${c.key} ${c.verified ? "verified" : "tofu"} ${tag}`.trimEnd()); + } + }, + + // §5.8: admit a contact to the main tier; their token travels in our next + // message to them. Pushed to the server right away. + async accept() { + const state = load(); + const address = rest[0]; + const contact = state.contacts?.[address]; + if (!contact) throw new proto.SmolError(`no key for ${address} yet`); + state.accepted = { ...(state.accepted || {}), + [address]: { identity: state.accepted?.[address]?.identity ?? contact.key, active: true } }; + state.syncOk = true; + save(state); + const held = await pushTokens(state); + console.log(`accepted ${address}; server now holds ${held} accept token(s)`); + }, + + // §5.8: withdraw a contact's accept token; their mail lands in requests + // from their next message on. + async block() { + const state = load(); + const address = rest[0]; + if (!state.accepted?.[address]) throw new proto.SmolError(`${address} was never accepted`); + state.accepted[address] = { ...state.accepted[address], active: false }; + save(state); + const held = await pushTokens(state); + console.log(`blocked ${address}; server now holds ${held} accept token(s)`); }, async send() { const state = load(); - const me = proto.identityFromSeed(unhex(state.seed)); + const me = currentIdentity(state); + const master = unhex(state.master); const addr = proto.parseAddress(rest[0]); let recipient; if (addr.identity) { @@ -204,11 +270,17 @@ const commands = { if (state.account && !rest.includes("--anonymous")) fields.push(["Reply-To", proto.parseAddress(`${state.account.user}@${state.account.host}` + (state.account.port === proto.DEFAULT_PORT ? "" : `:${state.account.port}`)).uri(me.publicKey)]); + // §5.8: hand an accepted correspondent the token for our own mailbox. + const accepted = state.accepted?.[addr.short]; + if (accepted?.active) fields.push(["Accept", b32encode(proto.tokenFor(master, b32decode(accepted.identity)))]); const body = utf8Bytes(proto.buildFrontmatter(fields, (parseOption("--body") || "") + "\n")); const envelope = proto.seal(me, recipient, body); + // §5.8: our token for their mailbox, if they have given us one. + const held = state.tokens?.[addr.short]; + const mac = held ? proto.acceptMac(b32decode(held.token), proto.messageId(envelope)) : null; const opened = await open(addr.host, addr.port, { requirePin: false }); try { - await proto.sendOp(opened.session, envelope); + await proto.sendOp(opened.session, envelope, mac); } finally { opened.session.stream.close(); } @@ -217,29 +289,37 @@ const commands = { envelope: hex(proto.seal(me, me.publicKey, body)), sentAt: proto.nowSeconds(), }]; save(state); - console.log(`sent ${hex(proto.messageId(envelope)).slice(0, 8)} to ${addr.short}`); + console.log(`sent ${hex(proto.messageId(envelope)).slice(0, 16)} to ${addr.short}` + + (mac ? " (accepted)" : "")); }, async fetch() { const state = load(); - const me = proto.identityFromSeed(unhex(state.seed)); + const me = currentIdentity(state); const account = proto.parseAddress(`${state.account.user}@${state.account.host}` + (state.account.port === proto.DEFAULT_PORT ? "" : `:${state.account.port}`)); const ids = identities(state); + const keep = rest.includes("--keep"); + if (rest.includes("--reset")) { state.afterTime = 0; state.afterId = hex(new Uint8Array(proto.ID_LEN)); } + let [afterTime, afterId] = keep ? cursor(state) : [0, new Uint8Array(proto.ID_LEN)]; const opened = await open(account.host, account.port, { requirePin: true }); let total = 0, stored = 0, rejected = 0; try { - await proto.authenticate(opened.session, opened.handshakeHash, account.user, me); + const { sync, tokens } = tokenSet(state); + await proto.authenticate(opened.session, opened.handshakeHash, account.user, me, { sync, tokens }); for (;;) { - const records = await proto.fetchOp(opened.session); + const records = await proto.fetchOp(opened.session, afterTime, afterId); if (!records.length) break; const acked = []; for (const record of records) { total++; + afterTime = record.receivedAt; + afterId = record.id; + let unsealed; try { if (!timingSafeEqual(proto.messageId(record.envelope), record.id)) throw new proto.SmolError("id does not match the envelope"); - proto.unseal(ids, record.envelope); + unsealed = proto.unseal(ids, record.envelope); } catch (err) { rejected++; console.error(`warning: ${hex(record.id)}: ${err.message}; left on server`); @@ -249,25 +329,45 @@ const commands = { if (!state.inbox.some(m => m.id === hex(record.id))) { state.inbox.push({ id: hex(record.id), envelope: hex(record.envelope), receivedAt: record.receivedAt, + isRequest: record.isRequest, keptOnServer: keep, }); stored++; + const { fields } = proto.parseFrontmatter(new TextDecoder().decode(unsealed.body)); + const raw = fields.accept; + if (raw) { + try { + const token = b32decode(raw); + if (token.length === proto.TOKEN_LEN) { + const address = addressOfSigner(state, unsealed.sender, fields["reply-to"]); + if (address) state.tokens = { ...(state.tokens || {}), [address]: { token: b32encode(token) } }; + } + } catch { /* malformed token: ignore */ } + } } acked.push(record.id); } - if (!acked.length) break; - await proto.deleteOp(opened.session, acked); + if (keep) { + state.afterTime = afterTime; + state.afterId = hex(afterId); + } else if (acked.length) { + await proto.deleteOp(opened.session, acked); + } } } finally { opened.session.stream.close(); } + if (!keep) { state.afterTime = 0; state.afterId = hex(new Uint8Array(proto.ID_LEN)); } save(state); console.log(`${total} message(s): ${stored} new, ${rejected} rejected`); }, list() { const state = load(); - for (const m of state.inbox || []) describe(state, m, "inbox"); - for (const m of state.sent || []) describe(state, m, "sent"); + const wantRequests = rest.includes("--requests"); + for (const m of state.inbox || []) { + if (Boolean(m.isRequest) === wantRequests) describe(state, m, "inbox"); + } + if (rest.includes("--sent")) for (const m of state.sent || []) describe(state, m, "sent"); }, read() { @@ -281,19 +381,21 @@ const commands = { async rotate() { const state = load(); - const old = proto.identityFromSeed(unhex(state.seed)); + const old = currentIdentity(state); + const master = unhex(state.master); const account = proto.parseAddress(`${state.account.user}@${state.account.host}` + (state.account.port === proto.DEFAULT_PORT ? "" : `:${state.account.port}`)); - const fresh = proto.newIdentity(); - const cert = proto.makeCert(old, fresh.seed); + const nextIndex = (state.rotations ?? 0) + 1; + const freshSeed = proto.identitySeed(master, nextIndex); + const fresh = proto.identityFromSeed(freshSeed); + const cert = proto.makeCert(account.user, old, freshSeed); const opened = await open(account.host, account.port, { requirePin: true }); try { - await proto.registerOp(opened.session, { username: account.user, identity: fresh, cert }); + await proto.registerOp(opened.session, opened.serverStatic, { username: account.user, identity: fresh, cert }); } finally { opened.session.stream.close(); } - state.retired = [...(state.retired || []), state.seed]; - state.seed = hex(fresh.seed); + state.rotations = nextIndex; save(state); console.log(`rotated ${account.short}`); console.log(`new key: ${b32encode(fresh.publicKey)}`); @@ -301,8 +403,25 @@ const commands = { }, }; +async function pushTokens(state) { + if (!state.account) { + console.error("warning: not registered; the set will be pushed with your first fetch"); + return 0; + } + const me = currentIdentity(state); + const account = proto.parseAddress(`${state.account.user}@${state.account.host}` + + (state.account.port === proto.DEFAULT_PORT ? "" : `:${state.account.port}`)); + const opened = await open(account.host, account.port, { requirePin: true }); + try { + const { sync, tokens } = tokenSet(state); + return await proto.authenticate(opened.session, opened.handshakeHash, account.user, me, { sync, tokens }); + } finally { + opened.session.stream.close(); + } +} + function describe(state, row, box, verbose = false) { - const id = box === "inbox" ? row.id : row.id; + const id = row.id; try { const opened = proto.unseal(identities(state), unhex(row.envelope)); const { fields, body } = proto.parseFrontmatter(new TextDecoder().decode(opened.body)); @@ -311,12 +430,12 @@ function describe(state, row, box, verbose = false) { ?? `<${b32encode(opened.sender).slice(0, 20)}…>`; const stamp = new Date((opened.time) * 1000).toISOString().slice(0, 16).replace("T", " "); if (!verbose) return console.log( - `${id.slice(0, 8)} ${stamp} ${who.padEnd(28).slice(0, 28)} ${fields.Subject || "(no subject)"}`); + `${id.slice(0, 8)} ${stamp} ${who.padEnd(28).slice(0, 28)} ${fields.subject || "(no subject)"}`); console.log(`id: ${id}`); console.log(`${box === "sent" ? "to" : "from"}: ${who}`); console.log(`key: ${b32encode(opened.sender)}`); console.log(`date: ${new Date(opened.time * 1000).toISOString()}`); - for (const [k, v] of Object.entries(fields)) console.log(`${k.toLowerCase()}: ${v}`); + for (const [k, v] of Object.entries(fields)) if (k !== "accept") console.log(`${k}: ${v}`); console.log("signature verified\n"); console.log(body); } catch (err) { diff --git a/test/e2e.py b/test/e2e.py index cadd852..746197b 100644 --- a/test/e2e.py +++ b/test/e2e.py @@ -85,12 +85,18 @@ def main() -> int: run(bob + ["register", f"bob@{host}"]) check("bob registers on the reference server", True) - # --- reference client sends to the JS client + # bob needs alice as a known contact before he can accept her (§5.8). + run(bob + ["resolve", f"alice@{host}"]) + + # --- reference client sends to the JS client, unsolicited: bob has not + # accepted alice yet, so it lands in his requests tier, not the main one run(alice + ["send", f"bob@{host}", "--subject", "greetings", "--body", "hello bob"]) print(run(bob + ["fetch"]).strip()) - bob_list = run(bob + ["list"]) - check("bob lists alice's message", "greetings" in bob_list, bob_list) - bob_read = run(bob + ["read", re.match(r"([0-9a-f]{8})", bob_list).group(1)]) + bob_requests = run(bob + ["list", "--requests"]) + check("bob's requests tier holds alice's unsolicited message", "greetings" in bob_requests, + bob_requests) + check("bob's main tier is still empty", "greetings" not in run(bob + ["list"])) + bob_read = run(bob + ["read", re.match(r"([0-9a-f]{8})", bob_requests).group(1)]) alice_pub = re.search(r"public key: (\S+)", run(alice + ["whoami"])).group(1) check("bob reads the message", "hello bob" in bob_read, bob_read) check("bob sees alice's key", alice_pub in bob_read) @@ -99,23 +105,34 @@ def main() -> int: check("reference client emits the signed reply address", "reply-to: smol://alice@" in bob_read, bob_read) - # --- JS client replies, through the bridge's WebSocket relay + # --- bob accepts alice (§5.8): pushed to the server right away, so her + # future mail reaches his main tier + accepted = run(bob + ["accept", f"alice@{host}"]) + check("bob's server now holds an accept token", "server now holds 1 accept token" in accepted, + accepted) + + # --- JS client replies, through the bridge's WebSocket relay. Bob has + # accepted alice, so this carries the token issued to her — but it is + # still unsolicited from alice's side, landing in her requests tier. ws_env = dict(os.environ, GSMOL_TRANSPORT="ws", GSMOL_BRIDGE=f"ws://127.0.0.1:{bridge_port}") reply_id = re.search(r"sent ([0-9a-f]+)", run(bob + ["send", f"alice@{host}", "--subject", "re: greetings", "--body", "hello alice"], env=ws_env)).group(1) check("bob sends through the bridge", bool(reply_id)) run(alice + ["fetch"]) - alice_list = run(alice + ["list"]) - check("alice lists bob's reply", "re: greetings" in alice_list, alice_list) + alice_requests = run(alice + ["list", "--requests"]) + check("alice's requests tier holds bob's reply", "re: greetings" in alice_requests, + alice_requests) alice_read = run(alice + ["read", reply_id[:8]]) check("alice reads bob's reply", "hello alice" in alice_read, alice_read) check("reference client verifies the JS signature", "signature verified" in alice_read) # bob's reply carries his signed smol:// address; a receiver that does # not know the field must still preserve and display it (SPEC.md §5.5) check("reference client preserves the signed reply address", - "reply-to:smol://bob@" in alice_read, alice_read) + bool(re.search(r"reply-to:\s+smol://bob@", alice_read)), alice_read) - # --- rotation: the JS client rotates, the reference client walks the chain + # --- having now learned bob's accept token from that reply's Accept + # field, alice's next message to him carries a matching MAC and + # reaches his main tier even after he rotates his key bob_rotated = run(bob + ["rotate"]) new_key = re.search(r"new key: (\S+)", bob_rotated).group(1) resolved = run(alice + ["resolve", f"bob@{host}"]) @@ -124,7 +141,8 @@ def main() -> int: "--anonymous"]) print(run(bob + ["fetch"]).strip()) bob_list = run(bob + ["list"]) - check("bob reads mail addressed to the new key", "post-rotation" in bob_list) + check("bob's main tier reaches mail addressed to the new key", "post-rotation" in bob_list, + bob_list) # --anonymous omits the reply address; the field list must show none post_read = run(bob + ["read", re.search(r"([0-9a-f]{8}) .*post-rotation", bob_list).group(1)]) diff --git a/test/gen_vectors.py b/test/gen_vectors.py index f9867b8..7edac53 100644 --- a/test/gen_vectors.py +++ b/test/gen_vectors.py @@ -23,6 +23,7 @@ import nacl.bindings as sodium from noise.connection import Keypair, NoiseConnection LABEL_SEAL, LABEL_MSG, LABEL_ID = b"smolmail/1 seal", b"smolmail/1 msg", b"smolmail/1 id" +LABEL_IDENTITY, LABEL_ACCEPT, LABEL_MAC = b"smolmail/1 identity", b"smolmail/1 accept", b"smolmail/1 mac" TIME = 1730000000 # fixed so the envelope vector is reproducible @@ -139,9 +140,26 @@ def main() -> int: "sender_seed": sender_seed.hex(), "recipient_seed": recipient_seed.hex(), "esk": esk.hex(), "body": body.hex(), "time": TIME, "envelope": envelope.hex(), - "id": hashlib.sha256(LABEL_ID + envelope).digest()[:16].hex(), + "id": hashlib.sha256(LABEL_ID + envelope).digest().hex(), "unpadded_plaintext_len": len(plaintext)} + # --- §2/§5.8: master-derived rotation seeds and accept tokens + master = seed("master") + correspondent = seed("correspondent identity") + accept_key = hkdf_sha256(master, b"", LABEL_ACCEPT, 32) + token = hmac.new(accept_key, correspondent, hashlib.sha256).digest() + mid = seed("token message id") + out["tokens"] = { + "master": master.hex(), + "identity_seed_0": hkdf_sha256(master, b"", LABEL_IDENTITY + struct.pack(">I", 0), 32).hex(), + "identity_seed_1": hkdf_sha256(master, b"", LABEL_IDENTITY + struct.pack(">I", 1), 32).hex(), + "accept_key": accept_key.hex(), + "correspondent": correspondent.hex(), + "token": token.hex(), + "message_id": mid.hex(), + "accept_mac": hmac.new(token, LABEL_MAC + mid, hashlib.sha256).digest().hex(), + } + # --- Noise NX transcript with every key fixed i_e, r_e, s_priv = seed("nx initiator eph"), seed("nx responder eph"), seed("nx server static") s_pub = sodium.crypto_scalarmult_base(s_priv) diff --git a/test/vectors.json b/test/vectors.json index 61fb897..3bf13a2 100644 --- a/test/vectors.json +++ b/test/vectors.json @@ -190,9 +190,19 @@ "body": "2d2d2d0a5375626a6563743a20766563746f720a2d2d2d0a68656c6c6f20626f620a", "time": 1730000000, "envelope": "534d4f4c01e048814b56d9b82e54fd367d3c980661313cc6a3d81a80315561fc0ac87f81184e49921528d72321669ae1b275229800d8786c1ecdd7d9331bcb2cc64dc27c0399b106b5061e9105d8adf82f6b7464930fa31cb08ba85dba4764ce14cf3ddc32599f43fea73ced76ffa3a3b768e5a127034f5e82d667687369c19f060e8eafb09da0e212683290f4e1a73ce072b94f053c9088652109d3639f7b9aa0d48619626ecefe33855aade6ab8bf9de14ebb7cf07eec0ef9c193ae4537c370183e0c8f7cd7230471b9d8e7389ac654e1b09dc9b0160c875270fdad218f0c9da735bab", - "id": "b05d15a2ab5293164eda564de02a6901", + "id": "b05d15a2ab5293164eda564de02a69019aa97895ff988f3d9fa2be5126516553", "unpadded_plaintext_len": 143 }, + "tokens": { + "master": "fc613b4dfd6736a7bd268c8a0e74ed0d1c04a959f59dd74ef2874983fd443fc9", + "identity_seed_0": "35c2f058e7568f68e99e4fba7a730bd2d3cf7e1f6a2a266a2c5f5ca150d0074c", + "identity_seed_1": "fef4bd7f73b6459031a79dd589011a120c2cbc19b693b918f4aa9a8afe9dba7b", + "accept_key": "4fbaf4883885875100dc157864977e9a533866b15adc9daed35eedcc5f62fb20", + "correspondent": "fbaa80b2e87dc300cd33a2497dfe672c47ca8c8af099db8d45644054dff85297", + "token": "fe3085aeb2952b2bdd5abdcf8908ffd00054e871f47dfed2aa541cdb0e2006d6", + "message_id": "7c65ce5d23376fb508b4dda83a271095fd41e0ce47604213153520b0a0db50ac", + "accept_mac": "1d56673df4952dedcc1e40dfce2ad7820864090809a381e225e0f2fafa7cbfc7" + }, "noise": { "initiator_eph_priv": "af5f15993914cfd4e308856810d483ab25a383bfb2d708a0e15f80275f1fe6c2", "responder_eph_priv": "1c21927bb3e579efb69c937a2bf2e299ca1da9c83a62fb7f8ea1a375eb6f1c80", diff --git a/test/vectors.test.mjs b/test/vectors.test.mjs index c969c42..13130bd 100644 --- a/test/vectors.test.mjs +++ b/test/vectors.test.mjs @@ -71,6 +71,18 @@ for (const v of vectors.ed_to_x25519) { eq(`x_pub[${v.name}]`, hex(crypto.ed25519ToX25519(crypto.ed25519PublicKey(unhex(v.seed)))), v.x_pub); } +// --- §2/§5.8: master-derived rotation seeds and accept tokens ----------------- + +{ + const t = vectors.tokens; + const master = unhex(t.master); + eq("identity-seed-0", hex(proto.identitySeed(master, 0)), t.identity_seed_0); + eq("identity-seed-1", hex(proto.identitySeed(master, 1)), t.identity_seed_1); + eq("accept-key", hex(proto.acceptKeyFor(master)), t.accept_key); + eq("token-for", hex(proto.tokenFor(master, unhex(t.correspondent))), t.token); + eq("accept-mac", hex(proto.acceptMac(unhex(t.token), unhex(t.message_id))), t.accept_mac); +} + // --- §5 envelope -------------------------------------------------------------- { @@ -125,19 +137,22 @@ for (const v of vectors.ed_to_x25519) { // --- frontmatter (§5.5) ------------------------------------------------------- { - const spec = "---\nSubject: Re: the thing\nIn-Reply-To: 4f2a1c9e8b7d6a5f3e2d1c0b9a8f7e6d\nX-Mood: cautiously optimistic\n---\nBody text starts here."; + const rid = "4f2a1c9e8b7d6a5f3e2d1c0b9a8f7e6d5c4b3a291807f6e5d4c3b2a1908f7e6d5"; // §5.4: 64 hex chars + const spec = `---\nSubject: Re: the thing\nIn-Reply-To: ${rid}\nX-Mood: cautiously optimistic\n---\nBody text starts here.`; const { fields, body } = proto.parseFrontmatter(spec); - eq("fm-subject", fields.Subject, "Re: the thing"); - eq("fm-reply", fields["In-Reply-To"], "4f2a1c9e8b7d6a5f3e2d1c0b9a8f7e6d"); + // §5.5: keys are compared case-insensitively, so they come back lowercased. + eq("fm-subject", fields.subject, "Re: the thing"); + eq("fm-reply", fields["in-reply-to"], rid); + eq("fm-case-insensitive", proto.parseFrontmatter("---\nSUBJECT: hi\n---\nx").fields.subject, "hi"); eq("fm-body", body, "Body text starts here."); // a malformed line invalidates the whole block, which fails closed toward display eq("fm-malformed", proto.parseFrontmatter("---\nno colon here\n---\nrest").body, "---\nno colon here\n---\nrest"); eq("fm-unterminated", proto.parseFrontmatter("---\nSubject: x\nno end").body, "---\nSubject: x\nno end"); - eq("fm-first-wins", proto.parseFrontmatter("---\nA: 1\nA: 2\n---\ntext").fields.A, "1"); + eq("fm-first-wins", proto.parseFrontmatter("---\nA: 1\nA: 2\n---\ntext").fields.a, "1"); eq("fm-escape", proto.buildFrontmatter([], "---\nactual body"), "---\n---\n---\nactual body"); eq("fm-build", proto.buildFrontmatter([["Subject", "hi"]], "there"), "---\nSubject: hi\n---\nthere"); eq("fm-no-block", proto.buildFrontmatter([], "plain"), "plain"); - eq("fm-too-many-keys", proto.parseFrontmatter("---\n" + "X: y\n".repeat(65) + "---\nbody").fields.Subject ?? "none", "none"); + eq("fm-too-many-keys", proto.parseFrontmatter("---\n" + "X: y\n".repeat(65) + "---\nbody").fields.subject ?? "none", "none"); } // --- addresses (§3) ----------------------------------------------------------- @@ -167,21 +182,23 @@ for (const v of vectors.ed_to_x25519) { // --- rotation chains (§7) ------------------------------------------------------ { + const user = "alice"; const old = proto.identityFromSeed(crypto.randomBytes(32)); const mid = crypto.randomBytes(32); - const midPub = crypto.ed25519PublicKey(mid); const fresh = crypto.randomBytes(32); const freshPub = crypto.ed25519PublicKey(fresh); const when = proto.nowSeconds(); - const chain = [proto.makeCert(old, mid, when), proto.makeCert(proto.identityFromSeed(mid), fresh, when)]; - ok("chain-valid", proto.walkChain(old.publicKey, freshPub, chain)); - ok("chain-unchanged", proto.walkChain(old.publicKey, old.publicKey, [])); - ok("chain-missing-link", !proto.walkChain(old.publicKey, freshPub, chain.slice(1))); + const chain = [proto.makeCert(user, old, mid, when), + proto.makeCert(user, proto.identityFromSeed(mid), fresh, when)]; + ok("chain-valid", proto.walkChain(user, old.publicKey, freshPub, chain)); + ok("chain-unchanged", proto.walkChain(user, old.publicKey, old.publicKey, [])); + ok("chain-missing-link", !proto.walkChain(user, old.publicKey, freshPub, chain.slice(1))); + ok("chain-wrong-username", !proto.walkChain("bob", old.publicKey, freshPub, chain)); const forged = chain.slice(); - forged[1] = proto.makeCert(proto.identityFromSeed(mid), crypto.randomBytes(32), when); - ok("chain-broken", !proto.walkChain(old.publicKey, freshPub, forged)); - ok("chain-oversize", !proto.walkChain(old.publicKey, freshPub, Array(17).fill(chain[0]))); - eq("cert-len", String(chain[0].length), "136"); + forged[1] = proto.makeCert(user, proto.identityFromSeed(mid), crypto.randomBytes(32), when); + ok("chain-broken", !proto.walkChain(user, old.publicKey, freshPub, forged)); + ok("chain-oversize", !proto.walkChain(user, old.publicKey, freshPub, Array(17).fill(chain[0]))); + eq("cert-len", String(chain[0].length), "200"); } // --- response framing guards (§6.1) ------------------------------------------- diff --git a/web/index.html b/web/index.html index 4e06cc6..cb44152 100644 --- a/web/index.html +++ b/web/index.html @@ -36,13 +36,13 @@

welcome

-

Identity is one 32-byte seed. Back it up: it is the only secret, and losing it means losing the address.

+

Identity is one 32-byte master secret. Back it up: it is the only secret, and losing it means losing the address.

-

or restore from a saved seed:

+

or restore from a saved master:

- +
@@ -100,7 +104,7 @@

1 your server

Pin the key of the server that will hold your mail. Get it from its operator over a - channel you already trust — registering, recalling and fetching all refuse an unpinned server (SPEC.md §4).

+ channel you already trust — registering, recalling and fetching all refuse an unpinned server.

@@ -119,7 +123,7 @@
already registered elsewhere? -

Restoring a seed brings back the identity, not the memory of what address it was +

Restoring a master brings back the identity, not the memory of what address it was registered under in a different browser — the server was never asked. Recall it here: this looks the address up and checks it resolves to this identity's key, without registering anything.

@@ -134,11 +138,23 @@
+
+

fetch

+ +
+

backup

-

Mail, once fetched, exists only here — the server deletes it as it's picked up - (SPEC.md §5). This backs up the local inbox, sent mail, contacts and pinned servers into one - file. It never includes the seed: that has its own reveal-and-copy flow above, meant for a +

Mail, once fetched, exists only here unless "leave mail on server" is on above. + This backs up the local inbox, requests, sent mail, contacts and pinned servers into one file. + It never includes the master: that has its own reveal-and-copy flow above, meant for a password manager, not a downloadable file.

diff --git a/web/js/app.js b/web/js/app.js index 8f0d787..965d65e 100644 --- a/web/js/app.js +++ b/web/js/app.js @@ -1,11 +1,12 @@ // UI wiring for the smol mail client: identity setup, folders, reading, // composing, fetching with verify-before-ack, and the §4/§7 trust surfaces. -import { hex, timingSafeEqual, utf8Bytes } from "./crypto.js"; +import { hex, timingSafeEqual, unhex, utf8Bytes } from "./crypto.js"; import { - DEFAULT_PORT, SmolError, authenticate, b32decode, b32encode, buildFrontmatter, deleteOp, - fetchOp, fingerprint, makeCert, messageId, newIdentity, openSession, parseAddress, - parseFrontmatter, registerOp, resolveOp, seal, sendOp, unseal, walkChain, + DEFAULT_PORT, ID_LEN, MAX_CHAIN, TOKEN_LEN, SmolError, acceptMac, authenticate, b32decode, + b32encode, buildFrontmatter, deleteOp, fetchOp, fingerprint, identitySeed, identityFromSeed, + makeCert, messageId, newMaster, openSession, parseAddress, parseFrontmatter, registerOp, + resolveOp, seal, sendOp, tokenFor, unseal, walkChain, } from "./proto.js"; import * as store from "./store.js"; import { connectStream } from "./transport.js"; @@ -37,11 +38,25 @@ function icon(name) { const placeholder = (text, warn = false) => el("p", { className: warn ? "placeholder warn" : "placeholder", textContent: text }); +// A native shown with showModal() renders in the browser's top +// layer, above the rest of the page — a toast anchored to sits +// visually behind an open dialog's backdrop no matter its z-index. Keeping +// the toast inside whichever dialog is open puts it in that same top-layer +// subtree, so a status raised while settings or compose is open (e.g. after +// "register") actually gets seen instead of landing, barely visible, behind it. +function reparentStatus() { + const toast = $("status"); + const target = document.querySelector("dialog[open]") ?? document.body; + if (toast.parentElement !== target) target.append(toast); +} +for (const dialog of document.querySelectorAll("dialog")) dialog.addEventListener("close", reparentStatus); + // The status line is a toast: click to dismiss, empty means hidden. Plain // notices fade; warnings stay until dismissed, since those carry the §4/§8 // trust messages the user must actually read. let toastTimer = null; function status(text, warn = false) { + reparentStatus(); const toast = $("status"); toast.textContent = text; toast.hidden = !text; @@ -125,7 +140,7 @@ async function connect(addr, { requirePin }) { const opened = await openSession(stream, addr.host, pinned); if (!pinned) status(`${addr.host} is not pinned; its key is ${b32encode(opened.serverStatic)}.\n` + - `RESOLVE results from this session are UNVERIFIED (SPEC.md §8).`, true); + `Lookups from this session are UNVERIFIED.`, true); return opened; } @@ -151,18 +166,22 @@ function showApp() { } $("create-identity").onclick = () => { - store.setIdentity(newIdentity().seed); + store.setIdentity(newMaster()); showApp(); - status("identity created. Back up the seed in settings now; it is the only secret."); + status("identity created. Back up the master in settings now; it is the only secret."); openSettings(); }; $("restore-identity").onclick = async () => { - const text = $("restore-seed").value.trim(); - if (!/^[0-9a-fA-F]{64}$/.test(text)) return status("a seed is 64 hex characters", true); - const seedBytes = Uint8Array.from(text.match(/../g), b => parseInt(b, 16)); + const text = $("restore-master").value.trim(); + if (!/^[0-9a-fA-F]{64}$/.test(text)) return status("a master is 64 hex characters", true); + const masterBytes = Uint8Array.from(text.match(/../g), b => parseInt(b, 16)); try { - store.setIdentity(seedBytes); + // §2: a master alone does not say which rotation index a server has + // bound; assume 0 (a never-rotated identity, the common case) until an + // address lets us confirm it below. Either way the accepted-correspondent + // set is unknown, so it must not overwrite the server's (syncOk: false). + store.restoreMaster(masterBytes, 0); } catch (err) { return status(err.message, true); } @@ -198,7 +217,7 @@ $("restore-identity").onclick = async () => { if (addressText) { try { - const addr = await recallAccount(addressText); + const addr = await recallAndDiscoverRotation(masterBytes, addressText); lines.push(`recalled ${addr.short}.`); renderAccount(); renderSettings(); @@ -260,6 +279,7 @@ function resetView() { // leaving the inbox used to blank the badge. async function updateUnreadBadge() { $("count-inbox").textContent = (await store.unreadCount()) || ""; + $("count-requests").textContent = (await store.requestsUnreadCount()) || ""; } async function renderFolder() { @@ -278,7 +298,8 @@ async function renderFolder() { if (token !== renderToken) return; list.replaceChildren(); if (!rows.length) { - list.append(placeholder(folder === "inbox" ? "no mail — fetch" : "nothing sent")); + list.append(placeholder(folder === "inbox" ? "no mail — fetch" + : folder === "requests" ? "no unsolicited mail" : "nothing sent")); return void updateListStatus(); } for (let offset = 0; offset < rows.length; offset += BATCH) { @@ -292,8 +313,9 @@ async function renderFolder() { // A real button, so every row is reachable and openable from the keyboard. function messageRow(row) { const opened = describe(row); - // Only the inbox has a read state; sent copies were never "unread". - const unread = folder === "inbox" && !store.isRead(row.id); + // Both mail tiers have a read state; sent copies were never "unread". + const inbound = folder === "inbox" || folder === "requests"; + const unread = inbound && !store.isRead(row.id); const item = el("button", { type: "button", className: `item${row.id === selectedId ? " selected" : ""}${unread ? " unread" : ""}`, @@ -302,7 +324,7 @@ function messageRow(row) { if (row.id === selectedId) item.setAttribute("aria-current", "true"); const who = el("div", { className: "who", textContent: folder === "sent" ? row.recipient : (opened.fromShort ?? "?") }); - if (folder === "inbox" && !opened.error && !opened.knownSender) + if (inbound && !opened.error && !opened.knownSender) who.append(el("span", { className: "dot", title: "sender key not bound to an address" })); item.append(who, el("div", { className: "mid" }, @@ -356,7 +378,7 @@ function openEnvelope(row) { const opened = unseal(store.identities(), row.envelope); const { fields, body } = parseFrontmatter(new TextDecoder().decode(opened.body)); entry = { ...opened, fields, body, - subject: fields.Subject || "", + subject: fields.subject || "", snippet: body.replace(/\s+/g, " ").trim().slice(0, 140) }; } catch (err) { entry = { error: err.message }; @@ -376,9 +398,9 @@ function describe(row) { if (opened.error) return opened; const known = store.addressForKey(opened.sender); let replyAddress = null; - if (opened.fields["Reply-To"]) { + if (opened.fields["reply-to"]) { try { - const claimed = parseAddress(opened.fields["Reply-To"]); + const claimed = parseAddress(opened.fields["reply-to"]); // Only a full smol:// address whose key matches the signer counts; // anything else is ordinary text. if (claimed.identity && timingSafeEqual(claimed.identity, opened.sender)) @@ -419,7 +441,8 @@ async function openMessage(id) { onclick: () => { $("app").classList.remove("reading"); } }, icon("back"), "messages"); if (opened.error) return void view.replaceChildren(back, placeholder(opened.error, true)); - const fields = Object.entries(opened.fields).filter(([k]) => k !== "Subject"); + // "accept" and "in-reply-to" are protocol machinery (§5.5, §5.8), not content. + const fields = Object.entries(opened.fields).filter(([k]) => k !== "subject" && k !== "accept"); const actions = el("div", { className: "actions" }); view.replaceChildren(...[ back, @@ -433,7 +456,32 @@ async function openMessage(id) { el("pre", { textContent: opened.body }), actions, ].filter(node => node != null)); - if (folder !== "inbox") return; + + // Deleting removes the local copy, and the server's too if "leave mail on + // server" left one there to remove (settings; SPEC.md §10). + actions.append(el("button", { type: "button", onclick: async () => { + try { + await deleteMessage(folder, row); + status("message deleted"); + selectedId = null; + resetView(); + renderFolder(); + } catch (err) { + status(err.message, true); // the local copy is kept if a needed server delete failed + } + } }, "delete")); + + if (folder !== "inbox" && folder !== "requests") return; + + if (folder === "requests" && opened.knownSender) { + const known = store.addressForKey(opened.sender); + if (!store.accepted(known)?.active) { + actions.before(placeholder( + `unsolicited mail from ${known}, not yet accepted into your main tier`)); + actions.append(el("button", { type: "button", onclick: () => acceptContact(known) }, + "accept — future mail lands in your main tier")); + } + } if (opened.knownSender) { actions.append(el("button", { type: "button", onclick: () => startReply(row) }, @@ -479,6 +527,28 @@ async function openMessage(id) { } } +// Deletes a message locally, and from the server too if it might still be +// sitting there (only possible when "leave mail on server" was on when it was +// fetched — SPEC.md §10). Sent copies are local-only; there is nothing +// server-side to remove for them (§5.6). Throws, leaving the local copy in +// place, if a needed server-side delete fails — otherwise a message could +// look gone locally while silently persisting on the server. +async function deleteMessage(folder, row) { + if (folder !== "sent" && row.keptOnServer) { + const me = store.identity(); + const addr = accountAddress(); + if (!me || !addr) throw new SmolError("not registered; cannot reach the server to delete this message"); + const opened = await connect(addr, { requirePin: true }); + try { + await authenticate(opened.session, opened.handshakeHash, addr.user, me); + await deleteOp(opened.session, [unhex(row.id)]); + } finally { + opened.session.stream.close(); + } + } + await store.removeMessage(folder, row.id); +} + // §4/§8: a signature always verifies here, because unseal() throws otherwise. // What can be weak is the binding between that key and an address, so that is // what the chip reports. @@ -585,7 +655,8 @@ compose.querySelector("form").onsubmit = async (event) => { async function sendMail(toText, subject, body, replyTo, anonymous) { const me = store.identity(); - if (!me) throw new SmolError("no identity yet"); + const master = store.master(); + if (!me || !master) throw new SmolError("no identity yet"); const addr = parseAddress(toText); const recipient = await resolveRecipient(addr); const account = accountAddress(); @@ -593,12 +664,19 @@ async function sendMail(toText, subject, body, replyTo, anonymous) { // A signed Reply-To lets a first-time recipient name and answer us (§5.5 // allows unknown keys); "anonymous" omits it. if (account && !anonymous) fields.push(["Reply-To", account.uri(me.publicKey)]); + // §5.8: hand an accepted correspondent the token for our own mailbox, so a + // first reply from them reaches our main tier. + const accepted = store.accepted(addr.short); + if (accepted?.active) fields.push(["Accept", b32encode(tokenFor(master, accepted.identity))]); const bodyBytes = utf8Bytes(buildFrontmatter(fields.filter(([, v]) => v), body.replace(/\s+$/, "") + "\n")); const envelope = seal(me, recipient, bodyBytes); + // §5.8: our token for their mailbox, if they have given us one. + const held = store.tokenFrom(addr.short); + const mac = held ? acceptMac(held, messageId(envelope)) : null; const opened = await connect(addr, { requirePin: false }); try { - await sendOp(opened.session, envelope); + await sendOp(opened.session, envelope, mac); } finally { opened.session.stream.close(); } @@ -609,7 +687,7 @@ async function sendMail(toText, subject, body, replyTo, anonymous) { envelope: seal(me, me.publicKey, bodyBytes), sentAt: Math.floor(Date.now() / 1000), }); - status(`sent to ${addr.short}`); + status(`sent to ${addr.short}${mac ? " (accepted)" : ""}`); if (folder === "sent") renderFolder(); } @@ -635,10 +713,41 @@ async function resolveRecipient(addr) { // --- fetch ---------------------------------------------------------------------- +// The address we know a signer by: a contact, or the Reply-To it signed for +// itself. Naming a mailbox is not trusting a key, so nothing is pinned here +// (§5.7, §8). +function addressOfSigner(sender, replyToField) { + const known = store.addressForKey(sender); + if (known) return known; + if (!replyToField) return null; + try { + const parsed = parseAddress(replyToField); + if (parsed.identity && timingSafeEqual(parsed.identity, sender)) return parsed.short; + } catch { /* malformed claim: no address to learn a token under */ } + return null; +} + +// §5.8: an Accept field is bound to the signer of the message that carried +// it, which unseal() has already verified. +function learnTokenFrom(unsealed, opened) { + const raw = opened.fields["accept"]; + if (!raw) return; + let token; + try { + token = b32decode(raw); + } catch { + return; + } + if (token.length !== TOKEN_LEN) return; + const address = addressOfSigner(unsealed.sender, opened.fields["reply-to"]); + if (address) store.learnToken(address, token); +} + $("fetch").onclick = async () => { const me = store.identity(); + const master = store.master(); const addr = accountAddress(); - if (!me) return status("no identity yet", true); + if (!me || !master) return status("no identity yet", true); if (!addr) return status("not registered; register in settings first", true); const button = $("fetch"), label = button.querySelector(".label"); const resting = label.textContent; @@ -647,15 +756,24 @@ $("fetch").onclick = async () => { status("fetching…"); let stored = 0, verified = 0; const rejected = []; + // §10: acknowledging (deleting) is the default; "leave mail on server" pages + // forward by cursor instead, so already-fetched mail is never re-downloaded + // even though it isn't deleted (storeIfNew also dedupes, as a second line + // of defense). + const leaveOnServer = store.leaveOnServer(); + let [afterTime, afterId] = store.cursor(); try { const opened = await connect(addr, { requirePin: true }); try { - await authenticate(opened.session, opened.handshakeHash, addr.user, me); + const { sync, tokens } = store.tokenSet(master); + await authenticate(opened.session, opened.handshakeHash, addr.user, me, { sync, tokens }); for (;;) { - const records = await fetchOp(opened.session); + const records = await fetchOp(opened.session, afterTime, afterId); if (!records.length) break; const acked = []; for (const record of records) { + afterTime = record.receivedAt; + afterId = record.id; let unsealed; try { if (!timingSafeEqual(messageId(record.envelope), record.id)) @@ -668,29 +786,39 @@ $("fetch").onclick = async () => { rejected.push(`${hex(record.id)}: ${err.message}`); continue; } - const row = { id: hex(record.id), envelope: record.envelope, receivedAt: record.receivedAt }; + const row = { id: hex(record.id), envelope: record.envelope, receivedAt: record.receivedAt, + tier: record.isRequest ? store.TIER_REQUESTS : store.TIER_MAIN, keptOnServer: leaveOnServer }; if (await store.storeIfNew("inbox", row)) { stored++; // A signed Reply-To is exactly as verified as the signature // unseal() just checked — see bindReplyAddress(). describe() // both parses the same claim the reader would show and warms its // cache, so opening the message right after costs nothing extra. - const claim = describe(row).replyAddress; - if (claim) { + const described = describe(row); + if (described.replyAddress) { try { - bindReplyAddress(claim, unsealed.sender); + bindReplyAddress(described.replyAddress, unsealed.sender); verified++; } catch { /* already known under a different key: leave it be */ } } + learnTokenFrom(unsealed, described); } acked.push(record.id); } - if (!acked.length) break; - await deleteOp(opened.session, acked); + if (leaveOnServer) { + // Persisted per batch, so an interrupted fetch resumes here rather + // than re-paging from the start next time. + store.setCursor(afterTime, afterId); + } else if (acked.length) { + await deleteOp(opened.session, acked); + } } } finally { opened.session.stream.close(); } + // Everything acknowledged is deleted, so the next fetch starts fresh; a + // record left on the server (rejected above) simply resurfaces then. + if (!leaveOnServer) store.setCursor(0, new Uint8Array(ID_LEN)); status([`${stored} new, ${rejected.length} rejected`, verified ? `${verified} sender${verified === 1 ? "" : "s"} verified from a signed Reply-To` : null, ...rejected.map(line => ` ${line}`), @@ -710,6 +838,7 @@ $("fetch").onclick = async () => { function renderContacts(list) { const contacts = store.allContacts(); list.replaceChildren(...contacts.map(({ address, key, verified, history }) => { + const accepted = store.accepted(address); const item = el("button", { type: "button", className: `item contact${address === selectedId ? " selected" : ""}`, @@ -722,6 +851,7 @@ function renderContacts(list) { el("div", { textContent: fingerprint(b32decode(key)) }), el("span", { className: verified ? "badge ok" : "badge", textContent: verified ? "verified key" : "tofu" }), + accepted?.active ? el("span", { className: "badge ok", textContent: "accepted" }) : null, history.length ? el("span", { className: "badge", textContent: `${history.length} previous` }) : null)); item.dataset.search = @@ -753,6 +883,18 @@ function openContact(address) { } } }, icon("refresh"), "re-resolve"); + const isAccepted = Boolean(store.accepted(address)?.active); + const tokenButton = el("button", { type: "button", + textContent: isAccepted ? "block" : "accept", + onclick: async (event) => { + event.target.disabled = true; + try { + if (isAccepted) await blockContact(address); else await acceptContact(address); + } finally { + event.target.disabled = false; + } + } }); + view.replaceChildren(...[ el("button", { type: "button", className: "back", onclick: () => { $("app").classList.remove("reading"); } }, icon("back"), "contacts"), @@ -763,10 +905,16 @@ function openContact(address) { field("fingerprint", fingerprint(known.key)), field("public key", b32encode(known.key)), field("address", addr.uri(known.key), false, copyButton(addr.uri(known.key), "the address")), + el("div", { className: "seed-card" }, + el("h4", { textContent: "accept token" }), + el("p", { className: "hint", textContent: isAccepted + ? "This contact's mail lands in your main tier." + : "This contact's mail lands in requests until accepted." }), + el("div", { className: "actions" }, tokenButton)), known.history.length ? el("section", { className: "history" }, el("h3", { textContent: `previous keys (${known.history.length})` }), el("p", { className: "hint", textContent: - "Replaced by a signed rotation chain (SPEC.md §7). Mail signed with these was sent " + + "Replaced by a signed rotation chain. Mail signed with these was sent " + "before the change; rotation is not revocation, so a stolen key can still rotate onward." }), ...known.history.map(({ key, until }) => el("div", { className: "keyblock" }, el("div", { textContent: fingerprint(b32decode(key)) }), @@ -796,7 +944,7 @@ async function refreshContact(address) { status(`${addr.short} pinned (trust on first use${opened.pinned ? "" : ", UNVERIFIED server"})`); } else if (timingSafeEqual(known.key, current)) { status(`${addr.short}: key unchanged`); - } else if (walkChain(known.key, current, chain)) { + } else if (walkChain(addr.user, known.key, current, chain)) { store.saveContact(addr.short, current, known.verified); status(`${addr.short} rotated its key; a signed chain confirms it.\nnow ${b32encode(current)}`, true); } else { @@ -810,6 +958,59 @@ async function refreshContact(address) { else renderFolder(); } +// --- accept tokens (§5.8) -------------------------------------------------------- + +// An accept or a block only takes effect once the server holds the changed +// set, so it is pushed right away rather than at the next fetch. +async function pushTokens() { + const me = store.identity(); + const master = store.master(); + const addr = accountAddress(); + if (!me || !master) throw new SmolError("no identity yet"); + if (!addr) { + status("not registered; the set will be pushed with your first fetch", true); + return 0; + } + const opened = await connect(addr, { requirePin: true }); + try { + const { sync, tokens } = store.tokenSet(master); + return await authenticate(opened.session, opened.handshakeHash, addr.user, me, { sync, tokens }); + } finally { + opened.session.stream.close(); + } +} + +// Admit a contact to the main tier; their token travels in our next message +// to them. +async function acceptContact(address) { + try { + const key = store.contact(address)?.key; + if (!key) throw new SmolError(`no key for ${address} yet`); + store.accept(address, key); + store.setSyncOk(true); + const held = await pushTokens(); + status(`${address} accepted; server now holds ${held} accept token(s)`); + renderFolder(); + if (folder === "contacts" && selectedId === address) openContact(address); + } catch (err) { + status(err.message, true); + } +} + +// Withdraw a contact's accept token; their mail lands in requests from their +// next message on. +async function blockContact(address) { + try { + store.block(address); + const held = await pushTokens(); + status(`${address} blocked; server now holds ${held} accept token(s)`); + renderFolder(); + if (folder === "contacts" && selectedId === address) openContact(address); + } catch (err) { + status(err.message, true); + } +} + // --- settings --------------------------------------------------------------------- const settings = $("settings"); @@ -822,6 +1023,8 @@ function openSettings() { $("settings-open").onclick = openSettings; $("settings-close").onclick = () => settings.close(); +$("leave-on-server").onchange = (event) => store.setLeaveOnServer(event.target.checked); + // Registering against an unpinned server is refused, so the dialog is ordered // by what depends on what; the chips say how far along that order you are. function step(id, done) { @@ -835,6 +1038,8 @@ function renderSettings() { const account = store.account(); const pins = store.allPins(); + $("leave-on-server").checked = store.leaveOnServer(); + step("step-server", pins.length > 0); step("step-address", Boolean(account)); @@ -850,16 +1055,18 @@ function renderSettings() { const card = $("identity-card"); card.replaceChildren(); if (!me) return; + const masterBytes = store.master(); const share = account ? accountAddress().uri(me.publicKey) : null; - const retired = store.identities().length - 1; + const rotations = store.rotations(); + const acceptedCount = store.allAccepted().filter(a => a.active).length; // Settings opens by itself right after an identity is created, which is a poor // moment to paint the only secret across the screen — so it is asked for. const masked = "\u2022".repeat(64); - const seed = el("div", { className: "seed", textContent: masked }); + const master = el("div", { className: "seed", textContent: masked }); const reveal = el("button", { type: "button", textContent: "reveal", onclick: () => { - const showing = seed.textContent !== masked; - seed.textContent = showing ? masked : hex(me.seed); + const showing = master.textContent !== masked; + master.textContent = showing ? masked : hex(masterBytes); reveal.textContent = showing ? "reveal" : "hide"; } }); @@ -869,27 +1076,30 @@ function renderSettings() { field("shareable", share ?? "register above to get one", !share, share ? copyButton(share, "your address") : null), el("div", { className: "seed-card" }, - el("h4", { textContent: "recovery seed" }), - el("div", { className: "row" }, seed, reveal, copyButton(hex(me.seed), "the seed")), + el("h4", { textContent: "recovery master" }), + el("div", { className: "row" }, master, reveal, copyButton(hex(masterBytes), "the master")), el("p", { textContent: "The only secret there is. Anyone who holds it can read your mail and " + "write as you, and losing it loses the address with it — there is no reset. Copy it " + "somewhere safe now." })), - field("retired keys", String(retired)), - retired ? el("p", { className: "hint", - textContent: "Kept so mail sealed to a superseded key stays readable." }) : null, + field("rotations", String(rotations)), + rotations ? el("p", { className: "hint", + textContent: "Earlier keys are re-derived on demand from the master, so mail sealed to a " + + "superseded key stays readable." }) : null, + field("accepted correspondents", String(acceptedCount) + + (store.syncOk() ? "" : " (not yet pushed to the server — accept them again)")), el("details", {}, el("summary", { textContent: "advanced" }), el("p", { className: "hint", textContent: - "Rotating issues a fresh key with a signed certificate proving it replaces this one. " + - "Contacts accept the change from that chain, and the old seed is kept, since mail sealed " + - "to it is readable with nothing else. Rotation is not revocation (SPEC.md \u00a77)." }), + "Rotating issues a fresh key — the next one derivable from the master — with a signed " + + "certificate proving it replaces this one. Contacts accept the change from that chain, and " + + "the old key stays readable with nothing else. Rotation is not revocation." }), el("button", { type: "button", textContent: "rotate identity", onclick: rotateIdentity }), el("p", { className: "hint", textContent: - "Log out to remove this identity from this browser entirely: the seed, pins, contacts and " + - "every cached message. Without a backup of the seed there is no way back in — same as any " + - "other loss of it. Fetching deletes from the server once acknowledged (SPEC.md §5), so a " + - "message already fetched here has no other copy anywhere; anything still unfetched stays " + - "on the server and comes back on the next fetch after logging back in." }), + "Log out to remove this identity from this browser entirely: the master, pins, contacts and " + + "every cached message. Without a backup of the master there is no way back in — same as any " + + "other loss of it. Fetching deletes from the server once acknowledged unless \u201cleave mail " + + "on server\u201d is on, so a message already fetched here may have no other " + + "copy anywhere; anything still on the server comes back on the next fetch after logging back in." }), el("button", { type: "button", className: "danger", textContent: "log out & erase this browser", onclick: logout })), ].filter(node => node != null)); @@ -948,13 +1158,18 @@ $("import-form").onsubmit = (event) => { $("register-form").onsubmit = async (event) => { event.preventDefault(); const form = event.target; + const submit = form.querySelector("button"); + const resting = submit.textContent; + submit.disabled = true; // a round trip to the server takes a moment; say so rather than sit inert + submit.textContent = "registering…"; try { const me = store.identity(); if (!me) throw new SmolError("no identity yet"); const addr = parseAddress(form.address.value.trim()); const opened = await connect(addr, { requirePin: true }); try { - await registerOp(opened.session, { username: addr.user, identity: me, token: form.token.value.trim() }); + await registerOp(opened.session, opened.serverStatic, + { username: addr.user, identity: me, token: form.token.value.trim() }); } finally { opened.session.stream.close(); } @@ -965,10 +1180,13 @@ $("register-form").onsubmit = async (event) => { form.reset(); } catch (err) { status(err.message, true); + } finally { + submit.disabled = false; + submit.textContent = resting; } }; -// Restoring a seed brings back the identity, not the memory of what address a +// Restoring a master brings back the identity, not the memory of what address a // *different* browser registered it under — "account" is local-only state, // never asked of the server. This binds it without REGISTER: RESOLVE the // address and require it name this exact key, so a typo or someone else's @@ -991,9 +1209,41 @@ async function recallAccount(text) { return addr; } +// §2: a master alone does not say which rotation index a server has bound, +// so restoring resolves the address and walks indices 0..MAX_CHAIN until one +// derives the key RESOLVE returned, correcting restore-identity's default +// guess of index 0. Also binds "account", like recallAccount(). +async function recallAndDiscoverRotation(masterBytes, text) { + const addr = parseAddress(text.trim()); + const opened = await connect(addr, { requirePin: true }); + let current; + try { + ({ identity: current } = await resolveOp(opened.session, addr.user)); + } finally { + opened.session.stream.close(); + } + let found = null; + for (let n = 0; n <= MAX_CHAIN; n++) { + if (timingSafeEqual(identityFromSeed(identitySeed(masterBytes, n)).publicKey, current)) { + found = n; + break; + } + } + if (found === null) + throw new SmolError(`the key bound to ${addr.short} is not derived from this master ` + + `within ${MAX_CHAIN} rotations`); + store.setRotationIndex(found); + store.setAccount(addr); + return addr; +} + $("recall-form").onsubmit = async (event) => { event.preventDefault(); const form = event.target; + const submit = form.querySelector("button"); + const resting = submit.textContent; + submit.disabled = true; // a round trip to the server takes a moment; say so rather than sit inert + submit.textContent = "recalling…"; try { const me = store.identity(); const addr = await recallAccount(form.address.value); @@ -1003,6 +1253,9 @@ $("recall-form").onsubmit = async (event) => { form.reset(); } catch (err) { status(err.message, true); + } finally { + submit.disabled = false; + submit.textContent = resting; } }; @@ -1017,7 +1270,7 @@ $("export-data").onclick = async () => { // No counts in the message: the file itself is sealed precisely so an // observer learns nothing from it, and a summary here would just move // that same leak from the file to the screen right next to it. - status("exported — sealed to this identity; only a browser with this seed restored can open it"); + status("exported — sealed to this identity; only a browser with this master restored can open it"); } catch (err) { status(err.message, true); } @@ -1046,40 +1299,50 @@ $("import-data").onchange = async (event) => { } }; -async function rotateIdentity() { +// §7: rotate to the next index's derived key and rebind the account with a +// signed certificate. The superseded key stays derivable from the master, +// since mail sealed to it stays readable with nothing else. +async function rotateIdentity(event) { const me = store.identity(); + const master = store.master(); const addr = accountAddress(); - if (!me || !addr) return status("rotate needs a registered account", true); + if (!me || !master || !addr) return status("rotate needs a registered account", true); if (!window.confirm("Rotate the identity key?\n\nContacts accept the change from the signed " + "chain. The old key is kept, since mail sealed to it stays readable with nothing else.")) return; + const button = event.target; + button.disabled = true; // a round trip to the server takes a moment; say so rather than sit inert + button.textContent = "rotating…"; try { - const fresh = newIdentity(); - const cert = makeCert(me, fresh.seed); + const freshSeed = identitySeed(master, store.rotations() + 1); + const fresh = identityFromSeed(freshSeed); + const cert = makeCert(addr.user, me, freshSeed); const opened = await connect(addr, { requirePin: true }); try { - await registerOp(opened.session, { username: addr.user, identity: fresh, cert }); + await registerOp(opened.session, opened.serverStatic, { username: addr.user, identity: fresh, cert }); } finally { opened.session.stream.close(); } - store.rotateIdentity(fresh.seed); + store.advanceRotation(); // The key set grew, so a cached "not one of our keys" failure may be stale. openedCache.clear(); status(`rotated ${addr.short}\nnew key: ${b32encode(fresh.publicKey)}`); renderAccount(); - renderSettings(); + renderSettings(); // rebuilds the identity card, this button included renderFolder(); } catch (err) { + button.disabled = false; + button.textContent = "rotate identity"; status(err.message, true); } } -// Unlike rotate, nothing is kept: this is the seed's only local copy, gone. +// Unlike rotate, nothing is kept: this is the master's only local copy, gone. async function logout() { if (!window.confirm( "Log out and erase this browser?\n\n" + - "This deletes the seed, pins, contacts and every cached message from this browser. Make sure " + - "the seed is backed up elsewhere first — without it there is no way back in." + "This deletes the master, pins, contacts and every cached message from this browser. Make sure " + + "the master is backed up elsewhere first — without it there is no way back in." )) return; await store.clearAll(); location.reload(); diff --git a/web/js/proto.js b/web/js/proto.js index 3908879..e8ed6ec 100644 --- a/web/js/proto.js +++ b/web/js/proto.js @@ -1,18 +1,20 @@ -// Smol Mail protocol, version 1 (../smolmail SPEC.md): addresses, sealed and -// signed envelopes, body frontmatter, key rotation, and the framed request -// and response bodies of the five operations. +// Smol Mail protocol, version 1.1 (../smolmail SPEC.md): addresses, sealed and +// signed envelopes, body frontmatter, key rotation, accept tokens, and the +// framed request and response bodies of the five operations. import { aeadDecrypt, aeadEncrypt, concat, ed25519PublicKey, ed25519SeedToX25519, - ed25519Sign, ed25519ToX25519, ed25519Verify, hkdfSha256, randomBytes, sha256, timingSafeEqual, - utf8Bytes, x25519, x25519Base, + ed25519Sign, ed25519ToX25519, ed25519Verify, hkdfSha256, hmacSha256, randomBytes, sha256, + timingSafeEqual, utf8Bytes, x25519, x25519Base, } from "./crypto.js"; import { NxInitiator } from "./noise.js"; export const DEFAULT_PORT = 1961; -export const KEY_LEN = 32, SIG_LEN = 64, CERT_LEN = 136, ID_LEN = 16; +export const KEY_LEN = 32, SIG_LEN = 64, CERT_LEN = 200, ID_LEN = 32, TOKEN_LEN = 32; export const MAX_FRAME = 1 << 20, NOISE_PAYLOAD = 65535 - 16, PAD_TO = 1024; export const ENVELOPE_HEADER = 69, PAYLOAD_HEADER = 45, MAX_CHAIN = 16; +export const MAX_SKEW = 86400; // §5.3: how far ahead of our clock a payload may be dated +export const FLAG_REQUESTS = 0x01; // §6.1: set when a FETCH record missed an accept token const FRONTMATTER_MAX = 4096, FRONTMATTER_KEYS = 64; export const OP = { AUTH: 0x00, RESOLVE: 0x01, SEND: 0x02, FETCH: 0x03, DELETE: 0x04, REGISTER: 0x05 }; @@ -25,7 +27,9 @@ export class SmolError extends Error {} const LABEL = Object.freeze({ auth: utf8Bytes("smolmail/1 auth"), seal: utf8Bytes("smolmail/1 seal"), msg: utf8Bytes("smolmail/1 msg"), id: utf8Bytes("smolmail/1 id"), - rotate: utf8Bytes("smolmail/1 rotate"), + rotate: utf8Bytes("smolmail/1 rotate"), identity: utf8Bytes("smolmail/1 identity"), + accept: utf8Bytes("smolmail/1 accept"), mac: utf8Bytes("smolmail/1 mac"), + register: utf8Bytes("smolmail/1 register"), }); // --- encoding helpers ------------------------------------------------------- @@ -89,16 +93,37 @@ export class Reader { get done() { return this.pos === this.buf.length; } } -// --- identity --------------------------------------------------------------- +// --- identity (§2) ------------------------------------------------------------ -// §2: an Ed25519 keypair with the X25519 agreement keys derived from it. +// An Ed25519 keypair with the X25519 agreement keys derived from it. export function identityFromSeed(seed) { if (seed.length !== KEY_LEN) throw new SmolError(`identity seed must be ${KEY_LEN} bytes`); return { seed, publicKey: ed25519PublicKey(seed) }; } -export function newIdentity() { - return identityFromSeed(randomBytes(KEY_LEN)); +export function newMaster() { + return randomBytes(KEY_LEN); +} + +// §2: the only secret a user holds. Every rotation index's signing seed, and +// the accept key, are derived from it with HKDF. +export function identitySeed(master, index) { + return hkdfSha256(master, new Uint8Array(0), concat(LABEL.identity, u32BE(index))); +} + +export function acceptKeyFor(master) { + return hkdfSha256(master, new Uint8Array(0), LABEL.accept); +} + +// §5.8: the token this account issues to one correspondent, independent of +// the rotation index so it survives the owner's key rotation. +export function tokenFor(master, correspondentIdentity) { + return hmacSha256(acceptKeyFor(master), correspondentIdentity); +} + +// §5.8: what a sender attaches to SEND to reach the recipient's main tier. +export function acceptMac(token, id) { + return hmacSha256(token, concat(LABEL.mac, id)); } // --- addressing (§3) -------------------------------------------------------- @@ -132,8 +157,10 @@ export function parseAddress(text) { // --- message format (§5) ---------------------------------------------------- +// §5.4: derived from the envelope so no sender can choose it; used whole, +// nothing truncates it. export function messageId(envelope) { - return sha256(concat(LABEL.id, envelope)).slice(0, ID_LEN); + return sha256(concat(LABEL.id, envelope)); } // §5.2 and §5.3. The ephemeral key is thrown away after sealing, so the sender @@ -174,6 +201,7 @@ export function unseal(identities, envelope) { const body = r.take(bodyLen), signature = r.take(SIG_LEN); // trailing bytes are padding if (!ed25519Verify(sender, concat(LABEL.msg, to, epk, plaintext.slice(0, PAYLOAD_HEADER), body), signature)) throw new SmolError("signature does not verify"); + if (when > BigInt(nowSeconds() + MAX_SKEW)) throw new SmolError("payload is dated in the future"); return { sender, time: Number(when), body, id: messageId(envelope) }; } @@ -183,7 +211,8 @@ const FM_KEY = /^[A-Za-z0-9-]{1,64}$/; // A flat `Key: value` block, deliberately not YAML. Any malformed line // invalidates the whole block, which is then returned as ordinary body text: -// frontmatter fails closed toward display, never toward silent discard. +// frontmatter fails closed toward display, never toward silent discard. Keys +// are compared case-insensitively (§5.5), so they are returned lowercased. export function parseFrontmatter(text) { if (!text.startsWith("---\n")) return { fields: {}, body: text }; const lines = text.split("\n"); @@ -197,7 +226,7 @@ export function parseFrontmatter(text) { for (let i = 0; i < block.length; i++) { const line = block[i], colon = line.indexOf(":"); if (colon < 0 || !FM_KEY.test(line.slice(0, colon))) return { fields: {}, body: text }; - const key = line.slice(0, colon); + const key = line.slice(0, colon).toLowerCase(); if (!(key in fields)) fields[key] = line.slice(colon + 1).trim(); // first occurrence wins } return { fields, body: rest }; @@ -212,28 +241,35 @@ export function buildFrontmatter(fields, body) { // --- key rotation (§7) ------------------------------------------------------- -export function makeCert(oldIdentity, newSeed, when = nowSeconds()) { +// old_pub 32 || new_pub 32 || time 8 || sig_old 64 || sig_new 64. Both keys +// sign, so the old key alone cannot hand the username to a key nobody +// controls; the username is covered but not carried, so a verifier always +// supplies the one it is checking. +export function makeCert(username, oldIdentity, newSeed, when = nowSeconds()) { const newPub = ed25519PublicKey(newSeed), time = i64BE(when); + const signed = concat(LABEL.rotate, utf8Bytes(username), oldIdentity.publicKey, newPub, time); return concat(oldIdentity.publicKey, newPub, time, - ed25519Sign(oldIdentity.seed, concat(LABEL.rotate, oldIdentity.publicKey, newPub, time))); + ed25519Sign(oldIdentity.seed, signed), ed25519Sign(newSeed, signed)); } // Accept a key change only when a signed chain leads from the key we hold to -// the one the server now returns (§7). -export function walkChain(pinned, current, chain) { +// the one the server now returns, both keys signing each link (§7). +export function walkChain(username, pinned, current, chain) { const same = (a, b) => timingSafeEqual(a, b); if (same(pinned, current)) return true; if (!chain.length || chain.length > MAX_CHAIN) return false; let key = pinned, started = false; for (const cert of chain) { - const old = cert.slice(0, 32), next = cert.slice(32, 64), when = cert.slice(64, 72), sig = cert.slice(72); + const old = cert.slice(0, 32), next = cert.slice(32, 64), when = cert.slice(64, 72); + const sigOld = cert.slice(72, 136), sigNew = cert.slice(136, 200); if (!started) { if (!same(old, key)) continue; // a link predating the key we hold started = true; } else if (!same(old, key)) { return false; // the chain is not continuous } - if (!ed25519Verify(old, concat(LABEL.rotate, old, next, when), sig)) return false; + const signed = concat(LABEL.rotate, utf8Bytes(username), old, next, when); + if (!ed25519Verify(old, signed, sigOld) || !ed25519Verify(next, signed, sigNew)) return false; key = next; } return started && same(key, current); @@ -356,13 +392,20 @@ export function expectOk(status, what) { } // §4 session authentication: sign the handshake hash, which binds the -// signature to this session's server ephemeral and cannot be replayed. -export async function authenticate(session, handshakeHash, username, identity) { +// signature to this session's server ephemeral and cannot be replayed, and +// push the accept token set (§5.8). `sync = 0` leaves the server's stored set +// untouched and `tokens` MUST then be empty; `sync = 1` replaces it exactly. +// Returns the number of accept tokens the server now holds. +export async function authenticate(session, handshakeHash, username, identity, { sync = 0, tokens = [] } = {}) { const name = utf8Bytes(username); if (name.length > 255) throw new SmolError("username too long"); + if (tokens.length > 0xffff) throw new SmolError("too many accept tokens for one AUTH"); const body = concat(Uint8Array.of(name.length), name, identity.publicKey, - ed25519Sign(identity.seed, concat(LABEL.auth, handshakeHash))); - expectOk((await session.call(OP.AUTH, body)).status, "authentication"); + ed25519Sign(identity.seed, concat(LABEL.auth, handshakeHash)), + Uint8Array.of(sync), u16BE(tokens.length), ...tokens); + const { status, body: reply } = await session.call(OP.AUTH, body); + expectOk(status, "authentication"); + return reply.length >= 2 ? new Reader(reply).u16() : 0; } // RESOLVE, returning the current key and its rotation chain (§6.1). @@ -375,19 +418,26 @@ export async function resolveOp(session, user) { return { identity: r.take(KEY_LEN), chain: Array.from({ length: r.u8() }, () => r.take(CERT_LEN)) }; } -export async function sendOp(session, envelope) { - const { status, body } = await session.call(OP.SEND, envelope); +// §5.8: `mac` is the sender's proof of an accept token, absent or TOKEN_LEN bytes. +export async function sendOp(session, envelope, mac = null) { + const macBytes = mac ?? new Uint8Array(0); + if (macBytes.length && macBytes.length !== TOKEN_LEN) + throw new SmolError(`accept MAC must be ${TOKEN_LEN} bytes`); + const body = concat(Uint8Array.of(macBytes.length), macBytes, envelope); + const { status, body: reply } = await session.call(OP.SEND, body); expectOk(status, "sending"); - return body.length === ID_LEN ? body : messageId(envelope); + return reply.length === ID_LEN ? reply : messageId(envelope); } -export async function fetchOp(session) { - const { status, body } = await session.call(OP.FETCH); +// §6.1: pages forward from a cursor; an all-zero id starts at the beginning. +export async function fetchOp(session, afterReceivedAt = 0, afterId = new Uint8Array(ID_LEN)) { + const body = concat(i64BE(afterReceivedAt), afterId); + const { status, body: reply } = await session.call(OP.FETCH, body); expectOk(status, "fetching"); - const r = new Reader(body); + const r = new Reader(reply); return Array.from({ length: r.u16() }, () => { - const id = r.take(ID_LEN), receivedAt = Number(r.i64()); - return { id, receivedAt, envelope: r.take(r.u32()) }; + const id = r.take(ID_LEN), receivedAt = Number(r.i64()), flags = r.u8(); + return { id, receivedAt, flags, envelope: r.take(r.u32()), isRequest: Boolean(flags & FLAG_REQUESTS) }; }); } @@ -399,12 +449,19 @@ export async function deleteOp(session, ids) { return new Reader(reply).u16(); } -export async function registerOp(session, { username, identity, token = "", cert = null }) { +// §6.1: the signature is proof of possession, bound to the server that will +// store the binding so it cannot be replayed to another server. +export function registerSigned(serverStatic, username, identity) { + return concat(LABEL.register, serverStatic, utf8Bytes(username), identity); +} + +export async function registerOp(session, serverStatic, { username, identity, token = "", cert = null }) { const name = utf8Bytes(username); const tokenBytes = utf8Bytes(token); if (name.length > 255 || tokenBytes.length > 255 || cert && cert.length > 255) throw new SmolError("REGISTER field too long"); const body = concat(Uint8Array.of(name.length), name, identity.publicKey, + ed25519Sign(identity.seed, registerSigned(serverStatic, username, identity.publicKey)), Uint8Array.of(tokenBytes.length), tokenBytes, Uint8Array.of(cert ? cert.length : 0), cert ?? new Uint8Array(0)); const { status } = await session.call(OP.REGISTER, body); diff --git a/web/js/store.js b/web/js/store.js index 03aaf9c..c0e23a0 100644 --- a/web/js/store.js +++ b/web/js/store.js @@ -1,12 +1,14 @@ // Browser state: identity, pins, contacts and read markers in localStorage; // sealed envelopes in IndexedDB, opened only on demand, so nothing at rest -// is plaintext (the seed excepted — it is the user's browser profile). +// is plaintext (the master secret excepted — it is the user's browser profile). import { aeadDecrypt, aeadEncrypt, hex, hkdfSha256, randomBytes, unhex, utf8Bytes } from "./crypto.js"; -import { b32decode, b32encode, identityFromSeed } from "./proto.js"; +import { ID_LEN, MAX_CHAIN, b32decode, b32encode, identityFromSeed, identitySeed, tokenFor } from "./proto.js"; const KEY = "gsmol"; +export const TIER_MAIN = 0, TIER_REQUESTS = 1; + // Every getter reads the whole blob and some are called per message row, so the // parse is cached. `derived` holds the identities that follow from it, which // cost an Ed25519 scalar multiplication each. @@ -39,44 +41,82 @@ function update(fn) { save(next ?? state); } -// --- identity ---------------------------------------------------------------- +// --- identity (§2) ------------------------------------------------------------- -export function seed() { - const raw = load().seed; +export function master() { + const raw = load().master; return raw ? unhex(raw) : null; } +// The rotation index (§7) of the identity currently in use. +export function rotations() { + return load().rotations ?? 0; +} + export function identity() { return identities()[0] ?? null; } -export function setIdentity(newSeed) { - if (seed()) throw new Error("an identity already exists; rotate it instead"); - update(state => { state.seed = hex(newSeed); }); -} - -// Rotation (§7): the old seed is retained, since mail sealed to a superseded -// key is readable with nothing else. -export function rotateIdentity(newSeed) { - const old = seed(); - if (!old) throw new Error("no identity to rotate"); - update(state => { - state.retired = [...(state.retired || []), { seed: hex(old), at: Date.now() }]; - state.seed = hex(newSeed); - }); -} - -// §7: seeds rotated away from are retained, since mail sealed to a superseded -// key is readable with nothing else. +// §7: every key rotated away from is re-derivable from the master, since mail +// sealed to a superseded key is readable with nothing else. export function identities() { if (derived) return derived; - const raw = seed(); - if (!raw) return []; - const retired = (load().retired || []).map(({ seed: s }) => identityFromSeed(unhex(s))); - derived = [identityFromSeed(raw), ...retired]; + const m = master(); + if (!m) return []; + derived = []; + for (let n = rotations(); n >= 0; n--) derived.push(identityFromSeed(identitySeed(m, n))); return derived; } +function bindMaster(newMaster, rotationIndex, syncOk) { + if (master()) throw new Error("an identity already exists; rotate it instead"); + update(state => { + state.master = hex(newMaster); + state.rotations = rotationIndex; + state.syncOk = syncOk; + }); + setCursor(0, new Uint8Array(ID_LEN)); +} + +// A fresh identity: rotation index 0, and an empty accepted set is already +// complete, so it may sync. +export function setIdentity(newMaster) { + bindMaster(newMaster, 0, true); +} + +// §2: recovering a master alone does not recover which correspondents were +// accepted, so that set must not overwrite the server's until rebuilt. +export function restoreMaster(newMaster, rotationIndex = 0) { + bindMaster(newMaster, rotationIndex, false); +} + +// Corrects the rotation index once a RESOLVE against the account's address +// reveals which one the server actually has bound (see restoreMaster()'s +// default of 0, the common case of a never-rotated identity). +export function setRotationIndex(n) { + update(state => { state.rotations = n; }); +} + +// Whether the accepted-correspondent set held here may replace the server's +// on the next AUTH — false right after a restore from the master alone, whose +// empty set must not erase the server's (§4). +export function syncOk() { + return load().syncOk ?? true; +} + +export function setSyncOk(ok) { + update(state => { state.syncOk = ok; }); +} + +// Rotation (§7): only the index advances; the superseded key stays derivable +// from the master, so nothing has to be archived. +export function advanceRotation() { + const current = rotations(); + if (!master()) throw new Error("no identity to rotate"); + if (current >= MAX_CHAIN) throw new Error(`the rotation chain is full at ${MAX_CHAIN} links`); + update(state => { state.rotations = current + 1; }); +} + // --- account and server pins --------------------------------------------------- export function account() { @@ -111,6 +151,32 @@ export function allPins() { return Object.entries(load().servers || {}).map(([host, key]) => ({ host, key })); } +// --- FETCH behavior and cursor (§6.1) ------------------------------------------- + +// When true, fetch does not acknowledge (delete) what it retrieves — mail +// stays on the server until explicitly deleted. Defaults to the original +// behavior: fetched mail is acknowledged immediately. +export function leaveOnServer() { + return load().leaveOnServer ?? false; +} + +export function setLeaveOnServer(value) { + update(state => { state.leaveOnServer = value; }); +} + +export function cursor() { + const state = load(); + const afterId = state.afterId; + return [state.afterTime ?? 0, afterId ? unhex(afterId) : new Uint8Array(ID_LEN)]; +} + +export function setCursor(afterTime, afterId) { + update(state => { + state.afterTime = afterTime; + state.afterId = hex(afterId); + }); +} + // --- contacts ------------------------------------------------------------------ export function contact(address) { @@ -144,6 +210,70 @@ export function allContacts() { .map(([address, c]) => ({ address, key: c.key, verified: c.verified, history: c.history ?? [] })); } +// --- accept tokens (§5.8) -------------------------------------------------------- + +export function accepted(address) { + const a = (load().accepted || {})[address]; + return a ? { identity: b32decode(a.identity), active: a.active } : null; +} + +// Admit a contact to the main tier. The identity is frozen at acceptance — a +// re-accept after a block must not change which key the token is derived +// from (§5.8). +export function accept(address, identityBytes) { + update(state => { + const table = { ...(state.accepted || {}) }; + const previous = table[address]; + table[address] = { + identity: previous?.identity ?? b32encode(identityBytes), + active: true, + addedAt: previous?.addedAt ?? Date.now(), + }; + state.accepted = table; + }); +} + +// Withdraw a contact's accept token; their mail lands in the requests tier +// from their next message on. Throws if the contact was never accepted. +export function block(address) { + if (!(load().accepted || {})[address]) throw new Error(`${address} was never accepted`); + update(state => { + const table = { ...(state.accepted || {}) }; + table[address] = { ...table[address], active: false }; + state.accepted = table; + }); +} + +export function allAccepted() { + return Object.entries(load().accepted || {}) + .map(([address, a]) => ({ address, identity: b32decode(a.identity), active: a.active })); +} + +// §4: the tokens to push with AUTH, and whether to push at all. A client that +// cannot vouch for its own set — one restored from the master alone — must +// not replace the server's with an incomplete one. +export function tokenSet(masterBytes) { + if (!syncOk()) return { sync: 0, tokens: [] }; + const active = allAccepted().filter(a => a.active).sort((a, b) => a.address.localeCompare(b.address)); + return { sync: 1, tokens: active.map(a => tokenFor(masterBytes, a.identity)) }; +} + +// A token received from a correspondent, filed under the address that issued +// it: an address outlives the keys behind it, so the token keeps working +// across the issuer's rotations (§5.8). +export function tokenFrom(address) { + const raw = (load().tokens || {})[address]; + return raw ? b32decode(raw.token) : null; +} + +export function learnToken(address, tokenBytes) { + update(state => { + const tokens = { ...(state.tokens || {}) }; + tokens[address] = { token: b32encode(tokenBytes), seenAt: Date.now() }; + state.tokens = tokens; + }); +} + // --- read markers --------------------------------------------------------------- export function markRead(idHex) { @@ -201,6 +331,11 @@ function tx(db, box, mode, fn) { }); } +// "requests" is a view over the same physical "inbox" records, filtered by +// tier (§5.8) — not a separate folder, so a message keeps one identity +// regardless of which tier it arrived in. +const physicalFolder = (folder) => folder === "requests" ? "inbox" : folder; + export async function storeMessage(box, record) { const db = await openDb(); await tx(db, box, "readwrite", store => store.put(record)); @@ -215,27 +350,37 @@ export async function storeIfNew(box, record) { return record; } -export async function listMessages(box) { +export async function listMessages(folder) { + const physical = physicalFolder(folder); const db = await openDb(); - const rows = await tx(db, box, "readonly", store => store.getAll()); - return rows.sort((a, b) => (b.receivedAt ?? b.sentAt) - (a.receivedAt ?? a.sentAt)); // newest first + const rows = await tx(db, physical, "readonly", store => store.getAll()); + const wantTier = folder === "requests" ? TIER_REQUESTS : TIER_MAIN; + const filtered = physical === "inbox" ? rows.filter(r => (r.tier ?? TIER_MAIN) === wantTier) : rows; + return filtered.sort((a, b) => (b.receivedAt ?? b.sentAt) - (a.receivedAt ?? a.sentAt)); // newest first } -export async function getMessage(box, id) { +export async function getMessage(folder, id) { const db = await openDb(); - return await tx(db, box, "readonly", store => store.get(id)) ?? null; + return await tx(db, physicalFolder(folder), "readonly", store => store.get(id)) ?? null; +} + +export async function removeMessage(folder, id) { + const db = await openDb(); + await tx(db, physicalFolder(folder), "readwrite", store => store.delete(id)); } // The unread badge must survive leaving the inbox, so it is counted from ids // and read markers alone — no envelope is opened. -export async function unreadCount() { - const db = await openDb(); - const ids = await tx(db, "inbox", "readonly", store => store.getAllKeys()); +async function unreadIn(folder) { + const rows = await listMessages(folder); const read = load().read || {}; - return ids.filter(id => !read[id]).length; + return rows.filter(row => !read[row.id]).length; } -// --- export / import: mail, contacts, pins — never the seed --------------------- +export const unreadCount = () => unreadIn("inbox"); +export const requestsUnreadCount = () => unreadIn("requests"); + +// --- export / import: mail, contacts, pins — never the master -------------------- // btoa/atob work on JS's UTF-16 "binary string" convention (one code unit per // byte); this just bridges that to Uint8Array. Browser-only, like the rest of @@ -246,38 +391,42 @@ const fromBase64 = (text) => Uint8Array.from(atob(text), c => c.charCodeAt(0)); // A gsmol-only construction, not the wire protocol's own HKDF namespace // ("smolmail/1 ..." in proto.js) — export/import has no counterpart in // SPEC.md, so it gets its own label rather than borrowing the protocol's. -// Sealing to a key derived from the identity's own seed means the file is +// Sealing to a key derived from the identity's own master means the file is // opaque without it — no new passphrase to manage, and it can only ever be -// opened where that seed is already restored, exactly the situation import +// opened where that master is already restored, exactly the situation import // already assumes. const EXPORT_LABEL = utf8Bytes("gsmol/1 export"); -const exportKey = (seed) => hkdfSha256(seed, new Uint8Array(0), EXPORT_LABEL, 32); +const exportKey = (masterBytes) => hkdfSha256(masterBytes, new Uint8Array(0), EXPORT_LABEL, 32); -// Deliberately excludes the seed: it already has its own reveal-and-copy flow -// in settings, meant for a password manager, not a downloadable file. Beyond -// that, everything else in here was worth hiding too — contacts and pins are -// a social graph and a list of which mail servers you use, not just the mail -// SPEC.md §5 makes irreplaceable once fetched — so the whole payload is -// sealed, not just the parts that were already ciphertext at rest. +// Deliberately excludes the master: it already has its own reveal-and-copy +// flow in settings, meant for a password manager, not a downloadable file. +// Beyond that, everything else in here was worth hiding too — contacts and +// pins are a social graph and a list of which mail servers you use, not just +// the mail SPEC.md §5 makes irreplaceable once fetched — so the whole payload +// is sealed, not just the parts that were already ciphertext at rest. export async function exportData() { - const me = identity(); - if (!me) throw new Error("no identity yet"); - const [inbox, sent] = await Promise.all([listMessages("inbox"), listMessages("sent")]); + const m = master(); + if (!m) throw new Error("no identity yet"); + const [inbox, requests, sent] = await Promise.all( + [listMessages("inbox"), listMessages("requests"), listMessages("sent")]); const payload = { servers: Object.fromEntries(allPins().map(({ host, key }) => [host, key])), contacts: Object.fromEntries(allContacts().map(({ address, key, verified, history }) => [address, { key, verified, history }])), - inbox: inbox.map(row => ({ id: row.id, receivedAt: row.receivedAt, envelope: toBase64(row.envelope) })), + inbox: [...inbox, ...requests].map(row => ({ + id: row.id, receivedAt: row.receivedAt, envelope: toBase64(row.envelope), + tier: row.tier ?? TIER_MAIN, keptOnServer: row.keptOnServer ?? false, + })), sent: sent.map(row => ({ id: row.id, recipient: row.recipient, sentAt: row.sentAt, envelope: toBase64(row.envelope) })), }; // Random per export: the key is the same every time (deterministic from the - // seed), so nonce reuse has to be ruled out the way it always is under a + // master), so nonce reuse has to be ruled out the way it always is under a // fixed key — a fresh 96-bit nonce per seal, not a fixed one like seal()'s // in proto.js gets away with (there, a fresh ephemeral key each message // makes the derived key itself unique, so a zero nonce is safe). const nonce = randomBytes(12); - const ciphertext = aeadEncrypt(exportKey(me.seed), nonce, utf8Bytes(JSON.stringify(payload)), new Uint8Array(0)); + const ciphertext = aeadEncrypt(exportKey(m), nonce, utf8Bytes(JSON.stringify(payload)), new Uint8Array(0)); return { gsmolExport: 2, exportedAt: Date.now(), @@ -300,11 +449,11 @@ const isRecord = (value) => typeof value === "object" && value !== null && !Arra export async function importData(data) { let payload; if (data?.gsmolExport === 2) { - const me = identity(); - if (!me) throw new Error("no identity yet — restore it before importing"); + const m = master(); + if (!m) throw new Error("no identity yet — restore it before importing"); try { const plaintext = aeadDecrypt( - exportKey(me.seed), fromBase64(data.nonce), fromBase64(data.ciphertext), new Uint8Array(0)); + exportKey(m), fromBase64(data.nonce), fromBase64(data.ciphertext), new Uint8Array(0)); payload = JSON.parse(new TextDecoder().decode(plaintext)); } catch { throw new Error("couldn't decrypt — exported by a different identity, or the file is corrupted"); @@ -352,7 +501,8 @@ export async function importData(data) { for (const row of Array.isArray(rows) ? rows : []) { try { const record = box === "inbox" - ? { id: row.id, receivedAt: row.receivedAt, envelope: fromBase64(row.envelope) } + ? { id: row.id, receivedAt: row.receivedAt, envelope: fromBase64(row.envelope), + tier: row.tier ?? TIER_MAIN, keptOnServer: Boolean(row.keptOnServer) } : { id: row.id, recipient: row.recipient, sentAt: row.sentAt, envelope: fromBase64(row.envelope) }; if (await storeIfNew(box, record)) summary.mailAdded++; } catch { summary.malformed++; } @@ -362,8 +512,8 @@ export async function importData(data) { return summary; } -// Logout: erases the seed, pins, contacts and every cached message from this -// browser. Closes the held connection first so the delete isn't left +// Logout: erases the master, pins, contacts and every cached message from +// this browser. Closes the held connection first so the delete isn't left // "blocked" waiting for a handle that never closes on its own. export async function clearAll() { localStorage.removeItem(KEY); diff --git a/web/style.css b/web/style.css index ecfdff9..961aa09 100644 --- a/web/style.css +++ b/web/style.css @@ -356,6 +356,14 @@ dialog::backdrop { background: var(--backdrop); } #compose textarea { font-family: inherit; resize: vertical; } #compose footer { display: flex; gap: 8px; align-items: center; justify-content: flex-end; padding-top: 4px; flex-wrap: wrap; } #compose .anon { font-size: 13px; color: var(--muted); margin-right: auto; display: flex; align-items: center; gap: 6px; white-space: nowrap; } + +/* A checkbox with a multi-line label, e.g. settings' "leave mail on server" — + the checkbox sits at the label's cap height rather than centered against + the full wrapped block, which is what made it look adrift. */ +#settings label.check-row { flex-direction: row; align-items: flex-start; gap: 10px; flex: none; } +#settings label.check-row input { flex: none; margin: 3px 0 0; } +.check-row .check-title { display: block; color: var(--fg); font-size: 14px; } +.check-row .hint { display: block; margin: 2px 0 0; } /* Without this every action read exactly like "cancel" and "close". */ .filled { background: var(--accent); color: var(--on-accent); font-weight: 500; padding: 8px 20px; } .filled:hover { background: var(--accent); box-shadow: var(--shadow); }