randogoth/fumi
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat: implement Phase 1 TCP client

Browse source
This commit is contained in:
randogoth 2026-09-28 16:03:53 +03:00
parent 1690aed3dc
commit 37df976fde
19 changed files with 5631 additions and 377 deletions
Download patch file Download diff file Expand all files Collapse all files

408
PLAN.md Normal file
View file

@ -0,0 +1,408 @@
# 文 fumi
A Rust implementation of the [Smol Mail](https://code.randogoth.com/randogoth/smolmail) client: a minimalist, end-to-end encrypted mail protocol over a Noise-secured TCP connection, with an optional Reticulum (RNS) mesh carrier behind the `rns` build feature. `fumi` implements the client side only — deriving identities, sealing and opening mail, and talking to a mailbox — and is the counterpart to [bunshin](https://code.randogoth.com/randogoth/bunshin), the server.
**Status: plan only.** Nothing in `src/` exists yet. Everything below is the intended design; the coverage tables are targets, not claims.
## Scope
| | v1.1 (TCP) | v1.2 (RNS) |
|---|---|---|
| `AUTH`, `RESOLVE`, `SEND`, `FETCH`, `DELETE`, `REGISTER` | planned | planned, `rns` feature |
| Envelope sealing and opening (§5.1–5.4) | planned | identical, transport does not touch it |
| Frontmatter (§5.5), sent copies (§5.6), `Reply-To` (§5.7) | planned | identical |
| Accept tokens (§5.8) | planned | identical |
| Key rotation and chain walking (§7) | planned | identical |
| Server pinning | Noise static key | none — the destination hash is the pin (§13.4) |
Section numbers are [SPEC.md](https://code.randogoth.com/randogoth/smolmail/src/branch/main/SPEC.md) throughout; §13 onwards is [RNS.md](https://code.randogoth.com/randogoth/smolmail/src/branch/main/RNS.md).
## Addressing
```
alice@example.org[:1961] short form, resolved via the server
smol://alice@example.org/mfrggzdfzt... self-certifying, 52-char base32 identity
smol+rns://alice@8f2c1d0a7b6e5f4c3d2a1b0e9f8c7d6a short form over Reticulum
smol+rns://alice@8f2c1d0a7b6e5f4c3d2a1b0e9f8c7d6a/mfrggzdfzt... self-certifying over Reticulum
```
The path component is the 32-byte identity key in unpadded lowercase base32 — **52 characters** (§3), not 32. The RNS authority is the server's 16-byte destination hash as exactly 32 lowercase hex characters (§13.2), compared in full, with no port and no bare `user@host` form. The scheme selects the transport; there is nothing else to configure and nothing to negotiate.
---
## Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ main.rs — CLI │
│ keygen whoami restore rotate trust register resolve │
│ import contacts accept block send fetch delete list read │
└──────────────────────────────┬──────────────────────────────┘
▼
┌─────────────────────────────────────────────────────────────┐
│ client.rs — the six operations, AUTH session setup, chain │
│ walking, seal/open pipelines, token sync, outbound queue │
└───┬──────────────┬──────────────┬──────────────┬────────────┘
▼ ▼ ▼ ▼
┌────────────┐ ┌────────────┐ ┌────────────┐ ┌────────────────┐
│ account.rs │ │ message.rs │ │ store.rs │ │ address.rs │
│ master │ │ envelope │ │ SQLite │ │ the four forms │
│ rotation │ │ payload │ │ contacts │ │ usernames │
│ tokens │ │ frontmatter│ │ tokens │ │ URIs │
│ certs │ │ message id │ │ seen, mail │ │ fingerprints │
└──────┬─────┘ └─────┬──────┘ └────────────┘ └────────────────┘
▼ ▼
┌───────────────────────────────┐
│ crypto.rs │
│ hkdf, hmac, sha256, base32 │
│ Ed25519/X25519 map, §2 checks │
└───────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ transport.rs — trait Transport { request, bind, close }, │
│ TransportBindValues, operation and status constants │
└───────────────────▲─────────────────────▲───────────────────┘
implements │ │ implements
┌────────┴────────┐ ┌─────────┴──────────────┐
│ tcp.rs │ │ rns/ (feature "rns") │
│ Noise_NX │ │ shim FFI, links │
│ u32 framing │ │ path discovery │
│ static-key pin │ │ │
└─────────────────┘ └────────────────────────┘
```
`client.rs` depends on the `Transport` trait; the two carriers implement it. Only `transport.rs`'s bind values and the framing differ between them — every operation body, every signature and the whole envelope format are byte-identical (§13.5, §15).
## Modules
| File | Contents |
|---|---|
| `crypto.rs` | base32 (unpadded lowercase, as `bunshin/src/crypto.rs`), HKDF-SHA256, HMAC-SHA256, SHA-256, the Ed25519→X25519 map with §2's checks |
| `address.rs` | `Address` parsing for all four forms, username validation (§3), self-certifying URI rendering, fingerprints |
| `account.rs` | master secret, `seed_n` derivation, the current rotation index, accept-key and per-correspondent token derivation, rotation certificates |
| `message.rs` | envelope seal/open (§5.1–5.3), message id (§5.4), frontmatter parse/build (§5.5) |
| `transport.rs` | `Transport` trait, `TransportBindValues`, operation and status constants |
| `tcp.rs` | Noise_NX initiator, `len u32 \|\| op u8 \|\| body` framing, static-key pinning |
| `rns/` | `mod.rs`, `ffi.rs`, `transport.rs` — behind `#[cfg(feature = "rns")]` |
| `client.rs` | the six operations, AUTH session setup, chain walking, token sync, send/fetch pipelines |
| `store.rs` | SQLite: state, contacts, accepted, tokens, seen ids, inbox, sent |
| `error.rs` | `SmolError` and the status-code mapping |
| `lib.rs`, `main.rs` | library surface and CLI |
## Dependencies
```toml
[package]
name = "fumi"
version = "0.1.0"
edition = "2021"
description = "Smol Mail client"
[features]
# RNS carrier over microReticulum (§13); needs MICRORETICULUM_SOURCE_DIR at
# build time, provided by the flake. Same shape as bunshin's.
rns = ["dep:cc"]
[dependencies]
snow = "0.9" # Noise_NX, TCP carrier only
chacha20poly1305 = "0.10" # envelope AEAD (§5.1) — snow does not expose one
ed25519-dalek = { version = "2", features = ["rand_core"] }
x25519-dalek = { version = "2", features = ["static_secrets"] }
sha2 = "0.10"
hmac = "0.12" # accept tokens and their MACs (§5.8)
hkdf = "0.12"
rand_core = { version = "0.6", features = ["getrandom"] }
rusqlite = { version = "0.32", features = ["bundled"] }
data-encoding = "2" # base32 and HEXLOWER; no separate hex crate
clap = { version = "4", features = ["derive"] }
log = "0.4"
env_logger = "0.11"
anyhow = "1"
[build-dependencies]
cc = { version = "1", optional = true } # compiles the RNS shim, see Phase 2
```
`chacha20poly1305` is the dependency `bunshin` does not have and `fumi` cannot do without: the server stores envelopes without opening them, the client opens them. No `regex` — §3's username grammar is a dozen lines of `matches!`, the way `bunshin/src/proto.rs` does it. No `hex` — `data_encoding::HEXLOWER` already covers RNS destination hashes.
The Ed25519→X25519 map (§2) uses `ed25519_dalek::VerifyingKey::to_montgomery()` for the public half and `SigningKey::to_scalar_bytes()` fed to `x25519_dalek::StaticSecret` for the private half, which is exactly what libsodium's `crypto_sign_ed25519_{pk,sk}_to_curve25519` produce. §2 requires rejecting an all-zero agreement output and a low-order received ephemeral; `x25519-dalek` signals neither, so `crypto.rs` checks the output for all-zero and refuses, which covers both.
---
## Cryptographic detail
The one place a client cannot be approximately right. Taken from §5.2–§5.4; `smolmail.py`'s `seal`/`unseal` are the executable reference.
### Sealing
```
esk, epk = X25519_keygen()
shared = X25519(esk, to_x25519(recipient_identity)) # reject all-zero
key = HKDF-SHA256(shared, salt = epk || recipient_identity,
info = "smolmail/1 seal", len = 32) # one key, not two
wipe(esk)
header = version u8 || sender 32 || time i64 || body_len u32
signature = Ed25519(sender, "smolmail/1 msg" || to || epk || header || body)
plaintext = header || body || signature || padding # padding to 1024, MAY, default on
aad = "SMOL" || version u8 || to 32 || epk 32 # 69 bytes, the envelope header
envelope = aad || ChaCha20-Poly1305(key, nonce = 0^12, aad, plaintext)
id = SHA-256("smolmail/1 id" || envelope)
```
- `"smolmail/1 seal"` is the HKDF info; `"smolmail/1 msg"` is the signature label. They are different labels for different jobs (§11).
- The salt is `epk || recipient_identity`, not empty. It binds the key to this exact sealing.
- **The sender is inside the ciphertext.** Nothing outside the AEAD names a sender — that is the whole of §5.2's unlinkability and §9's privacy claim. Frontmatter lives in `body`, so it is encrypted too.
- The nonce is all-zero and that is correct: `key` is used exactly once because `esk` is fresh per message.
- `body_len` makes the padding unambiguous and the tag protects it.
### Opening
Mirror image, plus the checks a receiver **MUST** perform (§5.3): the envelope's `to` is one of our own keys (including superseded ones, §7), the signature verifies against the `sender` it carries, `time` is not more than 86400 s ahead of our clock, and trailing bytes past `body_len + 64` are ignored as padding. A larger backwards skew MAY be surfaced.
### Accept tokens (§5.8)
```
accept = HKDF-SHA256(master, salt = "", info = "smolmail/1 accept", len = 32)
t = HMAC-SHA256(accept, correspondent_identity)
mac = HMAC-SHA256(t, "smolmail/1 mac" || id)
```
`accept` does not depend on the rotation index, so tokens survive our rotation; `t` is frozen at the identity the correspondent had when accepted, so it survives theirs. The correspondent's copy of `t` travels as a base32 `Accept` frontmatter field inside a sealed, signed payload, and is filed under the address that signed it — never under the key, which rotates.
---
## Client obligations
The requirements that have no natural home in an operation handler and are therefore the ones an implementation forgets. Each is owned by exactly one module.
| Requirement | Section | Owner |
|---|---|---|
| Reject a username that is not 1–63 of `[a-z0-9._-]`, does not start and end alphanumeric, or has adjacent separators; normalise to lowercase | §3 | `address.rs` |
| Abort on a pinned-key mismatch and surface it; mark `RESOLVE` from an unpinned session unverified | §4 | `tcp.rs`, `client.rs` |
| Refuse `REGISTER`, `FETCH`, `DELETE` against a server whose key did not come from a trusted channel | §4 | `client.rs` |
| `AUTH` with `sync = 0` unless the local token set is known complete — a master-restored client must not erase the server's set with an empty one | §4 | `account.rs`, `store.rs` (`sync_ok` flag) |
| Reject all-zero X25519 output and low-order ephemerals | §2 | `crypto.rs` |
| Verify the payload signature, check `to` is ours, enforce the 86400 s forward skew | §5.3 | `message.rs` |
| Keep a sent copy sealed to our own key | §5.6 | `client.rs` |
| `Reply-To`: act on it only when the URI's key equals `sender`, never let it replace a bound key | §5.7 | `client.rs` |
| Frontmatter: 4 KiB, 64 keys, first occurrence wins, case-insensitive keys, malformed block falls back to plain text, no YAML parser | §5.5 | `message.rs` |
| Walk the rotation chain from the pinned key, verify both signatures per link, max 16 links; a broken, absent or over-long chain needs user confirmation; surface every rotation | §7 | `client.rs` |
| Be able to re-derive every superseded private key and accept mail addressed to it | §7 | `account.rs` |
| Keep a seen-id set so a replayed envelope is not re-stored after deletion | §10 | `store.rs` |
| Page `FETCH` forward by `(received_at, id)` until a response is empty; deletion is a separate decision | §6.1 | `client.rs` |
| Treat an unknown status code as failure, never retry automatically, surface the number, never parse the reason string | §12 | `error.rs` |
| Own the outbound queue and retry; resending is safe because ids are derived | §6, §13.10 | `store.rs`, `client.rs` |
## Local store
One SQLite file, shared by both carriers (§15). Following the reference client's schema:
```
state one row: username, host, port, scheme, rotations, cursor (after_time, after_id), sync_ok
servers host -> pinned Noise static key, pinned_at (TCP only; RNS has nothing to pin)
contacts address -> identity, verified, seen_at (verified = key came from a smol:// URI)
accepted address -> identity frozen at acceptance, active (our tokens, issued to others)
tokens address -> token (their tokens, issued to us)
seen id -> at (§10 replay guard)
inbox id -> envelope, received_at, tier (sealed at rest, opened on demand)
sent id -> recipient, envelope, sent_at (§5.6 copies)
```
Envelopes are stored sealed; no plaintext at rest.
---
## CLI
One master per store, selected by the global `--key` and `--db` flags — a second identity is a second pair of files. This drops the multi-account juggling an earlier draft had and with it the ambiguity of which account a bare `fetch <address>` meant: the home address lives in `state`, so only the commands that address someone else take an address.
```
fumi [--key identity.key] [--db fumi.db] [--timeout 30] <command>
# identity
fumi keygen [--force] create a master secret
fumi whoami address, public key, fingerprint, rotation index
fumi restore <address> recover local state from the master alone
fumi rotate advance the rotation index, sign and push the certificate
# servers and contacts
fumi trust <host> <key-b32> [--force] pin a server's Noise static key (TCP only)
fumi register <address> [--invite TOKEN] bind this identity to a username
fumi resolve <address> look up a contact's key, walk the chain, pin it
fumi import <smol-uri> add a contact from a self-certifying address
fumi contacts known keys and how each was learned
# accept tokens
fumi accept <address> issue a token, admit to the main tier, sync the set
fumi block <address> withdraw it, sync the set
# mail
fumi send <address> [--subject S] [--body TEXT | --file F | -] [--header K:V]
[--reply-to ID] [--anonymous] [--no-pad]
fumi fetch [--keep] [--reset] retrieve, verify, store, acknowledge
fumi delete <id>... remove from the server explicitly
fumi list [--sent] [--requests]
fumi read <id> [--sent]
# RNS carrier (feature "rns"); addresses select it by scheme
fumi --rns-udp <host:port> --rns-udp-forward <host:port> [--rns-storage DIR] <command>
```
`trust` takes a host rather than a full address because a pin is per server, not per user. `register <address>` binds the username the address already carries, so there is no separate `--username`. Without the `rns` feature, a `smol+rns://` address fails with a message naming the feature rather than a parse error.
## Key differences from bunshin
| Aspect | bunshin (server) | fumi (client) |
|---|---|---|
| Noise role | responder, holds the static key | initiator, has no static key (NX) |
| Session | accepts connections | initiates, one per operation batch |
| Trust | publishes its static key | pins it and aborts on mismatch |
| Signatures | verifies AUTH and REGISTER | produces them over the bind values |
| Envelopes | stores them sealed, never opens one | seals and opens; needs the AEAD directly |
| Rate limits | enforces | respects, and never retries automatically |
| RNS role | `IN`/`SINGLE` destination, announces, handles requests | path discovery, `OUT`/`SINGLE`, opens links, sends requests |
---
## Phase 1: TCP client
| # | Module | Notes |
|---|---|---|
| 1 | `crypto.rs` | base32, HKDF, HMAC, the X25519 map and §2's checks |
| 2 | `address.rs` | all four address forms, username grammar |
| 3 | `account.rs` | master, `seed_n`, accept key, tokens, rotation certificates |
| 4 | `message.rs` | seal, open, message id, frontmatter |
| 5 | `transport.rs` | trait, `TransportBindValues::tcp`, constants |
| 6 | `tcp.rs` | Noise_NX initiator, framing, pinning |
| 7 | `store.rs` | schema above |
| 8 | `client.rs` | six operations, chain walking, token sync, fetch pipeline |
| 9 | `main.rs` | CLI |
Modules 1–4 are pure and fully unit-testable against the reference client's vectors before a socket is opened.
## Phase 2: RNS carrier
Built the way [bunshin's RNS carrier](https://code.randogoth.com/randogoth/bunshin/src/branch/main/RNS.md) was, and for the same reasons: [microReticulum](https://github.com/attermann/microReticulum) (C++17, Apache-2.0, on upstream's community-implementations list) behind a thin C ABI shim, pinned at the commit bunshin uses — `40fa628`, 2026-07-20. The Python-sidecar and native-Rust options are not chosen: the first doubles the session rules across two implementations, the second has no listed-upstream candidate. Revisit only if a Rust stack gets listed.
No `libffi`: it builds calls at runtime, which is not what linking a C++ library needs. `build.rs` compiles the shim with `cc` and the library with cmake, exactly as bunshin does.
| # | Work | Notes |
|---|---|---|
| 1 | `shim/` | the C ABI below, plus `udp_interface.{h,cpp}` from bunshin |
| 2 | `build.rs` | cmake over the vendored checkout, then `cc` for the shim |
| 3 | `rns/ffi.rs` | `extern "C"` declarations and the safe wrapper |
| 4 | `rns/transport.rs` | `impl Transport`, `TransportBindValues::rns`, timeouts, teardown |
| 5 | `address.rs`, `main.rs` | `smol+rns://` dispatch and the `--rns-*` flags |
| 6 | `flake.nix` | pin microReticulum and every FetchContent dependency |
| 7 | tests | bind vectors, then end-to-end against `smolmaild_rns.py` |
`transport.rs` itself does not change: the trait already carries the bind values, which is the only thing the two carriers disagree about.
### Client-side surface
microReticulum's client API is present and mirrors the Python reference's flow:
| Need | API | Header |
|---|---|---|
| Path discovery | `Transport::has_path`, `Transport::request_path` | `Transport.h` |
| Server identity | `Identity::recall(destination_hash)` | `Identity.h` |
| Destination | `Destination(identity, OUT, SINGLE, "smolmail", "server")` | `Destination.h` |
| Link | `Link(destination, established_cb, closed_cb)`, `Link::link_id()`, `teardown()` | `Link.h` |
| Request | `Link::request(path, data, response_cb, failed_cb, progress_cb, timeout)` | `Link.h:194` |
| Response | `RequestReceipt::get_response()`, `get_status()` | `Link.h:96` |
Every callback is a bare function pointer with no userdata, the same constraint bunshin hit. A CLI has one request in flight at a time, so the shim keeps a single response slot plus a condvar rather than a map.
### Shim ABI
```c
/* All calls block the caller; the shim owns the Reticulum loop thread and
every callback fires there, as in bunshin's shim. */
int smolmail_rns_start(const char *storage_dir,
const char *udp_listen_host, uint16_t udp_listen_port,
const char *udp_forward_host, uint16_t udp_forward_port);
/* Requests a path if none is known and waits for it, recalls the identity,
builds the OUT/SINGLE destination, opens the link and waits for ACTIVE.
Returns the 16-byte link id, which the bind values are derived from. */
int smolmail_rns_connect(const uint8_t *destination_hash, uint32_t timeout_ms,
uint8_t *link_id_out);
/* request is `op u8 || body`, response is `status u8 || payload`. */
int smolmail_rns_request(const uint8_t *request, size_t request_len,
uint32_t timeout_ms, uint8_t *out, size_t cap, size_t *out_len);
void smolmail_rns_close(void);
```
No client Reticulum identity is created or loaded, and the shim never calls `Link::identify`: a server MUST NOT require identification (§13.8), and a durable client handle is exactly what the wire format is built to withhold.
### Client requirements specific to this carrier
- Request a path and **wait** for it before constructing the link. Creating one first makes RNS assume the maximum hop count and fail after minutes rather than promptly (§13.3). `Identity::recall` can still return nothing immediately after a path appears; that is a retry, not an error.
- Set an explicit request timeout. Reticulum's default is derived from the round-trip time and covers a packet, not a `FETCH` page (§13.5). Default 30 s, as the reference client.
- One fresh link per session, never reused across authentications — `link_id` is what stops an `AUTH` replaying on another link (§13.6).
- No path, a dead link, a rejected resource and a timeout are local errors. They MUST NOT be reported as status codes (§13.5).
- Tear the link down when done rather than leave it on keepalives (§13.10).
- Remember a size cap learned from status 6 or 7 instead of discovering it twice (§13.7). Servers run ~32 KiB envelope and fetch budgets on mesh.
- Retrying a `SEND` of unknown outcome is safe and expected here: ids are derived and a resend returns the same id with no new message (§10, §13.10).
### Bind values
`TransportBindValues` is lifted from `bunshin/src/bind.rs` unchanged, including its test vectors — the client produces the signatures the server verifies, so sharing the construction is the point.
```
h = SHA-256("smolmail/1 bind" || destination || link_id) # replaces the Noise handshake hash
server_static = SHA-256("smolmail/1 bind" || destination) # replaces the Noise static key
```
### Build and interfaces
`build.rs` copies the microReticulum checkout from `MICRORETICULUM_SOURCE_DIR` into `OUT_DIR`, applies the one-line `#include <cstdint>` patch upstream master needs under current libstdc++, configures cmake with every `RNS_<DEP>_SOURCE_DIR` override the flake supplies, then compiles the shim with `cc` and links both archives. The flake vendors each FetchContent dependency as a pinned `flake = false` input so the sandboxed build never fetches.
microReticulum ships no interfaces, only examples, so `shim/udp_interface.{h,cpp}` comes over from bunshin verbatim (Apache-2.0). One asymmetry: bunshin can answer to the source of the last datagram it received, but a client speaks first, so `--rns-udp-forward` is effectively required and should point at an `rnsd` or at the server's UDP interface.
Two upstream divergences bunshin already found and that apply here too: `Curve25519::eval` does not clamp the scalar, so a destination hash must never be re-derived in Rust — but a client only ever consumes a hash from an address, so this stays a non-issue as long as nothing tries to verify one locally. And microReticulum splices request and response payloads into msgpack envelopes verbatim; bunshin's shim packs and unpacks on the server side, and the client shim should expect the same in both directions. Verify against `smolmaild_rns.py` before trusting the symmetry.
---
## Testing
### Unit
Address parsing and rejection (52-char base32, 32-char hex, port rejection on `smol+rns://`, bad usernames); identity derivation and rotation against vectors from `smolmail.py`; seal/open round trip; `unseal` of an envelope produced by the reference client, and the reverse; frontmatter parser against §5.5's rules including the malformed-block fallback; token and MAC derivation; `TransportBindValues` against bunshin's vectors; chain walking, including the broken, over-long and absent cases.
### Integration
Against a local `bunshin`: register, resolve, send, fetch, delete, accept-token sync with both `sync` values, key rotation followed by a fetch of mail addressed to the superseded key, quota and rate-limit responses, and an unknown status code.
### Interoperability
Both directions against `smolmail.py`/`smolmaild.py` on TCP and `smolmail_rns.py`/`smolmaild_rns.py` on RNS, plus the cross-transport case §15 promises: send over TCP, fetch over RNS, one identity and one store.
---
## Project structure
```
fumi/
├── Cargo.toml
├── build.rs # rns feature only
├── flake.nix # vendors microReticulum and its deps
├── shim/
│ ├── smolmail_rns.{h,cpp} # C ABI over microReticulum, client side
│ └── udp_interface.{h,cpp} # from bunshin, Apache-2.0
├── src/
│ ├── main.rs lib.rs error.rs
│ ├── client.rs account.rs address.rs message.rs store.rs crypto.rs
│ ├── transport.rs tcp.rs
│ └── rns/ mod.rs ffi.rs transport.rs
├── tests/
└── README.md
```
## References
- [SPEC.md](https://code.randogoth.com/randogoth/smolmail/src/branch/main/SPEC.md) — protocol 1.1
- [RNS.md](https://code.randogoth.com/randogoth/smolmail/src/branch/main/RNS.md) — the 1.2 addendum
- [smolmail.py](https://code.randogoth.com/randogoth/smolmail/src/branch/main/smolmail.py), [smolmail_rns.py](https://code.randogoth.com/randogoth/smolmail/src/branch/main/smolmail_rns.py) — reference clients
- [bunshin](https://code.randogoth.com/randogoth/bunshin) and its [RNS.md](https://code.randogoth.com/randogoth/bunshin/src/branch/main/RNS.md) — the server, and the carrier this plan follows
- [microReticulum](https://github.com/attermann/microReticulum) — C++ Reticulum, pinned at `40fa628`