diff --git a/README.md b/README.md index 5d18732..3651e4e 100644 --- a/README.md +++ b/README.md @@ -64,7 +64,7 @@ nix build # or: nix develop -c cargo build nix develop -c cargo test --workspace ``` -The crate is a workspace: `core/` is the `fumi-core` library — everything but the command line — and `cli/` is the `fumi` binary. GUI hosts depend on `fumi-core` alone; it pulls no argument parsing, no logging globals, and no keyfile conventions. `bundled-sqlite` (default) vendors libsqlite3; embedders that link their own SQLite, as Android's NDK does, build with `default-features = false`. +The crate is a workspace: `core/` is the `fumi-core` library — everything but the command line — and `cli/` is the `fumi` binary. GUI hosts depend on `fumi-core` alone; it pulls no argument parsing, no logging globals, and no keyfile conventions. `store` (default) is the SQLite local-state module and the operations that take it; `bundled-sqlite` (default) vendors libsqlite3, so embedders that link their own SQLite, as Android's NDK does, build with `default-features = false` and that flag alone. With both defaults off the crate keeps identities, addresses, seal/open and the raw transport operations (`resolve`, `walk_chain`, `accept_mac`) — for hosts with their own database and thin FFI consumers that need only the crypto and envelope layer. `bunshin` and the reference clients make a complete test rig: register against a local server, exchange mail in both directions, then rotate and fetch the mail that the superseded key still receives. @@ -73,7 +73,8 @@ The crate is a workspace: `core/` is the `fumi-core` library — everything but `fumi-core` is the protocol core of a GUI client, not just this CLI's engine. The contract an embedder can rely on: - **Secrets stay bytes in the API, files stay in the CLI.** `Account::new(master, index)` takes the master as bytes and `Store::open(path)` takes only mail and state; `identity.key` next to `fumi.db` is a CLI convention. A GUI keeps the master wherever its platform wants it (Android Keystore, OS keychain) and hands it over per operation. -- **One shareable store handle.** `Store` is `Send + Sync`: every operation locks an inner mutex, and the database runs in WAL mode, so a host holds one `Store` behind an executor and reads it while a fetch writes. Each envelope is committed as it is verified — a concurrent reader listing the inbox sees fetch progress without any callback. +- **A storeless build.** `default-features = false` drops rusqlite entirely: identities, addresses, seal/open and the raw transport operations remain, for hosts with their own database. `store` and `bundled-sqlite` compose with it as separate features. +- **One shareable store handle.** `Store` is `Send + Sync`: every operation locks an inner mutex, and the database runs in WAL mode, so a host holds one `Store` behind an executor and reads it while a fetch writes. Each envelope is committed as it is verified — a concurrent reader listing the inbox sees fetch progress without any callback. `Store::open_in_memory()` gives a scratch store with the schema and version already in place — for tests, foreign importers assembling rows in code, and FFI round-trips that never touch disk. - **Decisions, not prose.** `error::Error` is an enum: `NotPinned`, `PinMismatch`, `KeyChanged { address, known, offered }`, `NotRegistered`, `AuthFailed`, `RateLimited`, `QuotaExceeded`, `SchemaVersion`, ... — the distinctions a UI routes on. `Display` still produces the message the CLI prints. - **Long operations can stop.** `fetch_with` takes a `FetchOptions` with a cancellation token checked between pages; the cursor is saved per page, so a cancelled fetch resumes rather than repeats. - **Composable flows.** `restore` remains the one-call convenience, but its pieces are public: `connect` + `resolve`, `account::find_rotation_index`, then the `Store` setters — a wizard can hold the master and the address before it has connectivity, and show each step. Trust outcomes arrive as values: `TrustChange` from `trust_key` and `send`, `Session::unpinned_static` for the sec 4 warning. @@ -82,6 +83,25 @@ The crate is a workspace: `core/` is the `fumi-core` library — everything but The intended call graph: `Store::open` once, `Account::new(master, rotations)` per session, then `client::*` operations taking `&Store` and `&Account`. The store outlives accounts; nothing in the library caches between calls. +## The store schema + +The schema is a contract, not an implementation detail: `SCHEMA_VERSION` is 1, written to SQLite's `PRAGMA user_version` at creation and checked at open, so a foreign importer — a migration from another client, a sync tool — can build rows directly instead of reading `store.rs`. `Store::open_in_memory()` gives an importer a scratch database with schema and version in place. Within a version the layout below is stable; a version bump comes with a documented migration. + +Every mail column holds a sealed envelope, never plaintext. Timestamps are Unix seconds; keys and ids are 32-byte blobs. + +| Table | Columns | Meaning and invariants | +|---|---|---| +| `state` | one row, `id = 1`, always present | `username`, `host`, `port`, `scheme` (`tcp` or `rns`) are the account; `rotations` is the sec 2 rotation index the account is at; `after_time` and `after_id` are the sec 6.1 fetch cursor; `sync_ok` (0/1) says whether the local accept-token set may replace the server's — a client restored from the master alone must not erase it (sec 4) | +| `servers` | `host`, `static`, `pinned_at` | The sec 4 pins: a server's Noise static key, keyed by host without port | +| `contacts` | `address`, `identity`, `verified`, `seen_at` | Every key known for an address; `verified` is 1 only when it came from a self-certifying URI | +| `accepted` | `address`, `identity`, `active`, `added_at` | Main-tier admission (sec 5.8). `identity` is frozen at acceptance because the token the correspondent holds is derived from it; `active = 0` is a block and keeps the row | +| `tokens` | `address`, `token`, `seen_at` | Accept tokens correspondents issued us, filed under their address, which outlives the keys behind it (sec 5.8) | +| `seen` | `id`, `at` | Every message id ever fetched; a replay is not re-stored even after a local delete (sec 10) | +| `inbox` | `id`, `envelope`, `received_at`, `tier` | Sealed mail; `tier` is 0 main, 1 requests (arrived without a matching accept token) | +| `sent` | `id`, `recipient`, `envelope`, `sent_at` | Sealed self-copies (sec 5.6), with the recipient for the listing | + +An importer should set `user_version` to 1, keep the single `state` row, and respect the frozen `accepted.identity` and `sync_ok` semantics above — those are the two rows where a wrong guess loses data (an erased server-side token set, or a contact's token silently changing). + ## Modules | File | Contents | @@ -93,8 +113,8 @@ The intended call graph: `Store::open` once, `Account::new(master, rotations)` p | `core/src/transport.rs` | `Transport` trait, bind values, operation and status constants | | `core/src/tcp.rs` | Noise_NX initiator, `len u32 \|\| op u8 \|\| body` framing, static-key pinning | | `core/src/rns/` | the Reticulum carrier behind the `rns` feature: FFI over `core/shim/`, path discovery, one link per session | -| `core/src/client.rs` | the six operations, AUTH, chain walking, token sync, fetch pipeline with cancellation | -| `core/src/store.rs` | SQLite: state, contacts, accepted, tokens, seen ids, inbox, sent; `Send + Sync`, WAL, versioned schema | +| `core/src/client.rs` | the six operations, AUTH, chain walking, token sync, fetch pipeline with cancellation; store-taking items behind the `store` feature | +| `core/src/store.rs` | SQLite behind the `store` feature: state, contacts, accepted, tokens, seen ids, inbox, sent; `Send + Sync`, WAL, versioned schema | | `core/src/error.rs` | the `Error` enum embedders match on, the status-code mapping | | `core/vectors.json` | the committed reference vectors, the tests' source of truth | | `cli/src/main.rs` | the command line: keyfile and `--db` conventions, hints, output | diff --git a/core/Cargo.toml b/core/Cargo.toml index 9c23d4f..d025ca5 100644 --- a/core/Cargo.toml +++ b/core/Cargo.toml @@ -6,7 +6,12 @@ description = "Smol Mail client library: identities, sealing, mailbox operations license = "Apache-2.0" [features] -default = ["bundled-sqlite"] +default = ["store", "bundled-sqlite"] +# Local state in SQLite: the store module and the client operations that take +# one. Off, the crate keeps identities, addresses, seal/open and the raw +# transport operations for hosts with their own database and thin FFI +# consumers that need only the crypto and envelope layer. +store = ["dep:rusqlite"] # Bundle libsqlite3 so the store needs no system SQLite; embedders with their # own SQLite (Android's NDK case) build with default-features = false. bundled-sqlite = ["rusqlite/bundled"] @@ -23,7 +28,7 @@ sha2 = "0.10" hmac = "0.12" hkdf = "0.12" rand_core = { version = "0.6", features = ["getrandom"] } -rusqlite = "0.32" +rusqlite = { version = "0.32", optional = true } data-encoding = "2" [dev-dependencies] diff --git a/core/src/client.rs b/core/src/client.rs index 737e3c6..c659ca5 100644 --- a/core/src/client.rs +++ b/core/src/client.rs @@ -1,31 +1,48 @@ //! The six operations, AUTH session setup, chain walking, token sync and the //! send/fetch pipelines. Everything here is carrier-neutral: it speaks //! through the `Transport` trait and the bodies are byte-identical on both -//! carriers (RNS.md sec 13.5, sec 15). +//! carriers (RNS.md sec 13.5, sec 15). The operations that take a `Store` +//! need the `store` feature; `resolve`, `walk_chain` and `accept_mac` are +//! always built. +use crate::account::RotationCert; +use crate::crypto::{ct_eq, hmac_sha256, KEY_LEN}; +use crate::error::{expect_ok, Error}; +use crate::transport::{ + Reader, Transport, CERT_LEN, ID_LEN, LABEL_MAC, MAX_CHAIN, OP_RESOLVE, TOKEN_LEN, +}; + +#[cfg(feature = "store")] +use crate::account::{Account, Identity}; +#[cfg(feature = "store")] use std::collections::BTreeMap; -use crate::account::{Account, Identity, RotationCert}; +#[cfg(feature = "store")] use crate::address::{Address, Scheme}; -use crate::crypto::{b32, ct_eq, hmac_sha256, KEY_LEN}; -use crate::error::{expect_ok, Error}; +#[cfg(feature = "store")] +use crate::crypto::b32; +#[cfg(feature = "store")] use crate::message::{ accept_field, build_frontmatter, message_id, parse_frontmatter, seal, unseal, Opened, }; +#[cfg(feature = "store")] use crate::store::{now, Store, Stored}; +#[cfg(feature = "store")] use crate::tcp::TcpTransport; +#[cfg(feature = "store")] use crate::transport::{ - Reader, Transport, CERT_LEN, FLAG_REQUESTS, ID_LEN, LABEL_AUTH, LABEL_MAC, LABEL_REGISTER, - MAX_CHAIN, OP_AUTH, OP_DELETE, OP_FETCH, OP_REGISTER, OP_RESOLVE, OP_SEND, TOKEN_LEN, + FLAG_REQUESTS, LABEL_AUTH, LABEL_REGISTER, OP_AUTH, OP_DELETE, OP_FETCH, OP_REGISTER, OP_SEND, }; /// A connection plus what the caller needs to know about how it was made: /// the server's static key when no pin was checked, for the sec 4 warning. +#[cfg(feature = "store")] pub struct Session { transport: Box, pub unpinned_static: Option<[u8; KEY_LEN]>, } +#[cfg(feature = "store")] impl Session { pub fn transport(&mut self) -> &mut dyn Transport { self.transport.as_mut() @@ -39,6 +56,7 @@ impl Session { /// Opens a session, enforcing sec 4's rule about unpinned servers. TCP pins /// the server's Noise static key; on RNS the destination hash is the pin /// (RNS.md sec 13.4). +#[cfg(feature = "store")] pub fn connect( store: &Store, addr: &Address, @@ -174,6 +192,7 @@ pub enum TrustChange { /// Resolves a contact and applies sec 8's trust rules. A key change without /// a valid chain is an error here: the user confirms out of band and /// imports, rather than clicking through. +#[cfg(feature = "store")] pub fn trust_key( store: &Store, transport: &mut dyn Transport, @@ -211,6 +230,7 @@ pub fn trust_key( /// trailing block carries the mailbox's tokens (sec 5.8): `sync = 0` leaves /// the stored set untouched, `sync = 1` replaces it with exactly what /// follows. +#[cfg(feature = "store")] pub fn authenticate( transport: &mut dyn Transport, username: &str, @@ -242,6 +262,7 @@ pub fn authenticate( } /// What pushing the accept-token set achieved (sec 5.8). +#[cfg(feature = "store")] pub enum Pushed { /// Not registered yet; the set travels with the first fetch instead. NotRegistered, @@ -251,6 +272,7 @@ pub enum Pushed { /// An accept or a block only takes effect once the server holds the changed /// set, so it is pushed now rather than at the next fetch. +#[cfg(feature = "store")] pub fn push_tokens(store: &Store, account: &Account, timeout: u64) -> Result { let Some(addr) = store.account()? else { return Ok(Pushed::NotRegistered); @@ -264,6 +286,7 @@ pub fn push_tokens(store: &Store, account: &Account, timeout: u64) -> Result Result { let Some(addr) = store.account()? else { return Err(Error::NotRegistered); @@ -360,6 +385,7 @@ pub fn rotate(store: &Store, account: &Account, timeout: u64) -> Result Result { let mut session = connect(store, addr, false, timeout)?; let (identity, _) = resolve(session.transport(), &addr.user)?; @@ -379,6 +405,7 @@ pub fn restore(store: &Store, master: &[u8; KEY_LEN], addr: &Address, timeout: u } /// One message to send, as the CLI assembled it. +#[cfg(feature = "store")] pub struct SendDraft<'a> { pub address: &'a Address, pub text: String, @@ -393,6 +420,7 @@ pub struct SendDraft<'a> { } /// What `send` did, for the caller to report. +#[cfg(feature = "store")] pub struct Sent { pub id: [u8; ID_LEN], pub bytes: usize, @@ -407,6 +435,7 @@ pub struct Sent { /// Seals and delivers one message (sec 5, sec 6.1). Recipient selection /// prefers a key we already trust: a self-certifying address, then a stored /// contact, and only then RESOLVE with trust on first use. +#[cfg(feature = "store")] pub fn send( store: &Store, account: &Account, @@ -490,12 +519,14 @@ pub fn send( } /// One rejected fetch record: the id and why it did not enter the inbox. +#[cfg(feature = "store")] pub struct Rejected { pub id: [u8; ID_LEN], pub reason: String, } /// What `fetch` did, for the caller to report. +#[cfg(feature = "store")] pub struct Fetched { pub total: u32, pub stored: u32, @@ -512,6 +543,7 @@ pub struct Fetched { /// The knobs a host with a UI needs on `fetch_with`. Progress needs no /// callback: each envelope is committed to the `Store` as it is verified, so /// a concurrent reader (the store is `Send + Sync`) can list incrementally. +#[cfg(feature = "store")] pub struct FetchOptions<'a> { /// Remember the cursor instead of acknowledging with DELETE. pub keep: bool, @@ -524,6 +556,7 @@ pub struct FetchOptions<'a> { pub cancel: Option<&'a std::sync::atomic::AtomicBool>, } +#[cfg(feature = "store")] impl FetchOptions<'_> { /// The defaults a plain fetch uses. fn new(keep: bool, reset: bool, timeout: u64) -> Self { @@ -541,6 +574,7 @@ impl FetchOptions<'_> { /// it takes, so it always pages from the start; `keep` instead remembers the /// cursor. A message that fails verification is left on the server, so a /// client-side bug cannot lose mail. +#[cfg(feature = "store")] pub fn fetch( store: &Store, account: &Account, @@ -552,6 +586,7 @@ pub fn fetch( } /// `fetch` with the full option set, for hosts that need cancellation. +#[cfg(feature = "store")] pub fn fetch_with( store: &Store, account: &Account, @@ -641,6 +676,7 @@ pub fn fetch_with( Ok(summary) } +#[cfg(feature = "store")] fn verify_and_open( account: &Account, mid: &[u8; ID_LEN], @@ -655,6 +691,7 @@ fn verify_and_open( /// Files the accept token a verified payload carried, under the address that /// signed it (sec 5.8). The signature has already been checked by `unseal`, /// so the attribution is the signer's own claim. +#[cfg(feature = "store")] fn learn_token(store: &Store, opened: &Opened) -> Result<(), Error> { let text = String::from_utf8_lossy(&opened.body).into_owned(); let (fields, _) = parse_frontmatter(&text); @@ -669,6 +706,7 @@ fn learn_token(store: &Store, opened: &Opened) -> Result<(), Error> { } } +#[cfg(feature = "store")] fn delete_ids(transport: &mut dyn Transport, ids: &[[u8; ID_LEN]]) -> Result { let mut body = Vec::with_capacity(2 + ids.len() * ID_LEN); body.extend_from_slice(&(ids.len() as u16).to_be_bytes()); @@ -687,6 +725,7 @@ fn delete_ids(transport: &mut dyn Transport, ids: &[[u8; ID_LEN]]) -> Result Result { let opened = unseal(account.keys(), &stored.envelope, now())?; let text = String::from_utf8_lossy(&opened.body).into_owned(); @@ -747,7 +789,7 @@ pub fn accept_mac(token: &[u8; TOKEN_LEN], mid: &[u8; ID_LEN]) -> [u8; 32] { } /// Proof of possession over a server's static key (sec 6.1), for tests. -#[cfg(test)] +#[cfg(all(test, feature = "store"))] fn register_signed(server_static: &[u8], username: &str, identity: &[u8]) -> Vec { [LABEL_REGISTER, server_static, username.as_bytes(), identity].concat() } @@ -755,7 +797,7 @@ fn register_signed(server_static: &[u8], username: &str, identity: &[u8]) -> Vec #[cfg(test)] mod tests { use super::*; - use crate::account::{identity_seed, verify_sig, Identity, RotationCert}; + use crate::account::{identity_seed, verify_sig, Account, Identity, RotationCert}; use crate::transport::{TransportBindValues, SIG_LEN}; fn master() -> [u8; 32] { @@ -850,6 +892,7 @@ mod tests { ); } + #[cfg(feature = "store")] #[test] fn bind_values_feed_the_auth_and_register_signatures() { let bind = TransportBindValues::tcp(&[2u8; 32], &[7u8; 32]).unwrap(); diff --git a/core/src/error.rs b/core/src/error.rs index b535235..c2f1347 100644 --- a/core/src/error.rs +++ b/core/src/error.rs @@ -55,6 +55,7 @@ pub enum Error { port: u16, source: std::io::Error, }, + #[cfg(feature = "store")] Storage(rusqlite::Error), Noise(snow::Error), Io(std::io::Error), @@ -111,6 +112,7 @@ impl fmt::Display for Error { Self::Unreachable { host, port, source } => { write!(f, "cannot reach {host}:{port}: {source}") } + #[cfg(feature = "store")] Self::Storage(e) => write!(f, "storage error: {e}"), Self::Noise(e) => write!(f, "noise error: {e}"), Self::Io(e) => write!(f, "{e}"), @@ -123,6 +125,7 @@ impl std::error::Error for Error { fn source(&self) -> Option<&(dyn std::error::Error + 'static)> { match self { Self::Unreachable { source, .. } => Some(source), + #[cfg(feature = "store")] Self::Storage(e) => Some(e), Self::Noise(e) => Some(e), Self::Io(e) => Some(e), @@ -131,6 +134,7 @@ impl std::error::Error for Error { } } +#[cfg(feature = "store")] impl From for Error { fn from(e: rusqlite::Error) -> Self { Error::Storage(e) diff --git a/core/src/lib.rs b/core/src/lib.rs index 9ec0c0e..cac0109 100644 --- a/core/src/lib.rs +++ b/core/src/lib.rs @@ -14,6 +14,7 @@ pub mod error; pub mod message; #[cfg(feature = "rns")] pub mod rns; +#[cfg(feature = "store")] pub mod store; pub mod tcp; pub mod transport; diff --git a/core/src/store.rs b/core/src/store.rs index 838cfdf..595e345 100644 --- a/core/src/store.rs +++ b/core/src/store.rs @@ -84,12 +84,29 @@ pub type ContactRow = (String, [u8; KEY_LEN], bool, Option); pub const SCHEMA_VERSION: i32 = 1; impl Store { + /// Opens the store at `path`. SQLite's own `:memory:` path works too; + /// `open_in_memory` spells that case without the magic string. pub fn open(path: &Path) -> Result { - let db = Connection::open(path)?; + Store::init(Connection::open(path)?) + } + + /// An in-memory store: the same schema, version and contract, gone with + /// the handle. For tests, foreign importers assembling rows in code, + /// and FFI round-trips that never touch disk. + pub fn open_in_memory() -> Result { + Store::init(Connection::open_in_memory()?) + } + + fn init(db: Connection) -> Result { // WAL keeps concurrent readers cheap while a writer commits, and a // short busy timeout absorbs the contention the mutex cannot (a // second connection in the same process, or a CLI run alongside). - db.pragma_update(None, "journal_mode", "WAL")?; + // An in-memory database journals in memory, so WAL applies to + // files only. + let journal: String = db.query_row("PRAGMA journal_mode", [], |row| row.get(0))?; + if journal != "memory" { + db.pragma_update(None, "journal_mode", "WAL")?; + } db.busy_timeout(std::time::Duration::from_secs(5))?; let version: i32 = db.query_row("PRAGMA user_version", [], |row| row.get(0))?; if version == 0 { @@ -706,6 +723,22 @@ mod tests { } } + #[test] + fn in_memory_store_round_trips() { + let store = Store::open_in_memory().expect("memory store opens"); + let addr = Address::parse("alice@example.org").unwrap(); + store.set_account(&addr).unwrap(); + assert_eq!( + store.account().unwrap().unwrap().short(), + "alice@example.org" + ); + store.store_inbox(&[1u8; 32], b"envelope", 5, false).unwrap(); + assert_eq!(store.mail("inbox").unwrap().len(), 1); + // The same handle through the path spelling. + let path = std::path::Path::new(":memory:"); + assert!(Store::open(path).is_ok()); + } + #[test] fn store_is_shareable_across_threads() { fn assert_send_sync() {}