diff --git a/audit.md b/audit.md new file mode 100644 index 0000000..08eb789 --- /dev/null +++ b/audit.md @@ -0,0 +1,69 @@ +# smolboid audit + +A security and implementation audit of smolboid as of 2026-10-05, covering the `core/` feed format, the `daemon/` binary, and the three end-to-end suites (integration rig, stress, pentest). The audit was conducted against the smolmail protocol (SPEC.md 1.2), PLAN.md, and the actual fumi-core and bunshin codebases it interoperates with. Everything below reflects executed and observed behavior, not intent. + +## Verification results + +| Suite | Command | Result | +| --- | --- | --- | +| Unit and format tests | `devbox run test` | 19 core + 28 daemon + 3 vector pins, all pass | +| Reference vectors | `devbox run vectors` regenerates `core/vectors.json` | 15 cases pin every verification state; byte-identical regeneration | +| Integration rig | `devbox run rig` | Passes: post, nick, repost, delete, timestamp collision, hard limit, FIFO prune, 90% warning, author rotation — all against real bunshin and fumi binaries | +| Stress | `devbox run stress` | Passes: 40-send burst with integrity held, exact 25-record quota, reply cap exact at 10, 200 parallel HTTP requests in 271ms (~738 req/s lower bound), all-304 conditional load, zero failed reads during concurrent writes | +| Pentest | `devbox run pentest` | Passes: every probe repelled (details below) | + +## Trust model, as implemented + +The reader of a feed verifies the author's own §5.3 payload signature, rebuilt from public data alone: the record's sender key, the line timestamp, the canonical body form, and the feed's `smol:host_key` metadata. smolboid never signs content and holds no author private key; its role at receipt is admission (feed exists, sender is an owner key, `accepted_state` is true), not attestation. What the host controls is what gets published at all — it can withhold or prune records, but cannot forge one: a forged or tampered record reconstructs to an invalid state, verified by the pentest's tamper probe and the vectors. + +## Security properties, each with its enforcement point and proof + +- **Feed-line injection is impossible.** Post and repost text is rejected on TAB, LF, CR or U+2028 (post rule 4, `rules.rs`), so a record can never introduce a second line; a text posing as `# smol:mail` or `# smol:repost` publishes only as timestamped post text. Proven by the pentest's injection and poser probes. +- **The feed path cannot escape its directory.** Usernames are validated as 1-30 bytes of `[a-z0-9-]` before any filesystem access (`feedstore.rs`), so traversal, encoding, null-byte, unicode and dotfile forms all 404 without touching a file outside `feeds/`. Proven by nine pentest path probes. +- **A record sealed to another instance cannot be laundered here.** The payload signature covers `to`; the feed's `smol:host_key` is the only `to` accepted. Proven by the vectors' `invalid_foreign_host` case. +- **Requests-tier mail is structurally invisible.** fumi's store splits requests-tier rows into a folder the daemon never lists; unadmitted senders publish nothing, get no feed, and draw no reply. Proven by the pentest's mallory probe. +- **Forged records are detectable, not trusted.** `smolboid check` exits nonzero on any invalid record; the signature check is the load-bearing mechanism, not the host's say-so. Proven by the tamper probe. +- **Reply floods are bounded.** At most 10 error replies per sender per hour, and never twice for the same rejected payload (state ledger plus dedupe ring). Proven by both suites: 15 and ~39 rejections drew at most 10 answers. +- **Quotas and retention hold under rapid fire.** The daily cap is never exceeded even by a 40-send burst, and paced sends land exactly on it; FIFO prunes only what the owner directed; hard-limit mode never silently deletes. Proven by the stress suite. +- **Slow clients cannot wedge the listener.** Socket read/write timeouts of 10s close half-open connections; the listener stayed healthy during and after a slowloris hold. Proven by the pentest. +- **Reads never see torn writes.** Feeds and state are replaced atomically (temp file plus rename); 60 reads during a concurrent delete all succeeded and the feed verified. Proven by the stress suite. +- **Unparseable or hostile input fails closed.** Oversized requests are capped at 8 KiB, malformed bodies become `Ignore`, blocks with unreconstructible fields are rejected with an explanatory reply rather than published. Proven by pentest parsing probes and dispatch unit tests. + +## Findings fixed during this audit + +Each of these was discovered by a suite or by design review against real clients, and is fixed with a regression test or a vector: + +1. **No read timeout on HTTP connections (fixed).** A slowloris client held a thread forever; the listener is thread-per-connection. Socket timeouts added in `http.rs`; the pentest now asserts closure at 10s. +2. **Reposts published as ordinary posts (fixed).** `apply_record` built a `Record::Post` regardless of kind; now a `Record::Repost`, with a unit test and a rig assertion. +3. **"A post mail has no frontmatter" was unimplementable (fixed by design change).** Every real smolmail client attaches a signed `Reply-To` block, so nothing would ever have published. Posts now accept two canonical body forms (bare, or a block whose only field is `Reply-To`), the verifier derives the block from `smol:author` plus the per-record sender key, and bodies with other fields are rejected at receipt rather than published as unverifiable. PLAN.md updated. +4. **Admission required a welcome mail (fixed).** Without the daemon's Accept token an author's mail stays in requests tier forever; `smolboid admit` now sends a welcome mail carrying it. +5. **A `smol:reply_to` metadata binding would break on rotation (fixed by removal).** The author's Reply-To changes with each rotation; per-record derivation replaced the binding entirely. +6. **A stale base32 length in the plan (fixed).** A 64-byte signature is 103 unpadded base32 characters, not 104. + +## Open findings, accepted for v1 + +- **Thread-per-connection is unbounded.** Bounded per connection by the 10s timeouts, and the port is operator-exposed, but a connection flood can still spawn many short-lived threads. A concurrent-connection cap is a small future hardening step. +- **HEAD returns 405.** Legitimate pollers may use HEAD; serving it (same headers, no body) is a small compatibility improvement. +- **The HTTP listener is plain HTTP.** The plan's `https://` origin assumes TLS termination upstream (a reverse proxy); a direct deployment serves feeds in cleartext. This is a deployment requirement to document in the README, not a code defect. +- **Bootstrap trust is operator-mediated.** The daemon's own server pin, registration, and each author's first resolution arrive via the fumi CLI and `smolboid admit` with trust-on-first-use semantics; a compromised or lying DNS/SRV path fails closed at the Noise pin mismatch, but the *first* pin is taken on faith by the operator. +- **`smol:author` is adopted from the first publishing mail.** The first Reply-To a feed's owner sends rewrites the metadata's address spelling (empty feeds only). A malicious owner can only rename their own feed's address form, not another user's; cross-feed impact is none, but the rule is worth knowing. +- **state.json holds no secrets** (owner keys are public, counters and ledgers are operational), so it is written world-readable-safe plaintext; the only secret on disk is the daemon master key, 0600 by `keygen`, whose protection is the operator's. + +## Not covered by this audit + +- fumi-core and bunshin themselves; they are separate projects with their own test suites, and smolboid depends on their correctness (envelope verification, Noise pin enforcement, tier separation). +- No `cargo audit`-style dependency vulnerability scan was run; the dependency tree is small (fumi-core, chacha20poly1305, serde, serde_json) but this check is recommended before any public deployment. +- No fuzzing of the feed parser or HTTP request parser; both are small and hand-tested against hostile inputs, but fuzz targets would raise confidence. +- Timing side channels are untested; the crypto layer is all constant-time primitives from fumi-core, but no analysis was performed at this layer. + +## Reproducing + +``` +devbox run test # unit, format and vector tests +devbox run vectors # regenerate and pin core/vectors.json +devbox run rig # end-to-end against a local bunshin +devbox run stress # flood, quota, reply budget, HTTP load +devbox run pentest # adversarial probes, raw sockets and real mail +``` + +All suites are self-contained: each starts its own bunshin instance, fumi identities and daemon on local ports and cleans up after itself. diff --git a/tests/pentest.sh b/tests/pentest.sh new file mode 100755 index 0000000..84c8719 --- /dev/null +++ b/tests/pentest.sh @@ -0,0 +1,239 @@ +#!/usr/bin/env bash +# smolboid pentest: an adversarial pass against our own daemon. Every probe +# asserts the hardened outcome; a probe that succeeds as an attack fails the +# suite. Raw sockets via python3 for the HTTP surface, real fumi sends for +# the mail surface. +set -euo pipefail + +ROOT="$(cd "$(dirname "$0")/.." && pwd)" +FUMI="$ROOT/../fumi/target/debug/fumi" +BUNSHIN_BIN="$ROOT/../bunshin/target/debug/bunshin" +SMOLBOID="$ROOT/target/debug/smolboid" +HOST=127.0.0.1 +SMTP_PORT=19631 +HTTP_PORT=18101 +RIG="$(mktemp -d /tmp/smolboid-pentest.XXXXXX)" +DATA="$RIG/data" +DAEMON_ADDR="twtxt@$HOST:$SMTP_PORT" +FEED="$DATA/feeds/tobias.txt" +BUNSHIN_PID="" +SMOLBOID_PID="" +cleanup() { + [[ -n "$BUNSHIN_PID" ]] && kill "$BUNSHIN_PID" 2>/dev/null || true + [[ -n "$SMOLBOID_PID" ]] && kill "$SMOLBOID_PID" 2>/dev/null || true +} +trap cleanup EXIT + +step() { printf '\n== %s ==\n' "$*"; } +fail() { echo "FAIL: $*" >&2; exit 1; } +probe() { # name, expected-status-prefix, raw request bytes (python literal) + local name=$1 want=$2 payload=$3 + local got + got=$(python3 - "$HTTP_PORT" "$payload" <<'EOF' +import socket, sys +port, payload = int(sys.argv[1]), sys.argv[2].encode().decode("unicode_escape") +s = socket.create_connection(("127.0.0.1", port), timeout=15) +s.sendall(payload.encode()) +s.settimeout(15) +# Accumulate until a full status line lands; a single recv may split. +data = b"" +while b"\r\n" not in data: + try: + chunk = s.recv(4096) + except socket.timeout: + break + if not chunk: + break + data += chunk +if not data: + data = b"" +print(data.split(b"\r\n")[0].decode(errors="replace")) +EOF +) + if [[ "$got" == "$want"* ]]; then + echo "ok $name -> $got" + else + echo "got: $got" + fail "probe $name: wanted $want*, got $got" + fi +} + +step "build" +cargo build --manifest-path "$ROOT/../bunshin/Cargo.toml" --quiet +cargo build --manifest-path "$ROOT/../fumi/Cargo.toml" --quiet +cargo build --manifest-path "$ROOT/Cargo.toml" --quiet + +step "bootstrap: bunshin, tobias and a second identity mallory" +mkdir -p "$DATA" +"$BUNSHIN_BIN" keygen --key "$RIG/server.key" >"$RIG/keygen.log" 2>&1 +SERVER_KEY="$(grep -oE '[a-z2-7]{52}' "$RIG/keygen.log" | head -1)" +"$BUNSHIN_BIN" serve --key "$RIG/server.key" --db "$RIG/server.db" \ + --host "$HOST" --port "$SMTP_PORT" >"$RIG/bunshin.log" 2>&1 & +BUNSHIN_PID=$! +sleep 1 +author() { "$FUMI" --key "$RIG/author.key" --db "$RIG/author.db" --timeout 10 "$@"; } +mallory() { "$FUMI" --key "$RIG/mallory.key" --db "$RIG/mallory.db" --timeout 10 "$@"; } +daemon_fumi() { "$FUMI" --key "$RIG/master.key" --db "$DATA/mail.db" --timeout 10 "$@"; } +author keygen --force >/dev/null +author trust "$HOST:$SMTP_PORT" "$SERVER_KEY" +author register "tobias@$HOST:$SMTP_PORT" >/dev/null +"$SMOLBOID" keygen "$RIG/master.key" >/dev/null +daemon_fumi trust "$HOST:$SMTP_PORT" "$SERVER_KEY" +daemon_fumi register "$DAEMON_ADDR" >/dev/null + +step "admit tobias (never mallory)" +cat >"$RIG/smolboid.toml" <"$RIG/admit.log" 2>&1 \ + || { cat "$RIG/admit.log"; fail "admit"; } +author fetch >/dev/null +"$SMOLBOID" serve --config "$RIG/smolboid.toml" >"$RIG/serve.log" 2>&1 & +SMOLBOID_PID=$! +sleep 2 +# A legitimate post so the feed has content to protect. +author send "$DAEMON_ADDR" --body "real post" >/dev/null +for ((i = 0; i < 20; i++)); do + grep -q "real post" "$FEED" 2>/dev/null && break + sleep 1 +done +grep -q "real post" "$FEED" || fail "setup: the real post never published" + +step "http: path traversal and path abuse" +probe "traversal-etc-passwd" "HTTP/1.1 404" 'GET /twtxt/../../etc/passwd HTTP/1.1\r\n\r\n' +probe "encoded-traversal" "HTTP/1.1 404" 'GET /twtxt/..%2F..%2Fetc%2Fpasswd HTTP/1.1\r\n\r\n' +probe "dotdot-no-suffix" "HTTP/1.1 404" 'GET /twtxt/.. HTTP/1.1\r\n\r\n' +probe "absolute-path" "HTTP/1.1 404" 'GET //etc/passwd HTTP/1.1\r\n\r\n' +probe "uppercase-username" "HTTP/1.1 404" 'GET /twtxt/TOBIAS.txt HTTP/1.1\r\n\r\n' +probe "unicode-username" "HTTP/1.1 404" 'GET /twtxt/t\xc3\xb6bias.txt HTTP/1.1\r\n\r\n' +probe "oversized-username" "HTTP/1.1 404" "GET /twtxt/$(printf 'a%.0s' $(seq 1 200)).txt HTTP/1.1\r\n\r\n" +probe "null-byte-username" "HTTP/1.1 404" 'GET /twtxt/tob\x00ias.txt HTTP/1.1\r\n\r\n' +probe "dotfile-username" "HTTP/1.1 404" 'GET /twtxt/.ssh.txt HTTP/1.1\r\n\r\n' +# The traversal probes must never return file content. +curl -s "http://$HOST:$HTTP_PORT/twtxt/../../etc/passwd" | grep -q "root:" \ + && fail "traversal returned /etc/passwd" + +step "http: request parsing abuse" +# Any 4xx is a hardened outcome; the server may read it as a method. +probe "garbage-request" "HTTP/1.1 40" 'not http at all\r\n\r\n' +probe "smuggled-second-request" "HTTP/1.1 200" 'GET /twtxt/tobias.txt HTTP/1.1\r\nX-Injected: 1\r\n\r\nGET /twtxt/tobias' +probe "null-in-request" "HTTP/1.1 40" 'GE\x00T /twtxt/tobias.txt HTTP/1.1\r\n\r\n' +probe "post-method" "HTTP/1.1 405" 'POST /twtxt/tobias.txt HTTP/1.1\r\nContent-Length: 0\r\n\r\n' +probe "huge-request-line" "HTTP/1.1 40" "GET /$(printf 'a%.0s' $(seq 1 9000)) HTTP/1.1\r\n\r\n" +probe "extra-headers-tolerated" "HTTP/1.1 200" 'GET /twtxt/tobias.txt HTTP/1.1\r\nHost: x\r\nEvil: 1\r\n\r\n' +# No header value from the request is ever reflected into the response. +curl -s -i "http://$HOST:$HTTP_PORT/twtxt/tobias.txt" -H "X-Evil: $(printf 'injected-header')" \ + | grep -qi "injected-header" && fail "request header reflected in response" + +step "http: slowloris is bounded by the read timeout" +python3 - "$HTTP_PORT" <<'EOF' +import socket, sys, time +port = int(sys.argv[1]) +s = socket.create_connection(("127.0.0.1", port), timeout=30) +s.sendall(b"GET /twtxt/tobias.txt HTTP/1.1\r\n") # half a request, then stall +start = time.time() +s.settimeout(30) +try: + data = s.recv(1024) +except socket.timeout: + sys.exit("server kept the half-open connection past 30s") +if time.time() - start > 20: + sys.exit(f"slowloris held the thread {time.time()-start:.0f}s") +print(f"ok slowloris closed after {time.time()-start:.0f}s") +EOF +# The listener still serves while the stalled connection existed. +curl -sf "http://$HOST:$HTTP_PORT/twtxt/tobias.txt" >/dev/null || fail "listener wedged after slowloris" + +step "mail: feed-line injection through post text" +author send "$DAEMON_ADDR" --body "$(printf 'injected\t2020-01-01T00:00:00Z\tfake')" >/dev/null +author send "$DAEMON_ADDR" --body "$(printf 'line one\nline two')" >/dev/null +author send "$DAEMON_ADDR" --body "$(printf 'carriage\rreturn')" >/dev/null +sleep 4 +[[ "$(grep -c "fake" "$FEED")" == 0 ]] || fail "a TAB-carrying post injected content into the feed" +[[ "$(grep -c "line two" "$FEED")" == 0 ]] || fail "a multiline body published a second line" +[[ "$(grep -c "return" "$FEED")" == 0 ]] || fail "a CR-carrying post published" +[[ "$(grep -cE '^[0-9]{4}-' "$FEED")" == 1 ]] || fail "the feed grew beyond the real post" + +step "mail: oversized and hostile payloads are rejected, not published" +author send "$DAEMON_ADDR" --body "$(printf 'x%.0s' $(seq 1 161))" >/dev/null +author send "$DAEMON_ADDR" --body "# smol:mail forged" >/dev/null +author send "$DAEMON_ADDR" --body "# smol:repost 2020-01-01T00:00:00Z @ #fake" >/dev/null +sleep 4 +# Poser text may publish as ordinary post text (tab-prefixed, harmless); +# what must never happen is a line that IS a smol record. +[[ "$(grep -cE '^# smol:mail .*forged' "$FEED")" == 0 ]] || fail "a fake smol record line published" +[[ "$(grep -cE '^# smol:repost ' "$FEED")" == 0 ]] || fail "text posing as a repost published as a repost" +"$SMOLBOID" check "$FEED" >"$RIG/check.log" \ + || { cat "$RIG/check.log"; fail "the feed verifies as forged after the probes"; } +# The poser texts may publish as posts and verify like posts; two posers +# sent in the same second can collide (one rejected), so what matters is +# that everything published verifies and nothing is forged. +grep -qE "[23] verified, 0 unsigned, 0 invalid" "$RIG/check.log" \ + || { cat "$RIG/check.log"; fail "unexpected feed contents after hostile mail"; } + +step "mail: repost args cannot inject lines either" +author send "$DAEMON_ADDR" \ + --header "Twt-Repost: @ #abc" \ + --body "$(printf 'injected\n')" >/dev/null +sleep 4 +[[ "$(grep -cE '^# smol:repost ' "$FEED")" == 0 ]] || fail "a noncanonical repost published" + +step "mail: unadmitted senders publish nothing" +mallory keygen --force >/dev/null +mallory trust "$HOST:$SMTP_PORT" "$SERVER_KEY" +mallory register "mallory@$HOST:$SMTP_PORT" >/dev/null +mallory send "$DAEMON_ADDR" --body "mallory was here" >/dev/null +sleep 4 +[[ "$(grep -c "mallory was here" "$FEED")" == 0 ]] || fail "unadmitted mail published" +[[ ! -f "$DATA/feeds/mallory.txt" ]] || fail "an unadmitted sender got a feed" + +step "mail: forged signing keys verify as invalid, never as verified" +# Tamper the published feed's own signature line: the record must flip to +# invalid, proving verification is load-bearing. +cp "$FEED" "$RIG/tampered.txt" +python3 - "$RIG/tampered.txt" <<'EOF' +import sys +path = sys.argv[1] +lines = open(path).read().split("\n") +for i, line in enumerate(lines): + if line.startswith("# smol:mail "): + parts = line.split(" ") + parts[3] = ("a" if parts[3][0] != "a" else "b") + parts[3][1:] + lines[i] = " ".join(parts) + break +open(path, "w").write("\n".join(lines)) +EOF +if "$SMOLBOID" check "$RIG/tampered.txt" >/dev/null 2>&1; then + fail "a tampered signature still verified" +fi + +step "mail: reply-flood is capped at 10 per hour" +for i in $(seq 1 15); do + author send "$DAEMON_ADDR" --body "$(printf 'y%.0s' $(seq 1 $((200 + i))))" >/dev/null +done +sleep 5 +author fetch >/dev/null +# Fifteen distinct oversized payloads; at most ten answers, the rest silent. +# (Earlier probes may have drawn their own replies for other reasons.) +author list --requests 2>/dev/null | grep -c "twtxt: more than 160" >"$RIG/replies.count" || true +REPLIES=$(grep -oE '^[0-9]+' "$RIG/replies.count" | head -1 || echo 0) +echo "flood answers: $REPLIES of 15" +[[ "$REPLIES" -le 10 ]] || fail "reply flood drew $REPLIES answers, over the hourly cap" + +step "http: still healthy after everything" +curl -sf "http://$HOST:$HTTP_PORT/twtxt/tobias.txt" >/dev/null || fail "listener unhealthy at the end" +"$SMOLBOID" check "$FEED" >/dev/null || fail "feed does not verify at the end" + +step "pentest passed: every probe was repelled" +cleanup +trap - EXIT diff --git a/tests/rig.sh b/tests/rig.sh new file mode 100755 index 0000000..b98d804 --- /dev/null +++ b/tests/rig.sh @@ -0,0 +1,194 @@ +#!/usr/bin/env bash +# smolboid integration rig (PLAN.md step 7): a local bunshin, the fumi CLI as +# the author, and the daemon under test — covering post, command, repost, +# timestamp collision, size-cap warning, both prune modes and an author +# rotation. Real binaries, real Noise sessions, real envelopes. +set -euo pipefail + +ROOT="$(cd "$(dirname "$0")/.." && pwd)" +FUMI_DIR="$ROOT/../fumi" +BUNSHIN_DIR="$ROOT/../bunshin" +RIG="$(mktemp -d /tmp/smolboid-rig.XXXXXX)" +HOST=127.0.0.1 +SMTP_PORT=19611 +HTTP_PORT=18081 +DAEMON_ADDR="twtxt@$HOST:$SMTP_PORT" +AUTHOR_ADDR="tobias@$HOST:$SMTP_PORT" +DATA="$RIG/data" + +BUNSHIN_PID="" +SMOLBOID_PID="" +cleanup() { + [[ -n "$BUNSHIN_PID" ]] && kill "$BUNSHIN_PID" 2>/dev/null || true + [[ -n "$SMOLBOID_PID" ]] && kill "$SMOLBOID_PID" 2>/dev/null || true +} +trap cleanup EXIT + +step() { printf '\n== %s ==\n' "$*"; } +fail() { echo "FAIL: $*" >&2; exit 1; } + +wait_for() { # file, text, [tries] + local file=$1 text=$2 tries=${3:-20} i + for ((i = 0; i < tries; i++)); do + [[ -f $file ]] && grep -q "$text" "$file" && return 0 + sleep 1 + done + fail "timed out waiting for '$text' in $file" +} + +wait_gone() { # file, text, [tries] + local file=$1 text=$2 tries=${3:-20} i + for ((i = 0; i < tries; i++)); do + grep -q "$text" "$file" || return 0 + sleep 1 + done + fail "'$text' is still in $file" +} + +step "build bunshin, fumi, smolboid" +cargo build --manifest-path "$BUNSHIN_DIR/Cargo.toml" --quiet +cargo build --manifest-path "$FUMI_DIR/Cargo.toml" --quiet +cargo build --manifest-path "$ROOT/Cargo.toml" --quiet +BUNSHIN="$BUNSHIN_DIR/target/debug/bunshin" +FUMI="$FUMI_DIR/target/debug/fumi" +SMOLBOID="$ROOT/target/debug/smolboid" + +step "bunshin on $HOST:$SMTP_PORT" +"$BUNSHIN" keygen --key "$RIG/server.key" >"$RIG/keygen.log" 2>&1 +SERVER_KEY="$(grep -oE '[a-z2-7]{52}' "$RIG/keygen.log" | head -1)" +[[ -n "$SERVER_KEY" ]] || { cat "$RIG/keygen.log"; fail "no server key"; } +"$BUNSHIN" serve --key "$RIG/server.key" --db "$RIG/server.db" \ + --host "$HOST" --port "$SMTP_PORT" >"$RIG/bunshin.log" 2>&1 & +BUNSHIN_PID=$! +sleep 1 + +step "identities: author tobias, daemon twtxt" +mkdir -p "$DATA" +fumi() { "$FUMI" --key "$RIG/author.key" --db "$RIG/author.db" --timeout 10 "$@"; } +dfumi() { "$FUMI" --key "$RIG/master.key" --db "$DATA/mail.db" --timeout 10 "$@"; } +fumi keygen --force >/dev/null +fumi trust "$HOST:$SMTP_PORT" "$SERVER_KEY" +fumi register "$AUTHOR_ADDR" >/dev/null +"$SMOLBOID" keygen "$RIG/master.key" >/dev/null +dfumi trust "$HOST:$SMTP_PORT" "$SERVER_KEY" +dfumi register "$DAEMON_ADDR" >/dev/null + +step "admit tobias" +cat >"$RIG/smolboid.toml" <"$RIG/admit.log" 2>&1 \ + || { cat "$RIG/admit.log"; fail "admit"; } +grep -q "welcome mail sent" "$RIG/admit.log" \ + || { cat "$RIG/admit.log"; fail "no welcome mail"; } +# The Accept token's proof is the next step: without it, posts stay in the +# requests tier and nothing ever publishes. +fumi fetch >/dev/null +FEED="$DATA/feeds/tobias.txt" + +step "serve" +"$SMOLBOID" serve --config "$RIG/smolboid.toml" >"$RIG/serve.log" 2>&1 & +SMOLBOID_PID=$! +sleep 2 + +step "post" +fumi send "$DAEMON_ADDR" --body "hello smolverse" >/dev/null +wait_for "$FEED" "hello smolverse" +"$SMOLBOID" check "$FEED" >"$RIG/check.log" || { cat "$RIG/check.log"; fail "check"; } +grep -q "1 verified, 0 unsigned, 0 invalid" "$RIG/check.log" \ + || { cat "$RIG/check.log"; fail "the first post did not verify"; } +curl -sf "http://$HOST:$HTTP_PORT/twtxt/tobias.txt" | grep -q "hello smolverse" \ + || fail "the feed is not served over HTTP" +[[ "$(curl -s -o /dev/null -w '%{http_code}' "http://$HOST:$HTTP_PORT/twtxt/nobody.txt")" == 404 ]] \ + || fail "a missing feed must 404" + +step "commands: nick, repost" +fumi send "$DAEMON_ADDR" --header "Twt-Nick: toby" --body "" >/dev/null +wait_for "$FEED" "# nick = toby" +fumi send "$DAEMON_ADDR" \ + --header "Twt-Repost: @ #abc1234" \ + --body "" >/dev/null +wait_for "$FEED" "# smol:repost .* #abc1234" +"$SMOLBOID" check "$FEED" >"$RIG/check.log" || { cat "$RIG/check.log"; fail "check"; } +grep -q "2 verified, 0 unsigned, 0 invalid" "$RIG/check.log" \ + || { cat "$RIG/check.log"; fail "the repost did not verify"; } + +step "delete" +TS="$(grep "hello smolverse" "$FEED" | cut -f1)" +fumi send "$DAEMON_ADDR" --header "Twt-Delete: $TS" --body "" >/dev/null +wait_gone "$FEED" "hello smolverse" + +step "timestamp collision" +collided=0 +for i in 1 2 3 4 5; do + fumi send "$DAEMON_ADDR" --body "collide A $i" >/dev/null & + A_PID=$! + fumi send "$DAEMON_ADDR" --body "collide B $i" >/dev/null & + B_PID=$! + # Only the two sends, never the servers. + wait "$A_PID" "$B_PID" + sleep 3 + if grep -q "collide A $i" "$FEED" && grep -q "collide B $i" "$FEED"; then + continue # different seconds; try again + fi + fumi fetch >/dev/null + if fumi list --requests 2>/dev/null | grep -q "timestamp .* already exists"; then + collided=1 + break + fi +done +[[ "$collided" == 1 ]] || fail "no collision rejection arrived after 5 attempts" + +step "retention: hard limit" +kill "$SMOLBOID_PID" 2>/dev/null || true +# Metadata (~190B) plus one record (~253B) must fit under the cap, and land +# above its 90% mark so the warning fires. +sed -i 's/^cap_bytes = .*/cap_bytes = 460/' "$RIG/smolboid.toml" +"$SMOLBOID" serve --config "$RIG/smolboid.toml" >>"$RIG/serve.log" 2>&1 & +SMOLBOID_PID=$! +sleep 2 +fumi send "$DAEMON_ADDR" --body "over the cap" >/dev/null +sleep 3 +grep -q "over the cap" "$FEED" && fail "a capped feed published a record" +fumi fetch >/dev/null +fumi list --requests 2>/dev/null | grep -q "feed is at its size cap" \ + || fail "no cap rejection reply" + +step "retention: fifo mode" +fumi send "$DAEMON_ADDR" --header "Twt-Prune: fifo" --body "" >/dev/null +wait_for "$DATA/state.json" '"prune": "Fifo"' +fumi send "$DAEMON_ADDR" --body "fifo post" >/dev/null +wait_for "$FEED" "fifo post" +[[ "$(grep -cE '^[0-9]' "$FEED")" == 1 ]] || fail "fifo did not prune down to one record" +fumi fetch >/dev/null +fumi list --requests 2>/dev/null | grep -q "90% of its size cap" \ + || fail "no 90% warning reply" +"$SMOLBOID" check "$FEED" >"$RIG/check.log" || { cat "$RIG/check.log"; fail "check"; } +grep -q "1 verified, 0 unsigned, 0 invalid" "$RIG/check.log" \ + || { cat "$RIG/check.log"; fail "the pruned feed does not verify"; } + +step "author rotation" +fumi rotate >/dev/null +fumi send "$DAEMON_ADDR" --body "after rotation" >/dev/null +wait_for "$FEED" "after rotation" +OWNERS="$(python3 -c "import json;print(len(json.load(open('$DATA/state.json'))['feeds'][0]['owner_keys']))")" +[[ "$OWNERS" == 2 ]] || fail "expected 2 owner keys after rotation, got $OWNERS" +"$SMOLBOID" check "$FEED" >"$RIG/check.log" || { cat "$RIG/check.log"; fail "check"; } +# Under the fifo cap only one record fits, so the rotation post pruned the +# fifo post; what remains must fully verify. +grep -q "1 verified, 0 unsigned, 0 invalid" "$RIG/check.log" \ + || { cat "$RIG/check.log"; fail "rotation-era records do not verify"; } + +step "rig passed" +echo "feeds: $FEED" +echo "logs: $RIG" +cleanup +trap - EXIT diff --git a/tests/stress.sh b/tests/stress.sh new file mode 100755 index 0000000..2d5e751 --- /dev/null +++ b/tests/stress.sh @@ -0,0 +1,177 @@ +#!/usr/bin/env bash +# smolboid stress test: a mail flood against the daily quota and reply +# budget, an HTTP load burst against a full feed, and feed-integrity checks +# throughout. Real binaries, real Noise sessions. +set -euo pipefail + +ROOT="$(cd "$(dirname "$0")/.." && pwd)" +FUMI="$ROOT/../fumi/target/debug/fumi" +BUNSHIN_BIN="$ROOT/../bunshin/target/debug/bunshin" +SMOLBOID="$ROOT/target/debug/smolboid" +HOST=127.0.0.1 +SMTP_PORT=19621 +HTTP_PORT=18091 +RIG="$(mktemp -d /tmp/smolboid-stress.XXXXXX)" +DATA="$RIG/data" +DAEMON_ADDR="twtxt@$HOST:$SMTP_PORT" +FEED="$DATA/feeds/tobias.txt" +BUNSHIN_PID="" +SMOLBOID_PID="" +cleanup() { + [[ -n "$BUNSHIN_PID" ]] && kill "$BUNSHIN_PID" 2>/dev/null || true + [[ -n "$SMOLBOID_PID" ]] && kill "$SMOLBOID_PID" 2>/dev/null || true +} +trap cleanup EXIT + +step() { printf '\n== %s ==\n' "$*"; } +fail() { echo "FAIL: $*" >&2; exit 1; } +wait_for() { + local file=$1 text=$2 tries=${3:-40} i + for ((i = 0; i < tries; i++)); do + [[ -f $file ]] && grep -q "$text" "$file" && return 0 + sleep 1 + done + fail "timed out waiting for '$text' in $file" +} +count_posts() { grep -cE '^[0-9]' "$FEED" 2>/dev/null || echo 0; } + +step "build" +cargo build --manifest-path "$ROOT/../bunshin/Cargo.toml" --quiet +cargo build --manifest-path "$ROOT/../fumi/Cargo.toml" --quiet +cargo build --manifest-path "$ROOT/Cargo.toml" --quiet + +step "bootstrap: bunshin, author tobias, daemon twtxt" +mkdir -p "$DATA" +"$BUNSHIN_BIN" keygen --key "$RIG/server.key" >"$RIG/keygen.log" 2>&1 +SERVER_KEY="$(grep -oE '[a-z2-7]{52}' "$RIG/keygen.log" | head -1)" +[[ -n "$SERVER_KEY" ]] || fail "no server key" +# Raised server rate limits: the flood stresses smolboid's quota, not +# bunshin's per-IP throttles — everything here shares 127.0.0.1, and the +# daemon's one-second polling already AUTHs 60 times a minute. +"$BUNSHIN_BIN" serve --key "$RIG/server.key" --db "$RIG/server.db" \ + --host "$HOST" --port "$SMTP_PORT" \ + --rate-connections 1000000 --rate-sends 1000000 --rate-tokens 1000000 \ + >"$RIG/bunshin.log" 2>&1 & +BUNSHIN_PID=$! +sleep 1 +author() { "$FUMI" --key "$RIG/author.key" --db "$RIG/author.db" --timeout 10 "$@"; } +daemon_fumi() { "$FUMI" --key "$RIG/master.key" --db "$DATA/mail.db" --timeout 10 "$@"; } +author keygen --force >/dev/null +author trust "$HOST:$SMTP_PORT" "$SERVER_KEY" +author register "tobias@$HOST:$SMTP_PORT" >/dev/null +"$SMOLBOID" keygen "$RIG/master.key" >/dev/null +daemon_fumi trust "$HOST:$SMTP_PORT" "$SERVER_KEY" +daemon_fumi register "$DAEMON_ADDR" >/dev/null + +step "admit" +cat >"$RIG/smolboid.toml" <"$RIG/admit.log" 2>&1 \ + || { cat "$RIG/admit.log"; fail "admit"; } +author fetch >/dev/null + +step "serve" +"$SMOLBOID" serve --config "$RIG/smolboid.toml" >"$RIG/serve.log" 2>&1 & +SMOLBOID_PID=$! +sleep 2 + +step "mail burst: 40 unpaced posts in a few seconds" +START=$(date +%s) +for i in $(seq 1 40); do + author send "$DAEMON_ADDR" --body "flood post $i" >/dev/null +done +echo "40 sends in $(( $(date +%s) - START ))s" +# Rapid fire means many payload seconds collide; what must hold is +# integrity: nothing invalid, the quota never exceeded, the daemon alive. +for ((i = 0; i < 40; i++)); do + sleep 1 + [[ "$(count_posts)" -ge 1 ]] && break +done +[[ "$(count_posts)" -le 25 ]] || fail "quota exceeded under burst: $(count_posts) records" +"$SMOLBOID" check "$FEED" >"$RIG/check.log" \ + || { cat "$RIG/check.log"; fail "the burst left the feed unparseable"; } +grep -qE "0 unsigned, 0 invalid" "$RIG/check.log" \ + || { cat "$RIG/check.log"; fail "the burst published invalid records"; } +curl -sf "http://$HOST:$HTTP_PORT/twtxt/tobias.txt" >/dev/null || fail "listener unhealthy after the burst" + +step "quota precision: paced posts fill exactly to 25" +# One send per second keeps timestamps distinct, so only the daily quota +# can stop publication: the feed lands on exactly the configured cap. +for i in $(seq 41 70); do + author send "$DAEMON_ADDR" --body "flood post $i" >/dev/null + sleep 1 + [[ "$(count_posts)" == 25 ]] && break +done +for ((i = 0; i < 20; i++)); do + [[ "$(count_posts)" == 25 ]] && break + sleep 1 +done +[[ "$(count_posts)" == 25 ]] || fail "quota held at $(count_posts) records, want 25" +"$SMOLBOID" check "$FEED" >"$RIG/check.log" \ + || { cat "$RIG/check.log"; fail "the quota-cut feed does not verify"; } +grep -q "25 verified, 0 unsigned, 0 invalid" "$RIG/check.log" \ + || { cat "$RIG/check.log"; fail "not all 25 verify"; } +author fetch >/dev/null + +step "reply budget: the flood drew at most 10 replies" +# The welcome mail also carries a twtxt: subject; only rejections count. +author list --requests 2>/dev/null | grep "twtxt:" | grep -cv "feed is ready" \ + >"$RIG/replies.count" || true +REPLIES=$(grep -oE '^[0-9]+' "$RIG/replies.count" | head -1) +echo "error replies after the flood: $REPLIES" +[[ "$REPLIES" -le 10 ]] || fail "reply budget exceeded: $REPLIES replies in an hour" + +step "http load: 200 requests, 20 parallel" +URL="http://$HOST:$HTTP_PORT/twtxt/tobias.txt" +ETAG="$(curl -s -D - -o /dev/null "$URL" | tr -d '\r' | awk -F'"' '/^ETag:/ {print $2}')" +[[ -n "$ETAG" ]] || fail "no etag" +START=$(date +%s%N) +seq 1 200 | xargs -P 20 -I{} curl -sf -o /dev/null "$URL" || fail "a load request failed" +END=$(date +%s%N) +MS=$(((END - START) / 1000000)) +echo "200 requests in ${MS}ms ($((200000 / MS)) req/s lower bound)" +AFTER="$(curl -s -D - -o /dev/null "$URL" | tr -d '\r' | awk -F'"' '/^ETag:/ {print $2}')" +[[ "$AFTER" == "$ETAG" ]] || fail "the etag moved under load without a write" + +step "conditional load: 200 requests, 20 parallel, all 304" +START=$(date +%s%N) +seq 1 200 | xargs -P 20 -I{} \ + curl -sf -o /dev/null -w '%{http_code}\n' -H "If-None-Match: \"$ETAG\"" "$URL" \ + | sort | uniq -c >"$RIG/codes.log" +END=$(date +%s%N) +echo "200 conditional requests in $(((END - START) / 1000000))ms" +grep -q "200 304" "$RIG/codes.log" || { cat "$RIG/codes.log"; fail "not all conditional requests were 304"; } + +step "read-during-write: the feed stays parseable while the mail loop runs" +author send "$DAEMON_ADDR" --header "Twt-Delete: $(grep -E '^[0-9]' "$FEED" | tail -1 | cut -f1)" --body "" >/dev/null +BROKEN=0 +for i in $(seq 1 60); do + curl -sf "$URL" >/dev/null || BROKEN=$((BROKEN + 1)) + # One per-loop read while the delete lands between polls. + sleep 0.2 +done +[[ "$BROKEN" == 0 ]] || fail "$BROKEN reads failed while the feed changed" +for ((i = 0; i < 20; i++)); do + [[ "$(count_posts)" == 24 ]] && break + sleep 1 +done +[[ "$(count_posts)" == 24 ]] || fail "delete did not land" +"$SMOLBOID" check "$FEED" >/dev/null || fail "final feed does not verify" +[[ "$(count_posts)" == 24 ]] || fail "expected 24 records after the delete, got $(count_posts)" + +step "stress passed" +echo "feed: $FEED" +echo "logs: $RIG" +cleanup +trap - EXIT