randogoth/itsybitsy
1
0
Fork 0
Code Issues Pull requests Projects Releases 3 Packages Wiki Activity Actions

feat: serve gemini behind a tls terminator

Browse source
This commit is contained in:
randogoth 2026-10-04 22:04:46 +03:00
parent dc2114a4d2
commit c29364d8e0
8 changed files with 395 additions and 39 deletions
Download patch file Download diff file Expand all files Collapse all files

39
README.md
View file

@ -2,13 +2,13 @@
[![License: Apache-2.0](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE) [![AI-DECLARATION: copilot](https://img.shields.io/badge/%E4%B7%BC%20AI--DECLARATION-copilot-fee2e2?labelColor=fee2e2)](https://ai-declaration.md)
Serve folders of Markdown to different domains over HTTP, [Spartan](https://portal.mozz.us/gemini/spartan.mozz.us/), [Nex](https://nightfall.city/nex/info/specification.txt) and [Gopher](https://www.rfc-editor.org/rfc/rfc1436), rendered on request with no build step.
Serve folders of Markdown to different domains over HTTP, [Gemini](https://geminiprotocol.net/docs/specification.gmi), [Spartan](https://portal.mozz.us/gemini/spartan.mozz.us/), [Nex](https://nightfall.city/nex/info/specification.txt) and [Gopher](https://www.rfc-editor.org/rfc/rfc1436), rendered on request with no build step.
A Rust re-implementation of [smolweb](https://code.randogoth.com/randogoth/smolweb), which serves one folder from one process with its output formats hardcoded and its configuration in per-file frontmatter. itsybitsy changes all three: one process serves many domains, each output format is a separate crate behind a cargo feature, and configuration lives in a TOML file per content directory.
## Status
It serves. HTTP, Spartan, Nex and Gopher are working across five output formats, with virtual hosting wherever the protocol carries a hostname. Gemini is the one protocol left.
It serves. All five protocols work across five output formats, with virtual hosting wherever the protocol carries a hostname. Gemini expects TLS to be terminated in front of it, which is the one deployment requirement itsybitsy does not satisfy by itself.
| Area | State |
| --- | --- |
@ -21,7 +21,8 @@ It serves. HTTP, Spartan, Nex and Gopher are working across five output formats,
| FIGlet banners and hyphenation, behind features | implemented |
| WML decks and card sub-documents, behind a feature | implemented |
| Gopher, with dot-stuffing and the RFC 4266 root menu | implemented |
| Gemini | planned |
| Gemini, behind a TLS terminator | implemented |
| Built-in TLS, so no terminator is needed | not planned for now |
## Configuration
@ -108,6 +109,8 @@ Configuration keys are not carried over verbatim either: md2txt accepts three sp
wapdown's own parser produces neither tables nor lists, so its WML decks render a bullet list as literal text and drop tables entirely; both come out as real markup here, lists as marked lines and tables as `<table>`, which WML has.
Gemini diverges in three places. smolweb answers 59 (bad request) to any URL that is not `gemini://`, which tells a client its request was malformed when an `https://` URL is merely for somewhere this server does not fetch from; that is 53 (proxy request refused) here, and 59 is kept for a URL that genuinely does not parse. smolweb builds absolute redirect targets from the port it is bound to, which is wrong behind a terminator; these are relative. And smolweb terminates TLS itself with a certificate named per site, so `tls` is not a configuration key here at all — naming one is an error rather than a setting that quietly does nothing.
smolweb inherits five invented directives from its two renderer libraries: `{.card}`, `{.include}`, `![[ ]]`, `#[label](art.txt)` and MultiMarkdown attribute lists (`{: .center}`) — two incompatible brace grammars, with art alignment expressible two different ways in one line. None of it appeared in real content, so itsybitsy expresses the same capabilities with standard constructs instead. Parsing once rather than once per library also means the text formats gain setext headings and the WAP formats gain tables and ASCII art, none of which their original parser handled.
## Output formats
@ -150,12 +153,40 @@ A banner that will not fit the line, names a font this build lacks, or is asked
| HTTP | `Host` header | `GET` and `HEAD`, negotiated by `Accept` |
| Spartan | the host field of its request line | One format per listener |
| Nex | nothing; its listener names one site | No status line at all |
| Gemini | the authority of the URL it is sent | TLS terminated in front; one format per listener |
| Gopher | nothing; its listener names one site | Item type 0 and the root menu |
A protocol with no hostname in its requests cannot be routed by one, so its listener names a single site outright and configuration refuses to leave that implicit once more than one site exists. HTTP and Spartan fall back to `default_site` when a request names a host this server does not know.
A protocol with no hostname in its requests cannot be routed by one, so its listener names a single site outright and configuration refuses to leave that implicit once more than one site exists. HTTP, Gemini and Spartan fall back to `default_site` when a request names a host this server does not know.
Neither Nex nor Gopher has a redirect status, so a canonical target is resolved server-side and the real content comes back on the first request rather than a bounce.
### Gemini and TLS
itsybitsy does not speak TLS. The Gemini listener expects plaintext, so a TLS wrapper terminates in front of it and forwards to a loopback port:
```toml
[listener.gemini]
protocol = "gemini"
# The terminator owns 1965; this is its back end.
bind = "127.0.0.1:11965"
formats = ["gemtext"]
default_site = "smol"
```
```ini
; /etc/stunnel/gemini.conf
[gemini]
accept = 1965
connect = 127.0.0.1:11965
cert = /var/lib/itsybitsy/gemini.pem
```
`stunnel` and `ghostunnel` are built for exactly this and support SNI, so each virtual host can present its own certificate. Use a long-lived self-signed certificate rather than an ACME one: Gemini clients pin the fingerprint they first saw, and a renewal that changes it is indistinguishable from an interception.
Two things follow from terminating in front, and neither is a limitation that can be configured away. Every connection appears to come from the terminator, so request logs show its address unless it speaks the PROXY protocol, which itsybitsy does not yet read. And a client certificate never reaches us, so statuses 60 to 62 are not implementable — itsybitsy serves static documents and has nothing to authenticate, so only the logs are poorer for it.
Redirects are relative references, which the spec permits and which are the only correct form here: the port this listener is bound to is the terminator's back end, not a port any client reached, so an absolute URL built from it would point somewhere unpublished.
Gopher serves item type 0 (text) and one menu: a prefix-less `gopher://host/` means item type 1 by RFC 4266, so an empty selector gets a one-item menu pointing at `/` rather than the root document, which would be the wrong type. Text items are dot-stuffed and terminated with a lone dot; binary items are sent raw, since dot-stuffing would corrupt them and a terminator would become part of the file. There are no generated directory listings and no type 7 search.
## Card sub-documents

223
bin/src/proto/gemini.rs Normal file
View file

@ -0,0 +1,223 @@
//! Gemini: one absolute URL in, a status line and a body out.
//!
//! TLS is terminated in front of this listener, by `stunnel`, `ghostunnel` or any
//! other TLS wrapper, so what arrives here is plaintext. That keeps a TLS stack,
//! certificate loading and SNI out of the binary entirely; the protocol below is
//! the whole of Gemini that is not TLS. Two consequences are worth knowing: every
//! connection appears to come from the terminator unless it speaks the PROXY
//! protocol, and a client certificate can never reach us, so statuses 60 to 62
//! are not implementable here. itsybitsy serves static documents and has nothing
//! to authenticate, so only the logs are poorer for it.
use std::io::{BufReader, Write};
use std::net::TcpStream;
use anyhow::Result;
use itsybitsy_core::site::{Resolution, Resource};
use crate::proto::{for_log, read_line_capped};
use crate::serve::Listener;
/// The spec caps a request URL at 1024 bytes; this cap counts its CRLF too.
const MAX_REQUEST: usize = 1026;
pub fn serve(listener: &Listener, mut stream: TcpStream) -> Result<()> {
let line = {
let mut reader = BufReader::new(stream.try_clone()?);
read_line_capped(&mut reader, MAX_REQUEST)
};
let Some(line) = line else {
log::warn!("{}: unreadable or oversized request", listener.name);
return header(&mut stream, 59, "Bad request");
};
let request = match parse(&line) {
Ok(request) => request,
Err(Refusal { status, meta }) => {
log::info!("{} gemini refused {}: {meta}", listener.name, for_log(&line));
return header(&mut stream, status, meta);
}
};
// A host this server does not answer for is a request to fetch from
// elsewhere, which is what 53 is for; falling back to `default_site` first
// matches how the other host-addressed listeners behave.
let Some(site) = listener.site_for(request.host) else {
log::info!("{} gemini unknown host {}", listener.name, for_log(request.host));
return header(&mut stream, 53, "Proxy request refused");
};
log::info!("{} gemini {} {}", listener.name, for_log(request.host), for_log(request.path));
let format = &listener.formats[0];
match site.resolve(request.path) {
Ok(Resolution::Found(Resource::Document { page, .. })) => match page.body(format) {
Some(body) => body_response(&mut stream, listener.media_type(format), body),
None => header(&mut stream, 40, "Temporary failure"),
},
Ok(Resolution::Found(Resource::Part { parent_url, slug, page })) => {
match page.part(format, &slug) {
Some(body) => body_response(&mut stream, listener.media_type(format), body),
// Sub-documents belong to the paginating formats; Gemini serves
// the whole document instead of a card that does not exist here.
None => header(&mut stream, 31, &parent_url),
}
}
Ok(Resolution::Found(Resource::Raw { path, media_type })) => {
write!(stream, "20 {media_type}\r\n")?;
let mut file = std::fs::File::open(&path)?;
std::io::copy(&mut file, &mut stream)?;
Ok(())
}
Ok(Resolution::Redirect(location)) => header(&mut stream, 31, &location),
Ok(Resolution::NotFound) => header(&mut stream, 51, "Not found"),
Err(err) => {
log::warn!("{} gemini {}: {err}", listener.name, for_log(request.path));
// 40 rather than 50: the cause is a document or a directory config an
// operator can fix, so a client is right to try again later.
header(&mut stream, 40, "Temporary failure")
}
}
}
/// A request that will not be served, and the status saying why.
struct Refusal {
status: u8,
meta: &'static str,
}
struct Request<'a> {
/// The URL's authority, port included. Host normalisation strips the port.
host: &'a str,
path: &'a str,
}
/// Pick apart `gemini://host/path?query`.
///
/// Hand-rolled rather than taking a URL crate as a dependency: only the scheme,
/// the authority and the remainder are wanted, and path cleaning already drops
/// the query. The split between 53 and 59 follows the spec's own wording, where
/// 53 is "a resource at a domain not served by the server" and 59 is a request
/// the server could not parse: a URL that parses but names another scheme or
/// host is a proxy request, while one that does not parse is a bad request.
/// smolweb answers 59 to both, including to an ordinary `https://` URL.
fn parse(line: &str) -> Result<Request<'_>, Refusal> {
let Some((scheme, rest)) = line.split_once("://") else {
return Err(Refusal { status: 59, meta: "Bad request: an absolute URL is required" });
};
if !scheme.eq_ignore_ascii_case("gemini") {
return Err(Refusal { status: 53, meta: "Proxy request refused" });
}
let end = rest.find(['/', '?', '#']).unwrap_or(rest.len());
let (host, path) = rest.split_at(end);
if host.is_empty() {
return Err(Refusal { status: 59, meta: "Bad request: the URL names no host" });
}
// The spec forbids userinfo outright and forbids a client from sending a
// fragment, so neither is merely a request for somewhere else: 53 would
// suggest the URL was fine and only the host was foreign.
if host.contains('@') {
return Err(Refusal { status: 59, meta: "Bad request: userinfo is not allowed" });
}
if path.contains('#') {
return Err(Refusal { status: 59, meta: "Bad request: a fragment is not allowed" });
}
// `gemini://host` with nothing after the authority addresses the root.
let path = if path.is_empty() { "/" } else { path };
Ok(Request { host, path })
}
/// A status line and nothing else. `meta` is a redirect target for 3x and a
/// human-readable reason otherwise.
fn header(stream: &mut TcpStream, status: u8, meta: &str) -> Result<()> {
write!(stream, "{status} {meta}\r\n")?;
Ok(())
}
/// A 20 and a body. The redirect targets this listener emits are relative
/// references, which the spec permits and which are the only correct form here:
/// the port this socket is bound to is the terminator's back end, not the port a
/// client reached, so an absolute URL built from it would send clients to a port
/// that is not published.
fn body_response(stream: &mut TcpStream, media_type: &str, body: &[u8]) -> Result<()> {
write!(stream, "20 {media_type}\r\n")?;
stream.write_all(body)?;
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
fn ok(line: &str) -> (String, String) {
let request = parse(line).unwrap_or_else(|_| panic!("{line} should parse"));
(request.host.to_string(), request.path.to_string())
}
fn refused(line: &str) -> u8 {
parse(line).err().unwrap_or_else(|| panic!("{line} should be refused")).status
}
#[test]
fn splits_an_ordinary_request() {
assert_eq!(
ok("gemini://example.org/notes/one"),
("example.org".into(), "/notes/one".into())
);
assert_eq!(ok("gemini://example.org/"), ("example.org".into(), "/".into()));
}
#[test]
fn a_bare_authority_addresses_the_root() {
assert_eq!(ok("gemini://example.org"), ("example.org".into(), "/".into()));
}
#[test]
fn keeps_the_port_on_the_host_for_normalisation_to_strip() {
assert_eq!(ok("gemini://example.org:1965/"), ("example.org:1965".into(), "/".into()));
}
#[test]
fn keeps_the_query_for_path_cleaning_to_drop() {
assert_eq!(
ok("gemini://example.org/?format=text"),
("example.org".into(), "/?format=text".into())
);
// A query with no path at all still leaves the path non-empty.
assert_eq!(ok("gemini://example.org?q=1"), ("example.org".into(), "?q=1".into()));
}
#[test]
fn the_scheme_is_matched_without_regard_to_case() {
assert_eq!(ok("GEMINI://example.org/").0, "example.org");
}
#[test]
fn another_scheme_is_a_proxy_request_not_a_parse_failure() {
// smolweb answers 59 here, which tells a client its request was malformed
// when it was merely for somewhere this server does not fetch from.
assert_eq!(refused("https://example.org/"), 53);
assert_eq!(refused("gopher://example.org/"), 53);
}
#[test]
fn a_relative_or_schemeless_request_is_a_bad_request() {
assert_eq!(refused("/notes/one"), 59);
assert_eq!(refused("example.org/notes"), 59);
assert_eq!(refused(""), 59);
}
#[test]
fn userinfo_and_fragments_are_bad_requests() {
// Both are forbidden by the spec rather than simply unsupported.
assert_eq!(refused("gemini://user@example.org/"), 59);
assert_eq!(refused("gemini://example.org/page#section"), 59);
assert_eq!(refused("gemini://example.org#section"), 59);
}
#[test]
fn an_empty_authority_is_a_bad_request() {
assert_eq!(refused("gemini:///notes"), 59);
}
}

1
bin/src/proto/mod.rs
View file

@ -1,5 +1,6 @@
//! Shared plumbing for the protocol listeners.
pub mod gemini;
pub mod gopher;
pub mod http;
pub mod negotiate;

10
bin/src/serve.rs
View file

@ -196,17 +196,13 @@ fn handle(listener: &Listener, stream: TcpStream) {
// One malformed document must not take the process down, so a panic in a
// handler is caught and logged. This is the Rust equivalent of smolweb's
// bare `except Exception`, except the cause is recorded rather than lost.
let caught = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
match listener.protocol {
let caught =
std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| match listener.protocol {
Protocol::Http => proto::http::serve(listener, stream),
Protocol::Spartan => proto::spartan::serve(listener, stream),
Protocol::Nex => proto::nex::serve(listener, stream),
Protocol::Gopher => proto::gopher::serve(listener, stream),
// Configuration validation refuses this, so it cannot be reached.
Protocol::Gemini => {
unreachable!("gemini is refused at startup until its listener exists")
}
}
Protocol::Gemini => proto::gemini::serve(listener, stream),
}));
match caught {

120
bin/tests/listeners.rs
View file

@ -66,6 +66,17 @@ bind = "127.0.0.1:0"
formats = ["gemtext"]
default_site = "one"
[listener.gemini]
protocol = "gemini"
bind = "127.0.0.1:0"
formats = ["gemtext"]
default_site = "one"
[listener.gemstrict]
protocol = "gemini"
bind = "127.0.0.1:0"
formats = ["gemtext"]
[listener.nex]
protocol = "nex"
bind = "127.0.0.1:0"
@ -367,6 +378,97 @@ fn spartan_refuses_a_malformed_request_line() {
assert_eq!(server.send("spartan", b"one.test / notanumber\r\n"), "4 Bad request\r\n");
}
// -- Gemini --------------------------------------------------------------
//
// TLS is terminated in front of this listener, so these tests speak plaintext
// to it exactly as a terminator's back end would.
#[test]
fn gemini_serves_gemtext() {
let server = Server::start();
let response = server.send("gemini", b"gemini://one.test/\r\n");
assert!(response.starts_with("20 text/gemini; charset=utf-8\r\n"), "{response}");
assert!(response.contains("# One"), "{response}");
}
#[test]
fn gemini_routes_by_the_url_authority() {
let server = Server::start();
assert!(server.send("gemini", b"gemini://one.test/\r\n").contains("# One"));
assert!(server.send("gemini", b"gemini://two.test/\r\n").contains("# Two"));
}
#[test]
fn gemini_accepts_the_port_in_the_authority() {
// A client that reached a non-default port puts it in the URL it sends.
let server = Server::start();
assert!(server.send("gemini", b"gemini://one.test:1965/\r\n").contains("# One"));
}
#[test]
fn gemini_redirects_the_md_form_to_a_relative_target() {
// Relative rather than absolute: the port this socket is bound to is the
// terminator's back end, not the one the client reached.
let server = Server::start();
assert_eq!(server.send("gemini", b"gemini://one.test/about.md\r\n"), "31 /about\r\n");
}
#[test]
fn gemini_reports_a_missing_page() {
let server = Server::start();
assert_eq!(server.send("gemini", b"gemini://one.test/nope\r\n"), "51 Not found\r\n");
}
#[test]
fn gemini_serves_a_raw_file_with_its_media_type() {
let server = Server::start();
let response = server.send("gemini", b"gemini://one.test/img.png\r\n");
assert!(response.starts_with("20 image/png\r\n"), "{response}");
assert!(response.ends_with("not really a png"), "{response}");
}
#[test]
fn gemini_falls_back_to_the_default_site_for_an_unknown_host() {
let server = Server::start();
assert!(server.send("gemini", b"gemini://elsewhere.test/\r\n").contains("# One"));
}
#[test]
fn gemini_refuses_an_unknown_host_without_a_default_site() {
let server = Server::start();
let response = server.send("gemstrict", b"gemini://elsewhere.test/\r\n");
assert_eq!(response, "53 Proxy request refused\r\n");
}
#[test]
fn gemini_refuses_another_scheme_as_a_proxy_request() {
let server = Server::start();
let response = server.send("gemini", b"https://one.test/\r\n");
assert_eq!(response, "53 Proxy request refused\r\n");
}
#[test]
fn gemini_rejects_a_request_that_is_not_an_absolute_url() {
let server = Server::start();
let response = server.send("gemini", b"/about\r\n");
assert!(response.starts_with("59 Bad request"), "{response}");
}
#[test]
fn gemini_rejects_userinfo_and_fragments() {
let server = Server::start();
assert!(server.send("gemini", b"gemini://u@one.test/\r\n").starts_with("59 "));
assert!(server.send("gemini", b"gemini://one.test/about#top\r\n").starts_with("59 "));
}
#[test]
fn gemini_rejects_a_url_over_the_spec_limit() {
// 1024 bytes of URL plus CRLF is the most the spec allows.
let server = Server::start();
let long = format!("gemini://one.test/{}\r\n", "a".repeat(1100));
assert!(server.send("gemini", long.as_bytes()).starts_with("59 "), "oversized URL");
}
// -- Nex -----------------------------------------------------------------
#[test]
@ -542,12 +644,19 @@ protocol = "http"
bind = "127.0.0.1:0"
formats = ["wml", "html"]
default_site = "one"
[listener.gemini]
protocol = "gemini"
bind = "127.0.0.1:0"
formats = ["gemtext"]
default_site = "one"
"#,
root = root.to_str().unwrap(),
);
let config: ServerConfig = toml::from_str(&text).unwrap();
let registry = Arc::new(itsybitsy::registry::build());
let formats: Vec<String> = ["wml", "html"].iter().map(|s| s.to_string()).collect();
let formats: Vec<String> =
["wml", "html", "gemtext"].iter().map(|s| s.to_string()).collect();
let sites = Arc::new(SiteSet::build(&config, registry.clone(), formats).unwrap());
let bound = serve::bind(&config, registry, sites).unwrap();
@ -599,6 +708,15 @@ default_site = "one"
assert_eq!(split(&response).0, "HTTP/1.1 404 Not Found");
}
#[test]
fn a_gemini_client_is_redirected_to_the_parent_page() {
// Gemini never serves WML, so a card URL always resolves to the whole
// document rather than to a card that does not exist in gemtext.
let server = wml_server();
let response = server.send("gemini", b"gemini://one.test/trail/weather\r\n");
assert_eq!(response, "31 /trail\r\n");
}
#[test]
fn the_index_deck_links_out_to_each_card() {
let server = wml_server();

6
content/index.md
View file

@ -1,6 +1,6 @@
# itsybitsy
A small daemon serving this folder over HTTP, [Spartan](/about) and Nex.
A small daemon serving this folder over HTTP, [Spartan](/about), Gemini, Nex and Gopher.
- North Loop: open
- South Loop: closed
@ -9,6 +9,10 @@ A small daemon serving this folder over HTTP, [Spartan](/about) and Nex.
| --- | --- |
| HTTP | 8080 |
| Spartan | 3000 |
| Gemini | 1965 |
| Nex | 1900 |
| Gopher | 7070 |
Gemini here is plaintext: a TLS terminator in front owns the published port.
See the [about page](/about) for more.

25
core/src/config.rs
View file

@ -48,16 +48,6 @@ pub struct SiteSpec {
/// `default_site`.
#[serde(default)]
pub hosts: Vec<String>,
/// Gemini only; unused until that listener exists.
#[serde(default)]
pub tls: Option<TlsSpec>,
}
#[derive(Debug, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct TlsSpec {
pub cert: PathBuf,
pub key: PathBuf,
}
#[derive(Debug, Deserialize)]
@ -125,15 +115,6 @@ impl Protocol {
pub fn negotiates(self) -> bool {
matches!(self, Protocol::Http)
}
/// Whether a listener exists for this protocol.
///
/// Gemini needs a TLS stack and per-host certificate selection, which is its
/// own piece of work; naming it in a configuration is refused here rather
/// than panicking on the first connection.
pub fn is_implemented(self) -> bool {
!matches!(self, Protocol::Gemini)
}
}
/// What validation established, and what `--check` prints.
@ -263,12 +244,6 @@ impl ServerConfig {
fn validate_listeners(&self, available: &[&str]) -> Result<BTreeSet<String>, Error> {
let mut formats = BTreeSet::new();
for (name, spec) in &self.listener {
if !spec.protocol.is_implemented() {
return Err(Error::config(format!(
"listener '{name}': {} is not implemented yet",
spec.protocol.as_str()
)));
}
if spec.formats.is_empty() {
return Err(Error::config(format!("listener '{name}': formats must not be empty")));
}

8
itsybitsy.toml
View file

@ -22,6 +22,14 @@ bind = "127.0.0.1:1900"
formats = ["text"]
site = "demo"
# Plaintext: a TLS terminator would own 1965 and forward here. Poke it with
# printf 'gemini://localhost/\r\n' | nc localhost 1965
[listener.gemini]
protocol = "gemini"
bind = "127.0.0.1:1965"
formats = ["gemtext"]
default_site = "demo"
[listener.gopher]
protocol = "gopher"
bind = "127.0.0.1:7070"