231 lines
16 KiB
Markdown
231 lines
16 KiB
Markdown
# 🕸 itsybitsy
|
|
|
|
[](LICENSE) [](https://ai-declaration.md)
|
|
|
|
Serve folders of Markdown to different domains over HTTP, [Gemini](https://geminiprotocol.net/docs/specification.gmi), [Spartan](https://portal.mozz.us/gemini/spartan.mozz.us/), [Nex](https://nightfall.city/nex/info/specification.txt) and [Gopher](https://www.rfc-editor.org/rfc/rfc1436), rendered on request with no build step.
|
|
|
|
A Rust re-implementation of [smolweb](https://code.randogoth.com/randogoth/smolweb), which serves one folder from one process with its output formats hardcoded and its configuration in per-file frontmatter. itsybitsy changes all three: one process serves many domains, each output format is a separate crate behind a cargo feature, and configuration lives in a TOML file per content directory.
|
|
|
|
## Status
|
|
|
|
It serves. All five protocols work across five output formats, with virtual hosting wherever the protocol carries a hostname. Gemini expects TLS to be terminated in front of it, which is the one deployment requirement itsybitsy does not satisfy by itself.
|
|
|
|
| Area | State |
|
|
| --- | --- |
|
|
| Server configuration, virtual hosts, `--check` | implemented |
|
|
| Path resolution, traversal containment, live reload | implemented |
|
|
| Markdown parsing, includes, ASCII art, card breaks | implemented |
|
|
| gemtext, XHTML-MP and HTML output | implemented |
|
|
| HTTP, Spartan and Nex listeners, virtual hosting | implemented |
|
|
| Fixed-width text output (Nex, later Gopher) | implemented |
|
|
| FIGlet banners and hyphenation, behind features | implemented |
|
|
| WML decks and card sub-documents, behind a feature | implemented |
|
|
| Gopher, with dot-stuffing and the RFC 4266 root menu | implemented |
|
|
| Gemini, behind a TLS terminator | implemented |
|
|
| Built-in TLS, so no terminator is needed | not planned for now |
|
|
|
|
## Configuration
|
|
|
|
Two files. A server file names the virtual hosts and the listeners:
|
|
|
|
```toml
|
|
version = 1
|
|
|
|
[site.smol]
|
|
root = "/srv/smol/content"
|
|
hosts = ["smol.place", "www.smol.place"]
|
|
|
|
[listener.web]
|
|
protocol = "http"
|
|
bind = "0.0.0.0:8080"
|
|
formats = ["xhtmlmp", "html"]
|
|
default_site = "smol"
|
|
|
|
[listener.nex]
|
|
protocol = "nex"
|
|
bind = "0.0.0.0:1900"
|
|
formats = ["gemtext"]
|
|
site = "smol"
|
|
```
|
|
|
|
A page is rendered into the union of what the enabled listeners can serve, and no more: a configuration with no HTTP listener never builds any HTML. HTTP and Spartan route by the hostname the request carries, falling back to `default_site`; Nex carries none, so its listener names one site outright.
|
|
|
|
A `.itsybitsy.toml` in any content directory says how the Markdown in **that directory** renders. There is no inheritance: a subdirectory without its own file uses the built-in defaults rather than its parent's. Repetition in a deep tree is the price of never having to look elsewhere to know how a page renders.
|
|
|
|
```toml
|
|
[defaults]
|
|
max_age = 3600
|
|
margin_left = 2
|
|
h1_style = "underline:="
|
|
|
|
[page."index.md"]
|
|
title = "Notes"
|
|
max_age = 300
|
|
```
|
|
|
|
`[defaults]` applies to every Markdown file in the directory; a `[page."name.md"]` table overrides it for one file. `title` is per-page by nature, so setting it in `[defaults]` is an error; left unset, it is derived from the page's first level-1 heading.
|
|
|
|
The fixed-width text format reads `margin_left`, `margin_right`, `paragraph_spacing`, `h1_style` through `h6_style`, `blockquote_bars`, `list_indent`, `code_block_line_numbers` and `wrap_code_blocks`. A heading style is `underline`, `underline:<char>`, `figlet`, `figlet:<font>`, `markers` or `plain`; by default the first three levels are underlined with `=`, `-` and `~`, and deeper ones keep their `#` markers, since underline characters run out before heading levels do. Editing the file re-renders that directory's pages on the next request, and a file that fails to parse makes them error rather than silently falling back to defaults.
|
|
|
|
Because the name begins with a dot, the same rule that keeps `.secret.md` out of the URL space keeps this file out of it too.
|
|
|
|
`--check` validates the configuration and reports the routing it resolved, without binding a port:
|
|
|
|
```console
|
|
$ itsybitsy --config /etc/itsybitsy.toml --check
|
|
sites:
|
|
smol: /srv/smol/content
|
|
hosts:
|
|
smol.place -> smol
|
|
www.smol.place -> smol
|
|
listeners:
|
|
web: http on 0.0.0.0:8080 [xhtmlmp, html] by host, default smol
|
|
formats rendered per page: html, xhtmlmp
|
|
```
|
|
|
|
Everything the schema cannot express is checked here rather than on first request: a format no enabled feature provides, two sites claiming one hostname, a content root that does not exist, a server file sitting inside a content root where it would be served.
|
|
|
|
## Markdown
|
|
|
|
Markdown is parsed once per document into one representation that every output format reads, rather than once per format. Everything itsybitsy adds to CommonMark is either a construct other tools already understand or invisible to them, so a document stays portable:
|
|
|
|
| Written | Means |
|
|
| --- | --- |
|
|
| `![[path]]` | splice that file's lines in here |
|
|
| `` | an image whose target is a text file, inlined verbatim |
|
|
| `<!-- card Title -->` | a card divider, for formats that paginate |
|
|
| `---` | an untitled divider |
|
|
| `<!-- center -->`, `<!-- right margin=4 -->` | alignment for the block that follows |
|
|
|
|
Only the include is non-standard, and it is the spelling Obsidian and its relatives established; CommonMark renders it as literal text, which is why it is the one construct handled before parsing. The rest are ordinary HTML comments and images, interpreted after parsing — so an HTML comment that is not a recognised directive stays a comment, at the cost of a misspelled one doing nothing rather than complaining.
|
|
|
|
Alignment only reaches the fixed-width text formats; gemtext and HTML have no way to express it. An art label may carry its own `:center`, `:left` or `:right` token, which is removed from the label so it does not leak into the fallback text a gemtext or HTML client shows.
|
|
|
|
Include and art targets resolve relative to the file holding the reference and must stay inside the content root. Expansion is bounded on three axes — nesting depth, total lines, and total bytes — because a cycle check alone does not stop a long chain, and neither stops a diamond where two branches include the same file without ever repeating one on a single path.
|
|
|
|
### Differences from smolweb
|
|
|
|
Configuration keys are not carried over verbatim either: md2txt accepts three spellings of `paragraph_spacing`, and its `cache_control` is named for the header it lands in rather than the value it holds, which here is `max_age`. Its `text` and `nex` renderers differ only in heading decoration and in whether links are inlined or numbered — and the numbered form never writes the reference list its numbers point at, so there is one text renderer here rather than two. Tables, which both of its text renderers drop entirely, are rendered as aligned columns.
|
|
|
|
wapdown's own parser produces neither tables nor lists, so its WML decks render a bullet list as literal text and drop tables entirely; both come out as real markup here, lists as marked lines and tables as `<table>`, which WML has.
|
|
|
|
Gemini diverges in three places. smolweb answers 59 (bad request) to any URL that is not `gemini://`, which tells a client its request was malformed when an `https://` URL is merely for somewhere this server does not fetch from; that is 53 (proxy request refused) here, and 59 is kept for a URL that genuinely does not parse. smolweb builds absolute redirect targets from the port it is bound to, which is wrong behind a terminator; these are relative. And smolweb terminates TLS itself with a certificate named per site, so `tls` is not a configuration key here at all — naming one is an error rather than a setting that quietly does nothing.
|
|
|
|
smolweb inherits five invented directives from its two renderer libraries: `{.card}`, `{.include}`, `![[ ]]`, `#[label](art.txt)` and MultiMarkdown attribute lists (`{: .center}`) — two incompatible brace grammars, with art alignment expressible two different ways in one line. None of it appeared in real content, so itsybitsy expresses the same capabilities with standard constructs instead. Parsing once rather than once per library also means the text formats gain setext headings and the WAP formats gain tables and ASCII art, none of which their original parser handled.
|
|
|
|
## Output formats
|
|
|
|
Each format is a crate implementing one trait, compiled in behind a cargo feature. A renderer is handed the parsed document and may not open files or sockets, which is what keeps every path check in one place rather than in each format.
|
|
|
|
| Format | Served as | Notes |
|
|
| --- | --- | --- |
|
|
| `gemtext` | `text/gemini` | Unwrapped; Gemini and Spartan clients wrap for themselves |
|
|
| `xhtmlmp` | `application/vnd.wap.xhtml+xml` | Well-formed XML with the Mobile Profile doctype |
|
|
| `html` | `text/html` | The same markup without the XML declaration |
|
|
| `text` | `text/plain` | Wrapped to 80 columns, because Nex and Gopher clients do not wrap |
|
|
| `wml` | `text/vnd.wap.wml` | WAP 1.x decks, paginated to a per-card byte budget (feature `wml`) |
|
|
|
|
HTTP negotiates between them from `Accept`. Only a literal media type counts as a match, so a browser's `*/*` can never be read as willingness to receive a WAP format; `?format=<id>` overrides negotiation outright, and naming an id the listener does not serve is a bad request rather than a silent fallback.
|
|
|
|
A third party adds a format by writing a crate that depends on `itsybitsy-core`, adding it to the workspace and one `#[cfg]` arm in the binary's registry, then naming its id in `formats`. Every listener's formats are checked against the registry at startup, so a format switched off by a feature is a configuration error naming the missing id rather than a server error on the first request.
|
|
|
|
## Optional features
|
|
|
|
Two decorative capabilities for the text format are off by default, because most pages leave them alone and both cost binary size:
|
|
|
|
| Feature | Adds | Cost |
|
|
| --- | --- | --- |
|
|
| `wml` | WML 1.3 decks for WAP 1.x handsets | +37 KiB |
|
|
| `figlet` | `h1_style = "figlet"` banners, fonts `small` and `standard` | +60 KiB |
|
|
| `hyphenation` | `hyphenate = true`, English patterns | +127 KiB |
|
|
| `hyphenation-all` | every language the pattern crate carries | +3.0 MiB |
|
|
|
|
```bash
|
|
cargo build --release --features "figlet hyphenation"
|
|
```
|
|
|
|
A banner that will not fit the line, names a font this build lacks, or is asked for by a build without `figlet` falls back to the level's underline — a heading that cannot be decorated should not be lost. An unknown font name is logged, since that is almost always a typo. Likewise `hyphenate` has no effect without the feature, and an unknown `hyphen_lang` leaves the text unhyphenated; a ragged right edge is the plain-text convention anyway.
|
|
|
|
## Protocols
|
|
|
|
| Protocol | Routes by | Notes |
|
|
| --- | --- | --- |
|
|
| HTTP | `Host` header | `GET` and `HEAD`, negotiated by `Accept` |
|
|
| Spartan | the host field of its request line | One format per listener |
|
|
| Nex | nothing; its listener names one site | No status line at all |
|
|
| Gemini | the authority of the URL it is sent | TLS terminated in front; one format per listener |
|
|
| Gopher | nothing; its listener names one site | Item type 0 and the root menu |
|
|
|
|
A protocol with no hostname in its requests cannot be routed by one, so its listener names a single site outright and configuration refuses to leave that implicit once more than one site exists. HTTP, Gemini and Spartan fall back to `default_site` when a request names a host this server does not know.
|
|
|
|
Neither Nex nor Gopher has a redirect status, so a canonical target is resolved server-side and the real content comes back on the first request rather than a bounce.
|
|
|
|
### Gemini and TLS
|
|
|
|
itsybitsy does not speak TLS. The Gemini listener expects plaintext, so a TLS wrapper terminates in front of it and forwards to a loopback port:
|
|
|
|
```toml
|
|
[listener.gemini]
|
|
protocol = "gemini"
|
|
# The terminator owns 1965; this is its back end.
|
|
bind = "127.0.0.1:11965"
|
|
formats = ["gemtext"]
|
|
default_site = "smol"
|
|
```
|
|
|
|
```ini
|
|
; /etc/stunnel/gemini.conf
|
|
[gemini]
|
|
accept = 1965
|
|
connect = 127.0.0.1:11965
|
|
cert = /var/lib/itsybitsy/gemini.pem
|
|
```
|
|
|
|
`stunnel` and `ghostunnel` are built for exactly this and support SNI, so each virtual host can present its own certificate. Use a long-lived self-signed certificate rather than an ACME one: Gemini clients pin the fingerprint they first saw, and a renewal that changes it is indistinguishable from an interception.
|
|
|
|
Two things follow from terminating in front, and neither is a limitation that can be configured away. Every connection appears to come from the terminator, so request logs show its address unless it speaks the PROXY protocol, which itsybitsy does not yet read. And a client certificate never reaches us, so statuses 60 to 62 are not implementable — itsybitsy serves static documents and has nothing to authenticate, so only the logs are poorer for it.
|
|
|
|
Redirects are relative references, which the spec permits and which are the only correct form here: the port this listener is bound to is the terminator's back end, not a port any client reached, so an absolute URL built from it would point somewhere unpublished.
|
|
|
|
Gopher serves item type 0 (text) and one menu: a prefix-less `gopher://host/` means item type 1 by RFC 4266, so an empty selector gets a one-item menu pointing at `/` rather than the root document, which would be the wrong type. Text items are dot-stuffed and terminated with a lone dot; binary items are sent raw, since dot-stuffing would corrupt them and a terminator would become part of the file. There are no generated directory listings and no type 7 search.
|
|
|
|
## Card sub-documents
|
|
|
|
WML is the one output that paginates: a WAP 1.x handset has a hard per-card byte budget and refuses a deck that exceeds it. So a long page becomes a chain of screens, and a page with dividers becomes a menu card linking to the rest.
|
|
|
|
With `deck_per_card`, each card also gets its own URL under the page's:
|
|
|
|
| Request | WML client | Any other format |
|
|
| --- | --- | --- |
|
|
| `/trail` | the menu deck | the whole page |
|
|
| `/trail/weather` | that card's own deck | redirects to `/trail` |
|
|
| `/trail/nonexistent` | redirects to `/trail` | redirects to `/trail` |
|
|
| `/about/whatever` | not found | not found |
|
|
|
|
That URL space belongs to the format that claims it. A format which does not address a sub-document redirects to the parent rather than substituting something else, and a page with no dividers has no such URLs at all — `/about/whatever` is not a sub-document just because `/about` exists. Nex and Gopher have no redirect status, so they resolve to the parent's content directly instead of bouncing.
|
|
|
|
The deck keys are `split_level`, `split_on_rule`, `max_card_bytes`, `menu`, `menu_style`, `deck_per_card`, `template_nav`, `nav_next_label`, `nav_prev_label`, `nav_back_label`, `home_label` and `images`.
|
|
|
|
## URLs
|
|
|
|
Links should be root-relative and extensionless (`[about](/about)`, not `about.md`), so the same link resolves identically from every protocol.
|
|
|
|
| Request | Serves |
|
|
| --- | --- |
|
|
| `/` | `index.md` |
|
|
| `/foo` | `foo.md`, else `foo/index.md` |
|
|
| `/foo.md` | redirects to `/foo` |
|
|
| `/img.png` | the file itself, by media type |
|
|
|
|
Nothing outside the content root is reachable. A request target is percent-decoded before it is normalised, so an encoded `..` becomes a real one and gets clamped at the root rather than quietly matching nothing; any path component beginning with a dot is refused outright; and whatever survives is canonicalised and required to still be inside the root, which is what defeats a symlink pointing out of it. There are no generated directory listings — only `index.md`.
|
|
|
|
## Development
|
|
|
|
```bash
|
|
devbox run check # fmt --check, clippy -D warnings, test
|
|
devbox run test
|
|
devbox run build
|
|
devbox run run # serve ./content on localhost
|
|
```
|
|
|
|
`RUST_LOG` sets the log level, which defaults to `info`: one line per request, and the real cause on failure.
|