feat: initial implementation

This commit is contained in:
randogoth 2026-09-26 12:45:51 +03:00
commit c3cb1d7046
19 changed files with 1296 additions and 0 deletions

6
.gitignore vendored Normal file
View file

@ -0,0 +1,6 @@
__pycache__/
*.pyc
.venv/
cert.pem
key.pem
*.egg-info/

87
README.md Normal file
View file

@ -0,0 +1,87 @@
# smolweb
Serve a folder of Markdown as [Gemini](https://geminiprotocol.net/) capsules,
[Spartan](https://portal.mozz.us/gemini/spartan.mozz.us/) capsules, and HTTP
pages content-negotiated between WML (WAP 1.x) and XHTML-MP (WAP 2.0 and
modern browsers) — all rendered on the fly, no build step.
```bash
smolweb --root ./content --host example.com
```
Every listener binds by default: Gemini on `:1965` (TLS), Spartan on `:300`,
HTTP on `:8080`. An empty address disables one:
```bash
smolweb --root ./content --host example.com --spartan ""
```
## Rendering
Markdown is parsed once per file (cached by mtime, re-parsed the moment the
file changes) and rendered into every format from the same source:
- **gemtext** via [md2txt](https://code.randogoth.com/randogoth/md2txt)'s `gemini` renderer.
- **WML** and **XHTML-MP** via [wapdown](https://code.randogoth.com/randogoth/wapdown).
Both libraries read the same leading `---` frontmatter block independently
and ignore keys they don't recognise, so one file can freely mix wapdown's
deck-structure keys (`title`, `split_level`, `deck_per_card`, ...) with
md2txt's typography keys (`links_per_block`, ...).
wapdown's `{.card Title}` card-break directive has no meaning to md2txt's
parser, so it is stripped before gemtext rendering rather than leaking
through as literal text — a card break already renders as nothing in
XHTML-MP for the same reason (cards are a WAP 1.x screen-budget concern;
a Gemini client or browser just scrolls through one continuous document).
## URLs
Links should be root-relative and extensionless (`[about](/about)`, not
`about.md`) — the same link then resolves identically from a Gemini,
Spartan, or HTTP client:
| Request | Serves |
| --- | --- |
| `/` | `index.md` |
| `/foo` | `foo.md`, else `foo/index.md` |
| `/foo.md` | redirects to `/foo` |
| `/img.png` | served as-is, by MIME type |
A document with `deck_per_card: true` gets its own URL space for WML only:
`/trail` serves the whole document (the menu deck, if it has one), and
`/trail/weather` serves one card's own deck directly. Gemini, Spartan, and
an HTTP client negotiating gemtext/XHTML-MP all redirect `/trail/weather`
back to `/trail` instead — that URL space only ever makes sense for WML.
## HTTP content negotiation
| Accept | Served as |
| --- | --- |
| `text/vnd.wap.wml` (literal, non-wildcard) | WML |
| `application/vnd.wap.xhtml+xml` (literal, non-wildcard) | XHTML-MP |
| anything else, including a `*/*` wildcard or no header at all | the same XHTML-MP body, as `text/html` |
A `*/*` wildcard never selects WML — a browser's `Accept: text/html,
application/xhtml+xml;q=0.9,*/*;q=0.8` must never be handed WML through the
wildcard. `?format=wml`, `?format=xhtml`, or `?format=html` overrides
negotiation outright, for testing without real WAP hardware.
## TLS
If `--cert`/`--key` don't exist, a self-signed EC P-256 certificate is
generated on first run (via `openssl`, so devbox must provide it), valid for
ten years. Gemini clients use TOFU (trust-on-first-use) fingerprint pinning
rather than CA validation, so this is the norm rather than a warning.
## Development
```bash
devbox run sync # install everything, including wapdown/md2txt
devbox run test # pytest
devbox run run # serve ./content on localhost, Spartan on :3000 (unprivileged)
```
`pyproject.toml` currently points `wapdown`/`md2txt` at local paths (see the
comment there) rather than their git origins, since this checkout carries
unpushed changes to both.

10
content/about.md Normal file
View file

@ -0,0 +1,10 @@
---
title: About smolweb
---
# About
This capsule is served by **smolweb**, a tiny Python daemon that renders
Markdown on the fly for Gemini, Spartan, and HTTP (WML or XHTML-MP,
content-negotiated).
Back to [the index](/).

19
content/index.md Normal file
View file

@ -0,0 +1,19 @@
---
title: smolweb
links_per_block: true
link_markers: false
---
# smolweb
A small daemon serving this folder as [Gemini](/about), Spartan, WML, and XHTML-MP.
- North Loop: open
- South Loop: closed
| Format | Port |
| --- | --- |
| Gemini | 1965 |
| Spartan | 300 |
| HTTP | 8080 |
See the [about page](/about) for more.

16
devbox.json Normal file
View file

@ -0,0 +1,16 @@
{
"packages": [
"python313@latest",
"uv@latest",
"libxml2@latest",
"netcat-gnu@latest",
"openssl@latest"
],
"shell": {
"scripts": {
"sync": ["uv sync"],
"test": ["uv run pytest"],
"run": ["uv run smolweb --root ./content --host localhost --spartan :3000"]
}
}
}

39
pyproject.toml Normal file
View file

@ -0,0 +1,39 @@
[project]
name = "smolweb"
version = "0.1.0"
description = "Serve a folder of Markdown as gemtext, Spartan, WML, and XHTML-MP"
readme = "README.md"
requires-python = ">=3.13"
dependencies = [
"wapdown",
"md2txt",
]
[project.scripts]
smolweb = "smolweb.cli:main"
# Local path sources: `uv.toml` cannot hold a `[sources]` table (uv only
# reads that from pyproject.toml), so there is no separate override file to
# split "reproducible" from "editable" -- this table is the only place it
# can live. Both origins are real (code.randogoth.com), but this checkout's
# wapdown/md2txt have unpushed local changes (the xhtmlmp renderer, the
# gemtext fixes) that a git source can't see yet, so path is what actually
# works right now. Once those are pushed, swap to:
# wapdown = { git = "ssh://git@code.randogoth.com:2222/randogoth/wapdown.git" }
# md2txt = { git = "ssh://git@code.randogoth.com:2222/randogoth/md2txt.git" }
[tool.uv.sources]
wapdown = { path = "../wapdown", editable = true }
md2txt = { path = "../md2txt", editable = true }
[dependency-groups]
dev = ["pytest>=8.0"]
[tool.pytest.ini_options]
testpaths = ["tests"]
[build-system]
requires = ["setuptools>=68", "wheel"]
build-backend = "setuptools.build_meta"
[tool.setuptools.packages.find]
where = ["src"]

0
src/smolweb/__init__.py Normal file
View file

37
src/smolweb/cert.py Normal file
View file

@ -0,0 +1,37 @@
"""Self-signed TLS certificate autogeneration for the Gemini listener.
Gemini clients use TOFU (trust-on-first-use) fingerprint pinning rather
than CA validation, so a long-lived self-signed cert is the norm here, not
a warning-triggering exception -- see the Gemini protocol specification.
Generation shells out to `openssl` rather than adding a Python crypto
dependency, keeping smolweb's own dependency list at zero beyond
wapdown/md2txt; devbox supplies the binary.
"""
from __future__ import annotations
import subprocess
from pathlib import Path
VALIDITY_DAYS = 365 * 10 # TOFU pins the fingerprint; expiry only breaks trust, never renews it
def ensure_certificate(cert_path: Path, key_path: Path, host: str) -> None:
"""Generate a self-signed EC P-256 cert/key pair if either is missing."""
if cert_path.exists() and key_path.exists():
return
cert_path.parent.mkdir(parents=True, exist_ok=True)
key_path.parent.mkdir(parents=True, exist_ok=True)
subprocess.run(
[
"openssl", "req", "-x509", "-newkey", "ec",
"-pkeyopt", "ec_paramgen_curve:P-256",
"-nodes", "-days", str(VALIDITY_DAYS),
"-keyout", str(key_path), "-out", str(cert_path),
"-subj", f"/CN={host}",
"-addext", f"subjectAltName=DNS:{host}",
],
check=True,
capture_output=True,
)
key_path.chmod(0o600)

91
src/smolweb/cli.py Normal file
View file

@ -0,0 +1,91 @@
"""smolweb: serve a folder of Markdown as gemtext, Spartan, WML, and XHTML-MP."""
from __future__ import annotations
import argparse
import ssl
import sys
import threading
from pathlib import Path
from typing import Tuple
from .cert import ensure_certificate
from .servers.gemini import GeminiServer
from .servers.http import SmolwebHTTPServer
from .servers.spartan import SpartanServer
from .site import Site
def _parse_address(spec: str) -> Tuple[str, int]:
"""'[HOST]:PORT', e.g. ':1965' or 'localhost:1965'."""
host, _, port = spec.rpartition(":")
return host, int(port)
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
prog="smolweb",
description="Serve a folder of Markdown as gemtext, Spartan, WML, and XHTML-MP.",
)
parser.add_argument("--root", type=Path, default=Path("content"), help="Folder of Markdown to serve (default: ./content).")
parser.add_argument("--host", default="localhost", help="Hostname this server answers to (default: localhost).")
parser.add_argument("--gemini", default=":1965", metavar="[HOST]:PORT", help="Gemini listen address; empty disables it (default: :1965).")
parser.add_argument("--spartan", default=":300", metavar="[HOST]:PORT", help="Spartan listen address; empty disables it (default: :300).")
parser.add_argument("--http", default=":8080", metavar="[HOST]:PORT", help="HTTP listen address; empty disables it (default: :8080).")
parser.add_argument("--cert", type=Path, default=Path("cert.pem"), help="TLS certificate path, generated on first run if missing.")
parser.add_argument("--key", type=Path, default=Path("key.pem"), help="TLS key path, generated on first run if missing.")
return parser
def main(argv=None) -> int:
args = build_parser().parse_args(argv)
root = args.root.resolve()
if not root.is_dir():
sys.stderr.write(f"smolweb: {root} is not a directory\n")
return 2
site = Site(root)
threads = []
if args.gemini:
try:
ensure_certificate(args.cert, args.key, args.host)
except Exception as exc: # openssl missing, or a generation failure
sys.stderr.write(f"smolweb: could not prepare TLS certificate: {exc}\n")
return 2
ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
ssl_context.minimum_version = ssl.TLSVersion.TLSv1_2
ssl_context.load_cert_chain(str(args.cert), str(args.key))
gemini_addr = _parse_address(args.gemini)
threads.append(_serve_in_thread(GeminiServer(gemini_addr, site, args.host, ssl_context), "gemini", gemini_addr))
if args.spartan:
spartan_addr = _parse_address(args.spartan)
threads.append(_serve_in_thread(SpartanServer(spartan_addr, site), "spartan", spartan_addr))
if args.http:
http_addr = _parse_address(args.http)
threads.append(_serve_in_thread(SmolwebHTTPServer(http_addr, site), "http", http_addr))
if not threads:
sys.stderr.write("smolweb: every listener disabled, nothing to do\n")
return 2
try:
for thread in threads:
thread.join()
except KeyboardInterrupt:
pass
return 0
def _serve_in_thread(server, name: str, address: Tuple[str, int]) -> threading.Thread:
host, port = address
sys.stderr.write(f"smolweb: {name} listening on {host or '*'}:{port}\n")
thread = threading.Thread(target=server.serve_forever, name=name, daemon=True)
thread.start()
return thread
if __name__ == "__main__":
raise SystemExit(main())

83
src/smolweb/negotiate.py Normal file
View file

@ -0,0 +1,83 @@
"""HTTP Accept-header negotiation between WML and XHTML-MP.
The one rule that matters: a `*/*` wildcard must never select WML or the
WAP XHTML type. A modern browser sends `Accept: text/html,
application/xhtml+xml;q=0.9,*/*;q=0.8` -- a matcher that lets the wildcard
satisfy `text/vnd.wap.wml` would hand it WML, which most browsers refuse to
render as anything but `text/html` (see wapdown's README). Only a literal,
non-wildcard match may select WML or `application/vnd.wap.xhtml+xml`;
everything else, including no Accept header at all, gets XHTML-MP served
as `text/html`.
"""
from __future__ import annotations
from typing import List, NamedTuple, Optional
WML_TYPE = "text/vnd.wap.wml"
XHTML_MP_TYPE = "application/vnd.wap.xhtml+xml"
FORMAT_WML = "wml"
FORMAT_XHTML = "xhtml" # served as application/vnd.wap.xhtml+xml
FORMAT_HTML = "html" # same XHTML-MP body, served as text/html
CONTENT_TYPES = {
FORMAT_WML: f"{WML_TYPE}; charset=utf-8",
FORMAT_XHTML: f"{XHTML_MP_TYPE}; charset=utf-8",
FORMAT_HTML: "text/html; charset=utf-8",
}
class _MediaRange(NamedTuple):
kind: str # media type, lowercased, e.g. "text/vnd.wap.wml" or "*/*"
q: float
def _parse_accept(accept: str) -> List[_MediaRange]:
ranges: List[_MediaRange] = []
for part in accept.split(","):
part = part.strip()
if not part:
continue
kind, _, params = part.partition(";")
q = 1.0
for param in params.split(";"):
param = param.strip()
if param.startswith("q="):
try:
q = float(param[2:])
except ValueError:
q = 1.0
ranges.append(_MediaRange(kind.strip().lower(), q))
return ranges
def _best_q(ranges: List[_MediaRange], media_type: str) -> Optional[float]:
"""The highest q among *literal* (non-wildcard) matches for media_type."""
matches = [r.q for r in ranges if r.kind == media_type]
return max(matches) if matches else None
def negotiate(accept: Optional[str], *, format_override: Optional[str] = None) -> str:
"""Return one of FORMAT_WML, FORMAT_XHTML, FORMAT_HTML.
`format_override` is `?format=wml|xhtml|html` from the request's query
string, which wins outright -- essential for testing WAP 1.x behavior
without real WAP 1.x hardware.
"""
if format_override in (FORMAT_WML, FORMAT_XHTML, FORMAT_HTML):
return format_override
ranges = _parse_accept(accept) if accept else []
wml_q = _best_q(ranges, WML_TYPE)
xhtml_q = _best_q(ranges, XHTML_MP_TYPE)
if wml_q is not None and (xhtml_q is None or wml_q >= xhtml_q):
return FORMAT_WML
if xhtml_q is not None:
return FORMAT_XHTML
return FORMAT_HTML
def content_type_for(fmt: str) -> str:
return CONTENT_TYPES[fmt]

View file

View file

@ -0,0 +1,102 @@
"""Gemini protocol listener (:1965, TLS)."""
from __future__ import annotations
import socketserver
import ssl
from urllib.parse import urlsplit
from ..site import NotFound, RawFile, Redirect, Site, WmlCard
MAX_REQUEST_BYTES = 1024 + 2 # spec: URI MUST NOT exceed 1024 bytes, plus CRLF
SOCKET_TIMEOUT = 10.0
DEFAULT_PORT = 1965
class GeminiHandler(socketserver.StreamRequestHandler):
def handle(self) -> None:
self.request.settimeout(SOCKET_TIMEOUT)
try:
# A cap of MAX_REQUEST_BYTES+1 both enforces the spec's 1024-byte
# URI limit and bounds a slow/hostile client to one short read.
raw = self.rfile.readline(MAX_REQUEST_BYTES + 1)
except OSError:
return
if not raw.endswith(b"\r\n"):
self._send(59, "Bad request")
return
try:
line = raw[:-2].decode("utf-8")
except UnicodeDecodeError:
self._send(59, "Bad request")
return
self._respond(line)
def _respond(self, line: str) -> None:
parsed = urlsplit(line)
if parsed.scheme != "gemini":
self._send(59, "Bad request: only gemini:// URIs are accepted")
return
host = self.server.host
if host and parsed.hostname and parsed.hostname != host:
self._send(53, "Proxy request refused")
return
try:
resource = self.server.site.resolve(parsed.path or "/")
except Redirect as redirect:
self._send(31, self.server.absolute_uri(redirect.location))
return
except NotFound:
self._send(51, "Not found")
return
except Exception:
self._send(40, "Temporary failure")
return
if isinstance(resource, RawFile):
self._send_body(20, resource.mime, resource.data)
elif isinstance(resource, WmlCard):
# /trail/weather is WML-only URL space; Gemini never serves
# WML, so it always redirects to the whole document.
self._send(31, self.server.absolute_uri(resource.parent_url))
else:
self._send_body(20, "text/gemini; charset=utf-8", resource.gemtext)
def _send(self, status: int, meta: str) -> None:
try:
self.wfile.write(f"{status} {meta}\r\n".encode("utf-8"))
except OSError:
pass
def _send_body(self, status: int, meta: str, body: bytes) -> None:
try:
self.wfile.write(f"{status} {meta}\r\n".encode("utf-8"))
self.wfile.write(body)
except OSError:
pass
class GeminiServer(socketserver.ThreadingTCPServer):
daemon_threads = True
allow_reuse_address = True
def __init__(self, address, site: Site, host: str, ssl_context: ssl.SSLContext) -> None:
self.site = site
self.host = host
self._port = address[1]
self._ssl_context = ssl_context
super().__init__(address, GeminiHandler)
def get_request(self):
# Wrap each accepted connection individually (rather than the
# listening socket itself) so a failed handshake from one client
# raises OSError from here, which socketserver's request loop
# already catches and drops -- it can't take down the listener.
conn, addr = super().get_request()
conn.settimeout(SOCKET_TIMEOUT)
return self._ssl_context.wrap_socket(conn, server_side=True), addr
def absolute_uri(self, path: str) -> str:
port_suffix = "" if self._port == DEFAULT_PORT else f":{self._port}"
return f"gemini://{self.host}{port_suffix}{path}"

View file

@ -0,0 +1,89 @@
"""HTTP listener: WML for WAP 1.x, XHTML-MP for WAP 2.0 and modern browsers."""
from __future__ import annotations
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from urllib.parse import parse_qs, urlsplit
from ..negotiate import FORMAT_WML, content_type_for, negotiate
from ..site import NotFound, RawFile, Redirect, Site, WmlCard
class SmolwebHTTPHandler(BaseHTTPRequestHandler):
def do_GET(self) -> None:
parsed = urlsplit(self.path)
query = parse_qs(parsed.query)
format_override = (query.get("format") or [None])[0]
try:
resource = self.server.site.resolve(parsed.path)
except Redirect as redirect:
self.send_response(301)
self.send_header("Location", redirect.location)
self.end_headers()
return
except NotFound:
self.send_response(404)
self.send_header("Content-Type", "text/plain; charset=utf-8")
self.end_headers()
self.wfile.write(b"Not found\n")
return
except Exception:
self.send_response(500)
self.end_headers()
return
if isinstance(resource, RawFile):
self._send(200, resource.mime, resource.data)
return
fmt = negotiate(self.headers.get("Accept"), format_override=format_override)
if isinstance(resource, WmlCard):
# /trail/weather is WML-only URL space; a client negotiating
# anything else gets sent back to the whole document at /trail.
if fmt != FORMAT_WML:
self.send_response(301)
self.send_header("Location", resource.parent_url)
self.end_headers()
return
self._send(
200,
content_type_for(FORMAT_WML),
resource.wml,
vary_accept=True,
cache_control=resource.cache_control,
)
return
body = resource.wml_index if fmt == FORMAT_WML else resource.xhtml
self._send(
200,
content_type_for(fmt),
body,
vary_accept=True,
cache_control=resource.cache_control,
)
def _send(self, status: int, content_type: str, body: bytes, *, vary_accept: bool = False, cache_control=None) -> None:
self.send_response(status)
self.send_header("Content-Type", content_type)
self.send_header("Content-Length", str(len(body)))
if vary_accept:
# Without this, an intermediate cache serving one client's
# negotiated response to the next client would hand WML to a
# browser or XHTML-MP to a WAP 1.x handset.
self.send_header("Vary", "Accept")
if cache_control is not None:
self.send_header("Cache-Control", f"max-age={cache_control}")
self.end_headers()
if self.command != "HEAD":
self.wfile.write(body)
class SmolwebHTTPServer(ThreadingHTTPServer):
daemon_threads = True
allow_reuse_address = True
def __init__(self, address, site: Site) -> None:
self.site = site
super().__init__(address, SmolwebHTTPHandler)

View file

@ -0,0 +1,98 @@
"""Spartan protocol listener (:300, plaintext TCP)."""
from __future__ import annotations
import socketserver
from ..site import NotFound, RawFile, Redirect, Site, WmlCard
MAX_REQUEST_BYTES = 4096
SOCKET_TIMEOUT = 10.0
DRAIN_CHUNK = 65536
class SpartanHandler(socketserver.StreamRequestHandler):
def handle(self) -> None:
self.request.settimeout(SOCKET_TIMEOUT)
try:
raw = self.rfile.readline(MAX_REQUEST_BYTES + 1)
except OSError:
return
if not raw.endswith(b"\r\n"):
self._send(4, "Bad request")
return
try:
line = raw[:-2].decode("utf-8")
except UnicodeDecodeError:
self._send(4, "Bad request")
return
parts = line.split(" ")
if len(parts) != 3:
self._send(4, "Bad request")
return
_host, path, length_str = parts
try:
content_length = int(length_str)
except ValueError:
self._send(4, "Bad request")
return
if content_length > 0:
# v1 accepts no uploads; the =: prompt-link extension exists to
# feed one, which a static markdown folder never needs. The
# body must still be drained so the connection stays in sync.
self._drain(content_length)
self._send(4, "Uploads are not accepted")
return
try:
resource = self.server.site.resolve(path or "/")
except Redirect as redirect:
self._send(3, redirect.location)
return
except NotFound:
self._send(4, "Not found")
return
except Exception:
self._send(5, "Internal error")
return
if isinstance(resource, RawFile):
self._send_body(2, resource.mime, resource.data)
elif isinstance(resource, WmlCard):
# /trail/weather is WML-only URL space; Spartan never serves
# WML, so it always redirects to the whole document.
self._send(3, resource.parent_url)
else:
self._send_body(2, "text/gemini; charset=utf-8", resource.gemtext)
def _drain(self, length: int) -> None:
remaining = length
while remaining > 0:
chunk = self.rfile.read(min(remaining, DRAIN_CHUNK))
if not chunk:
break
remaining -= len(chunk)
def _send(self, status: int, content: str) -> None:
try:
self.wfile.write(f"{status} {content}\r\n".encode("utf-8"))
except OSError:
pass
def _send_body(self, status: int, content: str, body: bytes) -> None:
try:
self.wfile.write(f"{status} {content}\r\n".encode("utf-8"))
self.wfile.write(body)
except OSError:
pass
class SpartanServer(socketserver.ThreadingTCPServer):
daemon_threads = True
allow_reuse_address = True
def __init__(self, address, site: Site) -> None:
self.site = site
super().__init__(address, SpartanHandler)

267
src/smolweb/site.py Normal file
View file

@ -0,0 +1,267 @@
"""Path resolution, live-reload caching, and multi-format rendering.
A folder of Markdown is rendered on request into gemtext (md2txt), and WML
plus XHTML-MP (wapdown). Both libraries parse the same leading `---`
frontmatter block independently and ignore keys they don't recognise, so
one file can carry both a wapdown deck-structure key (`title`,
`split_level`, `deck_per_card`) and an md2txt typography key
(`links_per_block`) with no conflict.
"""
from __future__ import annotations
import mimetypes
import posixpath
import threading
from dataclasses import dataclass, field
from pathlib import Path
from typing import Dict, Optional, Union
from urllib.parse import unquote
from md2txt import convert_markdown
from md2txt.conversion.core import parse_frontmatter as md2txt_parse_frontmatter
from wapdown.cli import convert as wapdown_convert
from wapdown.config import DeckConfig
from wapdown.conversion.core import strip_frontmatter as wapdown_strip_frontmatter
from wapdown.parsers.markdown import CARD_BREAK_PATTERN
class NotFound(Exception):
"""No resource exists at the requested path."""
class Redirect(Exception):
"""The resource lives at a different, canonical path."""
def __init__(self, location: str) -> None:
super().__init__(location)
self.location = location
@dataclass
class Document:
"""Every rendered format for one Markdown file.
All three renderings are produced together and cached as a unit, keyed
by the source file's (mtime, size) -- one parse per change, not one per
request or one per format.
"""
gemtext: bytes
xhtml: bytes
wml_index: bytes # the deck served at the document's own URL
wml_cards: Dict[str, bytes] = field(default_factory=dict) # deck_per_card only
cache_control: Optional[int] = None
@dataclass
class RawFile:
data: bytes
mime: str
@dataclass
class WmlCard:
"""A WML-only card sub-path, e.g. /trail/weather under deck_per_card.
Gemtext and WML-over-Spartan never address a card directly, and HTTP
only does when negotiation actually picks WML -- that decision needs
the request's Accept header, which Site can't see. So resolve() hands
this back instead of a Document, and every caller that isn't serving
WML itself must redirect to `parent_url` rather than substitute some
other field of it.
"""
wml: bytes
parent_url: str
cache_control: Optional[int]
@dataclass
class _CacheEntry:
mtime: float
size: int
document: Document
class Site:
"""A folder of Markdown, rendered to gemtext/WML/XHTML-MP on request."""
def __init__(self, root: Path) -> None:
self.root = root.resolve()
self._cache: Dict[Path, _CacheEntry] = {}
self._lock = threading.RLock()
def resolve(self, url_path: str) -> Union[Document, RawFile, WmlCard]:
"""Resolve a request path, raising NotFound or Redirect as needed."""
clean = self._clean_path(url_path)
source = self._file_for(clean)
if source is not None:
if clean.endswith(".md"):
# /foo.md always redirects to its canonical extensionless
# URL -- the same root-relative link then works identically
# from a Gemini, Spartan, or HTTP client.
raise Redirect(self._url_for(source))
if source.suffix != ".md":
return RawFile(data=source.read_bytes(), mime=self._mime_for(source))
return self._render(source)
# Not a literal file or directory index: maybe a WML card sub-path
# of a deck_per_card document, e.g. /trail/weather for trail.md's
# "weather" card. A real file or directory always wins over this
# interpretation, checked above first. Requiring a real separator
# (not just a truthy `card`) matters: rpartition on a bare
# top-level segment like "nope" returns sep="" and parent="", which
# would otherwise look up "" -> the root index and misinterpret any
# unresolvable top-level path as one of its cards.
parent, sep, card = clean.rpartition("/")
if sep and card:
parent_source = self._file_for(parent)
if parent_source is not None and parent_source.suffix == ".md":
document = self._render(parent_source)
if document.wml_cards:
# A real deck_per_card document: an unmatched card name
# still redirects to it, since the card URL space
# exists here, just not under that particular slug.
# A plain document with no cards at all falls through
# to NotFound below instead -- /about/whatever is not
# a card sub-path just because /about exists.
if card in document.wml_cards:
return WmlCard(
wml=document.wml_cards[card],
parent_url=self._url_for(parent_source),
cache_control=document.cache_control,
)
raise Redirect(self._url_for(parent_source))
raise NotFound(url_path)
# -- Path resolution --------------------------------------------------
def _clean_path(self, url_path: str) -> str:
"""Normalize to a root-relative path with no leading/trailing slash.
A leading-slash `posixpath.normpath` clamps any amount of `..` at
the root -- `/../../etc/passwd` normalizes to `/etc/passwd`, never
escaping above `/` -- so this alone can't be tricked into a
traversal string; `_safe_file`'s symlink check is the second,
independent layer that guards the filesystem side of that.
"""
url_path = url_path.split("?", 1)[0].split("#", 1)[0]
# Decode before normalizing: an encoded traversal segment like
# "%2e%2e" must become a real ".." here so normpath's clamping
# actually applies to it, rather than silently missing a file.
url_path = unquote(url_path)
if not url_path.startswith("/"):
url_path = "/" + url_path
normalized = posixpath.normpath(url_path)
clean = normalized.lstrip("/")
if clean == ".":
clean = ""
if any(part.startswith(".") for part in clean.split("/") if part):
raise NotFound(url_path)
return clean
def _file_for(self, clean: str) -> Optional[Path]:
if clean == "":
candidate = self.root / "index.md"
return candidate if self._safe_file(candidate) else None
base = self.root / clean
if "." in Path(clean).name:
# Already has an extension (.md, .png, ...): resolve literally,
# never auto-append .md to an already-named file.
return base if self._safe_file(base) else None
md_candidate = base.with_suffix(".md")
if self._safe_file(md_candidate):
return md_candidate
index_candidate = base / "index.md"
if self._safe_file(index_candidate):
return index_candidate
return None
def _safe_file(self, path: Path) -> bool:
if not path.is_file():
return False
try:
resolved = path.resolve(strict=True)
except OSError:
return False
return resolved.is_relative_to(self.root)
def _url_for(self, source: Path) -> str:
rel = source.relative_to(self.root)
if rel.name == "index.md":
parent = rel.parent.as_posix()
return "/" if parent in (".", "") else f"/{parent}/"
return "/" + rel.with_suffix("").as_posix()
def _mime_for(self, path: Path) -> str:
mime, _ = mimetypes.guess_type(path.name)
return mime or "application/octet-stream"
# -- Rendering ----------------------------------------------------------
def _render(self, source: Path) -> Document:
stat = source.stat()
with self._lock:
cached = self._cache.get(source)
if cached is not None and cached.mtime == stat.st_mtime and cached.size == stat.st_size:
return cached.document
document = self._render_uncached(source)
with self._lock:
self._cache[source] = _CacheEntry(stat.st_mtime, stat.st_size, document)
return document
def _render_uncached(self, source: Path) -> Document:
lines = source.read_text(encoding="utf-8").splitlines(keepends=True)
base_path = source.parent
wap_fields, wap_body = wapdown_strip_frontmatter(lines)
config = DeckConfig.resolve(wap_fields)
md_fm, md_body = md2txt_parse_frontmatter(lines)
# md2txt's parser has no concept of wapdown's {.card Title} card-
# break directive -- left in, it leaks through verbatim as literal
# paragraph text. A card break already renders as nothing for
# XHTML-MP (cards are a WAP 1.x screen-budget concern a browser or
# Gemini client just scrolls past); stripping the directive line
# gives gemtext the same "continuous document" treatment.
md_body = [line for line in md_body if not CARD_BREAK_PATTERN.match(line)]
gemtext_lines = convert_markdown(
md_body, width=0, frontmatter=md_fm, renderer_name="gemini", base_path=base_path
)
gemtext = ("\n".join(gemtext_lines) + "\n").encode("utf-8")
xhtml = wapdown_convert(
wap_body, config=config, renderer_name="xhtmlmp", base_path=base_path
).encode("utf-8")
wml_output = wapdown_convert(
wap_body, config=config, renderer_name="wml", base_path=base_path
)
wml_index, wml_cards = self._split_wml_output(config, wml_output)
return Document(
gemtext=gemtext,
xhtml=xhtml,
wml_index=wml_index,
wml_cards=wml_cards,
cache_control=config.cache_control,
)
@staticmethod
def _split_wml_output(config: DeckConfig, wml_output) -> tuple[bytes, Dict[str, bytes]]:
docs = wml_output.documents
if not (config.deck_per_card and len(docs) > 1):
return docs[0].text.encode("utf-8"), {}
# The first document is always the entry point regardless of
# topology: the hub in menu mode, or the first section's own file
# in linear mode (wapdown has no hub there) -- `_emit_multi_deck`
# preserves card order, so index 0 is never ambiguous.
wml_index = docs[0].text.encode("utf-8")
wml_cards: Dict[str, bytes] = {}
for doc in docs[1:]:
name = doc.name or ""
if name.endswith(".wml"):
wml_cards[name[: -len(".wml")]] = doc.text.encode("utf-8")
return wml_index, wml_cards

28
tests/conftest.py Normal file
View file

@ -0,0 +1,28 @@
from __future__ import annotations
from pathlib import Path
import pytest
from smolweb.site import Site
@pytest.fixture
def site(tmp_path: Path) -> Site:
(tmp_path / "index.md").write_text("# Home\n\nHello.\n", encoding="utf-8")
(tmp_path / "about.md").write_text("---\ntitle: About\n---\n# About\n\nBody.\n", encoding="utf-8")
(tmp_path / "img.png").write_bytes(b"\x89PNG\r\n\x1a\nfake")
sub = tmp_path / "dir"
sub.mkdir()
(sub / "index.md").write_text("# Nested\n\nNested body.\n", encoding="utf-8")
(tmp_path / ".secret.md").write_text("# Hidden\n", encoding="utf-8")
(tmp_path / "trail.md").write_text(
"---\ntitle: Trail\ndeck_per_card: true\n---\n"
"Intro line.\n\n"
"{.card Weather}\n"
"Cold and clear. Permit fee: $5.\n\n"
"{.card Status}\n"
"- North: open\n",
encoding="utf-8",
)
return Site(tmp_path)

48
tests/test_negotiate.py Normal file
View file

@ -0,0 +1,48 @@
"""Accept-header negotiation between WML and XHTML-MP."""
from __future__ import annotations
import pytest
from smolweb.negotiate import FORMAT_HTML, FORMAT_WML, FORMAT_XHTML, negotiate
class TestWildcardNeverSelectsWml:
def test_modern_browser_accept_header_gets_html(self):
# The exact header Firefox/Chrome send: a browser must never be
# handed WML through the */* wildcard.
accept = "text/html,application/xhtml+xml;q=0.9,image/webp,*/*;q=0.8"
assert negotiate(accept) == FORMAT_HTML
def test_bare_wildcard_gets_html(self):
assert negotiate("*/*") == FORMAT_HTML
def test_no_accept_header_gets_html(self):
assert negotiate(None) == FORMAT_HTML
class TestLiteralMatches:
def test_wml_accept_header_gets_wml(self):
assert negotiate("text/vnd.wap.wml") == FORMAT_WML
def test_xhtml_mp_accept_header_gets_xhtml(self):
assert negotiate("application/vnd.wap.xhtml+xml") == FORMAT_XHTML
def test_wml_wins_when_both_present_with_higher_q(self):
accept = "text/vnd.wap.wml;q=1.0, application/vnd.wap.xhtml+xml;q=0.5"
assert negotiate(accept) == FORMAT_WML
def test_xhtml_wins_when_both_present_with_higher_q(self):
accept = "text/vnd.wap.wml;q=0.5, application/vnd.wap.xhtml+xml;q=1.0"
assert negotiate(accept) == FORMAT_XHTML
def test_unrelated_type_gets_html_fallback(self):
assert negotiate("application/json") == FORMAT_HTML
class TestFormatOverride:
@pytest.mark.parametrize("fmt", [FORMAT_WML, FORMAT_XHTML, FORMAT_HTML])
def test_override_wins_regardless_of_accept(self, fmt):
assert negotiate("*/*", format_override=fmt) == fmt
def test_invalid_override_is_ignored(self):
assert negotiate("text/vnd.wap.wml", format_override="bogus") == FORMAT_WML

141
tests/test_site.py Normal file
View file

@ -0,0 +1,141 @@
"""Path resolution: traversal safety, extension rules, and live reload."""
from __future__ import annotations
import os
from pathlib import Path
import pytest
from smolweb.site import Document, NotFound, RawFile, Redirect, Site, WmlCard
class TestPathTraversal:
@pytest.mark.parametrize(
"path",
[
"/../../etc/passwd",
"/../../../../../../etc/passwd",
"/foo/../../etc/passwd",
"/%2e%2e/etc/passwd", # percent-decoded to ".." before normalizing, then clamped
],
)
def test_traversal_attempts_are_refused(self, site: Site, path: str):
with pytest.raises(NotFound):
site.resolve(path)
def test_symlink_escaping_root_is_refused(self, tmp_path: Path):
outside = tmp_path / "outside"
outside.mkdir()
(outside / "secret.md").write_text("# Secret\n", encoding="utf-8")
root = tmp_path / "root"
root.mkdir()
(root / "escape.md").symlink_to(outside / "secret.md")
site = Site(root)
with pytest.raises(NotFound):
site.resolve("/escape")
def test_dotfile_paths_are_refused(self, site: Site):
with pytest.raises(NotFound):
site.resolve("/.secret")
class TestResolutionRules:
def test_root_serves_index(self, site: Site):
doc = site.resolve("/")
assert isinstance(doc, Document)
assert b"# Home" in doc.gemtext
def test_extensionless_path_serves_md_file(self, site: Site):
doc = site.resolve("/about")
assert isinstance(doc, Document)
assert b"# About" in doc.gemtext
def test_directory_serves_its_index(self, site: Site):
doc = site.resolve("/dir")
assert isinstance(doc, Document)
assert b"# Nested" in doc.gemtext
def test_md_extension_redirects_to_canonical_url(self, site: Site):
with pytest.raises(Redirect) as excinfo:
site.resolve("/about.md")
assert excinfo.value.location == "/about"
def test_missing_md_extension_is_not_found(self, site: Site):
with pytest.raises(NotFound):
site.resolve("/nonexistent.md")
def test_raw_file_is_served_with_mime_type(self, site: Site):
raw = site.resolve("/img.png")
assert isinstance(raw, RawFile)
assert raw.mime == "image/png"
assert raw.data.startswith(b"\x89PNG")
def test_missing_path_is_not_found(self, site: Site):
with pytest.raises(NotFound):
site.resolve("/nope")
def test_bare_unresolvable_segment_is_not_a_root_card(self, site: Site):
# Regression: rpartition("/") on a bare top-level segment like
# "nope" returns sep="" and parent="" -- which must not be
# misread as "card 'nope' of the root index".
with pytest.raises(NotFound):
site.resolve("/totally-unresolvable-segment")
class TestWmlCardUrls:
def test_card_subpath_returns_wml_card(self, site: Site):
result = site.resolve("/trail/weather")
assert isinstance(result, WmlCard)
assert b"Cold and clear" in result.wml
assert b"$$5" in result.wml # WML's own literal-$ escaping, correctly applied
def test_unknown_card_redirects_to_parent(self, site: Site):
with pytest.raises(Redirect) as excinfo:
site.resolve("/trail/nonexistent-card")
assert excinfo.value.location == "/trail"
def test_card_url_of_non_deck_per_card_document_is_not_found(self, site: Site):
# about.md exists but never opted into deck_per_card, so it has no
# wml_cards at all -- /about/whatever must not resolve as a card.
with pytest.raises(NotFound):
site.resolve("/about/whatever")
def test_directive_line_does_not_leak_into_gemtext(self, site: Site):
# md2txt's parser has no concept of wapdown's {.card} directive; it
# must be stripped before gemtext rendering, not passed through as
# literal paragraph text.
doc = site.resolve("/trail")
assert isinstance(doc, Document)
assert b"{.card" not in doc.gemtext
assert b"Cold and clear" in doc.gemtext
assert b"North: open" in doc.gemtext
def test_directive_line_does_not_leak_into_xhtml(self, site: Site):
doc = site.resolve("/trail")
assert b"{.card" not in doc.xhtml
class TestLiveReload:
def test_edited_file_is_rerendered(self, tmp_path: Path):
target = tmp_path / "index.md"
target.write_text("# First\n", encoding="utf-8")
site = Site(tmp_path)
first = site.resolve("/")
assert b"# First" in first.gemtext
# mtime granularity on some filesystems is coarse; force a change.
new_time = target.stat().st_mtime + 5
target.write_text("# Second\n", encoding="utf-8")
os.utime(target, (new_time, new_time))
second = site.resolve("/")
assert b"# Second" in second.gemtext
assert b"# First" not in second.gemtext
def test_unchanged_file_reuses_cache(self, tmp_path: Path):
target = tmp_path / "index.md"
target.write_text("# Same\n", encoding="utf-8")
site = Site(tmp_path)
first = site.resolve("/")
second = site.resolve("/")
assert first is second

135
uv.lock generated Normal file
View file

@ -0,0 +1,135 @@
version = 1
revision = 3
requires-python = ">=3.13"
[[package]]
name = "colorama"
version = "0.4.6"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/d8/53/6f443c9a4a8358a93a6792e2acffb9d9d5cb0a5cfd8802644b7b1c9a02e4/colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44", size = 27697, upload-time = "2022-10-25T02:36:22.414Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" },
]
[[package]]
name = "iniconfig"
version = "2.3.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/72/34/14ca021ce8e5dfedc35312d08ba8bf51fdd999c576889fc2c24cb97f4f10/iniconfig-2.3.0.tar.gz", hash = "sha256:c76315c77db068650d49c5b56314774a7804df16fee4402c1f19d6d15d8c4730", size = 20503, upload-time = "2025-10-18T21:55:43.219Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/cb/b1/3846dd7f199d53cb17f49cba7e651e9ce294d8497c8c150530ed11865bb8/iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12", size = 7484, upload-time = "2025-10-18T21:55:41.639Z" },
]
[[package]]
name = "md2txt"
version = "0.1.0"
source = { editable = "../md2txt" }
dependencies = [
{ name = "pyfiglet" },
{ name = "pyphen" },
]
[package.metadata]
requires-dist = [
{ name = "pyfiglet", specifier = ">=0.8.0" },
{ name = "pyphen", specifier = ">=0.17.2" },
]
[package.metadata.requires-dev]
dev = [{ name = "pytest", specifier = ">=8.0" }]
[[package]]
name = "packaging"
version = "26.3"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/7d/fa/3944b40b07da9ce895c0e6303a5ab7d53da063554f534556b134a54d6093/packaging-26.3.tar.gz", hash = "sha256:94edc256424af38762eb31306eed28beb9f0efc50a8837492c9d6fd6004aed79", size = 313412, upload-time = "2026-08-04T18:15:28.737Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/63/34/ba1c580383c9eada3711951fef0795c80b829a078d72188184bcab9dd527/packaging-26.3-py3-none-any.whl", hash = "sha256:d7193f7c8e4e93f444fde0262bf90af30e16fa0ad0ad44cb553c87339b23cd1c", size = 129956, upload-time = "2026-08-04T18:15:27.159Z" },
]
[[package]]
name = "pluggy"
version = "1.6.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/f9/e2/3e91f31a7d2b083fe6ef3fa267035b518369d9511ffab804f839851d2779/pluggy-1.6.0.tar.gz", hash = "sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3", size = 69412, upload-time = "2025-05-15T12:30:07.975Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/54/20/4d324d65cc6d9205fabedc306948156824eb9f0ee1633355a8f7ec5c66bf/pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746", size = 20538, upload-time = "2025-05-15T12:30:06.134Z" },
]
[[package]]
name = "pyfiglet"
version = "1.0.4"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/c8/e3/0a86276ad2c383ce08d76110a8eec2fe22e7051c4b8ba3fa163a0b08c428/pyfiglet-1.0.4.tar.gz", hash = "sha256:db9c9940ed1bf3048deff534ed52ff2dafbbc2cd7610b17bb5eca1df6d4278ef", size = 1560615, upload-time = "2025-08-15T18:32:47.302Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/9f/5c/fe9f95abd5eaedfa69f31e450f7e2768bef121dbdf25bcddee2cd3087a16/pyfiglet-1.0.4-py3-none-any.whl", hash = "sha256:65b57b7a8e1dff8a67dc8e940a117238661d5e14c3e49121032bd404d9b2b39f", size = 1806118, upload-time = "2025-08-15T18:32:45.556Z" },
]
[[package]]
name = "pygments"
version = "2.21.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/49/2e/ced460408999b33da6b31b0021b0f37d329e202d4169aeb164493778f25b/pygments-2.21.0.tar.gz", hash = "sha256:610ca751c9bc2492b38eb9a38a7fbc93edbbb2d7182edaf34e66ae493dee5c8c", size = 5005329, upload-time = "2026-08-17T08:02:48.824Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/71/46/17f022dd3e953bf20a04a028a21ec746d942f8d2af30fa0f124fa0e6a684/pygments-2.21.0-py3-none-any.whl", hash = "sha256:2363c69b61c4a97c838da3b130dcd6468f4848992b21a82f2a63ec34377137d9", size = 1250147, upload-time = "2026-08-17T08:02:44.912Z" },
]
[[package]]
name = "pyphen"
version = "0.18.1"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/94/47/8430452269cd28863d73b903d07d329d058cf762527ff211b3864ba61fc7/pyphen-0.18.1.tar.gz", hash = "sha256:dbae6fbbe4f01cb206108b43573d857c67107be9d0e38eb1b08d6fa2210634a7", size = 2116411, upload-time = "2026-08-14T11:30:12.083Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/ec/1d/23801cf008f71575f0a4800463f349afa5f04b27fe783178a45cdc4d5edf/pyphen-0.18.1-py3-none-any.whl", hash = "sha256:0aa9051e15928cecadd4c632cea0258ba57215b2a197a39baa46abcdb0f47e84", size = 2116143, upload-time = "2026-08-14T11:30:10.428Z" },
]
[[package]]
name = "pytest"
version = "9.1.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "colorama", marker = "sys_platform == 'win32'" },
{ name = "iniconfig" },
{ name = "packaging" },
{ name = "pluggy" },
{ name = "pygments" },
]
sdist = { url = "https://files.pythonhosted.org/packages/e4/47/b9efed96c114afcfa3c9d3fe98a76a1d14c74a9e266d397cf6eb64be5e01/pytest-9.1.1.tar.gz", hash = "sha256:1088fbde8f2b49d95a549a195707afa7a76a3ce9bcadc26b6d71f0ffda5fe313", size = 1636369, upload-time = "2026-06-19T10:58:32.857Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/24/25/1de2678b631f5a49215c6c96fff41ba892b0a34df68d6d80292b1b48aa7f/pytest-9.1.1-py3-none-any.whl", hash = "sha256:37a86b45efb9a47a61a36449063e8e18d0cab3161329fc099eb21783169c4f0c", size = 386536, upload-time = "2026-06-19T10:58:31.347Z" },
]
[[package]]
name = "smolweb"
version = "0.1.0"
source = { editable = "." }
dependencies = [
{ name = "md2txt" },
{ name = "wapdown" },
]
[package.dev-dependencies]
dev = [
{ name = "pytest" },
]
[package.metadata]
requires-dist = [
{ name = "md2txt", editable = "../md2txt" },
{ name = "wapdown", editable = "../wapdown" },
]
[package.metadata.requires-dev]
dev = [{ name = "pytest", specifier = ">=8.0" }]
[[package]]
name = "wapdown"
version = "0.1.0"
source = { editable = "../wapdown" }
[package.metadata]
requires-dist = [{ name = "lxml", marker = "extra == 'dtd'", specifier = ">=5.0" }]
provides-extras = ["dtd"]
[package.metadata.requires-dev]
dev = [{ name = "pytest", specifier = ">=8.0" }]