Compare commits
8 commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c08b03db7b | ||
|
|
3e659df9dd | ||
|
|
c9bde15f5c | ||
|
|
1b47145522 | ||
|
|
603f29b0f0 | ||
|
|
bb7a5111a1 | ||
|
|
19e7cfaca3 | ||
|
|
36faa4fb93 |
34 changed files with 1331 additions and 456 deletions
6
.dockerignore
Normal file
6
.dockerignore
Normal file
|
|
@ -0,0 +1,6 @@
|
||||||
|
/target
|
||||||
|
/result
|
||||||
|
/result-*
|
||||||
|
.git
|
||||||
|
.jj
|
||||||
|
.devbox
|
||||||
54
.forgejo/workflows/ci.yml
Normal file
54
.forgejo/workflows/ci.yml
Normal file
|
|
@ -0,0 +1,54 @@
|
||||||
|
name: CI
|
||||||
|
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
branches: [main]
|
||||||
|
pull_request:
|
||||||
|
workflow_dispatch:
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
check:
|
||||||
|
runs-on: linux-x86_64
|
||||||
|
steps:
|
||||||
|
- uses: https://code.forgejo.org/actions/checkout@v4
|
||||||
|
- uses: https://code.forgejo.org/actions/cache@v4
|
||||||
|
with:
|
||||||
|
path: |
|
||||||
|
~/.cargo/registry
|
||||||
|
target
|
||||||
|
key: cargo-linux-${{ hashFiles('Cargo.lock') }}
|
||||||
|
restore-keys: cargo-linux-
|
||||||
|
# Forgejo auto-injects GITHUB_TOKEN (GH Actions compatibility), scoped
|
||||||
|
# to this Forgejo instance. Nix auto-detects that env var and assumes
|
||||||
|
# it's a github.com credential, so it sends it to api.github.com when
|
||||||
|
# fetching nixpkgs tarballs for devbox - which rejects it with 401.
|
||||||
|
# Clearing it here falls back to unauthenticated (fine for public
|
||||||
|
# nixpkgs fetches, same as this succeeds locally with no token set).
|
||||||
|
- run: devbox run check
|
||||||
|
env:
|
||||||
|
GITHUB_TOKEN: ""
|
||||||
|
|
||||||
|
# Ships the static musl binary (flake.nix packages.static) the way
|
||||||
|
# `nix run .#release-static` already does from a dev machine: one release
|
||||||
|
# per commit landing on main, tagged by its short hash, created if absent
|
||||||
|
# and with its asset replaced if present. Gated on `check` so a failing
|
||||||
|
# build or test on main is never published - unlike a tag push, which only
|
||||||
|
# happens once a human has already decided a commit is good, a push to
|
||||||
|
# main is not itself that decision.
|
||||||
|
#
|
||||||
|
# `nix build`/`nix run` draw from the Nix store and its binary cache, not
|
||||||
|
# from ~/.cargo/registry or ./target, so there is nothing here for the
|
||||||
|
# `check` job's cache to help with.
|
||||||
|
release:
|
||||||
|
needs: check
|
||||||
|
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
|
||||||
|
runs-on: linux-x86_64
|
||||||
|
steps:
|
||||||
|
- uses: https://code.forgejo.org/actions/checkout@v4
|
||||||
|
# forge.token is Forgejo Actions' auto-injected, repo-scoped
|
||||||
|
# credential - the CI equivalent of running this app locally with
|
||||||
|
# FORGEJO_TOKEN set in .env.
|
||||||
|
- run: nix run .#release-static
|
||||||
|
env:
|
||||||
|
FORGEJO_TOKEN: ${{ forge.token }}
|
||||||
|
GITHUB_TOKEN: ""
|
||||||
|
|
@ -7,4 +7,8 @@ This format is based on [AI-DECLARATION.md](https://ai-declaration.md/en/0.1.2).
|
||||||
|
|
||||||
## Notes
|
## Notes
|
||||||
|
|
||||||
-
|
- **The idea was mine, and so were the calls.** itsybitsy was conceived as a Rust server around *md2txt* and *wapdown*, my own Python renderers, with three requirements fixed before any code existed: one process serving several domains, output formats as plugins, configuration in TOML per directory. Porting the renderers rather than shelling out to them, one crate per format behind a cargo feature, flat configuration with no inheritance, which protocols shipped and in what order, TLS terminated in front of Gemini: the model proposed and argued for each; I decided.
|
||||||
|
|
||||||
|
- **The model wrote everything.** *Claude* did the Rust, the test suites and the documentation, in milestones that each ended in something runnable and verified. Parity with the originals came from diffing real output against *md2txt*, *pyfiglet* and *wapdown*. Costs of cache memory and a TLS stack were measured before implementing these features.
|
||||||
|
|
||||||
|
- **The audit found real bugs.** Three defects, the worst of them a document that could kill the whole process and every virtual host with it. All three are fixed and pinned by regression tests. See the security audit section of the README for more info.
|
||||||
117
AUDIT.md
117
AUDIT.md
|
|
@ -1,117 +0,0 @@
|
||||||
# Security and robustness audit — 2026-10-04
|
|
||||||
|
|
||||||
Audit of itsybitsy at commit `c29364d8` ("serve gemini behind a tls terminator"), covering integration behaviour, adversarial input and load. Three defects were found and fixed in `49a63503`; the remaining items are recorded below rather than fixed, because each is a design decision rather than a bug.
|
|
||||||
|
|
||||||
The headline result is the first finding: a single content file could abort the whole process, taking every virtual host with it, in a way the existing panic handler could not catch.
|
|
||||||
|
|
||||||
## Method
|
|
||||||
|
|
||||||
A release binary built with `--features "wml figlet hyphenation"` was run against a purpose-built content tree with two sites and seven listeners — HTTP (three: negotiating, single-format, and one with `max_connections = 5`), Spartan, Gemini, Nex and Gopher. Requests were sent as raw bytes over real sockets, so each protocol's own parser was exercised rather than a client library's idea of it.
|
|
||||||
|
|
||||||
The tree deliberately contained things a well-behaved tree would not: a canary file outside both roots, symlinks pointing to that file, to `/etc/passwd` and to `/tmp`, a hidden page, files whose names carry CR LF and tab bytes, documents nested thousands of levels deep, an 8 MiB single line, a 200 000-row table, a 2.5 GB Markdown file, include cycles, a 40-deep include chain, and a 2²⁴ diamond include fan-out.
|
|
||||||
|
|
||||||
138 checks ran in four phases:
|
|
||||||
|
|
||||||
| Phase | Checks | Covers |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| Integration | 59 | All five protocols end to end, negotiation, virtual hosting, every output format, card sub-documents, live reload |
|
|
||||||
| Containment | 16 | 27 traversal encodings × 5 protocols, dotfiles, symlink escapes, cross-site isolation |
|
|
||||||
| Protocol abuse | 23 | Response splitting, request smuggling, header flooding, `Host` abuse, request caps, upload abuse, slow clients |
|
|
||||||
| Exhaustion | 25 | Nesting depth, oversized documents, include bombs, include and art containment |
|
|
||||||
| Load | 15 | Concurrency, throughput, the connection cap, memory growth |
|
|
||||||
|
|
||||||
Everything of lasting value is now in the repository's own suite as 13 regression tests. The ad-hoc harness was not kept.
|
|
||||||
|
|
||||||
## Findings
|
|
||||||
|
|
||||||
### 1. Deep nesting aborted the process — critical, fixed
|
|
||||||
|
|
||||||
A served document containing 10 000 nested block quotes killed the server outright:
|
|
||||||
|
|
||||||
```
|
|
||||||
thread '<unknown>' has overflowed its stack
|
|
||||||
fatal runtime error: stack overflow, aborting
|
|
||||||
[exited with code 134]
|
|
||||||
```
|
|
||||||
|
|
||||||
Severity comes from three things together. A stack overflow raises SIGABRT rather than unwinding, so the `catch_unwind` at the handler boundary — which exists precisely so one bad document cannot take the process down — could not contain it. One process serves every configured site, so the failure is not scoped to the site whose content caused it. And nothing restarts the process on its own.
|
|
||||||
|
|
||||||
The Markdown parser was not at fault: pulldown-cmark handled 100 000 levels without trouble, being iterative. The recursion was ours, in three separate walks over the parsed document — the directive pass, each renderer's block walk, and the derived `Drop` for `Block`. The depth at which it died therefore depended on available stack, not on any limit in the code: a debug test thread died at 1 000 levels where the release server survived to 10 000.
|
|
||||||
|
|
||||||
**Fix.** A single cap at the parse boundary (`MAX_NESTING = 100` in `core/src/parse.rs`), checked where the nesting depth is already explicit as the builder's open-container stacks. Rewriting three recursive walks would have been the alternative; capping once means none of them can see a tree deep enough to matter, including any walk added later. An over-deep document is refused whole — HTTP 500, Gemini 40, Spartan 5 — rather than truncated at the limit, which would serve a page missing most of its content with nothing to indicate it.
|
|
||||||
|
|
||||||
The limit is two orders of magnitude above real prose. Ten levels of nesting is already unusual.
|
|
||||||
|
|
||||||
A secondary result worth recording, because it stops someone adding a limit that is not needed: inline nesting cannot run away. 150 consecutive `*` produce 75 levels of emphasis and 150 consecutive `[` produce one, because pulldown-cmark pairs delimiters and forbids links from nesting at all. Only block containers are unbounded. A test pins this.
|
|
||||||
|
|
||||||
### 2. A filename could inject a response header — moderate, fixed
|
|
||||||
|
|
||||||
```
|
|
||||||
GET /ev%0d%0aX-Injected:%20yes.md
|
|
||||||
|
|
||||||
HTTP/1.1 301 Moved Permanently
|
|
||||||
Location: /ev
|
|
||||||
X-Injected: yes <- injected
|
|
||||||
Connection: close
|
|
||||||
```
|
|
||||||
|
|
||||||
`url_for` interpolated the resolved filename into a URL with no encoding, and that URL is emitted as a redirect target. Spartan (`3 /ev\r\nX-Injected: yes`) and Gemini (`31 ...`) terminate their status lines the same way and split identically — one defect, three protocols.
|
|
||||||
|
|
||||||
Exploiting it requires a file named `ev\r\nX-Injected: yes.md`, which is legal on Linux. Under the current threat model the content tree is author-controlled, which is what keeps this moderate rather than critical; it becomes remotely reachable the moment any part of a tree accepts contributions from someone who is not the operator.
|
|
||||||
|
|
||||||
**Fix.** `url_for` now percent-encodes everything outside RFC 3986's unreserved set, leaving `/` as the separator. `clean_path` already decodes on the way in, so URLs round-trip, and a test asserts that. This also repaired a quieter bug that had not been noticed: filenames containing a space, `?`, `#`, `%` or non-ASCII characters previously produced URLs that were wrong or truncated. `/my%20notes`, `/a%23b` and `/caf%C3%A9` now resolve.
|
|
||||||
|
|
||||||
### 3. Large files were read before being rejected — low, fixed
|
|
||||||
|
|
||||||
`preprocess::expand` called `fs::read_to_string` on each file, allocating it in full, and only then applied the 8 MiB expansion cap. A 2.5 GB Markdown file in the tree therefore cost 2.5 GB of transient allocation per request, despite no more than 8 MiB of it ever being usable. With the default `max_connections = 256`, concurrent requests multiply that.
|
|
||||||
|
|
||||||
**Fix.** The file's length is checked before it is opened, and the read itself goes through a capped reader so the bound holds even if the file grew since the check.
|
|
||||||
|
|
||||||
## Results after the fixes
|
|
||||||
|
|
||||||
Containment held everywhere. 27 traversal encodings — double-encoded, overlong UTF-8, backslash, NUL byte, `..;/`, absolute paths — were tried against all five protocols, 135 attempts, and none returned the canary or `/etc/passwd`. Symlinks out of the root were refused on the resolved path, not the requested one. Neither site could read the other's tree, and the server configuration file, which sits outside both roots, was unreachable by every spelling tried.
|
|
||||||
|
|
||||||
No request shape broke a handler. Absolute-form and authority-form targets, missing and bogus HTTP versions, bare LF line endings, NUL bytes, invalid UTF-8, a 5 000-byte method, `Transfer-Encoding` with `Content-Length`, and a pipelined second request all produced exactly one response per connection and left the server serving. A `Host` carrying CR LF, NUL, 5 000 bytes or non-ASCII injected nothing. Every protocol enforced its request cap: Gemini 1026 bytes, Spartan 4096, Nex 2048, Gopher 512. A Spartan upload claiming 10 GB was refused in under three seconds rather than drained.
|
|
||||||
|
|
||||||
Include bombs were all bounded: the cycle, the self-reference, the 40-deep chain and the 2²⁴ diamond fan-out each produced an error and a live server, the fan-out caught by the byte cap that a per-stack cycle check cannot see.
|
|
||||||
|
|
||||||
Load behaved well:
|
|
||||||
|
|
||||||
| Measurement | Result |
|
|
||||||
| --- | --- |
|
|
||||||
| 32 concurrent clients, 3 200 requests | 8 464 req/s, 0 errors, p50 3.5 ms, p99 7.7 ms |
|
|
||||||
| 64 concurrent clients | 8 009 req/s, 0 errors |
|
|
||||||
| Mixed load across all five protocols | 9 248 req/s, 0 errors |
|
|
||||||
| 6 400 requests, leak check | RSS flat at 122 660 KiB throughout |
|
|
||||||
| 8 concurrent readers of a 4 MiB page | RSS delta 0 KiB — one shared cached copy |
|
|
||||||
| Connection cap of 5, 10 excess connections | 10/10 closed immediately; 13 threads while held, 8 when idle |
|
|
||||||
|
|
||||||
## Outstanding
|
|
||||||
|
|
||||||
**The render cache is unbounded.** Measured from a cold start: 105 pages totalling 16 588 KiB of Markdown, rendered into five formats, grew RSS from 3 876 KiB to 106 092 KiB — a cache cost of **6.2× the source size**. Memory is flat under repeated requests, so this is growth by distinct page visited rather than a leak, and at capsule scale it is fine. 6.2× is the multiplier to size `cache_max_bytes` with when LRU eviction lands.
|
|
||||||
|
|
||||||
**An oversized header produces a TCP reset instead of the 400.** The server writes the status and closes while the client is still sending, so the client may never read the response. nginx behaves similarly. Fixing it means draining a bounded amount before closing, as the Spartan listener already does for uploads.
|
|
||||||
|
|
||||||
**`ListenerSpec.width` is accepted and ignored.** A per-listener width cannot work until the render cache is keyed on `(format, width)` rather than format alone, so this is not a one-line change. Either delete the key or key the cache.
|
|
||||||
|
|
||||||
**No graceful shutdown.** SIGTERM terminates mid-response. Queued for the operations milestone along with SIGHUP reload.
|
|
||||||
|
|
||||||
## What this audit did not cover
|
|
||||||
|
|
||||||
The Gemini listener expects TLS to be terminated in front of it, and **that leg was never tested** — no `stunnel`, `ghostunnel` or `openssl` was available on the audit machine and no network to fetch one. Only the plaintext protocol behind the terminator was exercised. The `stunnel` configuration in the README is written from its documentation, not from a run.
|
|
||||||
|
|
||||||
Also out of scope: no coverage-guided fuzzing of the Markdown parser or the protocol readers, which is the right tool for the input-handling code and would likely find more than hand-written cases do. All load testing was over loopback on one machine, so the figures measure the server rather than any network. No audit of the dependency tree for known advisories. The threat model throughout assumes the content tree is author-controlled; findings 1 and 2 both become materially more serious if that stops being true, and that is the assumption most worth revisiting.
|
|
||||||
|
|
||||||
## Reproducing
|
|
||||||
|
|
||||||
The three findings are pinned by tests in the repository:
|
|
||||||
|
|
||||||
- `core/src/parse.rs` — `nesting_past_the_cap_is_refused_rather_than_overflowing_the_stack`, `deeply_repeated_inline_markers_are_bounded_by_the_parser_itself`
|
|
||||||
- `core/src/path.rs` — `url_for_escapes_bytes_that_would_end_a_response_header`, `an_encoded_url_still_resolves_back_to_the_same_path`
|
|
||||||
- `core/src/preprocess.rs` — `an_oversized_file_is_refused_without_being_read_into_memory`
|
|
||||||
- `bin/tests/listeners.rs` — `a_document_nested_past_the_cap_is_an_error_and_the_server_survives`, `an_injected_redirect_target_is_escaped_on_every_protocol`
|
|
||||||
|
|
||||||
```bash
|
|
||||||
devbox run check # 320 tests
|
|
||||||
devbox run -- cargo test --workspace --features "wml figlet hyphenation" # 375 tests
|
|
||||||
```
|
|
||||||
29
Dockerfile
Normal file
29
Dockerfile
Normal file
|
|
@ -0,0 +1,29 @@
|
||||||
|
# Multi-stage build: a slim Rust toolchain compiles the release binary, and a
|
||||||
|
# slim Debian runtime carries it with no toolchain. The default entrypoint
|
||||||
|
# serves the demo site from /srv/itsybitsy; mount a real server file and
|
||||||
|
# content roots over those paths to serve something else.
|
||||||
|
|
||||||
|
FROM rust:1.97-slim AS build
|
||||||
|
WORKDIR /src
|
||||||
|
COPY Cargo.toml Cargo.lock ./
|
||||||
|
COPY core core
|
||||||
|
COPY gemtext gemtext
|
||||||
|
COPY wap wap
|
||||||
|
COPY text text
|
||||||
|
COPY bin bin
|
||||||
|
# The demo config serves wml decks, which the `wml` feature adds to the wap
|
||||||
|
# format; the listeners the config names are validated against the registry.
|
||||||
|
RUN cargo build --release --locked --package itsybitsy --features wml
|
||||||
|
|
||||||
|
FROM debian:bookworm-slim
|
||||||
|
RUN useradd --uid 1000 --create-home itsybitsy
|
||||||
|
WORKDIR /srv/itsybitsy
|
||||||
|
COPY --from=build /src/target/release/itsybitsy /usr/local/bin/itsybitsy
|
||||||
|
# The container config binds 0.0.0.0, because a loopback bind would leave
|
||||||
|
# every published port unreachable from outside the container. --chmod keeps
|
||||||
|
# the file readable by the runtime user whatever the checkout's own modes are.
|
||||||
|
COPY --chmod=644 docker/itsybitsy.toml ./itsybitsy.toml
|
||||||
|
COPY content ./content
|
||||||
|
USER itsybitsy
|
||||||
|
EXPOSE 8080 3000 1900 1965 7070
|
||||||
|
ENTRYPOINT ["itsybitsy", "--config", "/srv/itsybitsy/itsybitsy.toml"]
|
||||||
236
README.md
236
README.md
|
|
@ -1,32 +1,24 @@
|
||||||
# 🕸 itsybitsy
|
# 🕸 itsybitsy
|
||||||
|
|
||||||
[](LICENSE) [](https://ai-declaration.md)
|
[](LICENSE) [](https://ai-declaration.md)  [](https://github.com/jetify-com/devbox/)
|
||||||
|
|
||||||
Serve folders of Markdown to different domains over HTTP, [Gemini](https://geminiprotocol.net/docs/specification.gmi), [Spartan](https://portal.mozz.us/gemini/spartan.mozz.us/), [Nex](https://nightfall.city/nex/info/specification.txt) and [Gopher](https://www.rfc-editor.org/rfc/rfc1436), rendered on request with no build step.
|
|
||||||
|
|
||||||
A Rust re-implementation of [smolweb](https://code.randogoth.com/randogoth/smolweb), which serves one folder from one process with its output formats hardcoded and its configuration in per-file frontmatter. itsybitsy changes all three: one process serves many domains, each output format is a separate crate behind a cargo feature, and configuration lives in a TOML file per content directory.
|
Serve folders of Markdown to different domains over HTTP, [Gemini](https://geminiprotocol.net/docs/specification.gmi), [Spartan](https://portal.mozz.us/gemini/spartan.mozz.us/), [Nex](https://nightfall.city/nex/info/specification.txt) and [Gopher](https://www.rfc-editor.org/rfc/rfc1436), rendered on request with no build step. One process serves many virtual hosts, each output format is a separate crate behind a cargo feature, and configuration lives in TOML files. Gemini expects TLS to be terminated in front of it; that is the one deployment requirement itsybitsy does not satisfy by itself.
|
||||||
|
|
||||||
## Status
|
The output formats are Rust ports of previous projects: the standalone Markdown to esoteric text format converter [md2txt](https://code.randogoth.com/randogoth/md2txt) and a spinoff from that for WML, [wapdown](https://code.randogoth.com/randogoth/wapdown).
|
||||||
|
|
||||||
It serves. All five protocols work across five output formats, with virtual hosting wherever the protocol carries a hostname. Gemini expects TLS to be terminated in front of it, which is the one deployment requirement itsybitsy does not satisfy by itself.
|
## Deployments
|
||||||
|
|
||||||
| Area | State |
|
Itsybitsy can be seen in action running the Gemini/Spartan/Nex/Gopher content for https://smol.place, the Gemini/Spartan capsules for https://randogoth.com and https://wap.randogoth.com
|
||||||
| --- | --- |
|
|
||||||
| Server configuration, virtual hosts, `--check` | implemented |
|
|
||||||
| Path resolution, traversal containment, live reload | implemented |
|
|
||||||
| Markdown parsing, includes, ASCII art, card breaks | implemented |
|
|
||||||
| gemtext, XHTML-MP and HTML output | implemented |
|
|
||||||
| HTTP, Spartan and Nex listeners, virtual hosting | implemented |
|
|
||||||
| Fixed-width text output (Nex, later Gopher) | implemented |
|
|
||||||
| FIGlet banners and hyphenation, behind features | implemented |
|
|
||||||
| WML decks and card sub-documents, behind a feature | implemented |
|
|
||||||
| Gopher, with dot-stuffing and the RFC 4266 root menu | implemented |
|
|
||||||
| Gemini, behind a TLS terminator | implemented |
|
|
||||||
| Built-in TLS, so no terminator is needed | not planned for now |
|
|
||||||
|
|
||||||
## Configuration
|
## Usage
|
||||||
|
|
||||||
Two files. A server file names the virtual hosts and the listeners:
|
```bash
|
||||||
|
itsybitsy --config /etc/itsybitsy.toml # serve
|
||||||
|
itsybitsy --config /etc/itsybitsy.toml --check # validate configuration only
|
||||||
|
```
|
||||||
|
|
||||||
|
A server file names the sites and the listeners:
|
||||||
|
|
||||||
```toml
|
```toml
|
||||||
version = 1
|
version = 1
|
||||||
|
|
@ -48,167 +40,40 @@ formats = ["gemtext"]
|
||||||
site = "smol"
|
site = "smol"
|
||||||
```
|
```
|
||||||
|
|
||||||
A page is rendered into the union of what the enabled listeners can serve, and no more: a configuration with no HTTP listener never builds any HTML. HTTP and Spartan route by the hostname the request carries, falling back to `default_site`; Nex carries none, so its listener names one site outright.
|
HTTP and Spartan route by the hostname the request carries, falling back to `default_site`; Nex and Gopher carry none, so their listener names one site outright. Every format a listener serves must be compiled in, which `--check` reports. A `.itsybitsy.toml` in a content directory sets rendering options for that directory — `[defaults]` for all its Markdown files, `[page."name.md"]` for one — with no inheritance from parent directories. Editing it re-renders on the next request.
|
||||||
|
|
||||||
A `.itsybitsy.toml` in any content directory says how the Markdown in **that directory** renders. There is no inheritance: a subdirectory without its own file uses the built-in defaults rather than its parent's. Repetition in a deep tree is the price of never having to look elsewhere to know how a page renders.
|
A site can set `mews_profile = true` to render its `xhtmlmp` and `html` output as [Mews Profile](https://mews.page/spec/) pages: the XML declaration, the XHTML-MP 1.2 doctype, the conformance marker, the viewport tag and the default stylesheet link, with raw HTML and off-site or `data:` images dropped rather than passed through. The declaration stays in the `text/html` form too, unlike an ordinary page's, because section 3.1 requires it and a validator reads whichever bytes it was served; a browser logs a warning for it and renders the page the same. The stylesheet itself is served from an embedded copy, at whatever path the link's `href` resolves to, so there is nothing to place in the content root. Sites sharing one root must agree on this setting, since they share one render cache.
|
||||||
|
|
||||||
```toml
|
A site may also set `feed` to the URL of its Atom feed, which is then declared in the head of every `xhtmlmp` and `html` page so a client can subscribe without reading the body (Mews Profile 6.2). Give it absolute: a capsule rarely carries the feed its web site publishes, so a root-relative path would point at a document this server has no file for. Sites sharing one root must agree on it too.
|
||||||
[defaults]
|
|
||||||
max_age = 3600
|
|
||||||
margin_left = 2
|
|
||||||
h1_style = "underline:="
|
|
||||||
|
|
||||||
[page."index.md"]
|
### Formats and protocols
|
||||||
title = "Notes"
|
|
||||||
max_age = 300
|
|
||||||
```
|
|
||||||
|
|
||||||
`[defaults]` applies to every Markdown file in the directory; a `[page."name.md"]` table overrides it for one file. `title` is per-page by nature, so setting it in `[defaults]` is an error; left unset, it is derived from the page's first level-1 heading.
|
| Format | Served as | Behind feature |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `gemtext` | `text/gemini` | default |
|
||||||
|
| `xhtmlmp`, `html` | WAP and desktop markup | default |
|
||||||
|
| `text` | `text/plain`, wrapped to 80 columns | default |
|
||||||
|
| `wml` | WML 1.3 decks for WAP 1.x handsets | `wml` |
|
||||||
|
| FIGlet banners, hyphenation for `text` | | `figlet`, `hyphenation` |
|
||||||
|
|
||||||
The fixed-width text format reads `margin_left`, `margin_right`, `paragraph_spacing`, `h1_style` through `h6_style`, `blockquote_bars`, `list_indent`, `code_block_line_numbers` and `wrap_code_blocks`. A heading style is `underline`, `underline:<char>`, `figlet`, `figlet:<font>`, `markers` or `plain`; by default the first three levels are underlined with `=`, `-` and `~`, and deeper ones keep their `#` markers, since underline characters run out before heading levels do. Editing the file re-renders that directory's pages on the next request, and a file that fails to parse makes them error rather than silently falling back to defaults.
|
HTTP negotiates formats from `Accept` and `?format=<id>` overrides it. Two decorative text features are off by default to save binary size: `cargo build --release --features "figlet hyphenation"`.
|
||||||
|
|
||||||
Because the name begins with a dot, the same rule that keeps `.secret.md` out of the URL space keeps this file out of it too.
|
### Markdown
|
||||||
|
|
||||||
`--check` validates the configuration and reports the routing it resolved, without binding a port:
|
Markdown is parsed once per document and rendered to every format. Everything itsybitsy adds to CommonMark stays portable:
|
||||||
|
|
||||||
```console
|
|
||||||
$ itsybitsy --config /etc/itsybitsy.toml --check
|
|
||||||
sites:
|
|
||||||
smol: /srv/smol/content
|
|
||||||
hosts:
|
|
||||||
smol.place -> smol
|
|
||||||
www.smol.place -> smol
|
|
||||||
listeners:
|
|
||||||
web: http on 0.0.0.0:8080 [xhtmlmp, html] by host, default smol
|
|
||||||
formats rendered per page: html, xhtmlmp
|
|
||||||
```
|
|
||||||
|
|
||||||
Everything the schema cannot express is checked here rather than on first request: a format no enabled feature provides, two sites claiming one hostname, a content root that does not exist, a server file sitting inside a content root where it would be served.
|
|
||||||
|
|
||||||
## Markdown
|
|
||||||
|
|
||||||
Markdown is parsed once per document into one representation that every output format reads, rather than once per format. Everything itsybitsy adds to CommonMark is either a construct other tools already understand or invisible to them, so a document stays portable:
|
|
||||||
|
|
||||||
| Written | Means |
|
| Written | Means |
|
||||||
| --- | --- |
|
| --- | --- |
|
||||||
| `![[path]]` | splice that file's lines in here |
|
| `![[path]]` | splice that file's lines in here |
|
||||||
| `` | an image whose target is a text file, inlined verbatim |
|
| `` | an image whose target is a text file, inlined verbatim |
|
||||||
| `<!-- card Title -->` | a card divider, for formats that paginate |
|
| `<!-- card Title -->` | a card divider, for formats that paginate |
|
||||||
| `---` | an untitled divider |
|
|
||||||
| `<!-- center -->`, `<!-- right margin=4 -->` | alignment for the block that follows |
|
| `<!-- center -->`, `<!-- right margin=4 -->` | alignment for the block that follows |
|
||||||
|
| `<!-- only gemtext text -->` … `<!-- end -->` | a run of blocks only those formats get |
|
||||||
|
| `<!-- except wml -->` … `<!-- end -->` | a run of blocks every other format gets |
|
||||||
|
|
||||||
Only the include is non-standard, and it is the spelling Obsidian and its relatives established; CommonMark renders it as literal text, which is why it is the one construct handled before parsing. The rest are ordinary HTML comments and images, interpreted after parsing — so an HTML comment that is not a recognised directive stays a comment, at the cost of a misspelled one doing nothing rather than complaining.
|
A gate names format ids, not protocols: `gemtext` is what Gemini, Spartan, Nex and Gopher all serve, so there is no "only on Gemini". Gates nest, each one closes inside the quote or list item it was opened in, and one left unclosed is refused rather than run on to the end of the page. An id this build does not provide matches nothing, so `only wml` is hidden everywhere when the `wml` feature is off — and so is a misspelled id.
|
||||||
|
|
||||||
Alignment only reaches the fixed-width text formats; gemtext and HTML have no way to express it. An art label may carry its own `:center`, `:left` or `:right` token, which is removed from the label so it does not leak into the fallback text a gemtext or HTML client shows.
|
Links should be root-relative and extensionless (`[about](/about)`), so the same link resolves identically from every protocol:
|
||||||
|
|
||||||
Include and art targets resolve relative to the file holding the reference and must stay inside the content root. Expansion is bounded on three axes — nesting depth, total lines, and total bytes — because a cycle check alone does not stop a long chain, and neither stops a diamond where two branches include the same file without ever repeating one on a single path.
|
|
||||||
|
|
||||||
### Differences from smolweb
|
|
||||||
|
|
||||||
Configuration keys are not carried over verbatim either: md2txt accepts three spellings of `paragraph_spacing`, and its `cache_control` is named for the header it lands in rather than the value it holds, which here is `max_age`. Its `text` and `nex` renderers differ only in heading decoration and in whether links are inlined or numbered — and the numbered form never writes the reference list its numbers point at, so there is one text renderer here rather than two. Tables, which both of its text renderers drop entirely, are rendered as aligned columns.
|
|
||||||
|
|
||||||
wapdown's own parser produces neither tables nor lists, so its WML decks render a bullet list as literal text and drop tables entirely; both come out as real markup here, lists as marked lines and tables as `<table>`, which WML has.
|
|
||||||
|
|
||||||
Gemini diverges in three places. smolweb answers 59 (bad request) to any URL that is not `gemini://`, which tells a client its request was malformed when an `https://` URL is merely for somewhere this server does not fetch from; that is 53 (proxy request refused) here, and 59 is kept for a URL that genuinely does not parse. smolweb builds absolute redirect targets from the port it is bound to, which is wrong behind a terminator; these are relative. And smolweb terminates TLS itself with a certificate named per site, so `tls` is not a configuration key here at all — naming one is an error rather than a setting that quietly does nothing.
|
|
||||||
|
|
||||||
smolweb inherits five invented directives from its two renderer libraries: `{.card}`, `{.include}`, `![[ ]]`, `#[label](art.txt)` and MultiMarkdown attribute lists (`{: .center}`) — two incompatible brace grammars, with art alignment expressible two different ways in one line. None of it appeared in real content, so itsybitsy expresses the same capabilities with standard constructs instead. Parsing once rather than once per library also means the text formats gain setext headings and the WAP formats gain tables and ASCII art, none of which their original parser handled.
|
|
||||||
|
|
||||||
## Output formats
|
|
||||||
|
|
||||||
Each format is a crate implementing one trait, compiled in behind a cargo feature. A renderer is handed the parsed document and may not open files or sockets, which is what keeps every path check in one place rather than in each format.
|
|
||||||
|
|
||||||
| Format | Served as | Notes |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| `gemtext` | `text/gemini` | Unwrapped; Gemini and Spartan clients wrap for themselves |
|
|
||||||
| `xhtmlmp` | `application/vnd.wap.xhtml+xml` | Well-formed XML with the Mobile Profile doctype |
|
|
||||||
| `html` | `text/html` | The same markup without the XML declaration |
|
|
||||||
| `text` | `text/plain` | Wrapped to 80 columns, because Nex and Gopher clients do not wrap |
|
|
||||||
| `wml` | `text/vnd.wap.wml` | WAP 1.x decks, paginated to a per-card byte budget (feature `wml`) |
|
|
||||||
|
|
||||||
HTTP negotiates between them from `Accept`. Only a literal media type counts as a match, so a browser's `*/*` can never be read as willingness to receive a WAP format; `?format=<id>` overrides negotiation outright, and naming an id the listener does not serve is a bad request rather than a silent fallback.
|
|
||||||
|
|
||||||
A third party adds a format by writing a crate that depends on `itsybitsy-core`, adding it to the workspace and one `#[cfg]` arm in the binary's registry, then naming its id in `formats`. Every listener's formats are checked against the registry at startup, so a format switched off by a feature is a configuration error naming the missing id rather than a server error on the first request.
|
|
||||||
|
|
||||||
## Optional features
|
|
||||||
|
|
||||||
Two decorative capabilities for the text format are off by default, because most pages leave them alone and both cost binary size:
|
|
||||||
|
|
||||||
| Feature | Adds | Cost |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| `wml` | WML 1.3 decks for WAP 1.x handsets | +37 KiB |
|
|
||||||
| `figlet` | `h1_style = "figlet"` banners, fonts `small` and `standard` | +60 KiB |
|
|
||||||
| `hyphenation` | `hyphenate = true`, English patterns | +127 KiB |
|
|
||||||
| `hyphenation-all` | every language the pattern crate carries | +3.0 MiB |
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cargo build --release --features "figlet hyphenation"
|
|
||||||
```
|
|
||||||
|
|
||||||
A banner that will not fit the line, names a font this build lacks, or is asked for by a build without `figlet` falls back to the level's underline — a heading that cannot be decorated should not be lost. An unknown font name is logged, since that is almost always a typo. Likewise `hyphenate` has no effect without the feature, and an unknown `hyphen_lang` leaves the text unhyphenated; a ragged right edge is the plain-text convention anyway.
|
|
||||||
|
|
||||||
## Protocols
|
|
||||||
|
|
||||||
| Protocol | Routes by | Notes |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| HTTP | `Host` header | `GET` and `HEAD`, negotiated by `Accept` |
|
|
||||||
| Spartan | the host field of its request line | One format per listener |
|
|
||||||
| Nex | nothing; its listener names one site | No status line at all |
|
|
||||||
| Gemini | the authority of the URL it is sent | TLS terminated in front; one format per listener |
|
|
||||||
| Gopher | nothing; its listener names one site | Item type 0 and the root menu |
|
|
||||||
|
|
||||||
A protocol with no hostname in its requests cannot be routed by one, so its listener names a single site outright and configuration refuses to leave that implicit once more than one site exists. HTTP, Gemini and Spartan fall back to `default_site` when a request names a host this server does not know.
|
|
||||||
|
|
||||||
Neither Nex nor Gopher has a redirect status, so a canonical target is resolved server-side and the real content comes back on the first request rather than a bounce.
|
|
||||||
|
|
||||||
### Gemini and TLS
|
|
||||||
|
|
||||||
itsybitsy does not speak TLS. The Gemini listener expects plaintext, so a TLS wrapper terminates in front of it and forwards to a loopback port:
|
|
||||||
|
|
||||||
```toml
|
|
||||||
[listener.gemini]
|
|
||||||
protocol = "gemini"
|
|
||||||
# The terminator owns 1965; this is its back end.
|
|
||||||
bind = "127.0.0.1:11965"
|
|
||||||
formats = ["gemtext"]
|
|
||||||
default_site = "smol"
|
|
||||||
```
|
|
||||||
|
|
||||||
```ini
|
|
||||||
; /etc/stunnel/gemini.conf
|
|
||||||
[gemini]
|
|
||||||
accept = 1965
|
|
||||||
connect = 127.0.0.1:11965
|
|
||||||
cert = /var/lib/itsybitsy/gemini.pem
|
|
||||||
```
|
|
||||||
|
|
||||||
`stunnel` and `ghostunnel` are built for exactly this and support SNI, so each virtual host can present its own certificate. Use a long-lived self-signed certificate rather than an ACME one: Gemini clients pin the fingerprint they first saw, and a renewal that changes it is indistinguishable from an interception.
|
|
||||||
|
|
||||||
Two things follow from terminating in front, and neither is a limitation that can be configured away. Every connection appears to come from the terminator, so request logs show its address unless it speaks the PROXY protocol, which itsybitsy does not yet read. And a client certificate never reaches us, so statuses 60 to 62 are not implementable — itsybitsy serves static documents and has nothing to authenticate, so only the logs are poorer for it.
|
|
||||||
|
|
||||||
Redirects are relative references, which the spec permits and which are the only correct form here: the port this listener is bound to is the terminator's back end, not a port any client reached, so an absolute URL built from it would point somewhere unpublished.
|
|
||||||
|
|
||||||
Gopher serves item type 0 (text) and one menu: a prefix-less `gopher://host/` means item type 1 by RFC 4266, so an empty selector gets a one-item menu pointing at `/` rather than the root document, which would be the wrong type. Text items are dot-stuffed and terminated with a lone dot; binary items are sent raw, since dot-stuffing would corrupt them and a terminator would become part of the file. There are no generated directory listings and no type 7 search.
|
|
||||||
|
|
||||||
## Card sub-documents
|
|
||||||
|
|
||||||
WML is the one output that paginates: a WAP 1.x handset has a hard per-card byte budget and refuses a deck that exceeds it. So a long page becomes a chain of screens, and a page with dividers becomes a menu card linking to the rest.
|
|
||||||
|
|
||||||
With `deck_per_card`, each card also gets its own URL under the page's:
|
|
||||||
|
|
||||||
| Request | WML client | Any other format |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| `/trail` | the menu deck | the whole page |
|
|
||||||
| `/trail/weather` | that card's own deck | redirects to `/trail` |
|
|
||||||
| `/trail/nonexistent` | redirects to `/trail` | redirects to `/trail` |
|
|
||||||
| `/about/whatever` | not found | not found |
|
|
||||||
|
|
||||||
That URL space belongs to the format that claims it. A format which does not address a sub-document redirects to the parent rather than substituting something else, and a page with no dividers has no such URLs at all — `/about/whatever` is not a sub-document just because `/about` exists. Nex and Gopher have no redirect status, so they resolve to the parent's content directly instead of bouncing.
|
|
||||||
|
|
||||||
The deck keys are `split_level`, `split_on_rule`, `max_card_bytes`, `menu`, `menu_style`, `deck_per_card`, `template_nav`, `nav_next_label`, `nav_prev_label`, `nav_back_label`, `home_label` and `images`.
|
|
||||||
|
|
||||||
## URLs
|
|
||||||
|
|
||||||
Links should be root-relative and extensionless (`[about](/about)`, not `about.md`), so the same link resolves identically from every protocol.
|
|
||||||
|
|
||||||
| Request | Serves |
|
| Request | Serves |
|
||||||
| --- | --- |
|
| --- | --- |
|
||||||
|
|
@ -217,11 +82,46 @@ Links should be root-relative and extensionless (`[about](/about)`, not `about.m
|
||||||
| `/foo.md` | redirects to `/foo` |
|
| `/foo.md` | redirects to `/foo` |
|
||||||
| `/img.png` | the file itself, by media type |
|
| `/img.png` | the file itself, by media type |
|
||||||
|
|
||||||
Nothing outside the content root is reachable. A request target is percent-decoded before it is normalised, so an encoded `..` becomes a real one and gets clamped at the root rather than quietly matching nothing; any path component beginning with a dot is refused outright; and whatever survives is canonicalised and required to still be inside the root, which is what defeats a symlink pointing out of it. There are no generated directory listings — only `index.md`.
|
Nothing outside a content root is reachable, dot-prefixed components are refused, and include expansion is depth-, line- and byte-capped. There are no generated directory listings — only `index.md`.
|
||||||
|
|
||||||
URLs the server generates are percent-encoded. A file name can contain a space, a `?`, a `#` or even a CR LF, and these URLs are emitted as redirect targets, so an unencoded one would truncate the path or end the response header and let a crafted file name inject a header of its own into every protocol.
|
### Gemini and TLS
|
||||||
|
|
||||||
A document is also refused if it nests blocks more than 100 levels deep, or if it is larger than the 8 MiB an expansion may produce. Both are about the stack and the heap rather than about the content: every pass over a parsed document recurses, and a stack overflow aborts the process instead of unwinding, so one pathological file would take every virtual host down with it. Includes are separately capped at 16 levels and 200 000 lines, with the byte cap catching the diamond fan-out that a per-stack cycle check cannot see.
|
itsybitsy does not speak TLS. A TLS wrapper (`stunnel`, `ghostunnel`) or a load balancer owns 1965 and forwards plaintext to a loopback port the Gemini listener binds; it supports SNI, so each virtual host can present its own certificate. Use a long-lived self-signed certificate rather than an ACME one: Gemini clients pin the fingerprint they first saw. A TLS terminator in front is also how the container deployment below runs.
|
||||||
|
|
||||||
|
## Deployment
|
||||||
|
|
||||||
|
The image builds from the repository root and runs on any OCI container host:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker build -t itsybitsy .
|
||||||
|
docker run -p 8080:8080 -p 3000:3000 -p 1900:1900 -p 7070:7070 itsybitsy
|
||||||
|
```
|
||||||
|
|
||||||
|
It serves the demo site with `docker/itsybitsy.toml`, which binds `0.0.0.0` where the checkout's own server file binds loopback, so published ports are reachable. A real site mounts over those paths:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker run -p 8080:8080 \
|
||||||
|
-v /srv/mysite.toml:/srv/itsybitsy/itsybitsy.toml:ro \
|
||||||
|
-v /srv/mysite:/srv/content:ro itsybitsy
|
||||||
|
```
|
||||||
|
|
||||||
|
The container runs as an unprivileged user. Port 1965 stays unexposed on the host: the Gemini listener is plaintext and a TLS terminator must own it in front, as always.
|
||||||
|
|
||||||
|
The flake deploys without Docker:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nix run . -- --config ./itsybitsy.toml # build and serve
|
||||||
|
nix build .#static # fully static musl binary
|
||||||
|
scp result/bin/itsybitsy host:/usr/local/bin/
|
||||||
|
```
|
||||||
|
|
||||||
|
`packages.static` links against musl with no C dependency, so the one binary runs unchanged on any Linux host, including one that has never seen Nix; `nix run .#release-static` publishes it as a release asset.
|
||||||
|
|
||||||
|
## Security audit
|
||||||
|
|
||||||
|
An adversarial audit of integration, containment, protocol abuse and load (2026-10-04, 138 checks over raw sockets) found three defects, all fixed with regression tests: deeply nested documents overflowed the stack and aborted the process (parsing now refuses past 100 levels), a file name with CR LF could inject a response header via redirect targets (generated URLs are now percent-encoded), and a large Markdown file was read into memory before the 8 MiB expansion cap rejected it (reads are now capped). After the fixes, 135 traversal attempts across all five protocols returned nothing outside the content roots and include bombs were bounded, at roughly 8–9k req/s with flat memory.
|
||||||
|
|
||||||
|
Outstanding: the render cache is unbounded (6.2× the source size across formats), an oversized request header yields a TCP reset instead of a 400, a listener's `width` is ignored, and there is no graceful shutdown. The TLS leg in front of Gemini was not tested, and the threat model assumes the content tree is author-controlled.
|
||||||
|
|
||||||
## Development
|
## Development
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -68,6 +68,9 @@ pub fn serve(listener: &Listener, mut stream: TcpStream) -> Result<()> {
|
||||||
std::io::copy(&mut file, &mut stream)?;
|
std::io::copy(&mut file, &mut stream)?;
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
Ok(Resolution::Found(Resource::Embedded { bytes, media_type })) => {
|
||||||
|
body_response(&mut stream, media_type, bytes)
|
||||||
|
}
|
||||||
Ok(Resolution::Redirect(location)) => header(&mut stream, 31, &location),
|
Ok(Resolution::Redirect(location)) => header(&mut stream, 31, &location),
|
||||||
Ok(Resolution::NotFound) => header(&mut stream, 51, "Not found"),
|
Ok(Resolution::NotFound) => header(&mut stream, 51, "Not found"),
|
||||||
Err(err) => {
|
Err(err) => {
|
||||||
|
|
@ -99,7 +102,6 @@ struct Request<'a> {
|
||||||
/// 53 is "a resource at a domain not served by the server" and 59 is a request
|
/// 53 is "a resource at a domain not served by the server" and 59 is a request
|
||||||
/// the server could not parse: a URL that parses but names another scheme or
|
/// the server could not parse: a URL that parses but names another scheme or
|
||||||
/// host is a proxy request, while one that does not parse is a bad request.
|
/// host is a proxy request, while one that does not parse is a bad request.
|
||||||
/// smolweb answers 59 to both, including to an ordinary `https://` URL.
|
|
||||||
fn parse(line: &str) -> Result<Request<'_>, Refusal> {
|
fn parse(line: &str) -> Result<Request<'_>, Refusal> {
|
||||||
let Some((scheme, rest)) = line.split_once("://") else {
|
let Some((scheme, rest)) = line.split_once("://") else {
|
||||||
return Err(Refusal { status: 59, meta: "Bad request: an absolute URL is required" });
|
return Err(Refusal { status: 59, meta: "Bad request: an absolute URL is required" });
|
||||||
|
|
@ -195,8 +197,8 @@ mod tests {
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn another_scheme_is_a_proxy_request_not_a_parse_failure() {
|
fn another_scheme_is_a_proxy_request_not_a_parse_failure() {
|
||||||
// smolweb answers 59 here, which tells a client its request was malformed
|
// 59 would tell a client its request was malformed when it was merely for
|
||||||
// when it was merely for somewhere this server does not fetch from.
|
// somewhere this server does not fetch from.
|
||||||
assert_eq!(refused("https://example.org/"), 53);
|
assert_eq!(refused("https://example.org/"), 53);
|
||||||
assert_eq!(refused("gopher://example.org/"), 53);
|
assert_eq!(refused("gopher://example.org/"), 53);
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -16,7 +16,7 @@ use itsybitsy_core::site::{Resolution, Resource};
|
||||||
use crate::proto::for_log;
|
use crate::proto::for_log;
|
||||||
use crate::serve::Listener;
|
use crate::serve::Listener;
|
||||||
|
|
||||||
/// Gopher selectors are short; the cap is smolweb's.
|
/// Gopher selectors are short, and nothing in the protocol needs a long one.
|
||||||
const MAX_REQUEST: usize = 512;
|
const MAX_REQUEST: usize = 512;
|
||||||
|
|
||||||
pub fn serve(listener: &Listener, mut stream: TcpStream) -> Result<()> {
|
pub fn serve(listener: &Listener, mut stream: TcpStream) -> Result<()> {
|
||||||
|
|
@ -55,6 +55,10 @@ pub fn serve(listener: &Listener, mut stream: TcpStream) -> Result<()> {
|
||||||
std::io::copy(&mut file, &mut stream)?;
|
std::io::copy(&mut file, &mut stream)?;
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
Ok(Resolution::Found(Resource::Embedded { bytes, .. })) => {
|
||||||
|
stream.write_all(bytes)?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
// resolve_flat follows redirects and sub-documents to real content.
|
// resolve_flat follows redirects and sub-documents to real content.
|
||||||
Ok(_) => text(&mut stream, b"Not found\n"),
|
Ok(_) => text(&mut stream, b"Not found\n"),
|
||||||
Err(err) => {
|
Err(err) => {
|
||||||
|
|
|
||||||
|
|
@ -32,7 +32,8 @@ pub fn serve(listener: &Listener, mut stream: TcpStream) -> Result<()> {
|
||||||
if !version.starts_with("HTTP/1.") {
|
if !version.starts_with("HTTP/1.") {
|
||||||
return bad_request(&mut stream);
|
return bad_request(&mut stream);
|
||||||
}
|
}
|
||||||
// Origin-form only. smolweb's `urlsplit` would have mishandled the others.
|
// Origin-form only; the other request-target forms are refused, not
|
||||||
|
// half-supported.
|
||||||
if !target.starts_with('/') {
|
if !target.starts_with('/') {
|
||||||
return bad_request(&mut stream);
|
return bad_request(&mut stream);
|
||||||
}
|
}
|
||||||
|
|
@ -168,12 +169,15 @@ pub fn serve(listener: &Listener, mut stream: TcpStream) -> Result<()> {
|
||||||
write_head(&mut stream, 200, "OK", media_type, meta.len(), &[])?;
|
write_head(&mut stream, 200, "OK", media_type, meta.len(), &[])?;
|
||||||
if !head_only {
|
if !head_only {
|
||||||
let mut file = std::fs::File::open(&path)?;
|
let mut file = std::fs::File::open(&path)?;
|
||||||
// Streamed, not buffered: smolweb reads the whole file into
|
// Streamed, not buffered, so a large file costs the copy buffer
|
||||||
// memory on every request.
|
// rather than its own size on every request.
|
||||||
std::io::copy(&mut file, &mut stream)?;
|
std::io::copy(&mut file, &mut stream)?;
|
||||||
}
|
}
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
Ok(Resolution::Found(Resource::Embedded { bytes, media_type })) => {
|
||||||
|
respond(&mut stream, 200, "OK", media_type, bytes, &[], head_only)
|
||||||
|
}
|
||||||
Ok(Resolution::Redirect(location)) => respond(
|
Ok(Resolution::Redirect(location)) => respond(
|
||||||
&mut stream,
|
&mut stream,
|
||||||
301,
|
301,
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,7 @@ use itsybitsy_core::site::{Resolution, Resource};
|
||||||
use crate::proto::{for_log, read_line_capped};
|
use crate::proto::{for_log, read_line_capped};
|
||||||
use crate::serve::Listener;
|
use crate::serve::Listener;
|
||||||
|
|
||||||
/// Nex requests are a single path; the cap is smolweb's.
|
/// Nex requests are a single path, so the cap only has to be generous for one.
|
||||||
const MAX_REQUEST: usize = 2048;
|
const MAX_REQUEST: usize = 2048;
|
||||||
|
|
||||||
pub fn serve(listener: &Listener, mut stream: TcpStream) -> Result<()> {
|
pub fn serve(listener: &Listener, mut stream: TcpStream) -> Result<()> {
|
||||||
|
|
@ -45,6 +45,9 @@ pub fn serve(listener: &Listener, mut stream: TcpStream) -> Result<()> {
|
||||||
let mut file = std::fs::File::open(&path)?;
|
let mut file = std::fs::File::open(&path)?;
|
||||||
std::io::copy(&mut file, &mut stream)?;
|
std::io::copy(&mut file, &mut stream)?;
|
||||||
}
|
}
|
||||||
|
Ok(Resolution::Found(Resource::Embedded { bytes, .. })) => {
|
||||||
|
stream.write_all(bytes)?;
|
||||||
|
}
|
||||||
// resolve_flat follows redirects and sub-documents to real content, so
|
// resolve_flat follows redirects and sub-documents to real content, so
|
||||||
// neither reaches here; they are matched for completeness.
|
// neither reaches here; they are matched for completeness.
|
||||||
Ok(_) => stream.write_all(b"Not found\n")?,
|
Ok(_) => stream.write_all(b"Not found\n")?,
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,7 @@
|
||||||
//! Spartan: `HOST PATH LENGTH` in, a one-digit status and a body out.
|
//! Spartan: `HOST PATH LENGTH` in, a one-digit status and a body out.
|
||||||
//!
|
//!
|
||||||
//! The host field is what smolweb discards; here it selects the virtual host,
|
//! The host field selects the virtual host, falling back to the listener's
|
||||||
//! falling back to the listener's `default_site` when it names nothing known.
|
//! `default_site` when it names nothing known.
|
||||||
|
|
||||||
use std::io::{BufReader, Read, Write};
|
use std::io::{BufReader, Read, Write};
|
||||||
use std::net::TcpStream;
|
use std::net::TcpStream;
|
||||||
|
|
@ -73,6 +73,9 @@ pub fn serve(listener: &Listener, mut stream: TcpStream) -> Result<()> {
|
||||||
std::io::copy(&mut file, &mut stream)?;
|
std::io::copy(&mut file, &mut stream)?;
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
Ok(Resolution::Found(Resource::Embedded { bytes, media_type })) => {
|
||||||
|
status(&mut stream, 2, media_type, bytes)
|
||||||
|
}
|
||||||
Ok(Resolution::Redirect(location)) => status(&mut stream, 3, &location, b""),
|
Ok(Resolution::Redirect(location)) => status(&mut stream, 3, &location, b""),
|
||||||
Ok(Resolution::NotFound) => status(&mut stream, 4, "Not found", b""),
|
Ok(Resolution::NotFound) => status(&mut stream, 4, "Not found", b""),
|
||||||
Err(err) => {
|
Err(err) => {
|
||||||
|
|
|
||||||
|
|
@ -171,7 +171,7 @@ fn accept_loop(listener: Arc<Listener>, socket: TcpListener) {
|
||||||
};
|
};
|
||||||
|
|
||||||
// At the cap, refuse immediately rather than queueing threads without
|
// At the cap, refuse immediately rather than queueing threads without
|
||||||
// bound. smolweb has no cap at all.
|
// bound.
|
||||||
let open = listener.open.fetch_add(1, Ordering::SeqCst);
|
let open = listener.open.fetch_add(1, Ordering::SeqCst);
|
||||||
if open >= listener.max_connections {
|
if open >= listener.max_connections {
|
||||||
listener.open.fetch_sub(1, Ordering::SeqCst);
|
listener.open.fetch_sub(1, Ordering::SeqCst);
|
||||||
|
|
@ -194,8 +194,9 @@ fn handle(listener: &Listener, stream: TcpStream) {
|
||||||
}
|
}
|
||||||
|
|
||||||
// One malformed document must not take the process down, so a panic in a
|
// One malformed document must not take the process down, so a panic in a
|
||||||
// handler is caught and logged. This is the Rust equivalent of smolweb's
|
// handler is caught and logged with its cause rather than lost. A stack
|
||||||
// bare `except Exception`, except the cause is recorded rather than lost.
|
// overflow is not a panic and aborts regardless, which is why the parser caps
|
||||||
|
// how deeply a document may nest.
|
||||||
let caught =
|
let caught =
|
||||||
std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| match listener.protocol {
|
std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| match listener.protocol {
|
||||||
Protocol::Http => proto::http::serve(listener, stream),
|
Protocol::Http => proto::http::serve(listener, stream),
|
||||||
|
|
|
||||||
|
|
@ -348,7 +348,7 @@ fn spartan_serves_gemtext() {
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn spartan_routes_by_the_host_field_smolweb_discards() {
|
fn spartan_routes_by_the_host_field() {
|
||||||
let server = Server::start();
|
let server = Server::start();
|
||||||
assert!(server.send("spartan", b"one.test / 0\r\n").contains("# One"));
|
assert!(server.send("spartan", b"one.test / 0\r\n").contains("# One"));
|
||||||
assert!(server.send("spartan", b"two.test / 0\r\n").contains("# Two"));
|
assert!(server.send("spartan", b"two.test / 0\r\n").contains("# Two"));
|
||||||
|
|
@ -560,8 +560,8 @@ fn nothing_outside_the_root_is_reachable_over_any_protocol() {
|
||||||
|
|
||||||
// -- Gopher --------------------------------------------------------------
|
// -- Gopher --------------------------------------------------------------
|
||||||
//
|
//
|
||||||
// Ported from smolweb's tests/test_gopher.py, where the exact wire bytes are the
|
// The exact wire bytes are the assertion here: Gopher has no status line, so
|
||||||
// assertion: Gopher has no status line, so framing is all a client has.
|
// framing is all a client has.
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn an_empty_selector_gets_a_one_item_menu() {
|
fn an_empty_selector_gets_a_one_item_menu() {
|
||||||
|
|
@ -591,9 +591,8 @@ fn a_text_item_ends_with_the_lone_dot_terminator() {
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn a_missing_selector_is_a_well_formed_text_item() {
|
fn a_missing_selector_is_a_well_formed_text_item() {
|
||||||
// No status to report with, so the error is the item's content. smolweb says
|
// No status to report with, so the error is the item's content. One wording
|
||||||
// "Not found." here and "Not found" over HTTP and Nex; one wording is used
|
// is used across every protocol, so this matches the HTTP and Nex bodies.
|
||||||
// across every protocol instead.
|
|
||||||
let server = Server::start();
|
let server = Server::start();
|
||||||
assert_eq!(server.send("gopher", b"/nope\r\n"), "Not found\n.\r\n");
|
assert_eq!(server.send("gopher", b"/nope\r\n"), "Not found\n.\r\n");
|
||||||
}
|
}
|
||||||
|
|
|
||||||
106
core/assets/mews-0.1.css
Normal file
106
core/assets/mews-0.1.css
Normal file
|
|
@ -0,0 +1,106 @@
|
||||||
|
/* Mews Profile default stylesheet 0.1 */
|
||||||
|
|
||||||
|
/* Base: light theme */
|
||||||
|
body {
|
||||||
|
background-color: #faf8f3;
|
||||||
|
color: #1f1f1f;
|
||||||
|
font-family: "Atkinson Hyperlegible", Verdana, Tahoma, system-ui, sans-serif;
|
||||||
|
font-size: 106%;
|
||||||
|
line-height: 1.5;
|
||||||
|
margin: 0 auto;
|
||||||
|
padding: 1em;
|
||||||
|
max-width: 40em;
|
||||||
|
}
|
||||||
|
|
||||||
|
h1, h2, h3, h4, h5, h6 {
|
||||||
|
font-weight: bold;
|
||||||
|
line-height: 1.25;
|
||||||
|
margin: 1.6em 0 0.5em 0;
|
||||||
|
}
|
||||||
|
h1 { font-size: 1.6em; margin-top: 0.5em; }
|
||||||
|
h2 { font-size: 1.3em; }
|
||||||
|
h3 { font-size: 1.1em; }
|
||||||
|
h4, h5, h6 { font-size: 1em; }
|
||||||
|
|
||||||
|
p, ul, ol, dl, blockquote, pre, table, address {
|
||||||
|
margin: 0 0 1em 0;
|
||||||
|
}
|
||||||
|
ul, ol { padding-left: 1.5em; }
|
||||||
|
li { margin-bottom: 0.25em; }
|
||||||
|
dt { font-weight: bold; }
|
||||||
|
dd { margin: 0 0 0.5em 1.5em; }
|
||||||
|
|
||||||
|
a { color: #1a4f9c; text-decoration: underline; }
|
||||||
|
a:visited { color: #5b2a86; }
|
||||||
|
a:focus, a:active { outline: 2px solid #1a4f9c; }
|
||||||
|
|
||||||
|
blockquote {
|
||||||
|
margin-left: 0;
|
||||||
|
padding-left: 1em;
|
||||||
|
border-left: 3px solid #c9c4b8;
|
||||||
|
}
|
||||||
|
|
||||||
|
pre, code, kbd, samp {
|
||||||
|
font-family: "Source Code Pro", Consolas, Menlo, "DejaVu Sans Mono", monospace;
|
||||||
|
font-size: 0.9em;
|
||||||
|
}
|
||||||
|
pre {
|
||||||
|
background-color: #efece4;
|
||||||
|
padding: 0.75em;
|
||||||
|
overflow: auto;
|
||||||
|
}
|
||||||
|
|
||||||
|
hr {
|
||||||
|
border: 0;
|
||||||
|
border-top: 1px solid #c9c4b8;
|
||||||
|
margin: 2em 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
table { border-collapse: collapse; }
|
||||||
|
th, td {
|
||||||
|
border: 1px solid #c9c4b8;
|
||||||
|
padding: 0.3em 0.6em;
|
||||||
|
text-align: left;
|
||||||
|
vertical-align: top;
|
||||||
|
}
|
||||||
|
caption { font-weight: bold; text-align: left; }
|
||||||
|
|
||||||
|
img {
|
||||||
|
display: block;
|
||||||
|
max-width: 100%;
|
||||||
|
height: auto;
|
||||||
|
border: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
input, select, textarea { font-size: 1em; font-family: inherit; }
|
||||||
|
|
||||||
|
/* Body variants: light */
|
||||||
|
body.mews-warm { background-color: #fbf4e8; color: #2a2420; }
|
||||||
|
body.mews-warm a { color: #8a3d14; }
|
||||||
|
body.mews-cool { background-color: #f3f6fa; color: #1c2430; }
|
||||||
|
body.mews-cool a { color: #1a4f9c; }
|
||||||
|
body.mews-green { background-color: #f2f6ef; color: #1f2a1f; }
|
||||||
|
body.mews-green a { color: #24502a; }
|
||||||
|
body.mews-mono { background-color: #e8ebe1; color: #1e2219; }
|
||||||
|
body.mews-mono a { color: #1e2219; }
|
||||||
|
|
||||||
|
/* Dark theme: follows the system setting */
|
||||||
|
@media (prefers-color-scheme: dark) {
|
||||||
|
body { background-color: #18191b; color: #e3e1dc; }
|
||||||
|
a { color: #8fb8f5; }
|
||||||
|
a:visited { color: #c9a3f2; }
|
||||||
|
a:focus, a:active { outline-color: #8fb8f5; }
|
||||||
|
blockquote, hr, th, td { border-color: #45464a; }
|
||||||
|
pre { background-color: #222326; }
|
||||||
|
|
||||||
|
body.mews-warm { background-color: #1d1916; color: #e8dfd3; }
|
||||||
|
body.mews-warm a { color: #f0b48a; }
|
||||||
|
body.mews-cool { background-color: #161a20; color: #dde4ee; }
|
||||||
|
body.mews-cool a { color: #8fb8f5; }
|
||||||
|
body.mews-green { background-color: #161b16; color: #dbe6d8; }
|
||||||
|
body.mews-green a { color: #9fd39a; }
|
||||||
|
body.mews-mono { background-color: #1b1d18; color: #c8d0b8; }
|
||||||
|
body.mews-mono a { color: #c8d0b8; }
|
||||||
|
}
|
||||||
|
|
||||||
|
html { color-scheme: light dark; }
|
||||||
|
|
@ -3,8 +3,7 @@
|
||||||
//! Invalidation is by modification time and length, which is what makes editing
|
//! Invalidation is by modification time and length, which is what makes editing
|
||||||
//! a file enough to see the change on the next request with no watcher and no
|
//! a file enough to see the change on the next request with no watcher and no
|
||||||
//! restart. Values are built outside the lock, so two first hits on one file can
|
//! restart. Values are built outside the lock, so two first hits on one file can
|
||||||
//! both build it; the work is idempotent and the second insert wins, which is
|
//! both build it; the work is idempotent and the second insert wins.
|
||||||
//! the trade the Python makes too.
|
|
||||||
|
|
||||||
use std::collections::HashMap;
|
use std::collections::HashMap;
|
||||||
use std::fs;
|
use std::fs;
|
||||||
|
|
@ -109,9 +108,9 @@ mod tests {
|
||||||
|
|
||||||
use super::*;
|
use super::*;
|
||||||
|
|
||||||
/// Force a modification time change. Filesystem granularity is coarse
|
/// Force a modification time change. Filesystem granularity is coarse enough
|
||||||
/// enough that two writes in one test can otherwise share a timestamp,
|
/// that two writes in one test can otherwise share a timestamp, which would
|
||||||
/// which is why the Python's live-reload test does the same thing.
|
/// make a stale value look correctly cached.
|
||||||
fn bump_mtime(path: &Path) {
|
fn bump_mtime(path: &Path) {
|
||||||
let later = SystemTime::now() + Duration::from_secs(5);
|
let later = SystemTime::now() + Duration::from_secs(5);
|
||||||
File::options()
|
File::options()
|
||||||
|
|
@ -137,7 +136,8 @@ mod tests {
|
||||||
let first = cache.get_or_insert_with(&path, Stamp::of(&path).unwrap(), build).unwrap();
|
let first = cache.get_or_insert_with(&path, Stamp::of(&path).unwrap(), build).unwrap();
|
||||||
let second = cache.get_or_insert_with(&path, Stamp::of(&path).unwrap(), build).unwrap();
|
let second = cache.get_or_insert_with(&path, Stamp::of(&path).unwrap(), build).unwrap();
|
||||||
|
|
||||||
// Identity, not just equality: the Python asserts `first is second`.
|
// Identity, not equality: a repeat hit must return the cached value
|
||||||
|
// rather than an equal rebuild.
|
||||||
assert!(Arc::ptr_eq(&first, &second));
|
assert!(Arc::ptr_eq(&first, &second));
|
||||||
assert_eq!(builds.load(Ordering::Relaxed), 1);
|
assert_eq!(builds.load(Ordering::Relaxed), 1);
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -48,6 +48,20 @@ pub struct SiteSpec {
|
||||||
/// `default_site`.
|
/// `default_site`.
|
||||||
#[serde(default)]
|
#[serde(default)]
|
||||||
pub hosts: Vec<String>,
|
pub hosts: Vec<String>,
|
||||||
|
/// Render this site's `xhtmlmp`/`html` output as a Mews Profile page
|
||||||
|
/// (mews.page/spec): the XHTML-MP 1.2 doctype, the conformance marker, the
|
||||||
|
/// viewport tag and the default stylesheet link, with raw HTML and
|
||||||
|
/// off-site or `data:` images dropped rather than passed through, since the
|
||||||
|
/// profile's permitted-element list cannot otherwise be guaranteed.
|
||||||
|
#[serde(default)]
|
||||||
|
pub mews_profile: bool,
|
||||||
|
/// The site's Atom feed, declared in the head of its `xhtmlmp`/`html`
|
||||||
|
/// pages (mews.page/spec 6.2) so a client can subscribe without reading
|
||||||
|
/// the body. Absolute: a capsule rarely carries the feed its web site
|
||||||
|
/// publishes, so a root-relative path would point at a document this
|
||||||
|
/// server has no file for.
|
||||||
|
#[serde(default)]
|
||||||
|
pub feed: Option<String>,
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Debug, Deserialize)]
|
#[derive(Debug, Deserialize)]
|
||||||
|
|
@ -169,13 +183,38 @@ impl ServerConfig {
|
||||||
self.reject_config_inside_root(config_path, &roots)?;
|
self.reject_config_inside_root(config_path, &roots)?;
|
||||||
let hosts = self.build_host_map()?;
|
let hosts = self.build_host_map()?;
|
||||||
let formats = self.validate_listeners(available_formats)?;
|
let formats = self.validate_listeners(available_formats)?;
|
||||||
|
let shared_roots = group_shared_roots(&roots);
|
||||||
|
self.validate_shared_markup(&shared_roots)?;
|
||||||
|
|
||||||
Ok(Checked {
|
Ok(Checked { hosts, roots: roots.clone(), shared_roots, formats })
|
||||||
hosts,
|
}
|
||||||
roots: roots.clone(),
|
|
||||||
shared_roots: group_shared_roots(&roots),
|
/// Sites sharing a root share one [`crate::site::Site`] and so one render
|
||||||
formats,
|
/// cache; a page rendered once there cannot differ by which name served the
|
||||||
})
|
/// request, so every setting that changes its markup must agree within a
|
||||||
|
/// group.
|
||||||
|
fn validate_shared_markup(&self, shared_roots: &[Vec<String>]) -> Result<(), Error> {
|
||||||
|
self.validate_shared(shared_roots, "mews_profile", |spec| spec.mews_profile)?;
|
||||||
|
self.validate_shared(shared_roots, "feed", |spec| spec.feed.clone())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn validate_shared<T: PartialEq>(
|
||||||
|
&self,
|
||||||
|
shared_roots: &[Vec<String>],
|
||||||
|
key: &str,
|
||||||
|
of: impl Fn(&SiteSpec) -> T,
|
||||||
|
) -> Result<(), Error> {
|
||||||
|
for group in shared_roots {
|
||||||
|
let mut settings = group.iter().map(|name| of(&self.site[name]));
|
||||||
|
let Some(first) = settings.next() else { continue };
|
||||||
|
if settings.any(|value| value != first) {
|
||||||
|
return Err(Error::config(format!(
|
||||||
|
"sites {} share one root, so {key} must agree between them",
|
||||||
|
group.join(", ")
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Canonicalise every content root, which also proves it exists and is a
|
/// Canonicalise every content root, which also proves it exists and is a
|
||||||
|
|
@ -670,8 +709,8 @@ impl Default for PageSettings {
|
||||||
// the terminal edge.
|
// the terminal edge.
|
||||||
margin_left: 2,
|
margin_left: 2,
|
||||||
margin_right: 2,
|
margin_right: 2,
|
||||||
// One blank line between blocks. md2txt uses two, which reads as
|
// One blank line between blocks; two reads as double-spaced
|
||||||
// double-spaced throughout.
|
// throughout.
|
||||||
paragraph_spacing: 1,
|
paragraph_spacing: 1,
|
||||||
heading_styles: DEFAULT_HEADING_STYLES,
|
heading_styles: DEFAULT_HEADING_STYLES,
|
||||||
blockquote_bars: true,
|
blockquote_bars: true,
|
||||||
|
|
@ -915,6 +954,19 @@ mod tests {
|
||||||
assert_eq!(checked.shared_roots, vec![vec!["one".to_string(), "two".to_string()]]);
|
assert_eq!(checked.shared_roots, vec![vec!["one".to_string(), "two".to_string()]]);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn rejects_sites_sharing_one_root_with_differing_mews_profile() {
|
||||||
|
let dir = tempfile::tempdir().unwrap();
|
||||||
|
let root = dir.path().join("content");
|
||||||
|
fs::create_dir_all(&root).unwrap();
|
||||||
|
let two = format!(
|
||||||
|
"{HTTP}\n[site.two]\nroot = {:?}\nhosts = [\"two.test\"]\nmews_profile = true\n",
|
||||||
|
root.to_str().unwrap()
|
||||||
|
);
|
||||||
|
let err = check(dir.path(), &two).unwrap_err();
|
||||||
|
assert!(err.to_string().contains("mews_profile must agree"), "{err}");
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn rejects_a_config_file_inside_a_content_root() {
|
fn rejects_a_config_file_inside_a_content_root() {
|
||||||
let dir = tempfile::tempdir().unwrap();
|
let dir = tempfile::tempdir().unwrap();
|
||||||
|
|
|
||||||
|
|
@ -7,6 +7,8 @@
|
||||||
//! | --- | --- |
|
//! | --- | --- |
|
||||||
//! | `<!-- card Title -->` | a card divider; invisible to any other renderer |
|
//! | `<!-- card Title -->` | a card divider; invisible to any other renderer |
|
||||||
//! | `<!-- center -->` | alignment for the block that follows |
|
//! | `<!-- center -->` | alignment for the block that follows |
|
||||||
|
//! | `<!-- only gemtext -->` … `<!-- end -->` | a run of blocks only these output formats get |
|
||||||
|
//! | `<!-- except wml -->` … `<!-- end -->` | a run of blocks every other format gets |
|
||||||
//! | `` | an image whose target is text, inlined verbatim |
|
//! | `` | an image whose target is text, inlined verbatim |
|
||||||
//!
|
//!
|
||||||
//! An HTML comment that is not a recognised directive stays a comment. Comments
|
//! An HTML comment that is not a recognised directive stays a comment. Comments
|
||||||
|
|
@ -37,19 +39,32 @@ pub fn apply(doc: &mut Doc, base: &Path, root: &Path) -> Result<(), Error> {
|
||||||
fn rewrite(blocks: Vec<Block>, base: &Path, root: &Path) -> Result<Vec<Block>, Error> {
|
fn rewrite(blocks: Vec<Block>, base: &Path, root: &Path) -> Result<Vec<Block>, Error> {
|
||||||
let mut out = Vec::with_capacity(blocks.len());
|
let mut out = Vec::with_capacity(blocks.len());
|
||||||
let mut pending: Option<Directive> = None;
|
let mut pending: Option<Directive> = None;
|
||||||
|
// Gates in the order they were opened. Each recursion gets its own, so a
|
||||||
|
// gate opened inside a quote or a list item has to close inside it too.
|
||||||
|
let mut gates: Vec<Gate> = Vec::new();
|
||||||
|
|
||||||
for block in blocks {
|
for block in blocks {
|
||||||
if let Block::Html(html) = &block {
|
if let Block::Html(html) = &block {
|
||||||
match directive(html) {
|
match directive(html) {
|
||||||
Some(Directive::Card { title }) => {
|
Some(Directive::Card { title }) => {
|
||||||
out.push(Block::CardBreak { title });
|
out.extend(gated(&gates, vec![Block::CardBreak { title }]));
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
Some(align @ Directive::Align { .. }) => {
|
Some(align @ Directive::Align { .. }) => {
|
||||||
pending = Some(align);
|
pending = Some(align);
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
None => {}
|
Some(Directive::Only(gate)) => {
|
||||||
|
gates.push(gate);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
Some(Directive::End) if !gates.is_empty() => {
|
||||||
|
gates.pop();
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
// A close with no gate open is just a comment, as is anything
|
||||||
|
// else unrecognised.
|
||||||
|
Some(Directive::End) | None => {}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -70,24 +85,68 @@ fn rewrite(blocks: Vec<Block>, base: &Path, root: &Path) -> Result<Vec<Block>, E
|
||||||
other => vec![other],
|
other => vec![other],
|
||||||
};
|
};
|
||||||
|
|
||||||
match pending.take() {
|
let aligned = match pending.take() {
|
||||||
Some(Directive::Align { align, margin }) => out.extend(produced.into_iter().map(|b| {
|
Some(Directive::Align { align, margin }) => produced
|
||||||
// A block that already carries its own alignment keeps it: the
|
.into_iter()
|
||||||
// more specific marker wins over the one that precedes it.
|
.map(|b| {
|
||||||
match b {
|
// A block that already carries its own alignment keeps it:
|
||||||
aligned @ Block::Aligned { .. } => aligned,
|
// the more specific marker wins over the one before it.
|
||||||
other => Block::Aligned { align, margin, block: Box::new(other) },
|
match b {
|
||||||
}
|
aligned @ Block::Aligned { .. } => aligned,
|
||||||
})),
|
other => Block::Aligned { align, margin, block: Box::new(other) },
|
||||||
_ => out.extend(produced),
|
}
|
||||||
}
|
})
|
||||||
|
.collect(),
|
||||||
|
_ => produced,
|
||||||
|
};
|
||||||
|
out.extend(gated(&gates, aligned));
|
||||||
|
}
|
||||||
|
|
||||||
|
if !gates.is_empty() {
|
||||||
|
return Err(Error::directive(
|
||||||
|
"an <!-- only --> or <!-- except --> gate is never closed; add <!-- end -->",
|
||||||
|
));
|
||||||
}
|
}
|
||||||
Ok(out)
|
Ok(out)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Wrap each block in every open gate, the first opened outermost, so nested
|
||||||
|
/// gates compose: a block has to satisfy all of them to survive filtering.
|
||||||
|
fn gated(gates: &[Gate], blocks: Vec<Block>) -> Vec<Block> {
|
||||||
|
if gates.is_empty() {
|
||||||
|
return blocks;
|
||||||
|
}
|
||||||
|
blocks
|
||||||
|
.into_iter()
|
||||||
|
.map(|block| {
|
||||||
|
gates.iter().rev().fold(block, |inner, gate| Block::Gated {
|
||||||
|
formats: gate.formats.clone(),
|
||||||
|
negated: gate.negated,
|
||||||
|
block: Box::new(inner),
|
||||||
|
})
|
||||||
|
})
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
|
||||||
enum Directive {
|
enum Directive {
|
||||||
Card { title: Option<String> },
|
Card {
|
||||||
Align { align: Align, margin: Option<u16> },
|
title: Option<String>,
|
||||||
|
},
|
||||||
|
Align {
|
||||||
|
align: Align,
|
||||||
|
margin: Option<u16>,
|
||||||
|
},
|
||||||
|
/// Opens a run of blocks restricted to some output formats.
|
||||||
|
Only(Gate),
|
||||||
|
/// Closes the innermost open gate.
|
||||||
|
End,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// An open gate's condition: format ids, and whether they are the formats to
|
||||||
|
/// keep (`only`) or the ones to leave out (`except`).
|
||||||
|
struct Gate {
|
||||||
|
formats: Vec<String>,
|
||||||
|
negated: bool,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Recognise a directive in the text of an HTML block, or `None` for an ordinary
|
/// Recognise a directive in the text of an HTML block, or `None` for an ordinary
|
||||||
|
|
@ -100,11 +159,31 @@ fn directive(html: &str) -> Option<Directive> {
|
||||||
let title = words.collect::<Vec<_>>().join(" ");
|
let title = words.collect::<Vec<_>>().join(" ");
|
||||||
return Some(Directive::Card { title: (!title.is_empty()).then_some(title) });
|
return Some(Directive::Card { title: (!title.is_empty()).then_some(title) });
|
||||||
}
|
}
|
||||||
|
if let Some(negated) = gate_named(name) {
|
||||||
|
// A gate naming no format would hide or reveal everything by accident,
|
||||||
|
// so an argument-less one stays a comment.
|
||||||
|
let formats = words.map(str::to_string).collect::<Vec<_>>();
|
||||||
|
return (!formats.is_empty()).then_some(Directive::Only(Gate { formats, negated }));
|
||||||
|
}
|
||||||
|
if name == "end" {
|
||||||
|
// `<!-- end of the list -->` is prose, not a close.
|
||||||
|
return words.next().is_none().then_some(Directive::End);
|
||||||
|
}
|
||||||
let align = align_named(name)?;
|
let align = align_named(name)?;
|
||||||
let margin = words.find_map(|word| word.strip_prefix("margin=")?.parse().ok());
|
let margin = words.find_map(|word| word.strip_prefix("margin=")?.parse().ok());
|
||||||
Some(Directive::Align { align, margin })
|
Some(Directive::Align { align, margin })
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Whether a name opens a gate, and if so whether it names the formats to leave
|
||||||
|
/// out rather than the ones to keep.
|
||||||
|
fn gate_named(name: &str) -> Option<bool> {
|
||||||
|
match name {
|
||||||
|
"only" => Some(false),
|
||||||
|
"except" => Some(true),
|
||||||
|
_ => None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
fn align_named(name: &str) -> Option<Align> {
|
fn align_named(name: &str) -> Option<Align> {
|
||||||
match name {
|
match name {
|
||||||
"left" => Some(Align::Left),
|
"left" => Some(Align::Left),
|
||||||
|
|
@ -307,6 +386,128 @@ mod tests {
|
||||||
assert_eq!(doc.blocks.len(), 1);
|
assert_eq!(doc.blocks.len(), 1);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// -- Format gates ------------------------------------------------------
|
||||||
|
|
||||||
|
fn gate(formats: &[&str], negated: bool, block: Block) -> Block {
|
||||||
|
Block::Gated {
|
||||||
|
formats: formats.iter().map(|id| id.to_string()).collect(),
|
||||||
|
negated,
|
||||||
|
block: Box::new(block),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_gate_wraps_every_block_of_its_run() {
|
||||||
|
let tree = Tree::new();
|
||||||
|
let doc = tree.doc("<!-- only html -->\n\nA.\n\nB.\n\n<!-- end -->\n\nC.\n").unwrap();
|
||||||
|
assert_eq!(
|
||||||
|
doc.blocks,
|
||||||
|
vec![
|
||||||
|
gate(&["html"], false, para("A.")),
|
||||||
|
gate(&["html"], false, para("B.")),
|
||||||
|
// Past the close the run is over.
|
||||||
|
para("C."),
|
||||||
|
]
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn an_except_gate_names_the_formats_to_leave_out() {
|
||||||
|
let tree = Tree::new();
|
||||||
|
let doc = tree.doc("<!-- except wml text -->\nA.\n<!-- end -->\n").unwrap();
|
||||||
|
assert_eq!(doc.blocks, vec![gate(&["wml", "text"], true, para("A."))]);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn nested_gates_compose_with_the_first_opened_outermost() {
|
||||||
|
let tree = Tree::new();
|
||||||
|
let doc = tree
|
||||||
|
.doc("<!-- only html xhtmlmp -->\n<!-- except xhtmlmp -->\nA.\n<!-- end -->\n<!-- end -->\n")
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(
|
||||||
|
doc.blocks,
|
||||||
|
vec![gate(&["html", "xhtmlmp"], false, gate(&["xhtmlmp"], true, para("A.")))]
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_card_divider_inside_a_gate_is_gated_too() {
|
||||||
|
// Otherwise WML would paginate at a divider meant for another format.
|
||||||
|
let tree = Tree::new();
|
||||||
|
let doc = tree.doc("<!-- only wml -->\n<!-- card Next -->\n<!-- end -->\n").unwrap();
|
||||||
|
assert_eq!(
|
||||||
|
doc.blocks,
|
||||||
|
vec![gate(&["wml"], false, Block::CardBreak { title: Some("Next".into()) })]
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn an_alignment_before_a_gate_ends_up_inside_it() {
|
||||||
|
let tree = Tree::new();
|
||||||
|
let doc = tree.doc("<!-- center -->\n<!-- only text -->\nA.\n<!-- end -->\n").unwrap();
|
||||||
|
assert_eq!(
|
||||||
|
doc.blocks,
|
||||||
|
vec![gate(
|
||||||
|
&["text"],
|
||||||
|
false,
|
||||||
|
Block::Aligned { align: Align::Center, margin: None, block: Box::new(para("A.")) },
|
||||||
|
)]
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_gate_may_be_opened_inside_a_quote_or_a_list_item() {
|
||||||
|
let tree = Tree::new();
|
||||||
|
let doc = tree
|
||||||
|
.doc("> <!-- only text -->\n> A.\n> <!-- end -->\n\n- <!-- only text -->\n B.\n <!-- end -->\n")
|
||||||
|
.unwrap();
|
||||||
|
let Block::BlockQuote(inner) = &doc.blocks[0] else { panic!("expected a quote") };
|
||||||
|
assert_eq!(inner, &vec![gate(&["text"], false, para("A."))]);
|
||||||
|
let Block::List { items, .. } = &doc.blocks[1] else { panic!("expected a list") };
|
||||||
|
assert_eq!(items[0], vec![gate(&["text"], false, para("B."))]);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_gate_left_open_is_refused() {
|
||||||
|
// Running it silently to the end of the document would hide the rest of
|
||||||
|
// the page from some formats with nothing to notice it by.
|
||||||
|
let tree = Tree::new();
|
||||||
|
let err = tree.doc("<!-- only html -->\nA.\n").unwrap_err();
|
||||||
|
assert!(err.to_string().contains("never closed"), "{err}");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_gate_must_close_inside_the_quote_it_was_opened_in() {
|
||||||
|
let tree = Tree::new();
|
||||||
|
assert!(tree.doc("> <!-- only html -->\n> A.\n\n<!-- end -->\n").is_err());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_close_with_nothing_open_stays_a_comment() {
|
||||||
|
let tree = Tree::new();
|
||||||
|
let doc = tree.doc("<!-- end -->\nA.\n").unwrap();
|
||||||
|
assert!(matches!(&doc.blocks[0], Block::Html(html) if html.contains("end")));
|
||||||
|
assert_eq!(doc.blocks[1], para("A."));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_gate_naming_no_format_stays_a_comment() {
|
||||||
|
// It would otherwise hide or reveal everything by accident.
|
||||||
|
let tree = Tree::new();
|
||||||
|
let doc = tree.doc("<!-- only -->\nA.\n").unwrap();
|
||||||
|
assert!(matches!(&doc.blocks[0], Block::Html(_)));
|
||||||
|
assert_eq!(doc.blocks[1], para("A."));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_close_carrying_prose_stays_a_comment() {
|
||||||
|
let tree = Tree::new();
|
||||||
|
let doc =
|
||||||
|
tree.doc("<!-- only html -->\nA.\n<!-- end of the gate -->\n<!-- end -->\n").unwrap();
|
||||||
|
let Block::Gated { block, .. } = &doc.blocks[1] else { panic!("expected a gated block") };
|
||||||
|
assert!(matches!(block.as_ref(), Block::Html(html) if html.contains("end of the gate")));
|
||||||
|
}
|
||||||
|
|
||||||
// -- Art ---------------------------------------------------------------
|
// -- Art ---------------------------------------------------------------
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
|
|
|
||||||
|
|
@ -31,6 +31,12 @@ pub enum Error {
|
||||||
/// stack overflow aborts the process rather than unwinding.
|
/// stack overflow aborts the process rather than unwinding.
|
||||||
TooDeep { limit: usize },
|
TooDeep { limit: usize },
|
||||||
|
|
||||||
|
/// A directive the parser could read but not complete, such as an
|
||||||
|
/// `<!-- only ... -->` gate that is never closed. Refused rather than
|
||||||
|
/// guessed, because the guess would hide content from some formats and show
|
||||||
|
/// it on others with nothing to notice it by.
|
||||||
|
Directive { message: String },
|
||||||
|
|
||||||
/// An include or ASCII-art target cannot be used. `path` is where the
|
/// An include or ASCII-art target cannot be used. `path` is where the
|
||||||
/// directive actually pointed, resolved, which is the thing an author needs
|
/// directive actually pointed, resolved, which is the thing an author needs
|
||||||
/// to see when a relative target is wrong.
|
/// to see when a relative target is wrong.
|
||||||
|
|
@ -76,6 +82,7 @@ impl fmt::Display for Error {
|
||||||
Error::TooDeep { limit } => {
|
Error::TooDeep { limit } => {
|
||||||
write!(f, "document nests more than {limit} levels deep")
|
write!(f, "document nests more than {limit} levels deep")
|
||||||
}
|
}
|
||||||
|
Error::Directive { message } => write!(f, "{message}"),
|
||||||
Error::Include { path, reason } => {
|
Error::Include { path, reason } => {
|
||||||
write!(f, "include target {} {reason}", path.display())
|
write!(f, "include target {} {reason}", path.display())
|
||||||
}
|
}
|
||||||
|
|
@ -103,4 +110,8 @@ impl Error {
|
||||||
pub(crate) fn config(message: impl Into<String>) -> Self {
|
pub(crate) fn config(message: impl Into<String>) -> Self {
|
||||||
Error::Config { message: message.into() }
|
Error::Config { message: message.into() }
|
||||||
}
|
}
|
||||||
|
|
||||||
|
pub(crate) fn directive(message: impl Into<String>) -> Self {
|
||||||
|
Error::Directive { message: message.into() }
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
173
core/src/ir.rs
173
core/src/ir.rs
|
|
@ -1,15 +1,16 @@
|
||||||
//! The one parsed representation every output format consumes.
|
//! The one parsed representation every output format consumes.
|
||||||
//!
|
//!
|
||||||
//! smolweb parses each document twice, with two hand-rolled regex parsers that
|
//! One parse, shared by every format. Parsing per format is how two outputs come
|
||||||
//! share seven identical patterns but disagree on the edges: that is where the
|
//! to disagree about the same document: a directive one understands and another
|
||||||
//! `{.card}` directive leaks into gemtext as literal text, and why the two
|
//! emits as literal text, or two sets of defaults that drift apart. Parsing once
|
||||||
//! libraries carry two divergent sets of defaults. Parsing once into this
|
//! into this structure removes that class of bug rather than its instances.
|
||||||
//! structure removes the class of bug rather than the instances.
|
|
||||||
//!
|
//!
|
||||||
//! It is a flat block sequence rather than a tree of nodes because that is what
|
//! It is a flat block sequence rather than a tree of nodes because that is what
|
||||||
//! the consumers want: WML packs a linear run of blocks into byte-budgeted
|
//! the consumers want: WML packs a linear run of blocks into byte-budgeted
|
||||||
//! cards, and the text renderers are a fold over blocks.
|
//! cards, and the text renderers are a fold over blocks.
|
||||||
|
|
||||||
|
use std::borrow::Cow;
|
||||||
|
|
||||||
/// A parsed document.
|
/// A parsed document.
|
||||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
pub struct Doc {
|
pub struct Doc {
|
||||||
|
|
@ -64,6 +65,16 @@ pub enum Block {
|
||||||
margin: Option<u16>,
|
margin: Option<u16>,
|
||||||
block: Box<Block>,
|
block: Box<Block>,
|
||||||
},
|
},
|
||||||
|
/// A block an `<!-- only ... -->` or `<!-- except ... -->` run restricted to
|
||||||
|
/// some output formats. Resolved by [`Doc::for_format`] before rendering, so
|
||||||
|
/// no renderer meets one; it wraps rather than being a field for the same
|
||||||
|
/// reason `Aligned` does, and the two nest in either order.
|
||||||
|
Gated {
|
||||||
|
formats: Vec<String>,
|
||||||
|
/// `true` for `except`: keep the block for every format *but* these.
|
||||||
|
negated: bool,
|
||||||
|
block: Box<Block>,
|
||||||
|
},
|
||||||
/// Raw block HTML. Kept rather than dropped so XHTML-MP can pass it through
|
/// Raw block HTML. Kept rather than dropped so XHTML-MP can pass it through
|
||||||
/// and the text formats can strip it, instead of the parser deciding.
|
/// and the text formats can strip it, instead of the parser deciding.
|
||||||
Html(String),
|
Html(String),
|
||||||
|
|
@ -108,6 +119,19 @@ impl Doc {
|
||||||
out
|
out
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// This document as one output format sees it: a block gated to other
|
||||||
|
/// formats is dropped and a gate that passes is unwrapped.
|
||||||
|
///
|
||||||
|
/// Borrowed unchanged when there are no gates, which is the ordinary page.
|
||||||
|
/// `first_h1` is kept whatever happens: the title is resolved once per page,
|
||||||
|
/// so a heading inside a gate still names the page on every format.
|
||||||
|
pub fn for_format(&self, format: &str) -> Cow<'_, Doc> {
|
||||||
|
if !any_gated(&self.blocks) {
|
||||||
|
return Cow::Borrowed(self);
|
||||||
|
}
|
||||||
|
Cow::Owned(Doc { blocks: retain(&self.blocks, format), first_h1: self.first_h1.clone() })
|
||||||
|
}
|
||||||
|
|
||||||
fn write_plain(inline: &[Inline], out: &mut String) {
|
fn write_plain(inline: &[Inline], out: &mut String) {
|
||||||
for item in inline {
|
for item in inline {
|
||||||
match item {
|
match item {
|
||||||
|
|
@ -125,10 +149,149 @@ impl Doc {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Whether a gate appears anywhere in a run of blocks, including inside the
|
||||||
|
/// wrappers a gate can be written in.
|
||||||
|
fn any_gated(blocks: &[Block]) -> bool {
|
||||||
|
blocks.iter().any(|block| match block {
|
||||||
|
Block::Gated { .. } => true,
|
||||||
|
Block::Aligned { block, .. } => any_gated(std::slice::from_ref(block)),
|
||||||
|
Block::BlockQuote(inner) => any_gated(inner),
|
||||||
|
Block::List { items, .. } => items.iter().any(|item| any_gated(item)),
|
||||||
|
_ => false,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
fn retain(blocks: &[Block], format: &str) -> Vec<Block> {
|
||||||
|
blocks.iter().filter_map(|block| keep(block, format)).collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One block as `format` sees it, or `None` if a gate excludes it.
|
||||||
|
fn keep(block: &Block, format: &str) -> Option<Block> {
|
||||||
|
match block {
|
||||||
|
Block::Gated { formats, negated, block } => {
|
||||||
|
let named = formats.iter().any(|id| id == format);
|
||||||
|
// `only` keeps the formats it names, `except` keeps all the others.
|
||||||
|
(named != *negated).then(|| keep(block, format)).flatten()
|
||||||
|
}
|
||||||
|
Block::Aligned { align, margin, block } => Some(Block::Aligned {
|
||||||
|
align: *align,
|
||||||
|
margin: *margin,
|
||||||
|
block: Box::new(keep(block, format)?),
|
||||||
|
}),
|
||||||
|
Block::BlockQuote(inner) => Some(Block::BlockQuote(retain(inner, format))),
|
||||||
|
Block::List { ordered, start, items } => Some(Block::List {
|
||||||
|
ordered: *ordered,
|
||||||
|
start: *start,
|
||||||
|
items: items.iter().map(|item| retain(item, format)).collect(),
|
||||||
|
}),
|
||||||
|
other => Some(other.clone()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
|
|
||||||
|
fn para(text: &str) -> Block {
|
||||||
|
Block::Paragraph(vec![Inline::Text(text.into())])
|
||||||
|
}
|
||||||
|
|
||||||
|
fn gate(formats: &[&str], negated: bool, block: Block) -> Block {
|
||||||
|
Block::Gated {
|
||||||
|
formats: formats.iter().map(|id| id.to_string()).collect(),
|
||||||
|
negated,
|
||||||
|
block: Box::new(block),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn doc(blocks: Vec<Block>) -> Doc {
|
||||||
|
Doc { blocks, first_h1: None }
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_document_without_gates_is_borrowed_unchanged() {
|
||||||
|
let source = doc(vec![para("a")]);
|
||||||
|
assert!(matches!(source.for_format("html"), Cow::Borrowed(_)));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn an_only_gate_keeps_the_formats_it_names_and_drops_the_rest() {
|
||||||
|
let source = doc(vec![gate(&["html", "wml"], false, para("a")), para("b")]);
|
||||||
|
assert_eq!(source.for_format("html").blocks, vec![para("a"), para("b")]);
|
||||||
|
assert_eq!(source.for_format("gemtext").blocks, vec![para("b")]);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn an_except_gate_drops_the_formats_it_names() {
|
||||||
|
let source = doc(vec![gate(&["wml"], true, para("a"))]);
|
||||||
|
assert_eq!(source.for_format("wml").blocks, Vec::new());
|
||||||
|
assert_eq!(source.for_format("html").blocks, vec![para("a")]);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn nested_gates_must_both_pass() {
|
||||||
|
let source = doc(vec![gate(&["html", "wml"], false, gate(&["wml"], true, para("a")))]);
|
||||||
|
assert_eq!(source.for_format("html").blocks, vec![para("a")]);
|
||||||
|
assert_eq!(source.for_format("wml").blocks, Vec::new());
|
||||||
|
assert_eq!(source.for_format("text").blocks, Vec::new());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_format_no_gate_names_is_simply_not_named() {
|
||||||
|
// A build without the wml feature therefore hides an `only wml` run,
|
||||||
|
// which is the answer that run asked for.
|
||||||
|
let source = doc(vec![gate(&["wml"], false, para("a"))]);
|
||||||
|
assert_eq!(source.for_format("gemtext").blocks, Vec::new());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn gates_are_resolved_inside_quotes_lists_and_alignment() {
|
||||||
|
let aligned = Block::Aligned {
|
||||||
|
align: Align::Center,
|
||||||
|
margin: None,
|
||||||
|
block: Box::new(gate(&["html"], false, para("c"))),
|
||||||
|
};
|
||||||
|
let source = doc(vec![
|
||||||
|
Block::BlockQuote(vec![gate(&["html"], false, para("a")), para("b")]),
|
||||||
|
Block::List {
|
||||||
|
ordered: false,
|
||||||
|
start: 1,
|
||||||
|
items: vec![vec![gate(&["html"], false, para("x"))], vec![para("y")]],
|
||||||
|
},
|
||||||
|
aligned,
|
||||||
|
]);
|
||||||
|
|
||||||
|
let kept = source.for_format("html");
|
||||||
|
assert_eq!(kept.blocks[0], Block::BlockQuote(vec![para("a"), para("b")]));
|
||||||
|
let Block::List { items, .. } = &kept.blocks[1] else { panic!("expected a list") };
|
||||||
|
assert_eq!(items, &vec![vec![para("x")], vec![para("y")]]);
|
||||||
|
assert_eq!(
|
||||||
|
kept.blocks[2],
|
||||||
|
Block::Aligned { align: Align::Center, margin: None, block: Box::new(para("c")) }
|
||||||
|
);
|
||||||
|
|
||||||
|
let dropped = source.for_format("text");
|
||||||
|
assert_eq!(dropped.blocks[0], Block::BlockQuote(vec![para("b")]));
|
||||||
|
let Block::List { items, .. } = &dropped.blocks[1] else { panic!("expected a list") };
|
||||||
|
assert_eq!(items, &vec![Vec::new(), vec![para("y")]]);
|
||||||
|
// An alignment wrapping nothing that survives goes with it.
|
||||||
|
assert_eq!(dropped.blocks.len(), 2);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_title_survives_a_gate_around_the_heading() {
|
||||||
|
// One page, one title, whichever format asks for it.
|
||||||
|
let source = Doc {
|
||||||
|
blocks: vec![gate(
|
||||||
|
&["html"],
|
||||||
|
false,
|
||||||
|
Block::Heading { level: 1, inline: vec![Inline::Text("T".into())] },
|
||||||
|
)],
|
||||||
|
first_h1: Some("T".into()),
|
||||||
|
};
|
||||||
|
assert_eq!(source.for_format("gemtext").first_h1.as_deref(), Some("T"));
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn plain_text_flattens_markup_and_keeps_link_labels() {
|
fn plain_text_flattens_markup_and_keeps_link_labels() {
|
||||||
let inline = vec![
|
let inline = vec![
|
||||||
|
|
|
||||||
|
|
@ -1,9 +1,9 @@
|
||||||
//! Media types for files served byte for byte.
|
//! Media types for files served byte for byte.
|
||||||
//!
|
//!
|
||||||
//! A fixed table rather than a system lookup: Python's `mimetypes` consults
|
//! A fixed table rather than a system lookup. A lookup reads `/etc/mime.types`
|
||||||
//! `/etc/mime.types` where it exists, so smolweb's `Content-Type` for the same
|
//! where it exists, so the same file gets a different `Content-Type` depending on
|
||||||
//! file differs between hosts. Determinism is worth more here than coverage of
|
//! the host. Determinism is worth more here than coverage of the long tail, and
|
||||||
//! the long tail, and an unknown extension has a correct answer anyway.
|
//! an unknown extension has a correct answer anyway.
|
||||||
|
|
||||||
use std::path::Path;
|
use std::path::Path;
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -427,8 +427,6 @@ mod tests {
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn parses_setext_headings() {
|
fn parses_setext_headings() {
|
||||||
// wapdown's parser handles these and md2txt's does not, so unifying the
|
|
||||||
// two parsers gains them for the text formats.
|
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
blocks("Title\n=====\n"),
|
blocks("Title\n=====\n"),
|
||||||
vec![Block::Heading { level: 1, inline: text("Title") }]
|
vec![Block::Heading { level: 1, inline: text("Title") }]
|
||||||
|
|
|
||||||
|
|
@ -82,8 +82,8 @@ fn encode_path(path: &str) -> String {
|
||||||
/// Decode `%XX` escapes, leaving an invalid escape as the literal text it is.
|
/// Decode `%XX` escapes, leaving an invalid escape as the literal text it is.
|
||||||
///
|
///
|
||||||
/// Bytes that do not form valid UTF-8 become U+FFFD, which matches no filename,
|
/// Bytes that do not form valid UTF-8 become U+FFFD, which matches no filename,
|
||||||
/// so a malformed target resolves to nothing rather than erroring. That matches
|
/// so a malformed target resolves to nothing rather than erroring. A test pins
|
||||||
/// Python's lossy `unquote` and is pinned by a test.
|
/// that leniency, so it is not later tightened into an error.
|
||||||
fn percent_decode(raw: &str) -> String {
|
fn percent_decode(raw: &str) -> String {
|
||||||
let bytes = raw.as_bytes();
|
let bytes = raw.as_bytes();
|
||||||
let mut out = Vec::with_capacity(bytes.len());
|
let mut out = Vec::with_capacity(bytes.len());
|
||||||
|
|
@ -145,9 +145,8 @@ mod tests {
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn clamps_traversal_at_the_root() {
|
fn clamps_traversal_at_the_root() {
|
||||||
// Ported from smolweb's TestPathTraversal: these must never reach above
|
// These must never reach above the root, and since nothing is mounted at
|
||||||
// the root, and since nothing is mounted at the clamped path they
|
// the clamped path they resolve to a path that simply does not exist.
|
||||||
// resolve to a path that simply does not exist.
|
|
||||||
assert_eq!(clean("/../../etc/passwd"), "etc/passwd");
|
assert_eq!(clean("/../../etc/passwd"), "etc/passwd");
|
||||||
assert_eq!(clean("/../../../../../../etc/passwd"), "etc/passwd");
|
assert_eq!(clean("/../../../../../../etc/passwd"), "etc/passwd");
|
||||||
assert_eq!(clean("/foo/../../etc/passwd"), "etc/passwd");
|
assert_eq!(clean("/foo/../../etc/passwd"), "etc/passwd");
|
||||||
|
|
@ -159,7 +158,7 @@ mod tests {
|
||||||
// before the clamp runs, or it would be treated as a literal segment.
|
// before the clamp runs, or it would be treated as a literal segment.
|
||||||
assert_eq!(clean("/%2e%2e/etc/passwd"), "etc/passwd");
|
assert_eq!(clean("/%2e%2e/etc/passwd"), "etc/passwd");
|
||||||
assert_eq!(clean("/%2E%2E/etc/passwd"), "etc/passwd");
|
assert_eq!(clean("/%2E%2E/etc/passwd"), "etc/passwd");
|
||||||
// An encoded separator becomes a separator, as Python's unquote does.
|
// An encoded separator becomes a real one, so normalising then sees it.
|
||||||
assert_eq!(clean("/dir%2fpage"), "dir/page");
|
assert_eq!(clean("/dir%2fpage"), "dir/page");
|
||||||
assert_eq!(clean("/hello%20world"), "hello world");
|
assert_eq!(clean("/hello%20world"), "hello world");
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -7,10 +7,10 @@
|
||||||
//! [`crate::directives`].
|
//! [`crate::directives`].
|
||||||
//!
|
//!
|
||||||
//! Together with that module this is the only part of the pipeline that opens
|
//! Together with that module this is the only part of the pipeline that opens
|
||||||
//! files, which gives the root-containment check exactly one home. That closes
|
//! files, which gives the root-containment check exactly one home. An include
|
||||||
//! the traversal smolweb has: md2txt resolves an include target and checks only
|
//! target is canonicalised and required to be inside the root, so a target of
|
||||||
//! that it exists, so `{.include ../../../../etc/passwd}` in any served document
|
//! `../../../../etc/passwd` resolves to nothing instead of being read: checking
|
||||||
//! reads and emits that file.
|
//! only that a target exists is what makes includes a traversal.
|
||||||
|
|
||||||
use std::collections::BTreeSet;
|
use std::collections::BTreeSet;
|
||||||
use std::fs;
|
use std::fs;
|
||||||
|
|
@ -24,7 +24,7 @@ use crate::error::{Error, IncludeReason};
|
||||||
const MAX_DEPTH: usize = 16;
|
const MAX_DEPTH: usize = 16;
|
||||||
/// Caps on the expanded result. The cycle set is per-*stack*, so a diamond —
|
/// Caps on the expanded result. The cycle set is per-*stack*, so a diamond —
|
||||||
/// `a` includes `b` and `c`, both include `d` — fans out exponentially without
|
/// `a` includes `b` and `c`, both include `d` — fans out exponentially without
|
||||||
/// ever repeating a file on one path. smolweb has nothing that stops this.
|
/// ever repeating a file on one path, so only these caps stop it.
|
||||||
const MAX_LINES: usize = 200_000;
|
const MAX_LINES: usize = 200_000;
|
||||||
const MAX_BYTES: usize = 8 * 1024 * 1024;
|
const MAX_BYTES: usize = 8 * 1024 * 1024;
|
||||||
|
|
||||||
|
|
@ -159,7 +159,7 @@ mod tests {
|
||||||
assert_eq!(include_target("see ![[notes.md]] there"), None);
|
assert_eq!(include_target("see ![[notes.md]] there"), None);
|
||||||
assert_eq!(include_target("![[]]"), None);
|
assert_eq!(include_target("![[]]"), None);
|
||||||
assert_eq!(include_target("![[unterminated"), None);
|
assert_eq!(include_target("![[unterminated"), None);
|
||||||
// The directive smolweb also accepted is gone: one spelling, not two.
|
// The brace-directive form is not accepted: one spelling, not two.
|
||||||
assert_eq!(include_target("{.include notes.md}"), None);
|
assert_eq!(include_target("{.include notes.md}"), None);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -263,8 +263,8 @@ mod tests {
|
||||||
#[test]
|
#[test]
|
||||||
fn a_diamond_fan_out_is_stopped_by_the_size_cap() {
|
fn a_diamond_fan_out_is_stopped_by_the_size_cap() {
|
||||||
// Each level doubles and no file repeats on any single path, so neither
|
// Each level doubles and no file repeats on any single path, so neither
|
||||||
// the cycle set nor the depth cap catches it. smolweb expands this until
|
// the cycle set nor the depth cap catches it, which leaves the byte cap
|
||||||
// it runs out of memory.
|
// as the only thing that stops it.
|
||||||
let tree = Tree::new();
|
let tree = Tree::new();
|
||||||
tree.write("leaf.md", &"filler line\n".repeat(64));
|
tree.write("leaf.md", &"filler line\n".repeat(64));
|
||||||
let mut previous = "leaf.md".to_string();
|
let mut previous = "leaf.md".to_string();
|
||||||
|
|
|
||||||
|
|
@ -3,12 +3,12 @@
|
||||||
//! A format is a crate implementing [`Renderer`], registered at startup behind a
|
//! A format is a crate implementing [`Renderer`], registered at startup behind a
|
||||||
//! cargo feature. The trait takes a parsed [`Doc`] rather than source text so
|
//! cargo feature. The trait takes a parsed [`Doc`] rather than source text so
|
||||||
//! that every format reads one parse: gemtext's `=>` link catalogue and the text
|
//! that every format reads one parse: gemtext's `=>` link catalogue and the text
|
||||||
//! formats' `[n]` references must agree about link identity and order, and in
|
//! formats' `[n]` references must agree about link identity and order, which
|
||||||
//! smolweb they can disagree because each library re-parses.
|
//! cannot be relied on when each format parses the source for itself.
|
||||||
//!
|
//!
|
||||||
//! A renderer must not open files or sockets. Includes and art are already
|
//! A renderer must not open files or sockets. Includes, art and format gates are
|
||||||
//! resolved by the time it runs, which is what keeps the root-containment check
|
//! already resolved by the time it runs, which is what keeps the
|
||||||
//! in one place rather than in every format.
|
//! root-containment check in one place rather than in every format.
|
||||||
|
|
||||||
use std::collections::BTreeMap;
|
use std::collections::BTreeMap;
|
||||||
use std::sync::Arc;
|
use std::sync::Arc;
|
||||||
|
|
@ -39,6 +39,18 @@ pub trait Renderer: Send + Sync {
|
||||||
fn render(&self, doc: &Doc, ctx: &RenderCtx<'_>) -> Result<Rendered, Error>;
|
fn render(&self, doc: &Doc, ctx: &RenderCtx<'_>) -> Result<Rendered, Error>;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// What a Mews Profile page needs beyond the document itself (mews.page/spec).
|
||||||
|
/// Present when the site has opted in, absent otherwise, so one `Option` says
|
||||||
|
/// both whether the profile applies and what it applies with.
|
||||||
|
#[derive(Debug, Clone, Copy)]
|
||||||
|
pub struct Mews<'a> {
|
||||||
|
/// The site's own hostnames, for judging an image `src` as same-site (4.2).
|
||||||
|
pub hosts: &'a [String],
|
||||||
|
/// The site's Atom feed, declared in the head (6.2). Absolute, since a
|
||||||
|
/// capsule need not host the feed its web site publishes.
|
||||||
|
pub feed: Option<&'a str>,
|
||||||
|
}
|
||||||
|
|
||||||
pub struct RenderCtx<'a> {
|
pub struct RenderCtx<'a> {
|
||||||
/// Canonical root-relative URL of this document: `/`, `/about`, `/dir/`. A
|
/// Canonical root-relative URL of this document: `/`, `/about`, `/dir/`. A
|
||||||
/// format that addresses sub-documents builds their URLs from this.
|
/// format that addresses sub-documents builds their URLs from this.
|
||||||
|
|
@ -48,6 +60,10 @@ pub struct RenderCtx<'a> {
|
||||||
pub title: &'a str,
|
pub title: &'a str,
|
||||||
pub settings: &'a PageSettings,
|
pub settings: &'a PageSettings,
|
||||||
pub width: Option<u16>,
|
pub width: Option<u16>,
|
||||||
|
/// Mews Profile settings, present when the site has opted in. Threaded
|
||||||
|
/// through so a renderer that cares can switch its markup without reaching
|
||||||
|
/// into site configuration itself.
|
||||||
|
pub mews: Option<Mews<'a>>,
|
||||||
}
|
}
|
||||||
|
|
||||||
pub struct Rendered {
|
pub struct Rendered {
|
||||||
|
|
@ -112,6 +128,7 @@ impl Registry {
|
||||||
url: &str,
|
url: &str,
|
||||||
settings: &PageSettings,
|
settings: &PageSettings,
|
||||||
fallback_title: &str,
|
fallback_title: &str,
|
||||||
|
mews: Option<Mews<'_>>,
|
||||||
) -> Result<Page, Error> {
|
) -> Result<Page, Error> {
|
||||||
let title = settings
|
let title = settings
|
||||||
.title
|
.title
|
||||||
|
|
@ -131,9 +148,17 @@ impl Registry {
|
||||||
// caller passing an unknown id is a bug, not bad input.
|
// caller passing an unknown id is a bug, not bad input.
|
||||||
Error::config(format!("no renderer provides the format '{id}'"))
|
Error::config(format!("no renderer provides the format '{id}'"))
|
||||||
})?;
|
})?;
|
||||||
let ctx =
|
let ctx = RenderCtx {
|
||||||
RenderCtx { url, title: &page.title, settings, width: renderer.default_width() };
|
url,
|
||||||
let rendered = renderer.render(doc, &ctx)?;
|
title: &page.title,
|
||||||
|
settings,
|
||||||
|
width: renderer.default_width(),
|
||||||
|
mews,
|
||||||
|
};
|
||||||
|
// Format gates are resolved here, so no renderer meets one and a
|
||||||
|
// gated run costs nothing extra in the cache: bodies are already
|
||||||
|
// kept per format.
|
||||||
|
let rendered = renderer.render(&doc.for_format(id), &ctx)?;
|
||||||
for part in rendered.parts {
|
for part in rendered.parts {
|
||||||
page.parts.insert((id.clone(), part.slug), part.body);
|
page.parts.insert((id.clone(), part.slug), part.body);
|
||||||
}
|
}
|
||||||
|
|
@ -202,9 +227,15 @@ mod tests {
|
||||||
self.width
|
self.width
|
||||||
}
|
}
|
||||||
|
|
||||||
fn render(&self, _doc: &Doc, ctx: &RenderCtx<'_>) -> Result<Rendered, Error> {
|
fn render(&self, doc: &Doc, ctx: &RenderCtx<'_>) -> Result<Rendered, Error> {
|
||||||
Ok(Rendered::body(
|
Ok(Rendered::body(
|
||||||
format!("{} {} {:?} {}", self.id, ctx.url, ctx.width, ctx.title).into_bytes(),
|
// The blocks are printed too, so what a format was handed —
|
||||||
|
// after gates — is visible in the body.
|
||||||
|
format!(
|
||||||
|
"{} {} {:?} {} {:?} {:?}",
|
||||||
|
self.id, ctx.url, ctx.width, ctx.title, doc.blocks, ctx.mews
|
||||||
|
)
|
||||||
|
.into_bytes(),
|
||||||
))
|
))
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -232,7 +263,7 @@ mod tests {
|
||||||
#[test]
|
#[test]
|
||||||
fn renders_only_the_formats_asked_for() {
|
fn renders_only_the_formats_asked_for() {
|
||||||
let page = registry()
|
let page = registry()
|
||||||
.page(&["one".to_string()], &doc(None), "/x", &PageSettings::default(), "x")
|
.page(&["one".to_string()], &doc(None), "/x", &PageSettings::default(), "x", None)
|
||||||
.unwrap();
|
.unwrap();
|
||||||
assert!(page.body("one").is_some());
|
assert!(page.body("one").is_some());
|
||||||
assert!(page.body("two").is_none(), "a format no listener serves is not rendered");
|
assert!(page.body("two").is_none(), "a format no listener serves is not rendered");
|
||||||
|
|
@ -241,33 +272,72 @@ mod tests {
|
||||||
#[test]
|
#[test]
|
||||||
fn each_renderer_gets_its_own_declared_width() {
|
fn each_renderer_gets_its_own_declared_width() {
|
||||||
let formats = vec!["one".to_string(), "two".to_string()];
|
let formats = vec!["one".to_string(), "two".to_string()];
|
||||||
let page =
|
let page = registry()
|
||||||
registry().page(&formats, &doc(None), "/x", &PageSettings::default(), "x").unwrap();
|
.page(&formats, &doc(None), "/x", &PageSettings::default(), "x", None)
|
||||||
|
.unwrap();
|
||||||
assert!(String::from_utf8_lossy(page.body("one").unwrap()).contains("None"));
|
assert!(String::from_utf8_lossy(page.body("one").unwrap()).contains("None"));
|
||||||
assert!(String::from_utf8_lossy(page.body("two").unwrap()).contains("Some(80)"));
|
assert!(String::from_utf8_lossy(page.body("two").unwrap()).contains("Some(80)"));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn mews_settings_reach_the_renderer_when_the_site_opted_in() {
|
||||||
|
let formats = vec!["one".to_string()];
|
||||||
|
let hosts = vec!["example.test".to_string()];
|
||||||
|
let mews = Mews { hosts: &hosts, feed: Some("https://example.test/feed.xml") };
|
||||||
|
let page = registry()
|
||||||
|
.page(&formats, &doc(None), "/x", &PageSettings::default(), "x", Some(mews))
|
||||||
|
.unwrap();
|
||||||
|
let body = String::from_utf8_lossy(page.body("one").unwrap()).to_string();
|
||||||
|
assert!(body.contains("example.test"), "{body}");
|
||||||
|
assert!(body.contains("feed.xml"), "{body}");
|
||||||
|
|
||||||
|
let unset = registry()
|
||||||
|
.page(&formats, &doc(None), "/x", &PageSettings::default(), "x", None)
|
||||||
|
.unwrap();
|
||||||
|
assert!(String::from_utf8_lossy(unset.body("one").unwrap()).contains("None"));
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn the_title_falls_back_from_config_to_heading_to_file_name() {
|
fn the_title_falls_back_from_config_to_heading_to_file_name() {
|
||||||
let configured = PageSettings { title: Some("Configured".into()), ..Default::default() };
|
let configured = PageSettings { title: Some("Configured".into()), ..Default::default() };
|
||||||
let formats = vec!["one".to_string()];
|
let formats = vec!["one".to_string()];
|
||||||
let reg = registry();
|
let reg = registry();
|
||||||
|
|
||||||
let from_config = reg.page(&formats, &doc(Some("Heading")), "/x", &configured, "stem");
|
let from_config =
|
||||||
|
reg.page(&formats, &doc(Some("Heading")), "/x", &configured, "stem", None);
|
||||||
assert_eq!(from_config.unwrap().title, "Configured");
|
assert_eq!(from_config.unwrap().title, "Configured");
|
||||||
|
|
||||||
let from_heading =
|
let from_heading =
|
||||||
reg.page(&formats, &doc(Some("Heading")), "/x", &PageSettings::default(), "stem");
|
reg.page(&formats, &doc(Some("Heading")), "/x", &PageSettings::default(), "stem", None);
|
||||||
assert_eq!(from_heading.unwrap().title, "Heading");
|
assert_eq!(from_heading.unwrap().title, "Heading");
|
||||||
|
|
||||||
let from_stem = reg.page(&formats, &doc(None), "/x", &PageSettings::default(), "stem");
|
let from_stem =
|
||||||
|
reg.page(&formats, &doc(None), "/x", &PageSettings::default(), "stem", None);
|
||||||
assert_eq!(from_stem.unwrap().title, "stem");
|
assert_eq!(from_stem.unwrap().title, "stem");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_gated_block_reaches_only_the_formats_it_names() {
|
||||||
|
let gated = Doc {
|
||||||
|
blocks: vec![Block::Gated {
|
||||||
|
formats: vec!["one".to_string()],
|
||||||
|
negated: false,
|
||||||
|
block: Box::new(Block::Paragraph(vec![Inline::Text("secret".into())])),
|
||||||
|
}],
|
||||||
|
first_h1: None,
|
||||||
|
};
|
||||||
|
let formats = vec!["one".to_string(), "two".to_string()];
|
||||||
|
let page =
|
||||||
|
registry().page(&formats, &gated, "/x", &PageSettings::default(), "x", None).unwrap();
|
||||||
|
// The stub prints the blocks it was handed, so presence is visible.
|
||||||
|
assert!(String::from_utf8_lossy(page.body("one").unwrap()).contains("secret"));
|
||||||
|
assert!(!String::from_utf8_lossy(page.body("two").unwrap()).contains("secret"));
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn an_unknown_format_is_a_bug_not_bad_input() {
|
fn an_unknown_format_is_a_bug_not_bad_input() {
|
||||||
let err = registry()
|
let err = registry()
|
||||||
.page(&["absent".to_string()], &doc(None), "/x", &PageSettings::default(), "x")
|
.page(&["absent".to_string()], &doc(None), "/x", &PageSettings::default(), "x", None)
|
||||||
.unwrap_err();
|
.unwrap_err();
|
||||||
assert!(err.to_string().contains("no renderer provides"), "{err}");
|
assert!(err.to_string().contains("no renderer provides"), "{err}");
|
||||||
}
|
}
|
||||||
|
|
|
||||||
132
core/src/site.rs
132
core/src/site.rs
|
|
@ -16,13 +16,23 @@ use crate::error::Error;
|
||||||
use crate::mime;
|
use crate::mime;
|
||||||
use crate::parse;
|
use crate::parse;
|
||||||
use crate::path::{clean_path, url_for};
|
use crate::path::{clean_path, url_for};
|
||||||
use crate::render::{Page, Registry, title_from_stem};
|
use crate::render::{Mews, Page, Registry, title_from_stem};
|
||||||
|
|
||||||
/// How deep a chain of server-side redirects `resolve_flat` will follow. Every
|
/// How deep a chain of server-side redirects `resolve_flat` will follow. Every
|
||||||
/// redirect currently points at a directly resolvable document, so one hop is
|
/// redirect currently points at a directly resolvable document, so one hop is
|
||||||
/// always enough; the cap is there so a future rule cannot loop.
|
/// always enough; the cap is there so a future rule cannot loop.
|
||||||
const MAX_FLAT_HOPS: usize = 5;
|
const MAX_FLAT_HOPS: usize = 5;
|
||||||
|
|
||||||
|
/// Name a Mews page's stylesheet `<link>` carries (`wap::xhtmlmp::MEWS_HEAD_EXTRA`).
|
||||||
|
const MEWS_STYLESHEET_NAME: &str = "mews-0.1.css";
|
||||||
|
|
||||||
|
/// The stylesheet itself, byte for byte as published at mews.page/spec (5.1).
|
||||||
|
/// Served from here rather than from a file an operator must place in every
|
||||||
|
/// site's root: the link's name and content are pinned to the spec version a
|
||||||
|
/// site opts into, not something a site chooses, so there is nothing for a
|
||||||
|
/// copy on disk to add and every way for it to go stale or go missing.
|
||||||
|
const MEWS_STYLESHEET: &[u8] = include_bytes!("../assets/mews-0.1.css");
|
||||||
|
|
||||||
/// What a request path resolved to.
|
/// What a request path resolved to.
|
||||||
#[derive(Debug)]
|
#[derive(Debug)]
|
||||||
pub enum Resolution {
|
pub enum Resolution {
|
||||||
|
|
@ -38,6 +48,9 @@ pub enum Resource {
|
||||||
Document { url: String, page: Arc<Page> },
|
Document { url: String, page: Arc<Page> },
|
||||||
/// A file served byte for byte, streamed rather than buffered.
|
/// A file served byte for byte, streamed rather than buffered.
|
||||||
Raw { path: PathBuf, media_type: &'static str },
|
Raw { path: PathBuf, media_type: &'static str },
|
||||||
|
/// A byte slice built into the binary, served like `Raw` but with no file
|
||||||
|
/// to open: currently only the Mews Profile stylesheet.
|
||||||
|
Embedded { bytes: &'static [u8], media_type: &'static str },
|
||||||
/// A sub-document of a page, addressed under the page's own URL.
|
/// A sub-document of a page, addressed under the page's own URL.
|
||||||
///
|
///
|
||||||
/// Only a paginating format produces these, so whether this slug exists
|
/// Only a paginating format produces these, so whether this slug exists
|
||||||
|
|
@ -59,12 +72,33 @@ pub struct Site {
|
||||||
/// Formats every page here is rendered into: the union of what the enabled
|
/// Formats every page here is rendered into: the union of what the enabled
|
||||||
/// listeners can serve, so a site with no HTTP listener never renders HTML.
|
/// listeners can serve, so a site with no HTTP listener never renders HTML.
|
||||||
formats: Vec<String>,
|
formats: Vec<String>,
|
||||||
|
/// Whether this site's `xhtmlmp`/`html` output is rendered as Mews Profile
|
||||||
|
/// pages (`SiteSpec::mews_profile`). Validation requires every site sharing
|
||||||
|
/// this root to agree, so one value is correct for all of them.
|
||||||
|
mews_profile: bool,
|
||||||
|
/// This site's own hostnames, used to judge an image `src` as same-site
|
||||||
|
/// when `mews_profile` is set. Taken from whichever configured site name
|
||||||
|
/// opened this root first; a root reached under several names with
|
||||||
|
/// differing host lists keeps only that one's, which is an accepted
|
||||||
|
/// simplification rather than a union of them all.
|
||||||
|
hosts: Vec<String>,
|
||||||
|
/// This site's Atom feed (`SiteSpec::feed`), declared in the head of a
|
||||||
|
/// Mews page. Validation requires every site sharing this root to agree,
|
||||||
|
/// since the declaration is part of the cached markup.
|
||||||
|
feed: Option<String>,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl Site {
|
impl Site {
|
||||||
/// Open a content root, canonicalising it so containment checks have a
|
/// Open a content root, canonicalising it so containment checks have a
|
||||||
/// stable base and a missing root fails now rather than per request.
|
/// stable base and a missing root fails now rather than per request.
|
||||||
pub fn new(root: &Path, registry: Arc<Registry>, formats: Vec<String>) -> Result<Self, Error> {
|
pub fn new(
|
||||||
|
root: &Path,
|
||||||
|
registry: Arc<Registry>,
|
||||||
|
formats: Vec<String>,
|
||||||
|
mews_profile: bool,
|
||||||
|
hosts: Vec<String>,
|
||||||
|
feed: Option<String>,
|
||||||
|
) -> Result<Self, Error> {
|
||||||
let root =
|
let root =
|
||||||
root.canonicalize().map_err(|cause| Error::Io { path: root.to_path_buf(), cause })?;
|
root.canonicalize().map_err(|cause| Error::Io { path: root.to_path_buf(), cause })?;
|
||||||
if !root.is_dir() {
|
if !root.is_dir() {
|
||||||
|
|
@ -77,6 +111,9 @@ impl Site {
|
||||||
built_in: Arc::new(DirConfig::default()),
|
built_in: Arc::new(DirConfig::default()),
|
||||||
registry,
|
registry,
|
||||||
formats,
|
formats,
|
||||||
|
mews_profile,
|
||||||
|
hosts,
|
||||||
|
feed,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -92,8 +129,20 @@ impl Site {
|
||||||
/// | `/foo` | `foo.md`, else `foo/index.md` |
|
/// | `/foo` | `foo.md`, else `foo/index.md` |
|
||||||
/// | `/foo.md` | redirects to `/foo` |
|
/// | `/foo.md` | redirects to `/foo` |
|
||||||
/// | `/img.png` | the file itself, by media type |
|
/// | `/img.png` | the file itself, by media type |
|
||||||
|
/// | `/mews-0.1.css` | the embedded stylesheet, when `mews_profile` is set |
|
||||||
pub fn resolve(&self, url_path: &str) -> Result<Resolution, Error> {
|
pub fn resolve(&self, url_path: &str) -> Result<Resolution, Error> {
|
||||||
let Some(clean) = clean_path(url_path) else { return Ok(Resolution::NotFound) };
|
let Some(clean) = clean_path(url_path) else { return Ok(Resolution::NotFound) };
|
||||||
|
|
||||||
|
// Checked by name only, so the link resolves the same at any depth: a
|
||||||
|
// page under `/foo/bar` carries the same relative `href` as one at the
|
||||||
|
// root, and both must land here rather than on a per-directory copy.
|
||||||
|
if self.mews_profile && clean.rsplit('/').next() == Some(MEWS_STYLESHEET_NAME) {
|
||||||
|
return Ok(Resolution::Found(Resource::Embedded {
|
||||||
|
bytes: MEWS_STYLESHEET,
|
||||||
|
media_type: mime::media_type(Path::new(MEWS_STYLESHEET_NAME)),
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
|
||||||
let Some(source) = self.file_for(&clean) else { return self.part_for(&clean) };
|
let Some(source) = self.file_for(&clean) else { return self.part_for(&clean) };
|
||||||
|
|
||||||
// `/foo.md` always redirects to its extensionless form, so one
|
// `/foo.md` always redirects to its extensionless form, so one
|
||||||
|
|
@ -213,7 +262,10 @@ impl Site {
|
||||||
let settings = self.dir_config(dir)?.settings_for(name)?;
|
let settings = self.dir_config(dir)?.settings_for(name)?;
|
||||||
let doc = parse::document(source, &self.root)?;
|
let doc = parse::document(source, &self.root)?;
|
||||||
let stem = source.file_stem().and_then(|s| s.to_str()).unwrap_or_default();
|
let stem = source.file_stem().and_then(|s| s.to_str()).unwrap_or_default();
|
||||||
self.registry.page(&self.formats, &doc, url, &settings, &title_from_stem(stem))
|
let mews = self
|
||||||
|
.mews_profile
|
||||||
|
.then_some(Mews { hosts: self.hosts.as_slice(), feed: self.feed.as_deref() });
|
||||||
|
self.registry.page(&self.formats, &doc, url, &settings, &title_from_stem(stem), mews)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -249,7 +301,7 @@ mod tests {
|
||||||
}
|
}
|
||||||
|
|
||||||
fn render(&self, doc: &Doc, ctx: &RenderCtx<'_>) -> Result<Rendered, Error> {
|
fn render(&self, doc: &Doc, ctx: &RenderCtx<'_>) -> Result<Rendered, Error> {
|
||||||
let mut body = format!("title={}\n", ctx.title);
|
let mut body = format!("title={}\nmews={:?}\n", ctx.title, ctx.mews);
|
||||||
for block in &doc.blocks {
|
for block in &doc.blocks {
|
||||||
body.push_str(&format!("{block:?}\n"));
|
body.push_str(&format!("{block:?}\n"));
|
||||||
}
|
}
|
||||||
|
|
@ -264,11 +316,11 @@ mod tests {
|
||||||
}
|
}
|
||||||
|
|
||||||
fn open(root: &Path) -> Site {
|
fn open(root: &Path) -> Site {
|
||||||
Site::new(root, registry(), vec!["stub".to_string()]).unwrap()
|
Site::new(root, registry(), vec!["stub".to_string()], false, Vec::new(), None).unwrap()
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Mirrors smolweb's `tests/conftest.py` fixture, so its assertions port
|
/// One of each kind of thing a request can land on, shared by the resolution
|
||||||
/// across directly.
|
/// tests below.
|
||||||
fn fixture() -> (tempfile::TempDir, Site) {
|
fn fixture() -> (tempfile::TempDir, Site) {
|
||||||
let dir = tempfile::tempdir().unwrap();
|
let dir = tempfile::tempdir().unwrap();
|
||||||
let root = dir.path();
|
let root = dir.path();
|
||||||
|
|
@ -447,11 +499,34 @@ mod tests {
|
||||||
assert_not_found(&site, "/nope");
|
assert_not_found(&site, "/nope");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_mews_stylesheet_is_served_embedded_when_the_profile_is_on() {
|
||||||
|
let (dir, _site) = fixture();
|
||||||
|
let mews_site =
|
||||||
|
Site::new(dir.path(), registry(), vec!["stub".to_string()], true, Vec::new(), None)
|
||||||
|
.unwrap();
|
||||||
|
for path in ["/mews-0.1.css", "/dir/mews-0.1.css"] {
|
||||||
|
match mews_site.resolve(path).unwrap() {
|
||||||
|
Resolution::Found(Resource::Embedded { bytes, media_type }) => {
|
||||||
|
assert_eq!(media_type, "text/css; charset=utf-8");
|
||||||
|
assert_eq!(bytes, MEWS_STYLESHEET, "{path}");
|
||||||
|
}
|
||||||
|
other => panic!("expected the embedded stylesheet at {path}, got {other:?}"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_mews_stylesheet_name_is_an_ordinary_missing_file_when_the_profile_is_off() {
|
||||||
|
let (_dir, site) = fixture();
|
||||||
|
assert_not_found(&site, "/mews-0.1.css");
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn a_bare_unresolvable_segment_resolves_to_nothing() {
|
fn a_bare_unresolvable_segment_resolves_to_nothing() {
|
||||||
// Regression carried over from smolweb: the Python reached this path
|
// The obvious implementation splits on the last `/`, where a bare
|
||||||
// through `rpartition("/")`, where a bare top-level segment yields an
|
// top-level segment yields an empty parent that must not then be read as
|
||||||
// empty parent that must not be read as the root index.
|
// the root index.
|
||||||
let (_dir, site) = fixture();
|
let (_dir, site) = fixture();
|
||||||
assert_not_found(&site, "/totally-unresolvable-segment");
|
assert_not_found(&site, "/totally-unresolvable-segment");
|
||||||
}
|
}
|
||||||
|
|
@ -550,11 +625,25 @@ mod tests {
|
||||||
assert!(matches!(site.resolve("/about"), Err(Error::Toml { .. })));
|
assert!(matches!(site.resolve("/about"), Err(Error::Toml { .. })));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn mews_profile_and_hosts_reach_the_render_context() {
|
||||||
|
let (dir, _site) = fixture();
|
||||||
|
let hosts = vec!["example.test".to_string()];
|
||||||
|
let mews_site =
|
||||||
|
Site::new(dir.path(), registry(), vec!["stub".to_string()], true, hosts, None).unwrap();
|
||||||
|
let (_, page) = document(&mews_site, "/about");
|
||||||
|
let body = String::from_utf8(page.body("stub").unwrap().to_vec()).unwrap();
|
||||||
|
assert!(body.contains("hosts: [\"example.test\"]"), "{body}");
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn a_missing_root_is_rejected_at_construction() {
|
fn a_missing_root_is_rejected_at_construction() {
|
||||||
let dir = tempfile::tempdir().unwrap();
|
let dir = tempfile::tempdir().unwrap();
|
||||||
let absent = dir.path().join("absent");
|
let absent = dir.path().join("absent");
|
||||||
assert!(matches!(Site::new(&absent, registry(), vec![]), Err(Error::Io { .. })));
|
assert!(matches!(
|
||||||
|
Site::new(&absent, registry(), vec![], false, Vec::new(), None),
|
||||||
|
Err(Error::Io { .. })
|
||||||
|
));
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
|
|
@ -562,15 +651,18 @@ mod tests {
|
||||||
let dir = tempfile::tempdir().unwrap();
|
let dir = tempfile::tempdir().unwrap();
|
||||||
let file = dir.path().join("not-a-dir");
|
let file = dir.path().join("not-a-dir");
|
||||||
fs::write(&file, "x").unwrap();
|
fs::write(&file, "x").unwrap();
|
||||||
assert!(matches!(Site::new(&file, registry(), vec![]), Err(Error::Config { .. })));
|
assert!(matches!(
|
||||||
|
Site::new(&file, registry(), vec![], false, Vec::new(), None),
|
||||||
|
Err(Error::Config { .. })
|
||||||
|
));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod part_tests {
|
mod part_tests {
|
||||||
//! Ported from smolweb's `TestWmlCardUrls`. A stub renderer stands in for a
|
//! A stub renderer stands in for a paginating format, so these rules are
|
||||||
//! paginating format, so these rules are tested without the WML crate: core
|
//! tested without the WML crate: core does not know which formats paginate,
|
||||||
//! does not know which formats paginate, which is the point.
|
//! which is the point.
|
||||||
|
|
||||||
use std::fs;
|
use std::fs;
|
||||||
|
|
||||||
|
|
@ -622,7 +714,15 @@ mod part_tests {
|
||||||
|
|
||||||
let mut registry = Registry::new();
|
let mut registry = Registry::new();
|
||||||
registry.insert(Arc::new(Paginating)).unwrap();
|
registry.insert(Arc::new(Paginating)).unwrap();
|
||||||
let site = Site::new(root, Arc::new(registry), vec!["paginating".to_string()]).unwrap();
|
let site = Site::new(
|
||||||
|
root,
|
||||||
|
Arc::new(registry),
|
||||||
|
vec!["paginating".to_string()],
|
||||||
|
false,
|
||||||
|
Vec::new(),
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
(dir, site)
|
(dir, site)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -38,7 +38,15 @@ impl SiteSet {
|
||||||
SiteSet { sites: Vec::new(), by_host: HashMap::new(), by_name: HashMap::new() };
|
SiteSet { sites: Vec::new(), by_host: HashMap::new(), by_name: HashMap::new() };
|
||||||
|
|
||||||
for (name, spec) in &config.site {
|
for (name, spec) in &config.site {
|
||||||
let site = Site::new(&spec.root, registry.clone(), formats.clone())?;
|
let hosts = spec.hosts.iter().filter_map(|host| normalize_host(host)).collect();
|
||||||
|
let site = Site::new(
|
||||||
|
&spec.root,
|
||||||
|
registry.clone(),
|
||||||
|
formats.clone(),
|
||||||
|
spec.mews_profile,
|
||||||
|
hosts,
|
||||||
|
spec.feed.clone(),
|
||||||
|
)?;
|
||||||
// One `Site` per distinct root, so sites sharing a folder share its
|
// One `Site` per distinct root, so sites sharing a folder share its
|
||||||
// caches rather than each building their own.
|
// caches rather than each building their own.
|
||||||
let index = match set.sites.iter().position(|open| open.root() == site.root()) {
|
let index = match set.sites.iter().position(|open| open.root() == site.root()) {
|
||||||
|
|
|
||||||
37
docker/itsybitsy.toml
Normal file
37
docker/itsybitsy.toml
Normal file
|
|
@ -0,0 +1,37 @@
|
||||||
|
version = 1
|
||||||
|
|
||||||
|
[site.demo]
|
||||||
|
root = "./content"
|
||||||
|
hosts = ["localhost", "127.0.0.1"]
|
||||||
|
|
||||||
|
[listener.web]
|
||||||
|
protocol = "http"
|
||||||
|
bind = "0.0.0.0:8080"
|
||||||
|
formats = ["wml", "xhtmlmp", "html"]
|
||||||
|
default_site = "demo"
|
||||||
|
|
||||||
|
[listener.spartan]
|
||||||
|
protocol = "spartan"
|
||||||
|
bind = "0.0.0.0:3000"
|
||||||
|
formats = ["gemtext"]
|
||||||
|
default_site = "demo"
|
||||||
|
|
||||||
|
[listener.nex]
|
||||||
|
protocol = "nex"
|
||||||
|
bind = "0.0.0.0:1900"
|
||||||
|
formats = ["text"]
|
||||||
|
site = "demo"
|
||||||
|
|
||||||
|
# Plaintext: a TLS terminator in front of the container owns 1965 and forwards
|
||||||
|
# here, as it does when deployed bare.
|
||||||
|
[listener.gemini]
|
||||||
|
protocol = "gemini"
|
||||||
|
bind = "0.0.0.0:1965"
|
||||||
|
formats = ["gemtext"]
|
||||||
|
default_site = "demo"
|
||||||
|
|
||||||
|
[listener.gopher]
|
||||||
|
protocol = "gopher"
|
||||||
|
bind = "0.0.0.0:7070"
|
||||||
|
formats = ["text"]
|
||||||
|
site = "demo"
|
||||||
|
|
@ -113,7 +113,7 @@ impl Writer {
|
||||||
// number in the text to keep the sequence readable.
|
// number in the text to keep the sequence readable.
|
||||||
if *ordered {
|
if *ordered {
|
||||||
self.line(&format!("* {}. {text}", start + index as u64));
|
self.line(&format!("* {}. {text}", start + index as u64));
|
||||||
} else {
|
} else if !is_only_links_item(item) {
|
||||||
self.line(&format!("* {text}"));
|
self.line(&format!("* {text}"));
|
||||||
}
|
}
|
||||||
for block in item {
|
for block in item {
|
||||||
|
|
@ -143,6 +143,9 @@ impl Writer {
|
||||||
// Alignment and pagination have no expression here. Unwrapping keeps
|
// Alignment and pagination have no expression here. Unwrapping keeps
|
||||||
// the content; the presentation is simply not available.
|
// the content; the presentation is simply not available.
|
||||||
Block::Aligned { block, .. } => self.block(block),
|
Block::Aligned { block, .. } => self.block(block),
|
||||||
|
// Gates are filtered out before rendering; keeping the content is
|
||||||
|
// the harmless reading if one ever arrives here.
|
||||||
|
Block::Gated { block, .. } => self.block(block),
|
||||||
Block::CardBreak { .. } => {}
|
Block::CardBreak { .. } => {}
|
||||||
// Raw markup would be shown literally, which is worse than omitting it.
|
// Raw markup would be shown literally, which is worse than omitting it.
|
||||||
Block::Html(_) => {}
|
Block::Html(_) => {}
|
||||||
|
|
@ -198,6 +201,16 @@ impl Writer {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Whether a list item holds links and nothing else worth printing, in which
|
||||||
|
/// case the `=>` lines are the whole item and a marker line above them would
|
||||||
|
/// list every entry twice -- the same reading [`is_only_links`] gives a
|
||||||
|
/// paragraph. Markup other than one paragraph keeps its marker, since the
|
||||||
|
/// shape is then doing something a link catalogue alone would not carry; so
|
||||||
|
/// does an ordered item, whose number would go with it.
|
||||||
|
fn is_only_links_item(item: &[Block]) -> bool {
|
||||||
|
matches!(item, [Block::Paragraph(inline)] if is_only_links(inline))
|
||||||
|
}
|
||||||
|
|
||||||
/// Whether a paragraph holds links and nothing else worth printing.
|
/// Whether a paragraph holds links and nothing else worth printing.
|
||||||
fn is_only_links(inline: &[Inline]) -> bool {
|
fn is_only_links(inline: &[Inline]) -> bool {
|
||||||
let mut saw_link = false;
|
let mut saw_link = false;
|
||||||
|
|
@ -292,7 +305,7 @@ mod tests_support {
|
||||||
pub fn render(markdown: &str) -> String {
|
pub fn render(markdown: &str) -> String {
|
||||||
let doc = parse::markdown(markdown).unwrap();
|
let doc = parse::markdown(markdown).unwrap();
|
||||||
let settings = PageSettings::default();
|
let settings = PageSettings::default();
|
||||||
let ctx = RenderCtx { url: "/x", title: "T", settings: &settings, width: None };
|
let ctx = RenderCtx { url: "/x", title: "T", settings: &settings, width: None, mews: None };
|
||||||
let out = Gemtext.render(&doc, &ctx).unwrap();
|
let out = Gemtext.render(&doc, &ctx).unwrap();
|
||||||
String::from_utf8(out.body).unwrap()
|
String::from_utf8(out.body).unwrap()
|
||||||
}
|
}
|
||||||
|
|
@ -367,6 +380,19 @@ mod tests {
|
||||||
assert_eq!(render("- see [x](/x)\n"), "* see x\n=> /x x\n");
|
assert_eq!(render("- see [x](/x)\n"), "* see x\n=> /x x\n");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_list_of_bare_links_is_a_run_of_link_lines() {
|
||||||
|
// The shape a generator wants for a dated index: a real list, so the
|
||||||
|
// markup formats can give it <ul><li>, without gemtext printing both
|
||||||
|
// the marker line and the link line for every entry.
|
||||||
|
assert_eq!(render("- [a](/a)\n- [b](/b)\n"), "=> /a a\n=> /b b\n");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn an_ordered_list_of_bare_links_keeps_its_numbers() {
|
||||||
|
assert_eq!(render("1. [a](/a)\n"), "* 1. a\n=> /a a\n");
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn code_blocks_become_fences_carrying_their_info_string() {
|
fn code_blocks_become_fences_carrying_their_info_string() {
|
||||||
assert_eq!(render("```rust\nlet x = 1;\n```\n"), "```rust\nlet x = 1;\n```\n");
|
assert_eq!(render("```rust\nlet x = 1;\n```\n"), "```rust\nlet x = 1;\n```\n");
|
||||||
|
|
@ -395,7 +421,7 @@ mod tests {
|
||||||
// Pagination is a WML concern; a Gemini client scrolls one document.
|
// Pagination is a WML concern; a Gemini client scrolls one document.
|
||||||
let doc = parse::markdown("a\n\nb\n").unwrap();
|
let doc = parse::markdown("a\n\nb\n").unwrap();
|
||||||
let settings = PageSettings::default();
|
let settings = PageSettings::default();
|
||||||
let ctx = RenderCtx { url: "/x", title: "T", settings: &settings, width: None };
|
let ctx = RenderCtx { url: "/x", title: "T", settings: &settings, width: None, mews: None };
|
||||||
let plain = String::from_utf8(Gemtext.render(&doc, &ctx).unwrap().body).unwrap();
|
let plain = String::from_utf8(Gemtext.render(&doc, &ctx).unwrap().body).unwrap();
|
||||||
assert_eq!(plain, "a\n\nb\n");
|
assert_eq!(plain, "a\n\nb\n");
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -2,9 +2,7 @@
|
||||||
//!
|
//!
|
||||||
//! Plain text has no markup to carry emphasis, so it is dropped and the words
|
//! Plain text has no markup to carry emphasis, so it is dropped and the words
|
||||||
//! kept. A link becomes `label (url)`: self-contained, and readable without
|
//! kept. A link becomes `label (url)`: self-contained, and readable without
|
||||||
//! scrolling to a reference list somewhere else. md2txt's `text` renderer emits
|
//! scrolling to a reference list somewhere else.
|
||||||
//! numbered markers instead but never writes the list they point at, so the
|
|
||||||
//! numbers lead nowhere.
|
|
||||||
|
|
||||||
use itsybitsy_core::ir::{Doc, Inline};
|
use itsybitsy_core::ir::{Doc, Inline};
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -122,6 +122,9 @@ impl<'a> Layout<'a> {
|
||||||
prefix,
|
prefix,
|
||||||
Placement { align: Some(*align), inset: margin.unwrap_or(0) as usize },
|
Placement { align: Some(*align), inset: margin.unwrap_or(0) as usize },
|
||||||
),
|
),
|
||||||
|
// Gates are filtered out before rendering; keeping the content is
|
||||||
|
// the harmless reading if one ever arrives here.
|
||||||
|
Block::Gated { block, .. } => self.block(block, prefix, place),
|
||||||
// Nothing paginates here, so a divider has nothing to divide.
|
// Nothing paginates here, so a divider has nothing to divide.
|
||||||
Block::CardBreak { .. } => {}
|
Block::CardBreak { .. } => {}
|
||||||
Block::Html(_) => {}
|
Block::Html(_) => {}
|
||||||
|
|
|
||||||
|
|
@ -1,10 +1,10 @@
|
||||||
//! Fixed-width plain text, for Nex and later Gopher.
|
//! Fixed-width plain text, for Nex and later Gopher.
|
||||||
//!
|
//!
|
||||||
//! One renderer, not two. md2txt ships a `text` and a `nex` renderer that differ
|
//! One renderer, not two: whether a heading gets a FIGlet banner and how a link
|
||||||
//! in exactly two things — whether headings get FIGlet banners, and whether links
|
//! is written are both configuration, so Nex and Gopher are this renderer with
|
||||||
//! are inlined or numbered — and both of those are now configuration. Its
|
//! different settings rather than renderers of their own. A link is inlined as
|
||||||
//! numbered form never writes the reference list its numbers point at, so the
|
//! `label (url)` rather than numbered, because a numbered marker is only useful
|
||||||
//! inline form is the only one that works and is the default here.
|
//! with a reference list to point at.
|
||||||
//!
|
//!
|
||||||
//! Unlike gemtext this wraps, because Nex and Gopher clients do not.
|
//! Unlike gemtext this wraps, because Nex and Gopher clients do not.
|
||||||
|
|
||||||
|
|
@ -58,7 +58,7 @@ mod tests {
|
||||||
/// assertions read as the content itself.
|
/// assertions read as the content itself.
|
||||||
fn render_with(settings: &PageSettings, width: u16, markdown: &str) -> String {
|
fn render_with(settings: &PageSettings, width: u16, markdown: &str) -> String {
|
||||||
let doc = parse::markdown(markdown).unwrap();
|
let doc = parse::markdown(markdown).unwrap();
|
||||||
let ctx = RenderCtx { url: "/x", title: "T", settings, width: Some(width) };
|
let ctx = RenderCtx { url: "/x", title: "T", settings, width: Some(width), mews: None };
|
||||||
String::from_utf8(Text.render(&doc, &ctx).unwrap().body).unwrap()
|
String::from_utf8(Text.render(&doc, &ctx).unwrap().body).unwrap()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -115,7 +115,7 @@ mod tests {
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn one_blank_line_separates_blocks_by_default() {
|
fn one_blank_line_separates_blocks_by_default() {
|
||||||
// md2txt emits two, which reads as double-spaced throughout.
|
// Two would read as double-spaced throughout.
|
||||||
assert_eq!(render("a\n\nb\n"), "a\n\nb\n");
|
assert_eq!(render("a\n\nb\n"), "a\n\nb\n");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -184,7 +184,6 @@ mod tests {
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn tables_are_rendered_as_aligned_columns() {
|
fn tables_are_rendered_as_aligned_columns() {
|
||||||
// md2txt drops tables entirely; this is the flaw that fixes.
|
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
render("| Format | Port |\n| --- | --- |\n| Nex | 1900 |\n"),
|
render("| Format | Port |\n| --- | --- |\n| Nex | 1900 |\n"),
|
||||||
"Format Port\n------ ----\nNex 1900\n"
|
"Format Port\n------ ----\nNex 1900\n"
|
||||||
|
|
@ -214,7 +213,7 @@ mod tests {
|
||||||
/// `parse::document`, which needs a file, so the unit tests build them.
|
/// `parse::document`, which needs a file, so the unit tests build them.
|
||||||
fn render_blocks(settings: &PageSettings, width: u16, blocks: Vec<Block>) -> String {
|
fn render_blocks(settings: &PageSettings, width: u16, blocks: Vec<Block>) -> String {
|
||||||
let doc = Doc { blocks, first_h1: None };
|
let doc = Doc { blocks, first_h1: None };
|
||||||
let ctx = RenderCtx { url: "/x", title: "T", settings, width: Some(width) };
|
let ctx = RenderCtx { url: "/x", title: "T", settings, width: Some(width), mews: None };
|
||||||
String::from_utf8(Text.render(&doc, &ctx).unwrap().body).unwrap()
|
String::from_utf8(Text.render(&doc, &ctx).unwrap().body).unwrap()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -268,7 +267,8 @@ mod separation_tests {
|
||||||
fn render(markdown: &str) -> String {
|
fn render(markdown: &str) -> String {
|
||||||
let doc = parse::markdown(markdown).unwrap();
|
let doc = parse::markdown(markdown).unwrap();
|
||||||
let settings = PageSettings { margin_left: 0, margin_right: 0, ..Default::default() };
|
let settings = PageSettings { margin_left: 0, margin_right: 0, ..Default::default() };
|
||||||
let ctx = RenderCtx { url: "/x", title: "T", settings: &settings, width: Some(40) };
|
let ctx =
|
||||||
|
RenderCtx { url: "/x", title: "T", settings: &settings, width: Some(40), mews: None };
|
||||||
String::from_utf8(Text.render(&doc, &ctx).unwrap().body).unwrap()
|
String::from_utf8(Text.render(&doc, &ctx).unwrap().body).unwrap()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -300,7 +300,7 @@ mod optional_feature_tests {
|
||||||
|
|
||||||
fn render_with(settings: &PageSettings, width: u16, markdown: &str) -> String {
|
fn render_with(settings: &PageSettings, width: u16, markdown: &str) -> String {
|
||||||
let doc = parse::markdown(markdown).unwrap();
|
let doc = parse::markdown(markdown).unwrap();
|
||||||
let ctx = RenderCtx { url: "/x", title: "T", settings, width: Some(width) };
|
let ctx = RenderCtx { url: "/x", title: "T", settings, width: Some(width), mews: None };
|
||||||
String::from_utf8(Text.render(&doc, &ctx).unwrap().body).unwrap()
|
String::from_utf8(Text.render(&doc, &ctx).unwrap().body).unwrap()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,9 +1,9 @@
|
||||||
//! The WAP family: XHTML Mobile Profile, and the `text/html` body derived from it.
|
//! The WAP family: XHTML Mobile Profile, and the `text/html` body derived from it.
|
||||||
//!
|
//!
|
||||||
//! One crate rather than one per format, because the HTML form is the XHTML-MP
|
//! One crate rather than one per format, because the HTML form is the XHTML-MP
|
||||||
//! bytes with the XML declaration removed, and because WML's deck configuration
|
//! bytes, less the XML declaration on a page that is not a Mews page, and because
|
||||||
//! is shared with nothing outside this family. Splitting them would duplicate the
|
//! WML's deck configuration is shared with nothing outside this family. Splitting
|
||||||
//! escaper and the markup builder.
|
//! them would duplicate the escaper and the markup builder.
|
||||||
|
|
||||||
#[cfg(feature = "wml")]
|
#[cfg(feature = "wml")]
|
||||||
mod deck;
|
mod deck;
|
||||||
|
|
@ -29,7 +29,7 @@ impl Renderer for XhtmlMp {
|
||||||
}
|
}
|
||||||
|
|
||||||
fn render(&self, doc: &Doc, ctx: &RenderCtx<'_>) -> Result<Rendered, Error> {
|
fn render(&self, doc: &Doc, ctx: &RenderCtx<'_>) -> Result<Rendered, Error> {
|
||||||
Ok(Rendered::body(xhtmlmp::document(doc, ctx.title, true).into_bytes()))
|
Ok(Rendered::body(xhtmlmp::document(doc, ctx.title, true, ctx.mews).into_bytes()))
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -46,7 +46,7 @@ impl Renderer for Html {
|
||||||
}
|
}
|
||||||
|
|
||||||
fn render(&self, doc: &Doc, ctx: &RenderCtx<'_>) -> Result<Rendered, Error> {
|
fn render(&self, doc: &Doc, ctx: &RenderCtx<'_>) -> Result<Rendered, Error> {
|
||||||
Ok(Rendered::body(xhtmlmp::document(doc, ctx.title, false).into_bytes()))
|
Ok(Rendered::body(xhtmlmp::document(doc, ctx.title, false, ctx.mews).into_bytes()))
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -82,7 +82,13 @@ mod tests {
|
||||||
fn render(renderer: &dyn Renderer, markdown: &str) -> String {
|
fn render(renderer: &dyn Renderer, markdown: &str) -> String {
|
||||||
let doc = parse::markdown(markdown).unwrap();
|
let doc = parse::markdown(markdown).unwrap();
|
||||||
let settings = PageSettings::default();
|
let settings = PageSettings::default();
|
||||||
let ctx = RenderCtx { url: "/x", title: "The Title", settings: &settings, width: None };
|
let ctx = RenderCtx {
|
||||||
|
url: "/x",
|
||||||
|
title: "The Title",
|
||||||
|
settings: &settings,
|
||||||
|
width: None,
|
||||||
|
mews: None,
|
||||||
|
};
|
||||||
String::from_utf8(renderer.render(&doc, &ctx).unwrap().body).unwrap()
|
String::from_utf8(renderer.render(&doc, &ctx).unwrap().body).unwrap()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -178,7 +184,7 @@ mod tests {
|
||||||
first_h1: None,
|
first_h1: None,
|
||||||
};
|
};
|
||||||
let settings = PageSettings::default();
|
let settings = PageSettings::default();
|
||||||
let ctx = RenderCtx { url: "/x", title: "T", settings: &settings, width: None };
|
let ctx = RenderCtx { url: "/x", title: "T", settings: &settings, width: None, mews: None };
|
||||||
let out = String::from_utf8(Html.render(&doc, &ctx).unwrap().body).unwrap();
|
let out = String::from_utf8(Html.render(&doc, &ctx).unwrap().body).unwrap();
|
||||||
assert!(out.contains("<pre title=\"Dragon\">/\\</pre>"), "{out}");
|
assert!(out.contains("<pre title=\"Dragon\">/\\</pre>"), "{out}");
|
||||||
}
|
}
|
||||||
|
|
@ -195,7 +201,7 @@ mod tests {
|
||||||
first_h1: None,
|
first_h1: None,
|
||||||
};
|
};
|
||||||
let settings = PageSettings::default();
|
let settings = PageSettings::default();
|
||||||
let ctx = RenderCtx { url: "/x", title: "T", settings: &settings, width: None };
|
let ctx = RenderCtx { url: "/x", title: "T", settings: &settings, width: None, mews: None };
|
||||||
let out = String::from_utf8(Html.render(&doc, &ctx).unwrap().body).unwrap();
|
let out = String::from_utf8(Html.render(&doc, &ctx).unwrap().body).unwrap();
|
||||||
assert!(out.contains("<body>\n</body>"), "{out}");
|
assert!(out.contains("<body>\n</body>"), "{out}");
|
||||||
}
|
}
|
||||||
|
|
@ -218,7 +224,7 @@ mod deck_tests {
|
||||||
// Card dividers are comment directives, so the IR is built by hand here
|
// Card dividers are comment directives, so the IR is built by hand here
|
||||||
// the way `parse::document` would produce it.
|
// the way `parse::document` would produce it.
|
||||||
let doc = with_card_breaks(markdown);
|
let doc = with_card_breaks(markdown);
|
||||||
let ctx = RenderCtx { url: "/trail", title: "Trail", settings, width: None };
|
let ctx = RenderCtx { url: "/trail", title: "Trail", settings, width: None, mews: None };
|
||||||
Wml.render(&doc, &ctx).unwrap()
|
Wml.render(&doc, &ctx).unwrap()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -474,7 +480,13 @@ mod oracle_dump {
|
||||||
];
|
];
|
||||||
for (name, settings) in cases {
|
for (name, settings) in cases {
|
||||||
let doc = with_card_breaks(TRAIL);
|
let doc = with_card_breaks(TRAIL);
|
||||||
let ctx = RenderCtx { url: "/trail", title: "Trail", settings: &settings, width: None };
|
let ctx = RenderCtx {
|
||||||
|
url: "/trail",
|
||||||
|
title: "Trail",
|
||||||
|
settings: &settings,
|
||||||
|
width: None,
|
||||||
|
mews: None,
|
||||||
|
};
|
||||||
let out = Wml.render(&doc, &ctx).unwrap();
|
let out = Wml.render(&doc, &ctx).unwrap();
|
||||||
std::fs::write(format!("/tmp/mine-{name}.wml"), out.body).unwrap();
|
std::fs::write(format!("/tmp/mine-{name}.wml"), out.body).unwrap();
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -77,6 +77,9 @@ fn block_markup(block: &Block, settings: &PageSettings, out: &mut String) {
|
||||||
Block::Rule => out.push_str("<p>---</p>\n"),
|
Block::Rule => out.push_str("<p>---</p>\n"),
|
||||||
// Alignment is a fixed-width concern; a handset lays out its own screen.
|
// Alignment is a fixed-width concern; a handset lays out its own screen.
|
||||||
Block::Aligned { block, .. } => block_markup(block, settings, out),
|
Block::Aligned { block, .. } => block_markup(block, settings, out),
|
||||||
|
// Gates are filtered out before rendering; keeping the content is the
|
||||||
|
// harmless reading if one ever arrives here.
|
||||||
|
Block::Gated { block, .. } => block_markup(block, settings, out),
|
||||||
// Pagination here is the deck's own job, driven by the byte budget.
|
// Pagination here is the deck's own job, driven by the byte budget.
|
||||||
Block::CardBreak { .. } => {}
|
Block::CardBreak { .. } => {}
|
||||||
// Raw HTML is not WML and would not parse on a handset.
|
// Raw HTML is not WML and would not parse on a handset.
|
||||||
|
|
@ -184,7 +187,6 @@ mod tests {
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn tables_are_rendered_because_wml_has_them() {
|
fn tables_are_rendered_because_wml_has_them() {
|
||||||
// wapdown's own parser produces no tables, so this is new output.
|
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
render("| a | b |\n| --- | --- |\n| 1 | 2 |\n"),
|
render("| a | b |\n| --- | --- |\n| 1 | 2 |\n"),
|
||||||
"<table columns=\"2\">\n<tr><td>a</td><td>b</td></tr>\n<tr><td>1</td><td>2</td></tr>\n</table>\n"
|
"<table columns=\"2\">\n<tr><td>a</td><td>b</td></tr>\n<tr><td>1</td><td>2</td></tr>\n</table>\n"
|
||||||
|
|
|
||||||
|
|
@ -3,47 +3,87 @@
|
||||||
//! One document serves both: a WAP 2.0 client needs well-formed XML with the
|
//! One document serves both: a WAP 2.0 client needs well-formed XML with the
|
||||||
//! prolog and the Mobile Profile doctype, while a browser parsing the same bytes
|
//! prolog and the Mobile Profile doctype, while a browser parsing the same bytes
|
||||||
//! as `text/html` tolerates the leading `<?xml ...?>` as a bogus comment but
|
//! as `text/html` tolerates the leading `<?xml ...?>` as a bogus comment but
|
||||||
//! logs a warning for it. So the HTML form is the same markup with that one line
|
//! logs a warning for it. So an ordinary page's HTML form is the same markup
|
||||||
//! removed.
|
//! with that one line removed. A Mews page keeps it in both forms: see
|
||||||
|
//! [`document`].
|
||||||
|
|
||||||
|
use itsybitsy_core::config::normalize_host;
|
||||||
use itsybitsy_core::ir::{Block, Doc, Inline};
|
use itsybitsy_core::ir::{Block, Doc, Inline};
|
||||||
|
use itsybitsy_core::render::Mews;
|
||||||
|
|
||||||
use crate::escape;
|
use crate::escape;
|
||||||
|
|
||||||
/// The XML declaration, which only the XHTML-MP form carries.
|
/// The XML declaration, which every form of a Mews page carries and only the
|
||||||
|
/// XHTML-MP form of an ordinary one does.
|
||||||
const PROLOG: &str = "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n";
|
const PROLOG: &str = "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n";
|
||||||
|
|
||||||
const DOCTYPE: &str = "<!DOCTYPE html PUBLIC \"-//WAPFORUM//DTD XHTML Mobile 1.0//EN\" \
|
/// The ordinary doctype, used whenever a site has not opted into Mews Profile
|
||||||
\"http://www.wapforum.org/DTD/xhtml-mobile10.dtd\">\n";
|
/// compliance.
|
||||||
|
const DOCTYPE_1_0: &str = "<!DOCTYPE html PUBLIC \"-//WAPFORUM//DTD XHTML Mobile 1.0//EN\" \
|
||||||
|
\"http://www.wapforum.org/DTD/xhtml-mobile10.dtd\">\n";
|
||||||
|
|
||||||
|
/// Mews Profile pages carry the XHTML-MP 1.2 doctype instead (mews.page/spec
|
||||||
|
/// 3.1), which is what a Mews DTD or validator checks a page against.
|
||||||
|
const DOCTYPE_1_2: &str = "<!DOCTYPE html PUBLIC \"-//WAPFORUM//DTD XHTML Mobile 1.2//EN\" \
|
||||||
|
\"http://www.openmobilealliance.org/tech/DTD/xhtml-mobile12.dtd\">\n";
|
||||||
|
|
||||||
|
/// The conformance marker (3.2), viewport tag (3.3) and default stylesheet
|
||||||
|
/// link (5.2), pinned to spec version 0.1: a site opts into the profile as a
|
||||||
|
/// whole, not into picking its own version or stylesheet path.
|
||||||
|
const MEWS_HEAD_EXTRA: &str = "<meta name=\"mews-profile\" content=\"0.1\" />\n\
|
||||||
|
<meta name=\"viewport\" content=\"width=device-width\" />\n\
|
||||||
|
<link rel=\"stylesheet\" type=\"text/css\" href=\"mews-0.1.css\" />\n";
|
||||||
|
|
||||||
/// Build the whole document. `prolog` distinguishes the two media types.
|
/// Build the whole document. `prolog` distinguishes the two media types.
|
||||||
pub fn document(doc: &Doc, title: &str, prolog: bool) -> String {
|
/// `mews` is `Some` when the site has opted into Mews Profile compliance, and
|
||||||
|
/// `None` otherwise.
|
||||||
|
pub fn document(doc: &Doc, title: &str, prolog: bool, mews: Option<Mews<'_>>) -> String {
|
||||||
let mut out = String::new();
|
let mut out = String::new();
|
||||||
if prolog {
|
// A Mews page carries the declaration whatever its media type: 3.1 requires
|
||||||
|
// it, and a validator or directory checker judges the bytes it was handed,
|
||||||
|
// which for anything fetching the way a browser does is the `text/html`
|
||||||
|
// form. The cost is the console warning an HTML parser logs for a comment
|
||||||
|
// it did not expect; it reads the line as a comment before the doctype, so
|
||||||
|
// nothing about how the page renders changes.
|
||||||
|
if prolog || mews.is_some() {
|
||||||
out.push_str(PROLOG);
|
out.push_str(PROLOG);
|
||||||
}
|
}
|
||||||
out.push_str(DOCTYPE);
|
out.push_str(if mews.is_some() { DOCTYPE_1_2 } else { DOCTYPE_1_0 });
|
||||||
out.push_str("<html xmlns=\"http://www.w3.org/1999/xhtml\">\n");
|
out.push_str("<html xmlns=\"http://www.w3.org/1999/xhtml\">\n");
|
||||||
out.push_str(&format!("<head><title>{}</title></head>\n", escape::text(title)));
|
out.push_str(&format!("<head>\n<title>{}</title>\n", escape::text(title)));
|
||||||
|
if mews.is_some() {
|
||||||
|
out.push_str(MEWS_HEAD_EXTRA);
|
||||||
|
}
|
||||||
|
// The feed is declared here rather than linked from a page body, which is
|
||||||
|
// the only place a client reading the head can find it (6.2). The label is
|
||||||
|
// the spec example's, since the attribute names the feed rather than
|
||||||
|
// whichever page happens to carry the declaration.
|
||||||
|
if let Some(feed) = mews.and_then(|mews| mews.feed) {
|
||||||
|
out.push_str(&format!(
|
||||||
|
"<link rel=\"alternate\" type=\"application/atom+xml\" href=\"{}\" title=\"Posts\" />\n",
|
||||||
|
escape::attr(feed)
|
||||||
|
));
|
||||||
|
}
|
||||||
|
out.push_str("</head>\n");
|
||||||
out.push_str("<body>\n");
|
out.push_str("<body>\n");
|
||||||
blocks(&doc.blocks, &mut out);
|
blocks(&doc.blocks, mews, &mut out);
|
||||||
out.push_str("</body>\n</html>\n");
|
out.push_str("</body>\n</html>\n");
|
||||||
out
|
out
|
||||||
}
|
}
|
||||||
|
|
||||||
fn blocks(blocks: &[Block], out: &mut String) {
|
fn blocks(blocks: &[Block], mews: Option<Mews<'_>>, out: &mut String) {
|
||||||
for block in blocks {
|
for block in blocks {
|
||||||
self_block(block, out);
|
self_block(block, mews, out);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
fn self_block(block: &Block, out: &mut String) {
|
fn self_block(block: &Block, mews: Option<Mews<'_>>, out: &mut String) {
|
||||||
match block {
|
match block {
|
||||||
Block::Heading { level, inline } => {
|
Block::Heading { level, inline } => {
|
||||||
let level = (*level).clamp(1, 6);
|
let level = (*level).clamp(1, 6);
|
||||||
out.push_str(&format!("<h{level}>{}</h{level}>\n", inlines(inline)));
|
out.push_str(&format!("<h{level}>{}</h{level}>\n", inlines(inline, mews)));
|
||||||
}
|
}
|
||||||
Block::Paragraph(inline) => out.push_str(&format!("<p>{}</p>\n", inlines(inline))),
|
Block::Paragraph(inline) => out.push_str(&format!("<p>{}</p>\n", inlines(inline, mews))),
|
||||||
Block::CodeBlock { lines, .. } => {
|
Block::CodeBlock { lines, .. } => {
|
||||||
out.push_str("<pre>");
|
out.push_str("<pre>");
|
||||||
out.push_str(&escape::text(&lines.join("\n")));
|
out.push_str(&escape::text(&lines.join("\n")));
|
||||||
|
|
@ -58,7 +98,7 @@ fn self_block(block: &Block, out: &mut String) {
|
||||||
}
|
}
|
||||||
Block::BlockQuote(inner) => {
|
Block::BlockQuote(inner) => {
|
||||||
out.push_str("<blockquote>\n");
|
out.push_str("<blockquote>\n");
|
||||||
blocks(inner, out);
|
blocks(inner, mews, out);
|
||||||
out.push_str("</blockquote>\n");
|
out.push_str("</blockquote>\n");
|
||||||
}
|
}
|
||||||
Block::List { ordered, start, items } => {
|
Block::List { ordered, start, items } => {
|
||||||
|
|
@ -74,10 +114,10 @@ fn self_block(block: &Block, out: &mut String) {
|
||||||
out.push_str("<li>");
|
out.push_str("<li>");
|
||||||
// A single paragraph needs no block wrapper inside the item.
|
// A single paragraph needs no block wrapper inside the item.
|
||||||
match item.as_slice() {
|
match item.as_slice() {
|
||||||
[Block::Paragraph(inline)] => out.push_str(&inlines(inline)),
|
[Block::Paragraph(inline)] => out.push_str(&inlines(inline, mews)),
|
||||||
blocks_in_item => {
|
blocks_in_item => {
|
||||||
out.push('\n');
|
out.push('\n');
|
||||||
blocks(blocks_in_item, out);
|
blocks(blocks_in_item, mews, out);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
out.push_str("</li>\n");
|
out.push_str("</li>\n");
|
||||||
|
|
@ -89,44 +129,53 @@ fn self_block(block: &Block, out: &mut String) {
|
||||||
if !head.is_empty() {
|
if !head.is_empty() {
|
||||||
out.push_str("<tr>");
|
out.push_str("<tr>");
|
||||||
for cell in head {
|
for cell in head {
|
||||||
out.push_str(&format!("<th>{}</th>", inlines(cell)));
|
out.push_str(&format!("<th>{}</th>", inlines(cell, mews)));
|
||||||
}
|
}
|
||||||
out.push_str("</tr>\n");
|
out.push_str("</tr>\n");
|
||||||
}
|
}
|
||||||
for row in rows {
|
for row in rows {
|
||||||
out.push_str("<tr>");
|
out.push_str("<tr>");
|
||||||
for cell in row {
|
for cell in row {
|
||||||
out.push_str(&format!("<td>{}</td>", inlines(cell)));
|
out.push_str(&format!("<td>{}</td>", inlines(cell, mews)));
|
||||||
}
|
}
|
||||||
out.push_str("</tr>\n");
|
out.push_str("</tr>\n");
|
||||||
}
|
}
|
||||||
out.push_str("</table>\n");
|
out.push_str("</table>\n");
|
||||||
}
|
}
|
||||||
Block::Rule => out.push_str("<hr/>\n"),
|
Block::Rule => out.push_str("<hr/>\n"),
|
||||||
// Raw HTML is passed through: this is the one family of formats where it
|
// Raw HTML is ordinarily passed through, since this is the one family
|
||||||
// is already in the right language.
|
// of formats where it is already in the right language. A Mews page
|
||||||
|
// cannot make that guarantee — the markup might use an element or
|
||||||
|
// attribute the profile excludes — so it is dropped instead.
|
||||||
Block::Html(html) => {
|
Block::Html(html) => {
|
||||||
out.push_str(html.trim_end());
|
if mews.is_none() {
|
||||||
out.push('\n');
|
out.push_str(html.trim_end());
|
||||||
|
out.push('\n');
|
||||||
|
}
|
||||||
}
|
}
|
||||||
// Alignment reaches only the fixed-width text formats. Pagination is a
|
// Alignment reaches only the fixed-width text formats. Pagination is a
|
||||||
// WML concern; a browser scrolls one document.
|
// WML concern; a browser scrolls one document.
|
||||||
Block::Aligned { block, .. } => self_block(block, out),
|
Block::Aligned { block, .. } => self_block(block, mews, out),
|
||||||
|
// Gates are filtered out before rendering; keeping the content is the
|
||||||
|
// harmless reading if one ever arrives here.
|
||||||
|
Block::Gated { block, .. } => self_block(block, mews, out),
|
||||||
Block::CardBreak { .. } => {}
|
Block::CardBreak { .. } => {}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
fn inlines(inline: &[Inline]) -> String {
|
fn inlines(inline: &[Inline], mews: Option<Mews<'_>>) -> String {
|
||||||
let mut out = String::new();
|
let mut out = String::new();
|
||||||
for item in inline {
|
for item in inline {
|
||||||
match item {
|
match item {
|
||||||
Inline::Text(text) => out.push_str(&escape::text(text)),
|
Inline::Text(text) => out.push_str(&escape::text(text)),
|
||||||
Inline::Code(code) => out.push_str(&format!("<code>{}</code>", escape::text(code))),
|
Inline::Code(code) => out.push_str(&format!("<code>{}</code>", escape::text(code))),
|
||||||
Inline::Emph(inner) => out.push_str(&format!("<em>{}</em>", inlines(inner))),
|
Inline::Emph(inner) => out.push_str(&format!("<em>{}</em>", inlines(inner, mews))),
|
||||||
Inline::Strong(inner) => out.push_str(&format!("<strong>{}</strong>", inlines(inner))),
|
Inline::Strong(inner) => {
|
||||||
|
out.push_str(&format!("<strong>{}</strong>", inlines(inner, mews)))
|
||||||
|
}
|
||||||
// XHTML-MP 1.0 has no <del>, and <strike> is not in the profile, so
|
// XHTML-MP 1.0 has no <del>, and <strike> is not in the profile, so
|
||||||
// the text survives without its markup rather than being dropped.
|
// the text survives without its markup rather than being dropped.
|
||||||
Inline::Strike(inner) => out.push_str(&inlines(inner)),
|
Inline::Strike(inner) => out.push_str(&inlines(inner, mews)),
|
||||||
Inline::Link { href, title, label } => {
|
Inline::Link { href, title, label } => {
|
||||||
let title = title
|
let title = title
|
||||||
.as_deref()
|
.as_deref()
|
||||||
|
|
@ -135,10 +184,17 @@ fn inlines(inline: &[Inline]) -> String {
|
||||||
out.push_str(&format!(
|
out.push_str(&format!(
|
||||||
"<a href=\"{}\"{title}>{}</a>",
|
"<a href=\"{}\"{title}>{}</a>",
|
||||||
escape::attr(href),
|
escape::attr(href),
|
||||||
inlines(label)
|
inlines(label, mews)
|
||||||
));
|
));
|
||||||
}
|
}
|
||||||
Inline::Image { src, title, alt } => {
|
Inline::Image { src, title, alt } => {
|
||||||
|
// A Mews page's image must point at the same site and never be
|
||||||
|
// a `data:` URI (SPEC.md 4.2); one that does not is dropped to
|
||||||
|
// its alt text rather than rendered non-conformant.
|
||||||
|
if mews.is_some_and(|mews| !same_site_image(src, mews.hosts)) {
|
||||||
|
out.push_str(&escape::text(&Doc::plain_text(alt)));
|
||||||
|
continue;
|
||||||
|
}
|
||||||
let title = title
|
let title = title
|
||||||
.as_deref()
|
.as_deref()
|
||||||
.map(|t| format!(" title=\"{}\"", escape::attr(t)))
|
.map(|t| format!(" title=\"{}\"", escape::attr(t)))
|
||||||
|
|
@ -151,20 +207,161 @@ fn inlines(inline: &[Inline]) -> String {
|
||||||
}
|
}
|
||||||
Inline::SoftBreak => out.push('\n'),
|
Inline::SoftBreak => out.push('\n'),
|
||||||
Inline::HardBreak => out.push_str("<br/>\n"),
|
Inline::HardBreak => out.push_str("<br/>\n"),
|
||||||
Inline::Html(html) => out.push_str(html),
|
// See the Block::Html arm above: unverifiable in a Mews page.
|
||||||
|
Inline::Html(html) => {
|
||||||
|
if mews.is_none() {
|
||||||
|
out.push_str(html);
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
out
|
out
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Whether an image `src` satisfies the Mews same-site rule (SPEC.md 4.2). A
|
||||||
|
/// `data:` URI never qualifies. A relative or root-relative URL always does,
|
||||||
|
/// since it resolves against the page's own origin by definition. An absolute
|
||||||
|
/// or protocol-relative URL qualifies when its host equals one of the site's
|
||||||
|
/// own hostnames or is a subdomain of one — a looser stand-in for "the same
|
||||||
|
/// registered domain" that needs no public-suffix list, adequate for hosts
|
||||||
|
/// the site itself configured.
|
||||||
|
fn same_site_image(src: &str, hosts: &[String]) -> bool {
|
||||||
|
if src.len() >= 5 && src.as_bytes()[..5].eq_ignore_ascii_case(b"data:") {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
let Some(raw_host) = url_host(src) else { return true };
|
||||||
|
let Some(host) = normalize_host(raw_host) else { return false };
|
||||||
|
hosts.iter().any(|allowed| host == *allowed || host.ends_with(&format!(".{allowed}")))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The host component of an absolute (`scheme://host/...`) or
|
||||||
|
/// protocol-relative (`//host/...`) URL, or `None` for anything else: a bare
|
||||||
|
/// or root-relative path has no host of its own, so it always resolves
|
||||||
|
/// against the page's own site.
|
||||||
|
fn url_host(src: &str) -> Option<&str> {
|
||||||
|
let rest = match src.split_once("://") {
|
||||||
|
Some((_scheme, rest)) => rest,
|
||||||
|
None => src.strip_prefix("//")?,
|
||||||
|
};
|
||||||
|
Some(rest.split(['/', '?', '#']).next().unwrap_or(rest))
|
||||||
|
}
|
||||||
|
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
|
|
||||||
|
fn empty() -> Doc {
|
||||||
|
Doc { blocks: Vec::new(), first_h1: None }
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A site that has opted into the profile and declares no feed.
|
||||||
|
fn mews(hosts: &[String]) -> Mews<'_> {
|
||||||
|
Mews { hosts, feed: None }
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn the_prolog_is_present_only_when_asked_for() {
|
fn an_ordinary_page_carries_the_prolog_only_when_asked_for() {
|
||||||
let doc = Doc { blocks: Vec::new(), first_h1: None };
|
let doc = empty();
|
||||||
assert!(document(&doc, "T", true).starts_with("<?xml"));
|
assert!(document(&doc, "T", true, None).starts_with("<?xml"));
|
||||||
assert!(document(&doc, "T", false).starts_with("<!DOCTYPE html"));
|
assert!(document(&doc, "T", false, None).starts_with("<!DOCTYPE html"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_mews_page_carries_the_prolog_in_both_forms() {
|
||||||
|
// The text/html form is what a validator fetching like a browser is
|
||||||
|
// handed, and 3.1 is a MUST on the bytes it reads.
|
||||||
|
let hosts = vec!["example.test".to_string()];
|
||||||
|
for xml in [true, false] {
|
||||||
|
let out = document(&empty(), "T", xml, Some(mews(&hosts)));
|
||||||
|
assert!(out.starts_with("<?xml version=\"1.0\" encoding=\"utf-8\"?>"), "{out}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_feed_is_declared_in_the_head_when_the_site_names_one() {
|
||||||
|
let hosts = vec!["example.test".to_string()];
|
||||||
|
let feed = Mews { hosts: &hosts, feed: Some("https://example.test/feed.xml") };
|
||||||
|
let out = document(&empty(), "T", false, Some(feed));
|
||||||
|
assert!(
|
||||||
|
out.contains(
|
||||||
|
"<link rel=\"alternate\" type=\"application/atom+xml\" \
|
||||||
|
href=\"https://example.test/feed.xml\" title=\"Posts\" />"
|
||||||
|
),
|
||||||
|
"{out}"
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
out.split("</head>").next().is_some_and(|head| head.contains("alternate")),
|
||||||
|
"{out}"
|
||||||
|
);
|
||||||
|
|
||||||
|
let none = document(&empty(), "T", false, Some(mews(&hosts)));
|
||||||
|
assert!(!none.contains("alternate"), "{none}");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_plain_page_keeps_the_1_0_doctype_and_no_mews_head() {
|
||||||
|
let out = document(&empty(), "T", false, None);
|
||||||
|
assert!(out.contains("XHTML Mobile 1.0"), "{out}");
|
||||||
|
assert!(!out.contains("mews-profile"), "{out}");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_mews_page_carries_the_1_2_doctype_and_the_conformance_marker() {
|
||||||
|
let hosts = vec!["example.test".to_string()];
|
||||||
|
let out = document(&empty(), "T", false, Some(mews(&hosts)));
|
||||||
|
assert!(out.contains("XHTML Mobile 1.2"), "{out}");
|
||||||
|
assert!(out.contains("<meta name=\"mews-profile\" content=\"0.1\" />"), "{out}");
|
||||||
|
assert!(out.contains("<meta name=\"viewport\" content=\"width=device-width\" />"), "{out}");
|
||||||
|
assert!(
|
||||||
|
out.contains("<link rel=\"stylesheet\" type=\"text/css\" href=\"mews-0.1.css\" />"),
|
||||||
|
"{out}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_mews_page_drops_raw_html_it_cannot_vouch_for() {
|
||||||
|
let hosts = vec!["example.test".to_string()];
|
||||||
|
let doc = Doc { blocks: vec![Block::Html("<div>x</div>".into())], first_h1: None };
|
||||||
|
let out = document(&doc, "T", false, Some(mews(&hosts)));
|
||||||
|
assert!(!out.contains("<div>"), "{out}");
|
||||||
|
|
||||||
|
let plain = document(&doc, "T", false, None);
|
||||||
|
assert!(plain.contains("<div>x</div>"), "{plain}");
|
||||||
|
}
|
||||||
|
|
||||||
|
fn image_doc(src: &str) -> Doc {
|
||||||
|
Doc {
|
||||||
|
blocks: vec![Block::Paragraph(vec![Inline::Image {
|
||||||
|
src: src.into(),
|
||||||
|
title: None,
|
||||||
|
alt: vec![Inline::Text("a photo".into())],
|
||||||
|
}])],
|
||||||
|
first_h1: None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_mews_page_keeps_a_relative_or_same_site_image() {
|
||||||
|
let hosts = vec!["example.test".to_string()];
|
||||||
|
for src in ["/i.png", "i.png", "https://example.test/i.png", "//img.example.test/i.png"] {
|
||||||
|
let out = document(&image_doc(src), "T", false, Some(mews(&hosts)));
|
||||||
|
assert!(out.contains(&format!("src=\"{src}\"")), "{src}: {out}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_mews_page_drops_an_off_site_or_data_image_to_its_alt_text() {
|
||||||
|
let hosts = vec!["example.test".to_string()];
|
||||||
|
for src in ["https://elsewhere.test/i.png", "data:image/png;base64,AAAA"] {
|
||||||
|
let out = document(&image_doc(src), "T", false, Some(mews(&hosts)));
|
||||||
|
assert!(!out.contains("<img"), "{src}: {out}");
|
||||||
|
assert!(out.contains("a photo"), "{src}: {out}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_plain_page_keeps_every_image_regardless_of_its_host() {
|
||||||
|
let out = document(&image_doc("https://elsewhere.test/i.png"), "T", false, None);
|
||||||
|
assert!(out.contains("<img src=\"https://elsewhere.test/i.png\""), "{out}");
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue