diff --git a/Cargo.lock b/Cargo.lock index 23a476b..451954e 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -332,6 +332,7 @@ name = "itsybitsy-wap" version = "0.1.0" dependencies = [ "itsybitsy-core", + "log", ] [[package]] diff --git a/README.md b/README.md index 544d341..9c11f53 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ A Rust re-implementation of [smolweb](https://code.randogoth.com/randogoth/smolw ## Status -It serves. HTTP, Spartan and Nex are working, with gemtext, XHTML-MP and HTML output and virtual hosting across all three. +It serves. HTTP, Spartan and Nex are working across five output formats, with virtual hosting on all three. Gemini and Gopher are the remaining protocols. | Area | State | | --- | --- | @@ -19,7 +19,7 @@ It serves. HTTP, Spartan and Nex are working, with gemtext, XHTML-MP and HTML ou | HTTP, Spartan and Nex listeners, virtual hosting | implemented | | Fixed-width text output (Nex, later Gopher) | implemented | | FIGlet banners and hyphenation, behind features | implemented | -| WML and XHTML-MP decks | planned | +| WML decks and card sub-documents, behind a feature | implemented | | Gemini and Gopher | planned | ## Configuration @@ -105,6 +105,8 @@ Include and art targets resolve relative to the file holding the reference and m Configuration keys are not carried over verbatim either: md2txt accepts three spellings of `paragraph_spacing`, and its `cache_control` is named for the header it lands in rather than the value it holds, which here is `max_age`. Its `text` and `nex` renderers differ only in heading decoration and in whether links are inlined or numbered — and the numbered form never writes the reference list its numbers point at, so there is one text renderer here rather than two. Tables, which both of its text renderers drop entirely, are rendered as aligned columns. +wapdown's own parser produces neither tables nor lists, so its WML decks render a bullet list as literal text and drop tables entirely; both come out as real markup here, lists as marked lines and tables as ``, which WML has. + smolweb inherits five invented directives from its two renderer libraries: `{.card}`, `{.include}`, `![[ ]]`, `#[label](art.txt)` and MultiMarkdown attribute lists (`{: .center}`) — two incompatible brace grammars, with art alignment expressible two different ways in one line. None of it appeared in real content, so itsybitsy expresses the same capabilities with standard constructs instead. Parsing once rather than once per library also means the text formats gain setext headings and the WAP formats gain tables and ASCII art, none of which their original parser handled. ## Output formats @@ -117,6 +119,7 @@ Each format is a crate implementing one trait, compiled in behind a cargo featur | `xhtmlmp` | `application/vnd.wap.xhtml+xml` | Well-formed XML with the Mobile Profile doctype | | `html` | `text/html` | The same markup without the XML declaration | | `text` | `text/plain` | Wrapped to 80 columns, because Nex and Gopher clients do not wrap | +| `wml` | `text/vnd.wap.wml` | WAP 1.x decks, paginated to a per-card byte budget (feature `wml`) | HTTP negotiates between them from `Accept`. Only a literal media type counts as a match, so a browser's `*/*` can never be read as willingness to receive a WAP format; `?format=` overrides negotiation outright, and naming an id the listener does not serve is a bad request rather than a silent fallback. @@ -128,6 +131,7 @@ Two decorative capabilities for the text format are off by default, because most | Feature | Adds | Cost | | --- | --- | --- | +| `wml` | WML 1.3 decks for WAP 1.x handsets | +37 KiB | | `figlet` | `h1_style = "figlet"` banners, fonts `small` and `standard` | +60 KiB | | `hyphenation` | `hyphenate = true`, English patterns | +127 KiB | | `hyphenation-all` | every language the pattern crate carries | +3.0 MiB | @@ -138,6 +142,23 @@ cargo build --release --features "figlet hyphenation" A banner that will not fit the line, names a font this build lacks, or is asked for by a build without `figlet` falls back to the level's underline — a heading that cannot be decorated should not be lost. An unknown font name is logged, since that is almost always a typo. Likewise `hyphenate` has no effect without the feature, and an unknown `hyphen_lang` leaves the text unhyphenated; a ragged right edge is the plain-text convention anyway. +## Card sub-documents + +WML is the one output that paginates: a WAP 1.x handset has a hard per-card byte budget and refuses a deck that exceeds it. So a long page becomes a chain of screens, and a page with dividers becomes a menu card linking to the rest. + +With `deck_per_card`, each card also gets its own URL under the page's: + +| Request | WML client | Any other format | +| --- | --- | --- | +| `/trail` | the menu deck | the whole page | +| `/trail/weather` | that card's own deck | redirects to `/trail` | +| `/trail/nonexistent` | redirects to `/trail` | redirects to `/trail` | +| `/about/whatever` | not found | not found | + +That URL space belongs to the format that claims it. A format which does not address a sub-document redirects to the parent rather than substituting something else, and a page with no dividers has no such URLs at all — `/about/whatever` is not a sub-document just because `/about` exists. Nex and Gopher have no redirect status, so they resolve to the parent's content directly instead of bouncing. + +The deck keys are `split_level`, `split_on_rule`, `max_card_bytes`, `menu`, `menu_style`, `deck_per_card`, `template_nav`, `nav_next_label`, `nav_prev_label`, `nav_back_label`, `home_label` and `images`. + ## URLs Links should be root-relative and extensionless (`[about](/about)`, not `about.md`), so the same link resolves identically from every protocol. diff --git a/bin/Cargo.toml b/bin/Cargo.toml index e9224bb..82e4052 100644 --- a/bin/Cargo.toml +++ b/bin/Cargo.toml @@ -18,6 +18,8 @@ spartan = [] nex = [] gemtext = ["dep:itsybitsy-gemtext"] wap = ["dep:itsybitsy-wap"] +# WML 1.3 decks for WAP 1.x handsets. +wml = ["wap", "itsybitsy-wap/wml"] text = ["dep:itsybitsy-text"] # FIGlet banners and hyphenation for the text format, both off by default. figlet = ["text", "itsybitsy-text/figlet"] diff --git a/bin/src/proto/http.rs b/bin/src/proto/http.rs index 63c61f4..ac0814c 100644 --- a/bin/src/proto/http.rs +++ b/bin/src/proto/http.rs @@ -139,6 +139,30 @@ pub fn serve(listener: &Listener, mut stream: TcpStream) -> Result<()> { } respond(&mut stream, 200, "OK", listener.media_type(&format), body, &headers, head_only) } + Ok(Resolution::Found(Resource::Part { parent_url, slug, page })) => { + match page.part(&format, &slug) { + Some(body) => respond( + &mut stream, + 200, + "OK", + listener.media_type(&format), + body, + &[("Vary", "Accept")], + head_only, + ), + // This format does not address that sub-document, so the parent + // is where the content actually is. + None => respond( + &mut stream, + 301, + "Moved Permanently", + "text/plain; charset=utf-8", + b"", + &[("Location", parent_url.as_str()), ("Vary", "Accept")], + head_only, + ), + } + } Ok(Resolution::Found(Resource::Raw { path, media_type })) => { let meta = std::fs::metadata(&path)?; write_head(&mut stream, 200, "OK", media_type, meta.len(), &[])?; diff --git a/bin/src/proto/nex.rs b/bin/src/proto/nex.rs index 1583c0a..bceb2e4 100644 --- a/bin/src/proto/nex.rs +++ b/bin/src/proto/nex.rs @@ -45,7 +45,8 @@ pub fn serve(listener: &Listener, mut stream: TcpStream) -> Result<()> { let mut file = std::fs::File::open(&path)?; std::io::copy(&mut file, &mut stream)?; } - // resolve_flat never yields a redirect; it is matched for completeness. + // resolve_flat follows redirects and sub-documents to real content, so + // neither reaches here; they are matched for completeness. Ok(_) => stream.write_all(b"Not found\n")?, Err(err) => { log::warn!("{} nex {}: {err}", listener.name, for_log(path)); diff --git a/bin/src/proto/spartan.rs b/bin/src/proto/spartan.rs index 642ed7c..cb1a3ec 100644 --- a/bin/src/proto/spartan.rs +++ b/bin/src/proto/spartan.rs @@ -58,6 +58,15 @@ pub fn serve(listener: &Listener, mut stream: TcpStream) -> Result<()> { } None => status(&mut stream, 5, "Not rendered", b""), }, + Ok(Resolution::Found(Resource::Part { parent_url, slug, page })) => { + match page.part(format, &slug) { + Some(body) => { + let media_type = listener.media_type(format); + status(&mut stream, 2, media_type, body) + } + None => status(&mut stream, 3, &parent_url, b""), + } + } Ok(Resolution::Found(Resource::Raw { path, media_type })) => { write!(stream, "2 {media_type}\r\n")?; let mut file = std::fs::File::open(&path)?; diff --git a/bin/src/registry.rs b/bin/src/registry.rs index 49f5955..3e511a3 100644 --- a/bin/src/registry.rs +++ b/bin/src/registry.rs @@ -21,6 +21,8 @@ pub fn build() -> Registry { { registry.insert(Arc::new(itsybitsy_wap::XhtmlMp)).expect("duplicate renderer id"); registry.insert(Arc::new(itsybitsy_wap::Html)).expect("duplicate renderer id"); + #[cfg(feature = "wml")] + registry.insert(Arc::new(itsybitsy_wap::Wml)).expect("duplicate renderer id"); } registry diff --git a/bin/tests/listeners.rs b/bin/tests/listeners.rs index 3c5e7b3..6ae0e69 100644 --- a/bin/tests/listeners.rs +++ b/bin/tests/listeners.rs @@ -28,6 +28,12 @@ impl Server { write(dir.path(), "one/img.png", "not really a png"); write(dir.path(), "one/.itsybitsy.toml", "[defaults]\nmax_age = 120\n"); write(dir.path(), "two/index.md", "# Two\n\nSecond site.\n"); + write( + dir.path(), + "one/trail.md", + "Intro.\n\n\n\nCold and clear. Permit fee: $5.\n\n\ + \n\n- North: open\n", + ); let text = format!( r#" @@ -401,3 +407,107 @@ fn nothing_outside_the_root_is_reachable_over_any_protocol() { assert_eq!(server.send("spartan", b"one.test /.itsybitsy.toml 0\r\n"), "4 Not found\r\n"); assert_eq!(server.send("nex", b"/.itsybitsy.toml\r\n"), "Not found\n"); } + +// -- Sub-documents ------------------------------------------------------- + +/// A page that paginates addresses sub-documents under its own URL. Only a +/// format that paginates can serve them, so every other format redirects to the +/// parent rather than substituting something else. +#[cfg(feature = "wml")] +mod parts { + use super::*; + + /// Like `Server::start`, but with a listener that can serve WML. + fn wml_server() -> Server { + let dir = tempfile::tempdir().unwrap(); + let root = dir.path().join("one"); + fs::create_dir_all(&root).unwrap(); + fs::write(root.join("index.md"), "# One\n").unwrap(); + fs::write( + root.join("trail.md"), + "Intro.\n\n\n\nCold and clear. Permit fee: $5.\n\n\ + \n\n- North: open\n", + ) + .unwrap(); + fs::write(root.join("about.md"), "# About\n\nNo dividers here.\n").unwrap(); + fs::write(root.join(".itsybitsy.toml"), "[defaults]\ndeck_per_card = true\n").unwrap(); + + let text = format!( + r#" +version = 1 + +[site.one] +root = "{root}" +hosts = ["one.test"] + +[listener.web] +protocol = "http" +bind = "127.0.0.1:0" +formats = ["wml", "html"] +default_site = "one" +"#, + root = root.to_str().unwrap(), + ); + let config: ServerConfig = toml::from_str(&text).unwrap(); + let registry = Arc::new(itsybitsy::registry::build()); + let formats: Vec = ["wml", "html"].iter().map(|s| s.to_string()).collect(); + let sites = Arc::new(SiteSet::build(&config, registry.clone(), formats).unwrap()); + + let bound = serve::bind(&config, registry, sites).unwrap(); + let addrs = bound + .iter() + .map(|entry| (entry.listener.name.clone(), entry.local_addr().unwrap())) + .collect(); + serve::spawn_all(bound); + Server { addrs, _dir: dir } + } + + #[test] + fn a_wml_client_gets_the_card_itself() { + let server = wml_server(); + let response = + server.request("web", "one.test", "/trail/weather", "Accept: text/vnd.wap.wml\r\n"); + let (status, body) = split(&response); + assert_eq!(status, "HTTP/1.1 200 OK"); + assert!(response.contains("Content-Type: text/vnd.wap.wml"), "{response}"); + assert!(body.contains(""), "{body}"); + // WML reads a single dollar as a variable reference. + assert!(body.contains("Permit fee: $$5."), "{body}"); + } + + #[test] + fn a_browser_is_redirected_to_the_parent_page() { + // That URL space belongs to WML; HTML does not address it. + let server = wml_server(); + let response = server.get("one.test", "/trail/weather"); + assert_eq!(split(&response).0, "HTTP/1.1 301 Moved Permanently"); + assert!(response.contains("Location: /trail"), "{response}"); + assert!(response.contains("Vary: Accept"), "the redirect depends on Accept"); + } + + #[test] + fn an_unknown_card_redirects_even_for_a_wml_client() { + let server = wml_server(); + let response = + server.request("web", "one.test", "/trail/nonexistent", "Accept: text/vnd.wap.wml\r\n"); + assert_eq!(split(&response).0, "HTTP/1.1 301 Moved Permanently"); + assert!(response.contains("Location: /trail"), "{response}"); + } + + #[test] + fn a_page_that_does_not_paginate_has_no_such_urls() { + let server = wml_server(); + let response = + server.request("web", "one.test", "/about/whatever", "Accept: text/vnd.wap.wml\r\n"); + assert_eq!(split(&response).0, "HTTP/1.1 404 Not Found"); + } + + #[test] + fn the_index_deck_links_out_to_each_card() { + let server = wml_server(); + let response = server.request("web", "one.test", "/trail", "Accept: text/vnd.wap.wml\r\n"); + let (_, body) = split(&response); + assert!(body.contains("Weather"), "{body}"); + assert!(body.contains("Status"), "{body}"); + } +} diff --git a/content/.itsybitsy.toml b/content/.itsybitsy.toml index 6992982..9f0f6af 100644 --- a/content/.itsybitsy.toml +++ b/content/.itsybitsy.toml @@ -3,3 +3,4 @@ max_age = 60 [page."about.md"] title = "About itsybitsy" +deck_per_card = true diff --git a/core/src/config.rs b/core/src/config.rs index 41fe91a..bc9bfcb 100644 --- a/core/src/config.rs +++ b/core/src/config.rs @@ -357,6 +357,63 @@ fn apply(keys: &PageKeys, settings: &mut PageSettings) -> Result<(), Error> { if let Some(value) = keys.wrap_code_blocks { settings.wrap_code_blocks = value; } + if let Some(value) = keys.split_level { + if value > 6 { + return Err(Error::config(format!("split_level = {value} is above heading level 6"))); + } + settings.split_level = value; + } + if let Some(value) = keys.split_on_rule { + settings.split_on_rule = value; + } + if let Some(value) = keys.max_card_bytes { + settings.max_card_bytes = value; + } + if let Some(value) = keys.menu { + settings.menu = value; + } + if let Some(value) = &keys.menu_style { + settings.menu_style = match value.as_str() { + "links" => MenuStyle::Links, + "select" => MenuStyle::Select, + other => { + return Err(Error::config(format!( + "menu_style = '{other}' is not one of links or select" + ))); + } + }; + } + if let Some(value) = keys.deck_per_card { + settings.deck_per_card = value; + } + if let Some(value) = keys.template_nav { + settings.template_nav = value; + } + for (configured, target) in [ + (&keys.nav_next_label, &mut settings.nav_next_label), + (&keys.nav_prev_label, &mut settings.nav_prev_label), + (&keys.nav_back_label, &mut settings.nav_back_label), + ] { + if let Some(value) = configured { + *target = value.clone(); + } + } + if let Some(value) = &keys.home_label { + // An empty label means no softkey, rather than one with no text on it. + settings.home_label = (!value.is_empty()).then(|| value.clone()); + } + if let Some(value) = &keys.images { + settings.images = match value.as_str() { + "keep" => Images::Keep, + "alt" => Images::Alt, + "drop" => Images::Drop, + other => { + return Err(Error::config(format!( + "images = '{other}' is not one of keep, alt or drop" + ))); + } + }; + } let levels = [ &keys.h1_style, @@ -480,6 +537,49 @@ pub struct PageKeys { /// Wrap over-long code lines. Off by default, because wrapping code changes /// what it says; on, it is better than losing the end of the line. pub wrap_code_blocks: Option, + + // WML decks. Only the WML format reads these; it is the one output that + // paginates, because a WAP 1.x handset has a per-card byte budget. + /// Heading level that starts a new card, or 0 to disable. Ignored when the + /// document carries explicit card dividers. + pub split_level: Option, + /// Treat a thematic break as a card divider. + pub split_on_rule: Option, + /// Bytes one card may occupy, navigation included, or 0 for no limit. + pub max_card_bytes: Option, + /// Hub-and-spoke with a menu card, rather than cards chained in sequence. + pub menu: Option, + /// `links` for one link per line, `select` for a keypad-pickable list. + pub menu_style: Option, + /// Give each card its own deck, addressable under the page's own URL. + pub deck_per_card: Option, + /// Hoist navigation shared by every card into a `