Move the WML renderer out to wapdown
Every other renderer here -- text, micron, ama, gemini, nex -- is a
straight transform of a document into a flat, minimal text format. The
WML renderer was neither: it emitted an XML tree, and it restructured the
document rather than transforming it, splitting one file into many
<card>s and inventing navigation between them. That is why it was the
only renderer needing split_level / max_card_bytes / menu levers, the
only one buffering the whole event stream instead of streaming it, and
the only one that required a block kind in the shared parser --
CARD_BREAK, for its {.card} divider -- that every other renderer ignored
as a no-op.
It now lives in its own repository, where restructuring a document into a
navigable deck is the point rather than an intrusion:
https://codeberg.org/randogoth/wapdown
Removed here: renderers/wml.py, its CLI registration, the wml examples,
the README section, and the CARD_BREAK event and {.card} directive that
existed solely to serve it. A leftover {.card} line in an old document
will now render as literal text rather than being silently dropped.
Verified: all five remaining renderers produce byte-identical output to
before this change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
randogoth
parent
d5046543b3
commit
e708dce8f7
8 changed files with 12 additions and 544 deletions
44
README.md
44
README.md
|
|
@ -19,7 +19,7 @@ uv tool install --from git+https://codeberg.org/randogoth/md2txt/ md2txt
|
||||||
|
|
||||||
## CLI
|
## CLI
|
||||||
|
|
||||||
- `md2txt` – converts Markdown into plain text with extensive formatting support. It ships with the default `markdown` parser and `text` renderer plugins, registers optional `micron`, `ama`, `gemini`, and `wml` renderers for Micron/Ancient Machine Book, Gemtext, or WML (WAP) output, and exposes the core pipeline so you can add your own parser or renderer modules:
|
- `md2txt` – converts Markdown into plain text with extensive formatting support. It ships with the default `markdown` parser and `text` renderer plugins, registers optional `micron`, `ama`, `gemini`, and `nex` renderers for Micron/Ancient Machine Book, Gemtext, or Nex output, and exposes the core pipeline so you can add your own parser or renderer modules:
|
||||||
- FIGlet-rendered headings.
|
- FIGlet-rendered headings.
|
||||||
- Emphasis styles converted to spaced or delimited characters.
|
- Emphasis styles converted to spaced or delimited characters.
|
||||||
- Optional alignment and margin controls via HTML `<p>` attributes or MultiMarkdown attribute blocks (e.g. `{:.center margin=20px}`) applied as leading spaces.
|
- Optional alignment and margin controls via HTML `<p>` attributes or MultiMarkdown attribute blocks (e.g. `{:.center margin=20px}`) applied as leading spaces.
|
||||||
|
|
@ -36,7 +36,7 @@ md2txt input.md --width 72 # override column width
|
||||||
md2txt input.md --renderer micron # emit Micron-formatted output for NomadNet
|
md2txt input.md --renderer micron # emit Micron-formatted output for NomadNet
|
||||||
md2txt input.md --renderer ama # emit AMA markup for Ancient Machine Book
|
md2txt input.md --renderer ama # emit AMA markup for Ancient Machine Book
|
||||||
md2txt input.md --renderer gemini # emit Gemtext for Gemini capsules
|
md2txt input.md --renderer gemini # emit Gemtext for Gemini capsules
|
||||||
md2txt input.md --renderer wml # emit a single-card WML deck for WAP browsers
|
md2txt input.md --renderer nex # emit plain text for Nex capsules
|
||||||
md2txt input.md --renderer-option width=68 # pass column width to a renderer
|
md2txt input.md --renderer-option width=68 # pass column width to a renderer
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
@ -123,39 +123,13 @@ Inline or block-level ASCII art can be embedded directly in Markdown:
|
||||||
|
|
||||||
Generated text uses CRLF line endings to maintain DOS compatibility.
|
Generated text uses CRLF line endings to maintain DOS compatibility.
|
||||||
|
|
||||||
## WML Decks
|
## Nex
|
||||||
|
|
||||||
The `wml` renderer converts Markdown into a [WML](https://en.wikipedia.org/wiki/Wireless_Markup_Language) deck for old WAP browsers. WML screens are idiomatically short and button/link-navigated rather than one long scrolling page, so unlike the other renderers, `wml` can split a document into multiple `<card>`s — but only when you ask it to; a plain document with no options stays a single card.
|
The `nex` renderer converts Markdown into plain text for the [Nex](https://nightfall.city/nex/) protocol. Nex has no hypertext markup for hand-authored documents at all — its one special syntax, `=> URL`, is spec'd for server-auto-generated directory listings, not document bodies — so `nex` output is deliberately unmarked plain text, reusing `TextRenderer`'s wrapping/margin/hyphenation machinery (same approach as the `ama`/`micron` renderers):
|
||||||
|
|
||||||
Two independent, both-optional ways to define card boundaries:
|
- Headings get a plain setext-style underline (`=` for H1, `-` for H2+) instead of a FIGlet banner. This is a readability convention borrowed from Markdown itself, not a requirement of the Nex spec.
|
||||||
|
- `**bold**`/`*italic*`/`~~strikethrough~~` markers are stripped rather than translated — there's nothing in Nex to translate them into.
|
||||||
- **Manual dividers** — insert a `{.card}` or `{.card Title}` line wherever you want a new card to start, regardless of heading structure:
|
- Links and images are inlined as `label (url)` plain prose (or bare `url` when there's no distinct label/alt, or when the label equals the url) rather than footnoted or turned into `=> ` lines, since that syntax isn't confirmed safe inside a regular document body.
|
||||||
|
|
||||||
```markdown
|
|
||||||
Intro shown on the menu card...
|
|
||||||
|
|
||||||
{.card Weather}
|
|
||||||
Cold and clear...
|
|
||||||
|
|
||||||
{.card Trail Status}
|
|
||||||
- North Loop: open
|
|
||||||
```
|
|
||||||
|
|
||||||
- **Heading-level splitting** — opt in with `--renderer-option split_level=N` to start a new card at every heading of level `N` (e.g. `split_level=2` for the common one-H1-title-plus-H2-sections pattern). Ignored if the document contains any `{.card}` markers.
|
|
||||||
|
|
||||||
When a document resolves to two or more cards, `wml` builds a "menu" card by default — the classic WAP-portal pattern: a list of button/link choices leading to each section, each with a Back control (`--renderer-option menu=false` switches to a linear book instead, chaining cards with Next/Back). A section whose content would still be too large for one card even after splitting is automatically paginated further (`--renderer-option max_card_bytes=N`, default 1400 — a nod to classic WAP 1.x per-deck cache limits; `0` disables it).
|
|
||||||
|
|
||||||
| Renderer option | Default | Meaning |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| `title` | *(none)* | Overall deck title, used on the menu card (or the single card, if never split). |
|
|
||||||
| `split_level` | `0` (disabled) | Heading level that starts a new card. Ignored if `{.card}` markers are present. |
|
|
||||||
| `max_card_bytes` | `1400` | Byte safety-net per card; `0` disables. |
|
|
||||||
| `menu` | `true` | Hub-and-spoke menu card vs. linear Next/Back chaining (`menu=false`). |
|
|
||||||
| `nav_next_label` | `More` | Label for forward navigation (linear mode / overflow pagination). |
|
|
||||||
| `nav_prev_label` | `Prev` | Label for backward navigation between overflow pagination parts. |
|
|
||||||
| `nav_back_label` | `Back` | Label for backward navigation from a section back to the menu/previous section. |
|
|
||||||
|
|
||||||
WML 1.3 is the only supported DTD (fixed by design, not a flag). Since WML uses `$` for variable substitution, a literal `$` in your Markdown is escaped to `$$` in the output automatically. Images pass through as `<img src="..." alt="..."/>` referencing the original path/URL — no WBMP conversion is performed.
|
|
||||||
|
|
||||||
## Plugin Architecture
|
## Plugin Architecture
|
||||||
|
|
||||||
|
|
@ -186,6 +160,10 @@ Once registered (for example in a small module that imports `md2txt.cli`), the n
|
||||||
|
|
||||||
The shared preprocessing helpers—YAML frontmatter parsing, recursive include expansion, ASCII art sentinels, and pipe-table sentinels—also live in `src/md2txt/conversion/core.py`, allowing alternate front-ends to reuse exactly the same behaviour without duplicating code.
|
The shared preprocessing helpers—YAML frontmatter parsing, recursive include expansion, ASCII art sentinels, and pipe-table sentinels—also live in `src/md2txt/conversion/core.py`, allowing alternate front-ends to reuse exactly the same behaviour without duplicating code.
|
||||||
|
|
||||||
|
## WML / WAP Output
|
||||||
|
|
||||||
|
The `wml` renderer used to live here. It has moved to [wapdown](https://codeberg.org/randogoth/wapdown), because a WML deck is not a minimal text format: it is an XML tree, and producing one means restructuring a document into navigable cards rather than transforming it. That needed controls the exporters here have no use for, so it now has a repository of its own. The `{.card}` divider moved with it.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
* Directed and vibe coded by Tobias Raayoni Last
|
* Directed and vibe coded by Tobias Raayoni Last
|
||||||
|
|
|
||||||
|
|
@ -1,14 +0,0 @@
|
||||||
Live updates for the ridge trail network. Reception is spotty past the tree line, check before you go.
|
|
||||||
|
|
||||||
{.card Weather}
|
|
||||||
Cold and clear. Wind: **15 mph** gusting from the *northwest*. Permit fee: $5.
|
|
||||||
|
|
||||||
{.card Trail Status}
|
|
||||||
- North Loop: open
|
|
||||||
- South Loop: closed
|
|
||||||
- Summit Spur: open, ice above 2000m
|
|
||||||
|
|
||||||
{.card Contact}
|
|
||||||
Ranger station: [call dispatch](tel:+15555550123)
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -1,19 +0,0 @@
|
||||||
# Trail Conditions
|
|
||||||
|
|
||||||
Live updates for the ridge trail network. Reception is spotty past the tree line, check before you go.
|
|
||||||
|
|
||||||
## Weather
|
|
||||||
|
|
||||||
Cold and clear. Wind: **15 mph** gusting from the *northwest*. Permit fee: $5.
|
|
||||||
|
|
||||||
## Trail Status
|
|
||||||
|
|
||||||
- North Loop: open
|
|
||||||
- South Loop: closed
|
|
||||||
- Summit Spur: open, ice above 2000m
|
|
||||||
|
|
||||||
## Contact
|
|
||||||
|
|
||||||
Ranger station: [call dispatch](tel:+15555550123)
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -1,17 +0,0 @@
|
||||||
<?xml version="1.0"?>
|
|
||||||
<!DOCTYPE wml PUBLIC "-//WAPFORUM//DTD WML 1.3//EN" "http://www.wapforum.org/DTD/wml13.dtd">
|
|
||||||
<wml>
|
|
||||||
<card id="card1">
|
|
||||||
<p><b><big>Trail Conditions</big></b></p>
|
|
||||||
<p>Live updates for the ridge trail network. Reception is spotty past the tree line, check before you go.</p>
|
|
||||||
<p><b>Weather</b></p>
|
|
||||||
<p>Cold and clear. Wind: <b>15 mph</b> gusting from the <i>northwest</i>. Permit fee: $$5.</p>
|
|
||||||
<p><b>Trail Status</b></p>
|
|
||||||
<p>- North Loop: open</p>
|
|
||||||
<p>- South Loop: closed</p>
|
|
||||||
<p>- Summit Spur: open, ice above 2000m</p>
|
|
||||||
<p><b>Contact</b></p>
|
|
||||||
<p>Ranger station: <a href="tel:+15555550123">call dispatch</a></p>
|
|
||||||
<p><img src="https://example.com/map.png" alt="Trail map"/></p>
|
|
||||||
</card>
|
|
||||||
</wml>
|
|
||||||
|
|
@ -25,7 +25,7 @@ from .renderers.text import TextRenderer
|
||||||
from .renderers import ama # noqa: F401 # register AMA renderer plugin
|
from .renderers import ama # noqa: F401 # register AMA renderer plugin
|
||||||
from .renderers import gemini # noqa: F401 # register Gemini renderer plugin
|
from .renderers import gemini # noqa: F401 # register Gemini renderer plugin
|
||||||
from .renderers import micron # noqa: F401 # register Micron renderer plugin
|
from .renderers import micron # noqa: F401 # register Micron renderer plugin
|
||||||
from .renderers import wml # noqa: F401 # register WML renderer plugin
|
from .renderers import nex # noqa: F401 # register Nex renderer plugin
|
||||||
|
|
||||||
|
|
||||||
def _split_option(token: str) -> Tuple[str, str]:
|
def _split_option(token: str) -> Tuple[str, str]:
|
||||||
|
|
|
||||||
|
|
@ -15,7 +15,6 @@ class BlockKind(Enum):
|
||||||
TABLE = "table"
|
TABLE = "table"
|
||||||
BLANK_LINE = "blank_line"
|
BLANK_LINE = "blank_line"
|
||||||
CUSTOM_BLOCK = "custom_block"
|
CUSTOM_BLOCK = "custom_block"
|
||||||
CARD_BREAK = "card_break"
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
|
|
@ -103,18 +102,6 @@ class TablePayload:
|
||||||
rows: List[List[str]] # rows[0] is the header row
|
rows: List[List[str]] # rows[0] is the header row
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class CardBreakPayload:
|
|
||||||
"""Marks an explicit, renderer-agnostic 'start a new page/card here' hint.
|
|
||||||
|
|
||||||
Emitted for the `{.card}` / `{.card Title}` directive. Formats without a
|
|
||||||
notion of pages (plain text, Gemtext, Micron, AMA) simply ignore this
|
|
||||||
event kind via their existing unhandled-BlockKind no-op path.
|
|
||||||
"""
|
|
||||||
|
|
||||||
title: Optional[str] = None
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
class BlockEvent:
|
class BlockEvent:
|
||||||
kind: BlockKind
|
kind: BlockKind
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,6 @@ from ..models import (
|
||||||
BlockKind,
|
BlockKind,
|
||||||
BlockQuotePayload,
|
BlockQuotePayload,
|
||||||
BlockStyle,
|
BlockStyle,
|
||||||
CardBreakPayload,
|
|
||||||
CodeBlockPayload,
|
CodeBlockPayload,
|
||||||
HeadingPayload,
|
HeadingPayload,
|
||||||
ListItemPayload,
|
ListItemPayload,
|
||||||
|
|
@ -29,7 +28,6 @@ ORDERED_LIST_PATTERN = re.compile(r"^(\s*)(\d+\.)(\s+)(.*)$")
|
||||||
UNORDERED_LIST_PATTERN = re.compile(r"^(\s*)([*+-])(\s+)(.*)$")
|
UNORDERED_LIST_PATTERN = re.compile(r"^(\s*)([*+-])(\s+)(.*)$")
|
||||||
BLOCKQUOTE_PATTERN = re.compile(r"^\s{0,3}>(.*)$")
|
BLOCKQUOTE_PATTERN = re.compile(r"^\s{0,3}>(.*)$")
|
||||||
HORIZONTAL_RULE_PATTERN = re.compile(r"^\s*([-*_])(?:\s*\1){2,}\s*$")
|
HORIZONTAL_RULE_PATTERN = re.compile(r"^\s*([-*_])(?:\s*\1){2,}\s*$")
|
||||||
CARD_BREAK_PATTERN = re.compile(r"^\s*\{\s*\.card(?:\s+(?P<title>.+?))?\s*\}\s*$")
|
|
||||||
INLINE_PARA_RE = re.compile(r"^\s*<p\b([^>]*)>(.*?)</p>\s*$", re.IGNORECASE)
|
INLINE_PARA_RE = re.compile(r"^\s*<p\b([^>]*)>(.*?)</p>\s*$", re.IGNORECASE)
|
||||||
PARA_OPEN_RE = re.compile(r"^\s*<p\b([^>]*)>\s*$", re.IGNORECASE)
|
PARA_OPEN_RE = re.compile(r"^\s*<p\b([^>]*)>\s*$", re.IGNORECASE)
|
||||||
PARA_CLOSE_RE = re.compile(r"^\s*</p>\s*$", re.IGNORECASE)
|
PARA_CLOSE_RE = re.compile(r"^\s*</p>\s*$", re.IGNORECASE)
|
||||||
|
|
@ -117,22 +115,6 @@ class MarkdownParser:
|
||||||
yield event
|
yield event
|
||||||
indented_code_lines = []
|
indented_code_lines = []
|
||||||
|
|
||||||
card_break_match = CARD_BREAK_PATTERN.match(line)
|
|
||||||
if card_break_match:
|
|
||||||
event = self._flush_paragraph(current_paragraph)
|
|
||||||
if event is not None:
|
|
||||||
yield event
|
|
||||||
current_paragraph = []
|
|
||||||
title = card_break_match.group("title")
|
|
||||||
self._last_stylable_block = False
|
|
||||||
yield BlockEvent(
|
|
||||||
kind=BlockKind.CARD_BREAK,
|
|
||||||
payload=CardBreakPayload(title=title.strip() if title else None),
|
|
||||||
style=self._clone_style(),
|
|
||||||
stylable=False,
|
|
||||||
)
|
|
||||||
continue
|
|
||||||
|
|
||||||
inline_para = INLINE_PARA_RE.match(line.strip())
|
inline_para = INLINE_PARA_RE.match(line.strip())
|
||||||
if inline_para:
|
if inline_para:
|
||||||
event = self._flush_paragraph(current_paragraph)
|
event = self._flush_paragraph(current_paragraph)
|
||||||
|
|
|
||||||
|
|
@ -1,429 +0,0 @@
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import sys
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from typing import Any, List, Optional, Tuple
|
|
||||||
|
|
||||||
from ..models import (
|
|
||||||
BlockEvent,
|
|
||||||
BlockKind,
|
|
||||||
FrontMatter,
|
|
||||||
StyleUpdateEvent,
|
|
||||||
)
|
|
||||||
from ..plugins import register_renderer
|
|
||||||
from .text import (
|
|
||||||
BOLD_RE,
|
|
||||||
CODE_STASH_RE,
|
|
||||||
IMAGE_RE,
|
|
||||||
ITALIC_RE,
|
|
||||||
LINK_RE,
|
|
||||||
STRIKETHROUGH_RE,
|
|
||||||
UNDERLINE_EM_RE,
|
|
||||||
UNDERLINE_STRONG_RE,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
WML_PROLOG = (
|
|
||||||
'<?xml version="1.0"?>\n'
|
|
||||||
'<!DOCTYPE wml PUBLIC "-//WAPFORUM//DTD WML 1.3//EN" '
|
|
||||||
'"http://www.wapforum.org/DTD/wml13.dtd">\n'
|
|
||||||
)
|
|
||||||
|
|
||||||
# A conservative nod to classic WAP 1.x per-deck cache limits; purely a
|
|
||||||
# safety net for a section that's still too big after §-based splitting,
|
|
||||||
# not the primary splitting mechanism. Reserves headroom for the nav <do>
|
|
||||||
# markup each continuation card also carries.
|
|
||||||
DEFAULT_MAX_CARD_BYTES = 1400
|
|
||||||
_NAV_HEADROOM_BYTES = 160
|
|
||||||
|
|
||||||
|
|
||||||
def escape_wml(text: str) -> str:
|
|
||||||
"""XML-escape text for WML element content or a double-quoted attribute
|
|
||||||
value, plus WML's own quirk: a literal '$' must become '$$' since WML
|
|
||||||
uses '$' for variable substitution. Applied once, up front, to the whole
|
|
||||||
raw string before any inline markup is turned into tags (see
|
|
||||||
`WMLRenderer._process_inline`), so link/image URLs captured out of the
|
|
||||||
already-escaped text are correctly escaped for attribute use too.
|
|
||||||
"""
|
|
||||||
text = text.replace("&", "&")
|
|
||||||
text = text.replace("<", "<").replace(">", ">").replace('"', """)
|
|
||||||
return text.replace("$", "$$")
|
|
||||||
|
|
||||||
|
|
||||||
def _coerce_bool(value: Any, default: bool) -> bool:
|
|
||||||
if value is None:
|
|
||||||
return default
|
|
||||||
if isinstance(value, bool):
|
|
||||||
return value
|
|
||||||
return str(value).strip().lower() not in {"false", "no", "0", "off"}
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class _Section:
|
|
||||||
title: Optional[str]
|
|
||||||
events: List[BlockEvent]
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class _Card:
|
|
||||||
id: str
|
|
||||||
title: Optional[str]
|
|
||||||
fragments: List[str]
|
|
||||||
back_label: Optional[str] = None
|
|
||||||
next_id: Optional[str] = None
|
|
||||||
next_label: Optional[str] = None
|
|
||||||
|
|
||||||
|
|
||||||
class WMLRenderer:
|
|
||||||
"""Render Markdown events as a WML deck for WAP browsers.
|
|
||||||
|
|
||||||
Unlike TextRenderer/MicronRenderer/AmaRenderer, which emit output
|
|
||||||
incrementally per `handle_event`, this renderer buffers the whole event
|
|
||||||
stream and does the real work in `finalize()` — card boundaries, the
|
|
||||||
menu card's link list, and forward/backward nav targets all depend on
|
|
||||||
the full document's structure being known up front. It still satisfies
|
|
||||||
the same Renderer protocol (`handle_event`/`finalize`), so it plugs into
|
|
||||||
`run_pipeline` unchanged; `finalize()` just returns a single WML string
|
|
||||||
instead of a list of lines.
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(self, frontmatter: FrontMatter, **options: Any) -> None:
|
|
||||||
self.frontmatter = frontmatter
|
|
||||||
self._events: List[BlockEvent] = []
|
|
||||||
# 0 = disabled. Heading-based splitting is opt-in, not a default —
|
|
||||||
# a document with headings but no explicit split_level stays one card.
|
|
||||||
self.split_level = int(options.get("split_level", 0) or 0)
|
|
||||||
self.max_card_bytes = int(options.get("max_card_bytes", DEFAULT_MAX_CARD_BYTES) or 0)
|
|
||||||
self.menu = _coerce_bool(options.get("menu"), True)
|
|
||||||
self.nav_next_label = str(options.get("nav_next_label", "More"))
|
|
||||||
self.nav_prev_label = str(options.get("nav_prev_label", "Prev"))
|
|
||||||
self.nav_back_label = str(options.get("nav_back_label", "Back"))
|
|
||||||
title = options.get("title")
|
|
||||||
self.title: Optional[str] = str(title) if title else None
|
|
||||||
|
|
||||||
# -- Renderer protocol ---------------------------------------------------
|
|
||||||
|
|
||||||
def handle_event(self, event: "BlockEvent | StyleUpdateEvent") -> None:
|
|
||||||
if isinstance(event, BlockEvent):
|
|
||||||
self._events.append(event)
|
|
||||||
# StyleUpdateEvent: WML has no per-block alignment/style stack to
|
|
||||||
# apply this to, by design; ignored like other structural mismatches.
|
|
||||||
|
|
||||||
def finalize(self) -> str:
|
|
||||||
preamble_events, sections = self._split_sections(self._events)
|
|
||||||
if len(sections) < 2:
|
|
||||||
all_events = list(preamble_events)
|
|
||||||
if sections:
|
|
||||||
all_events.extend(sections[0].events)
|
|
||||||
fragments = self._render_events(all_events)
|
|
||||||
cards = self._pack_section(
|
|
||||||
"card1",
|
|
||||||
self.title,
|
|
||||||
fragments,
|
|
||||||
entry_back_label=None,
|
|
||||||
exit_next_id=None,
|
|
||||||
exit_next_label=None,
|
|
||||||
)
|
|
||||||
elif self.menu:
|
|
||||||
cards = self._build_hub_and_spoke(preamble_events, sections)
|
|
||||||
else:
|
|
||||||
cards = self._build_linear(preamble_events, sections)
|
|
||||||
body = "\n".join(self._render_card(card) for card in cards)
|
|
||||||
return f"{WML_PROLOG}<wml>\n{body}\n</wml>\n"
|
|
||||||
|
|
||||||
# -- Splitting into sections ----------------------------------------------
|
|
||||||
|
|
||||||
def _split_sections(self, events: List[BlockEvent]) -> Tuple[List[BlockEvent], List[_Section]]:
|
|
||||||
card_break_indices = [i for i, e in enumerate(events) if e.kind == BlockKind.CARD_BREAK]
|
|
||||||
if card_break_indices:
|
|
||||||
return self._split_by_card_breaks(events, card_break_indices)
|
|
||||||
if self.split_level > 0:
|
|
||||||
heading_indices = [
|
|
||||||
i
|
|
||||||
for i, e in enumerate(events)
|
|
||||||
if e.kind == BlockKind.HEADING and e.payload.level == self.split_level
|
|
||||||
]
|
|
||||||
if heading_indices:
|
|
||||||
return self._split_by_heading(events, heading_indices)
|
|
||||||
return list(events), []
|
|
||||||
|
|
||||||
def _split_by_card_breaks(
|
|
||||||
self, events: List[BlockEvent], indices: List[int]
|
|
||||||
) -> Tuple[List[BlockEvent], List[_Section]]:
|
|
||||||
preamble = events[: indices[0]]
|
|
||||||
boundaries = indices + [len(events)]
|
|
||||||
sections: List[_Section] = []
|
|
||||||
for idx in range(len(indices)):
|
|
||||||
start = indices[idx] + 1 # skip the {.card} marker event itself
|
|
||||||
end = boundaries[idx + 1]
|
|
||||||
section_events = events[start:end]
|
|
||||||
marker_title = events[indices[idx]].payload.title
|
|
||||||
title = marker_title or self._first_heading_text(section_events)
|
|
||||||
sections.append(_Section(title=title, events=section_events))
|
|
||||||
return preamble, sections
|
|
||||||
|
|
||||||
def _split_by_heading(
|
|
||||||
self, events: List[BlockEvent], indices: List[int]
|
|
||||||
) -> Tuple[List[BlockEvent], List[_Section]]:
|
|
||||||
preamble = events[: indices[0]]
|
|
||||||
boundaries = indices + [len(events)]
|
|
||||||
sections: List[_Section] = []
|
|
||||||
for idx in range(len(indices)):
|
|
||||||
start = indices[idx] # keep the heading event, it still renders in-body
|
|
||||||
end = boundaries[idx + 1]
|
|
||||||
section_events = events[start:end]
|
|
||||||
title = events[indices[idx]].payload.text
|
|
||||||
sections.append(_Section(title=title, events=section_events))
|
|
||||||
return preamble, sections
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _first_heading_text(events: List[BlockEvent]) -> Optional[str]:
|
|
||||||
for event in events:
|
|
||||||
if event.kind == BlockKind.HEADING:
|
|
||||||
return event.payload.text
|
|
||||||
return None
|
|
||||||
|
|
||||||
# -- Card/menu/navigation topology ---------------------------------------
|
|
||||||
|
|
||||||
def _build_hub_and_spoke(
|
|
||||||
self, preamble_events: List[BlockEvent], sections: List[_Section]
|
|
||||||
) -> List[_Card]:
|
|
||||||
menu_fragments = self._render_events(preamble_events, big_first_heading=True)
|
|
||||||
for i, section in enumerate(sections):
|
|
||||||
label = escape_wml(section.title or "Untitled")
|
|
||||||
menu_fragments.append(f'<p><a href="#card{i + 1}">{label}</a></p>')
|
|
||||||
menu_title = self.title or self._first_heading_text(preamble_events)
|
|
||||||
cards: List[_Card] = [
|
|
||||||
_Card(id="menu", title=menu_title, fragments=menu_fragments)
|
|
||||||
]
|
|
||||||
for i, section in enumerate(sections):
|
|
||||||
fragments = self._render_events(section.events, big_first_heading=True)
|
|
||||||
cards.extend(
|
|
||||||
self._pack_section(
|
|
||||||
f"card{i + 1}",
|
|
||||||
section.title or "Untitled",
|
|
||||||
fragments,
|
|
||||||
entry_back_label=self.nav_back_label,
|
|
||||||
exit_next_id=None,
|
|
||||||
exit_next_label=None,
|
|
||||||
)
|
|
||||||
)
|
|
||||||
return cards
|
|
||||||
|
|
||||||
def _build_linear(
|
|
||||||
self, preamble_events: List[BlockEvent], sections: List[_Section]
|
|
||||||
) -> List[_Card]:
|
|
||||||
cards: List[_Card] = []
|
|
||||||
total = len(sections)
|
|
||||||
for i, section in enumerate(sections):
|
|
||||||
events = section.events
|
|
||||||
is_first = i == 0
|
|
||||||
if is_first:
|
|
||||||
events = list(preamble_events) + list(events)
|
|
||||||
fragments = self._render_events(events, big_first_heading=True)
|
|
||||||
# Card 1 also carries the preamble, so prefer the deck-level
|
|
||||||
# title / preamble's own heading over this section's heading.
|
|
||||||
card_title = (
|
|
||||||
(self.title or self._first_heading_text(preamble_events) or section.title or "Untitled")
|
|
||||||
if is_first
|
|
||||||
else (section.title or "Untitled")
|
|
||||||
)
|
|
||||||
is_last = i == total - 1
|
|
||||||
next_id = f"card{i + 2}" if not is_last else None
|
|
||||||
cards.extend(
|
|
||||||
self._pack_section(
|
|
||||||
f"card{i + 1}",
|
|
||||||
card_title,
|
|
||||||
fragments,
|
|
||||||
entry_back_label=None if is_first else self.nav_back_label,
|
|
||||||
exit_next_id=next_id,
|
|
||||||
exit_next_label=self.nav_next_label if next_id else None,
|
|
||||||
)
|
|
||||||
)
|
|
||||||
return cards
|
|
||||||
|
|
||||||
def _pack_section(
|
|
||||||
self,
|
|
||||||
base_id: str,
|
|
||||||
title: Optional[str],
|
|
||||||
fragments: List[str],
|
|
||||||
*,
|
|
||||||
entry_back_label: Optional[str],
|
|
||||||
exit_next_id: Optional[str],
|
|
||||||
exit_next_label: Optional[str],
|
|
||||||
) -> List[_Card]:
|
|
||||||
"""Split one section's rendered fragments into one or more physical
|
|
||||||
<card>s if they'd exceed max_card_bytes (0 disables). This is the
|
|
||||||
secondary byte-overflow safety net, not the primary splitting
|
|
||||||
mechanism (that's §-based or manual {.card} splitting upstream).
|
|
||||||
"""
|
|
||||||
chunks = self._greedy_pack(fragments, base_id)
|
|
||||||
cards: List[_Card] = []
|
|
||||||
last = len(chunks) - 1
|
|
||||||
for i, chunk in enumerate(chunks):
|
|
||||||
part_id = base_id if i == 0 else f"{base_id}-p{i + 1}"
|
|
||||||
is_first = i == 0
|
|
||||||
is_last = i == last
|
|
||||||
back_label = entry_back_label if is_first else self.nav_prev_label
|
|
||||||
next_id = f"{base_id}-p{i + 2}" if not is_last else exit_next_id
|
|
||||||
next_label = self.nav_next_label if not is_last else exit_next_label
|
|
||||||
cards.append(
|
|
||||||
_Card(
|
|
||||||
id=part_id,
|
|
||||||
title=title if is_first else None,
|
|
||||||
fragments=chunk,
|
|
||||||
back_label=back_label,
|
|
||||||
next_id=next_id,
|
|
||||||
next_label=next_label,
|
|
||||||
)
|
|
||||||
)
|
|
||||||
return cards
|
|
||||||
|
|
||||||
def _greedy_pack(self, fragments: List[str], base_id: str) -> List[List[str]]:
|
|
||||||
if self.max_card_bytes <= 0:
|
|
||||||
return [fragments]
|
|
||||||
budget = max(1, self.max_card_bytes - _NAV_HEADROOM_BYTES)
|
|
||||||
chunks: List[List[str]] = []
|
|
||||||
current: List[str] = []
|
|
||||||
current_size = 0
|
|
||||||
for frag in fragments:
|
|
||||||
frag_size = len(frag.encode("utf-8"))
|
|
||||||
if frag_size > budget:
|
|
||||||
if current:
|
|
||||||
chunks.append(current)
|
|
||||||
current, current_size = [], 0
|
|
||||||
chunks.append([frag])
|
|
||||||
print(
|
|
||||||
f"wml: warning: a single block in '{base_id}' is {frag_size} bytes, "
|
|
||||||
f"exceeding max_card_bytes ({self.max_card_bytes}); emitting it as its own card",
|
|
||||||
file=sys.stderr,
|
|
||||||
)
|
|
||||||
continue
|
|
||||||
if current and current_size + frag_size > budget:
|
|
||||||
chunks.append(current)
|
|
||||||
current, current_size = [], 0
|
|
||||||
current.append(frag)
|
|
||||||
current_size += frag_size
|
|
||||||
if current:
|
|
||||||
chunks.append(current)
|
|
||||||
return chunks or [[]]
|
|
||||||
|
|
||||||
def _render_card(self, card: _Card) -> str:
|
|
||||||
attrs = f' title="{escape_wml(card.title)}"' if card.title else ""
|
|
||||||
parts = [f'<card id="{card.id}"{attrs}>']
|
|
||||||
if card.back_label:
|
|
||||||
parts.append(f'<do type="prev" label="{escape_wml(card.back_label)}"><prev/></do>')
|
|
||||||
if card.next_id:
|
|
||||||
label = escape_wml(card.next_label or self.nav_next_label)
|
|
||||||
parts.append(f'<do type="accept" label="{label}"><go href="#{card.next_id}"/></do>')
|
|
||||||
parts.extend(card.fragments)
|
|
||||||
parts.append("</card>")
|
|
||||||
return "\n".join(parts)
|
|
||||||
|
|
||||||
# -- Block markup mapping -------------------------------------------------
|
|
||||||
|
|
||||||
def _render_events(self, events: List[BlockEvent], *, big_first_heading: bool = True) -> List[str]:
|
|
||||||
fragments: List[str] = []
|
|
||||||
seen_heading = not big_first_heading
|
|
||||||
list_counter = 0
|
|
||||||
for event in events:
|
|
||||||
kind = event.kind
|
|
||||||
if kind == BlockKind.PARAGRAPH:
|
|
||||||
fragments.append(f"<p>{self._process_inline(event.payload.text)}</p>")
|
|
||||||
list_counter = 0
|
|
||||||
elif kind == BlockKind.HEADING:
|
|
||||||
text = self._process_inline(event.payload.text)
|
|
||||||
if not seen_heading:
|
|
||||||
fragments.append(f"<p><b><big>{text}</big></b></p>")
|
|
||||||
seen_heading = True
|
|
||||||
else:
|
|
||||||
fragments.append(f"<p><b>{text}</b></p>")
|
|
||||||
list_counter = 0
|
|
||||||
elif kind == BlockKind.LIST_ITEM:
|
|
||||||
text = self._process_inline(event.payload.text)
|
|
||||||
if event.payload.ordered:
|
|
||||||
list_counter += 1
|
|
||||||
fragments.append(f"<p>{list_counter}. {text}</p>")
|
|
||||||
else:
|
|
||||||
list_counter = 0
|
|
||||||
fragments.append(f"<p>- {text}</p>")
|
|
||||||
elif kind == BlockKind.CODE_BLOCK:
|
|
||||||
lines = [escape_wml(line) for line in event.payload.lines]
|
|
||||||
fragments.append(f"<p>{'<br/>'.join(lines)}</p>")
|
|
||||||
list_counter = 0
|
|
||||||
elif kind == BlockKind.BLOCKQUOTE:
|
|
||||||
fragments.append(f"<p><i>{self._process_inline(event.payload.text)}</i></p>")
|
|
||||||
list_counter = 0
|
|
||||||
elif kind == BlockKind.HORIZONTAL_RULE:
|
|
||||||
fragments.append("<p>------------</p>")
|
|
||||||
list_counter = 0
|
|
||||||
elif kind == BlockKind.TABLE:
|
|
||||||
fragments.append(self._render_table(event.payload.rows))
|
|
||||||
list_counter = 0
|
|
||||||
# BLANK_LINE, CARD_BREAK, CUSTOM_BLOCK: no WML output of their own.
|
|
||||||
return fragments
|
|
||||||
|
|
||||||
def _render_table(self, rows: List[List[str]]) -> str:
|
|
||||||
if not rows:
|
|
||||||
return ""
|
|
||||||
columns = max(len(row) for row in rows)
|
|
||||||
tr_parts: List[str] = []
|
|
||||||
for row_index, row in enumerate(rows):
|
|
||||||
cells = list(row) + [""] * (columns - len(row))
|
|
||||||
td_parts = []
|
|
||||||
for cell in cells:
|
|
||||||
content = self._process_inline(cell)
|
|
||||||
if row_index == 0:
|
|
||||||
content = f"<b>{content}</b>"
|
|
||||||
td_parts.append(f"<td>{content}</td>")
|
|
||||||
tr_parts.append(f"<tr>{''.join(td_parts)}</tr>")
|
|
||||||
return f'<table columns="{columns}">{"".join(tr_parts)}</table>'
|
|
||||||
|
|
||||||
# -- Inline markup mapping -------------------------------------------------
|
|
||||||
|
|
||||||
def _process_inline(self, text: str) -> str:
|
|
||||||
# Escape once, up front; the markdown syntax markers below aren't
|
|
||||||
# XML-significant characters, so inline tag insertion on top of the
|
|
||||||
# already-escaped text is always safe (see `escape_wml`'s docstring).
|
|
||||||
escaped = escape_wml(text)
|
|
||||||
|
|
||||||
code_segments: List[str] = []
|
|
||||||
|
|
||||||
def stash_code(match) -> str:
|
|
||||||
code_segments.append(match.group(0)[1:-1])
|
|
||||||
return f"\u0000CODE{len(code_segments) - 1}\u0000"
|
|
||||||
|
|
||||||
escaped = CODE_STASH_RE.sub(stash_code, escaped)
|
|
||||||
escaped = STRIKETHROUGH_RE.sub(lambda m: m.group(1), escaped) # no WML equivalent
|
|
||||||
escaped = BOLD_RE.sub(lambda m: f"<b>{m.group(1)}</b>", escaped)
|
|
||||||
escaped = UNDERLINE_STRONG_RE.sub(lambda m: f"<b>{m.group(1)}</b>", escaped)
|
|
||||||
escaped = ITALIC_RE.sub(lambda m: f"<i>{m.group(1)}</i>", escaped)
|
|
||||||
escaped = UNDERLINE_EM_RE.sub(lambda m: f"<i>{m.group(1)}</i>", escaped)
|
|
||||||
escaped = LINK_RE.sub(self._replace_link, escaped)
|
|
||||||
escaped = IMAGE_RE.sub(self._replace_image, escaped)
|
|
||||||
|
|
||||||
for idx, code in enumerate(code_segments):
|
|
||||||
escaped = escaped.replace(f"\u0000CODE{idx}\u0000", code)
|
|
||||||
return escaped
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _replace_link(match) -> str:
|
|
||||||
label = match.group(1).strip()
|
|
||||||
url = match.group(2).strip()
|
|
||||||
return f'<a href="{url}">{label}</a>'
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _replace_image(match) -> str:
|
|
||||||
alt = match.group(1).strip()
|
|
||||||
src = match.group(2).strip()
|
|
||||||
return f'<img src="{src}" alt="{alt}"/>'
|
|
||||||
|
|
||||||
|
|
||||||
def _wml_renderer_factory(*, frontmatter: FrontMatter, **options: Any) -> WMLRenderer:
|
|
||||||
return WMLRenderer(frontmatter, **options)
|
|
||||||
|
|
||||||
|
|
||||||
try:
|
|
||||||
register_renderer("wml", _wml_renderer_factory)
|
|
||||||
except ValueError:
|
|
||||||
pass
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue