Add WML renderer for WAP decks

Adds a `wml` renderer plugin converting Markdown into WML decks for
old WAP browsers, following the existing parser -> BlockEvent stream
-> renderer plugin architecture (see gemini/micron/ama renderers).

WML content is idiomatically short and button/link-navigated rather
than one long page, so unlike the other renderers, `wml` can split a
document into multiple <card>s -- but only opt-in:

- Manual `{.card}` / `{.card Title}` divider directives (new,
  format-agnostic BlockKind.CARD_BREAK; silently ignored by every
  other renderer via their existing unhandled-kind no-op path).
- Heading-level splitting via `--renderer-option split_level=N`,
  ignored when manual dividers are present.
- Plain documents with neither stay a single card (the safe default).

When a document resolves to 2+ cards, a hub-and-spoke menu card is
built by default (button/link choices -> section, Back to menu), or
`menu=false` switches to a linear Next/Back chain. A byte-overflow
safety net (`max_card_bytes`, default 1400) paginates any single
section that's still too large.

Also adds GFM pipe-table support (BlockKind.TABLE) via the same
sentinel-injection preprocessing trick already used for ASCII art
blocks, so the parser's line-by-line loop needed no lookahead rework.
Rendered as `<table>` in wml, silently ignored elsewhere.

No new dependencies. No devbox/jj/test-suite additions -- decided
against for this repo (see README for the full option reference).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
randogoth 2026-09-23 08:22:14 +03:00
parent b539d037c6
commit d5046543b3
9 changed files with 641 additions and 3 deletions

View file

@ -19,7 +19,7 @@ uv tool install --from git+https://codeberg.org/randogoth/md2txt/ md2txt
## CLI ## CLI
- `md2txt` – converts Markdown into plain text with extensive formatting support. It ships with the default `markdown` parser and `text` renderer plugins, registers optional `micron`, `ama`, and `gemini` renderers for Micron/Ancient Machine Book or Gemtext output, and exposes the core pipeline so you can add your own parser or renderer modules: - `md2txt` – converts Markdown into plain text with extensive formatting support. It ships with the default `markdown` parser and `text` renderer plugins, registers optional `micron`, `ama`, `gemini`, and `wml` renderers for Micron/Ancient Machine Book, Gemtext, or WML (WAP) output, and exposes the core pipeline so you can add your own parser or renderer modules:
- FIGlet-rendered headings. - FIGlet-rendered headings.
- Emphasis styles converted to spaced or delimited characters. - Emphasis styles converted to spaced or delimited characters.
- Optional alignment and margin controls via HTML `<p>` attributes or MultiMarkdown attribute blocks (e.g. `{:.center margin=20px}`) applied as leading spaces. - Optional alignment and margin controls via HTML `<p>` attributes or MultiMarkdown attribute blocks (e.g. `{:.center margin=20px}`) applied as leading spaces.
@ -36,6 +36,7 @@ md2txt input.md --width 72 # override column width
md2txt input.md --renderer micron # emit Micron-formatted output for NomadNet md2txt input.md --renderer micron # emit Micron-formatted output for NomadNet
md2txt input.md --renderer ama # emit AMA markup for Ancient Machine Book md2txt input.md --renderer ama # emit AMA markup for Ancient Machine Book
md2txt input.md --renderer gemini # emit Gemtext for Gemini capsules md2txt input.md --renderer gemini # emit Gemtext for Gemini capsules
md2txt input.md --renderer wml # emit a single-card WML deck for WAP browsers
md2txt input.md --renderer-option width=68 # pass column width to a renderer md2txt input.md --renderer-option width=68 # pass column width to a renderer
``` ```
@ -122,6 +123,40 @@ Inline or block-level ASCII art can be embedded directly in Markdown:
Generated text uses CRLF line endings to maintain DOS compatibility. Generated text uses CRLF line endings to maintain DOS compatibility.
## WML Decks
The `wml` renderer converts Markdown into a [WML](https://en.wikipedia.org/wiki/Wireless_Markup_Language) deck for old WAP browsers. WML screens are idiomatically short and button/link-navigated rather than one long scrolling page, so unlike the other renderers, `wml` can split a document into multiple `<card>`s — but only when you ask it to; a plain document with no options stays a single card.
Two independent, both-optional ways to define card boundaries:
- **Manual dividers** — insert a `{.card}` or `{.card Title}` line wherever you want a new card to start, regardless of heading structure:
```markdown
Intro shown on the menu card...
{.card Weather}
Cold and clear...
{.card Trail Status}
- North Loop: open
```
- **Heading-level splitting** — opt in with `--renderer-option split_level=N` to start a new card at every heading of level `N` (e.g. `split_level=2` for the common one-H1-title-plus-H2-sections pattern). Ignored if the document contains any `{.card}` markers.
When a document resolves to two or more cards, `wml` builds a "menu" card by default — the classic WAP-portal pattern: a list of button/link choices leading to each section, each with a Back control (`--renderer-option menu=false` switches to a linear book instead, chaining cards with Next/Back). A section whose content would still be too large for one card even after splitting is automatically paginated further (`--renderer-option max_card_bytes=N`, default 1400 — a nod to classic WAP 1.x per-deck cache limits; `0` disables it).
| Renderer option | Default | Meaning |
| --- | --- | --- |
| `title` | *(none)* | Overall deck title, used on the menu card (or the single card, if never split). |
| `split_level` | `0` (disabled) | Heading level that starts a new card. Ignored if `{.card}` markers are present. |
| `max_card_bytes` | `1400` | Byte safety-net per card; `0` disables. |
| `menu` | `true` | Hub-and-spoke menu card vs. linear Next/Back chaining (`menu=false`). |
| `nav_next_label` | `More` | Label for forward navigation (linear mode / overflow pagination). |
| `nav_prev_label` | `Prev` | Label for backward navigation between overflow pagination parts. |
| `nav_back_label` | `Back` | Label for backward navigation from a section back to the menu/previous section. |
WML 1.3 is the only supported DTD (fixed by design, not a flag). Since WML uses `$` for variable substitution, a literal `$` in your Markdown is escaped to `$$` in the output automatically. Images pass through as `<img src="..." alt="..."/>` referencing the original path/URL — no WBMP conversion is performed.
## Plugin Architecture ## Plugin Architecture
The conversion pipeline lives in `src/md2txt/conversion/core.py` and is exposed via `run_conversion`. It is designed around lightweight factories: The conversion pipeline lives in `src/md2txt/conversion/core.py` and is exposed via `run_conversion`. It is designed around lightweight factories:
@ -149,7 +184,7 @@ register_renderer("ansi", my_renderer_factory)
Once registered (for example in a small module that imports `md2txt.cli`), the new plugins are available via `--parser my-markdown` or `--renderer ansi`. The CLI lists registered plugin names in `--help`, and the helper functions `available_parsers()` / `available_renderers()` return the sorted names if you need to build higher-level tooling. Once registered (for example in a small module that imports `md2txt.cli`), the new plugins are available via `--parser my-markdown` or `--renderer ansi`. The CLI lists registered plugin names in `--help`, and the helper functions `available_parsers()` / `available_renderers()` return the sorted names if you need to build higher-level tooling.
The shared preprocessing helpers—YAML frontmatter parsing, recursive include expansion, and ASCII art sentinels—also live in `src/md2txt/conversion/core.py`, allowing alternate front-ends to reuse exactly the same behaviour without duplicating code. The shared preprocessing helpers—YAML frontmatter parsing, recursive include expansion, ASCII art sentinels, and pipe-table sentinels—also live in `src/md2txt/conversion/core.py`, allowing alternate front-ends to reuse exactly the same behaviour without duplicating code.
--- ---

14
examples/wml-manual.md Normal file
View file

@ -0,0 +1,14 @@
Live updates for the ridge trail network. Reception is spotty past the tree line, check before you go.
{.card Weather}
Cold and clear. Wind: **15 mph** gusting from the *northwest*. Permit fee: $5.
{.card Trail Status}
- North Loop: open
- South Loop: closed
- Summit Spur: open, ice above 2000m
{.card Contact}
Ranger station: [call dispatch](tel:+15555550123)
![Trail map](https://example.com/map.png)

19
examples/wml.md Normal file
View file

@ -0,0 +1,19 @@
# Trail Conditions
Live updates for the ridge trail network. Reception is spotty past the tree line, check before you go.
## Weather
Cold and clear. Wind: **15 mph** gusting from the *northwest*. Permit fee: $5.
## Trail Status
- North Loop: open
- South Loop: closed
- Summit Spur: open, ice above 2000m
## Contact
Ranger station: [call dispatch](tel:+15555550123)
![Trail map](https://example.com/map.png)

17
examples/wml.wml Normal file
View file

@ -0,0 +1,17 @@
<?xml version="1.0"?>
<!DOCTYPE wml PUBLIC "-//WAPFORUM//DTD WML 1.3//EN" "http://www.wapforum.org/DTD/wml13.dtd">
<wml>
<card id="card1">
<p><b><big>Trail Conditions</big></b></p>
<p>Live updates for the ridge trail network. Reception is spotty past the tree line, check before you go.</p>
<p><b>Weather</b></p>
<p>Cold and clear. Wind: <b>15 mph</b> gusting from the <i>northwest</i>. Permit fee: $$5.</p>
<p><b>Trail Status</b></p>
<p>- North Loop: open</p>
<p>- South Loop: closed</p>
<p>- Summit Spur: open, ice above 2000m</p>
<p><b>Contact</b></p>
<p>Ranger station: <a href="tel:+15555550123">call dispatch</a></p>
<p><img src="https://example.com/map.png" alt="Trail map"/></p>
</card>
</wml>

View file

@ -25,6 +25,7 @@ from .renderers.text import TextRenderer
from .renderers import ama # noqa: F401 # register AMA renderer plugin from .renderers import ama # noqa: F401 # register AMA renderer plugin
from .renderers import gemini # noqa: F401 # register Gemini renderer plugin from .renderers import gemini # noqa: F401 # register Gemini renderer plugin
from .renderers import micron # noqa: F401 # register Micron renderer plugin from .renderers import micron # noqa: F401 # register Micron renderer plugin
from .renderers import wml # noqa: F401 # register WML renderer plugin
def _split_option(token: str) -> Tuple[str, str]: def _split_option(token: str) -> Tuple[str, str]:

View file

@ -17,6 +17,8 @@ ASCII_BLOCK_PATTERN = re.compile(
ASCII_INLINE_PATTERN = re.compile(r"#\[(?P<label>[^\]]+)\]\((?P<target>[^)]+)\)") ASCII_INLINE_PATTERN = re.compile(r"#\[(?P<label>[^\]]+)\]\((?P<target>[^)]+)\)")
MMD_ATTR_TAIL_RE = re.compile(r"(.*?)\s*\{\s*:(.+?)\}\s*$") MMD_ATTR_TAIL_RE = re.compile(r"(.*?)\s*\{\s*:(.+?)\}\s*$")
ASCII_SENTINEL_PREFIX = "\u0000ASCII:" ASCII_SENTINEL_PREFIX = "\u0000ASCII:"
TABLE_SENTINEL_PREFIX = "\u0000TABLE:"
SEPARATOR_CELL_PATTERN = re.compile(r"^:?-{1,}:?$")
Event = BlockEvent | StyleUpdateEvent Event = BlockEvent | StyleUpdateEvent
RendererOutput = TypeVar("RendererOutput") RendererOutput = TypeVar("RendererOutput")
@ -269,6 +271,62 @@ def _parse_ascii_label(label: str) -> Tuple[str, str, Optional[str]]:
return block_type, block_name, align return block_type, block_name, align
def _split_pipe_row(line: str) -> List[str]:
trimmed = line.strip()
if trimmed.startswith("|"):
trimmed = trimmed[1:]
if trimmed.endswith("|"):
trimmed = trimmed[:-1]
return [cell.strip() for cell in trimmed.split("|")]
def _is_table_separator_row(line: str, expected_cells: int) -> bool:
stripped = line.strip()
if not stripped:
return False
cells = _split_pipe_row(stripped)
if len(cells) != expected_cells:
return False
return all(SEPARATOR_CELL_PATTERN.match(cell) for cell in cells)
def expand_tables(lines: List[str]) -> List[str]:
"""Collapse GFM-style pipe tables into single JSON sentinel lines.
Mirrors the ASCII-art sentinel trick above: the parser only ever sees one
physical line per table, so no lookahead/pushback is needed in its
line-by-line loop. Runs after `expand_includes` so tables inside included
files are also detected.
"""
expanded: List[str] = []
index = 0
total = len(lines)
while index < total:
line = lines[index]
stripped = line.rstrip("\n")
if stripped.startswith(ASCII_SENTINEL_PREFIX) or stripped.startswith(TABLE_SENTINEL_PREFIX):
expanded.append(line)
index += 1
continue
if "|" in stripped and stripped.strip() and index + 1 < total:
header_cells = _split_pipe_row(stripped)
if header_cells and _is_table_separator_row(lines[index + 1].rstrip("\n"), len(header_cells)):
rows = [header_cells]
index += 2 # header + separator row
while index < total:
row_line = lines[index].rstrip("\n")
if not row_line.strip() or "|" not in row_line:
break
rows.append(_split_pipe_row(row_line))
index += 1
sentinel = f"{TABLE_SENTINEL_PREFIX}{json.dumps({'rows': rows})}\n"
expanded.append(sentinel)
continue
expanded.append(line)
index += 1
return expanded
def run_pipeline( def run_pipeline(
lines: Iterable[str], lines: Iterable[str],
*, *,
@ -278,6 +336,7 @@ def run_pipeline(
) -> RendererOutput: ) -> RendererOutput:
base_dir = (base_path or Path.cwd()).resolve() base_dir = (base_path or Path.cwd()).resolve()
expanded_lines = expand_includes(list(lines), base_dir, set()) expanded_lines = expand_includes(list(lines), base_dir, set())
expanded_lines = expand_tables(expanded_lines)
for event in parser.parse(expanded_lines): for event in parser.parse(expanded_lines):
renderer.handle_event(event) renderer.handle_event(event)
return renderer.finalize() return renderer.finalize()

View file

@ -12,8 +12,10 @@ class BlockKind(Enum):
BLOCKQUOTE = "blockquote" BLOCKQUOTE = "blockquote"
LIST_ITEM = "list_item" LIST_ITEM = "list_item"
HORIZONTAL_RULE = "horizontal_rule" HORIZONTAL_RULE = "horizontal_rule"
TABLE = "table"
BLANK_LINE = "blank_line" BLANK_LINE = "blank_line"
CUSTOM_BLOCK = "custom_block" CUSTOM_BLOCK = "custom_block"
CARD_BREAK = "card_break"
@dataclass @dataclass
@ -96,6 +98,23 @@ class AsciiArtPayload:
pieces: List[AsciiArtPiece] pieces: List[AsciiArtPiece]
@dataclass
class TablePayload:
rows: List[List[str]] # rows[0] is the header row
@dataclass
class CardBreakPayload:
"""Marks an explicit, renderer-agnostic 'start a new page/card here' hint.
Emitted for the `{.card}` / `{.card Title}` directive. Formats without a
notion of pages (plain text, Gemtext, Micron, AMA) simply ignore this
event kind via their existing unhandled-BlockKind no-op path.
"""
title: Optional[str] = None
@dataclass @dataclass
class BlockEvent: class BlockEvent:
kind: BlockKind kind: BlockKind

View file

@ -5,7 +5,7 @@ import re
from pathlib import Path from pathlib import Path
from typing import Dict, Iterable, Iterator, List, Optional, Union from typing import Dict, Iterable, Iterator, List, Optional, Union
from ..conversion.core import ASCII_SENTINEL_PREFIX from ..conversion.core import ASCII_SENTINEL_PREFIX, TABLE_SENTINEL_PREFIX
from ..models import ( from ..models import (
AsciiArtPayload, AsciiArtPayload,
AsciiArtPiece, AsciiArtPiece,
@ -13,12 +13,14 @@ from ..models import (
BlockKind, BlockKind,
BlockQuotePayload, BlockQuotePayload,
BlockStyle, BlockStyle,
CardBreakPayload,
CodeBlockPayload, CodeBlockPayload,
HeadingPayload, HeadingPayload,
ListItemPayload, ListItemPayload,
ParagraphPayload, ParagraphPayload,
StyleSpec, StyleSpec,
StyleUpdateEvent, StyleUpdateEvent,
TablePayload,
) )
@ -27,6 +29,7 @@ ORDERED_LIST_PATTERN = re.compile(r"^(\s*)(\d+\.)(\s+)(.*)$")
UNORDERED_LIST_PATTERN = re.compile(r"^(\s*)([*+-])(\s+)(.*)$") UNORDERED_LIST_PATTERN = re.compile(r"^(\s*)([*+-])(\s+)(.*)$")
BLOCKQUOTE_PATTERN = re.compile(r"^\s{0,3}>(.*)$") BLOCKQUOTE_PATTERN = re.compile(r"^\s{0,3}>(.*)$")
HORIZONTAL_RULE_PATTERN = re.compile(r"^\s*([-*_])(?:\s*\1){2,}\s*$") HORIZONTAL_RULE_PATTERN = re.compile(r"^\s*([-*_])(?:\s*\1){2,}\s*$")
CARD_BREAK_PATTERN = re.compile(r"^\s*\{\s*\.card(?:\s+(?P<title>.+?))?\s*\}\s*$")
INLINE_PARA_RE = re.compile(r"^\s*<p\b([^>]*)>(.*?)</p>\s*$", re.IGNORECASE) INLINE_PARA_RE = re.compile(r"^\s*<p\b([^>]*)>(.*?)</p>\s*$", re.IGNORECASE)
PARA_OPEN_RE = re.compile(r"^\s*<p\b([^>]*)>\s*$", re.IGNORECASE) PARA_OPEN_RE = re.compile(r"^\s*<p\b([^>]*)>\s*$", re.IGNORECASE)
PARA_CLOSE_RE = re.compile(r"^\s*</p>\s*$", re.IGNORECASE) PARA_CLOSE_RE = re.compile(r"^\s*</p>\s*$", re.IGNORECASE)
@ -71,6 +74,23 @@ class MarkdownParser:
) )
continue continue
if line.startswith(TABLE_SENTINEL_PREFIX):
event = self._flush_paragraph(current_paragraph)
if event is not None:
yield event
current_paragraph = []
payload = self._build_table_payload(line)
style = self._combine_styles(self._current_style(), self._pending_block_style_spec)
self._pending_block_style_spec = None
self._last_stylable_block = True
yield BlockEvent(
kind=BlockKind.TABLE,
payload=payload,
style=style,
stylable=True,
)
continue
if in_code_block: if in_code_block:
if line.strip().startswith("```"): if line.strip().startswith("```"):
event = self._flush_code_block(code_lines) event = self._flush_code_block(code_lines)
@ -97,6 +117,22 @@ class MarkdownParser:
yield event yield event
indented_code_lines = [] indented_code_lines = []
card_break_match = CARD_BREAK_PATTERN.match(line)
if card_break_match:
event = self._flush_paragraph(current_paragraph)
if event is not None:
yield event
current_paragraph = []
title = card_break_match.group("title")
self._last_stylable_block = False
yield BlockEvent(
kind=BlockKind.CARD_BREAK,
payload=CardBreakPayload(title=title.strip() if title else None),
style=self._clone_style(),
stylable=False,
)
continue
inline_para = INLINE_PARA_RE.match(line.strip()) inline_para = INLINE_PARA_RE.match(line.strip())
if inline_para: if inline_para:
event = self._flush_paragraph(current_paragraph) event = self._flush_paragraph(current_paragraph)
@ -336,6 +372,15 @@ class MarkdownParser:
) )
return AsciiArtPayload(pieces=pieces) return AsciiArtPayload(pieces=pieces)
def _build_table_payload(self, sentinel_line: str) -> TablePayload:
payload = sentinel_line[len(TABLE_SENTINEL_PREFIX) :]
try:
data = json.loads(payload)
except json.JSONDecodeError as exc:
raise ValueError(f"Invalid table sentinel payload: {payload}") from exc
rows = data.get("rows", [])
return TablePayload(rows=[[str(cell) for cell in row] for row in rows])
def _decode_ascii_sentinel(self, line: str) -> List[Dict[str, str]]: def _decode_ascii_sentinel(self, line: str) -> List[Dict[str, str]]:
payload = line[len(ASCII_SENTINEL_PREFIX) :] payload = line[len(ASCII_SENTINEL_PREFIX) :]
try: try:

429
src/md2txt/renderers/wml.py Normal file
View file

@ -0,0 +1,429 @@
from __future__ import annotations
import sys
from dataclasses import dataclass
from typing import Any, List, Optional, Tuple
from ..models import (
BlockEvent,
BlockKind,
FrontMatter,
StyleUpdateEvent,
)
from ..plugins import register_renderer
from .text import (
BOLD_RE,
CODE_STASH_RE,
IMAGE_RE,
ITALIC_RE,
LINK_RE,
STRIKETHROUGH_RE,
UNDERLINE_EM_RE,
UNDERLINE_STRONG_RE,
)
WML_PROLOG = (
'<?xml version="1.0"?>\n'
'<!DOCTYPE wml PUBLIC "-//WAPFORUM//DTD WML 1.3//EN" '
'"http://www.wapforum.org/DTD/wml13.dtd">\n'
)
# A conservative nod to classic WAP 1.x per-deck cache limits; purely a
# safety net for a section that's still too big after §-based splitting,
# not the primary splitting mechanism. Reserves headroom for the nav <do>
# markup each continuation card also carries.
DEFAULT_MAX_CARD_BYTES = 1400
_NAV_HEADROOM_BYTES = 160
def escape_wml(text: str) -> str:
"""XML-escape text for WML element content or a double-quoted attribute
value, plus WML's own quirk: a literal '$' must become '$$' since WML
uses '$' for variable substitution. Applied once, up front, to the whole
raw string before any inline markup is turned into tags (see
`WMLRenderer._process_inline`), so link/image URLs captured out of the
already-escaped text are correctly escaped for attribute use too.
"""
text = text.replace("&", "&amp;")
text = text.replace("<", "&lt;").replace(">", "&gt;").replace('"', "&quot;")
return text.replace("$", "$$")
def _coerce_bool(value: Any, default: bool) -> bool:
if value is None:
return default
if isinstance(value, bool):
return value
return str(value).strip().lower() not in {"false", "no", "0", "off"}
@dataclass
class _Section:
title: Optional[str]
events: List[BlockEvent]
@dataclass
class _Card:
id: str
title: Optional[str]
fragments: List[str]
back_label: Optional[str] = None
next_id: Optional[str] = None
next_label: Optional[str] = None
class WMLRenderer:
"""Render Markdown events as a WML deck for WAP browsers.
Unlike TextRenderer/MicronRenderer/AmaRenderer, which emit output
incrementally per `handle_event`, this renderer buffers the whole event
stream and does the real work in `finalize()` — card boundaries, the
menu card's link list, and forward/backward nav targets all depend on
the full document's structure being known up front. It still satisfies
the same Renderer protocol (`handle_event`/`finalize`), so it plugs into
`run_pipeline` unchanged; `finalize()` just returns a single WML string
instead of a list of lines.
"""
def __init__(self, frontmatter: FrontMatter, **options: Any) -> None:
self.frontmatter = frontmatter
self._events: List[BlockEvent] = []
# 0 = disabled. Heading-based splitting is opt-in, not a default —
# a document with headings but no explicit split_level stays one card.
self.split_level = int(options.get("split_level", 0) or 0)
self.max_card_bytes = int(options.get("max_card_bytes", DEFAULT_MAX_CARD_BYTES) or 0)
self.menu = _coerce_bool(options.get("menu"), True)
self.nav_next_label = str(options.get("nav_next_label", "More"))
self.nav_prev_label = str(options.get("nav_prev_label", "Prev"))
self.nav_back_label = str(options.get("nav_back_label", "Back"))
title = options.get("title")
self.title: Optional[str] = str(title) if title else None
# -- Renderer protocol ---------------------------------------------------
def handle_event(self, event: "BlockEvent | StyleUpdateEvent") -> None:
if isinstance(event, BlockEvent):
self._events.append(event)
# StyleUpdateEvent: WML has no per-block alignment/style stack to
# apply this to, by design; ignored like other structural mismatches.
def finalize(self) -> str:
preamble_events, sections = self._split_sections(self._events)
if len(sections) < 2:
all_events = list(preamble_events)
if sections:
all_events.extend(sections[0].events)
fragments = self._render_events(all_events)
cards = self._pack_section(
"card1",
self.title,
fragments,
entry_back_label=None,
exit_next_id=None,
exit_next_label=None,
)
elif self.menu:
cards = self._build_hub_and_spoke(preamble_events, sections)
else:
cards = self._build_linear(preamble_events, sections)
body = "\n".join(self._render_card(card) for card in cards)
return f"{WML_PROLOG}<wml>\n{body}\n</wml>\n"
# -- Splitting into sections ----------------------------------------------
def _split_sections(self, events: List[BlockEvent]) -> Tuple[List[BlockEvent], List[_Section]]:
card_break_indices = [i for i, e in enumerate(events) if e.kind == BlockKind.CARD_BREAK]
if card_break_indices:
return self._split_by_card_breaks(events, card_break_indices)
if self.split_level > 0:
heading_indices = [
i
for i, e in enumerate(events)
if e.kind == BlockKind.HEADING and e.payload.level == self.split_level
]
if heading_indices:
return self._split_by_heading(events, heading_indices)
return list(events), []
def _split_by_card_breaks(
self, events: List[BlockEvent], indices: List[int]
) -> Tuple[List[BlockEvent], List[_Section]]:
preamble = events[: indices[0]]
boundaries = indices + [len(events)]
sections: List[_Section] = []
for idx in range(len(indices)):
start = indices[idx] + 1 # skip the {.card} marker event itself
end = boundaries[idx + 1]
section_events = events[start:end]
marker_title = events[indices[idx]].payload.title
title = marker_title or self._first_heading_text(section_events)
sections.append(_Section(title=title, events=section_events))
return preamble, sections
def _split_by_heading(
self, events: List[BlockEvent], indices: List[int]
) -> Tuple[List[BlockEvent], List[_Section]]:
preamble = events[: indices[0]]
boundaries = indices + [len(events)]
sections: List[_Section] = []
for idx in range(len(indices)):
start = indices[idx] # keep the heading event, it still renders in-body
end = boundaries[idx + 1]
section_events = events[start:end]
title = events[indices[idx]].payload.text
sections.append(_Section(title=title, events=section_events))
return preamble, sections
@staticmethod
def _first_heading_text(events: List[BlockEvent]) -> Optional[str]:
for event in events:
if event.kind == BlockKind.HEADING:
return event.payload.text
return None
# -- Card/menu/navigation topology ---------------------------------------
def _build_hub_and_spoke(
self, preamble_events: List[BlockEvent], sections: List[_Section]
) -> List[_Card]:
menu_fragments = self._render_events(preamble_events, big_first_heading=True)
for i, section in enumerate(sections):
label = escape_wml(section.title or "Untitled")
menu_fragments.append(f'<p><a href="#card{i + 1}">{label}</a></p>')
menu_title = self.title or self._first_heading_text(preamble_events)
cards: List[_Card] = [
_Card(id="menu", title=menu_title, fragments=menu_fragments)
]
for i, section in enumerate(sections):
fragments = self._render_events(section.events, big_first_heading=True)
cards.extend(
self._pack_section(
f"card{i + 1}",
section.title or "Untitled",
fragments,
entry_back_label=self.nav_back_label,
exit_next_id=None,
exit_next_label=None,
)
)
return cards
def _build_linear(
self, preamble_events: List[BlockEvent], sections: List[_Section]
) -> List[_Card]:
cards: List[_Card] = []
total = len(sections)
for i, section in enumerate(sections):
events = section.events
is_first = i == 0
if is_first:
events = list(preamble_events) + list(events)
fragments = self._render_events(events, big_first_heading=True)
# Card 1 also carries the preamble, so prefer the deck-level
# title / preamble's own heading over this section's heading.
card_title = (
(self.title or self._first_heading_text(preamble_events) or section.title or "Untitled")
if is_first
else (section.title or "Untitled")
)
is_last = i == total - 1
next_id = f"card{i + 2}" if not is_last else None
cards.extend(
self._pack_section(
f"card{i + 1}",
card_title,
fragments,
entry_back_label=None if is_first else self.nav_back_label,
exit_next_id=next_id,
exit_next_label=self.nav_next_label if next_id else None,
)
)
return cards
def _pack_section(
self,
base_id: str,
title: Optional[str],
fragments: List[str],
*,
entry_back_label: Optional[str],
exit_next_id: Optional[str],
exit_next_label: Optional[str],
) -> List[_Card]:
"""Split one section's rendered fragments into one or more physical
<card>s if they'd exceed max_card_bytes (0 disables). This is the
secondary byte-overflow safety net, not the primary splitting
mechanism (that's §-based or manual {.card} splitting upstream).
"""
chunks = self._greedy_pack(fragments, base_id)
cards: List[_Card] = []
last = len(chunks) - 1
for i, chunk in enumerate(chunks):
part_id = base_id if i == 0 else f"{base_id}-p{i + 1}"
is_first = i == 0
is_last = i == last
back_label = entry_back_label if is_first else self.nav_prev_label
next_id = f"{base_id}-p{i + 2}" if not is_last else exit_next_id
next_label = self.nav_next_label if not is_last else exit_next_label
cards.append(
_Card(
id=part_id,
title=title if is_first else None,
fragments=chunk,
back_label=back_label,
next_id=next_id,
next_label=next_label,
)
)
return cards
def _greedy_pack(self, fragments: List[str], base_id: str) -> List[List[str]]:
if self.max_card_bytes <= 0:
return [fragments]
budget = max(1, self.max_card_bytes - _NAV_HEADROOM_BYTES)
chunks: List[List[str]] = []
current: List[str] = []
current_size = 0
for frag in fragments:
frag_size = len(frag.encode("utf-8"))
if frag_size > budget:
if current:
chunks.append(current)
current, current_size = [], 0
chunks.append([frag])
print(
f"wml: warning: a single block in '{base_id}' is {frag_size} bytes, "
f"exceeding max_card_bytes ({self.max_card_bytes}); emitting it as its own card",
file=sys.stderr,
)
continue
if current and current_size + frag_size > budget:
chunks.append(current)
current, current_size = [], 0
current.append(frag)
current_size += frag_size
if current:
chunks.append(current)
return chunks or [[]]
def _render_card(self, card: _Card) -> str:
attrs = f' title="{escape_wml(card.title)}"' if card.title else ""
parts = [f'<card id="{card.id}"{attrs}>']
if card.back_label:
parts.append(f'<do type="prev" label="{escape_wml(card.back_label)}"><prev/></do>')
if card.next_id:
label = escape_wml(card.next_label or self.nav_next_label)
parts.append(f'<do type="accept" label="{label}"><go href="#{card.next_id}"/></do>')
parts.extend(card.fragments)
parts.append("</card>")
return "\n".join(parts)
# -- Block markup mapping -------------------------------------------------
def _render_events(self, events: List[BlockEvent], *, big_first_heading: bool = True) -> List[str]:
fragments: List[str] = []
seen_heading = not big_first_heading
list_counter = 0
for event in events:
kind = event.kind
if kind == BlockKind.PARAGRAPH:
fragments.append(f"<p>{self._process_inline(event.payload.text)}</p>")
list_counter = 0
elif kind == BlockKind.HEADING:
text = self._process_inline(event.payload.text)
if not seen_heading:
fragments.append(f"<p><b><big>{text}</big></b></p>")
seen_heading = True
else:
fragments.append(f"<p><b>{text}</b></p>")
list_counter = 0
elif kind == BlockKind.LIST_ITEM:
text = self._process_inline(event.payload.text)
if event.payload.ordered:
list_counter += 1
fragments.append(f"<p>{list_counter}. {text}</p>")
else:
list_counter = 0
fragments.append(f"<p>- {text}</p>")
elif kind == BlockKind.CODE_BLOCK:
lines = [escape_wml(line) for line in event.payload.lines]
fragments.append(f"<p>{'<br/>'.join(lines)}</p>")
list_counter = 0
elif kind == BlockKind.BLOCKQUOTE:
fragments.append(f"<p><i>{self._process_inline(event.payload.text)}</i></p>")
list_counter = 0
elif kind == BlockKind.HORIZONTAL_RULE:
fragments.append("<p>------------</p>")
list_counter = 0
elif kind == BlockKind.TABLE:
fragments.append(self._render_table(event.payload.rows))
list_counter = 0
# BLANK_LINE, CARD_BREAK, CUSTOM_BLOCK: no WML output of their own.
return fragments
def _render_table(self, rows: List[List[str]]) -> str:
if not rows:
return ""
columns = max(len(row) for row in rows)
tr_parts: List[str] = []
for row_index, row in enumerate(rows):
cells = list(row) + [""] * (columns - len(row))
td_parts = []
for cell in cells:
content = self._process_inline(cell)
if row_index == 0:
content = f"<b>{content}</b>"
td_parts.append(f"<td>{content}</td>")
tr_parts.append(f"<tr>{''.join(td_parts)}</tr>")
return f'<table columns="{columns}">{"".join(tr_parts)}</table>'
# -- Inline markup mapping -------------------------------------------------
def _process_inline(self, text: str) -> str:
# Escape once, up front; the markdown syntax markers below aren't
# XML-significant characters, so inline tag insertion on top of the
# already-escaped text is always safe (see `escape_wml`'s docstring).
escaped = escape_wml(text)
code_segments: List[str] = []
def stash_code(match) -> str:
code_segments.append(match.group(0)[1:-1])
return f"\u0000CODE{len(code_segments) - 1}\u0000"
escaped = CODE_STASH_RE.sub(stash_code, escaped)
escaped = STRIKETHROUGH_RE.sub(lambda m: m.group(1), escaped) # no WML equivalent
escaped = BOLD_RE.sub(lambda m: f"<b>{m.group(1)}</b>", escaped)
escaped = UNDERLINE_STRONG_RE.sub(lambda m: f"<b>{m.group(1)}</b>", escaped)
escaped = ITALIC_RE.sub(lambda m: f"<i>{m.group(1)}</i>", escaped)
escaped = UNDERLINE_EM_RE.sub(lambda m: f"<i>{m.group(1)}</i>", escaped)
escaped = LINK_RE.sub(self._replace_link, escaped)
escaped = IMAGE_RE.sub(self._replace_image, escaped)
for idx, code in enumerate(code_segments):
escaped = escaped.replace(f"\u0000CODE{idx}\u0000", code)
return escaped
@staticmethod
def _replace_link(match) -> str:
label = match.group(1).strip()
url = match.group(2).strip()
return f'<a href="{url}">{label}</a>'
@staticmethod
def _replace_image(match) -> str:
alt = match.group(1).strip()
src = match.group(2).strip()
return f'<img src="{src}" alt="{alt}"/>'
def _wml_renderer_factory(*, frontmatter: FrontMatter, **options: Any) -> WMLRenderer:
return WMLRenderer(frontmatter, **options)
try:
register_renderer("wml", _wml_renderer_factory)
except ValueError:
pass