md2txt/README.md

193 lines
11 KiB
Markdown
Raw Normal View History

2025-10-20 17:48:57 +03:00
```
                    $$\  $$$$$$\    $$\                 $$\     
                    $$ |$$  __$$\   $$ |                $$ |    
$$$$$$\$$$$\   $$$$$$$ |\__/  $$ |$$$$$$\   $$\   $$\ $$$$$$\   
$$  _$$  _$$\ $$  __$$ | $$$$$$  |\_$$  _|  \$$\ $$  |\_$$  _|  
$$ / $$ / $$ |$$ /  $$ |$$  ____/   $$ |     \$$$$  /   $$ |    
$$ | $$ | $$ |$$ |  $$ |$$ |        $$ |$$\  $$  $$<    $$ |$$\ 
$$ | $$ | $$ |\$$$$$$$ |$$$$$$$$\   \$$$$  |$$  /\$$\   \$$$$  |
\__| \__| \__| \_______|\________|   \____/ \__/  \__|   \____/ 
```
2025-10-19 22:58:10 +03:00
2025-10-20 22:04:59 +03:00
This repository contains the `md2txt` command line tool and supporting libraries for transforming Markdown into arcane formats that work well on retro hardware or constrained text viewers. A small plugin API lets you mix-and-match parsers and renderers so additional formats can plug into the same preprocessing pipeline.
2025-10-19 22:58:10 +03:00
2025-10-20 22:19:58 +03:00
Install the CLI directly from GitHub with [uv](https://github.com/astral-sh/uv):
```
2026-04-27 19:59:04 +02:00
uv tool install --from git+https://codeberg.org/randogoth/md2txt/ md2txt
2025-10-20 22:19:58 +03:00
```
2025-10-20 17:26:38 +03:00
## CLI
2025-10-19 22:58:10 +03:00
Add WML renderer for WAP decks Adds a `wml` renderer plugin converting Markdown into WML decks for old WAP browsers, following the existing parser -> BlockEvent stream -> renderer plugin architecture (see gemini/micron/ama renderers). WML content is idiomatically short and button/link-navigated rather than one long page, so unlike the other renderers, `wml` can split a document into multiple <card>s -- but only opt-in: - Manual `{.card}` / `{.card Title}` divider directives (new, format-agnostic BlockKind.CARD_BREAK; silently ignored by every other renderer via their existing unhandled-kind no-op path). - Heading-level splitting via `--renderer-option split_level=N`, ignored when manual dividers are present. - Plain documents with neither stay a single card (the safe default). When a document resolves to 2+ cards, a hub-and-spoke menu card is built by default (button/link choices -> section, Back to menu), or `menu=false` switches to a linear Next/Back chain. A byte-overflow safety net (`max_card_bytes`, default 1400) paginates any single section that's still too large. Also adds GFM pipe-table support (BlockKind.TABLE) via the same sentinel-injection preprocessing trick already used for ASCII art blocks, so the parser's line-by-line loop needed no lookahead rework. Rendered as `<table>` in wml, silently ignored elsewhere. No new dependencies. No devbox/jj/test-suite additions -- decided against for this repo (see README for the full option reference). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-23 08:22:14 +03:00
- `md2txt` – converts Markdown into plain text with extensive formatting support. It ships with the default `markdown` parser and `text` renderer plugins, registers optional `micron`, `ama`, `gemini`, and `wml` renderers for Micron/Ancient Machine Book, Gemtext, or WML (WAP) output, and exposes the core pipeline so you can add your own parser or renderer modules:
2025-10-20 17:48:57 +03:00
- FIGlet-rendered headings.
- Emphasis styles converted to spaced or delimited characters.
2025-10-19 22:58:10 +03:00
- Optional alignment and margin controls via HTML `<p>` attributes or MultiMarkdown attribute blocks (e.g. `{:.center margin=20px}`) applied as leading spaces.
2025-10-20 14:15:24 +03:00
- ASCII art injection with `#[label :align](art.txt)` syntax, supporting multiple art pieces per line and optional `{: .right}` style annotations.
- Per-document toggles for code block wrapping, numbering, blockquote decoration, and list indentation spacing.
2025-10-20 18:40:59 +03:00
- Link references can be emitted per block by setting `links_per_block: true` in frontmatter; otherwise they collect at the end of the document. Gemini output uses `=>` lines while the text renderer keeps numbered footnotes.
2025-10-19 22:58:10 +03:00
## Usage
```bash
2025-10-20 17:48:57 +03:00
md2txt input.md -o output.txt # convert to DOS-friendly text
md2txt input.md # write result to stdout
md2txt input.md --width 72 # override column width
2025-10-20 22:40:00 +03:00
md2txt input.md --renderer micron # emit Micron-formatted output for NomadNet
md2txt input.md --renderer ama # emit AMA markup for Ancient Machine Book
2025-10-20 18:40:59 +03:00
md2txt input.md --renderer gemini # emit Gemtext for Gemini capsules
Add WML renderer for WAP decks Adds a `wml` renderer plugin converting Markdown into WML decks for old WAP browsers, following the existing parser -> BlockEvent stream -> renderer plugin architecture (see gemini/micron/ama renderers). WML content is idiomatically short and button/link-navigated rather than one long page, so unlike the other renderers, `wml` can split a document into multiple <card>s -- but only opt-in: - Manual `{.card}` / `{.card Title}` divider directives (new, format-agnostic BlockKind.CARD_BREAK; silently ignored by every other renderer via their existing unhandled-kind no-op path). - Heading-level splitting via `--renderer-option split_level=N`, ignored when manual dividers are present. - Plain documents with neither stay a single card (the safe default). When a document resolves to 2+ cards, a hub-and-spoke menu card is built by default (button/link choices -> section, Back to menu), or `menu=false` switches to a linear Next/Back chain. A byte-overflow safety net (`max_card_bytes`, default 1400) paginates any single section that's still too large. Also adds GFM pipe-table support (BlockKind.TABLE) via the same sentinel-injection preprocessing trick already used for ASCII art blocks, so the parser's line-by-line loop needed no lookahead rework. Rendered as `<table>` in wml, silently ignored elsewhere. No new dependencies. No devbox/jj/test-suite additions -- decided against for this repo (see README for the full option reference). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-23 08:22:14 +03:00
md2txt input.md --renderer wml # emit a single-card WML deck for WAP browsers
2025-10-20 22:40:00 +03:00
md2txt input.md --renderer-option width=68 # pass column width to a renderer
2025-10-19 22:58:10 +03:00
```
2025-10-20 17:48:57 +03:00
- `--parser` and `--renderer` select a plugin by name (defaults are `markdown` and `text`).
- Repeatable `--parser-option KEY=VALUE` and `--renderer-option KEY=VALUE` pairs are forwarded to the plugin factories as keyword arguments in addition to the defaults supplied by the CLI.
- The CLI accepts `--help` for the full option list.
2025-10-19 22:58:10 +03:00
2025-10-20 17:48:57 +03:00
## Rendering Controls via Frontmatter
2025-10-19 22:58:10 +03:00
2025-10-20 17:48:57 +03:00
Fine-tune formatting by adding keys to frontmatter:
2025-10-19 22:58:10 +03:00
2025-10-20 17:48:57 +03:00
```yaml
2025-10-19 22:58:10 +03:00
---
h1_font: slant
h2_font: standard
h3_font: small
2025-10-19 23:22:15 +03:00
margin_left: 4
margin_right: 4
hyphenate: true
hyphen_lang: en_US
2025-10-19 22:58:10 +03:00
---
```
2025-10-20 17:48:57 +03:00
| Key | Description |
| --- | --- |
| `blockquote_bars: false` | Replace the default `|` prefix with three spaces. |
| `code_block_line_numbers: false` | Disable the `01 |` gutter in wrapped or unwrapped code blocks. |
| `code_block_wrap: <bool or int>` | Enables wrapping; when set to an integer it also controls the extra indentation applied to wrapped continuation lines. |
| `figlet_fallback: true` | force an H4-style fallback when a FIGlet banner would overflow; when omitted (the default), the banner is soft wrapping to honor width and margins. |
| `h#_font: <font>` | Accepts any known figlet font name or one of the keywords `caps` (force uppercase) and `title` (title case) to define headings.
| `hyphenate: true` | activates PyHyphen-based wrapping (optional `hyphen_lang` defaults to `en_US`) |
| `header_spacing` | controls how many blank lines precede headings (default `2`). |
2025-10-20 18:40:59 +03:00
| `links_per_block: true` | Emit link references directly below each block instead of collecting them at the end (supported by the text and Gemini renderers). |
2025-10-20 17:48:57 +03:00
| `list_marker_indent: <int>` | Extra spaces inserted before list markers. |
| `list_text_spacing: <int>` | Spaces between the marker and the wrapped list text. |
| `margin_left`/`margin_right` | add leading/trailing spaces to body text |
| `paragraph_spacing` | inserts blank lines between paragraphs |
| `wrap_code_blocks: true` | Wrap fenced/indented code blocks to fit the page width instead of using a gutter. |
All values are optional—defaults maintain the legacy behaviour.
2025-10-19 22:58:10 +03:00
## Styling via Attribute Blocks
You can influence alignment and margins in the source Markdown using either HTML paragraphs or MultiMarkdown attribute blocks:
```markdown
<p align="center" style="margin: 10px 20px;">
Centered paragraph with custom margins.
</p>
Paragraph styled via MMD attributes.
{: .right margin=12px }
## Centered Heading {: .center margin=8px }
```
Margins are interpreted as spaces; values using `px` are rounded to the nearest integer, and `margin: 0 auto;` centers content.
2025-10-20 14:15:24 +03:00
## File Includes
Two lightweight include syntaxes are recognised during preprocessing:
- `![[relative/path.md]]`
- `{.include relative/path.md}`
The paths are resolved relative to the file that contains the directive. Any YAML frontmatter in the included file is stripped before inlining, and include cycles are detected and rejected.
## ASCII Art Blocks
Inline or block-level ASCII art can be embedded directly in Markdown:
```markdown
#[dragon :left](examples/dragon.txt)
#[dragon :right](examples/dragon.txt){:.center}
#[dragon_a :left](a.txt) #[dragon_b :center](b.txt) #[dragon_c :right](c.txt)
```
- `#[label](file)` loads the contents of the text file and inserts it as a preformatted block.
- Colon tags inside the label (`:left`, `:center`, `:right`) steer the alignment of each art piece. Additional colon tags are ignored.
- When multiple directives appear on the same line, the pieces are laid out side-by-side when space permits, otherwise they fall back to stacked blocks.
2025-10-19 22:58:10 +03:00
## Output Line Endings
Generated text uses CRLF line endings to maintain DOS compatibility.
2025-10-20 15:03:34 +03:00
Add WML renderer for WAP decks Adds a `wml` renderer plugin converting Markdown into WML decks for old WAP browsers, following the existing parser -> BlockEvent stream -> renderer plugin architecture (see gemini/micron/ama renderers). WML content is idiomatically short and button/link-navigated rather than one long page, so unlike the other renderers, `wml` can split a document into multiple <card>s -- but only opt-in: - Manual `{.card}` / `{.card Title}` divider directives (new, format-agnostic BlockKind.CARD_BREAK; silently ignored by every other renderer via their existing unhandled-kind no-op path). - Heading-level splitting via `--renderer-option split_level=N`, ignored when manual dividers are present. - Plain documents with neither stay a single card (the safe default). When a document resolves to 2+ cards, a hub-and-spoke menu card is built by default (button/link choices -> section, Back to menu), or `menu=false` switches to a linear Next/Back chain. A byte-overflow safety net (`max_card_bytes`, default 1400) paginates any single section that's still too large. Also adds GFM pipe-table support (BlockKind.TABLE) via the same sentinel-injection preprocessing trick already used for ASCII art blocks, so the parser's line-by-line loop needed no lookahead rework. Rendered as `<table>` in wml, silently ignored elsewhere. No new dependencies. No devbox/jj/test-suite additions -- decided against for this repo (see README for the full option reference). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-23 08:22:14 +03:00
## WML Decks
The `wml` renderer converts Markdown into a [WML](https://en.wikipedia.org/wiki/Wireless_Markup_Language) deck for old WAP browsers. WML screens are idiomatically short and button/link-navigated rather than one long scrolling page, so unlike the other renderers, `wml` can split a document into multiple `<card>`s — but only when you ask it to; a plain document with no options stays a single card.
Two independent, both-optional ways to define card boundaries:
- **Manual dividers** — insert a `{.card}` or `{.card Title}` line wherever you want a new card to start, regardless of heading structure:
```markdown
Intro shown on the menu card...
{.card Weather}
Cold and clear...
{.card Trail Status}
- North Loop: open
```
- **Heading-level splitting** — opt in with `--renderer-option split_level=N` to start a new card at every heading of level `N` (e.g. `split_level=2` for the common one-H1-title-plus-H2-sections pattern). Ignored if the document contains any `{.card}` markers.
When a document resolves to two or more cards, `wml` builds a "menu" card by default — the classic WAP-portal pattern: a list of button/link choices leading to each section, each with a Back control (`--renderer-option menu=false` switches to a linear book instead, chaining cards with Next/Back). A section whose content would still be too large for one card even after splitting is automatically paginated further (`--renderer-option max_card_bytes=N`, default 1400 — a nod to classic WAP 1.x per-deck cache limits; `0` disables it).
| Renderer option | Default | Meaning |
| --- | --- | --- |
| `title` | *(none)* | Overall deck title, used on the menu card (or the single card, if never split). |
| `split_level` | `0` (disabled) | Heading level that starts a new card. Ignored if `{.card}` markers are present. |
| `max_card_bytes` | `1400` | Byte safety-net per card; `0` disables. |
| `menu` | `true` | Hub-and-spoke menu card vs. linear Next/Back chaining (`menu=false`). |
| `nav_next_label` | `More` | Label for forward navigation (linear mode / overflow pagination). |
| `nav_prev_label` | `Prev` | Label for backward navigation between overflow pagination parts. |
| `nav_back_label` | `Back` | Label for backward navigation from a section back to the menu/previous section. |
WML 1.3 is the only supported DTD (fixed by design, not a flag). Since WML uses `$` for variable substitution, a literal `$` in your Markdown is escaped to `$$` in the output automatically. Images pass through as `<img src="..." alt="..."/>` referencing the original path/URL — no WBMP conversion is performed.
2025-10-20 15:03:34 +03:00
## Plugin Architecture
2025-10-20 17:26:38 +03:00
The conversion pipeline lives in `src/md2txt/conversion/core.py` and is exposed via `run_conversion`. It is designed around lightweight factories:
2025-10-20 15:03:34 +03:00
- **Parser factories** receive `base_style: BlockStyle` plus any extra keyword arguments and must return an object with a `parse(lines: Iterable[str]) -> Iterator[BlockEvent | StyleUpdateEvent]` method. The bundled `MarkdownParser` implements this interface.
- **Renderer factories** receive `frontmatter: FrontMatter` and arbitrary keyword arguments and must return an object that provides `handle_event(event)` and `finalize() -> Any`. The `TextRenderer` returns a list of DOS-friendly output lines; other renderers may return any data appropriate for their target format.
2025-10-20 17:26:38 +03:00
Plugins register themselves through the helpers in `md2txt.plugins`:
2025-10-20 15:03:34 +03:00
```python
2025-10-20 17:26:38 +03:00
from md2txt.plugins import register_parser, register_renderer
2025-10-20 15:03:34 +03:00
def my_parser_factory(*, base_style, **options):
return MyParser(base_style, **options)
register_parser("my-markdown", my_parser_factory)
```
```python
def my_renderer_factory(*, frontmatter, **options):
return MyRenderer(frontmatter, target=options.get("target"))
register_renderer("ansi", my_renderer_factory)
```
2025-10-20 17:26:38 +03:00
Once registered (for example in a small module that imports `md2txt.cli`), the new plugins are available via `--parser my-markdown` or `--renderer ansi`. The CLI lists registered plugin names in `--help`, and the helper functions `available_parsers()` / `available_renderers()` return the sorted names if you need to build higher-level tooling.
2025-10-20 15:03:34 +03:00
Add WML renderer for WAP decks Adds a `wml` renderer plugin converting Markdown into WML decks for old WAP browsers, following the existing parser -> BlockEvent stream -> renderer plugin architecture (see gemini/micron/ama renderers). WML content is idiomatically short and button/link-navigated rather than one long page, so unlike the other renderers, `wml` can split a document into multiple <card>s -- but only opt-in: - Manual `{.card}` / `{.card Title}` divider directives (new, format-agnostic BlockKind.CARD_BREAK; silently ignored by every other renderer via their existing unhandled-kind no-op path). - Heading-level splitting via `--renderer-option split_level=N`, ignored when manual dividers are present. - Plain documents with neither stay a single card (the safe default). When a document resolves to 2+ cards, a hub-and-spoke menu card is built by default (button/link choices -> section, Back to menu), or `menu=false` switches to a linear Next/Back chain. A byte-overflow safety net (`max_card_bytes`, default 1400) paginates any single section that's still too large. Also adds GFM pipe-table support (BlockKind.TABLE) via the same sentinel-injection preprocessing trick already used for ASCII art blocks, so the parser's line-by-line loop needed no lookahead rework. Rendered as `<table>` in wml, silently ignored elsewhere. No new dependencies. No devbox/jj/test-suite additions -- decided against for this repo (see README for the full option reference). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-23 08:22:14 +03:00
The shared preprocessing helpers—YAML frontmatter parsing, recursive include expansion, ASCII art sentinels, and pipe-table sentinels—also live in `src/md2txt/conversion/core.py`, allowing alternate front-ends to reuse exactly the same behaviour without duplicating code.
2025-10-20 22:18:46 +03:00
---
* Directed and vibe coded by Tobias Raayoni Last
2025-10-20 22:19:58 +03:00
* Programmed by `gpt-5-codex`