md2txt/README.md

171 lines
9.9 KiB
Markdown
Raw Normal View History

2025-10-20 17:48:57 +03:00
```
                    $$\  $$$$$$\    $$\                 $$\     
                    $$ |$$  __$$\   $$ |                $$ |    
$$$$$$\$$$$\   $$$$$$$ |\__/  $$ |$$$$$$\   $$\   $$\ $$$$$$\   
$$  _$$  _$$\ $$  __$$ | $$$$$$  |\_$$  _|  \$$\ $$  |\_$$  _|  
$$ / $$ / $$ |$$ /  $$ |$$  ____/   $$ |     \$$$$  /   $$ |    
$$ | $$ | $$ |$$ |  $$ |$$ |        $$ |$$\  $$  $$<    $$ |$$\ 
$$ | $$ | $$ |\$$$$$$$ |$$$$$$$$\   \$$$$  |$$  /\$$\   \$$$$  |
\__| \__| \__| \_______|\________|   \____/ \__/  \__|   \____/ 
```
2025-10-19 22:58:10 +03:00
2025-10-20 22:04:59 +03:00
This repository contains the `md2txt` command line tool and supporting libraries for transforming Markdown into arcane formats that work well on retro hardware or constrained text viewers. A small plugin API lets you mix-and-match parsers and renderers so additional formats can plug into the same preprocessing pipeline.
2025-10-19 22:58:10 +03:00
2025-10-20 22:19:58 +03:00
Install the CLI directly from GitHub with [uv](https://github.com/astral-sh/uv):
```
uv tool install --from git+https://code.randogoth.com/randogoth/md2txt/ md2txt
2025-10-20 22:19:58 +03:00
```
2025-10-20 17:26:38 +03:00
## CLI
2025-10-19 22:58:10 +03:00
- `md2txt` – converts Markdown into plain text with extensive formatting support. It ships with the default `markdown` parser and `text` renderer plugins, registers optional `micron`, `ama`, `gemini`, and `nex` renderers for Micron/Ancient Machine Book, Gemtext, or Nex output, and exposes the core pipeline so you can add your own parser or renderer modules:
2025-10-20 17:48:57 +03:00
- FIGlet-rendered headings.
- Emphasis styles converted to spaced or delimited characters.
2025-10-19 22:58:10 +03:00
- Optional alignment and margin controls via HTML `<p>` attributes or MultiMarkdown attribute blocks (e.g. `{:.center margin=20px}`) applied as leading spaces.
2025-10-20 14:15:24 +03:00
- ASCII art injection with `#[label :align](art.txt)` syntax, supporting multiple art pieces per line and optional `{: .right}` style annotations.
- Per-document toggles for code block wrapping, numbering, blockquote decoration, and list indentation spacing.
2025-10-20 18:40:59 +03:00
- Link references can be emitted per block by setting `links_per_block: true` in frontmatter; otherwise they collect at the end of the document. Gemini output uses `=>` lines while the text renderer keeps numbered footnotes.
2025-10-19 22:58:10 +03:00
## Usage
```bash
2025-10-20 17:48:57 +03:00
md2txt input.md -o output.txt # convert to DOS-friendly text
md2txt input.md # write result to stdout
md2txt input.md --width 72 # override column width
2025-10-20 22:40:00 +03:00
md2txt input.md --renderer micron # emit Micron-formatted output for NomadNet
md2txt input.md --renderer ama # emit AMA markup for Ancient Machine Book
2025-10-20 18:40:59 +03:00
md2txt input.md --renderer gemini # emit Gemtext for Gemini capsules
md2txt input.md --renderer nex # emit plain text for Nex capsules
2025-10-20 22:40:00 +03:00
md2txt input.md --renderer-option width=68 # pass column width to a renderer
2025-10-19 22:58:10 +03:00
```
2025-10-20 17:48:57 +03:00
- `--parser` and `--renderer` select a plugin by name (defaults are `markdown` and `text`).
- Repeatable `--parser-option KEY=VALUE` and `--renderer-option KEY=VALUE` pairs are forwarded to the plugin factories as keyword arguments in addition to the defaults supplied by the CLI.
- The CLI accepts `--help` for the full option list.
2025-10-19 22:58:10 +03:00
2025-10-20 17:48:57 +03:00
## Rendering Controls via Frontmatter
2025-10-19 22:58:10 +03:00
2025-10-20 17:48:57 +03:00
Fine-tune formatting by adding keys to frontmatter:
2025-10-19 22:58:10 +03:00
2025-10-20 17:48:57 +03:00
```yaml
2025-10-19 22:58:10 +03:00
---
h1_font: slant
h2_font: standard
h3_font: small
2025-10-19 23:22:15 +03:00
margin_left: 4
margin_right: 4
hyphenate: true
hyphen_lang: en_US
2025-10-19 22:58:10 +03:00
---
```
2025-10-20 17:48:57 +03:00
| Key | Description |
| --- | --- |
| `blockquote_bars: false` | Replace the default `|` prefix with three spaces. |
| `code_block_line_numbers: false` | Disable the `01 |` gutter in wrapped or unwrapped code blocks. |
| `code_block_wrap: <bool or int>` | Enables wrapping; when set to an integer it also controls the extra indentation applied to wrapped continuation lines. |
| `figlet_fallback: true` | force an H4-style fallback when a FIGlet banner would overflow; when omitted (the default), the banner is soft wrapping to honor width and margins. |
| `h#_font: <font>` | Accepts any known figlet font name or one of the keywords `caps` (force uppercase) and `title` (title case) to define headings.
| `hyphenate: true` | activates PyHyphen-based wrapping (optional `hyphen_lang` defaults to `en_US`) |
| `header_spacing` | controls how many blank lines precede headings (default `2`). |
2025-10-20 18:40:59 +03:00
| `links_per_block: true` | Emit link references directly below each block instead of collecting them at the end (supported by the text and Gemini renderers). |
2025-10-20 17:48:57 +03:00
| `list_marker_indent: <int>` | Extra spaces inserted before list markers. |
| `list_text_spacing: <int>` | Spaces between the marker and the wrapped list text. |
| `margin_left`/`margin_right` | add leading/trailing spaces to body text |
| `paragraph_spacing` | inserts blank lines between paragraphs |
| `wrap_code_blocks: true` | Wrap fenced/indented code blocks to fit the page width instead of using a gutter. |
All values are optional—defaults maintain the legacy behaviour.
2025-10-19 22:58:10 +03:00
## Styling via Attribute Blocks
You can influence alignment and margins in the source Markdown using either HTML paragraphs or MultiMarkdown attribute blocks:
```markdown
<p align="center" style="margin: 10px 20px;">
Centered paragraph with custom margins.
</p>
Paragraph styled via MMD attributes.
{: .right margin=12px }
## Centered Heading {: .center margin=8px }
```
Margins are interpreted as spaces; values using `px` are rounded to the nearest integer, and `margin: 0 auto;` centers content.
2025-10-20 14:15:24 +03:00
## File Includes
Two lightweight include syntaxes are recognised during preprocessing:
- `![[relative/path.md]]`
- `{.include relative/path.md}`
The paths are resolved relative to the file that contains the directive. Any YAML frontmatter in the included file is stripped before inlining, and include cycles are detected and rejected.
## ASCII Art Blocks
Inline or block-level ASCII art can be embedded directly in Markdown:
```markdown
#[dragon :left](examples/dragon.txt)
#[dragon :right](examples/dragon.txt){:.center}
#[dragon_a :left](a.txt) #[dragon_b :center](b.txt) #[dragon_c :right](c.txt)
```
- `#[label](file)` loads the contents of the text file and inserts it as a preformatted block.
- Colon tags inside the label (`:left`, `:center`, `:right`) steer the alignment of each art piece. Additional colon tags are ignored.
- When multiple directives appear on the same line, the pieces are laid out side-by-side when space permits, otherwise they fall back to stacked blocks.
2025-10-19 22:58:10 +03:00
## Output Line Endings
Generated text uses CRLF line endings to maintain DOS compatibility.
2025-10-20 15:03:34 +03:00
## Nex
Add WML renderer for WAP decks Adds a `wml` renderer plugin converting Markdown into WML decks for old WAP browsers, following the existing parser -> BlockEvent stream -> renderer plugin architecture (see gemini/micron/ama renderers). WML content is idiomatically short and button/link-navigated rather than one long page, so unlike the other renderers, `wml` can split a document into multiple <card>s -- but only opt-in: - Manual `{.card}` / `{.card Title}` divider directives (new, format-agnostic BlockKind.CARD_BREAK; silently ignored by every other renderer via their existing unhandled-kind no-op path). - Heading-level splitting via `--renderer-option split_level=N`, ignored when manual dividers are present. - Plain documents with neither stay a single card (the safe default). When a document resolves to 2+ cards, a hub-and-spoke menu card is built by default (button/link choices -> section, Back to menu), or `menu=false` switches to a linear Next/Back chain. A byte-overflow safety net (`max_card_bytes`, default 1400) paginates any single section that's still too large. Also adds GFM pipe-table support (BlockKind.TABLE) via the same sentinel-injection preprocessing trick already used for ASCII art blocks, so the parser's line-by-line loop needed no lookahead rework. Rendered as `<table>` in wml, silently ignored elsewhere. No new dependencies. No devbox/jj/test-suite additions -- decided against for this repo (see README for the full option reference). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-23 08:22:14 +03:00
The `nex` renderer converts Markdown into plain text for the [Nex](https://nightfall.city/nex/) protocol. Nex has no hypertext markup for hand-authored documents at all — its one special syntax, `=> URL`, is spec'd for server-auto-generated directory listings, not document bodies — so `nex` output is deliberately unmarked plain text, reusing `TextRenderer`'s wrapping/margin/hyphenation machinery (same approach as the `ama`/`micron` renderers):
Add WML renderer for WAP decks Adds a `wml` renderer plugin converting Markdown into WML decks for old WAP browsers, following the existing parser -> BlockEvent stream -> renderer plugin architecture (see gemini/micron/ama renderers). WML content is idiomatically short and button/link-navigated rather than one long page, so unlike the other renderers, `wml` can split a document into multiple <card>s -- but only opt-in: - Manual `{.card}` / `{.card Title}` divider directives (new, format-agnostic BlockKind.CARD_BREAK; silently ignored by every other renderer via their existing unhandled-kind no-op path). - Heading-level splitting via `--renderer-option split_level=N`, ignored when manual dividers are present. - Plain documents with neither stay a single card (the safe default). When a document resolves to 2+ cards, a hub-and-spoke menu card is built by default (button/link choices -> section, Back to menu), or `menu=false` switches to a linear Next/Back chain. A byte-overflow safety net (`max_card_bytes`, default 1400) paginates any single section that's still too large. Also adds GFM pipe-table support (BlockKind.TABLE) via the same sentinel-injection preprocessing trick already used for ASCII art blocks, so the parser's line-by-line loop needed no lookahead rework. Rendered as `<table>` in wml, silently ignored elsewhere. No new dependencies. No devbox/jj/test-suite additions -- decided against for this repo (see README for the full option reference). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-23 08:22:14 +03:00
- Headings get a plain setext-style underline (`=` for H1, `-` for H2+) instead of a FIGlet banner. This is a readability convention borrowed from Markdown itself, not a requirement of the Nex spec.
- `**bold**`/`*italic*`/`~~strikethrough~~` markers are stripped rather than translated — there's nothing in Nex to translate them into.
- Links and images are inlined as `label (url)` plain prose (or bare `url` when there's no distinct label/alt, or when the label equals the url) rather than footnoted or turned into `=> ` lines, since that syntax isn't confirmed safe inside a regular document body.
Add WML renderer for WAP decks Adds a `wml` renderer plugin converting Markdown into WML decks for old WAP browsers, following the existing parser -> BlockEvent stream -> renderer plugin architecture (see gemini/micron/ama renderers). WML content is idiomatically short and button/link-navigated rather than one long page, so unlike the other renderers, `wml` can split a document into multiple <card>s -- but only opt-in: - Manual `{.card}` / `{.card Title}` divider directives (new, format-agnostic BlockKind.CARD_BREAK; silently ignored by every other renderer via their existing unhandled-kind no-op path). - Heading-level splitting via `--renderer-option split_level=N`, ignored when manual dividers are present. - Plain documents with neither stay a single card (the safe default). When a document resolves to 2+ cards, a hub-and-spoke menu card is built by default (button/link choices -> section, Back to menu), or `menu=false` switches to a linear Next/Back chain. A byte-overflow safety net (`max_card_bytes`, default 1400) paginates any single section that's still too large. Also adds GFM pipe-table support (BlockKind.TABLE) via the same sentinel-injection preprocessing trick already used for ASCII art blocks, so the parser's line-by-line loop needed no lookahead rework. Rendered as `<table>` in wml, silently ignored elsewhere. No new dependencies. No devbox/jj/test-suite additions -- decided against for this repo (see README for the full option reference). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-23 08:22:14 +03:00
2025-10-20 15:03:34 +03:00
## Plugin Architecture
2025-10-20 17:26:38 +03:00
The conversion pipeline lives in `src/md2txt/conversion/core.py` and is exposed via `run_conversion`. It is designed around lightweight factories:
2025-10-20 15:03:34 +03:00
- **Parser factories** receive `base_style: BlockStyle` plus any extra keyword arguments and must return an object with a `parse(lines: Iterable[str]) -> Iterator[BlockEvent | StyleUpdateEvent]` method. The bundled `MarkdownParser` implements this interface.
- **Renderer factories** receive `frontmatter: FrontMatter` and arbitrary keyword arguments and must return an object that provides `handle_event(event)` and `finalize() -> Any`. The `TextRenderer` returns a list of DOS-friendly output lines; other renderers may return any data appropriate for their target format.
2025-10-20 17:26:38 +03:00
Plugins register themselves through the helpers in `md2txt.plugins`:
2025-10-20 15:03:34 +03:00
```python
2025-10-20 17:26:38 +03:00
from md2txt.plugins import register_parser, register_renderer
2025-10-20 15:03:34 +03:00
def my_parser_factory(*, base_style, **options):
return MyParser(base_style, **options)
register_parser("my-markdown", my_parser_factory)
```
```python
def my_renderer_factory(*, frontmatter, **options):
return MyRenderer(frontmatter, target=options.get("target"))
register_renderer("ansi", my_renderer_factory)
```
2025-10-20 17:26:38 +03:00
Once registered (for example in a small module that imports `md2txt.cli`), the new plugins are available via `--parser my-markdown` or `--renderer ansi`. The CLI lists registered plugin names in `--help`, and the helper functions `available_parsers()` / `available_renderers()` return the sorted names if you need to build higher-level tooling.
2025-10-20 15:03:34 +03:00
Add WML renderer for WAP decks Adds a `wml` renderer plugin converting Markdown into WML decks for old WAP browsers, following the existing parser -> BlockEvent stream -> renderer plugin architecture (see gemini/micron/ama renderers). WML content is idiomatically short and button/link-navigated rather than one long page, so unlike the other renderers, `wml` can split a document into multiple <card>s -- but only opt-in: - Manual `{.card}` / `{.card Title}` divider directives (new, format-agnostic BlockKind.CARD_BREAK; silently ignored by every other renderer via their existing unhandled-kind no-op path). - Heading-level splitting via `--renderer-option split_level=N`, ignored when manual dividers are present. - Plain documents with neither stay a single card (the safe default). When a document resolves to 2+ cards, a hub-and-spoke menu card is built by default (button/link choices -> section, Back to menu), or `menu=false` switches to a linear Next/Back chain. A byte-overflow safety net (`max_card_bytes`, default 1400) paginates any single section that's still too large. Also adds GFM pipe-table support (BlockKind.TABLE) via the same sentinel-injection preprocessing trick already used for ASCII art blocks, so the parser's line-by-line loop needed no lookahead rework. Rendered as `<table>` in wml, silently ignored elsewhere. No new dependencies. No devbox/jj/test-suite additions -- decided against for this repo (see README for the full option reference). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-23 08:22:14 +03:00
The shared preprocessing helpers—YAML frontmatter parsing, recursive include expansion, ASCII art sentinels, and pipe-table sentinels—also live in `src/md2txt/conversion/core.py`, allowing alternate front-ends to reuse exactly the same behaviour without duplicating code.
2025-10-20 22:18:46 +03:00
## WML / WAP Output
The `wml` renderer used to live here. It has moved to [wapdown](https://code.randogoth.com/randogoth/wapdown), because a WML deck is not a minimal text format: it is an XML tree, and producing one means restructuring a document into navigable cards rather than transforming it. That needed controls the exporters here have no use for, so it now has a repository of its own. The `{.card}` divider moved with it.
2025-10-20 22:18:46 +03:00
---
* Directed and vibe coded by Tobias Raayoni Last
2025-10-20 22:19:58 +03:00
* Programmed by `gpt-5-codex`