diff --git a/README.md b/README.md index 48e1c3f..483a03c 100644 --- a/README.md +++ b/README.md @@ -1,11 +1,10 @@ -# md2amb utilities +# md2txt -This repository contains command line helpers for transforming Markdown into formats that work well on retro hardware or constrained text viewers. A small plugin API lets you mix-and-match parsers and renderers so additional formats can plug into the same preprocessing pipeline. +This repository contains the `md2txt` command line tool and supporting libraries for transforming Markdown into formats that work well on retro hardware or constrained text viewers. A small plugin API lets you mix-and-match parsers and renderers so additional formats can plug into the same preprocessing pipeline. -## Tools +## CLI -- `md2amb.py` – converts Markdown into Amber-screen formatted text (see script for details). -- `md2txt.py` – converts Markdown into 80-column, DOS-compatible plain text with extensive formatting support. It ships with the default `markdown` parser and `text` renderer plugins, registers optional `micron` and `ama` renderers for Micron/Ancient Machine Book output, and exposes the core pipeline so you can add your own parser or renderer modules: +- `md2txt` – converts Markdown into 80-column, DOS-compatible plain text with extensive formatting support. It ships with the default `markdown` parser and `text` renderer plugins, registers optional `micron` and `ama` renderers for Micron/Ancient Machine Book output, and exposes the core pipeline so you can add your own parser or renderer modules: - FIGlet-rendered headings (H1–H3) driven by optional YAML frontmatter (`h1_font`, `h2_font`, `h3_font`). - H4+ headings rendered in uppercase with dashed underlines. - Emphasis styles converted to spaced or delimited characters, e.g. `**bold**` → `B O L D`, `__strong__` → `_s_t_r_o_n_g_`, `~~strike~~` → `~s~t~r~i~k~e~`. @@ -13,7 +12,6 @@ This repository contains command line helpers for transforming Markdown into for - Code blocks numbered (`01 | line`), with both fenced and indented fences supported (see “Rendering Controls” for customisation). - Links transformed into `[label](n)` references, with footnote-style URL list at the end. - Optional alignment and margin controls via HTML `
` attributes or MultiMarkdown attribute blocks (e.g. `{:.center margin=20px}`) applied as leading spaces.
- - Recursive file includes using `![[file.md]]` or `{.include file.md}` (frontmatter inside the included file is ignored).
- ASCII art injection with `#[label :align](art.txt)` syntax, supporting multiple art pieces per line and optional `{: .right}` style annotations.
- Per-document toggles for code block wrapping, numbering, blockquote decoration, and list indentation spacing.
@@ -26,21 +24,18 @@ This repository contains command line helpers for transforming Markdown into for
## Usage
```bash
-python md2txt.py input.md -o output.txt # convert to DOS-friendly text
-python md2txt.py input.md # write result to stdout
-python md2txt.py input.md --width 72 # override column width
-python md2txt.py input.md --parser markdown --renderer micron # emit Micron-formatted output
-python md2txt.py input.md --renderer ama # emit AMB/AMA markup
-python md2txt.py input.md --renderer-option width=68 # pass KEY=VALUE to a renderer
+md2txt input.md -o output.txt # convert to DOS-friendly text
+md2txt input.md # write result to stdout
+md2txt input.md --width 72 # override column width
+md2txt input.md --parser markdown --renderer micron # emit Micron-formatted output
+md2txt input.md --renderer ama # emit AMB/AMA markup
+md2txt input.md --renderer-option width=68 # pass KEY=VALUE to a renderer
+
+# If the project is not installed yet:
+python -m md2txt input.md
```
-- `md2amb.py` – package Markdown (and linked Markdown files) into a self-contained `.amb` archive composed of `.ama` articles that honour the 78-column/64 KiB AMA constraints.
-
-```bash
-python md2amb.py --title "Your Manual" docs/index.md output/manual.amb
-```
-
-`--parser` and `--renderer` select a plugin by name (defaults are `markdown` and `text`). Repeatable `--parser-option KEY=VALUE` and `--renderer-option KEY=VALUE` pairs are forwarded to the plugin factories as keyword arguments in addition to the defaults supplied by the CLI. Both scripts accept `--help` for the full option list.
+`--parser` and `--renderer` select a plugin by name (defaults are `markdown` and `text`). Repeatable `--parser-option KEY=VALUE` and `--renderer-option KEY=VALUE` pairs are forwarded to the plugin factories as keyword arguments in addition to the defaults supplied by the CLI. The CLI accepts `--help` for the full option list.
## FIGlet Fonts via Frontmatter
@@ -126,15 +121,15 @@ Generated text uses CRLF line endings to maintain DOS compatibility.
## Plugin Architecture
-The conversion pipeline lives in `conversion_core.py` and is exposed via `run_conversion`. It is designed around lightweight factories:
+The conversion pipeline lives in `src/md2txt/conversion/core.py` and is exposed via `run_conversion`. It is designed around lightweight factories:
- **Parser factories** receive `base_style: BlockStyle` plus any extra keyword arguments and must return an object with a `parse(lines: Iterable[str]) -> Iterator[BlockEvent | StyleUpdateEvent]` method. The bundled `MarkdownParser` implements this interface.
- **Renderer factories** receive `frontmatter: FrontMatter` and arbitrary keyword arguments and must return an object that provides `handle_event(event)` and `finalize() -> Any`. The `TextRenderer` returns a list of DOS-friendly output lines; other renderers may return any data appropriate for their target format.
-Plugins register themselves through the helpers in `plugins.py`:
+Plugins register themselves through the helpers in `md2txt.plugins`:
```python
-from plugins import register_parser, register_renderer
+from md2txt.plugins import register_parser, register_renderer
def my_parser_factory(*, base_style, **options):
return MyParser(base_style, **options)
@@ -149,6 +144,6 @@ def my_renderer_factory(*, frontmatter, **options):
register_renderer("ansi", my_renderer_factory)
```
-Once registered (for example in a small module that imports `md2txt.py`), the new plugins are available via `--parser my-markdown` or `--renderer ansi`. The CLI lists registered plugin names in `--help`, and the helper functions `available_parsers()` / `available_renderers()` return the sorted names if you need to build higher-level tooling.
+Once registered (for example in a small module that imports `md2txt.cli`), the new plugins are available via `--parser my-markdown` or `--renderer ansi`. The CLI lists registered plugin names in `--help`, and the helper functions `available_parsers()` / `available_renderers()` return the sorted names if you need to build higher-level tooling.
-The shared preprocessing helpers—YAML frontmatter parsing, recursive include expansion, and ASCII art sentinels—also live in `conversion_core.py`, allowing alternate front-ends to reuse exactly the same behaviour without duplicating code.
+The shared preprocessing helpers—YAML frontmatter parsing, recursive include expansion, and ASCII art sentinels—also live in `src/md2txt/conversion/core.py`, allowing alternate front-ends to reuse exactly the same behaviour without duplicating code.
diff --git a/lorem.amb b/lorem.amb
deleted file mode 100644
index 4643777..0000000
Binary files a/lorem.amb and /dev/null differ
diff --git a/loremipsum.mu b/loremipsum.mu
deleted file mode 100644
index 6c8c720..0000000
--- a/loremipsum.mu
+++ /dev/null
@@ -1,237 +0,0 @@
-
- > Salientia
-
- >> Structa hinc vetitus factum
-
- Lorem markdownum omnes tellus, in, Herculeis
- `=reciprocalPower`= neque promissa est Latiae caput
- umero talia fidissima, nymphae. Foedera et sagittas
- tenetur adde matri esse: talia manibus.
-
- >> Fuit video longique dedisse
-
- Ille stupet, ultra `!incunabula agendum urbes`! mirer
- `!aper`!, metu. Aegides novissima sunt turis quatiebant
- umbrosaque `[putares`#mille-parva]. Metuunt eruiturque
- maximus velit, ater Vesta vulnus rustica verba! Mei
- vincite nulla, in Iovem: fama exurunt vernum!
-
- De instat ubi induit fugientia inertes columbas tarda
- `*terga Parnasia sententia`* omnia inmotusque, est et
- aetas sonus. Exhibuit coeperunt, canis. Suis minora,
- tempore fateor neve moderamine erit. Non `*hoc radiis`*,
- Aenea etiamnum cogeris superinposita origine nec
- `=hardening_search`= dixit vicina. Heros subito
- populifer adeunt quicquam cum `!ales`! Ixion angulus
- placet reddere.
-
- >> Tractaque serpens arva quoque
-
- > Include
-
- <>=======()
- (/\___ /|\\ ()==========<>_
- \_/ | \\ //|\ ______/ \)
- \_| \\ // | \_/
- \|\/|\_ // /\/
- (oo)\ \_// /
- //_/\_\/ / |
- @@/ |=\ \ |
- \_=\_ \ |
- \==\ \|\_ snd
- __(\===\( )\
- (((~) __(_/ |
- (((~) \ /
- ______/ /
- '------'
- <>=======()
- (/\___ /|\\ ()==========<>_
- \_/ | \\ //|\ ______/ \)
- \_| \\ // | \_/
- \|\/|\_ // /\/
- (oo)\ \_// /
- //_/\_\/ / |
- @@/ |=\ \ |
- \_=\_ \ |
- \==\ \|\_ snd
- __(\===\( )\
- (((~) __(_/ |
- (((~) \ /
- ______/ /
- '------'
- <>=======()
- (/\___ /|\\ ()==========<>_
- \_/ | \\ //|\ ______/ \)
- \_| \\ // | \_/
- \|\/|\_ // /\/
- (oo)\ \_// /
- //_/\_\/ / |
- @@/ |=\ \ |
- \_=\_ \ |
- \==\ \|\_ snd
- __(\===\( )\
- (((~) __(_/ |
- (((~) \ /
- ______/ /
- '------'
-
- > h1 Heading 8-)
-
- >> h2 Heading
-
- >>> h3 Heading
-
- >>> h4 Heading
-
- >>> h5 Heading
-
- >>> h6 Heading
-
- >> Horizontal Rules
-
- -
-
- -
-
- -
-
- >> Emphasis
-
- `!This is bold text`! `!This is bold text`! `*This is
- italic text`* `*This is italic text`* ~~Strikethrough~~
-
- >> Blockquotes
-
- >>>>Blockquotes can also be nested...
-
- >>>>>>>>by using additional greater-than signs right
- >>>>>>>>next to each other...
-
- >>>>>>>>>>>>or with spaces between arrows.
-
- >> Lists
-
- >>> Unordered
-
- + Create a list by starting a line with `=+`=, `=-`=, or
- `=*`=
- - Marker character change forces new list start:
- * Ac tristique libero volutpat at
- + Facilisis in pretium nisl aliquet
- - Nulla volutpat aliquam velit
- + Very easy
- >>> Ordered
-
- 1. Lorem ipsum dolor sit amet
- 2. Consectetur adipiscing elit
- 3. Integer molestie lorem at massa
- >> Code
-
- Inline `=code`=
-
- Indented code
-
- `=
- // Some comments
- line 1 of code
- line 2 of code
- line 3 of code
- `=
-
- Block code "fences"
-
- `=
- Sample text here...
- `=
-
- Syntax highlighting
-
- `=
- var foo = function (bar) {
- return bar++;
- };
-
- console.log(foo(5));
- `=
-
- >> Links
-
- `[link text`http://dev.nodeca.com]
-
- `[link with title`http://nodeca.github.io/pica/demo/
- "title text!"]
-
- >> Images
-
- `[Minion`https://octodex.github.com/images/minion.png] `
- [Stormtroopocat`https://octodex.github.com/images/stormt
- roopocat.jpg "The Stormtroopocat"]
-
- Inprudens dum memor alma, casses dedi Sinuessa iam quid
- adicit `=gate_truncate_wave`=. `[Geratur de`#salientia-
- iungit-contra-et], insolida temptat `=open`= carentia;
- genitorem solet, potens. Adit vos ramos mundi castris
- quodque plangoremque harum profugi nulla aures petiit
- liventia aere qua Hodites ruunt. Achaidos dilexisse
- `=redundancy`= praesagia tenuatus necis `!Capys
- recordor`! tenebras actis signis, reliquit, placidissime
- iram liquefactis longe nervi famem. Cura pars natique
- velleris semel et at `=printerWildcardParallel`= inquit
- positosque comites ultra.
-
- >> Quantaque augent
-
- Acuti ignotis ignis, ostendit nostras terras nondum,
- `[petit utque utero`#structa-hinc-vetitus-factum]
- sepulcrales, pluma reseratis Ismario. Tria silentia duae
- `*nymphe`*: certe parva cernam et dixit. `[Verborum
- sedisti`#mille-parva].
-
- `=
- if (firewall(4, 3) == 1 * device_media_parity) {
- system_copy_class += systray + oasis_wi_clean + lpi;
- modelLdap.search_secondary_socket *= bounce.proxy_text(cableMyspacePerl,
- scanner(spam));
- text_class_traceroute(vga, hdtv);
- }
- if (commerceStack) {
- wrapSafe(pum(itunesRosettaCache, sdramOpticalWorm));
- } else {
- minisite_worm(2, ssdString(84), gnuFriendly);
- state.qwerty(moduleJavascriptFloating + ugc, koffice,
- mail_system.prebinding_service_teraflops(mediaIcmpBoot));
- }
- memoryHexadecimalMultitasking += oop_android;
- var passive_vdu = bar;
- var ddrJava = leaf(unc_web);
- `=
-
- >>> Mille parva
-
- `!Minetur et`! aurum! Charaxi sanguine quo; vires
- subnectite `!pignora`!, omnique hostibus loquerentur
- aquae Solis ferox? `!Cum ne`!, putat utar ex ad aede
- mane. Vitalesque geminas exstimulat, iam vivunt dat
- artes et quem! Iam violas molire ipse, spoliabitur dente
- nervis regis, factos quae.
-
- >>>>Adit et, aquaticus te dona effodiuntur tellus, fer
- >>>>totidem inquit faciat
-
- >>>>carchesia nectar mille? Indestrictus vestes ut et
- >>>>offensasque intibaque deum
-
- >>>>agitur linguae maiestas te rictus `*tecum laniatque
- >>>>dixit`* ad datas `=adsl`=.
-
- >>>>Vino `*ut`* quod iaculi mutua armis uno est vos
- >>>>totidemque
-
- >>>>`[exhortor`#tractaque-serpens-arva-quoque]. Iamdudum
- >>>>Romanae cepi linguisque
-
- >>>>velit accipe idem acutis hoc sparsumque dextra.
- >>>>Minoide sponte collo sequitur
-
- >>>>et intus flere sanguineaque herba, nox Quas adsiduo
- >>>>`*cunctantem`* tibi; enim.
diff --git a/md2amb.py b/md2amb.py
deleted file mode 100644
index 67589f0..0000000
--- a/md2amb.py
+++ /dev/null
@@ -1,282 +0,0 @@
-#!/usr/bin/env python3
-from __future__ import annotations
-
-import argparse
-import re
-import struct
-from collections import deque
-from dataclasses import dataclass
-from pathlib import Path
-from typing import Dict, Iterable, List, Tuple
-
-import ama_renderer # noqa: F401 - ensure AMA renderer plugin registration
-from conversion_core import parse_frontmatter, run_conversion
-from markdown_parser import MarkdownParser
-from md_types import BlockStyle, FrontMatter
-from plugins import get_parser_factory, get_renderer_factory, register_parser
-from text_renderer import TextRenderer
-
-# Ensure markdown parser registered for standalone usage
-
-
-def _markdown_parser_factory(*, base_style: BlockStyle, **_: object) -> MarkdownParser:
- return MarkdownParser(base_style)
-
-
-try:
- register_parser("markdown", _markdown_parser_factory)
-except ValueError:
- pass
-
-
-MARKDOWN_LINK_RE = re.compile(r"(\[[^\]]*\]\()([^)]+)(\))")
-LOCAL_LINK_RE = re.compile(r"^[A-Za-z0-9_.~/\\-]+$")
-EXT_MD = {".md", ".markdown", ".mkd", ".mkdn"}
-AMA_MAX_BYTES = 65_535
-AMB_MAGIC = b"AMB1"
-LINK_CONTINUE_LABEL = "Continue"
-
-
-@dataclass
-class Article:
- source: Path
- ama_name: str
-
-
-def main(argv: Iterable[str] | None = None) -> int:
- parser = argparse.ArgumentParser(description="Convert Markdown into an AMB archive.")
- parser.add_argument("input", type=Path, help="Root Markdown file to convert.")
- parser.add_argument("output", type=Path, help="Output AMB filename.")
- parser.add_argument("--title", type=str, help="Optional book title.")
- args = parser.parse_args(list(argv) if argv is not None else None)
-
- input_path = args.input.resolve()
- if not input_path.exists():
- parser.error(f"Input file '{input_path}' does not exist.")
-
- amb_bytes = build_amb(
- root_markdown=input_path,
- title=args.title,
- )
- args.output.parent.mkdir(parents=True, exist_ok=True)
- args.output.write_bytes(amb_bytes)
- print(str(args.output))
- return 0
-
-
-def build_amb(root_markdown: Path, title: str | None) -> bytes:
- articles = collect_articles(root_markdown)
- ama_contents = render_articles(articles)
- files = assemble_files(ama_contents, title)
- return pack_amb(files)
-
-
-def collect_articles(root_markdown: Path) -> Dict[Path, Article]:
- queue: deque[Path] = deque([root_markdown])
- visited: Dict[Path, Article] = {}
- assigned_names: set[str] = set()
-
- while queue:
- current = queue.popleft()
- current = current.resolve()
- if current in visited:
- continue
- if not current.exists():
- raise FileNotFoundError(f"Referenced file '{current}' was not found.")
- if current == root_markdown:
- ama_name = "INDEX.AMA"
- else:
- ama_name = assign_ama_name(current.stem, assigned_names)
- assigned_names.add(ama_name)
- visited[current] = Article(source=current, ama_name=ama_name)
-
- for linked in find_local_markdown_links(current):
- queue.append(linked)
-
- return visited
-
-
-def find_local_markdown_links(markdown_path: Path) -> List[Path]:
- text = markdown_path.read_text(encoding="utf-8")
- results: List[Path] = []
-
- for _, target, _ in MARKDOWN_LINK_RE.findall(text):
- cleaned = target.strip()
- if not cleaned or cleaned.startswith("#"):
- continue
- if "://" in cleaned or cleaned.startswith(("mailto:", "ftp:", "gopher:", "tel:")):
- continue
- resolved = (markdown_path.parent / cleaned.split("#", 1)[0]).resolve()
- if resolved.suffix.lower() in EXT_MD:
- results.append(resolved)
- return results
-
-
-def assign_ama_name(stem: str, existing: set[str]) -> str:
- base = "".join((c if c.isalnum() else "_") for c in stem.upper())
- if not base:
- base = "ARTICLE"
- if base[0].isdigit():
- base = f"_{base}"
- base = base[:8]
-
- name = f"{base}.AMA"
- counter = 1
- while name in existing:
- suffix = f"{counter:02d}"
- trimmed = base[: max(1, 8 - len(suffix))]
- name = f"{trimmed}{suffix}.AMA"
- counter += 1
- return name
-
-
-def render_articles(articles: Dict[Path, Article]) -> Dict[str, List[str]]:
- parser_factory = get_parser_factory("markdown")
- renderer_factory = get_renderer_factory("ama")
- rendered: Dict[str, List[str]] = {}
-
- for path, article in articles.items():
- content = path.read_text(encoding="utf-8")
- rewritten = rewrite_links(content, path.parent, articles)
- frontmatter, body_lines = parse_frontmatter(rewritten.splitlines(keepends=True))
- ama_lines = run_conversion(
- body_lines,
- frontmatter=frontmatter,
- parser_factory=parser_factory,
- renderer_factory=renderer_factory,
- renderer_options={"width": 78},
- base_path=path.parent,
- )
- split_articles = split_article(article.ama_name, ama_lines)
- rendered.update(split_articles)
- return rendered
-
-
-def rewrite_links(markdown: str, base_dir: Path, articles: Dict[Path, Article]) -> str:
- def replacer(match: re.Match[str]) -> str:
- prefix, target, suffix = match.groups()
- cleaned = target.strip()
- candidate = (base_dir / cleaned.split("#", 1)[0]).resolve()
- if candidate in articles:
- mapped = articles[candidate].ama_name
- return f"{prefix}{mapped}{suffix}"
- return match.group(0)
-
- return MARKDOWN_LINK_RE.sub(replacer, markdown)
-
-
-def split_article(filename: str, lines: List[str]) -> Dict[str, List[str]]:
- encoded = "\n".join(lines).encode("utf-8")
- if len(encoded) <= AMA_MAX_BYTES:
- return {filename: lines}
-
- segments: List[List[str]] = []
- current: List[str] = []
- current_size = 0
-
- def flush_segment() -> None:
- nonlocal current, current_size
- if current:
- segments.append(current)
- current = []
- current_size = 0
-
- for line in lines:
- candidate_size = current_size + len((line + "\n").encode("utf-8"))
- if candidate_size > AMA_MAX_BYTES and current:
- flush_segment()
- current.append(line)
- current_size += len((line + "\n").encode("utf-8"))
- flush_segment()
-
- result: Dict[str, List[str]] = {}
- stem = Path(filename).stem
- generated_names = [filename]
-
- for idx in range(1, len(segments)):
- suffix = f"{idx:02d}"
- trimmed = stem[: max(1, 8 - len(suffix))]
- new_name = f"{trimmed}{suffix}.AMA"
- counter = 1
- while new_name in result or new_name in generated_names:
- suffix = f"{idx:02d}{counter}"
- trimmed = stem[: max(1, 8 - len(suffix))]
- new_name = f"{trimmed}{suffix}.AMA"
- counter += 1
- generated_names.append(new_name)
-
- for name, segment in zip(generated_names, segments, strict=False):
- result[name] = segment[:]
-
- for idx, name in enumerate(generated_names[:-1]):
- next_name = generated_names[idx + 1]
- result[name].append("")
- result[name].append(f"%l{next_name}:{LINK_CONTINUE_LABEL}%t")
- return result
-
-
-def assemble_files(ama_contents: Dict[str, List[str]], title: str | None) -> List[Tuple[str, bytes]]:
- files: List[Tuple[str, bytes]] = []
- if title:
- files.append(("TITLE", title.encode("ascii", "ignore")[:64]))
-
- index_bytes = encode_ama("INDEX.AMA", ama_contents.pop("INDEX.AMA"))
- files.append(("INDEX.AMA", index_bytes))
-
- for name, lines in sorted(ama_contents.items()):
- files.append((name, encode_ama(name, lines)))
-
- return files
-
-
-def encode_ama(name: str, lines: List[str]) -> bytes:
- content = "\n".join(lines).rstrip("\n") + "\n"
- data = content.encode("utf-8")
- if len(data) > AMA_MAX_BYTES:
- raise ValueError(f"Generated AMA article '{name}' exceeds {AMA_MAX_BYTES} bytes.")
- if any("\t" in line for line in lines):
- raise ValueError(f"Generated AMA article '{name}' contains tab characters.")
- return data
-
-
-def pack_amb(files: List[Tuple[str, bytes]]) -> bytes:
- entries = []
- offset = 6 + 20 * len(files)
- payloads = []
-
- for filename, data in files:
- canonical = filename.upper()
- if len(canonical) > 12:
- raise ValueError(f"Filename '{canonical}' does not fit 8.3 constraints.")
- payloads.append(data)
- checksum = bsd_checksum(data)
- entries.append((canonical, offset, len(data), checksum))
- offset += len(data)
-
- output = bytearray()
- output.extend(AMB_MAGIC)
- output.extend(struct.pack("