packaged
This commit is contained in:
parent
8cdedd43d6
commit
5067b9ec65
21 changed files with 114 additions and 566 deletions
43
README.md
43
README.md
|
|
@ -1,11 +1,10 @@
|
||||||
# md2amb utilities
|
# md2txt
|
||||||
|
|
||||||
This repository contains command line helpers for transforming Markdown into formats that work well on retro hardware or constrained text viewers. A small plugin API lets you mix-and-match parsers and renderers so additional formats can plug into the same preprocessing pipeline.
|
This repository contains the `md2txt` command line tool and supporting libraries for transforming Markdown into formats that work well on retro hardware or constrained text viewers. A small plugin API lets you mix-and-match parsers and renderers so additional formats can plug into the same preprocessing pipeline.
|
||||||
|
|
||||||
## Tools
|
## CLI
|
||||||
|
|
||||||
- `md2amb.py` – converts Markdown into Amber-screen formatted text (see script for details).
|
- `md2txt` – converts Markdown into 80-column, DOS-compatible plain text with extensive formatting support. It ships with the default `markdown` parser and `text` renderer plugins, registers optional `micron` and `ama` renderers for Micron/Ancient Machine Book output, and exposes the core pipeline so you can add your own parser or renderer modules:
|
||||||
- `md2txt.py` – converts Markdown into 80-column, DOS-compatible plain text with extensive formatting support. It ships with the default `markdown` parser and `text` renderer plugins, registers optional `micron` and `ama` renderers for Micron/Ancient Machine Book output, and exposes the core pipeline so you can add your own parser or renderer modules:
|
|
||||||
- FIGlet-rendered headings (H1–H3) driven by optional YAML frontmatter (`h1_font`, `h2_font`, `h3_font`).
|
- FIGlet-rendered headings (H1–H3) driven by optional YAML frontmatter (`h1_font`, `h2_font`, `h3_font`).
|
||||||
- H4+ headings rendered in uppercase with dashed underlines.
|
- H4+ headings rendered in uppercase with dashed underlines.
|
||||||
- Emphasis styles converted to spaced or delimited characters, e.g. `**bold**` → `B O L D`, `__strong__` → `_s_t_r_o_n_g_`, `~~strike~~` → `~s~t~r~i~k~e~`.
|
- Emphasis styles converted to spaced or delimited characters, e.g. `**bold**` → `B O L D`, `__strong__` → `_s_t_r_o_n_g_`, `~~strike~~` → `~s~t~r~i~k~e~`.
|
||||||
|
|
@ -13,7 +12,6 @@ This repository contains command line helpers for transforming Markdown into for
|
||||||
- Code blocks numbered (`01 | line`), with both fenced and indented fences supported (see “Rendering Controls” for customisation).
|
- Code blocks numbered (`01 | line`), with both fenced and indented fences supported (see “Rendering Controls” for customisation).
|
||||||
- Links transformed into `[label](n)` references, with footnote-style URL list at the end.
|
- Links transformed into `[label](n)` references, with footnote-style URL list at the end.
|
||||||
- Optional alignment and margin controls via HTML `<p>` attributes or MultiMarkdown attribute blocks (e.g. `{:.center margin=20px}`) applied as leading spaces.
|
- Optional alignment and margin controls via HTML `<p>` attributes or MultiMarkdown attribute blocks (e.g. `{:.center margin=20px}`) applied as leading spaces.
|
||||||
- Recursive file includes using `![[file.md]]` or `{.include file.md}` (frontmatter inside the included file is ignored).
|
|
||||||
- ASCII art injection with `#[label :align](art.txt)` syntax, supporting multiple art pieces per line and optional `{: .right}` style annotations.
|
- ASCII art injection with `#[label :align](art.txt)` syntax, supporting multiple art pieces per line and optional `{: .right}` style annotations.
|
||||||
- Per-document toggles for code block wrapping, numbering, blockquote decoration, and list indentation spacing.
|
- Per-document toggles for code block wrapping, numbering, blockquote decoration, and list indentation spacing.
|
||||||
|
|
||||||
|
|
@ -26,21 +24,18 @@ This repository contains command line helpers for transforming Markdown into for
|
||||||
## Usage
|
## Usage
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
python md2txt.py input.md -o output.txt # convert to DOS-friendly text
|
md2txt input.md -o output.txt # convert to DOS-friendly text
|
||||||
python md2txt.py input.md # write result to stdout
|
md2txt input.md # write result to stdout
|
||||||
python md2txt.py input.md --width 72 # override column width
|
md2txt input.md --width 72 # override column width
|
||||||
python md2txt.py input.md --parser markdown --renderer micron # emit Micron-formatted output
|
md2txt input.md --parser markdown --renderer micron # emit Micron-formatted output
|
||||||
python md2txt.py input.md --renderer ama # emit AMB/AMA markup
|
md2txt input.md --renderer ama # emit AMB/AMA markup
|
||||||
python md2txt.py input.md --renderer-option width=68 # pass KEY=VALUE to a renderer
|
md2txt input.md --renderer-option width=68 # pass KEY=VALUE to a renderer
|
||||||
|
|
||||||
|
# If the project is not installed yet:
|
||||||
|
python -m md2txt input.md
|
||||||
```
|
```
|
||||||
|
|
||||||
- `md2amb.py` – package Markdown (and linked Markdown files) into a self-contained `.amb` archive composed of `.ama` articles that honour the 78-column/64 KiB AMA constraints.
|
`--parser` and `--renderer` select a plugin by name (defaults are `markdown` and `text`). Repeatable `--parser-option KEY=VALUE` and `--renderer-option KEY=VALUE` pairs are forwarded to the plugin factories as keyword arguments in addition to the defaults supplied by the CLI. The CLI accepts `--help` for the full option list.
|
||||||
|
|
||||||
```bash
|
|
||||||
python md2amb.py --title "Your Manual" docs/index.md output/manual.amb
|
|
||||||
```
|
|
||||||
|
|
||||||
`--parser` and `--renderer` select a plugin by name (defaults are `markdown` and `text`). Repeatable `--parser-option KEY=VALUE` and `--renderer-option KEY=VALUE` pairs are forwarded to the plugin factories as keyword arguments in addition to the defaults supplied by the CLI. Both scripts accept `--help` for the full option list.
|
|
||||||
|
|
||||||
## FIGlet Fonts via Frontmatter
|
## FIGlet Fonts via Frontmatter
|
||||||
|
|
||||||
|
|
@ -126,15 +121,15 @@ Generated text uses CRLF line endings to maintain DOS compatibility.
|
||||||
|
|
||||||
## Plugin Architecture
|
## Plugin Architecture
|
||||||
|
|
||||||
The conversion pipeline lives in `conversion_core.py` and is exposed via `run_conversion`. It is designed around lightweight factories:
|
The conversion pipeline lives in `src/md2txt/conversion/core.py` and is exposed via `run_conversion`. It is designed around lightweight factories:
|
||||||
|
|
||||||
- **Parser factories** receive `base_style: BlockStyle` plus any extra keyword arguments and must return an object with a `parse(lines: Iterable[str]) -> Iterator[BlockEvent | StyleUpdateEvent]` method. The bundled `MarkdownParser` implements this interface.
|
- **Parser factories** receive `base_style: BlockStyle` plus any extra keyword arguments and must return an object with a `parse(lines: Iterable[str]) -> Iterator[BlockEvent | StyleUpdateEvent]` method. The bundled `MarkdownParser` implements this interface.
|
||||||
- **Renderer factories** receive `frontmatter: FrontMatter` and arbitrary keyword arguments and must return an object that provides `handle_event(event)` and `finalize() -> Any`. The `TextRenderer` returns a list of DOS-friendly output lines; other renderers may return any data appropriate for their target format.
|
- **Renderer factories** receive `frontmatter: FrontMatter` and arbitrary keyword arguments and must return an object that provides `handle_event(event)` and `finalize() -> Any`. The `TextRenderer` returns a list of DOS-friendly output lines; other renderers may return any data appropriate for their target format.
|
||||||
|
|
||||||
Plugins register themselves through the helpers in `plugins.py`:
|
Plugins register themselves through the helpers in `md2txt.plugins`:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
from plugins import register_parser, register_renderer
|
from md2txt.plugins import register_parser, register_renderer
|
||||||
|
|
||||||
def my_parser_factory(*, base_style, **options):
|
def my_parser_factory(*, base_style, **options):
|
||||||
return MyParser(base_style, **options)
|
return MyParser(base_style, **options)
|
||||||
|
|
@ -149,6 +144,6 @@ def my_renderer_factory(*, frontmatter, **options):
|
||||||
register_renderer("ansi", my_renderer_factory)
|
register_renderer("ansi", my_renderer_factory)
|
||||||
```
|
```
|
||||||
|
|
||||||
Once registered (for example in a small module that imports `md2txt.py`), the new plugins are available via `--parser my-markdown` or `--renderer ansi`. The CLI lists registered plugin names in `--help`, and the helper functions `available_parsers()` / `available_renderers()` return the sorted names if you need to build higher-level tooling.
|
Once registered (for example in a small module that imports `md2txt.cli`), the new plugins are available via `--parser my-markdown` or `--renderer ansi`. The CLI lists registered plugin names in `--help`, and the helper functions `available_parsers()` / `available_renderers()` return the sorted names if you need to build higher-level tooling.
|
||||||
|
|
||||||
The shared preprocessing helpers—YAML frontmatter parsing, recursive include expansion, and ASCII art sentinels—also live in `conversion_core.py`, allowing alternate front-ends to reuse exactly the same behaviour without duplicating code.
|
The shared preprocessing helpers—YAML frontmatter parsing, recursive include expansion, and ASCII art sentinels—also live in `src/md2txt/conversion/core.py`, allowing alternate front-ends to reuse exactly the same behaviour without duplicating code.
|
||||||
|
|
|
||||||
BIN
lorem.amb
BIN
lorem.amb
Binary file not shown.
237
loremipsum.mu
237
loremipsum.mu
|
|
@ -1,237 +0,0 @@
|
||||||
|
|
||||||
> Salientia
|
|
||||||
|
|
||||||
>> Structa hinc vetitus factum
|
|
||||||
|
|
||||||
Lorem markdownum omnes tellus, in, Herculeis
|
|
||||||
`=reciprocalPower`= neque promissa est Latiae caput
|
|
||||||
umero talia fidissima, nymphae. Foedera et sagittas
|
|
||||||
tenetur adde matri esse: talia manibus.
|
|
||||||
|
|
||||||
>> Fuit video longique dedisse
|
|
||||||
|
|
||||||
Ille stupet, ultra `!incunabula agendum urbes`! mirer
|
|
||||||
`!aper`!, metu. Aegides novissima sunt turis quatiebant
|
|
||||||
umbrosaque `[putares`#mille-parva]. Metuunt eruiturque
|
|
||||||
maximus velit, ater Vesta vulnus rustica verba! Mei
|
|
||||||
vincite nulla, in Iovem: fama exurunt vernum!
|
|
||||||
|
|
||||||
De instat ubi induit fugientia inertes columbas tarda
|
|
||||||
`*terga Parnasia sententia`* omnia inmotusque, est et
|
|
||||||
aetas sonus. Exhibuit coeperunt, canis. Suis minora,
|
|
||||||
tempore fateor neve moderamine erit. Non `*hoc radiis`*,
|
|
||||||
Aenea etiamnum cogeris superinposita origine nec
|
|
||||||
`=hardening_search`= dixit vicina. Heros subito
|
|
||||||
populifer adeunt quicquam cum `!ales`! Ixion angulus
|
|
||||||
placet reddere.
|
|
||||||
|
|
||||||
>> Tractaque serpens arva quoque
|
|
||||||
|
|
||||||
> Include
|
|
||||||
|
|
||||||
<>=======()
|
|
||||||
(/\___ /|\\ ()==========<>_
|
|
||||||
\_/ | \\ //|\ ______/ \)
|
|
||||||
\_| \\ // | \_/
|
|
||||||
\|\/|\_ // /\/
|
|
||||||
(oo)\ \_// /
|
|
||||||
//_/\_\/ / |
|
|
||||||
@@/ |=\ \ |
|
|
||||||
\_=\_ \ |
|
|
||||||
\==\ \|\_ snd
|
|
||||||
__(\===\( )\
|
|
||||||
(((~) __(_/ |
|
|
||||||
(((~) \ /
|
|
||||||
______/ /
|
|
||||||
'------'
|
|
||||||
<>=======()
|
|
||||||
(/\___ /|\\ ()==========<>_
|
|
||||||
\_/ | \\ //|\ ______/ \)
|
|
||||||
\_| \\ // | \_/
|
|
||||||
\|\/|\_ // /\/
|
|
||||||
(oo)\ \_// /
|
|
||||||
//_/\_\/ / |
|
|
||||||
@@/ |=\ \ |
|
|
||||||
\_=\_ \ |
|
|
||||||
\==\ \|\_ snd
|
|
||||||
__(\===\( )\
|
|
||||||
(((~) __(_/ |
|
|
||||||
(((~) \ /
|
|
||||||
______/ /
|
|
||||||
'------'
|
|
||||||
<>=======()
|
|
||||||
(/\___ /|\\ ()==========<>_
|
|
||||||
\_/ | \\ //|\ ______/ \)
|
|
||||||
\_| \\ // | \_/
|
|
||||||
\|\/|\_ // /\/
|
|
||||||
(oo)\ \_// /
|
|
||||||
//_/\_\/ / |
|
|
||||||
@@/ |=\ \ |
|
|
||||||
\_=\_ \ |
|
|
||||||
\==\ \|\_ snd
|
|
||||||
__(\===\( )\
|
|
||||||
(((~) __(_/ |
|
|
||||||
(((~) \ /
|
|
||||||
______/ /
|
|
||||||
'------'
|
|
||||||
|
|
||||||
> h1 Heading 8-)
|
|
||||||
|
|
||||||
>> h2 Heading
|
|
||||||
|
|
||||||
>>> h3 Heading
|
|
||||||
|
|
||||||
>>> h4 Heading
|
|
||||||
|
|
||||||
>>> h5 Heading
|
|
||||||
|
|
||||||
>>> h6 Heading
|
|
||||||
|
|
||||||
>> Horizontal Rules
|
|
||||||
|
|
||||||
-
|
|
||||||
|
|
||||||
-
|
|
||||||
|
|
||||||
-
|
|
||||||
|
|
||||||
>> Emphasis
|
|
||||||
|
|
||||||
`!This is bold text`! `!This is bold text`! `*This is
|
|
||||||
italic text`* `*This is italic text`* ~~Strikethrough~~
|
|
||||||
|
|
||||||
>> Blockquotes
|
|
||||||
|
|
||||||
>>>>Blockquotes can also be nested...
|
|
||||||
|
|
||||||
>>>>>>>>by using additional greater-than signs right
|
|
||||||
>>>>>>>>next to each other...
|
|
||||||
|
|
||||||
>>>>>>>>>>>>or with spaces between arrows.
|
|
||||||
|
|
||||||
>> Lists
|
|
||||||
|
|
||||||
>>> Unordered
|
|
||||||
|
|
||||||
+ Create a list by starting a line with `=+`=, `=-`=, or
|
|
||||||
`=*`=
|
|
||||||
- Marker character change forces new list start:
|
|
||||||
* Ac tristique libero volutpat at
|
|
||||||
+ Facilisis in pretium nisl aliquet
|
|
||||||
- Nulla volutpat aliquam velit
|
|
||||||
+ Very easy
|
|
||||||
>>> Ordered
|
|
||||||
|
|
||||||
1. Lorem ipsum dolor sit amet
|
|
||||||
2. Consectetur adipiscing elit
|
|
||||||
3. Integer molestie lorem at massa
|
|
||||||
>> Code
|
|
||||||
|
|
||||||
Inline `=code`=
|
|
||||||
|
|
||||||
Indented code
|
|
||||||
|
|
||||||
`=
|
|
||||||
// Some comments
|
|
||||||
line 1 of code
|
|
||||||
line 2 of code
|
|
||||||
line 3 of code
|
|
||||||
`=
|
|
||||||
|
|
||||||
Block code "fences"
|
|
||||||
|
|
||||||
`=
|
|
||||||
Sample text here...
|
|
||||||
`=
|
|
||||||
|
|
||||||
Syntax highlighting
|
|
||||||
|
|
||||||
`=
|
|
||||||
var foo = function (bar) {
|
|
||||||
return bar++;
|
|
||||||
};
|
|
||||||
|
|
||||||
console.log(foo(5));
|
|
||||||
`=
|
|
||||||
|
|
||||||
>> Links
|
|
||||||
|
|
||||||
`[link text`http://dev.nodeca.com]
|
|
||||||
|
|
||||||
`[link with title`http://nodeca.github.io/pica/demo/
|
|
||||||
"title text!"]
|
|
||||||
|
|
||||||
>> Images
|
|
||||||
|
|
||||||
`[Minion`https://octodex.github.com/images/minion.png] `
|
|
||||||
[Stormtroopocat`https://octodex.github.com/images/stormt
|
|
||||||
roopocat.jpg "The Stormtroopocat"]
|
|
||||||
|
|
||||||
Inprudens dum memor alma, casses dedi Sinuessa iam quid
|
|
||||||
adicit `=gate_truncate_wave`=. `[Geratur de`#salientia-
|
|
||||||
iungit-contra-et], insolida temptat `=open`= carentia;
|
|
||||||
genitorem solet, potens. Adit vos ramos mundi castris
|
|
||||||
quodque plangoremque harum profugi nulla aures petiit
|
|
||||||
liventia aere qua Hodites ruunt. Achaidos dilexisse
|
|
||||||
`=redundancy`= praesagia tenuatus necis `!Capys
|
|
||||||
recordor`! tenebras actis signis, reliquit, placidissime
|
|
||||||
iram liquefactis longe nervi famem. Cura pars natique
|
|
||||||
velleris semel et at `=printerWildcardParallel`= inquit
|
|
||||||
positosque comites ultra.
|
|
||||||
|
|
||||||
>> Quantaque augent
|
|
||||||
|
|
||||||
Acuti ignotis ignis, ostendit nostras terras nondum,
|
|
||||||
`[petit utque utero`#structa-hinc-vetitus-factum]
|
|
||||||
sepulcrales, pluma reseratis Ismario. Tria silentia duae
|
|
||||||
`*nymphe`*: certe parva cernam et dixit. `[Verborum
|
|
||||||
sedisti`#mille-parva].
|
|
||||||
|
|
||||||
`=
|
|
||||||
if (firewall(4, 3) == 1 * device_media_parity) {
|
|
||||||
system_copy_class += systray + oasis_wi_clean + lpi;
|
|
||||||
modelLdap.search_secondary_socket *= bounce.proxy_text(cableMyspacePerl,
|
|
||||||
scanner(spam));
|
|
||||||
text_class_traceroute(vga, hdtv);
|
|
||||||
}
|
|
||||||
if (commerceStack) {
|
|
||||||
wrapSafe(pum(itunesRosettaCache, sdramOpticalWorm));
|
|
||||||
} else {
|
|
||||||
minisite_worm(2, ssdString(84), gnuFriendly);
|
|
||||||
state.qwerty(moduleJavascriptFloating + ugc, koffice,
|
|
||||||
mail_system.prebinding_service_teraflops(mediaIcmpBoot));
|
|
||||||
}
|
|
||||||
memoryHexadecimalMultitasking += oop_android;
|
|
||||||
var passive_vdu = bar;
|
|
||||||
var ddrJava = leaf(unc_web);
|
|
||||||
`=
|
|
||||||
|
|
||||||
>>> Mille parva
|
|
||||||
|
|
||||||
`!Minetur et`! aurum! Charaxi sanguine quo; vires
|
|
||||||
subnectite `!pignora`!, omnique hostibus loquerentur
|
|
||||||
aquae Solis ferox? `!Cum ne`!, putat utar ex ad aede
|
|
||||||
mane. Vitalesque geminas exstimulat, iam vivunt dat
|
|
||||||
artes et quem! Iam violas molire ipse, spoliabitur dente
|
|
||||||
nervis regis, factos quae.
|
|
||||||
|
|
||||||
>>>>Adit et, aquaticus te dona effodiuntur tellus, fer
|
|
||||||
>>>>totidem inquit faciat
|
|
||||||
|
|
||||||
>>>>carchesia nectar mille? Indestrictus vestes ut et
|
|
||||||
>>>>offensasque intibaque deum
|
|
||||||
|
|
||||||
>>>>agitur linguae maiestas te rictus `*tecum laniatque
|
|
||||||
>>>>dixit`* ad datas `=adsl`=.
|
|
||||||
|
|
||||||
>>>>Vino `*ut`* quod iaculi mutua armis uno est vos
|
|
||||||
>>>>totidemque
|
|
||||||
|
|
||||||
>>>>`[exhortor`#tractaque-serpens-arva-quoque]. Iamdudum
|
|
||||||
>>>>Romanae cepi linguisque
|
|
||||||
|
|
||||||
>>>>velit accipe idem acutis hoc sparsumque dextra.
|
|
||||||
>>>>Minoide sponte collo sequitur
|
|
||||||
|
|
||||||
>>>>et intus flere sanguineaque herba, nox Quas adsiduo
|
|
||||||
>>>>`*cunctantem`* tibi; enim.
|
|
||||||
282
md2amb.py
282
md2amb.py
|
|
@ -1,282 +0,0 @@
|
||||||
#!/usr/bin/env python3
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import re
|
|
||||||
import struct
|
|
||||||
from collections import deque
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Dict, Iterable, List, Tuple
|
|
||||||
|
|
||||||
import ama_renderer # noqa: F401 - ensure AMA renderer plugin registration
|
|
||||||
from conversion_core import parse_frontmatter, run_conversion
|
|
||||||
from markdown_parser import MarkdownParser
|
|
||||||
from md_types import BlockStyle, FrontMatter
|
|
||||||
from plugins import get_parser_factory, get_renderer_factory, register_parser
|
|
||||||
from text_renderer import TextRenderer
|
|
||||||
|
|
||||||
# Ensure markdown parser registered for standalone usage
|
|
||||||
|
|
||||||
|
|
||||||
def _markdown_parser_factory(*, base_style: BlockStyle, **_: object) -> MarkdownParser:
|
|
||||||
return MarkdownParser(base_style)
|
|
||||||
|
|
||||||
|
|
||||||
try:
|
|
||||||
register_parser("markdown", _markdown_parser_factory)
|
|
||||||
except ValueError:
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
MARKDOWN_LINK_RE = re.compile(r"(\[[^\]]*\]\()([^)]+)(\))")
|
|
||||||
LOCAL_LINK_RE = re.compile(r"^[A-Za-z0-9_.~/\\-]+$")
|
|
||||||
EXT_MD = {".md", ".markdown", ".mkd", ".mkdn"}
|
|
||||||
AMA_MAX_BYTES = 65_535
|
|
||||||
AMB_MAGIC = b"AMB1"
|
|
||||||
LINK_CONTINUE_LABEL = "Continue"
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class Article:
|
|
||||||
source: Path
|
|
||||||
ama_name: str
|
|
||||||
|
|
||||||
|
|
||||||
def main(argv: Iterable[str] | None = None) -> int:
|
|
||||||
parser = argparse.ArgumentParser(description="Convert Markdown into an AMB archive.")
|
|
||||||
parser.add_argument("input", type=Path, help="Root Markdown file to convert.")
|
|
||||||
parser.add_argument("output", type=Path, help="Output AMB filename.")
|
|
||||||
parser.add_argument("--title", type=str, help="Optional book title.")
|
|
||||||
args = parser.parse_args(list(argv) if argv is not None else None)
|
|
||||||
|
|
||||||
input_path = args.input.resolve()
|
|
||||||
if not input_path.exists():
|
|
||||||
parser.error(f"Input file '{input_path}' does not exist.")
|
|
||||||
|
|
||||||
amb_bytes = build_amb(
|
|
||||||
root_markdown=input_path,
|
|
||||||
title=args.title,
|
|
||||||
)
|
|
||||||
args.output.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
args.output.write_bytes(amb_bytes)
|
|
||||||
print(str(args.output))
|
|
||||||
return 0
|
|
||||||
|
|
||||||
|
|
||||||
def build_amb(root_markdown: Path, title: str | None) -> bytes:
|
|
||||||
articles = collect_articles(root_markdown)
|
|
||||||
ama_contents = render_articles(articles)
|
|
||||||
files = assemble_files(ama_contents, title)
|
|
||||||
return pack_amb(files)
|
|
||||||
|
|
||||||
|
|
||||||
def collect_articles(root_markdown: Path) -> Dict[Path, Article]:
|
|
||||||
queue: deque[Path] = deque([root_markdown])
|
|
||||||
visited: Dict[Path, Article] = {}
|
|
||||||
assigned_names: set[str] = set()
|
|
||||||
|
|
||||||
while queue:
|
|
||||||
current = queue.popleft()
|
|
||||||
current = current.resolve()
|
|
||||||
if current in visited:
|
|
||||||
continue
|
|
||||||
if not current.exists():
|
|
||||||
raise FileNotFoundError(f"Referenced file '{current}' was not found.")
|
|
||||||
if current == root_markdown:
|
|
||||||
ama_name = "INDEX.AMA"
|
|
||||||
else:
|
|
||||||
ama_name = assign_ama_name(current.stem, assigned_names)
|
|
||||||
assigned_names.add(ama_name)
|
|
||||||
visited[current] = Article(source=current, ama_name=ama_name)
|
|
||||||
|
|
||||||
for linked in find_local_markdown_links(current):
|
|
||||||
queue.append(linked)
|
|
||||||
|
|
||||||
return visited
|
|
||||||
|
|
||||||
|
|
||||||
def find_local_markdown_links(markdown_path: Path) -> List[Path]:
|
|
||||||
text = markdown_path.read_text(encoding="utf-8")
|
|
||||||
results: List[Path] = []
|
|
||||||
|
|
||||||
for _, target, _ in MARKDOWN_LINK_RE.findall(text):
|
|
||||||
cleaned = target.strip()
|
|
||||||
if not cleaned or cleaned.startswith("#"):
|
|
||||||
continue
|
|
||||||
if "://" in cleaned or cleaned.startswith(("mailto:", "ftp:", "gopher:", "tel:")):
|
|
||||||
continue
|
|
||||||
resolved = (markdown_path.parent / cleaned.split("#", 1)[0]).resolve()
|
|
||||||
if resolved.suffix.lower() in EXT_MD:
|
|
||||||
results.append(resolved)
|
|
||||||
return results
|
|
||||||
|
|
||||||
|
|
||||||
def assign_ama_name(stem: str, existing: set[str]) -> str:
|
|
||||||
base = "".join((c if c.isalnum() else "_") for c in stem.upper())
|
|
||||||
if not base:
|
|
||||||
base = "ARTICLE"
|
|
||||||
if base[0].isdigit():
|
|
||||||
base = f"_{base}"
|
|
||||||
base = base[:8]
|
|
||||||
|
|
||||||
name = f"{base}.AMA"
|
|
||||||
counter = 1
|
|
||||||
while name in existing:
|
|
||||||
suffix = f"{counter:02d}"
|
|
||||||
trimmed = base[: max(1, 8 - len(suffix))]
|
|
||||||
name = f"{trimmed}{suffix}.AMA"
|
|
||||||
counter += 1
|
|
||||||
return name
|
|
||||||
|
|
||||||
|
|
||||||
def render_articles(articles: Dict[Path, Article]) -> Dict[str, List[str]]:
|
|
||||||
parser_factory = get_parser_factory("markdown")
|
|
||||||
renderer_factory = get_renderer_factory("ama")
|
|
||||||
rendered: Dict[str, List[str]] = {}
|
|
||||||
|
|
||||||
for path, article in articles.items():
|
|
||||||
content = path.read_text(encoding="utf-8")
|
|
||||||
rewritten = rewrite_links(content, path.parent, articles)
|
|
||||||
frontmatter, body_lines = parse_frontmatter(rewritten.splitlines(keepends=True))
|
|
||||||
ama_lines = run_conversion(
|
|
||||||
body_lines,
|
|
||||||
frontmatter=frontmatter,
|
|
||||||
parser_factory=parser_factory,
|
|
||||||
renderer_factory=renderer_factory,
|
|
||||||
renderer_options={"width": 78},
|
|
||||||
base_path=path.parent,
|
|
||||||
)
|
|
||||||
split_articles = split_article(article.ama_name, ama_lines)
|
|
||||||
rendered.update(split_articles)
|
|
||||||
return rendered
|
|
||||||
|
|
||||||
|
|
||||||
def rewrite_links(markdown: str, base_dir: Path, articles: Dict[Path, Article]) -> str:
|
|
||||||
def replacer(match: re.Match[str]) -> str:
|
|
||||||
prefix, target, suffix = match.groups()
|
|
||||||
cleaned = target.strip()
|
|
||||||
candidate = (base_dir / cleaned.split("#", 1)[0]).resolve()
|
|
||||||
if candidate in articles:
|
|
||||||
mapped = articles[candidate].ama_name
|
|
||||||
return f"{prefix}{mapped}{suffix}"
|
|
||||||
return match.group(0)
|
|
||||||
|
|
||||||
return MARKDOWN_LINK_RE.sub(replacer, markdown)
|
|
||||||
|
|
||||||
|
|
||||||
def split_article(filename: str, lines: List[str]) -> Dict[str, List[str]]:
|
|
||||||
encoded = "\n".join(lines).encode("utf-8")
|
|
||||||
if len(encoded) <= AMA_MAX_BYTES:
|
|
||||||
return {filename: lines}
|
|
||||||
|
|
||||||
segments: List[List[str]] = []
|
|
||||||
current: List[str] = []
|
|
||||||
current_size = 0
|
|
||||||
|
|
||||||
def flush_segment() -> None:
|
|
||||||
nonlocal current, current_size
|
|
||||||
if current:
|
|
||||||
segments.append(current)
|
|
||||||
current = []
|
|
||||||
current_size = 0
|
|
||||||
|
|
||||||
for line in lines:
|
|
||||||
candidate_size = current_size + len((line + "\n").encode("utf-8"))
|
|
||||||
if candidate_size > AMA_MAX_BYTES and current:
|
|
||||||
flush_segment()
|
|
||||||
current.append(line)
|
|
||||||
current_size += len((line + "\n").encode("utf-8"))
|
|
||||||
flush_segment()
|
|
||||||
|
|
||||||
result: Dict[str, List[str]] = {}
|
|
||||||
stem = Path(filename).stem
|
|
||||||
generated_names = [filename]
|
|
||||||
|
|
||||||
for idx in range(1, len(segments)):
|
|
||||||
suffix = f"{idx:02d}"
|
|
||||||
trimmed = stem[: max(1, 8 - len(suffix))]
|
|
||||||
new_name = f"{trimmed}{suffix}.AMA"
|
|
||||||
counter = 1
|
|
||||||
while new_name in result or new_name in generated_names:
|
|
||||||
suffix = f"{idx:02d}{counter}"
|
|
||||||
trimmed = stem[: max(1, 8 - len(suffix))]
|
|
||||||
new_name = f"{trimmed}{suffix}.AMA"
|
|
||||||
counter += 1
|
|
||||||
generated_names.append(new_name)
|
|
||||||
|
|
||||||
for name, segment in zip(generated_names, segments, strict=False):
|
|
||||||
result[name] = segment[:]
|
|
||||||
|
|
||||||
for idx, name in enumerate(generated_names[:-1]):
|
|
||||||
next_name = generated_names[idx + 1]
|
|
||||||
result[name].append("")
|
|
||||||
result[name].append(f"%l{next_name}:{LINK_CONTINUE_LABEL}%t")
|
|
||||||
return result
|
|
||||||
|
|
||||||
|
|
||||||
def assemble_files(ama_contents: Dict[str, List[str]], title: str | None) -> List[Tuple[str, bytes]]:
|
|
||||||
files: List[Tuple[str, bytes]] = []
|
|
||||||
if title:
|
|
||||||
files.append(("TITLE", title.encode("ascii", "ignore")[:64]))
|
|
||||||
|
|
||||||
index_bytes = encode_ama("INDEX.AMA", ama_contents.pop("INDEX.AMA"))
|
|
||||||
files.append(("INDEX.AMA", index_bytes))
|
|
||||||
|
|
||||||
for name, lines in sorted(ama_contents.items()):
|
|
||||||
files.append((name, encode_ama(name, lines)))
|
|
||||||
|
|
||||||
return files
|
|
||||||
|
|
||||||
|
|
||||||
def encode_ama(name: str, lines: List[str]) -> bytes:
|
|
||||||
content = "\n".join(lines).rstrip("\n") + "\n"
|
|
||||||
data = content.encode("utf-8")
|
|
||||||
if len(data) > AMA_MAX_BYTES:
|
|
||||||
raise ValueError(f"Generated AMA article '{name}' exceeds {AMA_MAX_BYTES} bytes.")
|
|
||||||
if any("\t" in line for line in lines):
|
|
||||||
raise ValueError(f"Generated AMA article '{name}' contains tab characters.")
|
|
||||||
return data
|
|
||||||
|
|
||||||
|
|
||||||
def pack_amb(files: List[Tuple[str, bytes]]) -> bytes:
|
|
||||||
entries = []
|
|
||||||
offset = 6 + 20 * len(files)
|
|
||||||
payloads = []
|
|
||||||
|
|
||||||
for filename, data in files:
|
|
||||||
canonical = filename.upper()
|
|
||||||
if len(canonical) > 12:
|
|
||||||
raise ValueError(f"Filename '{canonical}' does not fit 8.3 constraints.")
|
|
||||||
payloads.append(data)
|
|
||||||
checksum = bsd_checksum(data)
|
|
||||||
entries.append((canonical, offset, len(data), checksum))
|
|
||||||
offset += len(data)
|
|
||||||
|
|
||||||
output = bytearray()
|
|
||||||
output.extend(AMB_MAGIC)
|
|
||||||
output.extend(struct.pack("<H", len(entries)))
|
|
||||||
|
|
||||||
for name, file_offset, length, checksum in entries:
|
|
||||||
padded = name.encode("ascii", "ignore")
|
|
||||||
padded = padded + b"\x00" * (12 - len(padded))
|
|
||||||
output.extend(padded)
|
|
||||||
output.extend(struct.pack("<I", file_offset))
|
|
||||||
output.extend(struct.pack("<H", length))
|
|
||||||
output.extend(struct.pack("<H", checksum))
|
|
||||||
|
|
||||||
for data in payloads:
|
|
||||||
output.extend(data)
|
|
||||||
return bytes(output)
|
|
||||||
|
|
||||||
|
|
||||||
def bsd_checksum(data: bytes) -> int:
|
|
||||||
checksum = 0
|
|
||||||
for byte in data:
|
|
||||||
checksum = (checksum >> 1) | ((checksum & 1) << 15)
|
|
||||||
checksum = (checksum + byte) & 0xFFFF
|
|
||||||
return checksum
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
raise SystemExit(main())
|
|
||||||
BIN
output.amb
BIN
output.amb
Binary file not shown.
|
|
@ -1,10 +1,20 @@
|
||||||
[project]
|
[project]
|
||||||
name = "md2amb"
|
name = "md2txt"
|
||||||
version = "0.1.0"
|
version = "0.1.0"
|
||||||
description = "Add your description here"
|
description = "Markdown to plain-text conversion toolkit"
|
||||||
readme = "README.md"
|
readme = "README.md"
|
||||||
requires-python = ">=3.13"
|
requires-python = ">=3.13"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"pyfiglet>=0.8.0",
|
"pyfiglet>=0.8.0",
|
||||||
"pyphen>=0.17.2",
|
"pyphen>=0.17.2",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[project.scripts]
|
||||||
|
md2txt = "md2txt.cli:main"
|
||||||
|
|
||||||
|
[build-system]
|
||||||
|
requires = ["setuptools>=68", "wheel"]
|
||||||
|
build-backend = "setuptools.build_meta"
|
||||||
|
|
||||||
|
[tool.setuptools.packages.find]
|
||||||
|
where = ["src"]
|
||||||
|
|
|
||||||
5
src/md2txt/__init__.py
Normal file
5
src/md2txt/__init__.py
Normal file
|
|
@ -0,0 +1,5 @@
|
||||||
|
"""Public package interface for md2txt."""
|
||||||
|
|
||||||
|
from .cli import convert_markdown, main
|
||||||
|
|
||||||
|
__all__ = ["convert_markdown", "main"]
|
||||||
7
src/md2txt/__main__.py
Normal file
7
src/md2txt/__main__.py
Normal file
|
|
@ -0,0 +1,7 @@
|
||||||
|
"""Entry point for `python -m md2txt`."""
|
||||||
|
|
||||||
|
from .cli import main
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
|
|
@ -9,10 +9,10 @@ import sys
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Dict, Iterable, List, Optional, Tuple
|
from typing import Any, Dict, Iterable, List, Optional, Tuple
|
||||||
|
|
||||||
from conversion_core import parse_frontmatter, read_lines, run_conversion
|
from .conversion.core import parse_frontmatter, read_lines, run_conversion
|
||||||
from md_types import BlockStyle, FrontMatter
|
from .models import BlockStyle, FrontMatter
|
||||||
from markdown_parser import MarkdownParser
|
from .parsers.markdown import MarkdownParser
|
||||||
from plugins import (
|
from .plugins import (
|
||||||
available_parsers,
|
available_parsers,
|
||||||
available_renderers,
|
available_renderers,
|
||||||
get_parser_factory,
|
get_parser_factory,
|
||||||
|
|
@ -20,10 +20,10 @@ from plugins import (
|
||||||
register_parser,
|
register_parser,
|
||||||
register_renderer,
|
register_renderer,
|
||||||
)
|
)
|
||||||
from text_renderer import TextRenderer
|
from .renderers.text import TextRenderer
|
||||||
|
|
||||||
import micron_renderer # noqa: F401 # register micron renderer plugin
|
from .renderers import micron # noqa: F401 # register micron renderer plugin
|
||||||
import ama_renderer # noqa: F401 # register AMA renderer plugin
|
from .renderers import ama # noqa: F401 # register AMA renderer plugin
|
||||||
|
|
||||||
|
|
||||||
def _split_option(token: str) -> Tuple[str, str]:
|
def _split_option(token: str) -> Tuple[str, str]:
|
||||||
19
src/md2txt/conversion/__init__.py
Normal file
19
src/md2txt/conversion/__init__.py
Normal file
|
|
@ -0,0 +1,19 @@
|
||||||
|
"""Conversion pipeline helpers."""
|
||||||
|
|
||||||
|
from .core import (
|
||||||
|
ParserFactory,
|
||||||
|
RendererFactory,
|
||||||
|
parse_frontmatter,
|
||||||
|
read_lines,
|
||||||
|
run_conversion,
|
||||||
|
run_pipeline,
|
||||||
|
)
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"ParserFactory",
|
||||||
|
"RendererFactory",
|
||||||
|
"parse_frontmatter",
|
||||||
|
"read_lines",
|
||||||
|
"run_conversion",
|
||||||
|
"run_pipeline",
|
||||||
|
]
|
||||||
|
|
@ -5,7 +5,7 @@ import re
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Dict, Iterable, Iterator, List, Optional, Protocol, Set, Tuple, TypeVar
|
from typing import Any, Dict, Iterable, Iterator, List, Optional, Protocol, Set, Tuple, TypeVar
|
||||||
|
|
||||||
from md_types import BlockEvent, BlockStyle, FrontMatter, StyleUpdateEvent
|
from ..models import BlockEvent, BlockStyle, FrontMatter, StyleUpdateEvent
|
||||||
|
|
||||||
|
|
||||||
FRONTMATTER_PATTERN = re.compile(r"^---\s*$")
|
FRONTMATTER_PATTERN = re.compile(r"^---\s*$")
|
||||||
5
src/md2txt/parsers/__init__.py
Normal file
5
src/md2txt/parsers/__init__.py
Normal file
|
|
@ -0,0 +1,5 @@
|
||||||
|
"""Bundled parser implementations."""
|
||||||
|
|
||||||
|
from .markdown import MarkdownParser
|
||||||
|
|
||||||
|
__all__ = ["MarkdownParser"]
|
||||||
|
|
@ -5,8 +5,8 @@ import re
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Dict, Iterable, Iterator, List, Optional, Union
|
from typing import Dict, Iterable, Iterator, List, Optional, Union
|
||||||
|
|
||||||
from conversion_core import ASCII_SENTINEL_PREFIX
|
from ..conversion.core import ASCII_SENTINEL_PREFIX
|
||||||
from md_types import (
|
from ..models import (
|
||||||
AsciiArtPayload,
|
AsciiArtPayload,
|
||||||
AsciiArtPiece,
|
AsciiArtPiece,
|
||||||
BlockEvent,
|
BlockEvent,
|
||||||
|
|
@ -2,8 +2,8 @@ from __future__ import annotations
|
||||||
|
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from conversion_core import ParserFactory, RendererFactory
|
from ..conversion.core import ParserFactory, RendererFactory
|
||||||
from plugin_registry import PluginRegistry
|
from .registry import PluginRegistry
|
||||||
|
|
||||||
|
|
||||||
parser_plugins = PluginRegistry[ParserFactory]()
|
parser_plugins = PluginRegistry[ParserFactory]()
|
||||||
|
|
@ -32,3 +32,13 @@ def available_parsers() -> list[str]:
|
||||||
|
|
||||||
def available_renderers() -> list[str]:
|
def available_renderers() -> list[str]:
|
||||||
return renderer_plugins.names()
|
return renderer_plugins.names()
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"available_parsers",
|
||||||
|
"available_renderers",
|
||||||
|
"get_parser_factory",
|
||||||
|
"get_renderer_factory",
|
||||||
|
"register_parser",
|
||||||
|
"register_renderer",
|
||||||
|
]
|
||||||
7
src/md2txt/renderers/__init__.py
Normal file
7
src/md2txt/renderers/__init__.py
Normal file
|
|
@ -0,0 +1,7 @@
|
||||||
|
"""Bundled renderer implementations."""
|
||||||
|
|
||||||
|
from .ama import AmaRenderer
|
||||||
|
from .micron import MicronRenderer
|
||||||
|
from .text import TextRenderer
|
||||||
|
|
||||||
|
__all__ = ["AmaRenderer", "MicronRenderer", "TextRenderer"]
|
||||||
|
|
@ -5,7 +5,7 @@ from functools import partial
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Callable, List
|
from typing import Any, Callable, List
|
||||||
|
|
||||||
from md_types import (
|
from ..models import (
|
||||||
AsciiArtPayload,
|
AsciiArtPayload,
|
||||||
BlockQuotePayload,
|
BlockQuotePayload,
|
||||||
BlockStyle,
|
BlockStyle,
|
||||||
|
|
@ -16,8 +16,8 @@ from md_types import (
|
||||||
ParagraphPayload,
|
ParagraphPayload,
|
||||||
StyleSpec,
|
StyleSpec,
|
||||||
)
|
)
|
||||||
from plugins import register_renderer
|
from ..plugins import register_renderer
|
||||||
from text_renderer import (
|
from .text import (
|
||||||
BOLD_RE,
|
BOLD_RE,
|
||||||
CODE_STASH_RE,
|
CODE_STASH_RE,
|
||||||
IMAGE_RE,
|
IMAGE_RE,
|
||||||
|
|
@ -3,9 +3,18 @@ from __future__ import annotations
|
||||||
from functools import partial
|
from functools import partial
|
||||||
from typing import Any, List
|
from typing import Any, List
|
||||||
|
|
||||||
from md_types import AsciiArtPayload, BlockQuotePayload, BlockStyle, CodeBlockPayload, FrontMatter, HeadingPayload, ListItemPayload, ParagraphPayload
|
from ..models import (
|
||||||
from plugins import register_renderer
|
AsciiArtPayload,
|
||||||
from text_renderer import (
|
BlockQuotePayload,
|
||||||
|
BlockStyle,
|
||||||
|
CodeBlockPayload,
|
||||||
|
FrontMatter,
|
||||||
|
HeadingPayload,
|
||||||
|
ListItemPayload,
|
||||||
|
ParagraphPayload,
|
||||||
|
)
|
||||||
|
from ..plugins import register_renderer
|
||||||
|
from .text import (
|
||||||
BOLD_RE,
|
BOLD_RE,
|
||||||
CODE_STASH_RE,
|
CODE_STASH_RE,
|
||||||
IMAGE_RE,
|
IMAGE_RE,
|
||||||
|
|
@ -7,7 +7,7 @@ from dataclasses import dataclass
|
||||||
from functools import partial
|
from functools import partial
|
||||||
from typing import Callable, Dict, List, Optional, Tuple
|
from typing import Callable, Dict, List, Optional, Tuple
|
||||||
|
|
||||||
from md_types import (
|
from ..models import (
|
||||||
AsciiArtPayload,
|
AsciiArtPayload,
|
||||||
AsciiArtPiece,
|
AsciiArtPiece,
|
||||||
BlockEvent,
|
BlockEvent,
|
||||||
4
uv.lock
generated
4
uv.lock
generated
|
|
@ -3,9 +3,9 @@ revision = 3
|
||||||
requires-python = ">=3.13"
|
requires-python = ">=3.13"
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "md2amb"
|
name = "md2txt"
|
||||||
version = "0.1.0"
|
version = "0.1.0"
|
||||||
source = { virtual = "." }
|
source = { editable = "." }
|
||||||
dependencies = [
|
dependencies = [
|
||||||
{ name = "pyfiglet" },
|
{ name = "pyfiglet" },
|
||||||
{ name = "pyphen" },
|
{ name = "pyphen" },
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue