packaged
This commit is contained in:
parent
8cdedd43d6
commit
5067b9ec65
21 changed files with 114 additions and 566 deletions
43
README.md
43
README.md
|
|
@ -1,11 +1,10 @@
|
|||
# md2amb utilities
|
||||
# md2txt
|
||||
|
||||
This repository contains command line helpers for transforming Markdown into formats that work well on retro hardware or constrained text viewers. A small plugin API lets you mix-and-match parsers and renderers so additional formats can plug into the same preprocessing pipeline.
|
||||
This repository contains the `md2txt` command line tool and supporting libraries for transforming Markdown into formats that work well on retro hardware or constrained text viewers. A small plugin API lets you mix-and-match parsers and renderers so additional formats can plug into the same preprocessing pipeline.
|
||||
|
||||
## Tools
|
||||
## CLI
|
||||
|
||||
- `md2amb.py` – converts Markdown into Amber-screen formatted text (see script for details).
|
||||
- `md2txt.py` – converts Markdown into 80-column, DOS-compatible plain text with extensive formatting support. It ships with the default `markdown` parser and `text` renderer plugins, registers optional `micron` and `ama` renderers for Micron/Ancient Machine Book output, and exposes the core pipeline so you can add your own parser or renderer modules:
|
||||
- `md2txt` – converts Markdown into 80-column, DOS-compatible plain text with extensive formatting support. It ships with the default `markdown` parser and `text` renderer plugins, registers optional `micron` and `ama` renderers for Micron/Ancient Machine Book output, and exposes the core pipeline so you can add your own parser or renderer modules:
|
||||
- FIGlet-rendered headings (H1–H3) driven by optional YAML frontmatter (`h1_font`, `h2_font`, `h3_font`).
|
||||
- H4+ headings rendered in uppercase with dashed underlines.
|
||||
- Emphasis styles converted to spaced or delimited characters, e.g. `**bold**` → `B O L D`, `__strong__` → `_s_t_r_o_n_g_`, `~~strike~~` → `~s~t~r~i~k~e~`.
|
||||
|
|
@ -13,7 +12,6 @@ This repository contains command line helpers for transforming Markdown into for
|
|||
- Code blocks numbered (`01 | line`), with both fenced and indented fences supported (see “Rendering Controls” for customisation).
|
||||
- Links transformed into `[label](n)` references, with footnote-style URL list at the end.
|
||||
- Optional alignment and margin controls via HTML `<p>` attributes or MultiMarkdown attribute blocks (e.g. `{:.center margin=20px}`) applied as leading spaces.
|
||||
- Recursive file includes using `![[file.md]]` or `{.include file.md}` (frontmatter inside the included file is ignored).
|
||||
- ASCII art injection with `#[label :align](art.txt)` syntax, supporting multiple art pieces per line and optional `{: .right}` style annotations.
|
||||
- Per-document toggles for code block wrapping, numbering, blockquote decoration, and list indentation spacing.
|
||||
|
||||
|
|
@ -26,21 +24,18 @@ This repository contains command line helpers for transforming Markdown into for
|
|||
## Usage
|
||||
|
||||
```bash
|
||||
python md2txt.py input.md -o output.txt # convert to DOS-friendly text
|
||||
python md2txt.py input.md # write result to stdout
|
||||
python md2txt.py input.md --width 72 # override column width
|
||||
python md2txt.py input.md --parser markdown --renderer micron # emit Micron-formatted output
|
||||
python md2txt.py input.md --renderer ama # emit AMB/AMA markup
|
||||
python md2txt.py input.md --renderer-option width=68 # pass KEY=VALUE to a renderer
|
||||
md2txt input.md -o output.txt # convert to DOS-friendly text
|
||||
md2txt input.md # write result to stdout
|
||||
md2txt input.md --width 72 # override column width
|
||||
md2txt input.md --parser markdown --renderer micron # emit Micron-formatted output
|
||||
md2txt input.md --renderer ama # emit AMB/AMA markup
|
||||
md2txt input.md --renderer-option width=68 # pass KEY=VALUE to a renderer
|
||||
|
||||
# If the project is not installed yet:
|
||||
python -m md2txt input.md
|
||||
```
|
||||
|
||||
- `md2amb.py` – package Markdown (and linked Markdown files) into a self-contained `.amb` archive composed of `.ama` articles that honour the 78-column/64 KiB AMA constraints.
|
||||
|
||||
```bash
|
||||
python md2amb.py --title "Your Manual" docs/index.md output/manual.amb
|
||||
```
|
||||
|
||||
`--parser` and `--renderer` select a plugin by name (defaults are `markdown` and `text`). Repeatable `--parser-option KEY=VALUE` and `--renderer-option KEY=VALUE` pairs are forwarded to the plugin factories as keyword arguments in addition to the defaults supplied by the CLI. Both scripts accept `--help` for the full option list.
|
||||
`--parser` and `--renderer` select a plugin by name (defaults are `markdown` and `text`). Repeatable `--parser-option KEY=VALUE` and `--renderer-option KEY=VALUE` pairs are forwarded to the plugin factories as keyword arguments in addition to the defaults supplied by the CLI. The CLI accepts `--help` for the full option list.
|
||||
|
||||
## FIGlet Fonts via Frontmatter
|
||||
|
||||
|
|
@ -126,15 +121,15 @@ Generated text uses CRLF line endings to maintain DOS compatibility.
|
|||
|
||||
## Plugin Architecture
|
||||
|
||||
The conversion pipeline lives in `conversion_core.py` and is exposed via `run_conversion`. It is designed around lightweight factories:
|
||||
The conversion pipeline lives in `src/md2txt/conversion/core.py` and is exposed via `run_conversion`. It is designed around lightweight factories:
|
||||
|
||||
- **Parser factories** receive `base_style: BlockStyle` plus any extra keyword arguments and must return an object with a `parse(lines: Iterable[str]) -> Iterator[BlockEvent | StyleUpdateEvent]` method. The bundled `MarkdownParser` implements this interface.
|
||||
- **Renderer factories** receive `frontmatter: FrontMatter` and arbitrary keyword arguments and must return an object that provides `handle_event(event)` and `finalize() -> Any`. The `TextRenderer` returns a list of DOS-friendly output lines; other renderers may return any data appropriate for their target format.
|
||||
|
||||
Plugins register themselves through the helpers in `plugins.py`:
|
||||
Plugins register themselves through the helpers in `md2txt.plugins`:
|
||||
|
||||
```python
|
||||
from plugins import register_parser, register_renderer
|
||||
from md2txt.plugins import register_parser, register_renderer
|
||||
|
||||
def my_parser_factory(*, base_style, **options):
|
||||
return MyParser(base_style, **options)
|
||||
|
|
@ -149,6 +144,6 @@ def my_renderer_factory(*, frontmatter, **options):
|
|||
register_renderer("ansi", my_renderer_factory)
|
||||
```
|
||||
|
||||
Once registered (for example in a small module that imports `md2txt.py`), the new plugins are available via `--parser my-markdown` or `--renderer ansi`. The CLI lists registered plugin names in `--help`, and the helper functions `available_parsers()` / `available_renderers()` return the sorted names if you need to build higher-level tooling.
|
||||
Once registered (for example in a small module that imports `md2txt.cli`), the new plugins are available via `--parser my-markdown` or `--renderer ansi`. The CLI lists registered plugin names in `--help`, and the helper functions `available_parsers()` / `available_renderers()` return the sorted names if you need to build higher-level tooling.
|
||||
|
||||
The shared preprocessing helpers—YAML frontmatter parsing, recursive include expansion, and ASCII art sentinels—also live in `conversion_core.py`, allowing alternate front-ends to reuse exactly the same behaviour without duplicating code.
|
||||
The shared preprocessing helpers—YAML frontmatter parsing, recursive include expansion, and ASCII art sentinels—also live in `src/md2txt/conversion/core.py`, allowing alternate front-ends to reuse exactly the same behaviour without duplicating code.
|
||||
|
|
|
|||
BIN
lorem.amb
BIN
lorem.amb
Binary file not shown.
237
loremipsum.mu
237
loremipsum.mu
|
|
@ -1,237 +0,0 @@
|
|||
|
||||
> Salientia
|
||||
|
||||
>> Structa hinc vetitus factum
|
||||
|
||||
Lorem markdownum omnes tellus, in, Herculeis
|
||||
`=reciprocalPower`= neque promissa est Latiae caput
|
||||
umero talia fidissima, nymphae. Foedera et sagittas
|
||||
tenetur adde matri esse: talia manibus.
|
||||
|
||||
>> Fuit video longique dedisse
|
||||
|
||||
Ille stupet, ultra `!incunabula agendum urbes`! mirer
|
||||
`!aper`!, metu. Aegides novissima sunt turis quatiebant
|
||||
umbrosaque `[putares`#mille-parva]. Metuunt eruiturque
|
||||
maximus velit, ater Vesta vulnus rustica verba! Mei
|
||||
vincite nulla, in Iovem: fama exurunt vernum!
|
||||
|
||||
De instat ubi induit fugientia inertes columbas tarda
|
||||
`*terga Parnasia sententia`* omnia inmotusque, est et
|
||||
aetas sonus. Exhibuit coeperunt, canis. Suis minora,
|
||||
tempore fateor neve moderamine erit. Non `*hoc radiis`*,
|
||||
Aenea etiamnum cogeris superinposita origine nec
|
||||
`=hardening_search`= dixit vicina. Heros subito
|
||||
populifer adeunt quicquam cum `!ales`! Ixion angulus
|
||||
placet reddere.
|
||||
|
||||
>> Tractaque serpens arva quoque
|
||||
|
||||
> Include
|
||||
|
||||
<>=======()
|
||||
(/\___ /|\\ ()==========<>_
|
||||
\_/ | \\ //|\ ______/ \)
|
||||
\_| \\ // | \_/
|
||||
\|\/|\_ // /\/
|
||||
(oo)\ \_// /
|
||||
//_/\_\/ / |
|
||||
@@/ |=\ \ |
|
||||
\_=\_ \ |
|
||||
\==\ \|\_ snd
|
||||
__(\===\( )\
|
||||
(((~) __(_/ |
|
||||
(((~) \ /
|
||||
______/ /
|
||||
'------'
|
||||
<>=======()
|
||||
(/\___ /|\\ ()==========<>_
|
||||
\_/ | \\ //|\ ______/ \)
|
||||
\_| \\ // | \_/
|
||||
\|\/|\_ // /\/
|
||||
(oo)\ \_// /
|
||||
//_/\_\/ / |
|
||||
@@/ |=\ \ |
|
||||
\_=\_ \ |
|
||||
\==\ \|\_ snd
|
||||
__(\===\( )\
|
||||
(((~) __(_/ |
|
||||
(((~) \ /
|
||||
______/ /
|
||||
'------'
|
||||
<>=======()
|
||||
(/\___ /|\\ ()==========<>_
|
||||
\_/ | \\ //|\ ______/ \)
|
||||
\_| \\ // | \_/
|
||||
\|\/|\_ // /\/
|
||||
(oo)\ \_// /
|
||||
//_/\_\/ / |
|
||||
@@/ |=\ \ |
|
||||
\_=\_ \ |
|
||||
\==\ \|\_ snd
|
||||
__(\===\( )\
|
||||
(((~) __(_/ |
|
||||
(((~) \ /
|
||||
______/ /
|
||||
'------'
|
||||
|
||||
> h1 Heading 8-)
|
||||
|
||||
>> h2 Heading
|
||||
|
||||
>>> h3 Heading
|
||||
|
||||
>>> h4 Heading
|
||||
|
||||
>>> h5 Heading
|
||||
|
||||
>>> h6 Heading
|
||||
|
||||
>> Horizontal Rules
|
||||
|
||||
-
|
||||
|
||||
-
|
||||
|
||||
-
|
||||
|
||||
>> Emphasis
|
||||
|
||||
`!This is bold text`! `!This is bold text`! `*This is
|
||||
italic text`* `*This is italic text`* ~~Strikethrough~~
|
||||
|
||||
>> Blockquotes
|
||||
|
||||
>>>>Blockquotes can also be nested...
|
||||
|
||||
>>>>>>>>by using additional greater-than signs right
|
||||
>>>>>>>>next to each other...
|
||||
|
||||
>>>>>>>>>>>>or with spaces between arrows.
|
||||
|
||||
>> Lists
|
||||
|
||||
>>> Unordered
|
||||
|
||||
+ Create a list by starting a line with `=+`=, `=-`=, or
|
||||
`=*`=
|
||||
- Marker character change forces new list start:
|
||||
* Ac tristique libero volutpat at
|
||||
+ Facilisis in pretium nisl aliquet
|
||||
- Nulla volutpat aliquam velit
|
||||
+ Very easy
|
||||
>>> Ordered
|
||||
|
||||
1. Lorem ipsum dolor sit amet
|
||||
2. Consectetur adipiscing elit
|
||||
3. Integer molestie lorem at massa
|
||||
>> Code
|
||||
|
||||
Inline `=code`=
|
||||
|
||||
Indented code
|
||||
|
||||
`=
|
||||
// Some comments
|
||||
line 1 of code
|
||||
line 2 of code
|
||||
line 3 of code
|
||||
`=
|
||||
|
||||
Block code "fences"
|
||||
|
||||
`=
|
||||
Sample text here...
|
||||
`=
|
||||
|
||||
Syntax highlighting
|
||||
|
||||
`=
|
||||
var foo = function (bar) {
|
||||
return bar++;
|
||||
};
|
||||
|
||||
console.log(foo(5));
|
||||
`=
|
||||
|
||||
>> Links
|
||||
|
||||
`[link text`http://dev.nodeca.com]
|
||||
|
||||
`[link with title`http://nodeca.github.io/pica/demo/
|
||||
"title text!"]
|
||||
|
||||
>> Images
|
||||
|
||||
`[Minion`https://octodex.github.com/images/minion.png] `
|
||||
[Stormtroopocat`https://octodex.github.com/images/stormt
|
||||
roopocat.jpg "The Stormtroopocat"]
|
||||
|
||||
Inprudens dum memor alma, casses dedi Sinuessa iam quid
|
||||
adicit `=gate_truncate_wave`=. `[Geratur de`#salientia-
|
||||
iungit-contra-et], insolida temptat `=open`= carentia;
|
||||
genitorem solet, potens. Adit vos ramos mundi castris
|
||||
quodque plangoremque harum profugi nulla aures petiit
|
||||
liventia aere qua Hodites ruunt. Achaidos dilexisse
|
||||
`=redundancy`= praesagia tenuatus necis `!Capys
|
||||
recordor`! tenebras actis signis, reliquit, placidissime
|
||||
iram liquefactis longe nervi famem. Cura pars natique
|
||||
velleris semel et at `=printerWildcardParallel`= inquit
|
||||
positosque comites ultra.
|
||||
|
||||
>> Quantaque augent
|
||||
|
||||
Acuti ignotis ignis, ostendit nostras terras nondum,
|
||||
`[petit utque utero`#structa-hinc-vetitus-factum]
|
||||
sepulcrales, pluma reseratis Ismario. Tria silentia duae
|
||||
`*nymphe`*: certe parva cernam et dixit. `[Verborum
|
||||
sedisti`#mille-parva].
|
||||
|
||||
`=
|
||||
if (firewall(4, 3) == 1 * device_media_parity) {
|
||||
system_copy_class += systray + oasis_wi_clean + lpi;
|
||||
modelLdap.search_secondary_socket *= bounce.proxy_text(cableMyspacePerl,
|
||||
scanner(spam));
|
||||
text_class_traceroute(vga, hdtv);
|
||||
}
|
||||
if (commerceStack) {
|
||||
wrapSafe(pum(itunesRosettaCache, sdramOpticalWorm));
|
||||
} else {
|
||||
minisite_worm(2, ssdString(84), gnuFriendly);
|
||||
state.qwerty(moduleJavascriptFloating + ugc, koffice,
|
||||
mail_system.prebinding_service_teraflops(mediaIcmpBoot));
|
||||
}
|
||||
memoryHexadecimalMultitasking += oop_android;
|
||||
var passive_vdu = bar;
|
||||
var ddrJava = leaf(unc_web);
|
||||
`=
|
||||
|
||||
>>> Mille parva
|
||||
|
||||
`!Minetur et`! aurum! Charaxi sanguine quo; vires
|
||||
subnectite `!pignora`!, omnique hostibus loquerentur
|
||||
aquae Solis ferox? `!Cum ne`!, putat utar ex ad aede
|
||||
mane. Vitalesque geminas exstimulat, iam vivunt dat
|
||||
artes et quem! Iam violas molire ipse, spoliabitur dente
|
||||
nervis regis, factos quae.
|
||||
|
||||
>>>>Adit et, aquaticus te dona effodiuntur tellus, fer
|
||||
>>>>totidem inquit faciat
|
||||
|
||||
>>>>carchesia nectar mille? Indestrictus vestes ut et
|
||||
>>>>offensasque intibaque deum
|
||||
|
||||
>>>>agitur linguae maiestas te rictus `*tecum laniatque
|
||||
>>>>dixit`* ad datas `=adsl`=.
|
||||
|
||||
>>>>Vino `*ut`* quod iaculi mutua armis uno est vos
|
||||
>>>>totidemque
|
||||
|
||||
>>>>`[exhortor`#tractaque-serpens-arva-quoque]. Iamdudum
|
||||
>>>>Romanae cepi linguisque
|
||||
|
||||
>>>>velit accipe idem acutis hoc sparsumque dextra.
|
||||
>>>>Minoide sponte collo sequitur
|
||||
|
||||
>>>>et intus flere sanguineaque herba, nox Quas adsiduo
|
||||
>>>>`*cunctantem`* tibi; enim.
|
||||
282
md2amb.py
282
md2amb.py
|
|
@ -1,282 +0,0 @@
|
|||
#!/usr/bin/env python3
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import re
|
||||
import struct
|
||||
from collections import deque
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Dict, Iterable, List, Tuple
|
||||
|
||||
import ama_renderer # noqa: F401 - ensure AMA renderer plugin registration
|
||||
from conversion_core import parse_frontmatter, run_conversion
|
||||
from markdown_parser import MarkdownParser
|
||||
from md_types import BlockStyle, FrontMatter
|
||||
from plugins import get_parser_factory, get_renderer_factory, register_parser
|
||||
from text_renderer import TextRenderer
|
||||
|
||||
# Ensure markdown parser registered for standalone usage
|
||||
|
||||
|
||||
def _markdown_parser_factory(*, base_style: BlockStyle, **_: object) -> MarkdownParser:
|
||||
return MarkdownParser(base_style)
|
||||
|
||||
|
||||
try:
|
||||
register_parser("markdown", _markdown_parser_factory)
|
||||
except ValueError:
|
||||
pass
|
||||
|
||||
|
||||
MARKDOWN_LINK_RE = re.compile(r"(\[[^\]]*\]\()([^)]+)(\))")
|
||||
LOCAL_LINK_RE = re.compile(r"^[A-Za-z0-9_.~/\\-]+$")
|
||||
EXT_MD = {".md", ".markdown", ".mkd", ".mkdn"}
|
||||
AMA_MAX_BYTES = 65_535
|
||||
AMB_MAGIC = b"AMB1"
|
||||
LINK_CONTINUE_LABEL = "Continue"
|
||||
|
||||
|
||||
@dataclass
|
||||
class Article:
|
||||
source: Path
|
||||
ama_name: str
|
||||
|
||||
|
||||
def main(argv: Iterable[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(description="Convert Markdown into an AMB archive.")
|
||||
parser.add_argument("input", type=Path, help="Root Markdown file to convert.")
|
||||
parser.add_argument("output", type=Path, help="Output AMB filename.")
|
||||
parser.add_argument("--title", type=str, help="Optional book title.")
|
||||
args = parser.parse_args(list(argv) if argv is not None else None)
|
||||
|
||||
input_path = args.input.resolve()
|
||||
if not input_path.exists():
|
||||
parser.error(f"Input file '{input_path}' does not exist.")
|
||||
|
||||
amb_bytes = build_amb(
|
||||
root_markdown=input_path,
|
||||
title=args.title,
|
||||
)
|
||||
args.output.parent.mkdir(parents=True, exist_ok=True)
|
||||
args.output.write_bytes(amb_bytes)
|
||||
print(str(args.output))
|
||||
return 0
|
||||
|
||||
|
||||
def build_amb(root_markdown: Path, title: str | None) -> bytes:
|
||||
articles = collect_articles(root_markdown)
|
||||
ama_contents = render_articles(articles)
|
||||
files = assemble_files(ama_contents, title)
|
||||
return pack_amb(files)
|
||||
|
||||
|
||||
def collect_articles(root_markdown: Path) -> Dict[Path, Article]:
|
||||
queue: deque[Path] = deque([root_markdown])
|
||||
visited: Dict[Path, Article] = {}
|
||||
assigned_names: set[str] = set()
|
||||
|
||||
while queue:
|
||||
current = queue.popleft()
|
||||
current = current.resolve()
|
||||
if current in visited:
|
||||
continue
|
||||
if not current.exists():
|
||||
raise FileNotFoundError(f"Referenced file '{current}' was not found.")
|
||||
if current == root_markdown:
|
||||
ama_name = "INDEX.AMA"
|
||||
else:
|
||||
ama_name = assign_ama_name(current.stem, assigned_names)
|
||||
assigned_names.add(ama_name)
|
||||
visited[current] = Article(source=current, ama_name=ama_name)
|
||||
|
||||
for linked in find_local_markdown_links(current):
|
||||
queue.append(linked)
|
||||
|
||||
return visited
|
||||
|
||||
|
||||
def find_local_markdown_links(markdown_path: Path) -> List[Path]:
|
||||
text = markdown_path.read_text(encoding="utf-8")
|
||||
results: List[Path] = []
|
||||
|
||||
for _, target, _ in MARKDOWN_LINK_RE.findall(text):
|
||||
cleaned = target.strip()
|
||||
if not cleaned or cleaned.startswith("#"):
|
||||
continue
|
||||
if "://" in cleaned or cleaned.startswith(("mailto:", "ftp:", "gopher:", "tel:")):
|
||||
continue
|
||||
resolved = (markdown_path.parent / cleaned.split("#", 1)[0]).resolve()
|
||||
if resolved.suffix.lower() in EXT_MD:
|
||||
results.append(resolved)
|
||||
return results
|
||||
|
||||
|
||||
def assign_ama_name(stem: str, existing: set[str]) -> str:
|
||||
base = "".join((c if c.isalnum() else "_") for c in stem.upper())
|
||||
if not base:
|
||||
base = "ARTICLE"
|
||||
if base[0].isdigit():
|
||||
base = f"_{base}"
|
||||
base = base[:8]
|
||||
|
||||
name = f"{base}.AMA"
|
||||
counter = 1
|
||||
while name in existing:
|
||||
suffix = f"{counter:02d}"
|
||||
trimmed = base[: max(1, 8 - len(suffix))]
|
||||
name = f"{trimmed}{suffix}.AMA"
|
||||
counter += 1
|
||||
return name
|
||||
|
||||
|
||||
def render_articles(articles: Dict[Path, Article]) -> Dict[str, List[str]]:
|
||||
parser_factory = get_parser_factory("markdown")
|
||||
renderer_factory = get_renderer_factory("ama")
|
||||
rendered: Dict[str, List[str]] = {}
|
||||
|
||||
for path, article in articles.items():
|
||||
content = path.read_text(encoding="utf-8")
|
||||
rewritten = rewrite_links(content, path.parent, articles)
|
||||
frontmatter, body_lines = parse_frontmatter(rewritten.splitlines(keepends=True))
|
||||
ama_lines = run_conversion(
|
||||
body_lines,
|
||||
frontmatter=frontmatter,
|
||||
parser_factory=parser_factory,
|
||||
renderer_factory=renderer_factory,
|
||||
renderer_options={"width": 78},
|
||||
base_path=path.parent,
|
||||
)
|
||||
split_articles = split_article(article.ama_name, ama_lines)
|
||||
rendered.update(split_articles)
|
||||
return rendered
|
||||
|
||||
|
||||
def rewrite_links(markdown: str, base_dir: Path, articles: Dict[Path, Article]) -> str:
|
||||
def replacer(match: re.Match[str]) -> str:
|
||||
prefix, target, suffix = match.groups()
|
||||
cleaned = target.strip()
|
||||
candidate = (base_dir / cleaned.split("#", 1)[0]).resolve()
|
||||
if candidate in articles:
|
||||
mapped = articles[candidate].ama_name
|
||||
return f"{prefix}{mapped}{suffix}"
|
||||
return match.group(0)
|
||||
|
||||
return MARKDOWN_LINK_RE.sub(replacer, markdown)
|
||||
|
||||
|
||||
def split_article(filename: str, lines: List[str]) -> Dict[str, List[str]]:
|
||||
encoded = "\n".join(lines).encode("utf-8")
|
||||
if len(encoded) <= AMA_MAX_BYTES:
|
||||
return {filename: lines}
|
||||
|
||||
segments: List[List[str]] = []
|
||||
current: List[str] = []
|
||||
current_size = 0
|
||||
|
||||
def flush_segment() -> None:
|
||||
nonlocal current, current_size
|
||||
if current:
|
||||
segments.append(current)
|
||||
current = []
|
||||
current_size = 0
|
||||
|
||||
for line in lines:
|
||||
candidate_size = current_size + len((line + "\n").encode("utf-8"))
|
||||
if candidate_size > AMA_MAX_BYTES and current:
|
||||
flush_segment()
|
||||
current.append(line)
|
||||
current_size += len((line + "\n").encode("utf-8"))
|
||||
flush_segment()
|
||||
|
||||
result: Dict[str, List[str]] = {}
|
||||
stem = Path(filename).stem
|
||||
generated_names = [filename]
|
||||
|
||||
for idx in range(1, len(segments)):
|
||||
suffix = f"{idx:02d}"
|
||||
trimmed = stem[: max(1, 8 - len(suffix))]
|
||||
new_name = f"{trimmed}{suffix}.AMA"
|
||||
counter = 1
|
||||
while new_name in result or new_name in generated_names:
|
||||
suffix = f"{idx:02d}{counter}"
|
||||
trimmed = stem[: max(1, 8 - len(suffix))]
|
||||
new_name = f"{trimmed}{suffix}.AMA"
|
||||
counter += 1
|
||||
generated_names.append(new_name)
|
||||
|
||||
for name, segment in zip(generated_names, segments, strict=False):
|
||||
result[name] = segment[:]
|
||||
|
||||
for idx, name in enumerate(generated_names[:-1]):
|
||||
next_name = generated_names[idx + 1]
|
||||
result[name].append("")
|
||||
result[name].append(f"%l{next_name}:{LINK_CONTINUE_LABEL}%t")
|
||||
return result
|
||||
|
||||
|
||||
def assemble_files(ama_contents: Dict[str, List[str]], title: str | None) -> List[Tuple[str, bytes]]:
|
||||
files: List[Tuple[str, bytes]] = []
|
||||
if title:
|
||||
files.append(("TITLE", title.encode("ascii", "ignore")[:64]))
|
||||
|
||||
index_bytes = encode_ama("INDEX.AMA", ama_contents.pop("INDEX.AMA"))
|
||||
files.append(("INDEX.AMA", index_bytes))
|
||||
|
||||
for name, lines in sorted(ama_contents.items()):
|
||||
files.append((name, encode_ama(name, lines)))
|
||||
|
||||
return files
|
||||
|
||||
|
||||
def encode_ama(name: str, lines: List[str]) -> bytes:
|
||||
content = "\n".join(lines).rstrip("\n") + "\n"
|
||||
data = content.encode("utf-8")
|
||||
if len(data) > AMA_MAX_BYTES:
|
||||
raise ValueError(f"Generated AMA article '{name}' exceeds {AMA_MAX_BYTES} bytes.")
|
||||
if any("\t" in line for line in lines):
|
||||
raise ValueError(f"Generated AMA article '{name}' contains tab characters.")
|
||||
return data
|
||||
|
||||
|
||||
def pack_amb(files: List[Tuple[str, bytes]]) -> bytes:
|
||||
entries = []
|
||||
offset = 6 + 20 * len(files)
|
||||
payloads = []
|
||||
|
||||
for filename, data in files:
|
||||
canonical = filename.upper()
|
||||
if len(canonical) > 12:
|
||||
raise ValueError(f"Filename '{canonical}' does not fit 8.3 constraints.")
|
||||
payloads.append(data)
|
||||
checksum = bsd_checksum(data)
|
||||
entries.append((canonical, offset, len(data), checksum))
|
||||
offset += len(data)
|
||||
|
||||
output = bytearray()
|
||||
output.extend(AMB_MAGIC)
|
||||
output.extend(struct.pack("<H", len(entries)))
|
||||
|
||||
for name, file_offset, length, checksum in entries:
|
||||
padded = name.encode("ascii", "ignore")
|
||||
padded = padded + b"\x00" * (12 - len(padded))
|
||||
output.extend(padded)
|
||||
output.extend(struct.pack("<I", file_offset))
|
||||
output.extend(struct.pack("<H", length))
|
||||
output.extend(struct.pack("<H", checksum))
|
||||
|
||||
for data in payloads:
|
||||
output.extend(data)
|
||||
return bytes(output)
|
||||
|
||||
|
||||
def bsd_checksum(data: bytes) -> int:
|
||||
checksum = 0
|
||||
for byte in data:
|
||||
checksum = (checksum >> 1) | ((checksum & 1) << 15)
|
||||
checksum = (checksum + byte) & 0xFFFF
|
||||
return checksum
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
BIN
output.amb
BIN
output.amb
Binary file not shown.
|
|
@ -1,10 +1,20 @@
|
|||
[project]
|
||||
name = "md2amb"
|
||||
name = "md2txt"
|
||||
version = "0.1.0"
|
||||
description = "Add your description here"
|
||||
description = "Markdown to plain-text conversion toolkit"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.13"
|
||||
dependencies = [
|
||||
"pyfiglet>=0.8.0",
|
||||
"pyphen>=0.17.2",
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
md2txt = "md2txt.cli:main"
|
||||
|
||||
[build-system]
|
||||
requires = ["setuptools>=68", "wheel"]
|
||||
build-backend = "setuptools.build_meta"
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
where = ["src"]
|
||||
|
|
|
|||
5
src/md2txt/__init__.py
Normal file
5
src/md2txt/__init__.py
Normal file
|
|
@ -0,0 +1,5 @@
|
|||
"""Public package interface for md2txt."""
|
||||
|
||||
from .cli import convert_markdown, main
|
||||
|
||||
__all__ = ["convert_markdown", "main"]
|
||||
7
src/md2txt/__main__.py
Normal file
7
src/md2txt/__main__.py
Normal file
|
|
@ -0,0 +1,7 @@
|
|||
"""Entry point for `python -m md2txt`."""
|
||||
|
||||
from .cli import main
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
|
|
@ -9,10 +9,10 @@ import sys
|
|||
from pathlib import Path
|
||||
from typing import Any, Dict, Iterable, List, Optional, Tuple
|
||||
|
||||
from conversion_core import parse_frontmatter, read_lines, run_conversion
|
||||
from md_types import BlockStyle, FrontMatter
|
||||
from markdown_parser import MarkdownParser
|
||||
from plugins import (
|
||||
from .conversion.core import parse_frontmatter, read_lines, run_conversion
|
||||
from .models import BlockStyle, FrontMatter
|
||||
from .parsers.markdown import MarkdownParser
|
||||
from .plugins import (
|
||||
available_parsers,
|
||||
available_renderers,
|
||||
get_parser_factory,
|
||||
|
|
@ -20,10 +20,10 @@ from plugins import (
|
|||
register_parser,
|
||||
register_renderer,
|
||||
)
|
||||
from text_renderer import TextRenderer
|
||||
from .renderers.text import TextRenderer
|
||||
|
||||
import micron_renderer # noqa: F401 # register micron renderer plugin
|
||||
import ama_renderer # noqa: F401 # register AMA renderer plugin
|
||||
from .renderers import micron # noqa: F401 # register micron renderer plugin
|
||||
from .renderers import ama # noqa: F401 # register AMA renderer plugin
|
||||
|
||||
|
||||
def _split_option(token: str) -> Tuple[str, str]:
|
||||
19
src/md2txt/conversion/__init__.py
Normal file
19
src/md2txt/conversion/__init__.py
Normal file
|
|
@ -0,0 +1,19 @@
|
|||
"""Conversion pipeline helpers."""
|
||||
|
||||
from .core import (
|
||||
ParserFactory,
|
||||
RendererFactory,
|
||||
parse_frontmatter,
|
||||
read_lines,
|
||||
run_conversion,
|
||||
run_pipeline,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"ParserFactory",
|
||||
"RendererFactory",
|
||||
"parse_frontmatter",
|
||||
"read_lines",
|
||||
"run_conversion",
|
||||
"run_pipeline",
|
||||
]
|
||||
|
|
@ -5,7 +5,7 @@ import re
|
|||
from pathlib import Path
|
||||
from typing import Any, Dict, Iterable, Iterator, List, Optional, Protocol, Set, Tuple, TypeVar
|
||||
|
||||
from md_types import BlockEvent, BlockStyle, FrontMatter, StyleUpdateEvent
|
||||
from ..models import BlockEvent, BlockStyle, FrontMatter, StyleUpdateEvent
|
||||
|
||||
|
||||
FRONTMATTER_PATTERN = re.compile(r"^---\s*$")
|
||||
5
src/md2txt/parsers/__init__.py
Normal file
5
src/md2txt/parsers/__init__.py
Normal file
|
|
@ -0,0 +1,5 @@
|
|||
"""Bundled parser implementations."""
|
||||
|
||||
from .markdown import MarkdownParser
|
||||
|
||||
__all__ = ["MarkdownParser"]
|
||||
|
|
@ -5,8 +5,8 @@ import re
|
|||
from pathlib import Path
|
||||
from typing import Dict, Iterable, Iterator, List, Optional, Union
|
||||
|
||||
from conversion_core import ASCII_SENTINEL_PREFIX
|
||||
from md_types import (
|
||||
from ..conversion.core import ASCII_SENTINEL_PREFIX
|
||||
from ..models import (
|
||||
AsciiArtPayload,
|
||||
AsciiArtPiece,
|
||||
BlockEvent,
|
||||
|
|
@ -2,8 +2,8 @@ from __future__ import annotations
|
|||
|
||||
from typing import Any
|
||||
|
||||
from conversion_core import ParserFactory, RendererFactory
|
||||
from plugin_registry import PluginRegistry
|
||||
from ..conversion.core import ParserFactory, RendererFactory
|
||||
from .registry import PluginRegistry
|
||||
|
||||
|
||||
parser_plugins = PluginRegistry[ParserFactory]()
|
||||
|
|
@ -32,3 +32,13 @@ def available_parsers() -> list[str]:
|
|||
|
||||
def available_renderers() -> list[str]:
|
||||
return renderer_plugins.names()
|
||||
|
||||
|
||||
__all__ = [
|
||||
"available_parsers",
|
||||
"available_renderers",
|
||||
"get_parser_factory",
|
||||
"get_renderer_factory",
|
||||
"register_parser",
|
||||
"register_renderer",
|
||||
]
|
||||
7
src/md2txt/renderers/__init__.py
Normal file
7
src/md2txt/renderers/__init__.py
Normal file
|
|
@ -0,0 +1,7 @@
|
|||
"""Bundled renderer implementations."""
|
||||
|
||||
from .ama import AmaRenderer
|
||||
from .micron import MicronRenderer
|
||||
from .text import TextRenderer
|
||||
|
||||
__all__ = ["AmaRenderer", "MicronRenderer", "TextRenderer"]
|
||||
|
|
@ -5,7 +5,7 @@ from functools import partial
|
|||
from pathlib import Path
|
||||
from typing import Any, Callable, List
|
||||
|
||||
from md_types import (
|
||||
from ..models import (
|
||||
AsciiArtPayload,
|
||||
BlockQuotePayload,
|
||||
BlockStyle,
|
||||
|
|
@ -16,8 +16,8 @@ from md_types import (
|
|||
ParagraphPayload,
|
||||
StyleSpec,
|
||||
)
|
||||
from plugins import register_renderer
|
||||
from text_renderer import (
|
||||
from ..plugins import register_renderer
|
||||
from .text import (
|
||||
BOLD_RE,
|
||||
CODE_STASH_RE,
|
||||
IMAGE_RE,
|
||||
|
|
@ -3,9 +3,18 @@ from __future__ import annotations
|
|||
from functools import partial
|
||||
from typing import Any, List
|
||||
|
||||
from md_types import AsciiArtPayload, BlockQuotePayload, BlockStyle, CodeBlockPayload, FrontMatter, HeadingPayload, ListItemPayload, ParagraphPayload
|
||||
from plugins import register_renderer
|
||||
from text_renderer import (
|
||||
from ..models import (
|
||||
AsciiArtPayload,
|
||||
BlockQuotePayload,
|
||||
BlockStyle,
|
||||
CodeBlockPayload,
|
||||
FrontMatter,
|
||||
HeadingPayload,
|
||||
ListItemPayload,
|
||||
ParagraphPayload,
|
||||
)
|
||||
from ..plugins import register_renderer
|
||||
from .text import (
|
||||
BOLD_RE,
|
||||
CODE_STASH_RE,
|
||||
IMAGE_RE,
|
||||
|
|
@ -7,7 +7,7 @@ from dataclasses import dataclass
|
|||
from functools import partial
|
||||
from typing import Callable, Dict, List, Optional, Tuple
|
||||
|
||||
from md_types import (
|
||||
from ..models import (
|
||||
AsciiArtPayload,
|
||||
AsciiArtPiece,
|
||||
BlockEvent,
|
||||
4
uv.lock
generated
4
uv.lock
generated
|
|
@ -3,9 +3,9 @@ revision = 3
|
|||
requires-python = ">=3.13"
|
||||
|
||||
[[package]]
|
||||
name = "md2amb"
|
||||
name = "md2txt"
|
||||
version = "0.1.0"
|
||||
source = { virtual = "." }
|
||||
source = { editable = "." }
|
||||
dependencies = [
|
||||
{ name = "pyfiglet" },
|
||||
{ name = "pyphen" },
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue