randogoth/md2txt
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

gemini protocol gemtext support

Browse source
This commit is contained in:
randogoth 2025-10-20 18:40:59 +03:00
parent ae26bb205b
commit 315142005d
10 changed files with 620 additions and 17 deletions
Download patch file Download diff file Expand all files Collapse all files

5
README.md
View file

@ -13,12 +13,13 @@ This repository contains the `md2txt` command line tool and supporting libraries
## CLI
- `md2txt` – converts Markdown into plain text with extensive formatting support. It ships with the default `markdown` parser and `text` renderer plugins, registers optional `micron` and `ama` renderers for Micron/Ancient Machine Book output, and exposes the core pipeline so you can add your own parser or renderer modules:
- `md2txt` – converts Markdown into plain text with extensive formatting support. It ships with the default `markdown` parser and `text` renderer plugins, registers optional `micron`, `ama`, and `gemini` renderers for Micron/Ancient Machine Book or Gemtext output, and exposes the core pipeline so you can add your own parser or renderer modules:
- FIGlet-rendered headings.
- Emphasis styles converted to spaced or delimited characters.
- Optional alignment and margin controls via HTML `<p>` attributes or MultiMarkdown attribute blocks (e.g. `{:.center margin=20px}`) applied as leading spaces.
- ASCII art injection with `#[label :align](art.txt)` syntax, supporting multiple art pieces per line and optional `{: .right}` style annotations.
- Per-document toggles for code block wrapping, numbering, blockquote decoration, and list indentation spacing.
- Link references can be emitted per block by setting `links_per_block: true` in frontmatter; otherwise they collect at the end of the document. Gemini output uses `=>` lines while the text renderer keeps numbered footnotes.
## Usage
@ -28,6 +29,7 @@ md2txt input.md # write result to stdout
md2txt input.md --width 72 # override column width
md2txt input.md --renderer micron # emit Micron-formatted output
md2txt input.md --renderer ama # emit AMB/AMA markup
md2txt input.md --renderer gemini # emit Gemtext for Gemini capsules
md2txt input.md --renderer-option width=68 # pass KEY=VALUE to a renderer
```
@ -60,6 +62,7 @@ hyphen_lang: en_US
| `h#_font: <font>` | Accepts any known figlet font name or one of the keywords `caps` (force uppercase) and `title` (title case) to define headings.
| `hyphenate: true` | activates PyHyphen-based wrapping (optional `hyphen_lang` defaults to `en_US`) |
| `header_spacing` | controls how many blank lines precede headings (default `2`). |
| `links_per_block: true` | Emit link references directly below each block instead of collecting them at the end (supported by the text and Gemini renderers). |
| `list_marker_indent: <int>` | Extra spaces inserted before list markers. |
| `list_text_spacing: <int>` | Spaces between the marker and the wrapped list text. |
| `margin_left`/`margin_right` | add leading/trailing spaces to body text |

117
examples/loremipsum.gmi Normal file
View file

@ -0,0 +1,117 @@
```
___ _ _ _ _
/ __| __ _| (_)___ _ _| |_(_)__ _
\__ \/ _` | | / -_) ' \ _| / _` |
|___/\__,_|_|_\___|_||_\__|_\__,_|
```
## Structa hinc vetitus factum
Lorem markdownum omnes tellus, in, Herculeis reciprocalPower neque promissa est
Latiae caput umero talia fidissima, nymphae. Foedera et sagittas tenetur adde
matri esse: talia manibus.
## Fuit video longique dedisse
Ille stupet, ultra incunabula agendum urbes mirer aper, metu. Aegides novissima
sunt turis quatiebant umbrosaque putares [1]. Metuunt eruiturque maximus velit,
ater Vesta vulnus rustica verba! Mei vincite nulla, in Iovem: fama exurunt
vernum!
=> #mille-parva [1] putares
De instat ubi induit fugientia inertes columbas tarda terga Parnasia sententia
omnia inmotusque, est et aetas sonus. Exhibuit coeperunt, canis. Suis minora,
tempore fateor neve moderamine erit. Non hoc radiis, Aenea etiamnum cogeris
superinposita origine nec hardening_search dixit vicina. Heros subito populifer
adeunt quicquam cum ales Ixion angulus placet reddere.
## Tractaque serpens arva quoque
```
<>=======()
(/\___ /|\\ ()==========<>_
\_/ | \\ //|\ ______/ \)
\_| \\ // | \_/
\|\/|\_ // /\/
(oo)\ \_// /
//_/\_\/ / |
@@/ |=\ \ |
\_=\_ \ |
\==\ \|\_ snd
__(\===\( )\
(((~) __(_/ |
(((~) \ /
______/ /
'------'
```
Inprudens dum memor alma, casses dedi Sinuessa iam quid adicit
gate_truncate_wave. Geratur de [2], insolida temptat open carentia; genitorem
solet, potens. Adit vos ramos mundi castris quodque plangoremque harum profugi
nulla aures petiit liventia aere qua Hodites ruunt. Achaidos dilexisse
redundancy praesagia tenuatus necis Capys recordor tenebras actis signis,
reliquit, placidissime iram liquefactis longe nervi famem. Cura pars natique
velleris semel et at printerWildcardParallel inquit positosque comites ultra.
=> #salientia-iungit-contra-et [2] Geratur de
## Quantaque augent
Acuti ignotis ignis, ostendit nostras terras nondum, petit utque utero [3]
sepulcrales, pluma reseratis Ismario. Tria silentia duae nymphe: certe parva
cernam et dixit. Verborum sedisti [1].
=> #structa-hinc-vetitus-factum [3] petit utque utero
=> #mille-parva [1] putares
```
if (firewall(4, 3) == 1 * device_media_parity) {
system_copy_class += systray + oasis_wi_clean + lpi;
modelLdap.search_secondary_socket *= bounce.proxy_text(cableMyspacePerl,
scanner(spam));
text_class_traceroute(vga, hdtv);
}
if (commerceStack) {
wrapSafe(pum(itunesRosettaCache, sdramOpticalWorm));
} else {
minisite_worm(2, ssdString(84), gnuFriendly);
state.qwerty(moduleJavascriptFloating + ugc, koffice,
mail_system.prebinding_service_teraflops(mediaIcmpBoot));
}
memoryHexadecimalMultitasking += oop_android;
var passive_vdu = bar;
var ddrJava = leaf(unc_web);
```
### Mille parva
Minetur et aurum! Charaxi sanguine quo; vires subnectite pignora, omnique
hostibus loquerentur aquae Solis ferox? Cum ne, putat utar ex ad aede mane.
Vitalesque geminas exstimulat, iam vivunt dat artes et quem! Iam violas molire
ipse, spoliabitur dente nervis regis, factos quae.
> Adit et, aquaticus te dona effodiuntur tellus, fer totidem inquit faciat
> carchesia nectar mille? Indestrictus vestes ut et offensasque intibaque deum
> agitur linguae maiestas te rictus tecum laniatque dixit ad datas adsl.
> Vino ut quod iaculi mutua armis uno est vos totidemque
> exhortor [4]. Iamdudum Romanae cepi linguisque
=> #tractaque-serpens-arva-quoque [4] exhortor
> velit accipe idem acutis hoc sparsumque dextra. Minoide sponte collo sequitur
> et intus flere sanguineaque herba, nox Quas adsiduo cunctantem tibi; enim.

1
examples/loremipsum.md
View file

@ -6,6 +6,7 @@ margin_left: 4
margin_right: 4
paragraph_spacing: 2
code_block_wrap: 4
links_per_block: true
list_marker_indent: 2
list_text_spacing: 1
hyphenate: true

132
examples/markdown.gmi Normal file
View file

@ -0,0 +1,132 @@
```
_ _ _ _ _ _ ___ __
| |_ / | | || |___ __ _ __| (_)_ _ __ _ ( _ )__\ \
| ' \| | | __ / -_) _` / _` | | ' \/ _` | / _ \___| |
|_||_|_| |_||_\___\__,_\__,_|_|_||_\__, | \___/ | |
|___/ /_/
```
## h2 Heading
### h3 Heading
```
_ _ _ _ _ _ _
| |__ | || | | | | | ___ __ _ __| (_)_ __ __ _
| '_ \| || |_ | |_| |/ _ \/ _` |/ _` | | '_ \ / _` |
| | | |__ _| | _ | __/ (_| | (_| | | | | | (_| |
|_| |_| |_| |_| |_|\___|\__,_|\__,_|_|_| |_|\__, |
|___/
```
```
_ ____ _ _ _ _
| |__ | ___| | | | | ___ __ _ __| (_)_ __ __ _
| '_ \|___ \ | |_| |/ _ \/ _` |/ _` | | '_ \ / _` |
| | | |___) | | _ | __/ (_| | (_| | | | | | (_| |
|_| |_|____/ |_| |_|\___|\__,_|\__,_|_|_| |_|\__, |
|___/
```
```
_ __ _ _ _ _
| |__ / /_ | | | | ___ __ _ __| (_)_ __ __ _
| '_ \| '_ \ | |_| |/ _ \/ _` |/ _` | | '_ \ / _` |
| | | | (_) | | _ | __/ (_| | (_| | | | | | (_| |
|_| |_|\___/ |_| |_|\___|\__,_|\__,_|_|_| |_|\__, |
|___/
```
## Horizontal Rules
---
---
---
## Emphasis
This is bold text This is bold text This is italic text This is italic text
Strikethrough
## Blockquotes
> Blockquotes can also be nested...
> by using additional greater-than signs right next to each other...
> or with spaces between arrows.
## Lists
### Unordered
* Create a list by starting a line with +, -, or *
* Marker character change forces new list start:
* Ac tristique libero volutpat at
* Facilisis in pretium nisl aliquet
* Nulla volutpat aliquam velit
* Very easy
### Ordered
* Lorem ipsum dolor sit amet
* Consectetur adipiscing elit
* Integer molestie lorem at massa
## Code
Inline code
Indented code
```
// Some comments
line 1 of code
line 2 of code
line 3 of code
```
Block code "fences"
```
Sample text here...
```
Syntax highlighting
```
var foo = function (bar) {
return bar++;
};
console.log(foo(5));
```
## Links
link text [1]
link with title [2]
## Images
Minion [3] Stormtroopocat [4]
=> http://dev.nodeca.com [1] link text
=> http://nodeca.github.io/pica/demo/ "title text!" [2] link with title
=> https://octodex.github.com/images/minion.png [3] Minion
=> https://octodex.github.com/images/stormtroopocat.jpg "The Stormtroopocat" [4] Stormtroopocat

3
src/md2txt/cli.py
View file

@ -22,8 +22,9 @@ from .plugins import (
)
from .renderers.text import TextRenderer
from .renderers import micron # noqa: F401 # register micron renderer plugin
from .renderers import ama # noqa: F401 # register AMA renderer plugin
from .renderers import gemini # noqa: F401 # register Gemini renderer plugin
from .renderers import micron # noqa: F401 # register Micron renderer plugin
def _split_option(token: str) -> Tuple[str, str]:

2
src/md2txt/conversion/core.py
View file

@ -106,6 +106,7 @@ def parse_frontmatter(lines: List[str]) -> Tuple[FrontMatter, List[str]]:
blockquote_bars = _parse_bool(frontmatter.get("blockquote_bars"), True)
list_marker_indent = max(0, _parse_int(frontmatter.get("list_marker_indent"), 0))
list_text_spacing = max(0, _parse_int(frontmatter.get("list_text_spacing"), 1))
links_per_block = _parse_bool(frontmatter.get("links_per_block"), False)
fm = FrontMatter(
h1_font=frontmatter.get("h1_font", "standard").strip() or "standard",
h2_font=frontmatter.get("h2_font", "standard").strip() or "standard",
@ -123,6 +124,7 @@ def parse_frontmatter(lines: List[str]) -> Tuple[FrontMatter, List[str]]:
blockquote_bars=blockquote_bars,
list_marker_indent=list_marker_indent,
list_text_spacing=list_text_spacing,
links_per_block=links_per_block,
)
return fm, remaining

1
src/md2txt/models.py
View file

@ -48,6 +48,7 @@ class FrontMatter:
blockquote_bars: bool = True
list_marker_indent: int = 0
list_text_spacing: int = 1
links_per_block: bool = False
@dataclass

3
src/md2txt/renderers/__init__.py
View file

@ -1,7 +1,8 @@
"""Bundled renderer implementations."""
from .ama import AmaRenderer
from .gemini import GeminiRenderer
from .micron import MicronRenderer
from .text import TextRenderer
__all__ = ["AmaRenderer", "MicronRenderer", "TextRenderer"]
__all__ = ["AmaRenderer", "GeminiRenderer", "MicronRenderer", "TextRenderer"]

317
src/md2txt/renderers/gemini.py Normal file
View file

@ -0,0 +1,317 @@
from __future__ import annotations
import re
import textwrap
from typing import Any, Dict, List, Optional, Tuple
from ..models import (
AsciiArtPayload,
BlockEvent,
BlockKind,
BlockQuotePayload,
BlockStyle,
CodeBlockPayload,
FrontMatter,
HeadingPayload,
ListItemPayload,
ParagraphPayload,
StyleUpdateEvent,
)
from ..plugins import register_renderer
from .text import (
BOLD_RE,
CODE_STASH_RE,
IMAGE_RE,
ITALIC_RE,
LINK_RE,
STRIKETHROUGH_RE,
TextRenderer,
UNDERLINE_EM_RE,
UNDERLINE_STRONG_RE,
)
class GeminiRenderer:
"""Render Markdown events as Gemtext suitable for the Gemini protocol."""
def __init__(self, frontmatter: FrontMatter, *, width: int = 80, preformatted_alt: str | None = None, **_: Any) -> None:
self.frontmatter = frontmatter
self._text_renderer = TextRenderer(width=width, frontmatter=frontmatter)
self.width = max(20, width)
self.preformatted_alt = preformatted_alt or ""
self.output: List[str] = []
self._in_list: bool = False
self.links_per_block = frontmatter.links_per_block
self._link_indices: Dict[str, int] = {}
self._link_catalog: Dict[int, Tuple[str, Optional[str]]] = {}
self._pending_links: List[int] = []
def handle_event(self, event: BlockEvent | StyleUpdateEvent) -> None:
if isinstance(event, StyleUpdateEvent):
# Gemtext has no notion of per-block styling; ignore updates.
return
handler = {
BlockKind.PARAGRAPH: self._render_paragraph,
BlockKind.HEADING: self._render_heading,
BlockKind.CODE_BLOCK: self._render_code_block,
BlockKind.BLOCKQUOTE: self._render_blockquote,
BlockKind.LIST_ITEM: self._render_list_item,
BlockKind.HORIZONTAL_RULE: self._render_horizontal_rule,
BlockKind.BLANK_LINE: self._render_blank_line,
BlockKind.CUSTOM_BLOCK: self._render_custom_block,
}.get(event.kind)
if handler is not None:
handler(event.payload, event.style)
def finalize(self) -> List[str]:
self._end_list()
self._flush_pending_links(trailing_blank=False)
return self.output
# Rendering helpers -------------------------------------------------
def _render_paragraph(self, payload: ParagraphPayload, _style: BlockStyle) -> None:
self._end_list()
text, links = self._process_inline(payload.text)
if text:
self._ensure_blank_line()
self.output.extend(self._wrap_text(text))
self._handle_links(links)
def _render_heading(self, payload: HeadingPayload, style: BlockStyle) -> None:
self._end_list()
text, links = self._process_inline(payload.text)
def render_heading(target_style: BlockStyle) -> List[str]:
figlet = self._text_renderer._render_figlet_heading(payload.level, text, target_style)
if figlet:
banner = [line.rstrip() for line in figlet]
fence_header = self._preformatted_header()
lines = [fence_header, *banner, "```"]
return lines
level = max(1, min(3, payload.level))
heading = text
if not heading:
return []
marker = "#" * level
wrapped = self._wrap_text(
heading,
initial=f"{marker} ",
subsequent=f"{marker} ",
)
if not wrapped or wrapped[-1] != "":
wrapped.append("")
return wrapped
lines = render_heading(style)
if not lines:
return
self._ensure_blank_line()
self.output.extend(lines)
self._handle_links(links)
def _render_code_block(self, payload: CodeBlockPayload, _style: BlockStyle) -> None:
self._end_list()
self._ensure_blank_line()
self.output.append(self._preformatted_header())
for line in payload.lines:
self.output.append(line.rstrip("\n"))
self.output.append("```")
def _render_blockquote(self, payload: BlockQuotePayload, _style: BlockStyle) -> None:
self._end_list()
text, links = self._process_inline(payload.text)
if text:
self._ensure_blank_line()
wrapped = self._wrap_text(text, initial="> ", subsequent="> ")
self.output.extend(wrapped)
self._handle_links(links)
def _render_list_item(self, payload: ListItemPayload, _style: BlockStyle) -> None:
indent = payload.indent.replace("\t", " ")
depth = max(0, len(indent) // 2)
bullet_prefix = " " * depth + "* "
continuation = " " * depth + " "
text, links = self._process_inline(payload.text)
self._begin_list()
if text:
self.output.extend(self._wrap_text(text, initial=bullet_prefix, subsequent=continuation))
self._handle_links(links, leading_blank=False, trailing_blank=False)
def _render_horizontal_rule(self, _payload: object, _style: BlockStyle) -> None:
self._end_list()
self._ensure_blank_line()
self.output.append("---")
def _render_blank_line(self, *_: object) -> None:
self._end_list()
self._ensure_blank_line(force=True)
def _render_custom_block(self, payload: AsciiArtPayload, _style: BlockStyle) -> None:
lines: List[str] = []
for index, piece in enumerate(payload.pieces):
lines.extend(line.rstrip("\n") for line in piece.lines)
if index + 1 < len(payload.pieces):
lines.append("")
if not lines:
return
self._end_list()
self._ensure_blank_line()
self.output.append(self._preformatted_header())
self.output.extend(lines)
self.output.append("```")
# Internal utilities ------------------------------------------------
def _begin_list(self) -> None:
if not self._in_list:
self._ensure_blank_line()
self._in_list = True
def _end_list(self) -> None:
if self._in_list:
if self.output and self.output[-1] != "":
self.output.append("")
self._in_list = False
def _ensure_blank_line(self, *, force: bool = False) -> None:
if not self.output:
return
if force or self.output[-1] != "":
self.output.append("")
def _wrap_text(
self,
text: str,
*,
initial: str = "",
subsequent: Optional[str] = None,
) -> List[str]:
normalized = self._normalise_whitespace(text)
if not normalized:
return [initial.rstrip()] if initial else []
wrapper = textwrap.TextWrapper(
width=self.width,
expand_tabs=False,
replace_whitespace=False,
drop_whitespace=False,
break_long_words=False,
break_on_hyphens=True,
initial_indent=initial,
subsequent_indent=subsequent if subsequent is not None else initial,
)
return [line.rstrip() for line in wrapper.wrap(normalized)]
def _normalise_whitespace(self, text: str) -> str:
return re.sub(r"\s+", " ", text).strip()
def _register_link(self, url: str, label: Optional[str]) -> int:
key = url
index = self._link_indices.get(key)
if index is None:
index = len(self._link_indices) + 1
self._link_indices[key] = index
self._link_catalog[index] = (url, label)
else:
stored_url, stored_label = self._link_catalog.get(index, (url, None))
if label and not stored_label:
self._link_catalog[index] = (stored_url, label)
return index
def _process_inline(self, text: str) -> Tuple[str, List[int]]:
code_segments: List[str] = []
indices: List[int] = []
def stash_code(match):
segment = match.group(0)[1:-1]
code_segments.append(segment)
return f"\u0000CODE{len(code_segments) - 1}\u0000"
def register_link(label: str, target: str) -> str:
url = target.strip()
label_clean = label.strip()
display = label_clean or url
link_label = label_clean if label_clean and label_clean != url else None
index = self._register_link(url, link_label)
if index not in indices:
indices.append(index)
return f"{display} [{index}]"
text = CODE_STASH_RE.sub(stash_code, text)
text = STRIKETHROUGH_RE.sub(lambda m: m.group(1), text)
text = BOLD_RE.sub(lambda m: m.group(1), text)
text = ITALIC_RE.sub(lambda m: m.group(1), text)
text = UNDERLINE_STRONG_RE.sub(lambda m: m.group(1), text)
text = UNDERLINE_EM_RE.sub(lambda m: m.group(1), text)
text = LINK_RE.sub(lambda m: register_link(m.group(1), m.group(2)), text)
text = IMAGE_RE.sub(lambda m: register_link(m.group(1), m.group(2)), text)
for idx, code in enumerate(code_segments):
text = text.replace(f"\u0000CODE{idx}\u0000", code)
return self._normalise_whitespace(text), indices
def _handle_links(
self,
indices: List[int],
*,
leading_blank: bool = True,
trailing_blank: bool = True,
) -> None:
if not indices:
return
if self.links_per_block:
self._emit_links(indices, leading_blank=leading_blank, trailing_blank=trailing_blank)
else:
for index in indices:
if index not in self._pending_links:
self._pending_links.append(index)
def _flush_pending_links(self, *, trailing_blank: bool) -> None:
if not self._pending_links:
return
self._emit_links(self._pending_links, leading_blank=True, trailing_blank=trailing_blank)
self._pending_links.clear()
def _emit_links(
self,
indices: List[int],
*,
leading_blank: bool = True,
trailing_blank: bool = True,
) -> None:
if not indices:
return
trailing_blanks: List[str] = []
while self.output and self.output[-1] == "":
trailing_blanks.append(self.output.pop())
trailing_blanks.reverse()
if leading_blank and self.output and self.output[-1] != "":
self.output.append("")
for index in indices:
url, label = self._link_catalog.get(index, ("", None))
label_text = f"[{index}]"
if label:
label_text = f"{label_text} {label}"
line = f"=> {url}" if not label_text else f"=> {url} {label_text}"
self.output.append(line)
if trailing_blank and not trailing_blanks and self.output and self.output[-1] != "":
self.output.append("")
if trailing_blanks:
self.output.extend(trailing_blanks)
def _preformatted_header(self) -> str:
if self.preformatted_alt:
return f"``` {self.preformatted_alt}"
return "```"
def _gemini_renderer_factory(*, frontmatter: FrontMatter, **options: Any) -> GeminiRenderer:
width = int(options.get("width", 80))
alt = options.get("preformatted_alt")
return GeminiRenderer(frontmatter, width=width, preformatted_alt=alt)
try:
register_renderer("gemini", _gemini_renderer_factory)
except ValueError:
pass

48
src/md2txt/renderers/text.py
View file

@ -94,6 +94,7 @@ class TextRenderer:
)
self.list_marker_indent = max(0, frontmatter.list_marker_indent)
self.list_text_spacing = max(0, frontmatter.list_text_spacing)
self.links_per_block = frontmatter.links_per_block
self._base_style = BlockStyle(
align="left",
margin_left=max(0, frontmatter.margin_left),
@ -128,10 +129,30 @@ class TextRenderer:
self._apply_style_to_last_block(event.spec)
def finalize(self) -> List[str]:
if self.links:
if self.links_per_block:
self._flush_links_after_block(trailing_blank=False)
elif self.links:
self._emit_link_entries(self.links, trailing_blank=False)
self.links.clear()
self.link_indices.clear()
self._last_stylable_block = None
return self.output
def _handle_block_event(self, event: BlockEvent) -> None:
handler = self._handlers.get(event.kind)
if handler:
handler(event.payload, event.style)
if self.links_per_block:
self._flush_links_after_block()
else:
self._last_stylable_block = None
def _emit_link_entries(self, entries: List[tuple[int, str]], *, trailing_blank: bool) -> None:
if not entries:
return
if self.output and self.output[-1] != "":
self.output.append("")
for index, url in self.links:
for index, url in entries:
entry = f"[{index}] {url}"
self.output.extend(
self._wrap_text(
@ -141,15 +162,22 @@ class TextRenderer:
style=self._base_style,
)
)
self._last_stylable_block = None
return self.output
if trailing_blank and self.output and self.output[-1] != "":
self.output.append("")
def _handle_block_event(self, event: BlockEvent) -> None:
handler = self._handlers.get(event.kind)
if handler:
handler(event.payload, event.style)
else:
self._last_stylable_block = None
def _flush_links_after_block(self, *, trailing_blank: bool = True) -> None:
if not self.links:
return
trailing_blanks: List[str] = []
while self.output and self.output[-1] == "":
trailing_blanks.append(self.output.pop())
trailing_blanks.reverse()
add_trailing_blank = trailing_blank and not trailing_blanks
self._emit_link_entries(self.links, trailing_blank=add_trailing_blank)
self.links.clear()
self.link_indices.clear()
if trailing_blanks:
self.output.extend(trailing_blanks)
def _render_paragraph(self, payload: ParagraphPayload, style: BlockStyle) -> None:
processed = self._process_inline(payload.text)