feat: render randogoth.com as a small-net capsule
A Pelican generator that writes a capsule beside the HTML build: plain
Markdown plus per-directory .itsybitsy.toml sidecars, which a protocol daemon
converts per request to gemtext, XHTML-MP, WML or plain text.
The obstacle was recorded in the README as "smolweb reads YAML frontmatter
while Pelican uses bare Title: lines". Both halves turned out to be wrong:
itsybitsy parses no frontmatter at all, by design, and metadata lives outside
the document. So the transform is not a translation between dialects, it is a
removal, and a protocol daemon stays ignorant of one static generator.
Nothing in plugins/smolnet.py is specific to this site. Every difference is a
setting and every setting has a default, so another site enables it with
PLUGINS = ['smolnet']. It sits at the repo root for the same reason
flux.vision reaches into tobias.raayoni-last.com for lostimages.
Index entries are emitted as link-only paragraphs rather than Markdown lists,
which is not a style choice: the gemtext renderer prints a bare => line for a
paragraph holding nothing but links, and a bullet *and* a => line for a list
item, so a list would publish every entry twice. Gemini clients number link
lines themselves, so the board's dial-an-area-by-number idiom survives into
the capsule without emitting a single bracket. The trade-off is in the
docstring: the text renderer reflows such a run into prose, which costs
nothing here because randogoth.com is not getting Nex or Gopher.
Also drops a Zemanta tracking pixel that came in with a 2012 blog import and
is live on the site today, firing a third-party request on every view of that
article. It would have been worse in the capsule, where an image becomes a
visible link line advertising a dead tracker. It was the only one across all
three sites.
Verified by serving the output with the real daemon: 41 articles, 2 pages, 6
area indexes and an archive, no metadata surviving, no {filename} left,
trl@posteo.net revealed on Connect where the renderer would otherwise have
dropped it with its enclosing HTML, images served as image/png, and the index
rendering as a clean run of => lines in gemtext and as XHTML Mobile over HTTP.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
randogoth
parent
4853e5b360
commit
5131f1f53f
3 changed files with 366 additions and 7 deletions
350
plugins/smolnet.py
Normal file
350
plugins/smolnet.py
Normal file
|
|
@ -0,0 +1,350 @@
|
|||
"""Render a Pelican site as a small-net capsule beside its HTML.
|
||||
|
||||
A capsule is a directory of plain Markdown that a protocol daemon serves over
|
||||
Gemini, Spartan, Nex, Gopher and WAP, converting per request. This writes one
|
||||
out of the same content the HTML build uses, so the two never drift.
|
||||
|
||||
Written against `itsybitsy`, which parses **no frontmatter at all**: a document
|
||||
is pure Markdown and its metadata lives beside it in a per-directory sidecar.
|
||||
Pelican's own metadata is a block of bare `Title:`/`Date:` lines at the head of
|
||||
every file, so served unchanged it becomes the first paragraph of every page.
|
||||
Stripping that is most of what this plugin does.
|
||||
|
||||
Nothing here is specific to one site. Everything that differs between sites is a
|
||||
setting, and every setting has a default, so a site can switch it on with just
|
||||
`PLUGINS = ['smolnet']`.
|
||||
|
||||
SMOLNET_OUTPUT 'smolnet' directory under OUTPUT_PATH
|
||||
SMOLNET_SECTION_DIR 'section' subdirectory holding one index per category
|
||||
SMOLNET_LATEST 15 entries on the front page; 0 drops the section
|
||||
SMOLNET_SECTION_ORDER {} category name -> sort key, else alphabetical
|
||||
SMOLNET_LABELS {} override any heading, see LABELS
|
||||
SMOLNET_ASSETS None directories to copy; defaults to STATIC_PATHS
|
||||
SMOLNET_SIDECAR '.itsybitsy.toml'
|
||||
SMOLNET_ROT13_ATTR None attribute holding a rot13 address to reveal
|
||||
SMOLNET_TAGLINE None one line at the foot of the index
|
||||
|
||||
Why link-only paragraphs rather than lists: gemtext has one kind of link, a line
|
||||
beginning `=>`, so a renderer has to lift links out of the prose. itsybitsy's
|
||||
gemtext renderer emits a bare `=>` line for a paragraph that holds nothing but
|
||||
links, and tolerates soft breaks inside it, so a run of links on consecutive
|
||||
lines comes out as a clean run of link lines. A Markdown list would instead
|
||||
print a bullet *and* a link line for every entry, listing everything twice.
|
||||
|
||||
Gemini clients number link lines themselves, which means a menu of areas is
|
||||
dialled by number without this plugin writing a single bracket.
|
||||
|
||||
The trade-off, measured rather than assumed: that shape is exact in gemtext and
|
||||
in XHTML-MP, where each link keeps its own line, but the `text` renderer used
|
||||
for Nex and Gopher reflows the run into a paragraph, so an index comes out as
|
||||
`RANDOM · 4 (/area/random) PLACE · 7 (/area/place) …`. A Markdown list would be
|
||||
the better answer for those two and the worse answer for the other three. Links
|
||||
win, because gemtext is what a capsule is mostly read over; a site serving Nex
|
||||
or Gopher as its main face should flip this.
|
||||
"""
|
||||
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
|
||||
from pelican import signals
|
||||
from pelican.generators import Generator
|
||||
|
||||
LABELS = {
|
||||
"sections": "Sections",
|
||||
"latest": "Latest",
|
||||
"archive": "All entries",
|
||||
"pages": "Pages",
|
||||
"back": "Back to the index",
|
||||
}
|
||||
|
||||
# Pelican's metadata block: bare `Key: value` lines from the first line on,
|
||||
# with indented continuations, ending at the first blank line. This is
|
||||
# Python-Markdown's Meta extension, not YAML, and it has no delimiters.
|
||||
META_LINE = re.compile(r"^[A-Za-z][A-Za-z0-9_-]*:")
|
||||
META_CONT = re.compile(r"^\s+\S")
|
||||
|
||||
# Pelican's internal link syntax. Left alone these reach the capsule verbatim.
|
||||
INTERNAL = re.compile(r"\{(filename|static|attach)\}([^)\s]+)")
|
||||
|
||||
SUMMARY_MARKER = "<!-- PELICAN_END_SUMMARY -->"
|
||||
|
||||
|
||||
def rot13(text):
|
||||
"""The obfuscation used on addresses in the HTML templates."""
|
||||
out = []
|
||||
for ch in text:
|
||||
if "a" <= ch <= "z":
|
||||
out.append(chr((ord(ch) - 97 + 13) % 26 + 97))
|
||||
elif "A" <= ch <= "Z":
|
||||
out.append(chr((ord(ch) - 65 + 13) % 26 + 65))
|
||||
else:
|
||||
out.append(ch)
|
||||
return "".join(out)
|
||||
|
||||
|
||||
def strip_metadata(text):
|
||||
"""Drop the leading metadata block, returning the body alone.
|
||||
|
||||
Stops at the first line that is neither a `Key:` line nor an indented
|
||||
continuation of one, so a document that happens to open on prose is
|
||||
returned untouched rather than having its first paragraph eaten.
|
||||
"""
|
||||
lines = text.splitlines()
|
||||
i = 0
|
||||
seen = False
|
||||
while i < len(lines):
|
||||
line = lines[i]
|
||||
if not line.strip():
|
||||
if seen:
|
||||
i += 1
|
||||
break
|
||||
if META_LINE.match(line):
|
||||
seen = True
|
||||
elif seen and META_CONT.match(line):
|
||||
pass
|
||||
else:
|
||||
break
|
||||
i += 1
|
||||
if not seen:
|
||||
return text
|
||||
return "\n".join(lines[i:]).lstrip("\n")
|
||||
|
||||
|
||||
def rewrite_internal(text, by_source, warn):
|
||||
"""Resolve `{filename}other.md` against the documents actually built.
|
||||
|
||||
The target is a path relative to the referring file, which for a flat
|
||||
content directory means a bare basename. Anything that does not resolve is
|
||||
reported rather than silently emitted, because a literal `{filename}` in a
|
||||
capsule is both broken and ugly.
|
||||
"""
|
||||
|
||||
def sub(match):
|
||||
kind, target = match.group(1), match.group(2)
|
||||
key = os.path.basename(target)
|
||||
if kind == "filename" and key in by_source:
|
||||
return "/" + by_source[key]
|
||||
warn("smolnet: unresolved {%s}%s", kind, target)
|
||||
return "/" + key.rsplit(".", 1)[0]
|
||||
|
||||
return INTERNAL.sub(sub, text)
|
||||
|
||||
|
||||
def reveal_rot13(text, attr):
|
||||
"""Replace a block of HTML carrying a rot13 address with the address itself.
|
||||
|
||||
Address obfuscation defends against web crawlers and depends on JavaScript,
|
||||
so over Gemini it is both pointless and impossible. The renderer drops raw
|
||||
HTML blocks outright, which would take the address with them: on a page
|
||||
whose purpose is to carry it, that is a silent loss rather than a tidy-up.
|
||||
|
||||
The whole block is replaced rather than the tag that carries the attribute.
|
||||
Matching tags means guessing where the element ends, and the first attempt
|
||||
at this stopped at the first closing tag and left the rest of the markup
|
||||
stranded in the output. Markdown already separates block-level HTML by blank
|
||||
lines, so the block is a boundary the format gives us for free.
|
||||
"""
|
||||
if not attr:
|
||||
return text
|
||||
carrier = re.compile(r"\b" + re.escape(attr) + r'="([^"]+)"', re.IGNORECASE)
|
||||
blocks = re.split(r"(\n[ \t]*\n)", text)
|
||||
out = []
|
||||
for block in blocks:
|
||||
found = carrier.search(block)
|
||||
out.append(rot13(found.group(1)) if found else block)
|
||||
return "".join(out)
|
||||
|
||||
|
||||
def toml_escape(value):
|
||||
return str(value).replace("\\", "\\\\").replace('"', '\\"')
|
||||
|
||||
|
||||
def sidecar(titles):
|
||||
"""A `.itsybitsy.toml` naming each file beside it.
|
||||
|
||||
The daemon falls back to the first heading and then to the filename, so
|
||||
this is belt and braces, but it is the only place a title can live that
|
||||
survives a document having no heading of its own.
|
||||
"""
|
||||
lines = []
|
||||
for name in sorted(titles):
|
||||
lines.append('[page."%s"]' % toml_escape(name))
|
||||
lines.append('title = "%s"' % toml_escape(titles[name]))
|
||||
lines.append("")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def link(label, href):
|
||||
return "[%s](%s)" % (label.replace("]", ""), href)
|
||||
|
||||
|
||||
class CapsuleGenerator(Generator):
|
||||
"""Writes the capsule once the content generators have read everything."""
|
||||
|
||||
def generate_context(self):
|
||||
pass
|
||||
|
||||
def generate_output(self, writer=None):
|
||||
s = self.settings
|
||||
out = os.path.join(self.output_path, s.get("SMOLNET_OUTPUT", "smolnet"))
|
||||
section_dir = s.get("SMOLNET_SECTION_DIR", "section")
|
||||
labels = dict(LABELS, **s.get("SMOLNET_LABELS", {}))
|
||||
order = s.get("SMOLNET_SECTION_ORDER", {}) or {}
|
||||
latest_n = s.get("SMOLNET_LATEST", 15)
|
||||
rot_attr = s.get("SMOLNET_ROT13_ATTR")
|
||||
sidecar_name = s.get("SMOLNET_SIDECAR", ".itsybitsy.toml")
|
||||
|
||||
articles = [a for a in self.context.get("articles", [])]
|
||||
pages = [p for p in self.context.get("pages", [])]
|
||||
if not articles and not pages:
|
||||
return
|
||||
|
||||
# Map source basename -> slug, so {filename} can be resolved. Built from
|
||||
# everything published, pages included, since an article may link one.
|
||||
by_source = {}
|
||||
for doc in articles + pages:
|
||||
by_source[os.path.basename(doc.source_path)] = doc.slug
|
||||
|
||||
os.makedirs(out, exist_ok=True)
|
||||
root_titles = {}
|
||||
|
||||
def emit(doc, trail):
|
||||
body = strip_metadata(self._read(doc))
|
||||
body = body.replace(SUMMARY_MARKER, "")
|
||||
body = rewrite_internal(body, by_source, self._warn)
|
||||
body = reveal_rot13(body, rot_attr)
|
||||
# Blank lines, not newlines: a heading and the line under it are
|
||||
# separate blocks only if a blank line divides them, otherwise the
|
||||
# date runs into the first sentence as one paragraph.
|
||||
head = "# %s\n\n" % doc.title
|
||||
if trail:
|
||||
head += "%s\n\n" % trail
|
||||
text = head + body.strip() + "\n"
|
||||
name = "%s.md" % doc.slug
|
||||
with open(os.path.join(out, name), "w", encoding="utf-8") as fh:
|
||||
fh.write(text)
|
||||
root_titles[name] = doc.title
|
||||
|
||||
for article in articles:
|
||||
bits = []
|
||||
if getattr(article, "date", None):
|
||||
bits.append(article.date.strftime("%d %b %Y"))
|
||||
if getattr(article, "category", None):
|
||||
bits.append(str(article.category))
|
||||
emit(article, " · ".join(bits))
|
||||
for page in pages:
|
||||
emit(page, "")
|
||||
|
||||
# One index per category.
|
||||
sections = sorted(
|
||||
self.context.get("categories", []),
|
||||
key=lambda pair: (order.get(str(pair[0]), 99), str(pair[0])),
|
||||
)
|
||||
if sections:
|
||||
os.makedirs(os.path.join(out, section_dir), exist_ok=True)
|
||||
section_titles = {}
|
||||
for category, items in sections:
|
||||
lines = ["# %s" % category, ""]
|
||||
lines.append(self._links(sorted(items, key=lambda a: a.date, reverse=True)))
|
||||
lines += ["", link(labels["back"], "/")]
|
||||
name = "%s.md" % category.slug
|
||||
with open(os.path.join(out, section_dir, name), "w", encoding="utf-8") as fh:
|
||||
fh.write("\n".join(lines).rstrip() + "\n")
|
||||
section_titles[name] = str(category)
|
||||
self._write_sidecar(os.path.join(out, section_dir, sidecar_name), section_titles)
|
||||
|
||||
by_date = sorted(articles, key=lambda a: a.date, reverse=True)
|
||||
|
||||
# The full listing, so the index can stay short without hiding anything.
|
||||
archive = ["# %s" % labels["archive"], "", self._links(by_date), "",
|
||||
link(labels["back"], "/")]
|
||||
with open(os.path.join(out, "archive.md"), "w", encoding="utf-8") as fh:
|
||||
fh.write("\n".join(archive).rstrip() + "\n")
|
||||
root_titles["archive.md"] = labels["archive"]
|
||||
|
||||
index = ["# %s" % s.get("SITENAME", "index"), ""]
|
||||
if sections:
|
||||
index += ["## %s" % labels["sections"], ""]
|
||||
index.append("\n".join(
|
||||
link("%s · %d" % (category, len(items)),
|
||||
"/%s/%s" % (section_dir, category.slug))
|
||||
for category, items in sections
|
||||
))
|
||||
index.append("")
|
||||
if latest_n and by_date:
|
||||
index += ["## %s" % labels["latest"], "",
|
||||
self._links(by_date[:latest_n]), "",
|
||||
link(labels["archive"], "/archive"), ""]
|
||||
if pages:
|
||||
index += ["## %s" % labels["pages"], "",
|
||||
"\n".join(link(p.title, "/%s" % p.slug) for p in pages), ""]
|
||||
tagline = s.get("SMOLNET_TAGLINE")
|
||||
if tagline:
|
||||
index += ["---", "", tagline, ""]
|
||||
with open(os.path.join(out, "index.md"), "w", encoding="utf-8") as fh:
|
||||
fh.write("\n".join(index).rstrip() + "\n")
|
||||
root_titles["index.md"] = s.get("SITENAME", "index")
|
||||
|
||||
self._write_sidecar(os.path.join(out, sidecar_name), root_titles)
|
||||
self._copy_assets(out)
|
||||
|
||||
def _links(self, docs):
|
||||
"""One link per line in a single paragraph.
|
||||
|
||||
Deliberately not a Markdown list: a list item renders as a bullet *and*
|
||||
a link line, so every entry would appear twice.
|
||||
|
||||
Targets are root-relative. A section index sits one directory down, so
|
||||
relative targets would need `../` and would break again at any further
|
||||
depth; `/slug` is correct from anywhere and the daemon resolves it
|
||||
against the site root.
|
||||
"""
|
||||
rows = []
|
||||
for doc in docs:
|
||||
label = doc.title
|
||||
if getattr(doc, "date", None):
|
||||
label = "%s · %s" % (doc.date.strftime("%d %b %y"), doc.title)
|
||||
rows.append(link(label, "/%s" % doc.slug))
|
||||
return "\n".join(rows)
|
||||
|
||||
def _read(self, doc):
|
||||
with open(doc.source_path, encoding="utf-8") as fh:
|
||||
return fh.read()
|
||||
|
||||
def _write_sidecar(self, path, titles):
|
||||
with open(path, "w", encoding="utf-8") as fh:
|
||||
fh.write(sidecar(titles))
|
||||
|
||||
def _copy_assets(self, out):
|
||||
"""Copy the static directories in, so the capsule stands on its own.
|
||||
|
||||
A capsule whose images are fetched over HTTPS from the web site is not
|
||||
really a capsule, and link targets are resolved against the capsule
|
||||
root, so pointing outside it would not work anyway.
|
||||
"""
|
||||
s = self.settings
|
||||
names = s.get("SMOLNET_ASSETS")
|
||||
if names is None:
|
||||
names = [n for n in s.get("STATIC_PATHS", []) if not n.startswith(".")]
|
||||
for name in names:
|
||||
src = os.path.join(self.path, name)
|
||||
if not os.path.isdir(src):
|
||||
continue
|
||||
shutil.copytree(src, os.path.join(out, name), dirs_exist_ok=True)
|
||||
|
||||
def _warn(self, fmt, *args):
|
||||
try:
|
||||
from pelican.log import logger
|
||||
|
||||
logger.warning(fmt, *args)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def get_generators(pelican_object):
|
||||
return CapsuleGenerator
|
||||
|
||||
|
||||
def register():
|
||||
signals.get_generators.connect(get_generators)
|
||||
|
|
@ -50,8 +50,3 @@ you will definetly enjoy this flick as much as i did.
|
|||
|
||||
[trailer at apple.com
|
||||
](http://www.apple.com/trailers/lions_gate/crank2/)
|
||||
|
||||
<div class="zemanta-pixie">
|
||||
|
||||
|
||||
</div>
|
||||
|
|
|
|||
|
|
@ -9,8 +9,8 @@ DEFAULT_LANG = 'en'
|
|||
# *application*: grey chrome, dialogs, a menu bar. A board is a different thing,
|
||||
# and the one actually wanted here. mc-pelican stays in the tree as the fallback.
|
||||
THEME = 'bbs'
|
||||
PLUGIN_PATHS = ['plugins']
|
||||
PLUGINS = ['crtimages']
|
||||
PLUGIN_PATHS = ['plugins', '../plugins']
|
||||
PLUGINS = ['crtimages', 'smolnet']
|
||||
|
||||
# The sections are message areas, numbered so they can be dialled by key the way
|
||||
# a board's main menu works. No descriptions: the names are the whole listing,
|
||||
|
|
@ -20,6 +20,20 @@ BBS_AREAS = {
|
|||
'MACHINES': 4, 'GONZO': 5, 'ENDTIMES': 6,
|
||||
}
|
||||
|
||||
# The capsule mirrors the board: same areas, same order, same words. Gemini
|
||||
# clients number link lines themselves, so an area is still dialled by number
|
||||
# over there without the plugin writing a single bracket.
|
||||
SMOLNET_SECTION_DIR = 'area'
|
||||
SMOLNET_SECTION_ORDER = BBS_AREAS
|
||||
SMOLNET_LABELS = {
|
||||
'sections': 'Message areas', 'latest': 'Latest messages',
|
||||
'archive': 'All messages', 'back': 'Main menu', 'pages': 'System',
|
||||
}
|
||||
# The address is rot13 in the HTML because of web crawlers. That defence needs
|
||||
# JavaScript, which Gemini does not have, and the renderer drops raw HTML, so
|
||||
# without this the one thing the Connect page exists to carry would vanish.
|
||||
SMOLNET_ROT13_ATTR = 'data-r'
|
||||
|
||||
# Articles live flat in content/; assets are carved out of that.
|
||||
ARTICLE_PATHS = ['']
|
||||
ARTICLE_EXCLUDES = ['pages', 'library', 'images', 'files']
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue