fix: recover one hotlinked image, retire the other eleven

Twelve images across seven articles were hotlinked from servers that no longer
exist, so they were broken on the live sites. Checked every one against the
Wayback Machine, patiently enough to trust the answer: an earlier pass appeared
to find seven, but that was a test treating an empty response as a hit.

One survives. visualberlin.org's p1010789, a 2011 capture of the projector set
up against Mannheim city hall during the Flimmerkiste wildscreening, is now a
local asset in flux.vision/content/library/ and the reference points at it.

The other eleven were never captured. Both dead domains are archived, so this is
not a gap in coverage: of 8,438 visualberlin.org URLs and 204 for mutanten.org,
only that one image is there. mutanten.org's archive does not reach back to its
fp-content era at all, which is where the Logan's Run stills lived.

So lostimages.py learns about dead hosts, which it matches rather than fetching,
because a build must not depend on the network and a dead host is a fact about
the world rather than about the document. The references stay in the Markdown,
as the existing ones do: the reference records what was there.

It matters more on the capsule than on the web. A browser shows a broken image
as a silent gap, but gemtext has no inline images, so every one becomes a
visible => link line and the page would advertise a dead host as somewhere worth
going. smolnet.py therefore strips them too, reading the same host list, since
it works on the Markdown source and never sees Pelican's output.

lostimages.py moves to the repo-level plugins/ directory beside smolnet. It was
shared already, with flux.vision reaching into tobias.raayoni-last.com for it,
and randogoth.com now needs it too; a shared plugin living inside one site was
the wrong shape for three consumers.

Left alone deliberately: the two mutanten.org links in mutantenkino-website.md.
That entry is about having built the site, so the link is its subject rather
than decoration, and a dead link does not make the record wrong.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

flux.vision/content/flimmerkiste-2-wildscreening.md

@@ -33,7 +33,7 @@ generator, a nice 5000 ANSI projector, manfrotto tripod, V4 and laptops
 we walked through the Kreuzberg'ish 'Jungbusch' area towards the center
 marketplace where the huge city hall is located.
 
-![p1010789](http://visualberlin.org/wp-content/uploads/2009/05/p1010789-492x276.jpg "p1010789")
+![p1010789](/library/p1010789-492x276.jpg "p1010789")
 
 Equipment was set up quickly and we had a lot of fun, doing spontaneous
 mapping masks, jamming along, spreading Flimmerkiste flyers to curious

flux.vision/pelicanconf.py

@@ -7,9 +7,18 @@ TIMEZONE = 'Europe/Berlin'
 DEFAULT_LANG = 'en'
 
 THEME = 'flux'
-PLUGIN_PATHS = ['plugins', '../tobias.raayoni-last.com/plugins']
+PLUGIN_PATHS = ['plugins', '../plugins']
 PLUGINS = ['assetcheck', 'tileweight', 'lostimages']
 
+# Hosts that no longer exist, so their hotlinked images can never resolve.
+# Checked against Wayback: of twelve such images only one was ever captured,
+# and it now lives in content/library/. See lostimages.py.
+LOSTIMAGES_DEAD_HOSTS = [
+    'visualberlin.org', 'mutanten.org', 'web.mac.com',
+    'kino.web.de', 'lh4.google.it', 'l.yimg.com',
+]
+
+
 ARTICLE_PATHS = ['']
 ARTICLE_EXCLUDES = ['pages', 'library', 'images', 'recover']
 PAGE_PATHS = ['pages']

plugins/lostimages.py

@@ -0,0 +1,84 @@
+"""Drop <img> tags in article bodies whose image is gone for good.
+
+Two kinds of gone, same symptom:
+
+**Local.** The same situation as assetcheck.py, one level down: the recovered
+bodies kept their original /library/ references, and 22 of 200 point at files no
+archive holds (checked against Wayback; the CDX API returns no captures).
+
+**Remote.** Older posts hotlink images from other people's servers, and some of
+those servers no longer exist: visualberlin.org, mutanten.org, web.mac.com,
+kino.web.de, lh4.google.it, l.yimg.com. Name them in LOSTIMAGES_DEAD_HOSTS and
+their images are treated exactly like a lost local file. The host is matched
+rather than the URL fetched, because a build must not depend on the network and
+a dead host is a fact about the world, not about this document.
+
+Left alone either kind renders as a broken-image box, which on a phone reads as
+an unexplained 40px hole between paragraphs. On a gemtext capsule it is worse:
+an image becomes a visible link line, so the page advertises a dead host.
+
+The reference is still worth keeping in the Markdown source, so this strips it
+only from the generated output and leaves a comment naming the file, which keeps
+the output self-documenting for anyone who later finds the originals.
+"""
+import os
+import re
+
+from pelican import signals
+
+IMG = re.compile(r'<img[^>]*\bsrc="([^"]+)"[^>]*>')
+REMOTE = re.compile(r'^https?://([^/]+)/', re.IGNORECASE)
+ANCHOR = re.compile(r'<a[^>]*\bhref="(/library/[^"]+)"[^>]*>(.*?)</a>', re.S)
+
+
+def _strip(content):
+    body = getattr(content, "_content", None)
+    if not body or "<img" not in body:
+        return
+    base = content.settings.get("PATH", "content")
+    dead = {h.lower().lstrip(".") for h in
+            content.settings.get("LOSTIMAGES_DEAD_HOSTS", [])}
+    lost = []
+
+    def is_lost(src):
+        """True when this src can never resolve again."""
+        remote = REMOTE.match(src)
+        if remote:
+            host = remote.group(1).lower().split(":")[0]
+            # match the host and any subdomain of it, so naming
+            # visualberlin.org also covers www.visualberlin.org
+            return any(host == d or host.endswith("." + d) for d in dead)
+        if not src.startswith("/"):
+            return False
+        return not os.path.exists(os.path.join(base, src.lstrip("/")))
+
+    def name_of(src):
+        return os.path.basename(src.split("?")[0].rstrip("/")) or src
+
+    def anchor(m):
+        rel = m.group(1).lstrip("/")
+        if os.path.exists(os.path.join(base, rel)):
+            return m.group(0)
+        return m.group(2)
+
+    def sub(m):
+        src = m.group(1)
+        if not is_lost(src):
+            return m.group(0)
+        lost.append(name_of(src))
+        return "<!-- lost image: %s -->" % name_of(src)
+
+    body = IMG.sub(sub, body)
+    # a thumbnail that linked to the full-size file leaves the <a> behind, and
+    # its href 404s just as the <img> did
+    body = ANCHOR.sub(anchor, body)
+    if not lost:
+        return
+    # a paragraph that held nothing but the lost image would still occupy space
+    body = re.sub(r"<p>\s*((?:<!-- lost image: [^>]*-->\s*)+)</p>", r"\1", body)
+    content._content = body
+    content.lost_images = lost
+
+
+def register():
+    signals.content_object_init.connect(_strip)

plugins/smolnet.py

@@ -69,6 +69,12 @@ INTERNAL = re.compile(r"\{(filename|static|attach)\}([^)\s]+)")
 
 SUMMARY_MARKER = "<!-- PELICAN_END_SUMMARY -->"
 
+# Markdown image syntax, for dropping the ones whose host is gone. Matched on
+# the source rather than the rendered HTML because this plugin reads the
+# Markdown directly, so lostimages.py, which works on Pelican's output, never
+# sees the capsule.
+MD_IMAGE = re.compile(r'!\[[^\]]*\]\(\s*(https?://[^\s)]+)[^)]*\)')
+
 
 def rot13(text):
     """The obfuscation used on addresses in the HTML templates."""
@@ -156,6 +162,32 @@ def reveal_rot13(text, attr):
     return "".join(out)
 
 
+def drop_dead_images(text, dead):
+    """Remove images hotlinked from hosts that no longer exist.
+
+    It matters more here than on the web. A browser shows a broken image as a
+    silent gap, but gemtext has no inline images: every one becomes a visible
+    `=>` link line, so the page would advertise a dead host as though it were
+    somewhere worth going.
+
+    Shares LOSTIMAGES_DEAD_HOSTS with lostimages.py so there is one list, and
+    matches on the host rather than fetching, because a build must not depend on
+    the network.
+    """
+    if not dead:
+        return text
+
+    def sub(match):
+        host = match.group(1).split("//", 1)[1].split("/")[0].lower().split(":")[0]
+        if any(host == d or host.endswith("." + d) for d in dead):
+            return ""
+        return match.group(0)
+
+    text = MD_IMAGE.sub(sub, text)
+    # a line that held nothing but the dropped image would leave a blank
+    return re.sub(r"\n[ \t]*\n[ \t]*\n+", "\n\n", text)
+
+
 def toml_escape(value):
     return str(value).replace("\\", "\\\\").replace('"', '\\"')
 
@@ -193,6 +225,8 @@ class CapsuleGenerator(Generator):
         order = s.get("SMOLNET_SECTION_ORDER", {}) or {}
         latest_n = s.get("SMOLNET_LATEST", 15)
         rot_attr = s.get("SMOLNET_ROT13_ATTR")
+        dead_hosts = {h.lower().lstrip(".") for h in
+                      s.get("LOSTIMAGES_DEAD_HOSTS", [])}
         sidecar_name = s.get("SMOLNET_SIDECAR", ".itsybitsy.toml")
 
         articles = [a for a in self.context.get("articles", [])]
@@ -214,6 +248,7 @@ class CapsuleGenerator(Generator):
             body = body.replace(SUMMARY_MARKER, "")
             body = rewrite_internal(body, by_source, self._warn)
             body = reveal_rot13(body, rot_attr)
+            body = drop_dead_images(body, dead_hosts)
             # Blank lines, not newlines: a heading and the line under it are
             # separate blocks only if a blank line divides them, otherwise the
             # date runs into the first sentence as one paragraph.

randogoth.com/pelicanconf.py

@@ -10,7 +10,16 @@ DEFAULT_LANG = 'en'
 # and the one actually wanted here. mc-pelican stays in the tree as the fallback.
 THEME = 'bbs'
 PLUGIN_PATHS = ['plugins', '../plugins']
-PLUGINS = ['crtimages', 'smolnet']
+PLUGINS = ['crtimages', 'smolnet', 'lostimages']
+
+# Hosts that no longer exist, so their hotlinked images can never resolve.
+# Checked against Wayback: of twelve such images only one was ever captured,
+# and it now lives in content/library/. See lostimages.py.
+LOSTIMAGES_DEAD_HOSTS = [
+    'visualberlin.org', 'mutanten.org', 'web.mac.com',
+    'kino.web.de', 'lh4.google.it', 'l.yimg.com',
+]
+
 
 # The sections are message areas, numbered so they can be dialled by key the way
 # a board's main menu works. No descriptions: the names are the whole listing,

tobias.raayoni-last.com/pelicanconf.py

@@ -6,7 +6,7 @@ TIMEZONE = 'Asia/Jerusalem'
 DEFAULT_LANG = 'en'
 
 THEME = 'raayoni'
-PLUGIN_PATHS = ['plugins']
+PLUGIN_PATHS = ['plugins', '../plugins']
 PLUGINS = ['assetcheck', 'lostimages']
 # Dark is the register this work belongs to, so it is the default rather than
 # following the system. An explicit toggle (persisted in localStorage) wins.

tobias.raayoni-last.com/plugins/lostimages.py

@@ -1,55 +0,0 @@
-"""Drop <img> tags in article bodies whose file did not survive the recovery.
-
-The same situation as assetcheck.py, one level down: the recovered bodies kept
-their original /library/ references, and 22 of 200 point at files no archive
-holds (checked against Wayback; the CDX API returns no captures for them).
-
-Left alone they render as a broken-image box, which on a phone reads as an
-unexplained 40px hole between paragraphs. The reference is still worth keeping
-in the Markdown source, so this strips it only from the generated HTML and
-leaves a comment naming the file, which keeps the output self-documenting for
-anyone who later finds the originals on the offline drive.
-"""
-import os
-import re
-
-from pelican import signals
-
-IMG = re.compile(r'<img[^>]*\bsrc="(/[^"]+)"[^>]*>')
-ANCHOR = re.compile(r'<a[^>]*\bhref="(/library/[^"]+)"[^>]*>(.*?)</a>', re.S)
-
-
-def _strip(content):
-    body = getattr(content, "_content", None)
-    if not body or "<img" not in body:
-        return
-    base = content.settings.get("PATH", "content")
-    lost = []
-
-    def anchor(m):
-        rel = m.group(1).lstrip("/")
-        if os.path.exists(os.path.join(base, rel)):
-            return m.group(0)
-        return m.group(2)
-
-    def sub(m):
-        rel = m.group(1).lstrip("/")
-        if os.path.exists(os.path.join(base, rel)):
-            return m.group(0)
-        lost.append(os.path.basename(rel))
-        return "<!-- lost image: %s -->" % os.path.basename(rel)
-
-    body = IMG.sub(sub, body)
-    # a thumbnail that linked to the full-size file leaves the <a> behind, and
-    # its href 404s just as the <img> did
-    body = ANCHOR.sub(anchor, body)
-    if not lost:
-        return
-    # a paragraph that held nothing but the lost image would still occupy space
-    body = re.sub(r"<p>\s*((?:<!-- lost image: [^>]*-->\s*)+)</p>", r"\1", body)
-    content._content = body
-    content.lost_images = lost
-
-
-def register():
-    signals.content_object_init.connect(_strip)