Bare bones generative image making

I first tried portable pixmap graphics going through The Ray Tracer Challenge in JS during batch at RC, came across it again on the way to translating Scott Draves' Fuse (1991) for the web, then a few months later as well researching image segmentation looking at e.g., Pedro Felzenszwalb's C++ graph cut solver (2004).

It kind of rocks being able to compose images straight up out of color values both in science and in practice respectively when a pixel is not a little square and when, similar to say .obj files, printf(1) and bc(1) alone can get you truly far.

P3 800 450 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 10 10 10 21 21 21 32 32 32 43 43 43 53 53 53 64 64 64 74 74 74 83 83 83 93 93 93 101 101 101 110 110 110 117 117 117 124 124 124 129 129 129 134 134 134 137 137 137 140 140 140 140 140 140 140 140 140 137 137 137 134 134 134 129 129 129 134 134 134 137 137 137 140 140 140 140 140 140 140 140 140 137 137 137 134 134 134 129 129 129 124 124 124 117 117 117 118 118 118 122 122 122 125 125 125 127 127 127 128 128 128 127 127 127 125 125 125 122 122 122 118 118 118 115 115 115 128 128 128 140 140 140 153 153 153 166 166 166 178 178 178 191 191 191 204 204 204 217 217 217 230 230 230 242 242 242 255 255 255 242 242 242 230 230 230 217 217 217 204 204 204 191 191 191 178 178 178 166 166 166 153 153 153 140 140 140 128 128 128 115 115 115 102 102 102 89 89 89 76 76 76 64 64 64 51 51 51 38 38 38 26 26 26 13 13 13 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 4 4 15 15 15 27 27 27 38 38 38 49 49 49 60 60 60 71 71 71 82 82 82 92 92 92 101 101 101 111 111 111 119 119 119 128 …

Super neat Netpbm image formats make a lot of sense if you ever looked at <canvas> ImageData. This is ~1kB of .ppm cellular noise extract. The top three lines are the header. P3 is the magic number. Then you have dimensions and color max followed by space separated RGB triplets. If it only worked as-is browser side!

I had fun putting together a /dev/urandom driven rainbow generator last year and so took up drawing Worley noise and byte beat motifs, which are more specific types of problem, in the same vein as a drill. It's refreshing that for building pipelines like those I can rely on reading the manual and not have to spend much time looking up answers online when stuck. I do have Copilot, but it almost never helps beyond auto complete to be honest.

Not that I could write the book on it, but since many AI processes like Stable Diffusion start off as noise, making something out of nothing so to speak, which is a broadly common theme in software if you analyze it, I wanted to come up with more meaningful and browser compatible results as a next step.

Chaos

I have fuse mechanics already figured out, let me start there. Inspired by GNU Emacs Dissociated Press, cannibalistic bluff teen film Society (1992), cut-up technique, and implicitly maybe if I had to guess Carolee Schneemann and James Tenney's Fuses (1964-67) home movie full of lust, the algorithm is pure joy for being on the empirical side and open ended:

  1. Start with a series of images, divide them up into small tiles, typically 25x25px.
  2. Shuffle those around and place one of them in the middle of canvas.
  3. Fan out looking for locally coherent tiles array matches to assemble the final picture.

If the output can be merely as good as the input material, wicked kooky is a guarantee with this approach, when human parts like eyes and hands are involved in particular. I was wondering what happens processing text though. Noise, silent void, chaos and disorder, what's the difference? Joseph Haydn's The Creation (Die Schöpfung) Hob. XXI:2 (1797-98) strikes me as appropriate to be sampling from therefore.

I see Art Nouveau capital of the world Brussels Philharmonic have the libretto uploaded in both English and Old German. I expected it might have been symbolic to remove all the x's, but there are none, no joke. Too perfect, feeling irreverent and compelled to mangle up content to a degree:

\
# Download HTML.
curl -s https://www.brusselsphilharmonic.be/en/text-die-schöpfung |
# Target the first column for German, discard lint errors.
# XPath is like a jQuery for the command line.
xmllint --html --nowarning --xpath '//tr/td[1]//*/text()' 2> /dev/null - |
# Single space everything (squeeze).
tr -s [:space:] " " |
# Trim whitespace.
sed -e "s/ *$$//" |
# Introduce a few typos.
sed -e "s/ck/k/g;s/ht/th/;s/hr/rr/23" > libretto.de.txt

It may sound counterintuitive, but qlmanage(1) has a 800px soft cap when dealing with plain text files, but no issue rasterizing HTML or SVG <foreignObject>, which also means I can fit more letters into the master image using CSS, cool:

# Expanded for readability, should be line collapsed.
awk '{printf(
  "<svg xmlns=\"http://www.w3.org/2000/svg\" width=\"100%%\" height=\"100%%\">
    <style>
      pre {
        background: black;
        color: white;
        font-size: 5px;
        line-height: 1;
        white-space: wrap;
        word-spacing: -1ch;
      }
    </style>
    <foreignObject width=\"100%%\" height=\"100%%\">
      <pre xmlns=\"http://www.w3.org/1999/xhtml\">%s</pre>
    </foreignObject>
  </svg>",
$0)}' libretto.de.txt > master.svg
# Thumbnail in name only:
qlmanage master.svg -t -o . -s 2000

To draw samples from the master image on a Mac, sips(1) is agreeably JavaScript-able, for example:

// Basic sampler.js, call inside a for loop to derive fuse input images out of master:
// `for i in {1..8}; do sips -j sampler.js -o $i.png master.svg.png; done`
const w = Math.floor(Math.random() * 100) + 20
const h = Math.floor(Math.random() * 100) + 20
const canvas = new Canvas(w, h)

const [master] = sips.images
const x = Math.floor(Math.random() * master.size.width - w)
const y = Math.floor(Math.random() * master.size.height - h)

canvas.drawImage(master, x, y, w, h, 0, 0, w, h)

const tile = new Output(canvas, sips.outputPath)

tile.addToQueue()
Text and letter based fuse image

Holy Moly Biblical chaos on grid

Gibberish, happy to have traveled down this path, but would prefer some structure and some definition. Bank the templating via string interpolation and press on.

Formula

To have form, you want numbers and to arrive at numbers willy-nilly not, you have to have math, and to settle on a math you need to experiment a little. Would converting my .ppm byte beat maker to SVG lead to any gains?

As it happens no, I ran into floating point precision trouble with bc(1) using doubles internally, whereas float32 based patterns look unmistakably more absorbing in my opinion. I found no way of overriding the default setting and was convinced my calculator should be Perl instead, but was tired exploring more of this angle.

// TFW single vs. double precision floating point
// arithmetic will do your head in sometimes.
console.assert(42n ** 10n === BigInt(42 ** 10))
console.assert(43n ** 10n === BigInt(43 ** 10))
Byte beat pattern using float32 precision Same formula, but using float64 precision

The proof is in the pudding When higher precision comes at the cost of visual charm: Another morning lost beating around the bush.

Alright, one less idea to worry about, making progress, but obviously not there yet. I had ten days budgeted for this baby doll, and about a week's worth of sandglass expenditure into my quest, crunch time panic is creeping in. The pressure is on, the heat is up, the stakes are high, the rest of my schedule is on the line, how do I manage? Give up? Cave, lose heart and duck out? Go mad? Throw the towel in? Wave the white flag? Because that would be very sad.

LOL, not so fast Mr. Doubting Thomas Sir, hold your horses, dial the negativity down a notch please, hang on a minute and please allow me a moment of pause to reflect. Okay, fortunately, I still have a bit of rope left, why not tie up any metaphorical loose ends with a stevedore knot stacking pieces of DOM up and down and around? Bingo! Killing two birds, design and interpretation, with one stone? There you go buddy, gagnant, let's ride!

I thought it was going to be easy, but failed to get a proper formula out of ChatGPT here surprisingly enough. There was some confusion about the parametric equation being in existence, but the math is in fact beautifully simple and child's play really. Stevedore's is naturally a special case of Lissajous. If supporting folks on older browsers was no requirement, I might have written the whole thing in CSS:

x = R cos(nx t + φx), y = R cos(ny t + φy), z = R cos(nz t + φz)

Where R is the radius, nx = 3, ny = 2, nz = 5, φx = 1.5, φy = 0.2, and φz = 0.

Transform

Brilliant, I have my data source decided fine and dandy and would now like to compile a web document, preferably portable and self-contained to house the visual. SVG comes to mind again, but the lack of CSS 3d transforms is a showstopper.

Well, I had always hoped the XSLT toil of restyling nginx index listings might prove useful one day. Browsers are limited to version 1.0 of the spec (1999), which may not be the latest and greatest, but you have to appreciate the foresight in promoting separation of concerns regardless, yes? For example a data object of:

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet href="example.xsl" type="text/xsl"?>
<Article title="Die Schöpfung" lang="de">
  <Category hoboken="XXI:2">Oratorio</Category>
  <Composer>Joseph Haydn</Composer>
  <Description>
    Masterpiece! Depicts and celebrates the creation of
    the world as narrated in the Book of Genesis.
  </Description>
  <Librettist>Gottfried van Swieten</Librettist>
  <Completed>Herbst 1797</Completed>
  <Period>Classical</Period>
  <Premiere datetime="1798-04-30">30. 04. 1798</Premiere>
  <Premiere datetime="1799-03-19" type="public">19. 03. 1799</Premiere>
  <Section title="NR. 2. REZITATIV (RAPHAEL & CHOR)">
    <Lyric>
      <Character>RAPHAEL</Character>
      <Lines>
        Im Anfange schuf Gott Himmel und Erde,
        und die Erde war ohne Form und leer,
        und Finsternis war auf der Fläche der Tiefe.
      </Lines>
    </Lyric>
    …
  </Section>
</Article>

Yields:

Hat trick Same text as above taken from XML through XSL to HTML.

Given an example.xsl stylesheet along the lines of:

<?xml version="1.0"?>
<xsl:stylesheet
  version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output
    method="html"
    encoding="utf-8"
    doctype-system="about:legacy-compat"
    indent="yes"/>
  <xsl:template match="/">
    <html lang="{Article/@lang}">
      <head>
        <meta name="description" content="{Article/Description}"/>
        <title>
          <xsl:value-of select="Article/@title"/>
        </title>
        <style>…</style>
      </head>
      <body>
        <article>
          <h1><xsl:value-of select="Article/@title"/></h1>
          …
          <xsl:for-each select="Article/Section">
            <xsl:for-each select="Lyric">
              <p>
                <strong><xsl:value-of select="Character"/></strong>
                <span><xsl:value-of select="Lines"/></span>
              </p>
            </xsl:for-each>
          </xsl:for-each>
        </article>
      </body>
    </html>
  </xsl:template>
</xsl:stylesheet>

Fits like a glove. I can tell httpd.conf(5) to serve up the data file directly without HTML embedding even, grand.

Done and done Janus helps produce delightful stevedore knots, 70-80% bent, 10-25% washed out, 5-10% thorny, 90% unscripted. Made with builtin(1), perl(1), grep(1), awk(1), tail(1), printf(1), seq(1), jot(1), XML, XSL, and CSS FTW. 🤙

Not entirely a must, but if the HTML is rendered in-browser and the data is static, no reason being blindly purist about it, kindly ask JS to automatically reload the page before lunch.

// Might as well for the sake of it, because of CDATA retro attraction
// and meta#http-equiv being dumb. Run a tab switching aware timer and
// reload the page noons daily in near sync with new product delivery.
const then = new Date()
const deadline = new Date(then)

if (then.getHours() < 12) {
  deadline.setHours(12)
} else {
  deadline.setHours(36)
}

deadline.setSeconds(0)
deadline.setMinutes(0)

document.body
  .animate(null, { duration: 500, iterations: 1 })
  .addEventListener("finish", function loop(e) {
    const now = new Date()

    if (now > deadline) {
      location.reload()
    } else {
      e.target.play()
    }
  })

Give it a schmancy color palette, oversaturated neon possibly, call it "Untitled (Reimagined) No. XXIII," put on a? 𝔫𝔣𝔱, Bob's Yer Uncle! Just kidding. No offense darling, what is that, some sort of ribbon? So you found declarative ways of sketching pseudo 3d shapes using ancient general commands readily available on most systems? Big deal, is that best you can do? How do you bring real life traces into it?

Kicker, The

Ham Street Draw Dock ducks in a row

Summer is here Talk about getting all your ducks in a row.

Have you heard? Earth's rotation rate is variable and slowing down long term on average partly due to "tidal braking and the redistribution of mass within, including oceans and atmosphere," which is why our civilization invented and keeps track of leap seconds. Is that right? Our measure of time is inevitably convention? Lord have mercy, and besides undeniable uncertainty on other fronts the UTC scale differs in realization from lab to lab by a few nanoseconds? 🙀

# Probe for leap second updates in one fell swoop. Twice a year will suffice.
# NOTE: Remember to URL encode the @ sign when giving your email as password.
# Sourcing credentials from ~/.netrc here.
\
ftp -VMo - ftp://ftp.boulder.nist.gov/pub/time/leap-seconds.list | # Call NIST.
grep -o '^[^#]*' | # Strip out all the comments
tail -n1 | # Pluck out the last line
awk '{print $NF}' # Check current tally, game on if 38 or more.

Is there going to be further adjustments by 2035 when the new standard kicks in? Who knows. In the event that there is however, be advised, my code is watching and will be gladly taking a break to mark the occasion. With any luck then my efforts would not have been in absolute vain.

More