About 2 weeks ago, I decided to re-write my non technical blog in Astro. And this led to a lot of interesting things.

[Apologies for any typo: I wrote it late and didn't proof read it yet, I'll do it tomorrow, but it's a topic I've been focusing for a bit and I want to get it out]

Astro

The stack of the site was Ghost, Gatsby and Netlify, and I didn't want to change the first or the last, just how I built it to static. A friend recommended Astro a while back and whilst I was interested, I was very fond of Gatsby's image optimisation to do the switch. Lately I found out that Astro had a similar set of features behind an experimental flag for a bit, and I wanted to try them. So I did.

Rewriting the blog itself wasn't too painful: the site is rather straightforward as Astro supports React and I could have a transition by mounting the exact same components and there's a decent Astro Starter Ghost that smooths out things a lot.

The main problems started when I wanted to optimise the images. Astro offers 2 ways of dealing with images:

---
// import the Image component and the image
import { Image } from 'astro:assets';
import myImage from "../assets/my_image.png"; // Image is 1600x900
---

<!-- `alt` is mandatory on the Image component -->
<Image src={myImage} alt="A description of my image." />

or

---
import { getImage } from "astro:assets";
import myBackground from "../background.png"

const optimizedBackground = await getImage({src: myBackground, format: 'webp'})
---

<div style={`background-image: url(${optimizedBackground.src});`}></div>

neither was working out of the box with my setup though, as I was pulling HTML out of a CMS and mounting it with React.

Why React?

Let's open a brief parentesis on why I had to use React: the preview feature.

I didn't want to lose the ability to render my content in the actual page layout before publishing it, and that meant being able to load it client-side with an xhr call and apply the same manipulation logic I was applying during the build – more in a second.

I'm sure there are smarter ways of doing what I was doing, but as far as I understand, as I'm writing, rendering Astro clientside on a netlify function is an experimental and unstable feature and I had already my feature in React working out of the box.

I decided to invest in figuring out a way of dealing with the images instead.

Working with the HAST tree

As I mentioned, I was applying already a set of manipulation to the HTML received from Ghost. Not all of it was particularly clever, but I pulled in Rehype to swap some tags to custom components and apply custom logic.

That was a blessing. I since moved away from Rehype but by using hast-util-from-html and some related utilities I could do something on these lines.

import { getImage } from "astro:assets";
import { fromHtml } from "hast-util-from-html"
import { map } from "unist-util-map"

const { post } = Astro.props as Props

post.htmlAst = fromHtml(addNotes(post.html), { fragment: true })

map(post.htmlAst, function (node) {
  if (node.tagName === "img") {
    node.properties.src = await getImage({src: node.properties.src, format: 'webp'})
  }
  return node
})

That wasn't hard, except it sucks

Which is quite nice, but has 2 essential drawbacks:

  1. the build gets incrementally slower as I add more and more content (I was already tracking around 5 min per deploy
  2. it completely ignores the srcSet property of the img node.

The latter in particular was a problem: I could either process the srcSet urls too, but then the build time would've exploded, or remove the srcSet, which meant delivering non optimised images, and then what's the point of making the effort in the first place?

I was moaning to a friend about this, and he pointed me to an idea I had already implemented a while back in a branch and then forgot about it: CDNs.

The volume of my blog is very low, and most free tiers would be just fine. I picked imgKit and with a couple of lines of code and having added my domain to its configuration I got it up and running in minutes.

// utils/cdn.js

const config = {
  thumb: "tr:w-640,h-360,c-maintain_ratio",
  search: "tr:w-400,h-225,c-maintain_ratio",
  cover: "tr:w-1920,h-1080,c-maintain-ratio",
}

const isSafe = (url) => url.startsWith("https://cedmax.net")

export const withCdn = (url, type) => isSafe(url) ? 
    `https://ik.imagekit.io/cedmax/${config[type]}/${
        url.replace("https://cedmax.net/", "")
    }` : url

export const srcSet = (url) => {
  if (isSafe(url)) {
    return url
      .replace(/https:\/\/cedmax.net\//g, "https://ik.imagekit.io/cedmax/")
      .replace(/content\/images\/size\/w(\d+)\//g, "tr:w-$1/content/images/")
      .replace("//", "/")
  }

  return url
}

CDN FTW!

import { withCdn, srcSet } from "~/utils/cdn.js";
import { fromHtml } from "hast-util-from-html"
import { map } from "unist-util-map"

const { post } = Astro.props as Props

post.htmlAst = fromHtml(addNotes(post.html), { fragment: true })

map(post.htmlAst, function (node) {
  if (node.tagName === "img") {
    node.properties.src = withCdn(node.properties.src)
    node.properties.srcSet = srcSet(node.properties.srcSet)
  }
  return node
})

source set covered

This took the build down to 1 minute, and the amount of imagery won't change it.

Simplifying things

Another interesting benefit of moving from React to Astro is the simplifications I got along the way. Working with Astro is really refreshing as it puts you in a very different mindset as you can kinda stop thinking about refresh cycles, executions etc etc, and focus on the outcome.

The most astonishing one for me, was to figure out that as soon as I stop thinking about things in React terms that I had massively overengineered the notes.

I wanted to cross link annotations like this:[^1]. The first one should have been an anchor

<sup> <a href"#fn-1" id="bfn-1">1<a> </sup>

The second, opening the footnote, a span with the appropriate id, and then, after the following text, I wanted to add a link back to the original one

<span id="fn-1">1.</span> note text whatever it might be <a href="#bfn-1" aria-label="back to reading">↵</a>

Being in a React world, I thought this as a component, and I thought this as a component mounted as a replacement in the Rehype setup, so each wouldn't really have knowledge of any other. Cross linking references was a nightmare.

I eventually came up with this, and I was so proud of it working.

import React from "react"
import reactStringReplace from "react-string-replace"

function escapeRegExp(string) {
  return string.replace(/[.*+?^${}()|[\]\\]/g, "\\$&") // $& means the whole matched string
}

const footnotes = []
const Sup = ({ note }) => (
  <sup>
    {" "}
    <a
      data-smooth-scroll-to={`fn-${note}`}
      id={`bfn-${note}`}
      href={`#fn-${note}`}
    >
      {note}
    </a>
    {"  "}
  </sup>
)

const refMatch = (item) => {
  if (item instanceof Array) {
    return item.map(refMatch)
  }
  if (typeof item === "string") {
    const match = item.match(/\[\^(\d)+\]/g)
    if (match) {
      const footnum = match[0].replace(/[\\[^\]]/g, "")
      footnotes.push([match[0], footnum])
      const re = new RegExp(`\\[\\^(${escapeRegExp(footnum)})\\]`)
      return refMatch(
        reactStringReplace(item, re, () => (
          <Sup key={footnum} note={footnum} />
        )),
      )
    }
  }
  return item
}

const ftnMatch = (items) => {
  if (footnotes.length) {
    const matchedIdx = footnotes.findIndex(([match, noteRef]) => {
      const re = new RegExp(`${escapeRegExp(match)}`)
      return items.find((item, i) => {
        const isMatch = typeof item === "string" && item.match(re)
        if (isMatch) {
          const re2 = new RegExp(`\\[\\^(${escapeRegExp(noteRef)})\\]`)
          const component = reactStringReplace(item, re2, () => (
            <span key={`fn-${noteRef}`} id={`fn-${noteRef}`}>
              {noteRef}.
            </span>
          ))
          items.splice(i, 1, component)
          items.push(
            <a
              key={`bfn-${noteRef}`}
              aria-label="torna alla lettura del post"
              data-smooth-scroll-to={`bfn-${noteRef}`}
              href={`#bfn-${noteRef}`}
            >
              ↵
            </a>,
          )
        }
        return isMatch
      })
    })

    if (matchedIdx > -1) {
      footnotes.splice(matchedIdx, 1)
    }
  }
  return items
}

const WithNotes = ({ component: Comp, children, ...props }) => {
  return <Comp {...props}>{children && ftnMatch(children).map(refMatch)}</Comp>
}

export default WithNotes

What was I thinking?

I'm not going to explain the whole thing, mostly because it's a waste of time and because the moment I realised the alternative solution (linking things BEFORE translating them to HAST) I facedpalmed myself so hard I still wear the signs of it.

const escapeNoteBlock = (noteBlock) =>
  noteBlock.replace("[", "\\[").replace("^", "\\^").replace("]", "\\]")

export default (html) => {
  let matches
  while (
    (matches = [
      ...html.replace(/[\r|\n|\r\n]/g, "").matchAll(/(\[\^\d+\]).+\1/g),
    ][0])
  ) {
    const noteBlock = matches[1]
    const noteNum = noteBlock.match(/\d+/)[0]

    const noteLink = `<sup>&nbsp;<a data-smooth-scroll-to="fn-${noteNum}" id="bfn-${noteNum}" href="#fn-${noteNum}">${noteNum}</a>&nbsp;</sup>`
    const footNote = `<span key="fn-${noteNum}" id="fn-${noteNum}">${noteNum}.</span>`
    const footLink = `<a key="bfn-${noteNum}" aria-label="torna alla lettura del post" data-smooth-scroll-to="bfn-${noteNum}" href="#bfn-${noteNum}">↵</a>`

    html = html.replace(noteBlock, noteLink)

    const footNoteMatch = html
      .match(new RegExp(`${escapeNoteBlock(noteBlock)}.*?<\/p>`))[0]
      .replace("</p>", "")

    html = html.replace(footNoteMatch, `${footNoteMatch}${footLink}`)
    html = html.replace(noteBlock, footNote)
  }
  return html
}

I kinda like Regex, I do swear a lot using them though

A Regex matching pairs of [^\d] on the HTML and make the appropriate substitutions. THEN building to the HAST 🙈

Raindrop Integration

After rewriting the blog, other sites got my attentions: I've been collecting links to interesting articles for a decade and for the past few years I also started publishing songs that got stuck in my head.

I used to have Tumblr as a backend and then publish a static website pretty much in the same way I did with the blog, but I had it on my list to integrate everything on the same site for a while.

The simplicity Astro brought, made me want to do it straight away, but first I had to find a decent substitute for Tumblr.

The choice ended up being Raindrop. I used it already for bookmarking and I discovered that its API is quite flexible: I could upload up to 100 links in one call, and set the creation date at any point in time, maintaining the history.

Migrating from Tumblr became a mere exercise of parsing a Tumbr backup

const fs = require('fs')
const cheerio = require('cheerio')
const _ = require('date-fns')
const chunk = require('lodash/chunk')

const data = []
const dirname = './html/'
const filenames = fs.readdirSync(dirname)

filenames.forEach(filename => {
  const content = fs.readFileSync(dirname + filename, 'utf-8')
  const $ = cheerio.load(content)

  const url = $('iframe').attr('src')
  const title = $('iframe').attr('title')
  const date = $('#timestamp')
    .text()
    .trim()

  data.push({
    title,
    link: url,
    created: _.parse(date, 'MMMM do, yyyy h:mma', new Date()),
    pleaseParse: {},
    collection: {
      $id: process.env.COLLECTION_ID
    }
  })
})

const payloads = chunk(data, 100)

;(async () => {
  for (const payload of payloads) {
    axios.post(
      'https://api.raindrop.io/rest/v1/raindrops',
      { items: payload },
      {
        headers: {
          Authorization: `Bearer ${process.env.TOKEN}`
        }
      }
    )
  }
})()

This is nice

Pulling the page together was a breeze following the Ghost implementation, just the matter of calling the Raindrop API. 

import axios from "axios"

const {
  RAINDROP_BEARER: bearer,
  DSGN_ID: linkCollectionId,
  SONGWORM_ID: songsCollectionId
} = import.meta.env || process.env

const cache = {
  links: [],
  songworms: [],
}

const getUrl = (collectionId, page) =>
  `https://api.raindrop.io/rest/v1/raindrops/${collectionId}?sort=-created&perpage=50&page=${page}`

const fetchPage = async (url) => {
  const { data } = await axios.get(url, {
    headers: { Authorization: `Bearer ${bearer}` },
  })

  return data.items
}

const fetchAll = async (collection, collectionId) => {
  if (!cache[collection].length) {
    let page = 0
    let localitems = []
    do {
      localitems = await fetchPage(getUrl(collectionId, page))
      cache[collection] = [...cache[collection], ...localitems]
      page++
    } while (localitems.length > 0)
  }
  return cache[collection]
}

export async function getAllLinks() {
  const collection = "links"

return fetchAll(collection, linkCollectionId)
}

export async function getAllSongworms() {
  const collection = "songworms"

  return fetchAll(collection, songsCollectionId)
}

The cache is not technically needed for the build, but I didn't want to hammer the API during development

I connected Raindrop to the Netlify build with Pipedream because the free tier should cover for my needs, but there are surely other options out there if it doesn't work for you.

Conclusions

Astro gave me back the pleasure of fiddling with HTML and wiring APIs without having to bother about all the internal specifics of React, the performance bottlenecks, side effects, etc. It felt refreshing, like I was finally focusing on what mattered as opposed as trying to master a library.

I'm still very fond of React, don't get me wrong, but I don't think it'll be my go to for a quick website now on. I can still use it (and I'm using it) for client heavy logic pieces like the search result or the preview, as mentioned, but it's a limited and unobtrusive approach, instead of ubiquitous.

And these results are much easier to achieve

and that 93 is due to a Netlify feature