kmaasrud

Optimizing the hell out of my website

2022-09-26T13:00:00+02:00

Optimizing the hell out of my website

I considered my old website to be pretty minimal. It is mostly static and includes almost no JavaScript, but I wanted to go extreme.

Previously, my site weighed in at a pretty respectable 1.82 MB. That is not bad, and the size mostly comes from fonts which are easily cached. However, in my testing, it did take around 300 ms to load. That is not optimal IMO, so I want to cut down the number of requests in addition to the size.

Switching away from Hugo

Hugo has served as my static site generator of choice for a while now, and it has done it admirably. It does not contribute in any part to the size of my website (in fact, it actually has handy minimization techniques built-in.) However, I need greater flexibility in the steps I’m going to do to reduce my website’s footprint. Additionally, I wanted a more Unix-y and minimal feel to my site’s build process. Therefore, I’m going to make my own system, composed of regular Unix tools a small self-made program.

Inspired by mkws, I created sss. It is a small program that will take a directory of source files, pipe them through a collection of POSIX shell templates and create some output files. You can read more about how it works in it’s README, but it essentially just does some clever string interpolation. I was surprised at how quickly I got my Hugo setup converted to sss, and quickly proceeded to size optimization.

Going small

These are the steps I took in order to get my page as small as possible. The sizes reported are on-disk and not gzipped, so they only give an indication of how much I’ve saved.

Images

Images account for about half the size of the median web site¹. Simple optimization could reduce that by quite alot². We want to go extreme, which will involve the following process:

Start with a PNG file and optimize it with optipng, using the highest optimization settings (-o7.)
Convert the optimized PNG into WebP with the webp tools.

I did this with the following simple shell script, to automate the process for me:

#!/bin/sh

# Optimize PNG (make backup)
optipng -- -backup -o7 $1

# Convert into WebP
cwebp $1 -q $2 -o "${1%%.png}.webp"

# Remove original file
rm $1

The first parameter is the image file we want to optimize, and the second is the quality we want to use in the WebP compression (0-100 in increasing order of quality.)

The screenshot above had the initial size of ~1.2 MB, but the final optimized image ended up being ~22 kB. Quite a drastic improvement, if you ask me.

KaTeX

Math notation is very important for me, so KaTeX is a must. It is by no means big, but since I’m doing the Markdown rendering with Pandoc - which natively detects inline and display math - I figured I could get the rendering done at build time to save having to include the JavaScript library. This was as easy as using the Pandoc filter pandoc-katex (which luckily is available in nixpkgs.)

From there, I downloaded the KaTeX stylesheet and the corresponding fonts, in order to let me self-host them and save a few HTTP requests. We’re living in the modern days now, so I deleted everything except the woff2 fonts. Additionally, I went over the stylesheet and removed every redundant rule which I am not in need of (e.g. the \includegraphics support.)

This saved me ~600 kB worth of JavaScript and ~1 MB worth of fonts. However, the fonts don’t really count, since they are lazily loaded.

Fonts

Speaking of fonts, I would have to give up my beloved Inter. Instead, I’m opting at trusing the web browser and simply using the default fonts. I’m giving up a bit of predictability, but it is much better for the end user, as they can adapt the font choice to their preferences.

Once again considering the KaTeX fonts, I opted for setting font-display: swap for all of the fonts. This instructs the browser to actually render the markup, even if the fonts have not loaded yet, instead of blocking. This might lead to a bit of yank, but it ensures that the content is prioritized before the layout.

Removing Inter saved me ~800 kB, and I gained a bit of rendering speed by setting font-display: swap.

Stylesheets

I want my page to look the same everywhere, so like any good netizen, I use Normalize.css. To make it even more minimal, I removed any of the rules I’m never going to use (like forms and such), and inlined it into my main stylesheet. Speaking of my main stylesheet, I did as much as I could to keep it as short as possible, removing any fancy animations or stylings of stuff I were not planning on using anytime soon.

The stylesheet is now pretty optimized, so it can’t really be prioritized enough. For that reason, I decided that I wanted to inline everything in a <style> tag for every HTML file (except KaTeX, for caching reasons). Although this involves a (very small) penalty on the document size, it is far outweighed by the gains of not doing that extra GET request.

Lastly, I stumbled upon Lightning CSS and opted on using it to minify everything as much as possible. To me, it seems like it is the best CSS minimizer that’s currently available in nixpkgs.

The minimization saved me ~5 kB, and inlining the normalization and main stylesheet saved me two requests.

Minimization

Last, but not least, we arrive at minimizing the actual HTML file itself. There are a lot of interesting things which can be done to strip down byte after byte, like not closing some tags³, stripping out <html>, <head>, <body>, etc.⁴ and much more. This can quickly get tiresome to do manually, though, which is why we have minimizers. I found this one during my research, and after some testing, it gave me the best (smallest) results.

The pages are quite small to begin with, so HTML minimization only accounted for ~1 kB per page. Every byte counts, though!

Favicon

One extra step I really wanted to do, which does not contribute a lot, but is fun nonetheless, is to change the favicon to something smaller. Since SVG is an XML language which supports CSS and text tags, you can make an icon that just holds a single emoji⁵, which I presume is the most minimal you can get - unless you want to use a single square or something as your favicon. Just insert the following into your head, and exchange the emoji with whatever you prefer:

<link rel="icon" href='data:image/svg+xml,<svg xmlns=%22http://www.w3.org/2000/svg%22 viewBox=%220 0 100 100%22><text y=%22.9em%22 font-size=%2290%22>💭</text></svg>'>

The added advantage here, is that emojis look good on both light and dark mode, so you don’t need any clever media queries to make the favicon adapt to the device’s theme.

The differences here were in the range of single bytes, so I won’t bother you with any numbers.

Closing remarks

With the incredible networks speeds we currently have, all of this will likely be considered as unecessary by many. I would however implore you to think differently about it:

Take a second to think about the amount of visits bigger newspapers, social media platforms and the like get every day. Then think about the amount of data that is shipped to the end user that could be optimized, is unecessary or never even used. We could save a lot of bandwidth, computing power and subsequently energy by focusing on cutting out the cruft. As developers, we should serve as inspiration for this movement and implement this thinking into our own projects.

In some sense, we’re saving the world.

OPML is underrated

2024-02-07T10:52:00+01:00

OPML is underrated

As a response to the general enshittification of major platforms, I would say we are seeing a resurgence of the old web’s ethos, with personal blogs gaining traction and the concept of the small web on the rise. That might be colored by the digital communities I hang around in (which are mostly dominated by programmers) but it does at least empirically feel like a trend¹. That brings along with it a new interest in open web standards. Among them is RSS², which both I, and I think a lot others, have increasingly integrated into our digital routines to keep track of posts from people and sources we’re following. RSS aligns perfectly with the movement towards more personalized and controlled content consumption. Unlike the algorithm-driven feeds of other platforms — which often prioritize engagement over relevance or quality — RSS allows me to curate my own information stream. This feels important to me, as it gives me a level of autonomy over the content that shapes my views and knowledge, as opposed to handing that power over to advertisers.

There is an issue, though. RSS feeds can be a bit clunky to manage and keep track of. Their decentralized nature also makes discoverability an issue. Enter OPML, which is an outliner format that is most commonly used to store a list of feed subscriptions. I promise you; having a single file that stores all the feeds you’re interested in is a gamechanger, as it makes it significantly easier to organize, migrate, and share those feeds across different platforms and devices. Here’s an example:

<?xml version="1.0" encoding="utf-8"?>
<opml version="2.0">
  <head>
    <title>A list of feeds I follow</title>
  </head>
  <body>
    <outline text="My favorite blog" xmlUrl="https://a-cool-blog.tld/blog/feed.xml" type="rss"
      htmlUrl="https://a-cool-blog.tld/blog" description="You can also add a description" />
    <!-- more outline elements with feeds... -->
  </body>
</opml>

Each outline item must have the type rss (that goes for both RSS and Atom feeds) and must include the xmlUrl attribute. Optionally, you can specify some more attributes, like adding a title with text, a description with description and a link to the blog front page with htmlUrl — that added metadata can be very useful. Yes, it is XML-based, which I admit isn’t exactly the easiest format to work with, but it has a few advantages, which we’ll get back to.

With OPML, you don’t need separate applications or services to categorize feeds. Categorization can be achieved within a single OPML file through its outlining capabilities or by managing multiple OPML files, each dedicated to a different category or use-case. It is a very viable workflow to have one OPML file for your YouTube subscriptions, another for your favorite Twitter/X and Mastodon users, one more for news sites, and yet another for personal blogs — the world’s your oyster. However, there aren’t many application that support nested OPML outlines or categorizing based on different files, sadly, but there should be! This is a call to action, developers: Perfect side-project!

Beyond personal convenience, OPML has the potential to better the ecosystem of the small web. By not only sharing an RSS feed on your personal website, but also your list of subscribed feeds, we’re effectively creating a recommendation system that is based on concious curation, not statistical metrics. Your OPML file is now called a blogroll, and you officially get to call yourself a 90s web developer. Jokes aside, I believe the simple fact that there is a known person behind each recommendation is advantageous. Yes, this might promote smaller digital social circles, but I personally think the transparency of a known source is the best way to combat filter bubbles. That part is a whole sociological discussion in itself, so if you would like to discuss it further, I would love chatting about it on my mailing list.

Now, getting back to the fact that OPML is XML-based; I’d like to highlight an often-overlooked feature of this: The ability to use an XSL stylesheet to display the OPML file rendered through a HTML template when loaded in a browser. With this, you can add a short introduction and guide to the format, making the blogroll more accessible to those unfamiliar with it. It also opens the possibility to showcase each feed with added context or descriptions.

Here is an example XSL stylesheet you can use:

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:template match="/opml">
    <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
      <head>
        <title>
          <xsl:value-of select="head/title"/>
        </title>
        <meta name="viewport" content="width=device-width, initial-scale=1"/>
        <style> /* Insert CSS here */ </style>
      </head>
      <body>
        <p>
          This is a list of blogs and news sources I follow. The page
          is itself an <a href="http://opml.org/">OPML</a> file, which
          means you can copy the link into your RSS reader to
          subscribe to all the feeds listed below.
        </p>
        <ul>
          <xsl:apply-templates select="body/outline"/>
        </ul>
      </body>
    </html>
  </xsl:template>
  <xsl:template match="outline" xmlns="http://www.w3.org/1999/xhtml">
    <xsl:choose>
      <xsl:when test="@type">
        <xsl:choose>
          <xsl:when test="@xmlUrl">
            <li>
              <strong>
                <a href="{@htmlUrl}"><xsl:value-of select="@text"/></a>
                (<a class="feed" href="{@xmlUrl}">feed</a>)
              </strong>
              <xsl:choose>
                <xsl:when test="@description != ''">
                  <br/><xsl:value-of select="@description"/>
                </xsl:when>
              </xsl:choose>
            </li>
          </xsl:when>
        </xsl:choose>
      </xsl:when>
    </xsl:choose>
  </xsl:template>
</xsl:stylesheet>

You can link to the stylesheet in your OPML file by adding <?xml-stylesheet type="text/xsl" href="path/to/stylesheet.xsl"?> at the top. I actually do this on my own blogroll, so check that out if you want some inspiration.

While we’re all getting a bit fed up with the big platforms, OPML is like a breath of fresh air from the old web days. It’s all about making life easier when managing feeds, sharing cool finds, and stumbling upon new stuff. So I encourage you to create your own blogroll, slap it on your website, and share what you’re into. It’s a simple move, but it could spark some real connections and bring back a bit of that community vibe we miss.

My experience with Nix

2022-05-16T13:00:00+02:00

My experience with Nix

At the beginning of this year, I had read a lot about Nix and felt pretty confident that this wasn’t something for me. That was, until I did a voicecall with my good friend Victor. He was showing me these really neat tricks on how I could use Nix and Nix Flakes to handle configuration files, build software reproducibly and define my own development shell for each project I’m doing - I was blown away.

After that call, I immediately started with the (admittedly stupid) thing that every developer does when learning about a new technology. I started moving all my shit over to it: System configuration, package management, package configuration, etc. - I wanted it all reproducible. In this blog post I will document my experience so far. This is very far from a guide to using Nix, but you might find some useful hints from my mistakes.

Installing Nix

Okay, the first step was to aquire Nix. I use a Macbook Air M1, and I quickly discovered that Apple have made this a hastle. After Catalina, you are not able to write to the root directory of Macs, which meant getting the /nix directory up and running required hacky workarounds. Luckily, these are now intergrated into the installer, which made the install very simple.

$ sh <(curl -L https://nixos.org/nix/install)

did the trick for me. A reboot was needed, but then everything was up and running smoothly. Nice!

Setting up nix-darwin

The next step was to get my system built through Nix. This is a pretty wacky concept, but the advantages are incredible. Think of all the times you’ve bought a new computer or needed to do a factory reset. All those preferences you’ve tinkered with to perfection over the years: They do not need to be set up again! A git clone, one simple command, most likely a few minutes of building and you have all your apps and software up and running.

I started with this guide and tinkered a bit according to my system:

{
  description = "kmaasrud's system";

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
    darwin.url = "github:lnl7/nix-darwin/master";
    darwin.url.nixpkgs.follows = "nixpkgs";
  };

  outputs = { self, darwin, nixpkgs }: {
    darwinConfigurations = rec {
      mba = darwin.lib.darwinSystem {
        system = "aarch64-darwin";
        modules = [ ./configuration.nix ];
      };
    }
  };
}

I put some simple system configuration in configuration.nix, built the flake with

$ nix build .#darwinConfigurations.mba.system

and was joyful to see everything working. This gave me a symlinked directory named result, containing a binary sw/bin/darwin-rebuild which I could use to bootstrap the system from the same flake. To my dismay, I was met with an error that said I was missing the /run directory. The error message said I needed to include the line run\tprivate/var/run in /etc/synthetic.conf, which would create the firmlink I needed. I did this without luck and ended up giving up on the whole debacle after a long night of googling. That following morning - when I could think sensibly again - I had an idea only a genius like me could find: I did a reboot…

It worked.

Okay, 1-0 to Nix, but I had no reason to give up now. darwin-rebuild switch --flake . was working, and I was off to the races! The following days, weeks, months, I have been tinkering and my Nix system configuration to the core, and I am nowhere near considering myself done. If you are curious, you can explore my config yourself on GitHub.

Nix in projects

The whole system configuration thing is very cool, but I’ve found that the real magic of Nix comes when you start using it in your projects.

Nix serves as a very flexible build tool for your software. The builds are done inside a siloed environment, which means you need to declare all the dependencies of your build. This is immensely useful! Software is built the same on all computers, which especially makes CI a lot easier.

In my project MDoc, which depends on Tectonic (and subsequently a bunch of C/C++ libraries), I am able to do the build in a shell that has all the required dependencies without actually having to manually install said dependencies. They are listed in the following way:

buildInputs = with pkgs; [
  fontconfig
  graphite2
  harfbuzz
  icu
  libpng
  perl
  openssl
  zlib
] ++ lib.optionals stdenv.isDarwin [
  ApplicationServices
  Cocoa
];

Nix uses this list and makes sure these are linkable when building - so smooth! As you can see, software is built the same on all computers wasn’t an entirely true statement after all, since I need some specific dependencies for Darwin systems. However, when they can be conditionally defined this way, who am I to judge.

Conclusion

Allright, that is a misleading header; My Nix journey has only just begun. What I can say, is that Nix has changed the way I look at software. It would be very hard to go back to the old ways of a mutable system and non-reproducible development environments. I am convinced that Nix is the future - in some way or the other. It just needs a facelift, because there is no way I am accepting nix-* type commands.

A love letter to make

2023-04-19T13:00:00+02:00

A love letter to make

Dear make,

As I sit here thinking about you, I can’t help but feel a rush of excitement in my heart. I know that to some, you may seem like a dry and unromantic tool, but to me, you are the epitome of true love. Your simple syntax and powerful capabilities have stolen my heart and made my programming process so much smoother. Who knows how many keystrokes you have saved me since we first met? I used to spend hours typing out long and complex build commands, but now, with you by my side, those days are long gone. I shudder to think about how much time and energy I would have wasted without you.

Your knack for automating commands and streamlining my build, is like a loving partner who knows exactly what I need before I even ask. It’s as if you have a sixth sense for my needs and desires. You’re like a chameleon that can adapt to any language or environment. That kind of versatility is seriously impressive, and it makes me feel like you’re always up for a new adventure.

And what really sets you apart, is your reliability. You never let me down or give me any trouble. You always do exactly what I tell you to do, without any fuss or drama, and you’re always there wherever I go. That kind of dependability is what every programmer dreams of in a partner.

But, let’s be real here, make, you’re not perfect. I have to admit that sometimes you can be a little old-fashioned and quirky, with your insistence on using tabs instead of spaces, and your archaic syntax that can take some getting used to. And don’t even get me started on those .PHONY targets. Despite your quirks and flaws though, I still love you. Because when it comes down to it, your benefits far outweigh your drawbacks. You make my programming process so much more efficient and streamlined, and you save me countless hours of tedious work. Plus, you’re always there for me when I need you, and you never let me down.

So here’s to you, make - quirks and all. You may not be perfect, but you’re pretty darn close. Thank you for being the tool that I never knew I needed, and for making my life as a programmer so much easier and more enjoyable.

With love and a pinch of frustration,

A Hopelessly Devoted Programmer

You should track your finances in TOML

2023-11-13T13:00:00+01:00

You should track your finances in TOML

No, really! Let me explain.

I am quite particular with my finances. Having a proper accounting system helps me keep control of my spendings and what I am saving. Being a pretty nerdy guy, I quickly gravitated towards the plain text accounting movement, with which I ensure that I can control my data, switch systems if I want and do custom scripting around my ledger. I won’t delve into the details here, but the plaintextaccounting.org website is a great source of information, and I recommend reading through the Newcomer FAQ if you want an introduction to the concept.

The most popular tools for plain text accounting today are Beancount, Ledger and hledger. They are all fantastic tools, using similarly inspired DSLs that are pretty easy to learn. However, their syntax deviate in subtle ways. Here is an example entry written in Ledger first and then Beancount:

2023/11/06 My local grocery store | Grocery shopping
  Assets:ExampleBank:Checkings  $-20
  Expenses:Food:Groceries        $20

2023-11-06 * "My local grocery store" "Grocery shopping"
  Assets:ExampleBank:Checkings  -20 USD
  Expenses:Food:Groceries        20 USD

They are not compatible with eachother, and converting between the two is non-trivial. There are countless examples of features (transaction flags, prices, capitalization conventions, metadata, etc.) that are non-interoperable like this.

Now say you want to write some scripts for processing your journal. This means you’ll have to scour the internet for a parser supported by your language of choice, or write one on your own. Let me tell you, they are not abundant, and good luck finding a non-official library that is regularly updated. You might get forced to use a language you would not choose yourself, just to get some simple processing done.

Many of the reasons these tools have opted for a DSL, is to keep the amount of necessary typing to a minimum—an admirable goal which they achieve quite well in my opinion. However, most users use automated scripts, data-entry tools or at the least some editor completion to insert their data. Why, then, can’t we just use a standard plain text data serialization language?

Advantages of using a proper data serialization format

The advantages are obvious, given my gripes listed above; almost all of them are solved:

Interoperability: It is trivial to convert from one data format to another. Just look at the serde framework for Rust, or the countless number of conversion tools available online.
Language support: You can expect to find a high-quality maintaned library for parsing any of the big data formats in most languages. These often also deserialize into native data-structures, which should be easier to handle when scripting.
Standard: This hooks a bit into interoperability, but the key here is that you can expect your data to be easily used by or ported to any tool or service you want to use in the future. Not only that, but having a simple map to e.g. JSON makes the path to a web-accessible API quite clear.

Why TOML?

With these ideas in mind, why am I saying that TOML specifically is the solution? Well, the homepage says it all:

A config file format for humans.

The whole ethos around plain text accounting is that it can be read in it’s raw form and edited by hand if so desired. These goals are at the core of TOMLs design. Here is the equivalent transaction I showed above, written in my proposed TOML ledger specification:

[[transaction]]
date = 2023-11-06
description = "Grocery shopping"
payee = "My local grocery store"
postings = [
  { account = "Assets:ExampleBank:Checkings", amount = -20, currency = "USD" }
  { account = "Expenses:Food:Groceries", amount = 20, currency = "USD" }
]

Definitely more verbose, yes, but still reasonably short and totally legible (and perhaps even easier to understand without knowing the syntax beforehand.) Note also that TOML has a datetime type, which is perfectly suited for this use case.

Now this example leaves a lot to be desired. Evidently, writing out the currency in this way for each posting is not ideal, but could be solved by allowing the user to set a default currency. Accounts also borrow the colon-separated syntax from ledger, but I think this is also something that could be improved upon. There are also a lot of crucial details I have glossed over here—for example currency prices, posting metadata, balance assertions and more—but I would say these are all pretty low-hanging fruit.

If you like the concept and have thoughts or ideas that you would like to share, please send an email to my public inbox, and I’ll be happy to discuss! I’m doing some testing in a CLI called mynt, which initially is for personal use, but feel free to have a look at it for reference, contribute a patch if you see an improvement that can be made or even try it out yourself.