So when Pimoroni released its Badger 2350, I was very interested and I pre-ordered one straight away. It includes everything you need out of the box: wifi, bluetooth, buttons and a battery. You can get the full technical details on the Pimoroni website. You can program it using micropython and Pimoroni developed a few easy to use libraries to draw on the screen, read the buttons, connect to wifi and so on. I was not really excited by using python for this project to be honest. That’s ok, I’ll start with that, but I might rewrite that firmware in a lower level language later on, like Rust, Zig or even C. We will see.

Anyway I got mine in the mailbox a few weeks ago and then I thought “Now what? What am I going to do with it?”.

I first developed a simple badge for myself when I go to conferences, simple enough.

Then I got an idea. I have a Home Assistant installation at home connected to a few zigbee devices, a cat feeder, a couple of smart bulbs, plugs and a few temperature sensors. I have a temperature sensor in my office. I thought it would be fun to use that badge to display its data.

Now how can that work? zigbee2mqtt manages the zigbee devices and exposes them to Home Assistant via the message broker mosquitto. The data I’m interested in is available via mosquitto. If I had an MQTT client and listened to the sensor topic, I’d get the data I need. The badger would need to connect to the WIFI and implement an MQTT client. Looks like a plan.

Unfortunately after a few tests I saw that it couldn’t work that way. My router is too far away and the WIFI signal is too weak for the badge to connect to it.

Second option is for the badger to get the data via bluetooth. It is actually a better option for a low powered device like that. Bluetooth and especially BLE is more energy efficient than WIFI. And it happens that my MQTT broker runs on a Raspberry Pi 5 which has bluetooth capabilities. Here is a diagram of how that would work:

  flowchart LR
      A[Sensor] <-->|Zigbee| B[zigbee2mqtt]
      B <-->|MQTT| C[Bridge app]
      C <-->|BLE| D[Badger]
      B <-->|MQTT| E[Home Assistant]
  

All I need is some kind of bridge between the MQTT broker and the badge. Ok, fair enough, this won’t work out of the box.

But this is great actually! When starting this project, this is exactly the kind of problems I like to run into. I now need to write this program that will run on the Raspberry Pi, and plug the service into my NixOS configuration. I decided to use Rust for this project. I’m starting to have a bit of experience with that programming language and I know that the Rust ecosystem has everything I need to write this program.

I called that program mqttooth. Check it out. The source is available on GitHub along with my fork of the Badger 2350 firmware.

The Problem

As a longtime digital music enthusiast, I’ve spent decades curating my music library. Over the years, I’ve tried countless tools to keep it organized, ranging from Windows GUIs (forgive my younger self) to Linux command line utilities, and even web-based server apps.

My current workflow looks like this:

• Normalize audio files using mp3gain, aacgain, vorbisgain, or metaflac, depending on the format.
• Use beets to scrape metadata, tag files, and organize them.
• Check collection consistency with bliss.
• Edit tags as needed with various CLI tools, including eyeD3, id3, and tagutil.

Yes, it’s complicated. I need to simplify this workflow.

htagcli

Here is a short demo of htagcli in action:

While htagcli doesn’t yet streamline the entire process, it’s a solid step toward taking ownership of my music management workflow.

Currently, htagcli can replace both a command line tagger and bliss to check the library consistency. I plan to add auto-tagging features similar to beets, and I’m still considering how to handle normalization. We’ll see how it evolves.

I like these well-scoped personal projects, small but non-trivial, and which solve a real problem for me. Looking at the git history, I realized I started this three years ago but didn’t have the time to push it forward. I’m thrilled to finally have it in a very usable state.

Roadmap

Here are the features I plan to implement soon:

• Handle disc ID tags
• Check that cover art size is within a configurable range
• Ensure no missing tracks in an album
• Verify genre consistency at the artist level¹

Another major goal is to scrape metadata from third-party sources like MusicBrainz, Discogs, and Bandcamp to automatically tag files correctly.

I built a retro gaming handheld based on a raspberry compute module 3 sometime ago. It was a super fun project. I often think my work as a developper misses a bit of manual work. Something that I love to do. This project was great for that reason as it involved a bit of soldering, 3D printing, hot gluing and precise dremmelling.

Anyway, after I built it, I installed a custom Linux distribution based on retropie made by the author of the board. It’s a bit rough in the edge. But it’s been working fine so far, can’t complain.

Unfortunately, after a couple of years, you know, software gets out of date and that distribution needed to be maintained. Seeing that the latest image for retropie itself is from March 2022, what can you expect from a super niche distribution built on top of it? Not much, only a couple of forks here and there, but nothing actively maintained.

I wanted to tackle this issue and bring the latest and greatest software to this handheld I built myself which has been sitting on my desk for too long. Additionally, this is one of the last piece of hardware I have not nixified yet.

So here am I, installing NixOS on a gameboy handheld. How cool is that?

I named the project circuix-sword based on the name of the board: Circuit-Sword. If by chance you are one of the lucky few to have one of those, you can now run NixOS on it. It is very easy. Get started by downloading the latest image from the releases page. Then burn it on an SD card, and you are ready to go. All the instructions are in the README.md of the repository.

While working on this project, I ran into countless issues, which is part of the reasons I wanted to do it in the first place. I will describe only a few of them here in chronological order.

Boot the system

This one wasn’t too hard, I just downloaded the latest NixOS image for aarch64, and it was able to boot on the CM3 out of the box. I just plugged a screen to the HDMI out, a keyboard to the USB port, and I was ready to go.

WiFi

There is not much you can do without internet. I first reached out for a USB ethernet adapter I had around. It is good enough for hacking but not great for day to day use.

Hopefully, the Circuit-Sword has a WiFi chip on board. It is a Realtek 8723bs but requires two things for it to work properly:

• The right firmware. This is solved using the option: hardware.enableRedistributableFirmware = true; The closure induced by this option is quite big however (not less than 589 MiB). But we will address this issue later on.
• The sdio overlay. This requires messing up with the device tree. NixOS doesn’t provide a nice solution for that yet. But I found this one which works good enough for me.

Screen, first attempt

Next, I needed to get the built-in screen working. The Circuit-Sword has a 320x240 DPI screen. Making it work was just a matter of copying the right lines from the original distribution in the config.txt file.

While this worked fine for the console output, it does not for graphics. The drivers used in the original distribution have been deprecated to be replaced by the KMS DRM drivers.

Screen, second attempt

Ok, now let’s make those KMS DRM drivers work.

First, I needed to mess again with the device tree and enabled the vc4-kms-v3d overlay, which is the KMS driver. I also needed the vc4-kms-dpi-generic overlay to drive the DPI screen. That second overlay is especially tricky to get right as it needs a lot of parameters to work properly. One can map the original parameters from the config.txt file to the new format. But the documentation is not very clear, and it took me a fair deal of tinkering to get it right.

But wait, this is not enough. Now the SDL library needs to support the KMS DRM driver too. I needed to tweak slightly the compilation of SDL to enable it.

At this point, I was able to run retroarch and start games, that was already a big achievement!

Sound

The sound card is a MicroII chip. For some reason, it is stuck at a very high volume. The Linux module needs to be patched for it to work properly.

The cs-hud service

The cs-hud service is a custom service written in C especially for the board.

• It handles the safe-shutdown circuit
• When pressing the special mode button, it shows a help screen on top of the framebuffer. This screen shows the keyboard shortcuts along the current state: the levels of the volume, brightness and battery.

I integrated the source code in the git repo, created the derivation to build it and wrote the systemd service to run it. Standard NixOS stuff so far.

Unfortunately, with the new KMS setup, it is not possible to write an image on top of the framebuffer anymore. I then hacked the original code and removed pretty much everything related to that OSD feature. I kept the safe-shutdown handling, the brightness and volume control, and the battery level management.

On the way, I lost the ability to see all the info from that help screen. I can cope without it most of the time. But at some point I will definitely need to be able to see the battery level.

In the next section, I will explain how I solved that issue.

retroarch

The original distribution runs emulationstation. At first, I tried to make it work in NixOS. But soon I realized that emulationstation is actually some kind of frontend on top of retroarch. To make things simpler, I decided to just use plain retroarch directly, at least for a start. That project is involved enough already.

I have made two quality of life improvement changes to retroarch:

• I fixed the NetworkManager WiFi driver. It is now possible to set up the WiFi using the gamepad itself which is great as it wasn’t possible in the original distribution (merged upstream).
• I exposed the battery state to a unix socket using cs-hud and made it available in retroarch. This way, I can see the battery level in the retroarch interface. I keep that change in my fork as it has no chance to be merged upstream.

Closure size optimization

At this point, I had a working system, but it was way bigger than it should be: 1.81 GiB for the image of the version v0.0.2. I started to look at the closure to see what I could remove. This is achieved by tweaking the compilation options of various packages using nix overlays.

The most obvious thing was everything related to the desktop, namely: xorg, wayland, pulseaudio, pipewire and so on. gtk was especially difficult to get rid of. NetworkManager pulls it in the openconnect plugin but removing all its plugins was not enough for some reason. The solution was to get rid of openconnect itself.

The mesa library was pretty big as well (197 MiB). I trimmed it down to 20 MiB.

The redistribuable firmware derivation is about 589 MiB for only one single firmware needed! I did my own derivation for the firmware I need. The resulting derivation is now 41 KiB. That’s much better!

The retroarch assets derivation is still huge (494 MiB) but I’m ok with that. I don’t want to track down every single asset and check if it is needed or not.

I wanted to get rid of perl as well. But it didn’t work out nicely.

The final image is now around 800 MiB, under 1 GiB, which is not great but good enough.

Firmware flash

The last piece of the puzzle is the gamepad firmware that runs on the arduino. I wanted the source to be part of the repo as well. I also made it sure it compiles and make it possible to flash it out of the box.

Conclusion

This has been a difficult but fun journey. I love this project. Building the handheld was a great experience and porting the software to NixOS was also super fun.

There are so many different parts to it, going from very low to high level that makes it super interesting. From the Linux drivers that must be patched, to fine-tuning the booting process, the software that needs to be patched as well, some custom components, the gamepad firmware, not to mention the closure size optimization.

Really, there is something for everyone.

I have found out that NixOS is incredibly good at gluing all these pieces together. The fact that it is possible to patch every single package in one single repository is a huge win. That whole project took me some time. But the final amount of code is fairly reasonable. And I am very confident that I will be able to maintain it for a very long time.

This is a big deal because it directly affects the length of the feedback loop. I have been writing about it in the past. And I do think that a short feedback loop is the key to a good productivity.

Back to the story, until recently, ghci were unable to load multiple components, say an executable, like a test suite for example, and the library it depends on at the same time. And ghci is the centerpiece to get the fastest feedback loop when working with Haskell. Until now, working on a feature usually consists in doing some kind of back and forth between working on the library that contains the actual code and working on the test suite to make sure it works, restarting ghci between each iteration. This is far from being ideal and makes the feedback loop way longer than it needs to be.

That’s only a concern for cabal users by the way. stack have had a workaround for this for a while (see the documentation for stack ghci). What it does is merging all ghc options of the components before loading them in ghci. Basically, it is like creating a fake component which is the union of all wanted components. Pretty basic, but it has been an effective workaround for this issue.

But now it is possible to do it as well in cabal. At last, hurrah 🎉

Now, how does that work in practice?

The first thing is of course to have at least the versions of ghc 9.4and cabal 3.12.

Then cabal must be configured to enable the feature. When trying to load two components with cabal. It outputs a pretty informative message:

$ cabal repl test:my-app-tests lib:my-app
Error: [Cabal-7076]
Cannot open a repl for multiple components at once. The targets 'my-app' and 'tests' refer to different components..

Your compiler supports a multiple component repl but support is not enabled.
The experimental multi repl can be enabled by
  * Globally: Setting multi-repl: True in your .cabal/config
  * Project Wide: Setting multi-repl: True in your cabal.project file
  * Per Invocation: By passing --enable-multi-repl when starting the repl

As for me, I just turn it on globally. It is just too useful.

Once cabal is configured, that’s it really. If you try again that last command, that should work. Then after you edit a file in any of the loaded components, all required files will be recompiled by a simple :r in ghci.

The feedback loop I explained here works out of the box. Interestingly, it seems that it sometime makes ghc output the absolute path of the files that contains errors. This workaround is thus not always needed which is also awesome.

One caveat though, ghcid is able to run a command after the compilation succeeds. It is super useful to run the tests on any change for example. Let’s try this:

$ ghcid -c "cabal repl test:my-app-tests lib:my-app" -T :main
Command is not supported (yet) in multi-mode

Oh no, that doesn’t work.

But there is a way to work around this. I learned it in this video (recommended watch to see the feature in action). Instead or running a command, we’ll just execute the main. For example, if the main function is in the Main module, we can do this:

$ ghcid -c "cabal repl test:my-app-tests lib:my-app" -T Main.main

This works fine. Watch out that the executable must come first in the list of the components for some reason. If one needs to pass arguments to the main function, it is possible to use the :set command in ghci.

For example:

$ ghcid -c "cabal repl test:my-app-tests lib:my-app" -s ":set args -p /testpattern/" -T Main.main

That’s it. Once one tried it, it is hard to believe we have been waiting for so long for this.

Note: cabal documentation

But first, a short demo of me working on one of my projects:

Basically, I have a tmux session with two panes. On the left, I have my code editor neovim. On the right, I have the compiler running each time I save a file.

When the compiler finds an error in my code, with a shortcut I find the error in the compilation panel and yank it. Back to neovim, with another shortcut I jump straight to the error in my code, fix it, save the file, rinse and repeat.

Moving between panes

By default, moving the focus between panes in tmux is done with CTRL-b followed by an arrow key (as for me, I use CTRL-Space as my tmux prefix). In neovim, to move the focus between windows, one uses CTRL-w followed by a vim direction key: h, j, k and l.

I find it cumbersome. To make it easier, I use the neovim plugin tmux.nvim. With it, one can move the focus with CTRL-h, CTRL-j, CTRL-k and CTRL-l seemingly between tmux panes and neovim windows.

Check out the README.md of the plugin for details on the configuration.

Compilation pane

For the compilation pane, I run the compiler through feedback. By default, this tool watches for changes in files tracked by git and runs a command on any change. 99% of the time, this is what I want. For the remaining 1%, feedback has a few options to define the files to watch. Have a look at its repository for more information.

Now, to find the error in the compilation pane, tmux is able to search for regexes. I use this feature in my ~/.config/tmux/tmux.conf and map such searches to keybindings.

# Find errors and warnings in the current pane
bind C-e copy-mode \; send -X search-backward "[^\s]+:[^\s]+: (error|Error):"
bind C-w copy-mode \; send -X search-backward "[^\s]+:[^\s]+: (error|Error|warning|Warning):"

With this configuration, if I hit CTRL-Space (my tmux prefix) followed by CTRL-e, tmux will search in the current pane for an error and highlight it. With CTRL-Space followed by CTRL-w, tmux will search for an error or a warning.

Once the error has been found, I can yank it with Enter. The error string is now available in the system clipboard and, in neovim, in the + register.

Neovim pane

Back to neovim, to jump to the error I can enter command mode, type :edit and paste the error string with CTRL-SHIFT-v or CTRL-r-+.

:edit some/file.zig:12:10: error:

But don’t hit Enter yet, that’s not a proper vim command. I still need to edit the line such as:

:edit +12 some/file.zig

or even simpler:

:edit some/file.zig | 12

Now hit Enter and neovim will jump to the file and line number.

This is already better than trying to find the error by hand, open the file and scroll to the right line number. But we can do better.

Lua to the rescue

Here is a Lua module that can go in your ~/.config/nvim/lua directory. Let’s call it parse_error_string.lua for example.

local M = {}

local parse_error_string = function(str)
  -- src/Parser.hs:(30,3)-(32,11): warning:
  local file, line = str:match('([^:]+):%((%d+),%d+%)%-%(%d+,%d+%): .+')

  -- src/Parser.hs:10:3-8: error:
  if not file then
    file, line = str:match('([^:]+):(%d+):%d+%-%d+: .+:')
  end

  -- src/Model.hs:4:12: error:
  if not file then
    file, line = str:match('([^:]+):(%d+):%d+: .+:')
  end

  -- /absolute/path/to/some/java/source.java:229: error:
  if not file then
    file, line = str:match('([^:]+):(%d+): .+:')
  end

  return file, line
end

local function file_exists(filename)
  return vim.fn.filereadable(filename) == 1
end

local function open_file_line(filename, line)
  local line_arg = ''
  if line then
    line_arg = '+' .. line
  end

  vim.cmd(table.concat({ 'edit', line_arg, filename }, ' '))
end

function M.open_error(error_str)
  local filename, line = parse_error_string(error_str)

  if file_exists(filename) then
    open_file_line(filename, line)
  else
    vim.notify('Could not open file: ' .. filename, vim.log.levels.ERROR)
  end
end

function M.open_yanked_error()
  local error_str = vim.fn.getreg('+')
  M.open_error(error_str)
end

return M

The code is pretty straight forward. Most of the work is done in the parsing function parse_error_string which extract the filename and the line from the error string. Then the exposed function open_yanked_error reads the + register, parses the string it contains and opens the file at the right line.

As for calling the function itself, this is the mapping I use:

local parse_error_string = require('parse_error_string')

vim.keymap.set('n', '<Leader>fy', parse_error_string.open_yanked_error,
  { desc = 'Open yanked error' })

Now hitting <Leader>fy in normal mode will make neovim jump on the error found in the compilation pane.

Conclusion

I hope this post has demonstrated that it is not too difficult to set up a good development workflow in the terminal. With a few basic tools, some regexes, a bit of Lua and we can have a fast and efficient compilation loop that is language agnostic.

Feel free to adapt it to your needs and let me know if you like it.

Unfortunately, this doesn’t work with multi-package projects. This is because GHC outputs filenames relative to a package and not relative to where it runs (see this GHC issue and also this cabal issue). That means that neovim could be able to load the error file in the same way as previously, but it would not be able to find the files in the quickfix list, which would not be very useful.

To solve this, I came up with a relatively simple solution. I wrote some lua code to manually parse the error file and fill up the quickfix list prepending the relative path of the package currently loaded by ghcid. I mapped the lua function to a neovim user command, and it results in a very similar workflow as before. One just need to give to neovim the path to the package worked on.

See below for an example:

This code was present in my personal neovim config for a while. But to share it, I thought it would be easier to make a plugin out of it. And so I did. The plugin includes both approaches and is then working out of the box for both cases.

You can find it on GitHub.

The tooling in the Haskell ecosystem has greatly improved these last years. Especially, the introduction of Haskell Language Server dramatically lowered the barrier to entry for newcomers. But despite its undeniable quality, HLS still chokes on big codebases such as the ones we find in professional environments.

Also, for a fast feedback loop, it is not always the most suited tool to reach for. And having the fastest feedback loop is extremely important for a good developer experience.

ghcid

The tool that always works, no matter what, even on big codebases, is ghcid. I find myself constantly getting back to it for my feedback loops.

ghcid basically starts a ghci session and watches for changes in the loaded files. Just save a file with your editor, it instantaneously reloads it and prints the eventual errors or warnings, depending on your project configuration. It is extremely versatile and can adapt to many workflows.

This is how I use it.

I open a tmux session with two vertical panes. On the left, I have neovim with my Haskell project. On the right, I have ghcid running. When I save a file, ghcid reloads the file and prints the errors. I can then look at the errors and fix them in neovim, save again, repeat.

That’s pretty cool. And this kind of setup is super general by the way. It basically works with any language. Have an editor on the left, and runs a compilation loop on the right. There are a lot of tools that watch for file changes and run a command when a file is modified. I personally like entr and feedback for this kind of task.

But anyway, we can do better. neovim has a built-in feature for this kind of compilation loop. If your compilation task can produce a file (and ghcid can do that), neovim can parse it and populate the quickfix list with the errors and warnings. Then one can browse the quickfix list and fix the errors one after another.

This makes the feedback loop even faster: I save a Haskell file in my editor, ghcid automatically updates the error file, then, with one single command (:cfile or :cf), I reload the file in neovim and jump to the first error.

Simple, easy, fast.

What I like with this workflow is also that it is not intrusive at all. It doesn’t force me to fix any issue right now. I can jump to the error whenever I want while having the compilation state of my project right under my eyes all the time.

Configuration

Now let’s talk about the required configuration for this to work. First we must configure ghcid. This can be done at the root of the project in a file called .ghcid.

$ echo "-a -o errors.err" > .ghcid

This tells ghcid to output the errors in the file errors.err, the default error file for neovim (see :h errorfile). -a is not strictly necessary. It allows executing REPL commands in the ghci session. I find it sometimes useful. More information about this can be found on here.

Now we only need to tell neovim how to parse the error file. This is done with the option errorformat (see :h errorformat). A good place to set it is in the ftplugin file for Haskell. That file will be sourced whenever a Haskell file is opened.

$ cat ~/.config/nvim/after/ftplugin/haskell.lua

vim.opt_local.errorformat =
    -- %W multi-line warning
    -- For some reason, %m doesn't work with %\\?, we need to add two lines for
    -- each case
    "%W%f:(%l\\,%c)-(%e\\,%k): warning: %m," ..
    "%W%f:(%l\\,%c)-(%e\\,%k): warning:," ..
    "%W%f:%l:%c-%k: warning: %m," ..
    "%W%f:%l:%c-%k: warning:," ..
    "%W%f:%l:%c: warning: %m," ..
    "%W%f:%l:%c: warning:," ..

    -- %E multi-line error
    "%E%f:(%l\\,%c)-(%e\\,%k): error: %m," ..
    "%E%f:(%l\\,%c)-(%e\\,%k): error:," ..
    "%E%f:%l:%c-%k: error: %m," ..
    "%E%f:%l:%c-%k: error:," ..
    "%E%f:%l:%c: error: %m," ..
    "%E%f:%l:%c: error:," ..

    -- %Z Ends a multi-line message. We end it on the first line of the carret
    -- message.
    "%Z %\\+|%.%#," ..

    -- Continue a multi-line message
    "%C    %m," ..

    -- Swallow everything else
    "%-G%.%#"

And that’s it. Now when calling the command :cf in neovim, it will read the error file, parse it according to the errorformat option and jump to the first error.

Then one can browse the errors with the usual quickfix commands: :cnext, :cprev, :cfirst, :clast, etc…

Conclusion

This simple trick allows having a fast feedback loop when working with Haskell.

One shortcoming of this setup is that it doesn’t work when working in a multi-package project. This is because the errors returned by GHC are relative to the project. This can be easily solved by cd into the directory before starting ghcid.

I have a better solution for this as well, but it is slightly more involved and requires a bit of lua code. I will write about it in a future post.

That’s it for today. Let me know if you find this useful.

28-11-2024: Read the follow-up in this next post.

Related tools

• ghcid.nvim : A neovim plugin for ghcid. I was using it at some point. But I like my solution better because it is simpler and only use neovim built-in features.
• ghcidwatch : A rewrite of ghcid in rust developed by Mercury. I have never tried it, but it seems that it is able to write an error file in the same format as ghcid. It should then work with this setup.

It is a layout that organizes windows in columns while offering great flexibility when it comes to arrange the windows. I use it along a tabbed layout and with those two layouts, I do not need anything else really.

Let’s first start with a quick demo of the layout in action:

As one can see in the video, the layout organizes the windows in columns. You can have as many columns as you like. The windows can be freely moved from one column to another, left and right, and, in a single column, up and down. They can also be resized both horizontally and vertically. Adding new column is done by moving a window to the spot you want the column to appear. Simple, easy.

I also use this trick to dock windows together in tabs as showcased in the video.

I can change the focus of the windows using WindowNavigation. Every other command is handled by sending custom messages to the layout. See the source code for details.

With this layout, I can arrange the windows really in any way I want while keeping them always tiled.

I could not find anything like this in the xmonad-contrib repository. The closest I found was ResizableTall that I used for a while before developping my own. However, it is limited to two columns. Another interesting one is rowOfColumns. It is pretty similar to mine, but it wasn’t working very well with the tabbed sub-layout if I remember correctly.

That’s it. The layout has now been upstreamed and the source code is fully available in the xmonad-contrib repository.

With my years of developing experience, I was confident I could handle the rhythm of one puzzle a day and at the same time do it in a language I was not very familiar with, Rust. I have written a small project in Rust sometime ago, and I wanted to bring my knowledge one step further and learn this language for good. One month of intense problem-solving in Rust would make it at last.

Considering the language, Rust is actually a pretty decent choice when it comes to solving AOC puzzles. Indeed, strong typing offers nice guarantees that the code is going to do what it is meant to. And the good performance of the language allows brute force a lot of the problems without too much thinking.

The first thing that struck me was that the challenges were way harder than I expected. Here is how the whole thing is organized. You have one puzzle to solve a day. Each puzzle comes in two-parts. The first one is usually not too hard and can be solved with a direct or, call it naive, approach. The second part is somehow a variation of the first one. But there is a catch. Very often the solution developed for the first part blows up in complexity for the second part: time or memory.

And this is where the fun actually begins. How come those two closely related problems need to be solved in different algorithmic approaches? I can only bow to Eric Wastl, the person behind all this, for his amazing work. This two-parts organization is incredibly crafted. The same remark goes for the input data, which is different for every participant by the way. The tests are always very helpful and fit smoothly into unit tests. And not to mention the crystal clear explanations of each puzzle! All of this is very well done. And if AOC took me so much time to solve, I can not imagine how much time Eric Wastl takes to prepare it every year.

One thing that I did not foresee thus was that 25 days is a long time. After solving all the problems I was a bit exhausted. Depending on how clean you want your solution to be, I do, it can easily eat all your free time. Also, the puzzles are getting slightly harder each day. Interestingly, having a tight time constraint makes the challenge very close to real work in this regard. In my opinion, it is a good thing when it comes to learning a new language. It puts you in the shoes of a working developer. Debugging can be quite hard sometimes as some of the algorithms involve very big numbers. This also feels like real-life work. At day job, it is very often that we have some weird behavior that is hard to reproduce and inspect. Being creative on how to debug programs is a very important skill for a developer. Tackling that kind of problem helps to practice this skill.

I had heaps of fun with Advent of Code. All my solutions are on GitHub. I find it refreshing to hack on those puzzles. In everyday job, it is unfortunately very rare to run into such interesting algorithmic problems. And it feels good practicing pure algorithmic problem-solving.

To finish this post, I want to mention the leaderboard page from the website. It shows how much time the fastest people take to solve the problems. Sometimes a couple of minutes when it takes me hours. This is crazy. But let’s recognize that we are not in the same positions here. My primary goal is not to be the fastest but to learn a new language while having fun! That goal is achieved along the challenge. I’m happy with that.

I have been willing to learn FRP for a long time. Since I had the chance to have a little more time than usual these days, I thought it was the right time for me to dive into this paradigm and learn this stuff for good.

Among the different implementations of FRP in Haskell, I was especially interested in Reflex. What I find interesting in this implementation is that the very same code can be deployed on the web, as an Android app, or as an iPhone app which is spectacular when you think about it.

The idea of being capable of running Haskell code on a phone is so appealing that I could not resist. Once I heard about it, I knew that I needed to write an app at some point in my Haskell journey. And that promise of being able to run the very same code on different platforms. How true can that be? I needed to find out.

Unfortunately, examples of such apps in the wild are not that common, if not non-existent¹. I spent a fair amount of time looking for examples. Only tutorials are sometimes deployed on the web but nothing on Google Play (I did not dig in the app store as I don’t own an iPhone). And as a professional developer, I know there is often a big gap between tutorials and real-life applications.

I needed to find out by myself. And that is what I did with this project.

The project

As I like to do it, I needed an idea of something simple yet nontrivial to get my hands dirty. I found something that might be a good fit.

I am a big note taker. I take notes all the time about just anything, technical or not. I have tried many different tools. But what works best for me is plain markdown files in a private git repository.

What I am missing in my workflow is the ability to browse and search my notes from my phone. I could use the official GitHub app for that purpose. However, that would spoil the fun. I also know that there is a lot of value in owning the code of a tool you use. Once the code is written and works, it becomes easy to fix bugs and add features as needed.

The project would then be an app able to browse and search in a GitHub repository. It should also be able to render markdown files. For simplicity’s sake, I decided to use the GitHub API to talk to the Git repository.

Here is the summary of the things the app needs to do:

• have some settings
• validate and store the settings
• sends requests to the GitHub API
• react to responses, parse them, and show the results
• render markdown
• handle errors
• wrap everything in an easy-to-use UI

Experience

I had great fun learning FRP. As a developer, this is really the kind of experience I am looking at. A new concept that, once grokked, gives a deeper understanding of the domain, here, reactive UI development.

People usually say that FRP is hard to learn. I would not say that. But it is true that one has to change mindset to be able to be productive with it. And that is exactly the interesting point. My opinion is that it helps to understand the interactions of the widgets and user actions as a whole. It makes it clear that if something is difficult to implement in FRP that usually means the interactions are themselves complicated.

I put the app online and on the Play Store for you to try out (and me to use). As advertised by Obsidian Systems folks, it does work the same on the web and on Android. The whole code is available on GitHub.

Missing pieces and dark corners

I want to take advantage of this post to shed light on some dark corners I encountered during the development of this project.

Developer tools

For me, Obelisk is a developer tool, such as ghc, cabal, haskell-language-server, or even gcc. The documentation states that it must be installed.

But I don’t usually install the developer tools I use. I pull all the ones I need from a pinned version of nixpkgs and put them in scope with a nix-shell.

Unfortunately, the ob command is not in nixpkgs. So I came up with that shell.nix:

let
  obeliskSrc = fetchGit {
    url = "https://github.com/obsidiansystems/obelisk.git";
    ref = "refs/tags/v1.2.0.0";
  };
in
(import obeliskSrc { }).shell

One can see, that for this shell, we pin obelisk version v1.2.0.0. Yet the ob command, whatever version, doesn’t care about this and uses the rev pinned there instead. It fetches it, compiles it, and passes the commands to it. This is undocumented and can be super confusing if you don’t know about it.

Now I also like to have haskell-language-server available in my editor. This is not very hard to add it in the development shell, yet quite difficult if you don’t know how to do it. These are the relevant lines to be added to default.nix.

  shellToolOverrides = self: super: {
    haskell-language-server = pkgs.haskell.packages.ghc8107.haskell-language-server;
    implicit-hie = pkgs.haskell.packages.ghc8107.implicit-hie;
  };

At the end, working around those weird uses of nix, my workflow to work on this project is:

$ nix-shell

brings ob into scope. I can now use ob run or ob watch and develop. Then:

$ ob shell

brings haskell-language-server into scope. From this shell, I start my editor to have type hovers, auto-formatting and all the niceties that haskell-language-server provides.

Build and upload the Android app

The Play Store expects a signed AAB file. That file can be built with:

$ nix-build -A android.frontend --arg androidIsRelease true

The AAB file is then available in ./result/android-app-release.aab.

It must be signed with some java tools. Let’s start a shell with them:

$ nix-shell -p jdk17_headless

Now create a key store for storing the keys that will be used to sign the file.

$ keytool -genkey -v -keystore diverk.keystore -alias diverk -keyalg RSA -keysize 2048 -validity 10000

It will ask a few questions, along with a password. That one must be kept in a safe place. It will be needed for the next commands.

$ cp ./result/android-app-release.aab .
$ jarsigner -verbose -sigalg SHA256withRSA -digestalg SHA-256 -storepass the-passord-given-to-keytool -keystore diverk.keystore ./android-app-release.aab diverk

And that’s it! The AAB file is now signed and ready to be uploaded on the Play Store.

Deploy the app on the web

To deploy the app on the web, one must first build the derivation:

$ nix-build -A linuxExe
$ ls -L result
backend  frontend.jsexe.assets  static.assets  version

It contains the following files:

• backend is the executable of the HTTP server itself
• frontend.jsexe.assets is the web app compiled by ghcjs
• static.assets are the static assets used by the frontend

Now you can copy that derivation on any system having nix, reverse proxy it with nginx if you want, starts the backend, and it will serve your app on the web.

The backend supports many command line options. Find out which ones with backend --help.

Learning resources

Here you can find the best resources I have found on the subject:

• Presentation of the library by its creator Ryan Trinkle: part 1, part 2
• The official tutorial and my playground
• Great tutorials made by the Queensland Functional Programming Lab
• Real World Reflex: a video addressing more advanced concepts
  • video
  • slides