A Reproducible Development Environment on MacOS

For long I have fought with the inconsistencies of brew that, while certainly an impressive piece of software, has made several choices in its development that had the effect of distancing it ever far from traditional package managers.

A particular aspect that hurts me the most is its focus on the bleeding edge. Judging from how the community uses different managers to pin versions for the software they have installed, such as asdf or rbenv or pyenv, I don’t think I’m an edge case. But even those workarounds often aren’t enough, and it’s not uncommon to come back to a system with broken software due to linkage problems or a sudden major upgrade of a language runtime. This goes beyond programming languages and libraries, affecting my editor and my tools as well.

I’m not conservative regarding software, but I’m also not fond of rebuilding my system every handful of weeks. My Gentoo days are long past. I have been looking for a way to keeping the system a bit more stable for a while, particularly on the side of development packages.

Up so far I have tried MacPorts as an alternative, but it brings other issues. Mainly: its decreasing usage makes everything in the Mac ecosystem lopsided to brew’s court, which meant costly workarounds to install some packages, and absence of many.

I’ve been wanting to try a different approach: Linux distributions and their package managers usually deal well with these concepts. I could install Linux on my laptop, but there are two strong arguments against it: first, Linux is far from being supported in my personal daily driver—Wi-Fi is still unsupported on my late 2016 MacBook Pro (yes: still this one)—and secondly, I like the desktop ecosystem of MacOS a little better, and would like to keep using it while I can.

The other alternative, and the one I had been pursuing for a while, would be having my development environment in a virtual machine. This would bring the best parts of both worlds at the cost of performance, which is arguably a problem: most of the work I do doesn’t require a beefy computer. So I had a crack at it, or multiple.

A working setup

This has been an journey that has been going on for quite a while, coming from my days working at Cleverly. I have tried many variations alternating between Linux distributions and strategy: from actually using a GUI to keep myself to a terminal or even SSH only. Having some sort of interface made sense early on, so SSH only soon became out of the question. On the other hand, using a GUI was lagged too much to be usable.

I kept going back to the drawing board on this one, and never found anything that fit.

I then found Mitchell Hashimoto’s repository where he tackles this problem with a similar approach. He claimed his setup worked very well, and had minimal lag, so I wondered what were the main differences between his approach and mine were.

Then I realised he used VMWare Fusion instead of VirtualBox. So I tried it out.

It was night and day. Spawned a Debian box with i3 as the WM, and emacs GUI just flew. Gave it half the resources of the laptop (4 cores and 8GiB of RAM) and I managed to do a couple of comfortable programming sessions with both Ruby and Clojure.

I feel obligated to say that the clipboard between both parts also works well, so I can browse in MacOS and do my coding in the virtual environment.

Lastly, I found out that VMWare Fusion Player is free for non-commercial use, which is pretty much my case at the moment.

Reproducibility

So far my approach for reproducible systems would be Debian/Ubuntu with Ansible. That was my first approach to this development environment, and it worked well. Took me about two days to get the system to how I wanted it.

I had my eye on NixOS for quite a while, and found this to be the perfect excuse to try it out, seen that I even had Mitchell’s example and all. After a few hours editing my configuration.nix I had almost everything in place. I was blown away by how fast that was.

And the best part is: due to everything being declarative and written down in a configuration file, now I can go from no VM at all to readiness in minutes, with all missing being user data. Also, because of Nix’s shell environments, all the projects I use them with will have a reproducible environment at the distance of a nix-shell in the terminal. It’s glorious.

I’m stil learning about Nix, and I plan to release my configuration as soon as I learn more about flakes, overlays and packaging.

Other advantages

I’ve found that this approach also gives me some extra nice things:

GUI Tweaks and Performance

I’ve opted to follow Mitchell’s terminal based approach and use Emacs in its terminal-mode, in a terminal window instead of in a dedicated X window. Although graphical performance was far from being an issue, the complete absence of lag when using the terminal just won me over the nice things coming from the GUI version. I still use i3 to manage all terminal and non-terminal windows, even the occasional browser session.

For vim/nvim users there should be no compromise. I’m even tempted to make an attempt at a vim config later on, if I have the time.

I have no idea about how VSCode behaves. But if anything else fails, VSCode SSH mode works remarkably well, so in that case you may choose to have your development tools and languages inside the VM, but keep your editor out. I’d probably do that if this was my editor of choice.

Final thoughts

I’m happy with this setup. It’s snappy, comfortable to use and stable. It has been a few weeks since I set it up and it’s still behaving predictably.

This left me very curious with the prospect of using NixOS in other places, and Nix as the package manager for MacOS, so it’ll be something I’d like to try out. The tight coupling between system configuration and the system are very appealing, and I’d like to see how this performs in practice.