Anti-gallery post: gentoo and homebrew

Homebrew

You know, I know some people who use Macs. And recently I asked “what’s that thing you use to install stuff?” “Homebrew.” “Ah yeah. So Homebrew? Not Brew?” “… Homebrew.” And after a few minutes of looking into it, I finally understand: brew is the command, Homebrew is the name of the project. Which turned out to be just the beginning of learning new terminology. Right smack dab at the top of the manual for brew is a great big terminology section telling you that they insist on calling things formula, keg, rack, keg-only, Cellar, tap, and bottle, among other themed names.

Which hey, if giving every concept a themed term is the weirdest thing about a package manager, then that’s pretty good. I gotta say, from what I’ve seen, it is. At every turn, Homebrew seems to have a very normal way to do just about everything.

Getting it on Glitch: breaking the rules right out of the gate

Homebrew supports Linux. Parenthetically if you read only as far as the second line on their website, but no, they do.

They need root access in order to create their “Cellar,” their place where they store the built packages, in /home/linuxbrew (note to people keeping track: /home/linuxbrew is not the Cellar; the Cellar is further within that directory). Or you “may” install it to a different prefix. “May” is qualified from this on their page about using Homebrew on Linux Homebrew on Linux — Homebrew Documentation :

… you shouldn’t …
… buggy and unsupported …
If you decide to use another prefix: don’t open any issues, even if you think they are unrelated to your prefix choice. They will be closed without response.

So I’ve been very quiet about things, and I’ll only be talking about it here.

Getting it on Glitch: breaking more rules

From their installation documentation Installation — Homebrew Documentation :

Make sure you avoid installing into:

  • /tmp subdirectories because Homebrew gets upset.

We don’t want to waste a bunch of space in /app for what are essentially dependencies. Upset is better than nonexistent, right?

/tmp/homebrew seems reasonable. Note: if you’re stopping reading here, don’t use /tmp/homebrew. There’s a note about this later.

Getting it on Glitch: patches

Homebrew on Linux installs its own glibc, which we need to patch with our EPERM fixes. And they have a way to add custom patches: you have to edit the “formula” file, which is a description of how to build a package, adding a statement to it:

  patch do
    url "https://.../glibc-rhbz1869030-faccessat2-eperm.patch"
    sha256 "ed...41"
  end

Comparison with Nix: adding patches

At some level, it’s the same as Nix. You need to add an instruction to apply a patch in the build instructions. Both have that file with build instructions in enormous Git repos. In Homebrew, that repo is called a “tap,” while in Nix, that repo is called a “flake.”

But in Nix, you have an alternative to forking the flake repo: you can configure an “overlay” to say “instead of the list of patches from the flake, use the list of patches plus this other patch of mine.”

In Homebrew on the other hand, what seems to work is to make a copy of the file from that huge tap repo into a new tap. And your formula will have the same name, which will allow you to have your glibc used in place of the glibc from the main homebrew/core tap.

Getting it on Glitch: what just happened to my Homebrew installation

It turns out that the build procedure for glibc uses specifically /tmp/homebrew to unpack some temporary bootstrapping tools, and it deletes it after the build finishes. We’ll need to use a different directory to install Homebrew itself. I’m going with /tmp/h then :roll_eyes:

Getting it on Glitch: saving compiled things

So I built the custom glibc implementation. Homebrew calls prebuilt binary packages “bottles,” and brew has a suite of commands for producing them. Odd thing: they insist on having their own distribution of tar for creating these bottles, while they’re pretty happy using something else to install from bottles. Oh they have a term for installing a package from a bottle, too. “Pouring.” You’re welcome.

Comparison with Nix: binary package compression

Homebrew uses gzip to compress bottles https://github.com/Homebrew/brew/blob/4.1.21/Library/Homebrew/dev-cmd/bottle.rb#L455, for it speed. Nix has been using xz, which we earlier found out Speeding up a custom-built Nix package installer - #4 by wh0 its decompression was CPU-intensive enough to take longer than the download. So maybe gzip is a good thing on Glitch.

Comparison with Nix: binary package contents

Nix is much stricter about a package simply being a tree of files. Homebrew allows packages to have “post-install” steps, which may modify the package. Or to be more specific, the “keg.” That’s what they call an installed package.

The effect of this is that in Homebrew, you need to install a package in a “build-bottle” mode which defers the post-install steps in order to preserve a pristine keg to make a bottle. Then you run a command to create the bottle, and then you run a command to run the deferred post-install steps.

Nix doesn’t have this distinction. Rather, all the files have to be completely done after building, and they’re stored read-only in the Nix store. You can copy them into a binary cache whenever.

Comparison with Nix: installing binary packages

This bit about Homebrew is a little too defaults-focused for me. In Homebrew, information about where to find the bottle for a given package is stored in the formula. So you can’t go and build a bunch of bottles for a non-default Cellar and expect brew to install them automatically unless you also fork the whole tap and edit every formula you build.

brew can install single bottles from a filename, but it’s many times more fiddly when you have to deal with dependencies.

In Nix, the binary cache configuration is decoupled from the flake and derivations. You can set something in a config file to tell Nix where to get binary packages.

Getting it on Glitch: progress on bootstrapping

As the documentation had forewarned, getting Homebrew to set up a suite of packages in a different prefix was buggy. I got about as far as failing to compile krb5, a library deep, deep in the dependency graph on the way to installing node@20.

But secretly, I think it wouldn’t have worked even if I somehow had it installed in the normal prefix. There’s awkward stuff from the host system that leaks into the build. I think the subtext of “non-default Cellar directories don’t work” is that Homebrew significantly depends on bottles being built on CI servers that natively have the same glibc as the version in the distribution. Homebrew is not a good platform for building software from source on a host with older system software.

Comparison with Nix: relocatable bottles

Whereas in Nix fully treats the contents of a package as fixed, Homebrew has a system for patching packages at install time to fill in paths based on where they’re installed, including to non-default Cellar directories. That’s pretty darned merciful. I compiled gcc a total of zero times while working with Homebrew :pray:

Getting it on Glitch: breaking more rules

There’s a system that’s supposed to determine automatically if bottles are relocatable. You can override that decision with a --force-bottle when you run brew install. The packages I tried turned out to work enough in this mode.

Progress report

I then tried installing everything but glibc with --force-bottle, and seem to be able to run node@20 okay. Installing one of these “@version” things is what’s called “keg-only,” which means that brew won’t link it into your path, so you have to run /tmp/h/opt/node@20/bin/node. Their npm distribution uses #!/usr/bin/env node though, so you have to run /tmp/h/opt/node@20/bin/node /tmp/h/opt/node@20/bin/npm instead :skull:

Using Homebrew to set up Node.js 20 is kind of slow though.

speedrun.sh:

cd /tmp
mkdir /tmp/h
curl -L https://github.com/Homebrew/brew/tarball/master | tar -xz --strip 1 -C /tmp/h

mkdir -p /tmp/homebrew-cache
ln -snf /tmp/homebrew-cache ~/.cache/Homebrew

mkdir -p /tmp/h/Library/Taps/wh0
ln -sf /app/Taps/wh0/homebrew-glitch -t /tmp/h/Library/Taps/wh0

eval "$(/tmp/h/bin/brew shellenv)"

brew install ~/.data/glibc--*.tar.gz
brew deps -n node@20 | xargs -n1 brew install --force-bottle
brew install --force-bottle node@20
$ time sh -eux speedrun.sh
...
real    7m2.065s
user    3m10.872s
sys     0m44.692s

We have to work around limitations in brew install not recursively respecting --force-bottle (seems to be tracked? --force-bottle needs to be made recursive · Homebrew · Discussion #4875 · GitHub) which I’m doing with a lot of repeated invocations of brew, so we’re incurring several Ruby startups :weary:


and as usual, Glitch :・゚✧ here’s the project in whatever state it is in. that glibc bottle I copied to Glitch assets too https://cdn.glitch.me/db95827f-8afe-4ce8-a538-2332bf8ba093/glibc--2.35_1.x86_64_linux.bottle.tar.gz?v=1700547489094

1 Like