Components of the task
Alright, here’s an actual description of the components that make up the whole installation process, where it’s not just me messing around with the turbo encabulator script.
1. Download and process .narinfo files. These are simple text-based files containing, critically, the URL (relative, usually) of where to get the actual .nar file and the dependencies. You can download the first .narinfo file you need by constructing a URL given a binary cache server and the “hash” part of a Nix store path that the user wants to install (read more). And then you recursively get more .narinfo files for those dependencies.
2. Download the .nar files. You get the URLs for those from the .narinfo files. These can be large. And in the binary cache that I’m using to store my packages built for Glitch, (almost) everything is hosted on Glitch assets (read more).
3. Decompress the .nar files. Slight correction on the previous point: you don’t really download the .nar files directly from the binary cache server. The binary cache server keeps compressed .nar files, and the .narinfo tells you how they’re compressed. Currently the default is xz, and all the packages I’ve built are xz-compressed. You get those .nar.xz files and decompress them.
4. Read through the .nar and create files in the Nix store. At that point, there’s a way to parse the .nar file and get the various directories and files (read more). As we read through, we apply these actual changes, “to disk.”
Aside: error handling around streams in Node.js
It’s a pain. Remember that remark earlier about me wanting to do this without needing any dependencies not already included in the project container? That means nothing nice from npm. Just vanilla Node.js. And that means no node-fetch. Get ready for some builtin https module action 
.
I probably spent a whole week trying to figure out a reasonable way to make sure that something using the ‘response’ stream would be able to know if something went wrong with the HTTP(S) request.
It ended up something like this:
const https = require('https');
function deferredReject(p) {
p.catch((e) => { });
return p;
}
async function* getOk(url) {
const req = http.get(url);
const res = await new Promise((resolve, reject) => {
req.on('error', reject);
req.on('response', resolve);
});
if (!(res.statusCode >= 200 && res.statusCode < 300)) {
req.abort();
throw new Error(`get ${url} status ${res.statusCode} not ok`);
}
const done = deferredReject(new Promise((resolve, reject) => {
req.on('error', reject);
req.on('close', resolve);
}));
yield* res;
await done;
}
Anyone have a better way to do it?
I think Node.js 16 has better handling, where they emit request errors on the response too, but I feel like Node.js 10 would have an advantage of warm startup.
The slowness of it all
So eventually I got it working, and I tried it out on installing Emacs 28.2. It was slow. It took… let me pull up my notes here… about 2m32s to install that. And at that point I didn’t know if it was, like, slow considering what all it’s doing or anything. It was just slow compared to how soon I wanted to open Emacs.
In the next post, I’ll break down how long the different components ought to take, if you were to use normal tools. That way, we can start to judge if it’s slow because it just has a lot of work to do, or it’s slow because my Node.js code sucks, or it’s slow because Node.js is a bad framework, or whatever other reason.
aaah!