Node.js 22 starter

last thread: NixOS 24.05 packages

Project URL: Glitch :・゚✧

Here’s Node.js 22, which was packaged in Nixpkgs in the NixOS 24.05 release. My last post said that it took too long to build. Tomorrow I’ll write up how it’s possible now.

4 Likes

About Node.js taking too long to build, and version 22 so much so that it can’t finish in the 12-hour limit of Glitch project container lifetime, it was mostly that compiling native C++ code takes a lot of CPU.

My approach to working around this was that if we could split up the build so that it does some of the C++ compilation in one 12-hour block and the rest in another 12-hour block, and somehow combine the results, then we could get Node.js 22 built on Glitch.

What’s serving as a practical way to do this is ccache, that program that wraps a compiler and saves the output in case you want to compile the same input again. Nixpkgs has a way to build a package using ccache around the usual compiler (CCache - NixOS Wiki).

I set up ccache—which was tricky due to some ugliness in the way Nixpkgs defines the Node.js package—and started a build of Node.js 22. After eight-ish hours, the actual cache that ccache made was barely over 100 MB. I tarred it into my project’s .data dir and let the project go to sleep.

Then a while later I started up the project again, extracted the ccache cache back in place and started another attempt at the same build. ccache quickly let it get through the compilation tasks that were already done and cached. After that, it should take four hours to reach how far we originally got at the 12-hour limit last time, and it would have many more hours of project container time to go beyond that, and which I hoped, would continue to completion.

It didn’t do that though.

And that’s a good cliffhanger as any on which to say I’ll continue this story in the next post.

3 Likes

… It didn’t finish after 4 ish hours. It finished in minutes.

After the first project container had compiled for 8 ish hours, the second project container more or less took that stuff and packaged it up. There had been next to nothing left to compile.

That is to say, it took in total only about 8 hours. Why though?

I started with an empty ccache cache in the first project, so it couldn’t have shared anything from previous builds of Node.js. And we knew from previous container-out-of-time problems that it ought to take over 12 hours to compile everything. And ccache could only speed up repeats of identical compilations.

Then could it be that building Node.js involves some repeated identical compilations?

I did some research, which led me to think the answer was yes. I filed a bug about it.

Nixpkgs has since updated its 24.05 release with a rebuild of Node.js 22. I’ve just built it again, with ccache, without the project switch in the middle. That is, it built in a single project container starting with empty ccache cache. It took 8 hours and 2 minutes.

3 Likes