glibc
Somewhere between the version that was in earlier NixOS releases and this one, there was new logic added to remember which syscalls give ENOSYS and it’ll later remember to use the fallback. Or, at least there is for the clone3 one that I patched.
So I’ve had to rewrite the patch. It treats the EPERM we get with a custom behavior: use the fallback, but don’t go into never-try-again mode either. Which in retrospect, how did I decide that that’s how I want to do it? Something vaguely “patches should be something that could be added upstream are better, and a real EPERM on a non-Glitch system shouldn’t put the process into fallback-only mode.” But it’s not that meaningful, this project is full of bad-outside-of-Glitch patches.
Oh and for some reason is now a second place in the glibc codebase that also does this “try clone3 and see if it’s not supported” logic (spawni). I missed that the first time and had to recompile gcc an extra time 
libredirect
Libredirect, that thing that interposes on various filesystem-related libc functions to let you pretend file are in other places, has a few changes.
First, they’ve added a new test for NULL path arguments. Which is good. But it was inserted right between two tests I had disabled for incompatibility (they use system which doesn’t work), so I had to rewrite the patch to disable them.
Second, they’ve since removed the -ldl link flag, which could make it slightly more interoperable with older programs.
Before this removal, the libredirect.so file would get the new glibc libdl added to its rpath, which would cause problems when trying to load an old binary: the new libdl would be incompatible with the old libc in the process.
Earlier this year though, the team observed that the libcs they intend to package don’t really put anything in libdl treewide: remove -ldl linker flags by alyssais · Pull Request #212275 · NixOS/nixpkgs · GitHub, so they removed that flag. Huh, that PR was from back in January. Maybe it was already in since the last release, 23.05? Maybe I didn’t look into it back then.
Unfortunately, the old glibc on Glitch still requires libdl to provide the dlopen symbol that libredirect uses. In many programs this can still work, as long as the original program requests libdl, which seems to be common. But Glitch’s /bin/sh does not request libdl, so dlopen won’t be found, so libredirect won’t load correctly in /bin.sh, so it continues to be impossible to run Glitch’s /bin/sh (e.g. for the system function) with libredirect. Ugh.
python psutil
Remember that thing about Python APIs I had forgotten that the way I wrote that Nix “overlay” sets up the patch for a specific version of Python, 3.10 at the time. Now that the default Python 3 version has been upgraded to 3.11, I had to update the overlay file. Am I overlooking a simpler, non-Python-version-specific way to do this? Needs more research.
libuv
libuv, that async IO library used in Node.js, added a test for thread affinity. Thread affinity is super weird on Glitch, as we had earlier seen with the AWS SDK. I’ve added a patch to disable the thread affinity test.
I also looked into what the heck is so weird about thread affinity on Glitch.
import os
os.sched_getaffinity(os.getpid())
# {1, 2, 3}
os.sched_setaffinity(os.getpid(), [0])
# invalid argument
It seems there really are 4 logical CPUs, but we’re not allowed to use CPU 0. What’s the actual mechanism for this? Needs more research. It feels like I’ll find out with enough sifting through various cgroup options.
Anyway, that really throws off tests that expect reasonably “there must be at least one CPU, so let’s test on CPU 0.”
I guess we could make our patches smaller so that they test on CPU 1 instead of disable these affinity tests completely. But I’d need a lot of motivation to make that change.
Alright, that’s enough writing for now. I just keep uncovering more and more places I need to do additional research. But let me make a list of remaining topics so I don’t forget to mention them later.
- openssh
- openexr
- php pear installer thing
- nix sandbox test
- nix ‘installable’ derivation meaning
https://hydra.nixos.org/build/242650383#tabs-details
Closure size: 1577.11 MiB
good god, that’d take like 5 minutes just to download each time
update:
https://www.kernel.org/doc/Documentation/cgroup-v1/cpusets.txt
Cpusets provide a mechanism for assigning a set of CPUs and Memory
Nodes to a set of tasks.
That must be it.
The /proc/<pid>/status file for each task has four added lines,
displaying the task’s cpus_allowed (on which CPUs it may be scheduled)
and mems_allowed (on which Memory Nodes it may obtain memory)
$ cat /proc/$$/status
...
Cpus_allowed: e
Cpus_allowed_list: 1-3
Yup. (0xe being 0b1110.)
