Npm install not properly running recursively

Server runs an error telling met hat a dependcy is missing, but it’s a dependency of a dependency.

Investigating in terminal shows that while node_modules has my dependencies, if I go into one of those, and check its node_modules, it only has two dependency folders installed… and running npm install within the dependcy folder ‘fixes’ that issue (though I will have to do this for every dependency, as the problem is now a dependency of the next dependency.

What’s up? What should I do here? deleting node_modules and re-installing, and remixing project, have both not fixed the issue.

Node is >=16, npm is > 7. I tried switching to npm 8, but specifying that in package.json just causes an error instead of installing the newer npm.

Hm, might be a hint… the first dependecy I checked only had two folders in its node_modules, while after running npm install had dozens.

The next one I checked didn’t even have a node_modules folder. That suggests that it hit some problematic folder on the first one and then stopped its recursive process.

Perhaps a bug in npm 7? How can I upgrade to newer versions of npm? The normal way fails because symlinks don’t work properly or something. (I think it was npm install -g@latest)

Edit: express, another first level dependency, has 3 folders in its node_modules. Hm.

Ok, I think it’s because of a problem with the C++ build / install steps in one of my dependencies silently throwing things off.

I manually installed for the dependencies needed to get the server to get past the require statements at the top, and then saw an error from one of the dependencies throwing ENOENT for a folder/file it expected. Seeing it missing, and checking the dependecy’s package.json, I saw a ‘build’ command related to the error, and it seemed like a build step hadn’t been run. Running it, I got an out-of-memory error.

I assume this is somehow related to the failure of npm install.

That said, I have a pro account, so I have attempted to make this app “boosted”, so I could go from 512mb to 2gb memory.

Now I get a new infuriating problem–I can’t cd into the folder to run the command anymore.

I get this:

app@innovative-ambiguous-principal:~/node_modules 23:12 
$ ls
@babel  babel-loader  express  mediasoup  socket.io  webpack  webpack-cli  webpack-dev-middleware

app@innovative-ambiguous-principal:~/node_modules 23:12 
$ cd mediasoup 
-su: cd: mediasoup: No such file or directory

even though the folder is there… what’s up with the -su suddenly?

This is making me want to pull my hair out.

Seems that after boosting the app, I get new errors, as if I hadn’t manually run those npm-install commands before, and now it won’t let me run npm install, and shows my base level dependencies as missing.

So, it says ‘no mediasoup found’… even though I do see a mediasoup folder in the terminal when I type ls… but I can’t cd into that folder as if it doesn’t exist when I use cd. Something super funky is going on. refresh does not fix it, and npm install fails with

npm ERR! code ENOENT
npm ERR! syscall chmod
npm ERR! path /app/node_modules/webpack/bin/webpack.js

which, I mean, doesn’t make a ton of sense, npm should install that file there… and it used to be there before I boosted the app.

Feeling pretty disappointed in glitch right now.

that particular glitch seems borked, switched back to a previous one I had remixed from, boosted, ran npm install inside the dependencies, and now the server seems to run.

I think it’s running fine as far as I can tell at the moment. we’ll see.

Investigating in terminal shows that …

another place to look: check the project logs panel, particularly when Glitch is trying to install the project dependencies

if I go into one of those, and check its node_modules, it only has two dependency folders installed

The next one I checked didn’t even have a node_modules folder.

that’s part of how module work in node.js, where those recursive dependencies can come from node_modules, ../node_modules, ../../node_modules, etc. so each dependency’s module folder doesn’t need to have all of its own dependencies

Node is >=16, npm is > 7. I tried switching to npm 8

not sure if related, but try running enable-npm just to make sure Glitch isn’t somehow putting the project in pnpm mode. it shouldn’t at that version of node.js. this’ll also delete and recreate the node_modules

C++ build / install steps in one of my dependencies

Seeing it missing, and checking the dependecy’s package.json, I saw a ‘build’ command related to the error, and it seemed like a build step hadn’t been run. Running it, I got an out-of-memory error.

out of memory error referring to a message in the project log from Glitch saying that the project exceeded the memory limit, and not a message in the terminal where you ran the command, right? because if it’s the former, that’s something that might be helped by boosting. if it’s the latter, then I think boosting shouldn’t help.

programs can allocate and use memory freely up to the host server’s actual limit. the Glitch project memory quota, the thing that varies by boosted status or not, is enforced by a separate program that periodically checks the the project’s memory usage periodically, and if it’s over the limit for too long, you see the former error in the project log and Glitch kills all app processes in the container

by the way, :shushing_face: the grace period for memory overages is higher during the install phase

Now I get a new infuriating problem–I can’t cd into the folder to run the command anymore.

cursed. this I think comes from the fact that ~/node_modules is a symlink out to a separate shared mount, which sounds like it operates over the network. something’s out of sync, and I don’t know how to fix it. probably try stopping your project container and hope that it comes back up on an uncursed machine. instructions posted a long time ago for how to stop the container:

:grimacing: the above instructions are probably out of date with the API changes since then. see how to make the request here if the above doesn’t work:

1 Like

abandoning that remix and repeating my steps on a boosted “non-cursed” project that I had remixed from fixed it for me. I think the compilation steps that were demanding extra memory were a kind of long step on that processor.

that’s part of how module work in node.js, where those recursive dependencies can come from node_modules, ../node_modules, ../../node_modules, etc. so each dependency’s module folder doesn’t need to have all of its own dependencies

Sure, I am aware, except that it wasn’t installing the needed dependencies. When I would try to ‘require’ mediasoup, I’d get an error saying debug was missing, which was required by mediasoup in package.json but was not installed, and running npm install there would fix it. Same pattern with socket-io and express–missing sub-dependency, fixed by running npm install manually within the child folder.

so it sounds like, after all this, the native module built successfully too?

did the project logs say anything about npm giving up on installing dependencies?