[Resolved] Service Incident Update: Projects Loading Slowly or Not At All, NPM Reinstalls, & Missing Code

Ok I just checked and it’s only some files so hopefully I’m not totally ******.

4 files are just replaced with dots :cry: good thing I backed up my project. I think my .sqlite files are fine and I should be able to paste in the code of those 4 files since I haven’t changed them since June 14.

1 Like

Code pasted! Everything is working now!!

@wh0 im getting this > UnhandledPromiseRejectionWarning: SqliteError: disk I/O error
now. Do you know to fix it??

I’ve been seeing this intermittently. That is to say, after many subsequent attempts, a later disk write would succeed.


Above: a cluster of I/O errors observed last night. red-orange = I/O error, light blue = success. time on vertical axis, more recent entries on top, ~1 second per pixel. longer bars mean longer write latency.

But I wasn’t using sqlite in that case. I don’t know if the error would corrupt the database. (In my case, I was rewriting the entire file.) It could even be that these I/O errors are what cause files to become replaced with dots, e.g. when the Glitch editor encounters such an error.

So at some level, maybe the solution is to do your best to back up the database and try to rebuild your app to store its state on a third party service.

Or if you want to try something wild, just retry the write until it succeeds. Again, I don’t know if an error would corrupt the database.

I have the same issue :

node v12.0.0, with pnpm

9:01 PM


9:01 PM

Performing headless installation

9:01 PM

ERROR ENOSPC: no space left on device, open ‘/rbd/pnpm-volume/store/2/store.json.2640651246’

1 Like

@wh0 should I just try uploading the old sqlite file from my backup?? Don’t really care about the data since it isn’t important anyway. Do you think it’ll work?

Welp, I’ve got the exact same issue with pnpm…

could it be cus of the outage?!?!?

It no doubt is caused by the outage, so we’ll have to wait for Glitch to fix this…

Wow, that’s a good situation to be in. It’ll save you the trouble of figuring out the details of how sqlite works. If you can upload the backup version of the database–and it may take multiple tries if there are intermittent errors–then you can be sure (more sure anyway :roll_eyes: so much uncertainty right now) that whatever’s saved is not corrupted. At least not corrupted until the next write attempt.

When I type enable-pnpm or enable-npm I have this :

rm: cannot remove ‘node_modules’: Structure needs cleaning

ok ill try it soon!!

@wh0 should I download my current project just in case??

If we were to choose between backups or no backups, with all else being equal, we’d choose backups every time. What would the downsides even be?

1 Like

wait how do I upload files to glitch again :flushed: I pressed upload file and selected the file and it uploaded but how do I get it in my project?

wget <glitch cdn url>

the file is empty :flushed: :flushed:

Are referring to that thing where you upload an “asset” to a CDN?

And you uploaded it and the CDN gives back an empty file? You might be the first user to report that.

ok it failed the first time but I tried it again and now it works!!