I have an issue with a 261mb json file that gets corrupted while uploading
I tried uploading through the vscode extension (no error, but filesize 0b), the webbrowser (crashes) and ultimately imported through github (using LFS).
I am importing the json file with const ffhq = require('./data/ffhq-dataset-v2.json') in my server.js and locally that worked fine. If I upload a smaller size json it is also working.
I even upgraded my account because I thought that the 200mb limitation would be the problem, but after boosting my app I am still only using 3mb/400mb which is weird…
internal/modules/cjs/loader.js:787
throw err;
^
SyntaxError: /app/data/ffhq-dataset-v2.json: Unexpected token v in JSON at position 0
at JSON.parse (<anonymous>)
at Object.Module._extensions..json (internal/modules/cjs/loader.js:784:27)
at Module.load (internal/modules/cjs/loader.js:628:32)
at Function.Module._load (internal/modules/cjs/loader.js:555:12)
at Module.require (internal/modules/cjs/loader.js:666:19)
at require (internal/modules/cjs/helpers.js:16:16)
at Object.<anonymous> (/app/server.js:4:14)
at Module._compile (internal/modules/cjs/loader.js:759:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:770:10)
at Module.load (internal/modules/cjs/loader.js:628:32)
You’re mistaken. Our community had talked about using raw uncoordinated raw filesystem operations to manage application state on disk–many examples of which incidentally serialized data in JSON–as being more prone to corruption compared to a database management system.
Transferring a file without corruption is something Glitch should be able to do. But on the other hand, I think Glitch will soon stop supporting their VS Code extension, so you might have to transfer it some other way.
Thank you for all the replies.
It really seems that something happens to the file in transition.
As I said before Chrome crashes, VSCode only uploads ~1kb and using git lfs I only get the metadata
Any other way to upload a file?
Otherwise I’ll try to convert it into another database.
Ok, I was able to use wget to get the file from my dropbox
This seemed to be an unnecessary pain point.
Now my app is using 350/400mb space and 1/2gb RAM … living on the edge, haha.