Suggestions on how to upload a collection of assets that need to be in a fixed structure?

[blairmacintyre]

I am playing with creating augmented reality samples, for the Argon project, using Glitch. So far, it’s been great (esp the support turnaround to fix a few of the bugs I ran into)! One is this: Draggable Cubes

The argon.js framework (at http://argonjs.io) makes use of some parts of Cesium (http://cesiumjs.org). To do some of their operations, they need a collection of data files in a specific structure, which you point to by setting a variable:

var CESIUM_BASE_URL='https://bmaci.com/resources/cesium/';

Within that folder are a collection of directories and files that are accessed via URLs created from that variable. I’d love to be able to upload those files and keep them inside the glitch project, instead of hosting them on my personal domain (as this URL is doing).

This is a similar request to this one:

How can I add a new folder in my assets folder?

I would like to have my assets folder more ordered instead of having everything spread, is there any way to create a folder structure inside the assets folder?

But I’m not specifically interested in just being organized, I simply can’t use these assets if they can’t be in a fixed structure. I realize it may not be feasible to support something like this, but I wanted to check.

[pketh]

Hey Blair,
the assets you upload share the same base url (eg https://cdn.glitch.com/a7b7d5c3-b931-4ce0-a746-fe06ed8da62a%2Ffrog-twitter%402x.jpg?1488681391445) that you can use a decoder like http://meyerweb.com/eric/tools/dencoder/ to output into an human readable format (eg https://cdn.glitch.com/a7b7d5c3-b931-4ce0-a746-fe06ed8da62a/frog-twitter@2x.jpg?1488681391445)

From there it’s clear to see the path that your files all use. So in my case I could do

var CESIUM_BASE_URL='https://cdn.glitch.com/a7b7d5c3-b931-4ce0-a746-fe06ed8da62a/'; .

The /a7b7d5c3-b931-4ce0-a746-fe06ed8da62a/ part is based on your project uuid so it’ll be the same for all the files you upload into your project.

[etamponi]

Hey Blair,

In addition to what Pirijan already said, I’ve written a simple library: assets-lib that provides an express middleware to simulate a folder with all of your assets in it. Perhaps it would come handy for you!

[blairmacintyre]

Hi @pketh, that’s helpful but doesn’t quite solve my problem because of the additional directories. For example, if I set

var CESIUM_BASE_URL='https://cdn.glitch.com/a7b7d5c3-b931-4ce0-a746-fe06ed8da62a/'

then cesium will create URL’s of the form
https://cdn.glitch.com/a7b7d5c3-b931-4ce0-a746-fe06ed8da62a/Assets/IAU2006_XYS/IAU2000_XYS_0.json

I think the files are all in one big bin, so there is no way for me to create Assets/IAU2006_XYS/IAU2000_XYS_0.json when I upload IAU2000_XYS_0.json

[ngokevin]

I’m having trouble with this too. We want to upload 3D models which is a directory with 3D files (OBJ + MTL, glTF) and within those files, textures are referenced via relative paths. For example, https://github.com/KhronosGroup/glTF-Sample-Models/tree/master/1.0/VC/glTF-MaterialsCommon

Would like to be able to just upload the directory, and be able for the JS to load it out of the box with the relative paths. Otherwise, we have to point people to different upload flows (e.g., GitHub + rawgit.com), or I saw someone in the community gravitate towards neocities due to this specifically.

[pketh]

@ngokevin,
you can currently select all the files in the folder and drag them all in at once to get them all into assets. You can also set up your app so that it uses something like https://cdn.glitch.com/a7b7d5c3-b931-4ce0-a746-fe06ed8da62a (described above ) to reference relative model files).

dragging in the folder directly and it doing the right thing is something we do want to support in the future.

@blairmacintyre,
your assets are stored in a namespaced s3 bucket. The way s3 works is that it doesn’t have real folders - we simulate them with the name of the file. so folder/blah.png is basically a ‘folder’. we should be able to make relative paths in assets work by renaming an uploaded asset (eg. changing my-cool-file.jpg to folder/my-cool-file.jpg. it doesn’t yet but we’ll add that to our feature backlog.

programmatically, getting an app that’s making local/relative file system assumptions to output to a seperate CDN-ed bucket is a little trickier, but we’ll try and figure out a solution

[blairmacintyre]

@pketh

@blairmacintyre,
your assets are stored in a namespaced s3 bucket. The way s3 works is that it doesn’t have real folders - we simulate them with the name of the file. so folder/blah.png is basically a ‘folder’. we should be able to make relative paths in assets work by renaming an uploaded asset (eg. changing my-cool-file.jpg to folder/my-cool-file.jpg. it doesn’t yet but we’ll add that to our feature backlog.

So, I tried this, and it appears that it might work already? I edited one of the asset names (didn’t even notice the name was editable before) and then copied the URL, which looked like this:

https://cdn.glitch.com/160acd26-d5a4-4a5d-a6d6-4ed87fe43bad%2Fcesium%2FAssets%2FIAU2006_XYS%2FIAU2006_XYS_0.json?1490623741029

to this

https://cdn.glitch.com/160acd26-d5a4-4a5d-a6d6-4ed87fe43bad/cesium/Assets/IAU2006_XYS/IAU2006_XYS_0.json

and pasted it my browser bar and it downloaded the file. So I’ll try

https://cdn.glitch.com/160acd26-d5a4-4a5d-a6d6-4ed87fe43bad/cesium/

as the base and see if it works!

[pketh]

awesome, ya I suspected it might just work

[blairmacintyre]

Well, sort of worked. Next problem. I want to get rid of the hardcoded cdn.glitch.com part, so I took the assets.js loader that @etamponi suggested and modified to support paths (e.g., but changing the top to

router.get('/*', function (request, response) {
  var name = '%2F' + encodeURIComponent(request.url.substring(1)) + '"';

it redirects appropriately (e.g., setting my path for cesium to assets/cesium and when cesium requests assets/cesium/Assets/IAU2006_XYS/IAU2006_XYS_15.json, the server will use the asset loader and issue a redirect to https://cdn.glitch.com/160acd26-d5a4-4a5d-a6d6-4ed87fe43bad%2Fcesium%2FAssets%2FIAU2006_XYS%2FIAU2006_XYS_15.json

But, now I think I’m having CORS problems again, because if I load this URL directly in my browser it works, but in my web page I get a 403 error and Preflight response is not successful

From looking around, I think it’s CORS and so I tried adding some CORS headers

  response.set("Access-Control-Allow-Origin", "*");
  response.set("Access-Control-Allow-Methods", "GET,HEAD,OPTIONS,POST,PUT");
  response.set("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept, Authorization");

but it doesn’t seem to help (not sure if those are really the right headers, but…)

Thoughts?

[blairmacintyre]

I’ve also tried just including the express cors package, and using it instead of setting the response headers. no luck.

Boy, it sucks to not really understand what’s going on with the control flow here!

[etamponi]

Hi @blairmacintyre,

I sure see that using the assets CDN for your use case is painful! If the images you want to use fit in the project disk, you might want to upload them there instead of using the CDN. You lose the caching layer that CloudFront provides, but perhaps it’s a better solution while we knock off these details?

[blairmacintyre]

Project disk? Not sure what you mean by that.

[etamponi]

The files you see in Glitch are on a real server, that has a disk with space. Assets, on the other hand, are uploaded to a separate S3 bucket to be consumed by the CDN.

I mean that you might try to copy the files in the project, instead of uploading them in the assets CDN.

But if this makes things more complicated for you, just forget what I am saying In the meantime, I am trying to see if I can make my assets lib to work with sub folders.

[etamponi]

@blairmacintyre

I’ve updated the project assets-lib and it looks like it works also with subfolders now, can you give it another try?

[blairmacintyre]

I couldn’t see how to upload except to assets. If I uploaded files like this, they seemed to automatically go to assets.

The assets.js in my draggable-cubes example seems to almost work. It finds the right URL but has this error when my web page tries to access the redirect …

snt frm iOS kybrd

[etamponi]

@blairmacintyre

yes, what I am saying is that I fixed the error you see in the assets.js file you have in draggable-cubes. If you can copy over the file again from the assets-lib project, you can use it without errors.

[blairmacintyre]

Note to self: don’t reply via email. @etamponi, my reply was to this message, via email:

@etamponi

The files you see in Glitch are on a real server, that has a disk with space. Assets, on the other hand, are uploaded to a separate S3 bucket to be consumed by the CDN.

I mean that you might try to copy the files in the project, instead of uploading them in the assets CDN.

But if this makes things more complicated for you, just forget what I am saying In the meantime, I am trying to see if I can make my assets lib to work with sub folders.

I hadn’t yet seen

@etamponi

@blairmacintyre

I’ve updated the project assets-lib and it looks like it works also with subfolders now, can you give it another try?

I’ll try it!

[blairmacintyre]

Hmmm, interesting. So, in Safari it still gives the same error as before

But it seems to work in Chrome and Firefox. Perhaps my attempt worked there too. Hmmm.

[etamponi]

The version in http://assets-lib.glitch.me works fine for me in Safari

[blairmacintyre]

So, it worked for images at the top level: I have a crosshair_small.png that downloads fine with /assets/crosshair_small.png.

But, with the assets I renamed to have paths for names (such as cesium/Assets/IAU2006_XYS/IAU2006_XYS_15.json), it’s giving me that error.