Storage / persistence / database suggestions


#1

UPDATE: This post is probably subsumed by glitch.com/storage

There’s nothing built in for storing state accessible to multiple users, right? I see @nancy recommends DynamoDB in this awesome tutorial:

Is that the easiest way to get simple persistence? Any other recommendations? Firebase? Redis To Go?

UPDATE: Compiling list of options with example HyperDev Gomix Glitch projects:

  1. Amazon DynamoDB: example with Passport.js, HyperDev tutorial, HyperDev blog post
  2. Postgres on Heroku
  3. Google Firebase: example by @nakome, HyperDev blog post
  4. Mongo mLab: example
  5. RedisToGo: example
  6. Google Cloud Datastore?
  7. Microsoft Azure?
  8. IBM Cloudant?
  9. Airtable: example

Comparison of 8 different Redis options (including Redis To Go, listed above)

UPDATE: Each HyperDev project gets an Amazon DynamoDB. Example: https://hyperdev.com/#!/project/typhoon-pine


#2

Hi dreeves,

You should be able to make use of any external database/persistence service. I’ve personally used a Heroku Postgres Database by putting the DATABASE_URL in .env. I’ve also had good experience using Firebase with HyperDev. I image Redis To Go would work similarly, just use the url provisioned by your provider in your .env and add the appropriate npm module to your package.json.


#3

We have an example MongoDB project, that uses a free mlab mongodb instance (https://hyperdev.com/#!/project/navy-flower). There’s also a DynamoDB project that you could remix to get started with (https://hyperdev.com/#!/project/typhoon-pine).


#4

Sweet! Thanks @DanielX and @Gareth. In the meantime I set up a free Redis To Go instance and that’s working fine. Maybe I’ll wikify my original post above and collect all the suggestions and add links to example HyperDev projects that use them.

Meta question: Database questions are pretty orthogonal to HyperDev (unless y’all are thinking about building something in). I’ll assume we should generally avoid such questions and keep this forum to things specific to HyperDev.


#5

Database questions are fine :smile_cat:, most apps need databases and it’s a good discussion!

We might add something with more integrated databases in the future but for now our thinking is that any provider should work with a little manual set up . As we gain insight into what common problems people encounter, and their solutions, we’ll see about things we can do to make integrating databases/persistence more seamless.


#6

On a related topic, it appears that we cannot write to files from the application? I tried to use FileCookieStore and it stated that the file system is read-only. Is that correct? And if so, is that likely to change?

Thanks!


#7

That’s right - the filesystem within each project is stateless, so you can’t write to it.


#8

Oh, right. Hmm, but I can create files manually. Is there any way - within reason - to make that same faculty available to applications?


#9

I believe that you can write to /tmp inside the container. We’re exploring the ramifications of having the container file system being generally writable but haven’t yet reached a conclusion.


#10

Neat, I’ll have to try that out.


#11

I don’t seem to be able to edit the top post anymore. I was going to have it be a wiki that I or anyone could keep updating. In particular I wanted to add a link to the HyperDev blog post on using Firebase.

Relatedly, some feedback: I think this TMTOWTDI approach is bad for newbees. It feels overwhelming and you get decision paralysis. Just tell us what to do! :slight_smile: I mean, it’s great to have the option to use any datastore if you happen to already have a favorite but, speaking from personal experience, I got kind of frustrated finding myself down the rabbit hole of investigating them all and looking at example HyperDev projects that I wasn’t sure used the latest recommended practices, etc.

I guess the ideal would be for freshly created HyperDev projects to already have a working persistence solution. Then there’d really be no doubt. Even if it was commented out and you had to go get a key or something to actually use it, it would be so nice not to have all the doubts and second-guessing about the best way to do it.


#12

I’ve made your post a Wiki, so you can update it (I think). I’ve created a new DynamoDB example, which removes the need for the hyperweb package that typhoon-pine needed, in favor of straight Express (https://hyperdev.com/#!/project/linen-hunter) and added an Async DynamoDB example too (https://hyperdev.com/#!/project/caramel-knife).

DynamoDB is what we recommend for persistence, which is why a free instance is created with embedded credentials for each HyperDev project. We’ll be exposing it in a more user-friendly way going forward. We just have to get the back-end updates out first to deliver better stability.


#13

Hey all,

DynamoDB support is being deprecated with an upcoming infrastructure update. The short reason why is that we don’t feel like it’s a great or maintainable solution going forward.

here’s a longer explanation

As DanielX mentioned above, the best bet is to go with a third party db service right now.

thx!


#14

Third party DB services don’t seem like a great fit for HyperDev. If I remix a project, won’t we both end up with the same API key and write to the same database?

I think a local sqlite database would work really well. The dynamodb depcraction thread mentions that there will be a properly writeable FS in future. Where will this be announced once ready?


#15

Hey Wilfred,

When someone remixes your project, your .env file (where you’d store secrets like API keys) values are not copied to the new project.

yup, we’re also looking into sqlite as an opt-in option as well. When it’s ready for testing we’ll announce it in the forum. When we’re more confident in our implementation we’ll announce it in an in-app feature announcement.


#16

@Wilfred here’s a local SQLite database example using the local filesystem: https://gomix.com/#!/project/sqlite3-db