Redis data persistence

I am currently using redis in my project. However, sometimes all the data is lost when the app restarts. The redis is stored in .data/redis-stable/ folder. Is this intended or is there some configuration I could do to ensure data persistence?

The file system is persistent by default, there’s no configuration required. What happens when the data is lost, is the database file empty or no longer exists?

Sorry if i’m being random but, how did you installed redis on glitch @Andy-William?

1 Like

@Gareth I’m not sure where redis saves the database file, but the data is gone. All keys return empty value.

So far the data loss never happens when it restarts because I’m coding, but rather on other random time of the day (when I assume glitch updates so all projects got restarted).

@NikoBotDev
running the following command on .data

wget http://download.redis.io/redis-stable.tar.gz
tar xvzf redis-stable.tar.gz
cd redis-stable
make

source: https://redis.io/topics/quickstart

1 Like

Read about how to configure Redis persistence in the Redis documentation.

From redis-cli, you can check if persistence is enabled using CONFIG GET save and CONFIG GET appendonly.

You can check where the RDB is stored using CONFIG GET dbfilename, and the AOF log CONFIG GET appendfilename.

When following the steps you wrote above and start the server with .data/redis-stable/src/redis-server &, I see the following output on .data/redis-stable/src/redis-cli:

127.0.0.1:6379> CONFIG GET save
1) "save"
2) "3600 1 300 100 60 10000"
127.0.0.1:6379> CONFIG GET dbfilename
1) "dbfilename"
2) "dump.rdb"
127.0.0.1:6379> CONFIG GET appendonly
1) "appendonly"
2) "no"
127.0.0.1:6379> CONFIG GET appendfilename
(empty list or set)

This output means that Redis will save a dump file to .data/redis-stable/dump.rdb every hour there is a write, every 5 minutes if there have been at least 100 writes, and every minute if there have been at least 10000 writes. (If the Redis server is cleanly shut down, it will also write a dump file.) However, if there hasn’t been a write in a while and your container is shut down, Redis won’t get the chance to write its dump file, and you will lose data.

My recommendation is to turn on appendonly, so that your data is written to disk almost as soon as it can be. You’ll be much less likely to lose any data this way.

2 Likes

It turns out my config file wasn’t loaded to the server :sweat_smile:

It now properly loads and I changed to AOF also. The file is saved in /app/appendfile.aof though, so I think I need to change it to somewhere in .data.

Anyway, thank you for your help.

1 Like

@Andy-William I can’t start redis-server from glitch, can you help?

I did

tar xvzf redis-stable.tar.gz
cd redis-stable
make 

but bc I need permission (on glitch) to add the command to the .bash_profile, it doesn’t work on glitch, just on my local machine. Apparently it’s all about the binaries not working. So I can’t start the server (not by opening connection on my .js file and not even through the json file on the start section)

I really need redis connection for my code to run. It runs on my local machine bc I have everything properly installed

@tlowande try this …

Add a .gitignore file with the folliwing line

bin

Then redo your make like this …

(incorrect line deleted)

It should put the binaries into a new folder /app/bin/ . The nice is to avoid CPU overuse warnings.

Then you can run

/app/bin/redis-server

then I get

app@habitual-license:/tmp/redis-stable 14:58 
$ nice make install /app
cd src && make install
make[1]: Entering directory '/tmp/redis-stable/src'

Hint: It's a good idea to run 'make test' ;)

    INSTALL install
install: cannot create regular file '/usr/local/bin/redis-server': Permission denied
Makefile:306: recipe for target 'install' failed
make[1]: *** [install] Error 1
make[1]: Leaving directory '/tmp/redis-stable/src'
Makefile:9: recipe for target 'install' failed
make: *** [install] Error 2

so I manually created a bin folder then I was able to move the commands I needed to this folder and somehow it merged with original bin.

Funny thing is that by doing manny attempts, I got access to bin folder in one of my remixed projects and I don’t seem to be able to replicate it though

Sorry I pasted the wrong command, here is what actually worked!

echo bin >> /app/.gitignore
cd /tmp
wget http://download.redis.io/redis-stable.tar.gz
tar xvzf redis-stable.tar.gz
cd redis-stable
nice make PREFIX=/app install

nice prevents high CPU usage warning on container.
The above creates a folder /app/bin with the redis binaries.
The /tmp folder will clean itself up on the next project restart.

I then created redis.conf

# https://redis.io/topics/config

bind 127.0.0.1

protected-mode yes

port 6379

pidfile /app/.data/var/run/redis_6379.pid

dbfilename dump.rdb
dir /app/.data/

# glitch has memory constraints
maxmemory 400000000
maxmemory-policy noeviction

appendonly yes

auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

Start script in package.json

  "scripts": {
    "start": "/app/bin/redis-server /app/redis.conf & node server.js"
  },

The start script could be improved, for example only start redis if its not already running.

3 Likes

Thanks for the quick tutorial, but i get this error

Can't chdir to '/app/.data/': No such file or directory

do you know how to resolve?

EDIT:

im resolve this by use this command at console:

mkdir /app/.data

can u please help me more regarding package.json file. Means is there any error in code or not.

Also a big request to u , please provide a improved file also.
Thanks in advance…

i doubt that person will respond to you, they haven’t posted in a year.

1 Like

yaa, i also think so. Do u have any solution regarding this problem.
please let me know.

can u please help me more regarding package.json file. Means is there any error in code or not.

Also a big request to u , please provide a improved file also.
Thanks in advance…

do you receive any error when you put the code? if yes, could you send the error information?

see i am receving this error


on executing of package.json

{
“scripts”: {
“test”: “echo "Error" && exit 1”,
“start”: “/app/bin/redis-server /app/redis.conf & node server.js”
}
}

Make sure you use proper quotation mark " and all quotation mark is paired properly. You can use this website to validate https://jsonformatter.org/

2 Likes

thanks for this suggeston but problem and code remain same

can u please describe what you are saying?
if u did not understand the problem clearly u can refer to https://stackoverflow.com/questions/72032024/redis-server-on-glitch