"The app exceeded the memory limit" - caught in a loop. How to restart automatically / at all?


#1

Hi, I’m seeing some weird behavior with the memory limiter thing these last few days. I get the memory limit error, but then there’s no way to restart the server so I get a fresh slate. I’ve tried editing files (including my watch.json file), but nothing triggers a restart. What’s worse is that the server gets caught in a loop whenever it hits the first out of memory error, so the server stays down indefinitely after it hits its first memory limit:

The only way I’ve been able to stop it is to remix the app and then delete the old one. Then, usually about 12 hours later, it happens again. Edit: I found a faster way: Delete the project, then undelete it from the user profile area.

I’d prefer if the server just restarted rather than getting caught in this loop. Is there a way to trigger a restart whenever it gets paused? I’m guessing I’ve got a memory leak that begins to get out of hand after 12 hours.

So this post is two questions in one:

  • How do you escape this loop? Is there a console command? Is this expected behaviour?
  • Is there a way to automatically restart if the memory limit is reached - before this loop starts?

Here’s a minimal example that I created, but unfortunately I’m not sure if it exhibits the bug yet, as it usually takes about 12 hours:

Hopefully by the time the devs see this the bug has surfaced. If it doesn’t cause the error loop in the next 24 hours, I’ll try make it closer to the original project until it causes the problem. I’m using Uptime Robot to keep it awake.

Thanks!


#2

Hey @joe, you can use the refresh command in the console to restart your project manually instead of remixing or deleting/undeleting.

To watch memory usage you could monitor the value in /sys/fs/cgroup/memory/memory.usage_in_bytes and use it to determine when your project was getting low on memory at then use node’s child_process.exec() to execute the refresh command.


#3

Hey @cori, the refresh commant doesn’t seem to help. It’s as if the management software that controls the project isn’t properly terminating everything - the memory usage still stays high. This is immediately after a refresh:

That’s a screenshot from the minimal example (linked to above, also), so it does indeed replicate the bug. I’m keeping it alive with UptimeRobot, so hopefully you can have a poke around in the back-end and see what’s happening.

Thanks for the tips about checking memory usage and child_process.exec()!

Cheers,
Joe


#4

I just checked it, and I didn’t see the problem… if it happens again, can you please post a screenshot of the Console after you run this command:

ps aux

Thanks.


#5

hi @joe,

I was finally able to see the issue. Thanks for reporting! We hope to fix this issue soon :slight_smile:


#12

Hey @joe our team thinks this problem is fixed - please let us know if you encounter more problems!