Hello! I just got out of a meeting about the incident and wanted to share some additional info with you about what is going on:
- We had launched a few changes last week that significantly altered the way projects run. This caused some unexpected side effects that we have been working on since.
- We are making some massive changes to our infrastructure to ultimately get to a place of much higher stability for the Glitch platform. Because these changes are large and affect critical pieces of our infrastructure, we risk instability in the meantime - and this time the change we made had an unexpected impact on project start times. It was very unexpected, so it took us a while to even understand what was happening.
What we are doing about it:
- We are undoing part of the change that we made to our infrastructure, the one that most contributed to the high project start times.
- For our infrastructure changes going forward, we are going to try a different approach that we hope to have a more controlled effect on the system, trying as much as possible to leave existing projects alone and working on infrastructure changes in parallel.
- All of this is working toward a faster, stabler Glitch in the coming weeks.
Until this incident is resolved, I will be posting any new information that I can share in this thread.