As some of you have noticed, unfortunately, we experienced a four hour outage last week. People could not access their scripts through the browser and real-time sync was not enabled.
People could still write in the desktop app, but those that were online as the outage happened couldn’t.
The technical issue turned out to be trivial and no data was at risk. That said, it was an annoying issue for everybody working at that time, and we sincerely apologize for that.
Stability is our top priority and this should not have happened. Here is what we did to prevent such outages from happening in the future:
- The actual issue was caused by a misconfigured alert, which caused our database storage to fill up. We added the necessary alert to avoid that from happening in the future.
- As you know, our offline writing support is very robust. However, some people where kicked out when the crash happened, which made working offline impossible for them. We addressed this bug: Should we have another server side incident, you will be able to continue offline without interruptions.
- We’re improving our incident management processes: the incident took minutes to resolve, but significantly longer to escalate to engineering. We will avoid that in the future.
- We’re working on making exporting PDFs possible while offline, to further increase the robustness of our offline mode.
Thank you for your understanding and patience during the incident. If you have any questions or concerns, please do not hesitate to reach out.
– – Michi, Founder Arc Studio Pro