GitLab system status: Operational 🎉
@GitLab Twitter handle status: not so much ☹️
If you are reaching out to @GitLab for assistance, you may get a response from this handle (@gitlabstatus) or from one of team members' Twitter accounts.
ALT Twitter fail whale throwback with Tanukis carrying the whale
GitLab.com is experiencing high latencies due to Postgres database query issues. We are currently investigating why some queries are no longer being optimized correctly. monitor.gitlab.net/dashboard…
GitLab.com is operating normally once again. We are continuing to investigate the root causes of today's incident, and will conduct a post-mortem in gitlab.com/gitlab-com/infras…. Thanks for your understanding, and once again, apologies for any inconvenience caused.
Service is returning to normal on GitLab.com after our secondary instances lost replication while carrying out planned maintenance. Retrospective to follow at gitlab.com/gitlab-com/infras…. Apologies for any inconvenience.
GitLab.com continues to experience high latencies and queue times after a corrupt replication segment caused our secondary Postgres instances to stop replicating off the primary. We are working to resolve the problem.
Working doc here: docs.google.com/document/d/1…
Initial investigation indicates that during a deployment, a backward incompatibility resulted in cache poisoning. This caused a sharp increase in git errors between 09h35 and 10h05 UTC: Postmortem: gitlab.com/gitlab-com/infras…