mastodon.social - Backend outage – Incident details

Backend outage

ResolvedMajor outage2 hours 40 minutes

Affected

Website & API

Major outage from 10:33 PM to 10:42 PM, Degraded performance from 10:42 PM to 1:13 AM

Background queues

Major outage from 10:33 PM to 10:42 PM, Degraded performance from 10:42 PM to 1:13 AM

  • ResolvedUTC

    Kubernetes node issues have been resolved, and service is now fully restored.

  • IdentifiedUTC

    Website is operative again. Issue seems to be related to kubernetes pods becoming completely unresponsive. Continuing to investigate.

  • InvestigatingUTC

    We are experiencing issues with our redis caching system, as well as kubernetes node errors. We are currently investigating the issue.