mastodon.social - Server autoscaling/capacity issues – Incident details

Server autoscaling/capacity issues

ResolvedDegraded performance18 hours 30 minutes

Affected

Website & API

Degraded performance from 4:17 PM to 10:48 AM

Background queues

Degraded performance from 4:17 PM to 10:48 AM

Streaming API

Degraded performance from 4:17 PM to 10:48 AM

  • ResolvedUTC

    Capacity issues have been resolved, and the autoscaler has been adjusted.

  • IdentifiedUTC

    There was an issue with our cluster autoscaler where it seems to have scaled down the number of nodes we have for our cluster, resulting in strained resources. We are trying to scale back up, but Hetzner is having availability issues in the region we are deployed in. We are working to resolve the issue as quickly as possible.