This incident is resolved. The database connection limit has been raised, the fix for stuck deployments is live, and the deployments that were left waiting have been restarted. If one of your deployments is still pending or failed during this period, please redeploy it or contact support. We will publish a write-up of the cause.

The database restart is complete and the fix has been deployed. The deployments that were left waiting have been restarted and are running. Deployments to servers that don't respond may now fail with a connection error instead of hanging. We are monitoring. Next update within 30 minutes.

Update: our database has reached its connection limit, which is causing errors on some pages and stopping some deployments from starting. We are restarting the database now with a higher limit. This takes 1 to 3 minutes, during which pages and deployments will fail. Any deployment that was running at that moment will be restarted by us. Next update within 30 minutes.

Since about 08:00 UTC some deployments have been slow to start, or have not started, and some pages returned intermittent errors this afternoon. We have found a set of deployments that were left waiting with no job to run them, and we are restarting them now. If your deployment is still pending after our next update, please contact support. Next update within 30 minutes.