This page lists incidents by Namespace and known upstream issues.
This page lists incidents by Namespace and known upstream issues.
No components marked as affected
Resolved
All systems are operational.
Monitoring
We reached out to GitHub and they rolled back the experiment that led to the outage.
We are seeing signs of recovery.
We are still working through the backlog of affected jobs, but new jobs are good already.
Identified
We are working on reconciling the missed state from the invalid events.
We are also working on an alternative ingestion path to handle the corrupt event data gracefully.
Investigating
GitHub has been sending invalid job payloads which lead to jobs not being picked up. Our team is investigating.