No new documents are loaded

Resolved

Partial outage

Started over 5 years agoMarch 12, 2021Lasted about 12 hoursMarch 12, 20214:00 AM4:05 PMUTC

Affected

Anvil

Articles

Updates

Resolved
March 12, 2021 at 4:05 PMUTC
Resolved
March 12, 2021 at 4:05 PMUTC
The catchup job finished and all documents have been loaded.
Monitoring
March 12, 2021 at 3:12 PMUTC
Monitoring
March 12, 2021 at 3:12 PMUTC
The catchup job is currently running and is expected to complete before 11am (EST).
Identified
March 12, 2021 at 2:45 PMUTC
Identified
March 12, 2021 at 2:45 PMUTC
It appears that the 3am (UTC) loader job "got stuck" and was still running as of this morning. We've manually killed it so normal operations can resume. The catchup job is currently running.

Our monitoring catches jobs that fail, including jobs that fail because of a timeout. We are continuing to investigate the exact circumstances that lead to a job remaining in a running state without timing out.
Investigating
March 12, 2021 at 2:15 PMUTC
Investigating
March 12, 2021 at 2:15 PMUTC
We are currently investigating this incident.

Forge.AI - No new documents are loaded – Incident details