Outage Update

Hello caregivers, here’s the outage update as of Friday, Jul 20 at 17:45.

• Providence is making great progress across the System. A number of systems are back online, many more are very close.
• IT has 10 caregivers onsite. They’ll be working throughout the weekend to assist you.
• Command Center will continue to monitor progress.
• We will reassess the situation on Sunday morning to determine issues and plan accordingly for the upcoming week.
• The EPIC update is officially delayed.
• Helpdesk is receiving up to 5000 calls with wait times of up to 20 minutes. Impacted caregivers are still asked to perform repeated reboots of impacted workstations. Many have found success after as many as 30 reboot attempts.
• Providence is hearing of phishing attempts from bad actors claiming they are from CrowdStrike and Microsoft. Please use caution and do not click on any unfamiliar links or share personal/sensitive information with individuals on the phone.

Friday, Jul 19, 2024

A message from Chief Executive Brian Helleland

St. Joseph Hospital is impacted by the global Crowdstrike outage. Efforts are in place throughout the system and our IT teams have been working overnight to respond to the issue. A command center has been set up and our leadership team has met to report out. We have our priorities and are allocating resources.

The outage affected servers, workstations, devices and various applications that were on at time of the patch. If your workstation was off, it is safe to turn it back on, although workstations continue to experience unexpected restarts and ‘blue screens.’ If you cannot use your computer (e.g., “blue screen of death”), please attempt to reboot your computer. If rebooting your computer is unsuccessful after several attempts, please escalate the issue to your core leader. Impacted devices will need to be recovered one by one. Applications will be updated in the cloud. Downtime procedures are in place. We hope to fully recover over the next 24 hours. Third party vendors, physician offices, labs, etc., may take longer to recover.

Current status:

• EPIC is online, third-party vendors applications may have outages.
• All cloud applications were impacted.
• Most clinical devices that were turned on affected.
• We are prioritizing critical care areas.
• IT is going PC to PC to recover devices. Some will be down longer than others.
• IT has 25 computers that can be deployed in interim.
• Estimated timeline is 6-8 hours to get things back up.
• We expect an overflow of patients from Urgent Care, physician offices.

I’m including some FAQs for you here. We’ll keep you updated with statuses throughout the day as our systems come back online.

Brian Helleland
Chief Executive Providence St. Joseph Hospital
Chief Executive Orange County/High Desert, South Division