Crowdstrike Bug Shuts Down… Basically Everything

[iluvatar5150]

Last I read, the fix was to boot each machine into safe mode manually and delete a specific file. No word on how that’ll work with machines that use something like Bitlocker to encrypt their drives.

Oops.

https://www.cnn.com/webview/business/live-news/global-outage-intl-hnk

 

[iluvatar5150]

I wonder how @ThatRobGuy is doing this morning.

 

[Vambram]

Last I read, the fix was to boot each machine into safe mode manually and delete a specific file. No word on how that’ll work with machines that use something like Bitlocker to encrypt their drives.

Oops.

https://www.cnn.com/webview/business/live-news/global-outage-intl-hnk

Let us hope that this shall quickly get fixed and resolved all over the world. However, it doesn't look like that the fix is going to be easy OR quick.

 

[Goonie]

The effect of just one bug in one update is an alarm call. At the moment this looks like programming error, but I'm sure bad actors are paying attention.

 

[ThatRobGuy]

I wonder how @ThatRobGuy is doing this morning.

It hasn't been my most fun morning lol.

Been on the phone with various clients since 6am.

Weird stuff always seems to happen the Friday before I leave for vacation.

 

[RDKirk]

Glad I'm retired, or this would have been a bad day.

 

[ThatRobGuy]

Glad I'm retired, or this would have been a bad day.

Yep, and the issue is even "less fun" when it not only impacts your client's operations, but your own organization's internal operations as well.

Trying to get ourselves back up and running while simultaneously trying to fix client stuff as well...

Good times...(not)

 

[ThatRobGuy]

 

[iluvatar5150]

View attachment 351871

it's funny because it's true lol

 

[RocksInMyHead]

Fortunately, we appear to be completely unaffected.

 

[wing2000]

...same here. A handful of minor applications impacted out of hundreds.

 

[ThatRobGuy]

...same here. A handful of minor applications impacted out of hundreds.

Consider yourself lucky lol.

The issue is that a lot of the clients we work with, involve
A) A lot of interactions happening via API endpoints
B) ones who are in various stages of disaster recovery after previous cyber events, which means the particular Crowdstrike product (their Falcon endpoint sensor) is basically ubiquitous throughout their network. (which is the actual product of Crowdstrike that had the major issue)

 

[essentialsaltes]

Fake and old, but glorious:

 

[The IbanezerScrooge]

Weird. I work in IT and have seen no issues. Is this mainly Europe and Australia? I don't even actually know what CrowdStrike is\does.
Maybe I'm just in the walls of the Emerald City.

 

[ozso]

Weird. I work in IT and have seen no issues. Is this mainly Europe and Australia? I don't even actually know what CrowdStrike is\does.
Maybe I'm just in the walls of the Emerald City.

Everything is down in the greater Seattle area. First word early on is it was a hacker attack. I'm glad that's not the case.

 

[ThatRobGuy]

Weird. I work in IT and have seen no issues. Is this mainly Europe and Australia? I don't even actually know what CrowdStrike is\does.
Maybe I'm just in the walls of the Emerald City.

One of their main products/features "the falcon endpoint sensor" is (or now "was") considered to be the gold standard for prevention of things like ransomware and some of the harder to fight malware out there (like things that can infect VM hosts, etc...)

In order to do that, it requires a level of access to windows systems kernels that, while necessary to address the tougher malware, also means it has the potential to cause havoc as well.

It's used pretty heavily by some pretty major organizations (like hospitals, credit bureaus, airlines, etc...)

 

[adrianmonk]

Would have loved to say this was predicted but:

Though Crowdstrike released a report that companies skip reviews of updates half the time.

Firms skip security reviews of updates about half the time

Complicated, costly, time-consuming – pick three

www.theregister.com

 

[adrianmonk]

it's funny because it's true lol

As a mac user I can confirm this.

 

[adrianmonk]

One of their main products/features "the falcon endpoint sensor" is (or now "was") considered to be the gold standard for prevention of things like ransomware and some of the harder to fight malware out there (like things that can infect VM hosts, etc...)

In order to do that, it requires a level of access to windows systems kernels that, while necessary to address the tougher malware, also means it has the potential to cause havoc as well.

It's used pretty heavily by some pretty major organizations (like hospitals, credit bureaus, airlines, etc...)

Some games include kernel level drivers for their anti cheat solution which is another disaster waiting to happen. (well it has already happened with some games already).

 

[RDKirk]

One of their main products/features "the falcon endpoint sensor" is (or now "was") considered to be the gold standard for prevention of things like ransomware and some of the harder to fight malware out there (like things that can infect VM hosts, etc...)

In order to do that, it requires a level of access to windows systems kernels that, while necessary to address the tougher malware, also means it has the potential to cause havoc as well.

It's used pretty heavily by some pretty major organizations (like hospitals, credit bureaus, airlines, etc...)

But it actually worked as programmed, didn't it?