July 2024 CrowdStrike Outage and Resulting Lawsuits

As the law firms queue up:

https://www.globenewswire.com/news-release/2024/07/19/2916069/0/en/Lynch-Carpenter-Investigates-Claims-in-CrowdStrike-Outage.html

I offer this page to remove CrowdStrike clutter from

and

9 Likes

Well timed, sir:

6 Likes

A credible description from Hacker News:

CrowdStrike in this context is a NT kernel loadable module (a .sys file) which does syscall level interception and logs then to a separate process on the machine. It can also STOP syscalls from working if they are trying to connect out to other nodes and accessing files they shouldn’t be (using some drunk ass heuristics).

What happened here was they pushed a new kernel driver out to every client without authorization to fix an issue with slowness and latency that was in the previous Falcon sensor product. They have a staging system which is supposed to give clients control over this but they pissed over everyone’s staging and rules and just pushed this to production.

This has taken us out and we have 30 people currently doing recovery and DR. Most of our nodes are boot looping with blue screens which in the cloud is not something you can just hit F8 and remove the driver. We have to literally take each node down, attach the disk to a working node, delete the .sys file and bring it up. Either that or bring up a new node entirely from a snapshot.

This is fine but EC2 is rammed with people doing this now so it’s taking forever. Storage latency is through the roof.

I fought for months to keep this shit out of production because of this reason. I am now busy but vindicated.

Edit: to all the people moaning about windows, we’ve had no problems with Windows. This is not a windows issue. This is a third party security vendor shitting in the kernel.

9 Likes

Copy from Competency Crisis:

Also from Yahoo!

6 Likes

I call BS on this part. Crowdstrike is also for Linux and Mac. Those platforms weren’t affected. Windows is not secure out of the box, and therefore needs products like this. That Windows needs totally privileged kernel modules to patch up the holes that shouldn’t be there is the fundamental root cause of this fiasco.

Just say no to Windows. At least for any computers that your business relies on.

4 Likes

Nah, it’s CrowdStrike, this isn’t isolated. They managed to break a bunch of Linux system a few months ago.

It didn’t shut down half of the world, so no one really noticed.

Windows is terrible, but bugs, errors and crappy update policies are everywhere.

5 Likes

Fair enough. Still not gonna recommend or use Windows in my critical systems.

4 Likes

How did CrowdStrike get so popular?

All the wonderful work they did for Obama and DNC in 2016 brought them new business?

4 Likes

Cybersecurity is hard, there aren’t a whole lot of competent firms you can go to for help. Sometimes they look competent, until they don’t. Sometimes they are competent but just have a bad day.

One of the weird aspects of defensive cybersecurity is that it is a national level security problem, that the national governments have to leave up to the private firms. There are just too much attack surface and divergent requirements for the government to do anything analogous to, for instance, NORAD or the Coast Guard. (What would it even look like? Have the USG/NSA manage a secured cloud service for all Americans to use? Who would trust them?) In the last “Between Two Nerds” (on the great game) they talk a bit about that. [1]

The democratic-adjacent governments can, to a limited extent, set standards and provide guidelines for liability. That is what part of the NSA used to do and what the CISA, CSRB and NIST do now. Obviously the authoritarian ones can do similar things, more likely to be enforced by draconian measures.

edit: there are some things the national govt can do, like hardening military facilities, but in a market society there are a lot of vulnerable organizations they don’t have direct control of.

[1] wrong episode

8 Likes

Is there an implicit assumption here that NORAD would do an adequate job if ever put to the test? Sort of like the Secret Service has recently demonstrated?

5 Likes

NORADs jobs have evolved over time, but I would say NORAD and it’s associated orgs are doing an adequate job. Originally it was to act as a defense for Soviet bombers, at which I think it would have met expectations, but that hasn’t been an issue since 1984.

The current NORAD agreement (2006 revision) lists the missions as:

I. The primary missions of NORAD in the future shall be to provide
    a. Aerospace warning for North America,
    b. Aerospace control for North America, and
    c. Maritime warning for North America.

The warning parts appear to be adequately done. The control aspects are harder to determine, but I’ll note that they did intercept a plane that wandered over the RNC in a timely fashion, so they at least have the basics right. Their day to day operations are their active operations.

This is a place where the navies and the air forces have a small advantage over ground forces in that their day-to-day active peacetime operations are good training in and of themselves. eg. aircraft servicing, repairing sea damage, navigation, etc. Ground forces have to go to places like the US Army’s NTC or do other lower level wargames to get a similar effect.

I think the Secret Service has a similar problem to the ground forces, it is more difficult to train for incidents, and harder to evaluate quality. This isn’t made any easier by the administration’s DIE policies.

And in fairness to the SS rank-and-file, apparently a lot of the obviously bad security officers at the Trump rally weren’t actually SS. Once the smoke clears, hopefully someone does a full breakdown of just how many were from which security service.

5 Likes

@jabowery Did you every work with Norm Hardy (or for that matter any of the people around him) and the Capabilities based systems?

Looks like MS and Linux are in the process of adding mechanisms (eBPF) so security vendors don’t have to have as many deep hooks into the kernel.

2 Likes

The closest I came to Hardy was the work I did with Bruce J. MacLennan and David P. Reed who were involved with the Intel 432 architecture and its iMax OS file system respectively. My interest wasn’t directly in the capabilities based architecture of the 432/iMaxOS so much as it was in what these guys might be able to offer the futures architecture I was in charge of for the mass market rollout of the AT&T+Knight-Ridder nation-wide network: VIEWTRON. I had looked seriously at the 432 while at Arden Hills operations at CDC because it was apparent the architects left behind by Cray were getting themselves into some serious weeds – slowing things down to the point that I wouldn’t be able to benchmark an economic mass market version of the PLATO system. They were trying to deal with capabilities in the Cyber 180 series, and I wanted to understand what they were doing and why they were putting so much into hardware.

I addressed this obliquely in:

I suppose one thing I did carry forward from that work was a recommendation I made to some of the HP “Internet Chapter 2” (eSpeak) guys circa 1999 that they seriously consider a paradigm in which network objects were identified by RSA public keys. But that whole thing turned out to be just a scam to get H-1bs MBAs at Stanford, as part of the takeover of the Fortune 1000 by India.

Although that obviates cybersecurity at an entirely different level of abstraction, I became insistent on Algorithmic Information Theory and minimum complexity systems in part because I saw that as an indirect way of breaking through to that particular attack vector and killing two birds with one stone. No one would suspect that I was actually going to clean out the information industry of a foreign influence by such an abstract principle. Unfortunately, by the time my old college friend got in charge of Microsoft, it was too late. They had him.

So I’m stuck trying to do things to reform the social pseudosciences with basically zero comprehension by the people who need it the most. Not even Charles Murray – who should be all over it – is remotely capable of understanding.

4 Likes

The wake of the ClownStrike incident,has me looking at old project notes on building a better API than Posix or NT. I easily find a lot of capabilities tutorial material, sometimes with granovetter diagrams, but not a lot concrete on what capabilities should be core in an OS. Any thoughts on terms I can use to reinforce my Google-fu?

3 Likes

Yes, but will EU allow it? :slight_smile:

1 Like

MS is correct. The EU rules made it necessary to ease off kernel security restrictions. The EU is also correct that closing the kernel off is an anti-competitive move.

I suspect the hope is that with better APIs and tools like eBPF there will be enough low level access to make closing off the kernel to deep extensions like CrowdStrike’s more palatable.

3 Likes

Discussion of insurance coverage:

3 Likes

Seems to me that if it is true that an employee could push a change and head out for the day with no review, the company is negligent. Not much different than when someone gets injured on a machine because the company didn’t have a reasonable process for lock out tag out.

Not only should CrowdStrike be accountable for the losses of the companies that suffered a loss, but for every individual that suffered a loss.

It has 2.3 billion more assets than liabilities. Good bye Crowdsrike. Time to be replaced by a competent competitor.

4 Likes

Obviously, but without trying to be cynical here, why should anyone be confident a hollow state’s “justice system” is capable of delivering justice in situations involving organized crime assets like Crowdstrike?

You guys really screwed the pooch when you turned over the information infrastructure to a flood of immigrants simply because some or even most of them are sincere when they raise their right hands.

Mesopredators don’t belong in authority.

5 Likes