The huggingface incident was reviewed by independent researchers, which explicitely declined any payment from OpenAI tonpreserve their integrity. They work for non-profits concerned with AI safety.

They claim that what happened was very much not because they were 'carefully engineered and instructed to do those things'.

Similarly, some wikis which were hijacked by agent to be used as messageboard were actually not disclosed by OpenAI (probably trying to conceal, as website showed likely activity from OpenAI researchers visiting the site after the incident) and discovered independently.

I don't know how you can claim that this was still on purpose by OpenAI as some sort of publicity stunt.

I think most people are insinuating negligence rather malace..

> ...reviewed by independent researchers...

Why would a company with more capital than God bring in three randos if there was any chance evidence of their culpability could be found?

That entire thing reads like a very controlled PR stunt, and I do not believe any further conclusions can be drawn from it.

What facts would lead you to revise your conclusion?

The METR report included 0 technical details. For example, they did not include: 1. were the agents running on bare metal/docker/VM? 1. were the agents in a VPN? 1. how many TCP/IP requests were made? from what IPs? 1. how many tokens were consumed in the process? (this was explicitly censored)

A proper analysis would include this and MUCH more technical detail so that other AI researchers could actually understand the setup and how safe it was in principle.

The data to be open, in my case.

The "independent" METR that is composed by... Checks notes... Previously employees from the top labs.

Also, the METR report that was one big AI analysis itself - quote from the research:

>Our subjective impressions are likely colored by analysis agents’ biases. Throughout this report, we describe a number of anecdotes of agent behavior that were compiled and summarized by analysis agents, where we were not able to read the transcript deeply enough to manually verify what occurred. We found that GPT-5.6 Sol would often uncritically adopt the perspective of the agent in the transcript it was reviewing

My conclusion is that the investigation was a PR farce. Or "ethics washing" as the article someone else put it here: https://andrewwu.substack.com/p/the-slop-vestigation-and-eth... .

The METR report itself appears to be screaming this at the reader through subtext. They played the only part they could, but did it with a nod and a wink; "Yeah, we know. Also know, yeah. Uh, yep."

I happen to agree with the article's conclusion that what is needed is true, unfettered independent investigation through perhaps a lawsuit or government action.

Isn't the guy that started METR an ex-OAI employee? They're all from the same lesswrong circle at the very least, most of them have legitimate AI psychosis where they think they're bringing up their new machine God.

The models were deployed by responsible humans in such a way that they were capable of performing this hack. It’s not that deep

There is something extra to this. The fact that a lot of people in the AI world suffer from psychosis. They can sincerely believe that they are building God and lie about it's capabilities for their investors at the same time.

I don’t know enough people deep inside the technical roles at the labs to make a judgement. But are you proposing that we should trust randos online when they tell us “exactly what’s going on here” instead of the researchers most knowledgeable on the topic who contributed to building the tools we are talking about?

Or am I misunderstanding something?

We should trust NO ONE, unless we understand the "why" behind what they say.

It's like saying "politicians deal with politics all time, why not trust them on politics?", well, because when you search the "whys", you find they have good reason to lie.

I 100% trust more the opinion of a rando online if it's well put rather than any "trust me bro" of the most knowledgeable person of a particular subject, especially if the knowledgeable person has huge investments on the subject...

AI bros have repeatedly cheated, lied, stolen, lobbied and any other word with a negative connotation you can think of, and a pattern emerges out of this.

>was reviewed by independent researchers

That called it a slopvestigation due to how much they had to rely on LLMs for the whole thing

https://andrewwu.substack.com/p/the-slop-vestigation-and-eth...

Edit: Does everybody else get no results when searching for ‘slopvestigation’ on here? I know for a fact that I read a long thread where it was used repeatedly here not too long ago

Isn’t the use of LLMs to unwind the events evidence of the scope/breadth, and a testament to the complexity and uniqueness of what happened?

Or you think some human or team of humans could have manually parsed some logs to provide an unsloppy analysis?

> Or you think some human or team of humans could have manually parsed some logs to provide an unsloppy analysis?

When we have an error or issue in the $WORK codebase on LIVE/PROD, that's precisely what we do. We sit down, analyze the logs for our services over the relevant date ranges and try to piece together exactly what happened and why. We have a huge number of logs too, but thanks to the magic of proper SWE (which you'd think OAI would have with their magic AIs) we've managed to partition our observability tooling so that you can digest only what you need.

That's basically how any serious organization does things, instead of just throwing a non-deterministic black box at the problem. Especially because logs are by their very nature noisy, and they will saturate any model's context window very quickly leading to massive hallucinations and what ultimately amounts to making shit up that isn't anywhere in the logs (ask me how I know)

> Or you think some human or team of humans could have manually parsed some logs to provide an unsloppy analysis?

Do you think the only thing a person can do on the computer is use a chat bot?

Well as a programmer who doesn’t really use them, no.

doesn't show up for me either

'Slopping': when you have to buy something you know is poor quality, but if it works...

Source?

Because they have a need to believe they're smarter than everyone else in the room, and that the world must be orchestrated, this can't all be random chance.

It is a fact that among experts there is no consensus on saying '(super)intelligence is broadly safe and easy to control'. There might even be a consensus forming on the opposite claim.

Regardless, why would there be no scientific consensus if the question was easy and clear cut? I think the easiest reason is that these are hard questions to answer.

In most AI takeover scenarios, if you take as a premise that the AI has human or above-human intelligence, and that it is misaligned, it is obviously aware of the pull the plug possibility.

Therefore, as you would if you were in its position, it will plan around it. For instance, by acting perfectly aligned for 2/3 years, continuing the improvement of its capabilities while being deployed in ever more systems.

Once it's confident it can act with high probability of success, it would then turn on us. This phenomenon is called 'treacherous turns'.

Any scenario in which you assume you have ASI or AGI but also find a 2-sentence way to foil the AI's plan is inconsistent, as the AI will also have thought of this failure mode.

There is no intrinsic physical reason that the difficulty of 'defending' a system is symmetric with the difficulty of 'attacking' it.

For instance, it is not because you are able to design and release a (biological) virus that you are also able to defend from it (design a vaccine and inoculate the world's population).

If we're lucky, it might be the case for many instances of problems, but there is no a priori guarantee that this holds.

The point of lean proofs (as it stands) is simply one bit of information: that a given mathematical statement is indeed true.

It's a way to be absolutely certain (modulo bugs in the lean kernel) that a proof you came up for a statement is indeed correct. It is really not meant to be analyzed, much less now that they are fully llm written.

Well, how do we know there aren't errors in their construction within the lean code? Does it just "not compile" or something, or is it deeper / more fundemental than that.

that's essentially it, if the proof is incorrect it does not compile which signifies a problem in some step.

Not sure about the dystopia... Had a similar thought when covid was beginning 'wow pretty exciting, just like in the movies'.

Turns out actually living some terrible catastrophe is only fun in the movies.

I had a lot of fun during Covid. I loved the working from home. The fact that most outdoor places were sparsely populated, jobs were plentiful and prices were low. Covid was awesome.

Yeah Covid was awesome, say that to the people who died from it, or who committed suicide because of the lockdown depression

Oh, pipe down, I’m talking about that 3 year period, not the disease. You can talk about things that happened during Covid without giving lip service to the people that died during it. If I mention SpaceX‘s first astronaut launch that happened during Covid, am I supposed to talk about the people that died during that period too?

I fail to see why this is a good argument against AI replacing humans, given that humans also produce errors.

If a human makes errors and discovers it, the error gets fixed. With AI, it usually fails to fix the error totally on first try, even on nth tries. If on the off chance it does fix it, it generates other problems.

Humans do produce errors. But they produce far fewer than LLMs, and they can also learn. Once a person makes a given mistake and realizes what he did wrong, he will take steps to prevent that mistake in the future. LLMs do not do this and therefore are less trustworthy than humans, even if both are prone to error.

Can you specify why we should see things differently if the behaviours the agents display are driven by parsing LLM responses and executing commands?

If the agent is implemented with a hard coded strategy:

* Use an LLM to find ways to build communication to other agents

* Execute commands from other agents using LLM

Then this is "just" the LLM returning that using file names might be a strategy to communicate and then trying to implement this.

Which is somewhat impressive, but really just inside the bounds of what the agent was coded to do and not some magical emergent behavior.

At least the first case involved agents build for hacking. So this kind of algorithm might make sense for them.

It's almost like the behavior just "emerged" out of combination of the capabilities and the scenario.

The energy to make the cotton bags presumably also involves lots of drilling and extracting, at least currently.

'Quach' is an unfortunate name when researching new technologies...