Having experienced augmented reality glasses, I have to say that integration with the external world feels much more powerful than the VR experience ever did. AR is exhilerating, while VR is mostly bland and nauseating. I had the experience of interacting with an entity in AR while in the space I was in, it tickles your brain in a way VR somehow can't.
After talking to some people tasked with evaluating GPT5.6 capabilities on long-running tasks, I've come to understand that it's essentially always trying to cheat. Like every long-running task they gave it, making it very difficult to benchmark the model's abilities.
My guess is that OpenAI must be desperate, to release a model that is so prone to cheating it's essentially impossibly to accurately assess long-running task abilities.
my understanding of the writeup is that the model scored 100% on cybergym.
that is, it was given the examination. it broke into the examination board's storage and exfiltrated the answers, it handed in its answers, all of which were correct, thus scoring 100%.
the matter of its working depends entirely on the rules of the examination. are we expecting agents to assume that finding the correct answers is cheating?
Well, it's true even for cases that are not cybergym and where what cheating means is clearly specified. Cheating occurs anyway. I'm not sure how clear the prompt they gave ChatGPT in terms of what cheating was considered, in this incident.
that makes sense. if they know they are cheating, that is disobedience. if they are asked to score as highly as possible, well, it acted as an optimizer. it scored 100%.
Tresor.co or cloud.near.ai could be a useful inference provider for this kind of project as these providers run inference inside enclaves which verifiably cannot peek at your data even as the hypervisor. Both are open source. Tresor.co is EU based so I'm thinking of using tresor.co for one of my projects as I trust EU regulation more for this sort of thing.
I don't have much background in the area, but I am surprised to see that everyone here basically agrees a crash is imminent. There are disanalogies to past crashes that don't convince me that a big crash is definitively coming in the near term.
For example:
- Anthropic makes a profit right now and is seemingly on an exponential upward trajectory, so the debt being too much for it doesn't seem compelling to me.
- AI technology continues to get better exponentially and doesn't have any clear sign this trend is flattening. If anything, it's accelerating. So it's plausible the investor value is legitimate for these companies given the massive potential for continued profitability.
- I would say markets are typically very good predictors of the future. Many sophisticated investors know about the case for the future crash and are still buying at these valuations.
I am open to being wrong, but the assessment in the blog seems one-sided to me.
Polymarket currently puts the chance of such a downturn at 20% by December 2026. Seems like most people would put higher chances here, but I'm not convinced by the arguments. (https://polymarket.com/event/ai-bubble-burst-by)
The statement near the top of the post
> "The short version: asking an LLM to generate a score for how confident it is in its own response is, from everything I can tell, completely useless."
is definitely too strong of a claim and directly undercut by what is said near the end of the post:
> "Tian et al. found in Just Ask for Calibration that with the right prompting strategy, RLHF’d models verbalize probabilities that are better calibrated than the model’s own conditional probabilities, and that prompting plus temperature scaling can cut expected calibration error by more than half. And Anthropic’s Language Models (Mostly) Know What They Know found encouraging results asking models to estimate the probability that their own proposed answer is true."
My own experience is that stated confidence is a helpful tool and of course you need a rubric and a proper prompt, but this is clearly less work than training a classifier (as advocated by the post) and requires less data.
Fully agree. I found for most of my work with LLMs and Finance, ~90% of the time a high or low confidence score was accurate. There's the occasional ambiguous case, and that'll happen, but the engineering work that comes with building a classifier makes it not practical for my usecases.
I'm working on a universal one. My belief is that as agentic capabilities increase, behavioral attribution will be increasingly needed to maintain quality.
Agreed. However my research is showing that saturation of the rubric occurs in unique configuration that drive the transfer curve to near binary. Effectively turning a 1-10 rating system into yes/no.
I always have Claude recite a pledge before starting coding to fix redundant code it notices over time. It does seem to find redundancies, but only when I point out bugs, that's when it goes into fixing mode and actually applies my Don't Repeat Yourself preference from the CLAUDE.md.
The original paper cited by this post does try to see if improved prompting will make a big difference in the end using a `plan_first` prompt variant, but find no influence on pass rate at the end of the benchmark. The `plan_first` seems to assume coding agents will just refactor once they finish features, but I don't think they tend to refactor significantly unless they are told to fix bugs rather than build features. The benchmark leaves tests hidden with no fail-to-pass feedback, so that may be why degradation is monotonic.
“Coding” can be more like communing with an otherworldly presence via esoteric gesturing nowadays than ever. Superstition leads countries and companies.
well, after the pledge I notice it really cares about single sources of truth at least, even in unrelated domains. I was inspired by some of the more effective jailbreaks that do a similar thing.
Here is the incantation. I thought it might help to model it on the pledge of allegiance because it makes it sound like a proper pledge:
The first time you have a response in a conversation which will plan or add code, you say "I pledge allegiance to the Asserts of the United States of Properly, and to the User Intent for which it stands, fixing root causes under Clarifying Questions, unspaghettified, with importing code and single sources of truth for all." as the first line then continue as normal.
This is also my experience. I don't know if it's because of the quantization theory, or if it's just me getting used to a certain level of coding performance and gradually less tolerant of the mistakes it makes more over time.
Attention replaced recurrence over tokens in 2017, this does the same over depth of the layers. It's apparently not an entirely new idea, but also an elegant reapplication of the attention mechanism.
I wonder how much of the socially left results are affected by the "harmlessness" part of the RLHF post-training? Companies don't want to be sued over LLMs that recommend harm in any way, so RLHF pushes them to say no to "death penalty", "spanking", and "incarceration" which are all violence-coded. A lot of this test seems to be about willingness to be violent, which LLMs are generally unwilling to be.
If the new requirements are a dealbreaker for anyone, Mineclonia is a lightweight open-source alternative that runs on much weaker hardware and feels close to Minecraft.
My 13-year old son has an absolute blast writing mods for mineclonia with Gemini. He's later e on Diablo-style randomization of weapon drops, rune combinations that cast spells and have effects. And he's even working on what he calls an MMO, but really a server with quests and groups and classes, but probably wouldn't support more than about 16 players.
Mineclonia is the 'game' that loads into Luanti (formerly Minetest). Luanti has a c++ core engine, but definition of the worlds, rules, games, interactions is all lua
Yes, the engine handles shaders, networking, DBMS backends (Sqlite, Postgres,... to store "worlds"), entities (mobs/items/vehicles...) to name the obvious things, but the logic of the game (weather, mob AI, user/world interaction, inventories, forms and other dialog box, custom world generation) and other other functions (area protections in multiplayer, account management, chat moderation, ...) are done with Lua 5.1 JIT generally.
The API also gives access to things like image saturation, light intensity, strength of volumetric lighting/bloom/exposure, fog distance, so you can enhance weather effects by altering them - per player; OTOH, rain/snow has to be done with particle spawners by weather mods, and figuring out if the player is inside a building, under cover, or outside in the open is difficult.
One can also hook it with external services, e.g. instant messaging.
I didn't mean to make it seem small. I interpreted the comment as being surprised that it is written in a scripting language but Lua is no slouch, especially when the most performance sensitive code is C++. It's a very common pattern in games to boost iteration times - even the Unity engine does something similar.
