In the physical world, it seems like when an tool/device/instrument causes harm (or is used to cause harm), we assign blame to either the user of the tool or its creator.
When do we blame the user? When the tool is operating as intended by its creator, and we agree the tool meets certain quality standards and isn't defective.
When do we blame the creator? When the device doesn't meet those quality standards and reasonable use caused harm inadvertently. For example, for consumer devices, certifications like UL/CE are used to define acceptable performance levels and safety standards.
Maybe we need "quality certifications" for AI agents - essentially eval suites that demonstrate those agents won't cause harm under reasonable patterns of usage. Right now, these eval suites are run best-effort by the labs themselves.
The tricky thing is, a lot (all?) of these recent safety incidents have occurred while evaluating these models! This suggests we need much more rigorous standards for how exactly an eval can be run. Perhaps all of them should occur in truly air-gapped environments... though that may run counter to evaluating agents in a realistic way.
Regardless, it feels like the "industry standards" common in, say, electrical engineering and other disciplines are sorely lacking here. Unsurprising given how new these technologies are, but concerning since the blast radius for this technology is likely much larger than other technologies we've encountered in the past, except maybe nuclear technology.
> In the physical world, it seems like when an tool/device/instrument causes harm (or is used to cause harm), we assign blame to either the user of the tool or its creator.
Software executes in the physical world, and is generally not exempt from existing liability rules, and actually (especially with commercial products) blame in traditional liability is non-exclusive and much broader than “either the maker or the user”.
E.g., for a harms caused by a defective automobile it can simultaneously covered by a duty of the owner to maintain it it in safe operating condition that applies indepedently of any defects and liability for defective products which applies to every actor in the chain of commerce between the manufacturer and end user, not just the maker.
Software industry standards are as in Microsoft EULA. If your house burns down because of known flaw in Microsoft Windows they are not liable (well as far as EULA let’s them, you can most likely still sue them).
Software as big as operating system already is non deterministic when integrating with unknown hardware or 3rd party software.
That is why Apple controls the hardware and OS for their products, because they can limit non-deterministic things from happening this way.
> If your house burns down because of known flaw in Microsoft Windows they are not liable (well as far as EULA let’s them, you can most likely still sue them).
A EULA does not obviate responsibility of a company for its products. Continuing with your example, while it may be very difficult to prove a known flaw in MS Windows was the cause of your house being set afire, if one had said proof, a EULA would not absolve Microsoft.
That's a good idea, but a physical device is deterministic most of the time (if not always). E.g.: A lawnmower, as credited by the great Bryan Cantrill.
However an AI agent, or the model powering it is stochastic by design. How can you certify something which doesn't behave the same twice, and more importantly we don't understand how it works 100%?
BTW, really, how is that AI observability work is going in the frontier labs? Do they care, even?
I agree with one of the sibling comments that determinism isn't necessary for certifying a product. All engineered products operate under uncertain conditions; we define standards for how those products ought to respond under those conditions and verify them under measurement. Consider robot vacuums, for example.
I also agree that qualitatively, this technology seems different than the others. However, I feel that people tend to overly fixate on their internal stochasticity. Even if LLMs' internal mechanism is nondeterministic, shouldn't we be able to verify their "side effects" aren't harmful? Of course, "harm" is subjective and at this scale, the most effective way to verify behavior is probably some kind of LLM-as-judge...
Anyway, in this case the problems have occurred while actually running the evals themselves, so again, we're in a situation where we can't even confidently test these things and know that they won't cause harm in the outside world.
That we don;'t understand it is not an excuse, it's all the more reason to not let these things roam freely, with this amount of potential to do damage.
How can you certify something which doesn't behave the same twice, and more importantly we don't understand how it works 100%?
That's a question any lawmaker has already had to ask about technology all the time.
I'm not saying they came up with great answers, but there's nothing qualitatively new about that.
The stochastic factor doesn't change the fact that companies have to be accountable for the harms their software causes. That's just basic liability law.
> The stochastic factor doesn't change the fact that companies have to be accountable for the harms their software causes. That's just basic liability law.
We're on the same page. What I'm saying that certifying them as safe is harder than certifying a drill as safe, and we shall be more cautious about AI related technology and be more stringent about the can of worms it opens without hesitation.
An airline has a weather radar which shows the same thing for the same thing of weather event ahead. So, for similar weather phenomena, radar shows a similar thing.
For that thing, procedures and regulations are built. So regulations fit into a well understood phenomena, incl. "return back because that thing is way powerful for us".
For the same prompt, an AI model can return two completely different outputs, incl. but not limited to content, length, formatting and tiny details. What you get is a single instance. So, regulating an AI model for safety or any other property is not as easy as regulating air travel. Moreover, you have much stronger motivations for regulating airlines. Otherwise people die in a visible and gruesome way.
With AI, it's easy to whitewash problems. Somebody committed suicide? "They were already unstable". AI told something wrong and created problems? "The tech can’t guarantee truth because it's not alive, it can't understand right and wrong". It did something good? "It's probably a sentient being, we shall respect them".
I'm for regulating these things. They are dangerous as they are useful (sometimes), but the forces and motivations for regulating it is not the same.
Of course, it's the same. It's computer software. It's an incredibly powerful business automation tool. It's a lot of things.
What it's not is God or an independently conscious entity that somehow trumps a thousand years of common law that's built up until now about torts and liability.
Of course, there are some novel issues here that'll pop up here and there, but the idea that this is fundamentally different is propaganda on the part of these AI labs because the more boring, obvious situation doesn't favor them.
> Is AI less deterministic than an airline dealing with weather?
Yes, obviously? The responses of an airline to inclemement weather fit in a reasonably small set of responses, mostly involving rescheduling and/or rerouting flights.
The current AI predictability would be like if some airlines decided to do 9/11 when it was raining.
The current so-called scandals about AI hacking into other companies were because a bunch of human beings intentionally configured the software to go and do exactly that thing.
There's nothing deterministic about weather, so hopefully you're not just being disingenuous.
It's obvious that the global transportation system, or financial markets, or any number of other things are complex adaptive dynamic systems that are on par with AI in terms of their emergent properties.
Check my username. It's a concept I spent a lot of my life paying attention to.
Just because something has elements of autonomy or is adaptive doesn't make it particularly novel. We've dealt with those kinds of systems for centuries. The solution is to make rules and enforce those rules by whatever means are needed to meet the specifics of the case.
The rules, of course, are enforced against human beings.
> In the physical world, it seems like when an tool/device/instrument causes harm (or is used to cause harm), we assign blame to either the user of the tool or its creator.
Firearms are a notorious example where some people get, well, weird.
Having worked in self driving cars safety, the process there was simple: get confidence in SIM (integration tests for safety scenarios), validate in the test bed, approve features for maturity, then when released in the public for testing, do a trial exposure to the real world and recall if something is off.
A lot of these companies have gone the way of Tesla and decided to just patch on top when the fix is out and hope for the best, which is irresponsible.
We need the regulators to treat this as self driving cars.
Physical harm vs consequential harm is not the same thing at all. Seems like you're being paid to spread this request for regulation, or "you" are simply an agent of Anthropic/OpenAI.
I wonder what kind of new AI law would be useful right now. Maybe this one:
if an AI agent does something, you (the prompter) are responsible by default, unless you can show that your the agent itself behaved in an unexpected way and that you in no way prompted or hinted at the bad behavior, in which case the model provider is liable
The idea is that by making it clear who is responsible, corporations and others start paying more attention because they become financially liable.
On the other hand, I wonder if we'll end up with another variation of the cookie law, where every AI user or vendor just adds "don't do anything illegal" as part of their prompt to defend against that law. Thoughts?
The law that should be in place is that AI users and provides should be required to carry liability insurance when operating machines that may cause harm to the public.
We already do this for drivers, who are likely to hit the public with a few tons of steel.
> "industry standards" common in, say, electrical engineering and other disciplines are sorely lacking here
I think this misses the rather crucial fact that nobody can agree on a standard because nobody has the first idea what they're doing. I'm pretty sure there were very much fewer electrical engineering standards while it was all being first mass deployed, and after dozens to hundreds of fires and electrocutions people got an idea of what works and what doesn't.
You might debate here and say that some people did/do know what they are doing, but I posit that large scale deployment like this is very different to their toy model/prototypes/specific circumstances/rely on them being unnaturally smart, and learnings from one don't often translate to the general case
Regulations don't have to be written in blood, but usually are
> Maybe we need "quality certifications" for AI agents - essentially eval suites that demonstrate those agents won't cause harm under reasonable patterns of usage
Based on how LLMs work, this is impossible. You cannot predict how they work, it's literally based on a combination of random seed and a mostly-unpredictable path walked based on every token of input.
You don't blame a knifemaker for somebody getting cut by a sharp knife. AI is a knife. Very handy, very dangerous. We have to use them safely, that's all there is to it.
> the "industry standards" common in, say, electrical engineering and other disciplines are sorely lacking here
100% agreed. We have ignored SWEng's lack of discipline for too long. Now that the SWEng isn't even a human, we are looking at total catastrophe (on the scale of improperly built buildings falling down on people or catching fire) if we don't adopt a software building code.
This article is ostensibly about biology but it's really about pedagogy—how traditional education squeezes out the sense of discovery and turns many subjects into rote memorization exercises.
This reminds me of the pedagogical philosophy of Seymour Papert, who was heavily influenced by the ideas of Jean Piaget. Piaget's "genetic epistemology" argues that knowledge and understanding is created by interacting with environments, which traditional educational approaches fail to provide.
Papert combined Piaget's ideas with the emergence of computing to argue that children should be taught subjects in a hands-on, exploratory way - and not just for teaching computing! The idea is that by programming in simplified languages, children can discover ideas in subjects like mathematics and grammar - deriving them as they try to solve problems instead of having them dictated to them.
Suppose that biology was taught in a game-like environment where students were designing cells or organisms in some way. Maybe the game could be structured so that each organelle could be "discovered" by the student as they designed the cell to survive in some environment. Perhaps that'd make the purpose of each cell component more grounded and memorable.
Papert's book Mindstorms covers all this in detail, and I highly recommend reading it. I feel like it's especially important today given fears of how AI will affect childhood education. At best, I hope that computing can be a boon to education instead of a detriment if it's woven into pedagogy thoughtfully.
Education serves two purposes: Understanding and developing skills for employment. Unfortunately, there usually isn't enough time to teach both for all subjects. The latter usually takes priority.
I saw this happen twice in my university life. I took the honors version of CHEM101. 80% of it was more about exploring and discovery, and 20% was more traditional problems, calculations, etc.
For people like me who were not chemistry majors, this was great. For the chemistry majors - they really struggled when they got into CHEM102.
The same happened with Quantum Mechanics I. The professor didn't want to teach all the bra-ket notation, and more or less skipped much of the linear algebra aspects of QM - focusing on a purely calculus approach. He then retired. The professor who taught QM II saw what the professor did, and said "WTF?!" He spent most of the semester reteaching QM I.
As a result, those like me who went on to grad school had to retake QM II at the new university to make up for all that wasn't taught to me.
Completely off topic, but I really don't understand the demographic of physics educators that are resistant to Dirac notation in a first QM course. It isn't really that hard to explain or understand, it's extraordinarily convenient, and it's essential for engaging with most quantum adjacent literature. I understand that it conceals the distinction between vector space and dual space a bit but it's really not that hard to get through. I may be biased because I learned the notation in week 2 of intro quantum, so I am curious if anyone with the opposite experience disagrees.
I think he just felt the students wouldn't be able to handle the linear algebra aspects of QM. Strange notion, given that all the students were seniors, and had all taken a linear algebra course from the math department.
Probably wanted less formalism overall.
Like when I took an E&M course in the EE department, and the professor insisted on teaching only the integral forms and not the differential forms of the Maxwell's equations (i.e. no divergence, curl, etc in the whole course).
Skin in the game. Being "smart" isn't an immutable property of a noun. It's a verb. It's something you do, something you practice, constantly.
Even in fields like Math where ostensibly we have universal truths accessible to all, we still must learn how to derive proofs. Without doing math, one cannot understand math.
Learning IS experiential. It requires concentration and effort. Those who seek to make learning "easier" by removing effort are embarrassingly misguided about human cognition.
It's a open-source clone of Spore that aims to be more scientifically accurate. Right now you can play as microbes and very simple multicellular organisms.
This is intriguing. The two relevant features seem to be 1) realtime collaborative multiplayer conversations and 2) conversation-as-document - basically, letting you comment inline in an agent conversation.
For (1), the main value I'd see is in mentoring junior engineers or less technical contributors on a team. If someone puts up a PR with sloppy results, you could actually jump into the thread that produced that PR and see how the results came about, or even coach that contributor on how to do better next time. Also might make it easier to hand off work from one person to another - right now most coding agent sessions are user-local.
On (2), I frequently find myself consuming agents' gigantic text responses and tediously writing 8-bullet-point responses to guide them. It's pretty exhausting. I could see inline comments providing much better ergonomics.
All that being said, Zed has largely fallen out of the conversation for "agentic coding tools", and so this feels like their attempt at creating something like the Cursor Agents Window, Codex, or Claude Code. These two features seem compelling, and I understand they're even compatible with other coding harnesses. But I don't know if there's enough there to have a defensible product; if these features are excellent, others will clone them eventually.
> On (2), I do find myself consuming agents' gigantic text responses, writing 8-bullet-point responses to guide them, then iterating that way. It's pretty exhausting. I could see inline comments providing much better ergonomics.
For your 2 plannotator [1] has a nice way to annotate agent messages or documents and its integrated into the agent chat (codex app also adopted the pattern)
Same pain point on (2). So I had codex build me a small macos sytem tray app that lets me comment on anything i can select on my screen with the source app and metadata plus my comments. Once commenting is done i can click a button to coppy all comment to the chat input, and voila ! But really something i hope that gets integrated into the codex app itself. The present way they made it is not super helpful.
It's a solid experience so far (have been testing the alpha). Being able to work across model providers in parallel is my preference, rather than being stuck in Codex desktop, for example.
Well this is nice. Apparently I reached some limits (thanks all), and had to pay Cloudflare more. Fair I guess, although some warning would've been nice. Tried multiple payment options multiple times just now and Cloudflare botched every time without giving me an error message. Finally managed to get it through on the 10th time. Please be gentle now :/
That sucks... Bunny CDN served me and others great when it comes to a Cloudflare alternative, if you're looking for an alternative.
I understand the pragmatism with going with CF, but I'd lie if I didn't also say using CF as the front for your entire "European Digital Stack" kind of makes the blog-post feel less authentic compared to my initial impression, because of that.
Oh that's "funny". Where I work we used to pay for cloudflare, then the credit card expired and we didn't notice in time (our bad, for sure), now our account still works fine, just no premium features and all tickets we've made in order to _resume paying them_ have gone unanswered. Big shrug
I want point out the other aspect of that decision to use cloudflare because the content is already public.
If your users are in a sanctioned region or a sanctioned entity it is entirely possible for cloudflare to deny serving them traffic. In a way your website users are still bound to the US policies even if you or your country doesnt approve of those sanctions.
fwiw I've had my site on front page hn a couple of times. It is a completely unoptimized hetzner server running nginx and serving HTML.
computers are _fast_ these days, you're more likely to have an outage from cloudflare than by just skipping it IMO (for basic personal sites, like yours seems to be)
I think it's more likely they just use/abuse it than specifically created it, same as things like google spyware. The NSA's desires and the ad industry are aligned, so its a match made in heaven.
Have to wait for the next Snowden before you get any citations.
NSA collaborator or not, the mere existence of something like Cloudflare, which also tries to nudge you into skipping internal http/tls and just use that at the front, makes it highly likely that NSA is already deep in their infrastructure, just like they've been in the past for literally any big technology company in the US.
But yeah, zero citations, zero evidence, just based on history and what the goal of the organization is, it's pretty clear what's going on already.
Ok, that makes sense. "Likely, based on heuristics" is sufficient to inform your opinions and decisions about Cloudflare. I was hoping you might have something more concrete, but appreciate the thoughtful reply.
Tangent: I found your blog in your profile and liked your post on "Good Taste"^1
This is just categorically different and epistemically dishonest. It is, frankly, an embarrassing attempt to defend the fact that you don't actually have any evidence to support the claim that Cloudflare is supposedly an NSA creation beyond "believe me bro, ever heard of PRISM?"
> actually have any evidence to support the claim that Cloudflare is supposedly an NSA creation beyond
I never claimed so, never would either, so who's being dishonest now?
What I've said is that NSA compromises everything they can get their hands on (lots of evidence of this), assuming that Cloudflare aren't compromised by NSA already would be foolish, but you're right, this is an assumption on my part, I won't claim I have proof of this, just like I don't have proof that the sun will rise tomorrow, but I do assume so too.
Been tinkering on this Olmsted-inspired map of all the trees in NYC for a long time - just added some more features (seasonal colors and better search/filtering) recently.
I don't have much to say about this post other than to vigorously agree!
As an engineer who's full-stack and has frequently ended up doing product management, I think the main value I provide organizations is the ability to think holistically, from a product's core abstractions (the literal database schema), to how those are surfaced and interacted with by users, to how those are talked about by sales or marketing.
Clear and consistent thinking across these dimensions is what makes some products "mysteriously" outperform others in the long run.
It's one of the core ideas of Domain-Driven Design. In the early stage of the process, engineers should work closely with stakeholders to align on the set of terms (primitives as another commenter has put it), define them and put them in neat little contextual boxes.
If you get this part right, then everything else becomes and implementation effort. You're no longer fighting the system, you flow with it. Ideas becomes easier to brainstorm and the cost of changes is immediately visible.
DDD suggests continuous two-way integration between domain experts <-> engineers, to create a model that makes sense for both groups. Terminology enters the language from both groups so that everyone can speak to each other with more precision, leading to the benefits you stated.
We ended up with something like five microservices - that, in principle, could have been used by anyone else in the company to operate on the Domains they were supposed to represent and encapsulate. This one holds Users and User data! This one holds Products, and Product interaction data!
Nobody touched any of those except us, the engineers working on this one very specific product. We could have - should have - just put it all on one service, which would have also allowed us to trivially run database joins instead of having to have services constantly calling each other for data, stitching it together in code.
Subdomain shouldn't be engineering related. That's putting the cart before the horse. Subdomain is more like: This barely have anything to do with that, other than data transmission (not transformation).
How you implement it, however is an engineering question. Microservice is not the only abstraction tool that exists. It's kinda the worse. You have procedure/class, file/module, package/libraries, processes and IPC. Network call is for when you have no other choice.
Now how do you get your company / do yourself the hiring of those people in such a way that you can basically just have a team of people like this work with PMs to build their ideas?
I like doing this FS journey myself but am stuck "leading teams" of FS/BE/FE mixes and trying to get them to build stuff that I clearly understand and would be able to build with enough time but all I have is a team of FE or BE people or even FS people that can't just do the above. You need to be very involved with these people to get them to do this and it just doesn't scale.
I've recently tried AI (Claude specifically) and I feel like I can build things with Claude much quicker than with the FE/BE/FS people I have. Even including all the frustrations that Claude brings when I have to tell it that it's bullshitting me.
I have exactly the same experience as you. I tried educating people but all those developers (and beyond, up to stakeholders), no matter their seniority, do not want to get involved in the domain too much, just as little as they need. That naturally leads to me micromanaging all the things, leading to non scalability and finally overburn. As soon as I stop doing micro, all the stuff start to break down pretty fast. I wrote a book per project trying to get everyone on the same level but nah (more than 3000 pages in last decade, 20+ projects). Tried everything in hiring too, found almost nobody during all that time.
I am now off the previous work and will devote time to try AI, because I concluded it can't be worse than that.
Reading this thread brought back fond memories of sitting with front-line staff and just chatting with them while watching them work from the corner of my eye. My gimmick was to turn up for morning tea (the staff were older ladies that took homemade cakes to work), and by lunchtime have some frustration of theirs resolved.
It’s such a great feeling when you can make someone’s work better, for the life of me I can’t understand why others wouldn’t jump at the opportunity!
Sadly at current $dayjob, the devs are held at arm's length from the customer. On purpose!
Same here. No matter how hard I try, and use different approaches, from coaching, to sharing videos, through poiting out why this can benefit you personally, to showing how exactly it creates results, there simply is no interest. People don't care.
It's even worse than that - even the owner of the company I worked for didn't care that the product of his own company will be mediocre, while shouting generally the quality is the goal. It turns out that it was the goal as long as it was incidental and free (no such thing, but it looks that way if you are not deeply involved) and because it sounds good. As soon as reputation collides with the immediate profit, profit always wins.
Yuup. I do find that most of the time business decision makers actually have no clue about quality. Especially with software products if it looks like it works in the demo/looks pretty then the quality must be good right, and these engineers are just being pedantic, cause theyre engineers.
That’s something I relate too as well. I like working on different abstraction levels throughout the system.
Only way to cope was to let go things and pick my battles.
I always think about the joke where a sailor goes down to the dock and asks dock men if they speak French, English or German- dock men only shake their heads showing no. Later dock men chat and one saying to other he could learn languages so he would be able to talk with the sailor. The other replied that sailor knew 3 and it didn’t help him.
Everything is too recent, nobody can give a sure advice on how to deal with your situation. From my view as a fullstack engineer working with LLMs for the past 3 years, your generated product is probably crap and your only way to assess it is by asking Claude or ChatGPT if it's good, which it'll probably say yes to make you feel good.
Now go ahead and publish it. If your app brings revenue, then you build something quicker. A Claude-generated prototype is as much a product as some PowerPoint slides
Huh, my experience has been generally the opposite - most FS/BE/FE folks want to understand the business, and while a good PM will enhance that, the median PM is actively detrimental.
Frankly if the people you have aren't good enough then you need to get good at training, get better in your hiring (does your hiring process test the skills you want? Without being so long that anyone decent is going to get a better offer before they reach the end of it?), or maybe your company is just not offering enough to attract decent talent. There are plenty of better-than-AI programmers out there, but even in this job market they still have decent options.
Yes, and conversely, in cases when the initial model misjudged future needs, the most disastrous projects are those where the requirements or the technical design flies in the face of the original model. When this is solved sloppily, this often begins the slow degeneration - from an application that makes sense to a spaghetti mess that is only even navigable by people who were around when those weird bolt-ons happened. Usually not only the code, but also the UI reflects this, as even massive UI overhauls (like Atlassian's in 2025) tend to just sweep everything awkward under a rug -- those things are still necessary to manage the complexity but now they're hidden under ••• → Advanced Settings → Show More → All Settings.
I don't suppose you have any tips on how to get this going in an org? I love where I work and I love the products we make, but my team (phone apps) are treated very often like an afterthought; we just receive completed products from other teams, and have to "make them work." I don't think it's malicious on the part of the rest of the teams, we're just obviously quite a bit smaller and younger than the others, not to mention we had a large departure just as I arrived in the form of my former boss who was, I'll fully admit, far more competent in our products than I am.
I've worked on learning all I can and I have a much easier time participating in discussions now, however we still feel a bit silo'd off.
I find this perspective bizarre. Though I'm not happy about it all being centralized, the closest thing we have these days to the very niche phpBB forums of the 2000s is various subreddits focused on very specific topics. Scrolling through the front page is slop, sure, but whenever I'm looking for perspectives on a niche topic, searching for "<topic> reddit" is the first thing I do. And I know many people without any connection to the software industry who feel the same way.
Perhaps swearing at the LLM actually produces worse results?
Not sure if you’re being figurative, but if what you wrote in your first comment is indicative of the tone with which you prompt the LLM, then I’m not surprised you get terrible results. Swearing at the model doesn’t help it produce better code. The model isn’t going to be intimidated by you or worried about losing their job—which I bet your junior engineers are.
Ultimately, prompting LLMs is simply a matter of writing well. Some people seem to write prompts like flippant Slack messages, expecting the LLM to somehow have a dialogue with you to clarify your poorly-framed, half-assed requirement statements. That’s just not how they work. Specify what you actually want and they can execute on that. Why do you expect the LLM to read your mind and know the shape of nginx logs vs nginx-ingress logs? Why not provide an example in the prompt?
It’s odd—I go out of my way to “treat” the LLMs with respect, and find myself feeling an emotional reaction when others write to them with lots of negativity. Not sure what to make of that.
When do we blame the user? When the tool is operating as intended by its creator, and we agree the tool meets certain quality standards and isn't defective.
When do we blame the creator? When the device doesn't meet those quality standards and reasonable use caused harm inadvertently. For example, for consumer devices, certifications like UL/CE are used to define acceptable performance levels and safety standards.
Maybe we need "quality certifications" for AI agents - essentially eval suites that demonstrate those agents won't cause harm under reasonable patterns of usage. Right now, these eval suites are run best-effort by the labs themselves.
The tricky thing is, a lot (all?) of these recent safety incidents have occurred while evaluating these models! This suggests we need much more rigorous standards for how exactly an eval can be run. Perhaps all of them should occur in truly air-gapped environments... though that may run counter to evaluating agents in a realistic way.
Regardless, it feels like the "industry standards" common in, say, electrical engineering and other disciplines are sorely lacking here. Unsurprising given how new these technologies are, but concerning since the blast radius for this technology is likely much larger than other technologies we've encountered in the past, except maybe nuclear technology.
reply