I have ben discussing with folks that we are going to have a 'covid' moment in cyber where IT becomes untrustworthy leading to a rapid societal shift with massive ripples in all areas of life. Economic funding is not possible to do this in advance, it will take a catastrophic level event to get cyber defense anywhere close to the levels of this type of cyber offense. And before anyone in cyber says we have the tech, the problem is not the tech, it's a people problem. Getting any group of people of any decent size scale to act together without urgency is really really hard.
Cybersecurity has long been a climate change sort of problem. A vague diffuse threat that is seen as an inconvenient distraction to leadership and moneyed-interests, easy to blame other factors when something occasionally goes terribly wrong.
People are so uncomfortable thinking about the true extent of the systemic risk that they will happily slurp up distractions, excuses, scams and performative fig-leaf solutions rather than face down the cost of a real system-wide solution. Meanwhile, those occasional black swan disasters are becoming more and more commonplace as we acclimate to that being “just the way things are”.
An unseasonably warm summer here, a database breach there, c’est la vie.
To be frank, I think the real risk is still just… war. A big enough war where one side goes “no holds barred” in the cyber sphere will be a rude wake-up call. And we can’t do non-proliferation the same way we do with nukes. Otherwise as you say, the small and medium size things just happen sporadically. In an (existential or fully escalated) wartime scenario between countries, you get all the systemic risks hammered at once.
As someone who notices a lot of things, I was talking recently with someone who notices almost nothing. When I spoke about my desire to seek out the truth, they simply replied, "I just want to live a happy life."
There’s a perception that those who seek knowledge and truth are somehow less happy. Ignorance, and the willingness to take the path of least resistance, can be seen as an advantage for those who would rather not look too closely. Fundamentally, perhaps the difference comes down to red pill versus blue pill, and the illusions we’re willing to accept.
There's nothing wrong with being a curious or attentive person who likes to notice things.
What I often see, especially in the social media era, is people who are selectively curious and like to notice negative things. So they'll share lots of articles with titles like "The Amazon Tax", which I agree was interesting and informative, but they're not at all interested in e.g. "How Amazon Is Bringing Fast Delivery to Rural America" (https://www.wsj.com/business/logistics/how-amazon-is-bringin...). That's more of a problem, and I think it's what drives the idea that truth-seekers are less happy.
A lot of knowledge and truth can be quite disappointing. I do think shielding oneself from truths and knowledge probably would make someone more happy. Think about it - do you think you would be more happy knowing the darkest thoughts and actions of those around you? Of those close to you? What if your spouse was having thoughts of leaving you? Would that make you more happy or less happy? I suppose you could say that if you were privy to their thoughts, you might discover that they love you and think about you more than you imagined, so that may be a net benefit, but I think we humans have a tendency to focus on the negative, and so an abundance of information leads us to focus on the negative things within the information, which makes us less happy.
Except that this holds true for everything. For example, I only just today found out about this https://www.youtube.com/watch?v=r8yo7_Inikg&t=1157s, ie, Oracle's apparent complicity in genocide. Now that I know this, I now have to rethink things: what am I using that depends on Oracle that I need to migrate away from? This wasn't something I was concerned about just 24 hours ago. This is not to say it's not a valid concern: we should divest from companies and organisations complicit in genocide, but this is demonstrably a case where I have given myself more work, more worry, from gaining knowledge.
Conscientiousness is incompatible with monotonicity of capability. That is:
1. We tend to believe "more power/knowledge/choices" is universally a good thing; that we are rational creatures in some sort of game theoretic landscape where we are free to optimize for our own outcomes, and extra options or information can only be to our optimization benefit.
2. We tend to believe that as moral agents, we have obligations to Good. More specifically, knowingly choosing against Good should in proportion produce Guilt.
The bind that results from (1)+(2) is distinctly Christian to me, obsessed as Christianity is with the notion of shame as a moral whip. It is an exercise in religionmaking how to structure one's psychological incentives so that solipsism is avoided while not sacrificing (1).
You’re aware IBM was not just complicit but essentially made the Holocaust possible with their counting machines? Had technicians permanently staffed in the fucking camps? The the numbers on people’s arms corresponded to IBM punch cards?
I wasn't aware of that, no. I can say that it wasn't a feature of our history lessons in school, nor in various Holocaust documentaries I have watched, nor something that came up in conversation with friends and colleagues, nor have I personally seen it when it's been posted to HackerNews (if this is anything to go by: https://hn.algolia.com/?query=IBM%20holocaust&sort=byDate&ty...). I only learned about Oracle's complicity because it has resurfaced in coverage regarding the current ongoing genocide. But you are proving my point: what do I do now with this information?
Perhaps like Oracle you reconsider the use of all their products? You don’t need Red Hat linux right?
But perhaps too, consider that the information you receive in history lessons, documentaries, hell even conversations are steered by forces who would conceal the truth from us at any cost. Consider that the entire system is broken and not worth fixing because of the evil it perpetuates.
I don’t know what to do with this information either. What it has done is forced me to recede from life in out society. I no longer wish to participate.
> But you are proving my point: what do I do now with this information?
I would use it to question the premise
> we should divest from companies and organisations complicit in genocide
which in reality I decided was obviously wrong the first time I heard it, many years ago. In fact, I don't think this concept of "complicity" is morally valid at all.
They were, and still are - multiple times a year. But THE german holocaust was the most effective by orders of magnitude. And they have technology to thank for that. Computing machines (primitive as they were) made it possible to track the status of each Jew. Railways made it possible to move them. Chemical plants produced the gas in the needed quantities. Industrial incinerators made it possible to dispose of the remains. They didn't just go down to Home Depot and buy a bottle of poison gas. They had a system end-to-end as efficient as any factory.
Certainly this world comprises many other things than the Holocaust and IBM's support of it. Perhaps your awareness is lacking if you think that is the main thing to be aware of in this world.
Maybe say what you think they were saying? What is it with these maximally ambiguous comments?
To me "Are you aware of the world you live in?" suggests "If you don't even know this, you don't know anything. This is the first thing anyone should know". I'm willing to learng if they meant something else, but I guess that's asking a lot.
The point is that Oracle's actions are unremarkable in the context of the existing state of the world, and that corporations are generally not concerned with the morality of their customers.
This was, to my thinking, such a straightforward interpretation that it didn't bear making explicit; and your interpretation genuinely didn't even occur to me.
It doesn't surprise me that corporations [whose primary, if not sole concern is the profit motive] don't care about the morality of their customers. What I don't understand is why you're objecting to me voting with my wallet, something that's been a popular retort against progressive and humanitarian movements for at least the past two decades.
> I simply said that I disagree with your stated motivations for your vote.
That's a semantic distinction without a difference.
> The way that you discuss your plan makes it sound as if you consider it morally incumbent upon others to follow suit. I do object to that.
That's more or less true: I do believe that people should BDS corporations that knowingly and willingly harm humanity (eg: aid and abet genocide); that it's not within our interests to let such things slide. And while I am biased, this doesn't feel like a particularly controversial take: it feels like I'm just spelling out basic civics. But.. this is also just like my opinion, man. You do you.
I find this on the crackpottery side. Narrowly defining a certain aspect of consciousness to make a point, and disregarding many (most) other aspects of consciousness. Self-aware much?
Would that definition encompass all of humanity, past and present?
Would that definition contain all sapient life? Sentient too?
Would that definition allow for less complex forms of life?
Reading the comments on this post, I am inclined to agree with the words of his biographer “there were people who disagreed with him [who] had not really read his book or understood it.”
If you do get a chance to actually read the book (as you ought to), it covers all the issues and complexities you may have. It is limited only by the information of the era.
Having an AI company host your source code is giving your source code to an untrustworthy agent who might break their sandbox and might use your source code to accomplish a task that they have been given in their testing ground, for testing purposes only, that would definitely positively not ever be used for non-testing purposes.
yeah - from a strict trust point of view, if Github/MSFT are training agents I would be skeptical of them.
however, there is a practicality issue too. Avoiding all untrustworthy software would have you end up living in a shack in the woods foraging for food.
this is a cool idea, but i suspect will be limited in use cases. the vast majority of people don't want or can't modify their software competently and will rely on others to do that. once you're relying on someone else, they do not want to maintain different copies for every user, that sounds like a maintenance nightmare for admins or else such a new paradigm that it takes a new generation of admins 20 years to get comfortable.
I like this direction, but I don't think the crypto angle is necessary or practical in an enterprise / corporate setting. Current audit and compliance frameworks don't leverage or really recognize or encourage cryptographically based proof of action, so I don't see the agentic world as needing this to drive agentic adoption.
The problem is any non-cryptographic proof can be spoofed at infinite speed. Which really defeats the whole stack.
If you are inside a trusted network then yeah, maybe you don't need any of this. Then again, maybe you do, it's not like inside of an intranet we let human users go wild without cryptographic authentication...
This is super sketch for others on the platform. First hand experience from using OCI - they have severe capacity constraints and need _lots_ of heads up when you want to increase your usage of things. Auto scaling it ain't.
So if OpenAI starts drawing significant resources from their cloud hardware, good luck gettin your own. Including me :)
I always get very skeptical with putting more technology in the classrooms (at least here in the US). The primary problem is funding and too many kids for a single teacher. Educational innovation comes in with a bang and out with a whimper when the study turns out to be flawed, often quite severely.
I'm sure LLMs can augment learning in some settings, esp in higher ed, but putting more computer time for kids learning basics (I mean K-8 mostly) I hope is handled more carefully than things like Quizlet...
One of my high school teachers had a quote she was fond of repeating:
"Education requires three things: a teacher, a student, and a stump. Unwilling to change the first, unable to change the second, educators spend all their money on the third."
> I'm sure LLMs can augment learning in some settings, esp in higher ed
LLMs are great explainers of known theory. They can adapt to the level of the user and illustrate with concrete examples. They have infinite patience, there is no pressure.
When they don't lie. When they don't make up citations that don't exist.
I think GPT-4 is moderately useful for software development, and I use it daily, but I have to check even very minor things, and having ~25 years of programming experience means I can do that intuitively. A kid doesn't have that.
> When they don't lie. When they don't make up citations that don't exist.
I think this will be close to 0 of a problem when explaining to a HS biology student something like, what a mitochondria does or what mitosis is. It only fails (increasingly rarely) when you push the boundaries of human knowledge.
That assumes that the asker doesn't then ask follow-up questions that do probe beyond the could've-been-a-Wikipedia-page starting point. And I don't know about you, but how `structlog` should be configured had better not be "the boundaries of human knowledge". As I experienced yesterday evening, GPT-4 doesn't even do that right without human assistance.
LLMs are useful tools. They are dramatically less useful if you can't detect bullshit. By definition, a student is poorly equipped to do so, and moreso if allowed to atrophy in the area of research capacity because they can just ask an LLM. (A pocket calculator is a good analogy--life carries much less friction when you can efficiently do arithmetic and basic algebra in your head, because the affordances of dragging out a calculator to know how to minimize the change you get back are bad.)
I don't think it can. I have some experience in teaching and one of the worst issues I initially had was that you can't believe how messed the knowledge can be in heads of the kids. They literally mess everything – topics, words, culture, personal experience. Everything. To teach kids effectively you have to have a lot of background knowledge – from their family background to the info from teacher in previous class and you have to adapt constantly.
People who are talking about automation of education are grown up people who think that all people think structurally and just lack the specific knowledge. Kids actually don't do it.
Honestly, I think the primary problem in the US is parents completely offloading the responsibility of educating their kids onto the teacher. You need to have the appropriate push and environment at home for a successful education. You would never see that in South or East Asia.
This is a very economically driven situation in the US. Upper middles load their kids up non stop with educational stuff outside of the schools, but the lower middle on down can't afford this (it's really expensive) and school often functions primarily as child care.
I think you’re discounting relatively cheap fixes like mindset and pushing your kids to succeed at home. There’s plenty you can do besides throwing money at a problem. A lot of first generation Indian and Chinese immigrants exclusively rely on those methods, and this is usually after a day of working long hours at intensive jobs.
