In effect the source code is being copied by the LLM. This is what it's designed to do. LLMs are a lossy statistical compression of their training data.
If you give it a prompt telling it to replicate a product that's in its training set then its optimal next token prediction output is going to be to a lossy copy of that product's source code.
Whatever LLM they used copied the source code. It took their prompt and filled in the blanks of the spec by copying from the closest matching open source project. This is just what a next token predictor is going to do if you tell it to replicate software that's in its training set since that is the optimal way to predict the next tokens.
Not popping doesn't seem possible to me, since all investment has been directed towards it, massive construction projects have been started, etc... The revenue needed to keep OpenAI and Anthropic afloat is insane and there's no way businesses are going to start paying the real price when there's no ROI compared to human expertise.
AGI is fundamentally impossible through data scaling like they tried to claim and achieving AGI is what all this depends on. The long tail problem will remain undefeated and the IPOs are a desperate move to get the cash needed to scale one last time.
They could have just kept improving the technology without all the psychosis and finding use cases and ways to make it more reliable but instead they bet everything on a language model becoming their slave god.
Slowly deflating would be nice, but I don't see how. In any case the economy is getting wrecked and any goodwill tech companies had with employees is gone after going completely adversarial towards them as soon as they had an opportunity to. The most profitable use case of gen-AI is still spams and scams.
I wasted 2k on a color e-ink monitor that ended up being pretty much unusable. Reviews said as much but the risk was worth it to me for the chance to spend my days looking at something that doesn't feel like a screen. I believe it's an "if you build it they will come" thing especially for anyone working on a computer all day.
I don't use these tools, but wouldn't it be better to use them only after you do a manual review to see if they find anything you missed? Otherwise I could see reviewers getting false confidence and doing a less thorough review. This happens with seeing that unit tests pass.
One of the funniest things is how hard it was to get approval for a $100 software license but now people are being encouraged to burn thousands on tokens.
> "In its IPO filing, the company had said Cursor's access to developers' data, including coding requests and design decisions, could help improve its AI models such as Grok."
They're all stealing your IP and selling it back to your competitors in the form of tokens.
> "I want LLMs to code for me, but I want them to be trained on other people's code, not mine, duh".
Who ever said that? Have you actually heard that from your fellow programmers in real life?
If the code I wrote actually made even the slightest discernible difference in LLMs I'd be so honored. But it won't happen, as it's just 0.00001% of all the training data.
Tragedy of the Commons is not an analogy, it is an inevitable result of a large fraction of participants in a coordination game defecting due to perceived individual advantage.
Sounds good? They can pay for code they want to train on. There are plenty of companies sending me offers to code training materials for them for $50-100/hr. Don’t expect to charge me an arm and a leg for inference and then also train on my code.
There are already opt out buttons for training in Cursor and Claude Code… if you don’t want it then turn it off. If it was worth enough money to them they would offer a monetary incentive like discounts but none of them have yet
Interesting how our generation which grew up using Napster now has so many intellectual property extremists. By this logic, even humming a tune you heard on the radio is theft.
> I have absolutely zero interest in free. I honestly don't think I'm even remotely in the same demographic as people using free tiers / models. I want to pay. I don't want my data used for training...
They want to use LLMs trained on others code but don't want to contribute with their own.
It makes sense from a business perspective-SaaS firms value the ability of coding agents to accelerate development, but also worry the models will learn the secret sauce of their business and destroy its moat. So their desire to contractually exclude training on their data has some logic to it.
(Disclaimer: Not speaking for or about my current employer, just a general industry observation.)
I don't really use LLMs myself, but if someone wants to have any kind of software business then having the models trained on their products isn't ideal.
Cursor users are willfully providing it by using their product. Not unlike uploading a personal photo or video to social media -- that's not yours anymore. You gave all rights away when you put it on their servers.
> Not unlike uploading a personal photo or video to social media -- that's not yours anymore. You gave all rights away when you put it on their servers.
99% of users are unaware of that, so it’s false to say they "are willfully providing it".
> Not unlike uploading a personal photo or video to social media
I've granted them a limited license to use it.
> that's not yours anymore.
Not by any definition in the contract or in law is this true.
> You gave all rights away when you put it on their servers.
I gave away some rights. I also got something in return. Attention. And at the end of the day I'm completely entitled to turn around and sell copies of this work for profit. The only thing I can't do is sell an /exclusive/ license because that is no longer available.
None of this provides any implication for people who upload code to their own websites. Which these rapacious LLMs bots happily index, sometimes to the extent they actually crush the site, or create unusual costs for the owner.
Finally none of these LLM companies tell you where the source came from. Whether it is copyrighted, whether ownership rights are retained, or whether the code can be used publicly or not, and if so, which license it's covered by.
You're using the lens of social media contracts to understand something far larger and more important. It's lead you to some bizarre conclusions and huge oversites.
What if someone steals my work and then uploads it to facebook and claims it as their own? Do the rights no longer exist because it got uploaded to Meta?
Do you think that everyone using it and their employers are aware that they are giving their competition the ability to copy straight from their codebase when they ask it to replicate their product?
presumably they are subject to audits. i work at a tech company that has to deal with data compliance and we take it incredibly seriously and have automated systems tracking, tagging, and deleting data everywhere.
I work at a 20k-employee company with footprint in tech. A couple of weeks ago a decision was made to not renew Cursor license. This week it was made public and has been greeted with groans. Some employees have been publicly saying how ineffective and unproductive they are going to become due to this decision- which actually strikes me as really naive.
Personally I like to use a stable IDE. I used Cursor for a couple of days and then went back to VS Code, largely due to Cursor pushing agentic first approach with V3 update.
Indeed, according to Cursor leaderboard there were folks committing 200k-300k lines of code every month. And here I am intentionally slowing myself down manually approving every AI generated code change and terminal command.
> They're all stealing your IP and selling it back to your competitors in the form of tokens.
The users of those tools are stealing too. The model is trained on free software licensed under specific terms and the output of a prompt will strip those licenses and their terms.
It's not theft in the same way a con artist convinces you to give him all your money. The people using it or their employers just don't realize any competitor will be able to ask the LLM to replicate their product and it will copy the codebase they uploaded to them.
You could just test it for yourself if you're actually interested in finding out (assuming you are a decent programmer, but your bio suggests you have no technical expertise in the area).
First I never made any claims about myself or appealed to my authority, that was you. I still don't see any evidence of any technical expertise in ML or in programming anything more complicated than python scripts. Maybe you weren't aware but often on the internet people make false claims about their expertise on something. There's thousands of "Visionary leader driving ethical AI innovation and strategic growth" CEOs that know next to nothing about the actual technology.
Anyway like I said you can just test it out yourself and find out that I'm correct. Every skilled programmer already knows this and can predict what kind of complexity an LLM won't be able to handle. And anyone working on LLMs should know that they are completely dependent on their training data. The entire scaling hypothesis was based on this.
There's open source ML&DL projects dating back 12 years on my github, I don't know what you're on about.
Regardless, your claim is "LLMs being able to one-shot any higher-order complexity is entirely dependent on it already being in the training data." which is currently unfalsifiable, and known by every AI researcher to be most likely wrong. You've also said it's "been demonstrated". So stop wasting people's time and link to the demonstration instead of hand-waving a "do it yourself you're smart".
Good lord, it's like talking to a bloody climate change denier, I swear.
I have neural net applications I programmed entirely myself from 16 years ago and all I see from you are some python scripts so I guess I win this stupid game I wasn't even trying to play.
You haven't made a single technical point outside your bloviated claims to authority. I can safely assume you're a fraud because this is exactly how frauds speak. Actual scientists and engineers don't argue from authority they go and test the hypothesis for themselves, the fact that you balk at my suggestion to do this is amusing.
I said you can do a basic test because this is the best way to see it directly for yourself. It's very easy to do especially for an eminent machine learning visionary leader as yourself. You ask the LLM to produce two apps of similar complexity and technical challenge from a software perspective, one that is already in its training data and one that isn't and see which its more successful at. This isn't some controversial take, nor is it "unfalsifiable".
There's also hundreds of benchmarks demonstrating where the limitations are for LLMs, or you can study the progression of LLMs in mathematics and where the gains have been made and see that this also agrees with me. You can watch Chris Hay's videos demonstrating exactly how LLMs perform math layer by layer. Why is everyone using LLMs for search? Because it's an extremely efficient compression of all its training data. Did they figure out the Studio Ghibli art-style all on their own spontaneausly? No, they were trained on Studio Ghibli content. There's so many ways to come to this conclusion. But you seem to be too busy sniffing your own farts to be interested in learning anything about the field though.
This pointless test you're suggesting is a few countries removed from your initial suggestion, which was as a reminder: "The people using it or their employers just don't realize any competitor will be able to ask the LLM to replicate their product and it will copy the codebase they uploaded to them" (which, yes, is unfalsifiable given the search space)
I mean, are you seriously trying to back off that original ridiculous claim into a "code in the training data is more likely to appear in the output than code that isn't"? And I'm the fraud? As I said before, get over yourself.
Yes a test showing an LLM can reproduce an app in its training data and not an equivalent complexity app that is not in its training data is equivalent to proving the statement I made that having your codebase trained on will allow a competitor to copy your product.
That's certainly one way to do it, but where would you place them?
No, but seriously, you could imagine what we witnessed playing out in a high stakes Tom Clancy or Michael Crichton style fable.
The fiery blowhard Pentagon chief, the arrogant know it all tech bro lab head, an alarm being called in from a remote office and surfaced through Amazon.
This seems to be a good middle ground then. It allows for a way to prevent political projects getting grants under the guise of "scientific research", at least when they directly oppose the voters. I don't see any push to defund basic research, and if politicians start doing that there's at least a way for people to voice their disapproval through voting.
Aside from that, so much money was wasted on Alzheimer's research based on fraud.
If you give it a prompt telling it to replicate a product that's in its training set then its optimal next token prediction output is going to be to a lossy copy of that product's source code.