I consider myself an advanced industry representative in these things and would also borrow some time for detailed investigation. I'd happily contribute some valuable assets (tacos with cheese dip) into the matter, straight out of my personal drawers.
So, it looks like this tries to build a model (or rather, multiple layers of models combined with something called context mixing[0]), potentially analyzing the whole file first? Is it fair to call this a very fancy entropy coder, replacing the last phase of what a traditional audio codec does (the lossless part, after quantization)?
I like the code style, this is pretty nice C. It's refreshing to see human code.
I admit I'm not a fan of the loop macros, or writing constants like 0xd80000 in decimal, or having all of that code inlined with no further explanation or git history.
But I think it's just that this file is lifting parts of the logic from PAQ8 (the inlined num_states/next_state named in a comment sure look exactly like PAQ functions translated into C, which, kudos to PAQ for having extensive comments in their code by the way).
I'm a bit disappointed that PAQ isn't mentioned in the README, and only very obliquely in the code, if that's where those functions come from. PAQ is GPLv3, at least a mention could have been nice (assuming I'm not wrong about the source of that logic).
I use the PAQ terminology as it is likely to be familiar to other compression experts. However, the lineage to PAQ is very, very limited, and many other compressors also re-use its common ideas.
They seem to say they'll accept the outcome and pledge to engage in good-faith. Which is the only correct response, really, so time will tell if these words are more than generic PR.
Anyone can edit Wikipedia, editors are not employees. Even the admins are not employees.
(As far as I know you only have to sign any kind of formal documents if you have one of the few roles (as a non-employee) where you may have access to sensitive personal data, which is very few people.)
Hasn't been true in a very long time. Plenty of politically charged articles are locked so that only Wikipedia's editors can edit them, and you can guess which side they're taking. It's not a democracy.
Anyone can be a Wikipedia editor, assuming you haven't done something egregious and gotten a lifetime ban.
It takes no time at all to set up an editor account and you don't even have to divulge your real identity to do so. The Wikimedia Foundation appears to go to great lengths to defend their editors and has gone to court to defend their editors' rights to pseudo-anonymity.
Note: I am a former employee of Wikimedia and everything I saw while I worked there supports that the vast majority of staff, including management, care a great deal about protecting editors, promoting free knowledge, and making the project as open and inclusive as possible.
>Anyone can be a Wikipedia editor, assuming you haven't done something egregious and gotten a lifetime ban.
Firstly, not all articles are open to editing by gen-pop. See: Extended confirmed protection on many.
Secondly, some pages that are hot on the political side can then become subject to broader Wikipedia contentious topics / discretionary sanctions rules, so even eligible editors who qualified for extended confirmed protection edits, must fall in line with the approved narrative or risk sanctions and have their edits reverted to what the WMF staff admins and Arbitration Committee or ArbCom decide who gets to have the objectively correct viewpoint in certain cases and who doesn't.
>The Wikimedia Foundation appears to go to great lengths to defend their editors and has gone to court to defend their editors' rights to pseudo-anonymity.
OK, but why are you bringing this up? I never claimed otherwise. All I said wikipedia is very selective with who gets to edit some topics and what its admins and arbcom consider as "valid sources" and which not, leading to entrenched biases on political topics. Wikipedia defending editors is a good thing but an offtopic derailment addition to what I was originally talking about which had nothing to do with this.
Could you give an example of an article that was protected for reasons other than preventing vandalism? I've seen other people bring this up, but never with evidence.
You're entitled to no protection, nor are we entitled to you providing an example. But if you don't want to provide an example, your claim immediately turns to dust. This is just the fundamental core of rationalism.
The sky is green. I won't be taking any questions.
(actually the sky really was green a few days ago, related to tornado weather!)
> But if you don't want to provide an example, your claim immediately turns to dust.
That's not how it works in reality. If people disagree with the example it's dust anyway, except now you also wasted your time.
My opinion is irrelevant here since if people have made up their minds that WIkipedia is not biased and takes sides, then no example will suffice and they simple ask for an example just to have a target to shoot at.
I've been on this rodeo before. There's no guarantee of a good faith argument when politics are involved. I have no guarantee I'm not dealing with people arguing in bad faith trying to bait for a flame war, so I'd rather not even start to save time and sanity.
If you think it's dust, then so be it, I don't mind, I don't lose anything from this, nor do I gain anything from trying to convince you I'm right. I have my opinion, you have yours and we can leave it at that.
Even looking only at different cultures, saying nothing of nature, the degree to which people are willing to cooperate or defect against the group varies widely. The level of trust varies widely.
People aren't motivated solely by the risk of being caught. Some people tell themselves that everyone else must be trying to cheat, so it's okay for them to cheat. But this type of thinking isn't a law of nature, it's just a human flaw.
The amount of cheating is not a fundamental constant. Given that people are social animals and that we tend to be stronger as a group, in fact you'd expect that prosocial behavior is good for the species. If fucked up is to mean anything, poisoning the environment with forever chemicals ought to make the fucking cut, if you'll pardon my French.
I was never really blockchain-something. It was all the hype when I finished my software engineering masters (around 2017/18), so to understand the technology better I started a Podcast and wrote my master thesis about it. I never used it or worked in it afterwards though and I never thought it'd become more than a scam unfortunately
The homepage says something about AI guided fuzzing based on libfuzzer or AFL. Looks like they have the LLMs identify a bunch of interesting functions to test, generate some test harnesses, and then sort through the fuzzer findings at a high level, which sounds like a pretty good idea.
Thanks for figuring that out. Sort of sounds like AI programming programs to find vulnerabilities, of which fuzzing is one of the proven techniques to do it.
Back when I first heard of it, the HuggingFace hack reminded me about how you don't have to be particularly smart to find vulns, just aggressive in looking for them. AFL on its own "learned" how to construct well-formed JPEGs and PDFs by fuzzing decoders for these formats, back in the pre-LLM dark ages.
Of course the HuggingFace hack proved to be much more than that...
Setting a swarm of agents loose for hours to look for software vulnerabilities is far more compute-expensive than fuzzing. The industry has never thrown this kind of compute resources at pure fuzzing, in part because you can't get much VC money for that.
You also quickly get into diminishing returns with fuzzing. Generally a bug is either going to be found relatively quickly with a given fuzzing approach or it's going to be nearly impossible to find. You're usually better coming up with more intelligent fuzzing approaches than you are just dumping compute into it.
This sounds like a swarm of agents with particular prompting that happens to guide the LLMs toward doing a lot of fuzzing, so it's not either/or; you're getting all the compute requirements of both.
Fuzzing or having the LLM sort through where might be most useful to fuzz & sorting the results? Neither seem particularly compute intensive to me, fuzzing is a pretty standard step and having the LLM read through to find the most interesting areas to fuzz sounds a lot more efficient than leaving the whole task to the LLM.
GNU ld is indeed extremely slow, it is a very old codebase that isn't designed for modern systems, and it doesn't do any real work in parallel. GNU ld can be 2-10x slower than LLVM lld on real projects, and then mold is again ~5-10x faster that lld. The creator of mold is also the original creator of LLVM lld.
Traditionally the link time was a single-threaded step at the end of compilation, and it scaled very poorly. While compilation is embarrassingly parallel and has continued to scale. So link time was becoming a significant bottleneck, especially with incremental compilation. When you want to iterate on a large project, with GNU ld you might have to wait several minutes of link time on every dev build.
"...it doesn't do any real work in parallel." I think this is the biggest selling point.
Although, "with GNU ld you might have to wait several minutes of link time on every dev build." Meh, I might prefer to wait a few minutes and stick with something I know works.
"The creator of mold is also the original creator of LLVM lld" That's cool, why not implement these features in LLVM lld? Why not implement them in GNU ld? Is it a gatekeeping thing? Parallel ld would be very useful.
No, it's not a gatekeeping thing. You're coming at this with negative assumptions, so I don't think I can try to convince you. If you hate Rust and want something conservative, then use lld. It has some parallelism, but it's fairly coarse and still has single-threaded bottlenecks. It would have needed a pretty major re-architecture to modernize it.
As for trying to do that in BFD, patches welcome. Or maybe not welcome, I'm not sure the maintainers are really looking for a major redesign of GNU ld. But either way I'm not volunteering, it's much easier to start fresh than try to carve out and replace the entire guts of this old thing.
"You're coming at this with negative assumptions" not so much negative as skeptical. Everyone is pushing Rust and full rewrites, I'm still not convinced.
"But either way I'm not volunteering, it's much easier to start fresh than try to carve out and replace the entire guts of this old thing."
The absolute number of lifetime downloads can of course only go up (but that's even more vacuous).
reply