> It does make me wonder how much hidden knowledge is hidden in parts of the internet
That was the shining hope of the internet. "Information wants to be free" and now it could be available to anyone with a modem vs just in university or municipal libraries. Now, we have megasites that dominate the web's attention with very trivial and banal content that makes looking for that information nearly impossible.
Well, it's not as if that promise didn't come true. We do have the world's knowledge at our fingertips. It's more of a curation problem, and a natural result of a decentralized web. Wikipedia could be considered a "megasite", but I'd argue it's more valuable than a million independent sites.
Who knows what kind of weird mathematical or scientific things have been discovered by hobbyists and posted on some obscure forum thread or abandoned blog that wasn't appreciated by anyone?
Like, if someone had a proof for the Goldbach Conjecture, and for whatever reason posted about it on the WatchUSeek forums (just as a throwaway example), it could just lay dormant for who knows how long? The audience on WatchUSeek might not realize the significance of a such a discovery, and maybe not even the poster, so it doesn't get any recognition and just never gets picked up.
I mean, going with the example in the original post, I don't know much about Superpermutations (only a little after this post went viral), so if I had seen this 4chan post in the wild, I would just assume it's some guy doing a bit of math that I wasn't familiar with. I wouldn't be able to spot that this was actually a contribution to anything.
Is it better if there are lots of hidden gems tucked away on obscure websites? Actually I think that is a sort of interesting question. The world is a more interesting place if there are hidden bits I guess. And there will be some hidden pieces of info, so maybe it is good if there’s enough out there to incentivize exploration.
On the other hand, it is nice and convenient if everything is indexed.
It's kind of like that one scroll tucked away in some catacomb which nobody has looked at for hundreds of years that has the one bit of information that answers so many questions. The information is technically there, but you first need to know where to look. Discovery is always a problem.
Which is strange, because it’s rather trivial. You make your machine inside out. Now the entire universe is on the inside of it, so all energy it gets from external sources is technically on the inside.
(Some) people are also happy to pay to have someone piss on them, too. Do you think Microsoft should offer a Golden Shower tier of Copilot, complete with Trickle Down?
Oh, to be clear, there's a lot of bullshit on the internet, that might be the majority of stuff on the internet. Perpetual motion stuff is an example of bullshit.
Still, I do wonder what cool (and real) stuff on the internet that people don't really know about is actually significant.
One must hope that the spirit that compells malicious actors to archive private p*rn Discord servers on a publicly-accessible site eventually finds itself to people with access to technical servers. I get that everyone wants their secret hangout, but not everything needs to be a SCIF wannabe. Publish those logs.
That was the shining hope of the internet. "Information wants to be free" and now it could be available to anyone with a modem vs just in university or municipal libraries. Now, we have megasites that dominate the web's attention with very trivial and banal content that makes looking for that information nearly impossible.