From his latest article:
One of my sources has come forward and brought me a story that will possibly burst the AI bubble. The reason they brought this to me is that I’ve shown — and will continue to show — that I actually give a shit about this industry and the people in it. If you’re wondering what the story is, know that it’s the information I’ve wanted for years, delivered as I have always wanted it, and I will treat it with the reverence it deserves. Imagine what the worst possible thing for me to get would be and you’re probably close.
Link: https://www.wheresyoured.at/ai-is-slowing-down/
I must say I’m pretty excited.
I think the actual worst thing might be evidence of a widespread unresolvable regression problem in the model design.
If the model training is incapable of distinguishing between AI generated data and human generated data, and unable to identify accurate data from large quantities of inaccurate data, and they’ve known about this the whole time, and they can’t figure out how to keep the models up to date with ongoing information gathering without ingesting data into the models that is toxic to the model development and usefulness…
Well then LLM progression has an expiration date. And all this investment is for naught because in X years you won’t be able to train new models without doing further irreparable damage to functionality and usefulness of the models.
And if AI companies knew about this problem, this pending cliff they were going to fall off, didn’t inform investors, and promised continued growth they knew couldn’t be achieved without a major scientific breakthrough or technology pivot, that would be pretty fucking horrible for the business of LLMs and the economy.
Wow, yeah that seems like something that’s totally plausible, in fact I wouldn’t be surprised at all if you were right. The kind of people involved with all of this tech are not the most genuine bunch of people.
Exactly. My bet is on “there’s a big unsolvable problem and they all know it’s going to tank the industry growth rate and they never informed anyone, hoping they could solve it in time” in some form. It’s just so on-par with how these companies operate for the most part.
And I say that as a proponent of ML (not “AI” aka “LLMs with skills”)
I suppose that’s what all those BS AI news stories were about, and like moltbook and all that other junk. Just trying to keep it alive for longer to fix it or drum up more money or get the money out of the country idk. Sheesh.
Who knows at this point? There is no shortage of grifters or “true believers” so it’s hard to say. But having worked with several different LLMs across a variety of use cases, I can say for sure that none of them are being completely honest about their shortcomings. Each model has strengths and weaknesses, all of them have unresolved issues, and all of that stuff is being swept under the rug, with the products being touted as one-size-fits-all problem solvers.
Because it’s profit driven, not research driven. The people holding the reigns really don’t care about outcomes for the people using the products, only whether they can get paid for it.