LLMs are like slot machines, in that an incorrect answer (the slot machine eating your dollar) is unremarkable, while the LLM solving a problem (a jackpot) is amazing, and the latter stands out in your memory, causing you to overestimate the reliability of LLMs.

blog.glyph.im/2025/08/futzing-…

in reply to Cory Doctorow

I have argued that LLMs are like slot machines in another way: The business model. When you are paying by “the pull of the handle,” the vendor's incentive is not to solve your problem with a single pull, but to give the appearance of progress towards solving your problem.

Exactly the same as games companies that engineer their games to keep users buying coins, staring at the screen for eighteen hours a day. Very different model than selling tools or renting services.

This entry was edited (4 days ago)

Cory Doctorow reshared this.

in reply to Cory Doctorow

When the image generators came out, it reminded me of skilling up in video games. The way you can become an "expert" cook despite still being unable to nuke a pizza pocket irl without having different temperature zones. It felt like that and I could see how people would begin to see themselves as "artists" because it would feel like an accomplishment in a gamified way.

Or, well: twit.social/@JustinH/115010462…

This entry was edited (3 days ago)
in reply to Cory Doctorow

Recently at work, a manager pushing AI was presenting the "great stats" of AI use, and presented a "24% prompt acceptance rate" as if that didn't mean "76% prompt failure rate", aka, it fucks up the vast majority of the time. Even in these cases, of the 24% of prompts that were "accepted", there were no stats for how much additional work was needed to get the "accepted" result into an actually acceptable state.
in reply to Cory Doctorow

Better analogy - LLMs are like teenagers. They might know a couple facts through rote memorization, but they frequently make spurious extrapolations from their limited knowledge base and need frequent course-corrections from an experienced adult.

When the adults aren't steering the teenagers effectively, the teens tend to think maladaptive behavior is okay... just like LLMs.

Garbage in, garbage out is the same in both humans and machines.

in reply to Cory Doctorow

Related:

I was (ugh) using ChatGPT to see how much my students might be able to cheat on a set of quizzes I'm going to give them, and I ended up in so many words telling ChatGPT to "admit" that it's just spicy autocomplete. I had this feeling of accomplishment that I "got it" to say it.

And then I remembered: it *merely* tells me what I want to hear, just like someone who wants a cybergirlfriend, just like someone asking for stock tips, just like someone who wants it to be a god.