OC below by @[email protected]
What called my attention is that assessments of AI are becoming polarized and somewhat a matter of belief.
Some people firmly believe LLMs are helpful. But programming is a logical task and LLMs can’t think - only generate statistically plausible patterns.
The author of the article explains that this creates the same psychological hazards like astrology or tarot cards, psychological traps that have been exploited by psychics for centuries - and even very intelligent people can fall prey to these.
Finally what should cause alarm is that on top that LLMs can’t think, but people behave as if they do, there is no objective scientifically sound examination whether AI models can create any working software faster. Given that there are multi-billion dollar investments, and there was more than enough time to carry through controlled experiments, this should raise loud alarm bells.
Ah still rolling out the old “stochastic parrot” nonsense I see.
Anyway on to the actual article… I was hoping it wouldn’t make these basic mistakes:
Yes we do. Frankly if you’ve used it it’s so obviously better than regular JavaScript you probably don’t need more evidence (it’s like looking for “evidence” that film stars are more attractive than average people). But anyway we do have great papers like this one.
Anyway that’s slightly beside the point. I think the article is right that smart people are not invulnerable to manipulation or falling for “obviously” stupid ideas. I know plenty of very smart religious people for example.
However I think using this to dismiss LLMs is dumb, in the same way that his dismissal of Typescript is. LLMs aren’t homeopathy or religion.
I have used LLMs to get some work done and… guess what, it did the work! Do I trust it to do everything? Obviously not. But sometimes I don’t need perfect code. For example recently I asked it to create an example SystemVerilog file for me utilising as many syntax features as possible (testing an auto-formatter). It did a pretty good job. Saved some time. What psychological hazard have I fallen for exactly?
Overall, B-. Interesting ideas but flawed logic.
Ah still rolling out the old “computers think” pseudo-science.
Ah yes the old pointless vague anecdote.
Promoting pseudo-science.
Overall D. Neither interesting nor new nor useful.