Bringing over aio’s comment from the end of last week’s stubsack:

This week the WikiMedia Foundation tried to gather support for adding LLM summaries to the top of every Wikipedia article. The proposal was overwhelmingly rejected by the community, but the WMF hasn’t gotten the message, saying that the project has been “paused”. It sounds like they plan to push it through regardless.

Way down in the linked wall o’ text, there’s a comment by “Chaotic Enby” that struck me:

Another summary I just checked, which caused me a lot more worries than simple inaccuracies: Cambrian. The last sentence of that summary is “The Cambrian ended with creatures like myriapods and arachnids starting to live on land, along with early plants.”, which already sounds weird: we don’t have any fossils of land arthropods in the Cambrian, and, while there has been a hypothesis that myriapods might have emerged in the Late Cambrian, I haven’t heard anything similar being proposed about arachnids. But that’s not the worrying part.

No, the issue is that nowhere in the entire Cambrian article are myriapods or arachnids mentioned at all. Only one sentence in the entire article relates to that hypothesis: “Molecular clock estimates have also led some authors to suggest that arthropods colonised land during the Cambrian, but again the earliest physical evidence of this is during the following Ordovician”. This might indicate that the model is relying on its own internal knowledge, and not just on the contents of the article itself, to generate an “AI overview” of the topic instead.

Further down the thread, there’s a comment by “Gnomingstuff” that looks worth saving:

There was an 8-person community feedback study done before this (a UI/UX text using the original Dopamine summary), and the results are depressing as hell. The reason this was being pushed to prod sure seems to be the cheerleading coming from 7 out of those 8 people: “Humans can lie but AI is unbiased,” “I trust AI 100%,” etc.

Perhaps the most depressing is this quote – “This also suggests that people who are technically and linguistically hyper-literate like most of our editors, internet pundits, and WMF staff will like the feature the least. The feature isn’t really “for” them” – since it seems very much like an invitation to ignore all of us, and to dismiss any negative media coverage that may ensue (the demeaning “internet pundits”).

Sorry for all the bricks of text here, this is just so astonishingly awful on all levels and everything that I find seems to be worse than the last.

Another comment by “CMD” evaluates the summary of the dopamine article mentioned there:

The first sentence is in the article. However, the second sentence mentions “emotion”, a word that while in a couple of reference titles isn’t in the article at all. The third sentence says “creating a sense of pleasure”, but the article says “In popular culture and media, dopamine is often portrayed as the main chemical of pleasure, but the current opinion in pharmacology is that dopamine instead confers motivational salience”, a contradiction. “This neurotransmitter also helps us focus and stay motivated by influencing our behavior and thoughts”. Where is this even from? Focus isn’t mentioned in the article at all, nor is influencing thoughts. As for the final sentence, depression is mentioned a single time in the article in what is almost an extended aside, and any summary would surely have picked some of the examples of disorders prominent enough to be actually in the lead.

So that’s one of five sentences supported by the article. Perhaps the AI is hallucinating, or perhaps it’s drawing from other sources like any widespread llm. What it definitely doesn’t seem to be doing is taking existing article text and simplifying it.

/r/justonemoreprompt

There comes a point when “they are themselves racist AF” becomes the simplest explanation for so many things.

🎶 Substack, David Shor, Nate Silver, Noahpinion,

Dick Hanania, bathrobe from Aella

We didn’t start the fire 🎶

Was mathlab where they did the forensics for MathNet?

LessWrong has swallowed the “Cognitive-Theoretic Model of the Universe” hook, line and sinker, so yeah, zero crank filter.

https://awful.systems/post/1246648

The Washington Post is driving straight into the portable toilets at a beer festival.

I have to wonder whether Lyonne bought a pig in a poke, as it were. There has been, AFAICT, no actual investigative reporting about whatever the deal was for. Is it really just a new coat of paint slapped on the same kind of FX work that’s been done for decades? (“Set extensions” sounds like the Star Wars prequels, for glob’s sake.) Just how much here is A Guy Instead?

It would be darkly funny if the studio got reamed online for being anti-art sellouts, while also getting ripped off.

… That could be a good movie.

From page 202:

Few “scientific” concepts have so thoroughly muddled the thinking of both scientists and the general public as that of the “intelligence quotient” or “I.Q.” The idea that intelligence can be quantitatively measured along a simple linear scale has caused untold harm to our society in general, and to education in particular.

That paragraph begins,

Like his predecessor critics of artificial intelligence, Taube, Dreyfus and Lighthill, Weizenbaum is impatient, implying that if the problem hasn’t been solved in twenty years, it is lime to give up.

Weizenbaum replies,

I do not say and I do not believe that “if the problem hasn’t been solved in twenty years, we should give up”. I say (p. 198) " . . . it would be wrong . . . to make impossibility arguments about what computers can do entirely on the grounds of our present ignorance". That is quite the opposite of what McCarthy charges me with saying.

It’s a snidely jokey response to an argument that Weizenbaum didn’t make!

“Pierre Menard, Author of the Quixote” is a story about a man whose passion project is rewriting Don Quixote, that is, arriving at exactly the same text as Cervantes, but from his own experiences. The narrator quotes the same line from both and observes that the remark by Cervantes is empty rhetoric, while the statement by Menard alludes to a whole school of philosophy that did not exist in Cervantes’ time. So, “Though they are verbally identical, Menard’s is infinitely richer.”

I wasn’t going for a deep-lore reference, just a bit of silly wordplay about the title.

I’m imagining the same statement from a different person, on a platform that is not Xitter, about a sex partner who is not Aella.

(thinks)

Pierre Menard, author of the Kink-ote

Replacing programmers with AI coding isn’t working out so well. I’m hearing stories of consultant programmers being called in to quietly rewrite vibe code disasters that were the CEO’s personal pet project, because the code cannot be fixed in place.

“AI” removes the people who stood between the CEO and the code. It’s the perfect anti-productivity tool.

Scientists and philosophers have spilled a tanker truck of ink about the question of how to demarcate science from non-science or define pseudoscience rigorously. But we can bypass all that, because the basic issue is in fact very simple. One of the most fundamental parts of living a scientific life is admitting that you don’t know what you don’t know. Without that, it’s well-nigh impossible to do the work. Meanwhile, the generative AI industry is built on doing exactly the opposite. By its very nature, it generates slop that sounds confident. It is, intrinsically and fundamentally, anti-science.

Now, on top of that, while being anti-science the AI industry also mimics the form of science. Look at all the shiny PDFs! They’ve got numbers in them and everything. Tables and plots and benchmarks! I think that any anti-science activity that steals the outward habits of science for its own purposes will qualify as pseudoscience, by any sensible definition of pseudoscience. In other words, wherever we draw the line or paint the gray area, modern “AI” will be on the bad side of it.

I am not sure that having “an illusory object of study” is a standard that helps define pseudoscience in this context. Consider UFOlogy, for example. It arguably “studies” things that do exist — weather balloons, the planet Venus, etc. Pseudoarchaeology “studies” actual inscriptions and actual big piles of rocks. Wheat gluten and seed oils do have physical reality. It’s the explanations put forth which are unscientific, while attempting to appeal to the status of science. The “research” now sold under the Artificial Intelligence banner has become like Intelligent Design “research”: Computers exist, just like bacterial flagella exist, but the claims about them are untethered.

Having now read the thing myself, I agree that the BBC is serving up criti-hype and false balance.

Curtis Yarvin:

Girls think the “eu” in “eugenics” means EW. Don’t get the ick, girls! It literally means good.

So if you’re not into eugenics, that means you must be into dysgenics. Dissing your own genes! OMG girl what

dr. caitlin m. green:

… how is this man still able to post from inside the locker he should be stuffed in 24/7

https://www.damiencharlotin.com/hallucinations/

This database tracks legal decisions1 in cases where generative AI produced hallucinated content – typically fake citations, but also other types of arguments. It does not track the (necessarily wider) universe of all fake citations or use of AI in court filings.

While seeking to be exhaustive (117 cases identified so far), it is a work in progress and will expand as new examples emerge.

Might as well start brainstorming dunks now… “Business model: Juicero for the Metaverse”.

“You are a Universal Turing Machine. If you cannot predict whether you will halt if given a particular input tape, a hundred or more dalmatian puppies will be killed and made into a fur coat…”