- cross-posted to:
- [email protected]
- cross-posted to:
- [email protected]
You must log in or register to comment.
I am a professional academic linguist, and make no mistake, whether you are a Chomskyan or a Functionalist, Papa Noam (as we used to call him in grad school) is one of the most important figures in the field for the last 100 years. Perhaps the most important. I wish him well and will be sad when he eventually passes.
That’s to say nothing of his political endeavors.
If I may ask, what do you do as a professional academic linguist? Sounds like a interesting and niche field.
The more common definition of “linguist” is effectively “translator”, i.e. someone who is a native speaker of language X but can also speak language Y. That’s also the military definition.
In terms of the study of linguistics, in academia you can have a great many “linguists” who are not translators but are versed in the science of linguistics and can e.g. do grammatical analysis. It’s an entirely different skillset from “translator”, and in fact, one doesn’t need to speak language Y to do it.
So, mostly I’m differentiating myself from the translator-type linguist and saying I’m the linguistics-type linguist. And because I also do that for a living, I added professional.
Yeah bit what exactly does that entail? Studying over old books and figure out how the language has changed?
There are a great many languages which are undocumented entirely or are severely lacking in documentation. One part of my job is collecting data for such languages. Another part is more traditional computational linguistics, which in my case is primarily corpus analysis (still a relatively common step in the development of model training data).
Think of linguistics as “programming language but for languages”.
Is it ones and zeros, or is it more human readable? The differences between those, on an academic level is linguistics.
There is a full wiki article that probably gives a better impression of it than I can.
https://en.wikipedia.org/wiki/Linguistics_of_Noam_Chomsky
It’s really weird and interesting!
You sound like a cunning linguist
Are you also a master debater?
so who do we email now?
Will be forever beholden to his work on language and grammar. Made it bearable to sit through a pointless CompSci class.
I thought his language theories were disproven?
Yup.
Noam Chomsky set the field of linguistics and cognitive science back 50 years by proposing that there was a module in the brain specifically devoted to language, and that all grammar is innately stored in the human brain at birth. His theories were widely accepted by the linguistics community because of his constant evangelism, but he had no evidence whatsoever beyond “humans are special.” His theories have been time and again proven wrong by evolutionary biologists.
Even in modern times, he continued to push false pseudoscientific theories in artificial intelligence, a field where he had no experience. He claimed that AIs and animals can never become conscious because they can’t have the special magical consciousness module that humans have, again with no evidence. Similar arguments were used by phrenologists to justify the abuse of Blacks during the slave era, and are still used today to justify the abuse of animals on factory farms.
I’m sure he’s done some good in the world, but I don’t think his contributions to linguistics were one of them.
Chomsky is considered one of the founders of cognitive science. He was the only person who was able to argue away Skinner’s conceptualization of language. Were it not for him, behaviorism may still have been dominant.
Sure, his review of Skinner’s work was very popular, and it did change the field. But replacing one wrong theory with another wrong theory without evidence is not an improvement, and he continued to push his incorrect theory even after clear evidence to the contrary. He let his ego get in the way of the scientific process and held the field back as a result.
I urge anyone downvoting these comments to refute Steve. Downvoting without honest criticism is just petty
Steve sounds like your typical Redditor posting about a famous person who has made monumental strides in academia. I’m guessing Steve also has ready-to-paste arguments for anyone who says that Stephen Hawking was intelligent.
This is shocking to hear. I’m also extremely spooked that no experts have even attempted to dispute these claims. Was he really just some pseudo-academic celebrity? Is he merely the OG Jordan Peterson?
Haha, don’t worry, he was definitely not some kind of Jordan Peterson. JP intentionally abuses his popularity to spread hatred.
I don’t think Chomsky was as intentional about it. He just had opinions and happened to talk about them in a reassuring, intellectual tone. It’s more of a systemic issue, in that other people spread his ideas without thinking scientifically about them. I think it’s not healthy for the field that people idolize him so much. It makes it harder for linguistics to move toward the scientific method.
Hope this helps clarify.
I think there is also something structural about academia, at least culturally, that supports the kind of monolithic views of how things work, even when they’ve never been shown to be demonstrated by the evidence. Linguistics is far from the only field where I’ve seen this scenario play out.
Absolutely not. Your description of his theory is wrong. His argument is that humans innately possess universal grammar. That is not at all the same as “innately storing all grammar at birth.” It was obvious to him and anyone else at the time (and long before) that a kid born in America doesn’t spontaneously start speaking Swahili without first learning it. So what is universal grammar then? It’s the innate capacity for grammar itself.
Secondly, Chomsky did in fact present evidence for his theory. His most compelling evidence is summarized by the Poverty of the Stimulus argument. That is to say, human children begin to speak and understand their first language extremely rapidly given very few examples, mostly from listening to and interacting with their parents. More recently, given our experiences attempting to build AI in the form of Large Language Models, we’ve witnessed just how incredibly tiny the “training set” for human babies really is, compared to the sheer enormity of the text they feed into something like ChatGPT (a significant portion of all the text ever written in English).
This observation is what originally refuted Skinnerian behaviourist theories of language development. Skinner essentially tried to say that humans learn language by positive and negative reinforcement, the way we train a dog to do tricks. But that idea is obviously false to anyone who witnesses a child rapidly begin speaking. Language is compositional and infinitely productive, so to be able to acquire it so quickly implies that the brain is preconfigured or structured in a way that innately understands these features. That is Chomsky’s theory.
I mean, while I disagree with his thoughts on animal consciousness, which have been disproven in clinical settings (mirror-test), AI will obviously, at least not anytime soon, be conscious. We’re not even sure how consciousness works, although we are sure that we are not sure consciousness works. Without that knowledge, I posit AI will never become conscious.
Your position is:
We don’t know what it is, but AI certainly won’t have it?
Does not seem rigorous.
Lol. Lmao even.
Imagine thinking so rigidly. Plenty of things were known of before it was known how exactly they work. My position is:
We know something is causing this, but our understanding of it is currently infantile.
QED
The Chomsky hierarchy is still taught in computer science. That’s as much as I know of Chomsky. I don’t know anything about his philosophy or natural language contributions
I wont forgive him for torturing me with his pointless UG shit.
Aww man poor guy. I will never forget when I showed up on the Ali G show lol
It doesn’t sound like there’s necessarily been any recent change in his condition—their ultimate source is a month-old Reddit post from his former assistant, who was relaying information she’d heard from his family a month prior to that.
A few months is still recent for people not following his health closely.
True, but even the info from the family member was just confirming that his situation hadn’t improved since the “medical event” he suffered a full year ago.
Very sad to see our mortality as we watch Chomsky slip away. Let’s celebrate his life and his accomplishments - our eventual deterioration and demise is all a part of our grand story of life. He’s done amazing things with his time here and we’d all be lucky to have 95 great years like that.
Let’s celebrate his life and his accomplishments
I’m a big fan of his work in the field of genocide denial and atrocity apologetics, truly inspirational.
You changed my life Mister Chomsky. Thank you
Is he an idol for young people, but then later in life not any more? Politically?
Oh hey, I saw something by accident that I can contribute here: https://www.youtube.com/watch?v=KWKQIqzotLQ
It’s a video response from Chomsky’s current collaborators telling off these journalists for announcing a private health matter to the public & making it harder for Chomsky & his family & emphasising that even now into his 90s he is doing cutting-edge much-discussed intellectual work and that is the real news.
Nonononono
Saw his Manufacturing Consent documentary and it is pretty good
We’re all lucky he said as much as he did while he could speak.
Such a towering figure in the field of linguistics, politics and philosophy of psychology. Especially his linguistics work was something I found personally extremely engaging as a student. Reading his review of B.F. Skinner’s book was a highlight of my philosophy of psychology class.
I know he has been active since the turn of the century but his influence was definitely more of a 20th century phenomenon. Not many academics reach the level of popular awareness that he did.
For you yongsters out there, Chomsky vs Skinner was the original Kendrick Lamar vs Drake.
Chomsky vs Skinner was the original Kendrick Lamar vs Drake.
It’s extra ironic because their argument was over language acquisition and neither Lamar nor Drake can read.
What? This comes off super weird and bigoted because they definitely both can read.
Lamar and Drake are barely sentient narcissists. Don’t worry, no danger of offending them since they can’t read.
Who is Drake?
The guy from the meme.
Contemporary Phenomenologist with radical views about sexual liberation. Basically a latter day Foucault.
Ahh, I love fowcallt.
Such an interesting and weird life to have lived. Academically, he lived to see almost his entire body of work thoroughly disrupted by LLMs, specifically the concept of an universal grammar/ and the idea of an ‘innate language acquisition device’. I was in a live stream with him about 2 months or so after the first beta models from open AI had started to poke their way into the main-stream. It felt kind of sad, because its like, he was obviously very long in the tooth even then, but when your entire academic career is based on like “one thing”, as so many scientists and philosophers careers are, when that ‘one thing’ ends up being demonstrably false, it seems kind of… soul crushing? I saw it once before when the revised genomic classification of plants was being released and one of my professors (who i think was like, 1-2 years out from retirement), watched his life time body of work of taxonomic classification get “yeah nah dawged” by the revised genomic taxonomy. I’ll say he was not the most engaged instructor and actually insisted on teaching us wrong, which was super aggravating.
I still think Chomskys political work stands apart as a life well lived.
Universal grammar wasn’t his “entire body of work”—for the last 25 years he’s been at the forefront of the Minimalist program which is in no way affected by LLMs. And Chomsky’s criticism of LLMs as a model of the human brain’s language process is perfectly valid: while they can imitate human languages, they can also imitate artificial languages humans would never invent and can’t learn fluently. They’re like Ptolemy’s epicycles, that could accurately replicate the motions of the planets but could also imitate arbitrary orbits that violate Newtonian mechanics.
He has also written extensively about political ideologies, economics, and other issues. Linguistics is far from the entirety of his academic accomplishments.
Scrolled through every comment looking for this. Thanks comrade!
If you think that LLM’s prove anything of the sort, that they have even a smidgen of the implications for linguistics (or honestly any relevancy at all), you’re just completely confused.
Human beings do not learn language the way LLMs learn language. This shouldn’t need to be said but here we are, I guess. Fucking hate this timeline.
I’m not saying this from a defending LLMs point, I genuinely think they are a waste of so many things in this world and I can’t wait for this hype cycle to be over.
However, there is a lot of research and backing behind statistical learning in language acquisition, this is specifically the research subject of my friends. It’s a very big thing in intervention for delays in language.
It is opposed to Chomsky’s innate language theory, which at this point I think almost any linguist or language/speech sciences researcher would tell you isn’t a well accepted theory (at least as a holistic explanation, certainly it could still be true to an extent and a part of other systems).
tl;Dr LLMs are stupid, but it’s not broadly true that the way they “learn language” is entirely different from how humans do. The real difference is that they fail to actually learn anything even when imitating humans.
Be that as it may, we aren’t getting any answers from LLM’s. And given that Universal Grammar was the dominant view for so long, the jury is still out on a viable alternative.
Thank you for saying it. That really was a depressingly incurious comment.
Chomsky on the topic: https://www.nytimes.com/2023/03/08/opinion/noam-chomsky-chatgpt-ai.html
This would be true if chomskys claim was that he was simply studying human language acquisition and that machines are different, but his claim was that machines can’t learn human languages because they don’t have some intuitive innate grammar.
Saying an llm hasn’t learned language becomes harder and harder the more you talk to it and the more it starts walking like a duck and quacking like a duck. To make that claim you’ll need some evidence to counter the demonstrable understanding the llm displays. Chomsky in his nytimes response just gives his own unprovable theories on innate grammar and some examples of questions llms “can’t answer” but if you actually ask any modern llm they answer them fine.
You can define “learning” and “understanding” in a way that excludes llms but you’ll end up relying upon unprovable abstract theories until you can come up with an example of a question/prompt that any human would answer correctly and llms won’t to demonstrate that difference. I have yet to see any such examples. There’s plenty of evidence of them hallucinating when they reach the edge of their understanding, but that is something humans do as well.
Chomsky is still a very important figure and his work on politics with manufacturing consent is just as relevant as when it was written over 20 years ago. His work on language though is on shaky grounds and llms have made it even shakier.
Forget about defining learning. Just look at what “language” means in the dictionary. There are definitions that put an emphasis on using language to communicate thoughts or feelings, which LLMs don’t do.
Which definition of language is Chomsky referring to?
I’m not trying to be picky, but I think the answer might help determine whether or not LLMs are proving Chomsky wrong (which he might be, I don’t know!)
Having read your comment, I’m genuinely curious if you believe what you’re saying. Why make easily falsifiable claims? What’s the upside of lying about this obscure topic that you know nothing about? Is it a hobby?
I guess I can’t figure out your motive.
You are like, science denial-ism level of ignorance when it comes to this conversation, or perhaps, it might be that you don’t actually understand the underlying philosophy of scientific inquiry to understand why LLMs basically were able to break the back of both UH and innate acquisition.
You seem like the kind of person who cheer-leads “in the spirit of science” but doesn’t actually engage in it as a philosophical enterprise. You didn’t seem to notice the key point that @[email protected] made, which is that unlike UG, LLM’s are actually testable. That’s the whole thing right there, and if you don’t get the difference, that’s fine, but it speaks to your level of understanding to how one actually goes about conducting scientific inquiry.
And if you want to talk about incurious:
You might as well say that the longer you stare at a printed painting, the harder it is to deny that printers make art. LLM’s do not “understand” their outputs or their inputs. If we feed them nonsense, they output nonsense. There’s no underlying semantics whatsoever. LLM’s are a mathematical model.
Specifically “There’s no underlying semantics whatsoever” is the key lynch pin that UG demands that LLM’s demonstrate are not strictly necessary. Its exactly why Chomskys house of cards crumbles with the the counter-factual to UG/ Innate acquisition that LLM’s offer. I had a chance to ask him this question directly about 6 months prior to that op-ed being published. And he gave a response that’s about as incurious about why LLM, and basically, big complex networks in general, are able to learn as you’ve offered here. His was response was basically the same regurgitation on UG and innate acquisition that he offers in the op-ed. And the key point is that yes, LLM’s are just a big bucket of linear algebra; but they represent an actually testable instrument for learning how a language might be learned. This is the most striking part of Chomskys response and I found it particularly galling.
And it is interesting that yes, if you feed
LLM’stransformers (I’m going to start using the right term here: transformers) unstructured garbage, you get unstructured garbage out. However, if there is something there to learn, they seem to be at least some what effective at finding it. But that occurs in non-language based systems as well, including image transformers, transformers being used to predict series data like temperature or stock prices, even even DNA and RNA sequences. We’re probably going to be having transformers capable of translating animal vocalizations like whale and dolphin songs. If you have structured series data, it seems like transformers are effective at learning patterns and generating coherent responses.Here’s the thing. Chomsky UG represented a monolith in the world of language, language acquisition and learning, and frankly, was an actual barrier to progress in the entire domain, because we now have a counter factual where learning occurs and neither UG or innate acquisition are necessary or at all relevant. Its a complete collapse of the ideas, but its about as close as we’ll get because at least in one case of language acquisition, they are completely irrelevant.
And honestly, if you can’t handle criticism of ideas in the sciences, you don’t belong in the domain. Breaking other peoples ideas is fundamental the process, and its problematic when people assume you need some alternative in place to break someone elses work.
the key lynch pin that UG demands that LLM’s demonstrate are not strictly necessary
You know what, I’m going to be patient. Let’s syllogize your argument so everyone can get on the same page, shall we.
- LLM’s have various properties.
- ???
- Therefore, the UG hypothesis is wrong.
This argument is not valid, because it’s missing at least one premise. Once you come up with a valid argument, we can discuss its soundness and the plausibility of its premises. Until then, I can’t actually respond, because you haven’t said anything substantive.
The mainstream opinion in linguistics is that LLM’s are mostly irrelevant. If you believe otherwise — for instance, that LLM’s can offer insight into some abstract UG hypothesis about developmental neurobiology — explain why, and maybe publish your theory for peer review.
You don’t need to put project a false argument onto what I was saying.
Chomsky’s basic arguments:
1: UG requires understanding the semantic roles of words and phrases to map syntactic structures onto semantic structures.
2: UG posits certain principles of grammar are universal, and that syntactic and semantic representation is required as meaning changes with structure. The result is semantic universals - basic meanings that appear across all languages.
3: Semantic bootstrapping is then invoked to explain where children using their understanding of semantic categorizes to learning syntactic structures of language.
LLM’s torpedo all of this as totally unnecessary as fundamental to language acquisition, because they offer at least one example where none of the above need to be invoked. LLM’s have no innate understanding of language; its just pattern recognition and association. In UG semantics is intrinsically linked to syntactic structure. In this way, semantics are learned indirectly through exposure, rather than through an innate framework. LLM’s show that a UG and all of its complexity is totally unnecessary in at least one case of demonstrated language acquisition. That’s huge. Its beyond huge. It gives us a testable, falsifiable path forwards that UG didn’t.
The mainstream opinion in linguistics is that LLM’s are mostly irrelevant.
Largely, because Chomsky. To invoke Planck’s principle: Science advances one funeral at a time. Linguistics will finally be able to evolve past the rut its been in, and we now have real technical tools to do the kind of testable, reproducible, quantitative analysis at scale. We’re going to see more change in what we understand about language over the next five years than we’ve learned in the previous fifty. We didn’t have anything other than baby humans prior to now to study the properties of language acquisition. Language acquisition in humans is now a subset of the domain because we can actually talk about and study language acquisition outside of the context of humans. In a few more years, linguistics won’t look at-all like it did 4 years ago. If departments don’t adapt to this new paradigm, they’ll become like all those now laughable geography departments that didn’t adapt to the satellite revolution of the 1970s. Funny little backwaters of outdated modes of thinking the world has passed by. LLM’s for the study of language acquisition is like the invention of the microscope, and Chomsky completely missed the boat because it wasn’t his boat.
Your conclusion (which I assume is implied) might be something like,
- Mathematical models built on text data many times greater than the entirety of Wikipedia (which is 13GB) do a good job simulating human conversations (so, passing the Turing test), THEREFORE, homo sapiens lack an innate capacity for language (i.e., the UG Hypothesis is fundamentally mistaken).
My issue is that I just don’t see how to draw this conclusion from your premises. If you were to reformulate your premises into a valid argument structure, we can discuss them and find some common ground.
what an awful thing to write .
Tell us, what do you plan to do in life that someone won’t do better in future ?
I’d say his language theories had already been sufficiently challenged way before the advent of LLMs. But it’s hard to deny that they helped to lay the groundwork for what we know about linguistics today, even if they weren’t proven out in the end.
Two of my favorite Chomsky moments are his interview with Brian McGee and his debate with Foucault.
