• naevaTheRat@lemmy.dbzer0.com
    link
    fedilink
    English
    arrow-up
    24
    arrow-down
    2
    ·
    6 months ago

    Yeah this is the exact criticism. They recombine language pieces without really doing language. The end result looks like language, but it lacks any of the important characteristics of language such as meaning and intention.

    If I say “Two plus two is four” I am communicating my belief about mathematics.

    If an llm emits “two plus two is four” it is outputting a stochastically selected series of tokens linked by probabilities derived from training data. If the statement is true or false then that is accidental.

    Hence, stochastic parrot.

    • Ignotum@lemmy.world
      link
      fedilink
      English
      arrow-up
      1
      arrow-down
      18
      ·
      6 months ago

      If i train an LLM to do math, for the training data i generate a+b=cstatements, never showing it the same one twice.

      It would be pointless for it to “memorize” every single question and answer it gets since it would never see that question again. The only way it would be able to generate correct answers would be if it gained a concept of what numbers are, and how the add operation operates on them to create a new number.
      Rather than memorizing and parroting it would have to actually understand it in order to generate responses.

      It’s called generalization, it’s why large amounts of data is required (if you show the same data again and again then memorizing becomes a viable strategy)

      If I say “Two plus two is four” I am communicating my belief about mathematics.

      Seems like a pointless distinction, you were told it so you believe it to be the case? Why can’t we say the LLM outputs what it believes is the correct answer? You’re both just making some statement based on your prior experiences which may or may not be true

      • naevaTheRat@lemmy.dbzer0.com
        link
        fedilink
        English
        arrow-up
        17
        ·
        6 months ago

        You’re arguing against a position I didn’t put forward. Also

        Seems like a pointless distinction, you were told it so you believe it to be the case? Why can’t we say the LLM outputs what it believes is the correct answer? You’re both just making some statement based on your prior experiences which may or may not be true

        This is what excessive reduction does to a mfer. That is just such a hysterically absurd take.

          • naevaTheRat@lemmy.dbzer0.com
            link
            fedilink
            English
            arrow-up
            10
            ·
            6 months ago

            I’m a curmudgeonly physics nerd, it’s very strange being on the side of a debate going “No now come on, that’s way too reductive”

        • Ignotum@lemmy.world
          link
          fedilink
          English
          arrow-up
          2
          arrow-down
          8
          ·
          6 months ago

          The AI builds some kind of a model of the world in order to better understand the input and improve the output prediction,

          You have some mental model of how maths work which you have built up through school and other experiences in your life,

          You both are given a maths problem, you both give an answer based on your understanding of mathematics

          • naevaTheRat@lemmy.dbzer0.com
            link
            fedilink
            English
            arrow-up
            8
            arrow-down
            1
            ·
            edit-2
            6 months ago

            The algorithm assigns weights to nodes in a neural network. These weights are derived by statistical association of tokens in the training data after they have been cleaned.

            That is so enormously far from how we think humans learn (you don’t teach a kid to understand theory of mind by plopping them in front of the Gutenberg project and saying good luck, and yet they learn to explain theory of mind problems all the same) that it is just comically farcial to assume something similar is happening underneath.

            It is very interesting that llms are able to appear to be conversational but claiming they have some sort of mind with an understanding of maths is as ridiculous as suggesting a chess bot understands the Pauli exclusion principle because it never moves two pieces into the same physical space.

          • yuri@pawb.social
            link
            fedilink
            English
            arrow-up
            7
            ·
            6 months ago

            You’ve been speaking with your chest this whole time and now that we’re into the nitty gritty you really just said “The ai does… something!” It’s so general a description that by your measure automated thermostats are engaging in human reasoning when they make it a little bit cooler on a hot day.

            You might’ve been oversimplifying on purpose. I just can’t help but think you have no idea how LLMs work outside of this inherently flawed comparison to human thought.

            • Hackworth@lemmy.world
              link
              fedilink
              English
              arrow-up
              2
              arrow-down
              1
              ·
              6 months ago

              Not OP, but speaking from a fairly deep layman understanding of how LLMs work - all anyone really knows is that capabilities of fundamentally higher orders (like deception, which requires theory of mind) emerged by simply training larger networks. Since we don’t have a great understanding of how our own intelligence emerges from our wetware, we’re only guessing.

              • yuri@pawb.social
                link
                fedilink
                English
                arrow-up
                4
                ·
                6 months ago

                Something that looks like higher order reasoning emerged from training larger networks. At the end of the day it’s still just spicy autocomplete. Theoretically you could give it a large enough dataset to “predict” almost anything with really high accuracy, but all it’s doing is pattern recognition. One could argue that that’s all humans do, but that’s getting more into philosophy and skipping a lot of nuance.

                I’m not like, trying to argue with you by the way. Just having a fun time with this line of thought ^^

                • Hackworth@lemmy.world
                  link
                  fedilink
                  English
                  arrow-up
                  2
                  arrow-down
                  1
                  ·
                  edit-2
                  6 months ago

                  What makes the “spicy autocomplete” perspective incomplete is also what makes LLMs work. The “Attention is All You Need” paper that introduced attention transformers describes a type of self-awareness necessary to predict the next word. In the process of writing the next word of an essay, it navigates a 22,000-dimensional semantic space, And the similarity to the way humans experience language is more than philosophical - the advancements in LLMs have sparked a bunch of new research in neurology.

      • Kogasa@programming.dev
        link
        fedilink
        English
        arrow-up
        9
        arrow-down
        1
        ·
        6 months ago

        If you fine tune a LLM on math equations, odds are it won’t actually learn how to reliably solve novel problems. Just the same as it won’t become a subject matter expert on any topic, but it’s a lot harder to write simple math that “looks, but is not, correct” than it is to waffle vaguely about a topic. The idea of a LLM creating a robust model of the semantics of the text it’s trained on is, at face value, plausible; it just doesn’t seem to actually happen in practice.

        • Ignotum@lemmy.world
          link
          fedilink
          English
          arrow-up
          1
          arrow-down
          4
          ·
          6 months ago

          Prompt:

          What is 183649+72961?

          ChatGPT:

          The sum of 183649 and 72961 is 256610.

          It’s trained to generate what is most plausible, but with math, the only plausible response is the correct answer (assuming it has been trained on data where that has been the case)

          • Kogasa@programming.dev
            link
            fedilink
            English
            arrow-up
            5
            arrow-down
            1
            ·
            6 months ago

            ChatGPT uses auxiliary models to perform certain tasks like basic math and programming. Your explanation about plausibility is simply wrong.

            • Ignotum@lemmy.world
              link
              fedilink
              English
              arrow-up
              1
              arrow-down
              3
              ·
              6 months ago

              It has access to a python interpreter and can use that to do math, but it shows you that this is happening, and it did not when i asked it.

              I asked it to do another operation, this time specifying i wanted it to use an external tool, and it did

              You have access to a dictionary, that doesn’t prove you’re incapable of spelling simple words on your own, like goddamn people what’s with the hate boners for ai around here

              • Kogasa@programming.dev
                link
                fedilink
                English
                arrow-up
                5
                arrow-down
                1
                ·
                6 months ago

                It has access to a python interpreter and can use that to do math, but it shows you that this is happening, and it did not when i asked it.

                That’s not what I meant.

                You have access to a dictionary, that doesn’t prove you’re incapable of spelling simple words on your own, like goddamn people what’s with the hate boners for ai around here

                ??? You just don’t understand the difference between a LLM and a chat application using many different tools.