We have identified how millions of concepts are represented inside Claude Sonnet, one of our deployed large language models. This is the first ever detailed look inside a modern, production-grade large language model.

I often see a lot of people with outdated understanding of modern LLMs.

This is probably the best interpretability research to date, by the leading interpretability research team.

It’s worth a read if you want a peek behind the curtain on modern models.

  • Spedwell@lemmy.worldEnglish
    51·
    7 months ago

    concepts embedded in them

    internal model

    You used both phrases in this thread, but those are two very different things. It’s a stretch to say this research supports the latter.

    Yes, LLMs are still next-token generators. That is a descriptive statement about how they operate. They just have embedded knowledge that allows them to generate sometimes meaningful text.