A first-principles introduction to RLHF, what it changes in language models, and how it became part of modern post-training.
How linguistics learned to stop worrying and love the language models
Language models (LMs) can produce fluent, grammatical text. Nonetheless, some maintain that language models don’t really learn language and also, even if they did, that would not be informative for the study of human learning and processing. On the other side, there have been claims that the success of LMs obviates the need for studying linguistic theory and structure. We argue that both extremes are wrong. LMs can contribute to fundamental questions about linguistic structure, language processing, and learning. They force us to rethink arguments and ways of thinking that have been foundational in linguistics. While they do not replace linguistic structure and theory, they serve as model systems and working proofs of concept for gradient, usage-based approaches to language. We offer an optimistic take on the relationship between language models and linguistics.
Andrej Karpathy on Twitter / X
We'll be spending a lot more time trying to understand the outputs of language models. A few thoughts, tips & tricks:Writing. Something I've had success with: Ask your LLM to explain something in ASD-STE100, it's a controlled language specification originally developed for… pic.twitter.com/uMFWTcBCsl— Andrej Karpathy (@karpathy) October 2, 2026
Self-Instruct: Aligning Language Models with Self-Generated Instructions
Large "instruction-tuned" language models (i.e., finetuned to respond to instructions) have demonstrated a remarkable ability to generalize zero-shot to new tasks. Nevertheless, they depend...
Stanford CS336 Language Modeling from Scratch I 2025
Learn Dutch • Online Dutch Grammar Course
Learn Dutch grammar: Spelling and pronunciation, verbs, pronouns, nouns and articles, word order, and more. It includes sound files and exercises, and an active forum to post your questions..
RFC: Programming Languages Course Reboot, 2026
On the Biology of a Large Language Model
We investigate the internal mechanisms used by Claude 3.5 Haiku — Anthropic's lightweight production model — in a variety of contexts, using our circuit tracing methodology.
A break from programming languages
This is a blog post I have been considering writing for a long time.
Scaling Laws Across Model Architectures: A Comparative Analysis of...
The scaling of large language models (LLMs) is a critical research area for the efficiency and effectiveness of model training and deployment. Our work investigates the transferability and...
Are you Slop??
Are you Slop? Analyze your 'slop score' - a measure of how generic or unique your text appears to language models.
Prism: An Impure Functional Language With Typed Effects - Stephen Diehl
Personal blog of Stephen Diehl - Software engineer writing about technology, programming, and the future
Physics of LM: Part 4.2, Canon Layers at Scale where Synthetic Pretraining Resonates in Reality
The assistant axis: situating and stabilizing the character of large language models
Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.
