Andrej Karpathy on Twitter / X

Self-Instruct: Aligning Language Models with Self-Generated Instructions

Large "instruction-tuned" language models (i.e., finetuned to respond to instructions) have demonstrated a remarkable ability to generalize zero-shot to new tasks. Nevertheless, they depend...

Introduction | RLHF and Post-Training Book by Nathan Lambert

A first-principles introduction to RLHF, what it changes in language models, and how it became part of modern post-training.

Are you Slop??

Are you Slop? Analyze your 'slop score' - a measure of how generic or unique your text appears to language models.

How linguistics learned to stop worrying and love the language models

Language models (LMs) can produce fluent, grammatical text. Nonetheless, some maintain that language models don’t really learn language and also, even if they did, that would not be informative for the study of human learning and processing. On the other side, there have been claims that the success of LMs obviates the need for studying linguistic theory and structure. We argue that both extremes are wrong. LMs can contribute to fundamental questions about linguistic structure, language processing, and learning. They force us to rethink arguments and ways of thinking that have been foundational in linguistics. While they do not replace linguistic structure and theory, they serve as model systems and working proofs of concept for gradient, usage-based approaches to language. We offer an optimistic take on the relationship between language models and linguistics.

Scaling Laws Across Model Architectures: A Comparative Analysis of...

The scaling of large language models (LLMs) is a critical research area for the efficiency and effectiveness of model training and deployment. Our work investigates the transferability and...

PoliSim@CHI 2026

Large Language Models are rapidly evolving from text generators into reasoning systems that can act as autonomous agents. When placed in social contexts, these agents display emergent behaviors such as forming coalitions, spreading information, and making collective decisions.

Fast and Hard Code

People pick languages differently with LLMs.

The Koka Programming Language

Koka Language Specification

how to manage your AT Lexicons with lpm - underreacted

Astra for Coding: Why Are We Doing This Again?

Some thoughts on Astra and newfangled long-horizon models.

Wikipedia:Signs of AI writing

This is a list of writing and formatting conventions typical of AI chatbots such as ChatGPT, with real examples taken from Wikipedia articles and drafts. It is a field guide to help detect undisclosed AI-generated content on Wikipedia: while some of the signs may be broadly applicable, some may not apply in a non-Wikipedia context.[a] Not all text featuring these indicators is AI-generated, as the large language models that power AI chatbots are trained on human writing, including Wikipedia. Many elements of AI writing can be found in editorials, blogs, or fan fiction.

A break from programming languages

This is a blog post I have been considering writing for a long time.

Fight fire with pedantry

Over the last two years, I’ve heard a bunch of speculation about how to design a programming language specifically for an LLM. It usually centers around typechecking, formal verifiability, or context engineering. It’s a fun thought experiment, but always felt aimless. First, my experience has been that designing for humans is the best way to design for an LLM, as long as you restrict yourself to text. Second, being in-distribution is so critical that there’s no point unless you’re also training a frontier coding model.

Prism: An Impure Functional Language With Typed Effects - Stephen Diehl

Personal blog of Stephen Diehl - Software engineer writing about technology, programming, and the future