Systems design 3: LLMs and the semantic revolution

All you need is links

When Tim Berners Lee talks about how he designed the web, he uses an analogy from physics, describing his process as a quest to find “fundamental laws” which can generate a desired system:

Rotating the Space: On LLMs as a Medium for Thought

An essay on LLMs as a cognitive medium

Atomic Design by Brad Frost

Learn how to create and maintain digital design systems, allowing your team to roll out higher quality, more consistent UIs faster than ever before.

Fight fire with pedantry

Over the last two years, I’ve heard a bunch of speculation about how to design a programming language specifically for an LLM. It usually centers around typechecking, formal verifiability, or context engineering. It’s a fun thought experiment, but always felt aimless. First, my experience has been that designing for humans is the best way to design for an LLM, as long as you restrict yourself to text. Second, being in-distribution is so critical that there’s no point unless you’re also training a frontier coding model.

Artificial

An LLM is a computer program. We should talk about it like a computer program.

Lexicon Service Docs - at:// pizza thoughts

Riffing on some ways to make network services more legible to humans and machines

Conway's Law | Laws of Software Engineering

Organizations design systems that mirror their own communication structure.

On learning programming in an age of LLMs

Open answers to a reader's letter.

Extensible Software in the age of LLMs | Jeremy Morrell

Solid core + capability-based sandboxes + LLMs = Users with superpowers

Prism: An Impure Functional Language With Typed Effects - Stephen Diehl

Personal blog of Stephen Diehl - Software engineer writing about technology, programming, and the future

Harnessing Frustration: Using LLMs to Overcome Activation Energy

One of my biggest weaknesses as a software engineer is procrastination when facing a new project. When the scope is unclear, I have a tendency to wait until I feel I’ve “felt out” the problem to start doing anything. I know I’ll feel better and work much faster when I get “stuck in” but I still struggle with that first step, overcoming the “activation energy” required to engage with the details. LLMs have been a game-changer for me in this respect: I can just throw a couple of sentences at them with the shape of the problem. This leads to one of two outcomes: The LLM comes up with a good solution, usually in a slightly different way than what I was thinking. I realize “oh wow the solution is much simpler than I thought”. Straight away I start thinking about the consequences of implementing and improving what the LLM suggested. The LLM comes up with a solution that I intuitively recognize as “wrong”. My immediate reaction is frustration (“How could it get it so wrong”) which leads me to go back and forth with the model, explaining to it why its solution could not possibly work. But in the process of arguing with the model, my brain is churning away and generating variations or different approaches that could work. After a while, even if the AI is still on the wrong track, the debate will trigger a moment of inspiration where suddenly the solution will come to me. I’ll excitedly start up a new conversation and start working through it with the model. The key is the emotional reaction I have immediately to the LLM’s response, either excitement or frustration. By harnessing this immediate feedback loop, I get my brain out of its passive, procrastination mode. It’s almost like a jolt: either I’m thrilled because it’s simpler than I thought, or I’m spurred to action by the urge to correct a perceived ‘wrong’ answer. This forces me to engage with the problem in a meaningful way.

How LLMs Actually Work

A from-the-ground-up walkthrough of how modern LLMs work, from tokens to transformer blocks to the next-token loop

Design for a Living Planet: Settlement, Science, & the Human Future

In this brief, accessible volume, the authors — an urban philosopher and a mathematician-physicist — explain the surprising new findings from the sciences that are beginning to transform environmental design in the modern era. Authors Michael Mehaffy and Nikos Salingaros explore fractals, networks, self-organization, dynamical systems and other revolutionary ideas, describing them to non-science readers in a direct and engaging way. The book also examines fascinating new topics of design, including Agile, Wiki, Design Patterns and other “open-source” approaches from the software world. The authors conclude that a profound transformation is under way in modern design — and today’s students and practitioners will need to be aware of its implications for our future.“Lucidly describes what’s coming in the world of design — and what needs to come.” — Ward Cunningham, Inventor of wiki, and pioneer of Pattern Languages of Programming, Agile, and Scrum“Essential reading for all urban designers.” — Jeff Speck, Author of Walkable City“Brilliant.” — Charles Montgomery, Author of Happy City“Inspired, compelling and fascinating… Recognizes that a true architecture can be dug from the facts, insights, and theories, that occur with a broadening of science to include the human being.” — Christopher Alexander, Author of A Pattern Language and Notes on the Synthesis of FormSome comments on the individual chapters:“Packed with detail and beautiful in presentation.” — Gil Friend“Human society must find a path of retreat. Salingaros and Mehaffy point the way.” — David Brussat, Providence Journal“Michael Mehaffy and Nikos Salingaros have written some brilliant articles on how we can co-create cities which are truly resilient, rather than being ‘engineered resilient’.” — Smallworld Urbanism“For me, this essay was like a flash of insight, and I suddenly saw the world in a new light.” — Oeyvind Holmstad, Permaliv“We’ve just come across a very thoughtful article by Michael Mehaffy and Nikos Salingaros… [who] draw a number of lessons from biological systems and use them to draw conclusions about how resilient human systems must be designed.” — Resilient Design Institute“Salingaros and Mehaffy take us from the configuration of city spaces to the order of cells in living beings.” — Jaap Dawson, Delft Institute of Technology“If you wanted to know where the cutting edge was in urban design, it is here.” — Patrick J. Kennedy, CarFreeInBigD“This is the single most intelligent and illuminating article I’ve seen on Archdaily in 3 years.” — Nìming Pínglùn Zhě, ChinaMichael Mehaffy is an urbanist and design theorist, and a periodic visiting professor or adjunct in five graduate universities in four countries and three disciplines (architecture, urban planning and philosophy) including the University of Oregon (US) and the University of Strathclyde (UK). He has been a close associate of the architect and software pioneer Christopher Alexander, and a Research Associate with the Center for Environmental Structure, Alexander’s research center founded in 1967. He is currently executive director of Portland, Oregon based Sustasis Foundation, and editor of Sustasis Press.Nikos A. Salingaros is a mathematician and polymath known for his work on urban theory, architectural theory, complexity theory, and design philosophy. He has been a close collaborator of the architect and computer software pioneer Christopher Alexander. Salingaros published substantive research on Algebras, Mathematical Physics, Electromagnetic Fields, and Thermonuclear Fusion before turning his attention to Architecture and Urbanism. He is Professor of Mathematics at the University of Texas at San Antonio and has been on the Architecture faculties of universities in Italy, Mexico, and The Netherlands.

Hyperproblems: New Ways of Doing and Communicating Science - Hyperproblems

Hyperproblems: Hyperproblems are scientific challenges whose scale, complexity, novelty and interdependence overwhelm traditional research models, requiring…

LLMs break the internet. Signing everything fixes it.

The dead internet theory wasn't wrong, just early. Here's what to do next.