Writing note: The motivation and project are mine. ChatGPT assisted with drafting, editing, translation, and the original diagram and animation source.
Writing code again in the age of coding agents
While looking for my next job, I noticed something uncomfortable about my everyday work. I was using LLMs constantly. With coding agents helping me, I was writing less and less code myself.
It was convenient. But I also felt afraid, or at least deeply uneasy. I had spent years learning mathematics, machine learning, and how to build AI systems. Now I could ask an agent to implement something and receive working code before I had fully worked through the reasoning behind it. What would happen to my own understanding if I kept doing that?
I don’t think typing code by hand automatically makes someone a better researcher or engineer. And this is not an argument that coding agents are bad. What worried me was more personal: I did not want to become dependent on tools whose decisions I was losing the habit of questioning.
That discomfort is where LLM from First Principles began.
I wanted to write the code myself again
My initial idea was simple: take the mechanisms behind the language models I use every day, work out why they are designed as they are, and implement them myself. It would also be a chance to revisit things I had learned before, including ideas from my PhD that had become familiar enough to stop demanding close attention.
I wanted a more demanding standard than being able to recognize an equation or follow an explanation. Could I predict the shape of a tensor before running the next cell? Could I explain why a gradient reaches one parameter but not another? If I removed an operation, would I know what behavior changed?
For this project, from first principles means working backward from an implementation to its reasons: mathematical identities, assumptions, modeling choices, and engineering constraints. It does not mean that every part of modern LLM training can be uniquely derived from a handful of axioms. Often the point is to understand why a particular choice was made and what alternatives it leaves open.
I gradually settled on a routine: ask a question, derive the mechanism, write the implementation, inspect its behavior, and try to explain it. Sometimes the inspection sends me back to the question.
From tensors to post-training
The result is a learning repository containing 28 notebooks and a small reusable PyTorch package. The path starts with tensors, broadcasting, and automatic differentiation, then builds a decoder-only GPT and follows it through pretraining and post-training.
The early notebooks work through autograd, tokenization, optimization, and the training loop. The next ones build attention, positional information, a tiny GPT, generation, KV caching, linear attention, pretraining, and MoE. The later notebooks connect RL foundations and PPO to SFT, preference learning, DPO, and group-relative RL.
The notebooks are numbered 00–26, with PPO split into 21A and 21B. They contain derivations, small experiments, and saved observations. The reusable parts live in src/llmfp/, and a separate example shows how some of the pieces connect. This is an educational project, not a production LLM framework.
The difference between knowing the names of these mechanisms and implementing them becomes obvious in small details. In supervised fine-tuning, for instance, a prompt is context, but only the completion may contribute to the training loss. The token labels must be shifted and masked at the correct positions. If I shift them twice, the code can still execute while optimizing the wrong targets.
PPO brought another useful distinction: the old policy, whose probabilities were recorded when actions were sampled, is not the reference policy used for KL regularization. They can both appear as fixed log-probabilities in code, yet mean different things. Writing and checking the interfaces forced me to keep those roles separate.
These are not spectacular discoveries. They are exactly the sort of details I wanted to stop taking for granted.
When a lower loss did not mean a better answer
One small experiment from Lesson 25 on DPO captures the spirit of the repository.
In the notebook’s saved run, a tiny GPT was updated on three synthetic chosen/rejected response pairs. Its DPO training loss dropped from about 0.693 to 0.015. On six toy generation prompts, however, greedy decoding returned the expected answer 2 times before training and 2 times afterward.
One prompt was Say no. Before DPO, the model answered Yes; afterward, it answered Thanks. Neither was the requested No.
The preference margin for that training pair improved, but improving a reference-relative score for a pair of complete responses is not the same as producing the desired text one token at a time. The loss told me that the preference objective had been fitted on the examples supplied. Generation was a separate question.
Those are saved observations from a tiny teaching experiment, not a benchmark or evidence about DPO at scale. I kept them because an unsuccessful behavioral check can explain more than a smooth loss curve.
Learning with ChatGPT, without asking it to write the lesson
There is an obvious tension in all of this: I used ChatGPT extensively while building a project intended to help me rely less blindly on LLM-generated code.
I used it as a tutor. I asked about derivations, questioned explanations, compared implementation choices, and used code review to find errors in my reasoning. But for the notebook learning exercises, I disabled AI code completion. I typed the implementations myself, after thinking through what they should do.
This boundary is deliberately specific. It does not imply that AI was absent from repository maintenance, extracting reusable components, or integration examples. Nor does it mean that I should avoid mature PyTorch primitives forever. Once I understand an operation, using a well-tested implementation can free me to study the next mechanism.
The rule I tried to follow was simple:
If I cannot explain a line of code, it does not belong in the learning exercise yet.
AI can make it easy to obtain an answer. I wanted to make sure I still did the work of deciding whether I understood it.
Why I am sharing it
The source code is available at github.com/neuromaancer/llm-from-first-principles. If you want to see the pieces run together before opening the notebooks, there is a small CPU example using synthetic data, no downloaded checkpoint, and no API key:
git clone https://github.com/neuromaancer/llm-from-first-principles.git
cd llm-from-first-principles
uv sync
uv run python examples/tiny_sft_dpo.py
It starts with random GPT weights, fits the preferred responses in a supervised step, freezes a reference, applies DPO, and shows the generated outputs. The small training set is an exercise in connecting components; it should not be mistaken for evidence of generalization.
I am sharing this because I suspect others may recognize the discomfort that started it. We use language models to write, research, and program, often with very little need to look inside. That is useful. But some of us also want to understand the mechanisms underneath the tool we now use every day. I hope the notebooks offer one practical route back into that machinery.
This is also why I began the First Principles of LLMs & RL blog series. The repository lets me test whether I can build an idea; the articles let me test whether I can explain why it works. Neither task is quite finished by getting the code to run.
I will still use coding agents. What I want to keep is the ability to understand their work, question their choices, and know when I need to work something out for myself.