al.
A place to keep my sparks. All opinions are my own.
Browse series & topics
Latest posts
-
Attention: How Does a Model Decide Where to Look?
From human selective attention to recurrent encoder–decoders, additive attention, and learning how to read from context.
-
How Does Text Become a Vector?
From tokenization and word embeddings to learned representations and positional encodings: how text becomes a trainable numerical input.
-
How Does a Neural Network Know Which Way to Move?
How Taylor approximation, gradient descent, and backpropagation turn a model's prediction loss into a concrete parameter update.
-
Why Does −log p Keep Appearing?
From surprisal to entropy, cross-entropy, and KL divergence: why the same logarithm measures information and prediction loss.
-
Where Do Probabilities Come From?
How observations become model distributions: shared parameters, logits, Softmax, the chain rule, maximum likelihood, and negative log-likelihood.
-
Relearning LLMs and RL from First Principles
Why I am going back to the mathematics behind language models and reinforcement learning, and using derivation, implementation, and explanation to test what I actually understand.
-
A dialogue policy playground
Explore how task goals, social goals, and temperature affect a simple dialogue policy.