Learning in public
Studies & notes
Working through algorithms with code, derivations, and experiments.
6 studies
NLP · 2024
Attention in Sequence-to-Sequence Models
Derive attention weights, check their limiting behavior, and implement batch-first attention for German-to-English translation.
Attention / Sequence Models / Machine Translation
NLP · 2024
LSTMs: Parity, State Machines & POS Tagging
Explore what a small LSTM can remember through parity and grammar tasks, then examine a BiLSTM tagger's results and failure cases.
LSTM / Sequence Modeling / POS Tagging
NLP · 2024
Word Embeddings: PPMI, GloVe & Bias
Build count-based and learned word vectors, derive GloVe gradients, and interpret embedding visualizations and analogy results.
Word Embeddings / GloVe / PPMI
NLP · 2024
Language Model Decoding: Sampling & Beam Search
Compare temperature, top-k, nucleus sampling, and beam search, with attention to probability filtering, recurrent state, and reproducibility.
Text Generation / Sampling / Beam Search
Online Learning · 2023
Discounted UCB in a Changing Environment
Study how forgetting old rewards helps after a change point, and distinguish time-based discounting from the selected-arm update used in the archived experiment.
Nonstationary Bandits / UCB / Discounting
Online Learning · 2023
Bandit Algorithms: Methods & Implementation Review
Compare exploration strategies and inspect where the experimental implementations differ from standard UCB, Thompson sampling, and nonlinear bandit methods.
Bandits / UCB / Thompson Sampling