Notes & Signals

A place where I share my thoughts on speech processing and deep learning, academic life, and the books I happen to be reading — from investing and philosophy to fiction old and new.


Visual Information Theory, Part 2: Joint Entropy, Mutual Information, and Stochastic Processes

Extending entropy to two or more variables — joint entropy, conditional entropy, mutual information, fractional bits, and the entropy rate of a stochastic process.

16 min read   ·   2026

Visual Information Theory, Part 1: Probability, Codes, and Entropy

A visual and mathematically rigorous introduction to information content, entropy, cross-entropy, and KL divergence — for readers who want the intuition and the derivations.

17 min read   ·   2026