1 AM reflections

Things I learned the hard way, written up while they were still keeping me awake. From projects and coursework, and each one has something to play with.

• Gamma beat saturation: why screen colours lie on cheap LEDs

  Screen colours looked pastel on a A$49 LED bar. More saturation made it worse; a gamma curve fixed it. Why the error lived in the smallest part of each colour.

  from building Lantern
• What transfers to a GPT trained on 3.7M tokens, and what doesn't

  A 30M-parameter GPT on about 160 times less data than scaling laws suggest. The tokenizer mattered most, depth beat width, and SwiGLU did not survive the small data.

  from Advanced Machine Learning, ANU
• Predict the data, not the velocity, until you can't

  In flow matching, predicting the clean data held up at 32 dimensions where predicting the velocity collapsed. Why, what it cost to rescue, and why production models still predict velocity.

  from Advanced Machine Learning, ANU