Learning with Claude: Implementing a transformer from scratch
I tried to use Claude Code for self study. Starting here with a 10-ish day tutorial on implementation of the GPT-style transformer model. This was an enjoyable experience - after a few days I got to implement embedding, layernorm, multi-head attention, cross entropy and put them together into a decoder-only transformer that actually trains on my laptop. Beyond learning about transformer implementation, I realized that I could self study other subjects this way too - it will likely be more pleasant and motivating - feels a bit like Duolingo. I wish I had coding agents like this back in college and grad school (Maybe now is the time to revisit that measure theory class I struggled with most! or not). The tutorial itself may not be very useful for other people, although a couple observations may generalize. When using Claude for self study, I find it helpful to 1) talk with it, be explicit about my understanding, assumptions and confusions 2) have it break the learning down to bite size 3) use a plain text editor for assignments - even VS Code Copilot autocompletion implements transformers better than me. ...