https://yuejiang.org/tags/deep-learning/Recent content in Deep Learning on Yue Jiang | 姜岳HugoenWed, 15 Jul 2026 00:00:00 +0000https://yuejiang.org/2026/07/15/transformer-implementation/Wed, 15 Jul 2026 00:00:00 +0000https://yuejiang.org/2026/07/15/transformer-implementation/<p>I tried to use Claude Code for self study. Starting here with a 10-ish day <a href="https://github.com/Yue-Jiang/transformer-tutorial">tutorial</a> on implementation of the GPT-style transformer model. This was an enjoyable experience - after a few days I got to implement embedding, layernorm, multi-head attention, cross entropy and put them together into a decoder-only transformer that actually trains on my laptop. Beyond learning about transformer implementation, I realized that I could self study other subjects this way too - it will likely be more pleasant and motivating - feels a bit like Duolingo. I wish I had coding agents like this back in college and grad school (Maybe now is the time to revisit that measure theory class I struggled with most! or not). The tutorial itself may not be very useful for other people, although a couple observations may generalize. When using Claude for self study, I find it helpful to 1) talk with it, be explicit about my understanding, assumptions and confusions 2) have it break the learning down to bite size 3) use a plain text editor for assignments - even VS Code Copilot autocompletion implements transformers better than me.</p>