My implementation for Stanford CS336: Language Modeling from Scratch assignments. This repo contains code for the 5 hands-on assignments of the course, covering key topics in LLM: building core Transformer components (tokenizer, architecture, optimizer), implementing FlashAttention-2 with Triton, exploring the scaling law, etc.

Check out csdiy.wiki for more information about CS336 and many other excellent courses.