Generative AI
LLM Architecture
Open the box. Follow a prompt through tokenisation, embeddings, attention layers and the output head of a GPT-style model.
- Advanced
- About 8 hours
- 2 modules
- 8 lessons
- Free
This course is not released yet. We publish courses one at a time, each one complete. The full syllabus is below.
What you will learn
- The PipelineFrom Text to Tokens to Vectors, Decoder-Only Transformers, Attention in LLMs and more
- Scaling UpParameters and Layers, KV Caching, Quantisation and more
Course syllabus
Not released yet · full syllabus below
- 1
The Pipeline
4 lessons- 1From Text to Tokens to VectorsSoon
- 2Decoder-Only TransformersSoon
- 3Attention in LLMsSoon
- 4The Output Layer and LogitsSoon
- 2
Scaling Up
4 lessons- 5Parameters and LayersSoon
- 6KV CachingSoon
- 7QuantisationSoon
- 8Mixture of ExpertsSoon