English | MP4 | AVC 1920×1080 | AAC 44KHz 2ch | 52 lectures (7h 30m) | 5.55 GB
Generative AI, r1, LLMs, ChatGPT, GPT4, o1, Llama3, Decoders, T5, BERT, LoRA, FSDP, 4bit, Machine Learning, Data Science
Welcome to “LLMs Mastery: Complete Guide to Generative AI & Transformers”!
This practical course is designed to equip you with the knowledge and skills to build efficient, production-ready Large Language Models using cutting-edge technologies.
Key Topics Covered:
- Generative AI: Understand the principles and applications of Generative AI in creating new data instances.
- ChatGPT & GPT4: Dive into the workings of advanced AI models like ChatGPT and GPT4.
- LLMs: Start with the basics of LLMs, learning how they decode, process inputs and outputs, and how they are taught to communicate effectively.
- Encoder-Decoders: Master the concept of encoder-decoder models in the context of Transformers.
- T5, GPT2, BERT: Get hands-on experience with popular Transformer models such as T5, GPT2, and BERT.
- Machine Learning & Data: Understand the role of machine learning and data in training robust AI models.
- Advanced Techniques: Sophisticated training strategies like PeFT, LoRa, managing data memory and merging adapters.
- Specialised Skills: Cutting-edge training techniques, including 8-bit, 4-bit training and Flash-Attention.
- Scalable Solutions: Master the use of advanced tools like DeepSpeed and FSDP to efficiently scale model training.
What you’ll learn
- Grasp NLP Fundamentals: Understand the evolution from rule-based systems to advanced LLMs like Llama3, Gemma2, Phi3, and Mistral.
- Master Transformers & LLMs: Learn the architecture and application of Transformers in depth. Including tokenization, embeddings, pre-training & fine-tunning.
- Understand Generative AI Principles: Develop skills in building and fine-tuning generative models for real-world applications using RLHF and Chat-Templates
- Use Transformer Models: Overview LLMs and Encoder-Decoder models like BERT, GPT, T5, Llama and more in many different NLP tasks: Personal assistant, Reviews, QA
- Specialised Techniques: Implement 8-bit and 4-bit training, and use tools like DeepSpeed and FSDP, along with PeFT, LoRA, FlashAttention and more.
Table of Contents
Introduction Course Overview + What You’ll Learn
1 Introduction to Part 1 of the Course Transformers Fundamentals
2 Introduction to Part 2 of The Course LLMs
Getting Started How to Make the Best Use of this Course
3 Course Structure How to get the Most out of this Course
4 Environment Setup Prepare and Use the Resource of this Course Right
Overview of Natural Language Processing Bring Transformers into Perspective
5 Rule-Based Systems Era
6 Statistical Era
7 Machine Learning Era
8 Embeddings Era
9 NLP Evolution A Quick Practice Test
Transformers Introduction Important Concepts and Use-cases
10 Encoders, Decoders and The Attention Mechanism
11 Understanding the Transformer Architecture
12 Pre-training & Fine-tunning
13 Pre-training and Fine-tuning Transformers
14 Tokenisation & Embeddings
Popular Transformers Models Choose the Best Model for the Job
15 BERT
16 GPT
17 T5
18 Exploring T5 The Text-to-Text Transfer Transformer
Using Transformers Building Blocks and Hidden Gems (Practical)
19 Building Blocks
20 Tokenizers
21 Understanding Tokenizers for Transformers
22 Word Embeddings
23 Masked Language Modeling (MLM)
24 Semantic Search Index
Mastering Real-World Scenarios with Transformers and LLMs (Practical)
25 BERT (Encoder-model) for Extractive Question Answering
26 GPT (Decoder-model) for Instruction Following
27 Understanding GPT Instruct
28 T5 (Encoder-Decoder-model) for Writing Product Reviews
Introduction to Large Language Models
29 Decoding Large Language Models An Introduction
30 Introduction to Large Language Models (LLMs)
31 RLHF Teaching LLMs to Communicate Effectively
32 Understanding InputOutput in Language Models
33 Chat Templates Hands On Overview
34 Decision Frameworks for LLM Selection
35 Generation An Interactive Guide
Preparing for LLM Training (Practical)
36 Comprehensive Dive into Sequence Length
37 Token Counts Practical Intuition & Impact
38 Precision Matters Numerical Precision in Training
39 Sequences and Tokens
40 Navigating GPU Selection A Guide to Hardware Platforms
41 Practice Fundamentals Most Basic Form of Training LLMs
42 Practice Fundamentals Most Basic Form of Training LLMs – Part 2
43 Practice Fundamentals Most Basic Form of Training LLMs – Part 3
44 LLM Mastery Practice Test
Advanced LLM Training Techniques (Practical)
45 Understanding Practical Limitations
46 Boosting Efficiency PeFT and LoRa in Depth
47 Managing Data Memory Batch Size & Sequence Length
48 Advanced Solutions Gradient Accumulation & Checkpointing
49 Fitting Giants Practical Introduction to LoRA for Large Models
50 Expanding LoRA Adapter Merging and Effective Evaluations
51 Advanced Techniques
Exploring Specialised LLM Training Techniques (Practical)
52 Level-Up Giants 8-bit Training for Massive Models
53 Task-Focused Training Aim for Better Learning – Part 1
54 Task-Focused Training Aim for Better Learning – Part 2
55 Edge of Hardware Limits Scaling Inputs with Flash Attention 2
56 Edge of Hardware Limits Reaching 4bit Training with QLoRA
57 Specialised Techniques
Scale LLM Training with Advanced Tools (Practical)
58 Starting Strong Introductory Concepts for at Scale Training
59 Understanding DeepSpeed A Theoretical Overview
60 Implementing DeepSpeed A Hands-On Approach
61 FSDP Explained Theoretical Insights
62 Applying FSDP Real-World Usage and Best Practices
63 Scaling Training
64 Wrapping Up Course Conclusion, Recap, and Future Directions
Resolve the captcha to access the links!