LLM Reinforcement Learning Fine-Tuning DeepSeek Method GRPO

[EN] LLM Fine-Tuning and Reinforcement Learning with SFT, LoRA, DPO, and GRPO Custom Data HuggingFace

In this course, you will step into the world of Large Language Models (LLMs) and learn both fundamental and advanced end-to-end optimization methods. You’ll begin with the SFT (Supervised Fine-Tuning) approach, where you’ll discover how to properly prepare your data and create customized datasets using tokenizers and data collators through practical examples. During the SFT process, you’ll learn the key techniques for making large models lighter and more efficient with LoRA (Low-Rank Adaptation) and quantization, and explore step by step how to integrate them into your projects.

After solidifying the basics of SFT, we will move on to DPO (Direct Preference Optimization). DPO allows you to obtain user-focused results by directly reflecting user feedback in the model. You’ll learn how to format your data for this method, how to design a reward mechanism, and how to share models trained on popular platforms such as Hugging Face. Additionally, you’ll gain a deeper understanding of how data collators work in DPO processes, learning practical techniques for preparing and transforming datasets in various scenarios.

The most significant phase of the course is GRPO (Group Relative Policy Optimization), which has been gaining popularity for producing strong results. With GRPO, you will learn methods to optimize model behavior not only at the individual level but also within communities or across different user groups. This makes it more systematic and effective for large language models to serve diverse audiences or purposes. In this course, you’ll learn the fundamental principles of GRPO, and then solidify your knowledge by applying this technique with real-world datasets.

Throughout the training, we will cover key topics—LoRA, quantization, SFT, DPO, and especially GRPO—together, supporting each topic with project-oriented applications. By the end of this course, you will be fully equipped to manage every stage with confidence, from end-to-end data preparation to fine-tuning and group-based policy optimization. Developing modern and competitive LLM solutions that focus on both performance and user satisfaction in your own projects will become much easier.