К содержимому
learnspaceYOUR NEXT CHAPTER
ПРОСТРАНСТВО ОБУЧЕНИЯ
ГлавнаяКаталог курсовМоё обучениеCoursera

Знания без границ

Учитесь у лучших университетов и компаний мира.

Открыть Coursera
Интеграция
Пространство университета
Моё пространствоСтраница курса
↵
ЯЛичный кабинетСтудент
© 2026 LearnSpaceКаждый день — возможность узнать больше.Помощь
Deep Reinforcement Learning: From Theory to Practice · LearnSpace
Назад в каталог
courseraПрограммирование

Deep Reinforcement Learning: From Theory to Practice

Курс от University of Colorado Boulder
Средний≈ 18 чАнглийский
О курсеНавыкиПрограммаПреподаватели

О курсе

How can reinforcement learning scale beyond small tabular problems to high-dimensional environments such as games, robotics, and autonomous decision-making? This course introduces deep reinforcement learning, where reinforcement-learning algorithms are combined with neural-network-based function approximation. Learners begin by studying why tabular methods break down in large or continuous state spaces and how value functions, action-value functions, and policies can be represented by parameterized models. The course then develops value-based deep reinforcement learning methods, including fitted value iteration, Deep Q-Networks, replay buffers, target networks, Double DQN, dueling networks, and prioritized experience replay. Learners also study direct policy optimization through policy-gradient methods such as REINFORCE, as well as actor–critic methods that combine policy optimization with value estimation. The course introduces selected modern deep RL algorithms, such as PPO, DDPG, and SAC, with emphasis on implementation, stability, diagnosis, and empirical evaluation. By the end of the course, learners will be able to implement deep reinforcement-learning agents, diagnose common sources of instability, evaluate learned behavior using suitable experimental protocols, and report results in a reproducible way. This course can be taken for academic credit as part of CU Boulder’s Masters of Science in Computer Science (MS-CS) and Master of Science in Artificial Intelligence (MS-AI) degrees offered on the Coursera platform. These fully accredited graduate degrees offer targeted courses, short 8-week sessions, and pay-as-you-go tuition. Admission is based on performance in three preliminary courses, not academic history. CU degrees on Coursera are ideal for recent graduates or working professionals. Learn more: MS in Artificial Intelligence: https://www.coursera.org/degrees/ms-artificial-intelligence-boulder MS in Computer Science: https://coursera.org/degrees/ms-computer-science-boulder

Навыки, которые вы освоите

Reinforcement LearningDeep LearningSystem Design and ImplementationModel OptimizationModel TrainingDebuggingApplied Machine LearningAgentic systemsSampling (Statistics)Artificial Neural NetworksMachine Learning MethodsModel EvaluationMachine Learning AlgorithmsArtificial Intelligence and Machine Learning (AI/ML)Probability DistributionNetwork Model

Программа курса

6 модулей · 92 учебных материалов

01Function Approximation for RL 21 материалов

Welcome to the Course!

Course IntroductionВидео

Course Policies and Expectations

Earn Academic Credit for your Work!ЧтениеCourse SupportЧтениеAssessment ExpectationsЧтение

Why Function Approximation?

Учитесь у экспертов

Ashutosh Trivedi

Преподаватель курса

Deep Reinforcement Learning: From Theory to Practice
В каталоге вашей программы

Инвестируйте в себя

Новые знания — в удобное для вас время.

Начать на Coursera

Обучение откроется на Coursera
в новой вкладке

Обучение на Coursera

≈ 18 ч

6 модулей

Язык: Английский

Часть программы вашего университета
When Tabular RL BreaksВидео
Why Function ApproximationЧтение

Parameterized Value Functions

From Tables to Vθ and QθВидеоParameterized Value FunctionsЧтение

Learning from Targets

Losses, Gradients, and Semi-Gradient UpdatesВидеоLearning from TargetsЧтениеMonte Carlo TD TargetsЧтение

Why RL Is Harder

Bootstrapping, Changing Targets, and Correlated DataВидеоWhy Reinforcement Learning Is Harder Than Supervised LearningЧтение

The Deadly Triad

Function Approximation, Bootstrapping, and Off-Policy LearningВидеоThe Deadly TriadЧтение

Fitted Value Iteration

A Bridge from Value Iteration to Deep Q-LearningВидеоFitted Value IterationЧтение

Module Assignment

Module SummaryВидеоModule SummaryЧтениеAI Policy QuizЗаданиеFunction Approximation for RL Задание
02Deep Q-Learning and Value-Based Deep RL16 материалов

From Q-Learning to DQN

Replacing the Table by a Neural NetworkВидеоFrom Q-Learning to Deep Q-LearningЧтение

Q-Networks

Action Values as Neural Network OutputsВидеоQ-NetworksЧтение

The DQN Loss

Bellman Targets and Gradient UpdatesВидеоThe DQN LossЧтение

Experience Replay

Learning From Stored TransitionsВидеоExperience ReplayЧтение

Target Networks

Stabilizing Bootstrapped TargetsВидеоTarget NetworksЧтение

The DQN Algorithm

Putting the Pieces TogetherВидеоThe Complete DQN AlgorithmЧтение

DQN Improvements

Double DQN, Dueling Networks, and Practical RefinementsВидеоDQN ImprovementsЧтение

Module Assignment

Module SummaryЧтениеDeep Q-Learning and Value-Based Deep RLЗадание
03Policy Gradients and REINFORCE17 материалов

Why Policy Gradients?

Module IntroductionВидеоLearning Policies DirectlyВидео

Parameterized Stochastic Policies

From Action Values to πθ(a | s)Видео

The Policy-Gradient Objective

Expected Return as an Optimization ProblemВидеоWhy Optimize a Policy Directly?Чтение

The Score-Function Trick

How To Differentiate Through Sampled ActionsВидеоThe Score-Function IdentityЧтениеFrom Trajectory Probabilities to Policy GradientsЧтение

REINFORCE

Monte Carlo Policy-Gradient LearningВидеоREINFORCEЧтение

Baselines and Variance Reduction

Making Policy Gradients PracticalВидеоBaselines and AdvantagesЧтение

From REINFORCE to Actor-Critic

Why We Need Learned CriticsВидеоFrom REINFORCE to Actor-CriticЧтение

Module Assignment

Module SummaryВидеоModule SummaryЧтениеPolicy Gradients and REINFORCEЗадание
04Actor-Critic Methods14 материалов

Why Actor-Critic

Module IntroductionВидеоFrom REINFORCE to Learned CriticsВидео

The Actor and the Critic

Policies and Value Functions TogetherВидеоActor-Critic FoundationsЧтение

Advantage Estimates

Learning from Better-than-Expected OutcomesВидео

TD Actor-Critic

Using TD Errors to Update the PolicyВидео

Actor and Critic Losses

Training the Two NetworksВидеоTD Actor-Critic: Advantages, Updates, and LossesЧтение

Generalized Advantage Estimation

Balancing Bias and VarianceВидеоGeneralized Advantage EstimationЧтение

From Actor-Critic to PPO

Why Stable Policy Updates MatterВидеоFrom Actor-Critic to PPOЧтение

Module Assignment

Module SummaryЧтениеActor-Critic MethodsЗадание
05Modern Deep RL: PPO, DDPG, and SAC13 материалов

Why Modern Deep RL Algorithms?

Module IntroductionВидеоStability, Continuous Actions, and ExplorationВидео

PPO: Proximal Policy Optimization

Actor–Critic with Controlled UpdatesВидеоProbability Ratios and ClippingВидеоPPO: Proximal Policy OptimizationЧтение

DDPG: Deep Deterministic Policy Gradient

Continuous Control: Why Discrete-Action Methods Are Not EnoughВидеоDeterministic Actor-Critic for Continuous ActionsВидеоLearning Policies that ExploreВидеоDDPG: Deep Deterministic Policy GradientЧтение

SAC and Algorithm Comparison

Soft Actor–Critic and Modern Deep RL DesignВидеоSAC and Algorithm ComparisonЧтение

Module Assignment

Module SummaryЧтениеModern Deep RL: PPO, DDPG, and SACЗадание
06Practical Deep RL Implementation11 материалов

The Deep RL Training Loop

Module IntroductionВидеоHow Algorithms Become CodeВидеоTheory NotesЧтение

Rollouts and Replay Buffers

Organizing ExperienceВидео

Networks, Losses, and Optimization

Training Actors and CriticsВидео

Logging and Diagnostics

Knowing Whether Learning Is WorkingВидео

Evaluation and Reporting

Measuring Learned Behavior ReliablyВидеоDiagnostics, Evaluation, and ReportingЧтение

Module Assignment

Course SummaryВидеоCourse Synthesis: Deep Reinforcement Learning in PracticeЧтениеPractical Deep RL ImplementationЗадание