К содержимому
learnspaceYOUR NEXT CHAPTER
ПРОСТРАНСТВО ОБУЧЕНИЯ
ГлавнаяКаталог курсовМоё обучениеCoursera

Знания без границ

Учитесь у лучших университетов и компаний мира.

Открыть Coursera
Интеграция
Пространство университета
Моё пространствоСтраница курса
↵
ЯЛичный кабинетСтудент
© 2026 LearnSpaceКаждый день — возможность узнать больше.Помощь
Advanced Data Engineering · LearnSpace
Назад в каталог
courseraПрограммирование

Advanced Data Engineering

Курс от Duke University
Средний≈ 23.3 чАнглийский
О курсеНавыкиПрограммаПреподаватели

О курсе

In this advanced course, you will gain practical expertise in scaling data engineering systems using cutting-edge tools and techniques. This course is designed for data scientists, data engineers, and anyone with a foundational understanding of data handling who desires to escalate their skills to handle larger, more complex datasets efficiently. Throughout the course, you'll master the application of technologies such as Celery with RabbitMQ for scalable data consumption, Apache Airflow for optimized workflow management, and Vector and Graph databases for robust data management at scale. The course will culminate with hands-on projects that offer real-world experience, where you'll put your acquired skills to test in solving data engineering challenges. You will not only learn to create scalable data systems but also to analyze their performance and make necessary adjustments for optimum results. This invaluable experience in advanced data engineering techniques will prepare you for the demanding tasks of handling massive datasets, streamlining complex workflows, and optimizing data operations for businesses of any scale.

Навыки, которые вы освоите

Vector DatabasesMySQLApache AirflowScalabilityDatabase Management SystemsData PipelinesPerformance TuningDatabase ManagementData Import/Export

Программа курса

4 модулей · 134 учебных материалов

01Queues and Databases-RabbitMQ and MySQL43 материалов

About the course

Meet your instructor: Alfredo DezaВидеоConnect with your instructorЧтениеMeet your instructor: Noah GiftЧтениеAbout this courseВидео

Учитесь у экспертов

Noah Gift

Executive in Residence and Founder of Pragmatic AI Labs

Alfredo Deza

Adjunct Assistant Professor in the Pratt School of Engineering

Advanced Data Engineering
В каталоге вашей программы

Инвестируйте в себя

Новые знания — в удобное для вас время.

Начать на Coursera

Обучение откроется на Coursera
в новой вкладке

Обучение на Coursera

≈ 23.3 ч

4 модулей

Язык: Английский

Субтитры: Арабский, Французский, Узбекский, Украинский, Китайский (Китай), Греческий, Итальянский, Бразильский португальский, Вьетнамский, Нидерландский, Корейский, Немецкий, Русский, Тайский, Индонезийский, Шведский, Турецкий, Испанский, Хинди, Японский, Казахский, Венгерский, Польский

Часть программы вашего университета
Course structure and discussion etiquetteЧтение
Meet and greet (optional)Обсуждение
Report a Problem with the CourseЧтение

Introduction to RabbitMQ

Key TermsЧтениеIntroductionВидеоOverview of QueuesВидеоWhat is Celery?ВидеоIntroduction to CeleryЧтениеUse cases for RabbitMQВидеоUsing RabbitMQ with DockerЧтениеOverview of a Flask and Celery applicationВидеоSummaryВидеоExternal lab: Start RabbitMQ in a development environmentЧтениеIntroduction to RabbitMQ and FlaskЗадание

Using RabbitMQ with Flask and Celery

Key TermsЧтениеIntroductionВидеоConfiguring Celery with FlaskВидеоBuild a web app by using Python and FlaskЧтениеBackground tasks with CeleryЧтениеConnecting Celery with RabbitMQВидеоDefining a Celery task in FlaskВидеоFire and forget task in FlaskВидеоRetrieve values from asynchronous tasksВидеоSummaryВидеоExternal lab: Add a new Celery task for RabbitMQЧтениеRabbitMQ with Celery and FlaskЗадание

MySQL for Data Engineering

Key TermsЧтениеMySQL OverviewВидеоGetting Started with MySQLЧтениеMySQL from TerminalВидеоArchive and Drop DatabaseВидеоImport external database SakilaВидеоModify database SakilaВидеоBash pipelines with MySQLВидеоMySQL to Python Standard Library Web ServerВидеоLinux Hacking with MySQLЛабораторнаяQuiz-MySQL for Data EngineeringЗаданиеLesson ReflectionЧтениеQueues and Databases - Final week quizЗадание
02Optimizing Workflow Management at Scale with Apache Airflow34 материалов

Installing Apache Airflow

Key TermsЧтениеIntroductionВидеоWhat is Apache AirflowЧтениеWhat is Apache Airflow?ВидеоInstalling Apache Airflow from PyPIВидеоExploring the Airflow User InterfaceЧтениеUsing Apache Airflow with DockerВидеоExploring the Airflow UIВидеоExternal lab: Install Apache AirflowЧтениеLesson ReflectionЧтениеQuiz-Installing Apache AirflowЗадание

Apache Airflow Fundamentals

Key TermsЧтениеIntroductionВидеоExploring directed acyclic graphs (DAG)ВидеоExternal lab: Create a DAGЧтениеCreating a DAGВидеоArchitecture overviewЧтениеRunning a backfill

Creating a pipeline

Key TermsЧтениеIntroductionВидеоExternal Lab: Build a data pipeline for census dataЧтениеBuild Data Pipelines with Apache AirflowЧтениеIdentifying a task to build a DAGВидеоRetrieving remote dataВидеоCleaning and normalizing data
03Achieving Scalability with Vector, Graph, and Key/Value Databases29 материалов

Introduction to Vector Databases

Key TermsЧтениеPicking the proper databaseВидеоWhat are vector databases and how they workВидеоImplementing Semantic searchВидеоQuickstart QdrantВидеоQdrant Rust ClientВидеоVector Database ArchitecturesВидеоHands-on lab: Enhance Semantic SearchВидеоSocial Media RecommenderЛабораторнаяWhat is a Vector Database?ЧтениеExternal Lab: Run Quickstart of QdrantЧтениеExternal Lab: Extend Semantic SearchЧтениеJaccard indexЧтениеQuiz-Introduction to Vector DatabasesЗаданиеLesson ReflectionЧтение

Introduction to Graph Databases

Key TermsЧтениеGraph data models and database conceptsВидеоIntroduction to Amazon NeptuneВидеоGraph algorithms: UFC graph centrality in RustВидеоKosaraju Community Detection in GraphsВидеоShortest Path with GraphsВидео
04Real-world Advanced Data Engineering Projects28 материалов

Building a solution with DynamoDB with the AWS CLI

Key TermsЧтениеAmazon CodeCatalystЧтениеLearn AWS CloudShell for Dynamo DevelopmentВидеоLearn AWS CodeCatalyst for Dynamo DevelopmentВидеоLeveraging AWS CodeWhisperer for Dynamo DevelopmentВидеоCreate a Table with CLIВидеоPopulate a Table With Batching RecordsВидеоQuery a Table with RecordsВидеоProject WalkthroughВидеоExternal Lab: Extended DynamoDB ЧтениеLesson ReflectionЧтениеQuiz-Building a solution with DynamoDB with the AWS CLIЗадание

Persisting data through a multi-task DAG with Pandas

Key TermsЧтениеIntroductionВидеоOverview of a pipeline requirementsВидеоUsing SqlAlchemy with PandasВидеоQuick start for SQLAlchemyЧтениеPersisting data in a taskВидео

Next Steps

Final Quiz-Advanced Data EngineeringЗаданиеRecommended Next StepsЧтениеJupyter SandboxЛабораторнаяVS Code SandboxЛабораторнаяShare your learning experienceЧтение
Видео
Testing and validationВидео
SummaryВидео
Quiz-Apache Airflow FundamentalsЗадание
Lesson ReflectionЧтение
Видео
Inspecting the UI for resultsВидео
SummaryВидео
Lesson ReflectionЧтение
Quiz-Creating a pipelineЗадание
Quiz-Optimizing Workflow Management at Scale with Apache AirflowЗадание
Key Components of Rust CLI ToolВидео
Lab Walkthrough: Building a Rust Graph CLI ToolВидео
Rust CLI with ClapЧтение
External Lab: Rust Graph CLI ToolЧтение
Amazon NeptuneЧтение
Lesson ReflectionЧтение
Quiz-Introduction to Graph DatabasesЗадание
Final Quiz-Achieving Scalability with Vector, Graph, and Key/Value DatabasesЗадание
Reviewing the resultsВидео
SummaryВидео
Explore and analyze data with PythonЧтение
Lesson ReflectionЧтение
Quiz-Persisting data through a multi-task DAG with PandasЗадание