К содержимому
learnspaceYOUR NEXT CHAPTER
ПРОСТРАНСТВО ОБУЧЕНИЯ
ГлавнаяКаталог курсовМоё обучениеCoursera

Знания без границ

Учитесь у лучших университетов и компаний мира.

Открыть Coursera
Интеграция
Пространство университета
Моё пространствоСтраница курса
↵
ЯЛичный кабинетСтудент
© 2026 LearnSpaceКаждый день — возможность узнать больше.Помощь
Spark, Hadoop, and Snowflake for Data Engineering · LearnSpace
Назад в каталог
courseraАнализ данных

Spark, Hadoop, and Snowflake for Data Engineering

Курс от Duke University
Продвинутый≈ 29.9 чАнглийский
О курсеНавыкиПрограммаПреподаватели

О курсе

e.g. This is primarily aimed at first- and second-year undergraduates interested in engineering or science, along with high school students and professionals with an interest in programmingGain the skills for building efficient and scalable data pipelines. Explore essential data engineering platforms (Hadoop, Spark, and Snowflake) as well as learn how to optimize and manage them. Delve into Databricks, a powerful platform for executing data analytics and machine learning tasks, while honing your Python data science skills with PySpark. Finally, discover the key concepts of MLflow, an open-source platform for managing the end-to-end machine learning lifecycle, and learn how to integrate it with Databricks. This course is designed for learners who want to pursue or advance their career in data science or data engineering, or for software developers or engineers who want to grow their data management skill set. In addition to the technologies you will learn, you will also gain methodologies to help you hone your project management and workflow skills for data engineering, including applying Kaizen, DevOps, and Data Ops methodologies and best practices. With quizzes to test your knowledge throughout, this comprehensive course will help guide your learning journey to become a proficient data engineer, ready to tackle the challenges of today's data-driven world.

Навыки, которые вы освоите

PySparkDatabricksApache SparkMLOps (Machine Learning Operations)Snowflake SchemaDevOpsApache HadoopDistributed ComputingData IntegrationData TransformationPython ProgrammingData QualityModel DeploymentModel TrainingData ProcessingData ArchitectureSQLData WarehousingData PipelinesBig Data

Программа курса

4 модулей · 110 учебных материалов

01Overview and Introduction to PySpark30 материалов

Course Overview

Welcome to Data Engineering Platforms with Python!ЧтениеMeet your Co-Instructor: Kennedy BehrmanВидеоMeet your Co-Instructor: Noah GiftВидеоMeet and Greet (optional)Обсуждение

Учитесь у экспертов

Noah Gift

Executive in Residence and Founder of Pragmatic AI Labs

Kennedy Behrman

Senior Data Engineer

Matt Harrison

Преподаватель курса

Spark, Hadoop, and Snowflake for Data Engineering
В каталоге вашей программы

Инвестируйте в себя

Новые знания — в удобное для вас время.

Начать на Coursera

Обучение откроется на Coursera
в новой вкладке

Обучение на Coursera

≈ 29.9 ч

4 модулей

Язык: Английский

Субтитры: Арабский, Французский, Итальянский, Бразильский португальский, Корейский, Немецкий, Азербайджанский, Испанский, Японский

Часть программы вашего университета
Report a problem with the courseЧтение

Big Data Platforms: Hadoop and Spark

Overview of Big Data PlatformsВидеоBig Data PlatformsЗаданиеGetting Started with HadoopВидеоWhat is Apache Hadoop?ЧтениеApache Hadoop ConceptsЗаданиеGetting Started with SparkВидеоWhat is Apache Spark?ЧтениеUse Apache Spark in Azure Databricks (optional)ЧтениеApache Spark ConceptsЗаданиеChoosing between Hadoop and SparkЧтение

Resilient Distributed Datasets

Introduction to Resilient Distributed Datasets (RDD)ВидеоWhat are RDDs?ЧтениеRDD ConceptsЗаданиеResilient Distributed Datasets (RDD) DemoВидеоGetting Started: Creating RDD's with PySparkЧтениеPractice: Creating RDD's with PySparkЛабораторная

Spark SQL

Introduction to Spark SQLВидеоSpark SQL, Dataframes and DatasetsЧтениеSpark SQL ConceptsЗаданиеPySpark and Spark SQLЧтениеPySpark Dataframe Demo: Part 1ВидеоPySpark Dataframe Demo: Part 2ВидеоPySpark Dataframe ConceptsЗаданиеPractice: Reading Data into DataframesЛабораторная

Check for Understanding

PySparkЗадание
02Snowflake19 материалов

Snowflake Concepts and Architecture

What is Snowflake?ВидеоAccessing SnowflakeЧтениеDetailed View Inside SnowflakeЧтениеSnowflake ArchitectureЗаданиеSnowflake LayersВидеоSnowflake LayersЗаданиеSnowflake Web UIВидео

Snowflake Demo

Navigating SnowflakeВидеоNavigating SnowflakeЗаданиеSnowsight: The Snowflake Web InterfaceЧтениеCreating a Table in SnowflakeВидеоCreating a TableЗаданиеWorking with WarehousesЧтениеSnowflake Warehouses

Python Connector

Writing to SnowflakeВидеоWriting to SnowflakeЗаданиеReading from SnowflakeВидеоPython Connector DocumentationЧтение

Check for Understanding

SnowflakeЗадание
03Azure Databricks and MLFLow28 материалов

Getting Started with Databricks and Spark

What is Azure Databricks?ЧтениеAccessing DatabricksВидеоSpark Notebooks with DatabricksВидеоUsing Data with DatabricksВидеоWorking with Workspaces in DatabricksВидеоAdvanced Capabilities of DatabricksВидеоPySpark SQLЗадание

Foundational PySpark Skills and Databricks

Introduction to Databricks Machine LearningЧтениеPySpark Introduction on DatabricksВидеоExploring Databricks Azure FeaturesВидеоWhat is the Databricks File System (DBFS)?ЧтениеUsing the DBFS to AutoML WorkflowВидеоLoad, Register and Deploy ML ModelsВидео

Using MLFlow with Databricks

What is MLOps?ВидеоMLOps Workflow on Azure DatabricksЧтениеExploring Open-Source MLFlow FrameworksВидеоRunning MLFlow with DatabricksВидеоRun MLFlow Projects on Azure DatabricksЧтениеEnd to End Databricks MLFlowВидео

Check for Understanding

DataBricksЗадание
04DataOps and Operations Methodologies33 материалов

Getting Started with Kaizen Methodology for Data

Kaizen Methodology for DataВидеоKaizen MethodologyЗаданиеIntroducing GitHub CodeSpacesВидеоGitHub Codespaces OverviewЧтениеCompiling Python in GitHub CodespacesВидеоGetting Started with Amazon SageMaker Studio LabЧтениеWalking through Sagemaker Studio LabВидеоTeaching MLOps at Scale with GitHub (Optional)ЧтениеPytest Master Class (Optional)Видео

Getting Started with DevOps for Data

What is DevOps?ВидеоDevOps Key ConceptsВидеоGetting Started with DevOps and Cloud ComputingЧтениеDevOpsЗаданиеContinuous Integration OverviewВидеоBuild an NLP in Cloud9 with PythonВидео

Composing DataOps Solutions

What is DataOps?ВидеоDataOps and MLOps with SnowflakeВидеоDataOpsЗаданиеBuilding Cloud Pipelines with Step Functions and LambdaВидеоWhat is a Data Lake?ВидеоData Warehouse vs. Feature StoreВидео

Check for Understanding

DataOps and Operations MethodologiesЗаданиеNext StepsЧтениеShare your learning experienceЧтение
Видео
Serverless Compute with DatabricksЧтение
Databricks Model RegistryВидео
Model Serving on DatabricksВидео
PySpark DataFramesЗадание
Databricks Autologging with MLFlowВидео
Databricks AutologgingЧтение
MLFlow with DatabricksЗадание
ETL-Part-1: Keyword Extractor Tool to HashTag Tool Лабораторная
Build a Continuously Deployed Containerized FastAPI MicroserviceВидео
Hugo Continuous Deploy on AWSВидео
Container Based Continuous DeliveryВидео
Big Data ChallengesВидео
Types of Big Data ProcessingВидео
Real-World Data Engineering PipelineВидео
Data Feedback LoopВидео
Benefits of Serverless ETL TechnologiesЧтение
ETL-Part2: SQLite ETL DestinationЛабораторная