К содержимому
learnspaceYOUR NEXT CHAPTER
ПРОСТРАНСТВО ОБУЧЕНИЯ
ГлавнаяКаталог курсовМоё обучениеCoursera

Знания без границ

Учитесь у лучших университетов и компаний мира.

Открыть Coursera
Интеграция
Пространство университета
Моё пространствоСтраница курса
↵
ЯЛичный кабинетСтудент
© 2026 LearnSpaceКаждый день — возможность узнать больше.Помощь
Data Ingestion & Pipeline Implementation · LearnSpace
Назад в каталог
courseraАнализ данных

Data Ingestion & Pipeline Implementation

Курс от Coursera
Уровень не указан≈ 18.5 чАнглийский
О курсеНавыкиПрограммаПреподаватели

О курсе

Learn how to extract, migrate, and ingest data across multiple systems through this applied course in the Data Engineering Skill Path. You will develop critical competencies including collecting data from APIs, parsing semi-structured formats, designing and executing ETL/ELT migration jobs, building extraction routines with SDKs, loading large datasets into cloud warehouses, and implementing real-time ingestion pipelines using streaming systems. Through hands-on practice with IBM tooling, Snowflake, Kafka, and Python-based extraction workflows, you will learn to build scalable ingestion processes that reliably move data from source to structured storage. This course combines expertise from IBM and Snowflake, offering multiple perspectives on data ingestion across batch, streaming, and cloud-native environments. You will progress from collecting and preparing raw input data to designing migration plans, executing ETL/ELT scripts, applying bulk-loading techniques, and implementing event-driven pipelines. The curriculum balances conceptual foundations with practical exercises, preparing you to manage ingestion pipelines end-to-end. Perfect for aspiring data engineers and learners seeking strong, practical skills in extraction, migration, and ingestion across modern data platforms and architectures.

Навыки, которые вы освоите

Data GovernanceReal Time DataWeb ScrapingData PipelinesShell ScriptSnowflake SchemaScriptingApplication Programming Interface (API)Extract, Transform, LoadData SecurityData TransformationData ArchitectureBash (Scripting Language)Performance TuningData CollectionSQLData MigrationStored ProcedureData IntegrationApache Kafka

Программа курса

8 модулей · 102 учебных материалов

01Start Here: Get Oriented and Check Your Skills2 материалов
Start Here: How This Skill-Based Course WorksЧтениеSkill Diagnostic: Find Your Recommended Starting PointЗадание
02APIs and Data Collection22 материалов

Simple APIs

Application Program InterfaceВидео

Учитесь у экспертов

Professionals from the Industry

Преподаватель курса

Data Ingestion & Pipeline Implementation
В каталоге вашей программы

Инвестируйте в себя

Новые знания — в удобное для вас время.

Начать на Coursera

Обучение откроется на Coursera
в новой вкладке

Обучение на Coursera

≈ 18.5 ч

8 модулей

Язык: Английский

Субтитры: Арабский, Французский, Узбекский, Украинский, Китайский (Китай), Греческий, Итальянский, Бразильский португальский, Вьетнамский, Нидерландский, Корейский, Немецкий, Пушту, Русский, Тайский, Индонезийский, Шведский, Турецкий, Азербайджанский, Испанский, Хинди, Японский, Казахский, Венгерский, Польский

Часть программы вашего университета
Reading: Some Context on APIsPLUGIN
Hands-On Lab: Introduction to APIВнешний инструмент
Practice Quiz: Simple APIs Задание

REST APIs, Web Scraping, and Working with Files

REST APIs & HTTP Requests - Part 1ВидеоREST APIs & HTTP Requests - Part 2ВидеоReading: Web Scraping and HTML BasicsPLUGINHands-on Lab: Access REST APIs & Request HTTPВнешний инструментHands-On Lab: API ExamplesВнешний инструмент(Optional) HTML for Web ScrapingВидео(Optional) Web Scraping ВидеоReading: Web Scraping - A Key Tool in Data SciencePLUGINReading: Web Scraping Tables using PandasPLUGINHands-on Lab: Web ScrapingВнешний инструментWorking with Different File Formats ВидеоHands-on Lab: Working with different file formatsВнешний инструмент Practice Quiz: REST APIs, Web Scraping, and Working with FilesЗаданиеPractice Project: GDP Data Extraction and ProcessingВнешний инструмент

Summary, Cheatsheet, and Glossary

Summary: APIs and Data CollectionЧтениеCheat Sheet: APIs and Data CollectionPLUGINGlossary: APIs and Data CollectionPLUGIN

Wrap-Up

Python Cheat Sheet: The BasicsЧтение
03Bonus: Advanced SQL for Data Engineers18 материалов

About this Bonus module

About this Bonus ModuleЧтение

Views, Stored Procedures, and Transactions

ViewsВидеоHands-on Lab: Using ViewsВнешний инструментStored ProceduresВидеоHands-on Lab: Stored ProceduresВнешний инструментACID TransactionsВидеоHands-on Lab: Committing and Rolling Back a TransactionВнешний инструмент[Optional] Hands-on Labs Using IBM Db2ЧтениеSummary: Views, Stored Procedures, and TransactionsЧтениеPractice Quiz: Views, Stored Procedures, and TransactionsЗаданиеSQL Cheat Sheet: Views, Stored Procedures and TransactionsPLUGIN

JOIN Statements

Join OverviewВидеоInner JoinВидеоOuter JoinsВидеоHands-on Lab: JoinsВнешний инструментSummary: JOIN StatementsЧтениеPractice Quiz: Join StatementsЗаданиеSQL Cheat Sheet: JOIN Statements
04ETL & Data Pipelines: Tools and Techniques13 материалов

ETL using Shell Scripts

Linux Commands and Shell ScriptingЧтениеETL TechniquesЧтениеETL Using Shell ScriptingВидеоHands-On Lab: ETL using Shell ScriptsВнешний инструментSummary & HighlightsЧтениеPractice Quiz: ETL using Shell ScriptsЗадание

An Introduction to Data Pipelines

Introduction to Data PipelinesВидеоKey Data Pipeline ProcessesВидеоBatch versus Streaming Data Pipeline Use CasesВидеоData Pipeline Tools and TechnologiesВидеоInteractivity: Differentiate between Batch Processing and Stream ProcessingPLUGINSummary & HighlightsЧтениеPractice Quiz: An Introduction to Data PipelinesЗадание
05Batch data ingestion with Snowflake12 материалов
What is data ingestion?ВидеоBatch ingestion with SnowflakeВидеоLoading data from Snowflake MarketplaceВидеоLoading data using Snowflake’s web interfaceВидеоOptimize compute resources for efficient batch ingestionВидеоLoading data using Snowflake CLIВидеоLoading data using the COPY INTO commandВидеоScenario briefing and account setupЧтениеIngesting data from other data systems using connectorsВидеоRecap and best practices for batch ingestionВидеоAdditional resources on batch ingestion with SnowflakeЧтениеTroubleshooting errors encountered in the ModuleЧтение
06Building Streaming Pipelines using Kafka10 материалов

Using Apache Kafka to build Pipelines for Streaming Data

Distributed Event Streaming Platform ComponentsВидеоApache Kafka OverviewВидеоBuilding Event Streaming Pipelines using KafkaВидеоKafka Streaming ProcessВидеоHands-on Lab: Working with Streaming Data using KafkaВнешний инструмент[Optional] Hands-on Lab: Kafka Message Keys and OffsetВнешний инструментKafka Python ClientPLUGIN[Optional] Hands-on Lab: Kafka Python ClientВнешний инструментSummary & HighlightsЧтениеPractice Quiz: Building Streaming Pipelines using KafkaЗадание
07Data Engineering Lifecycle23 материалов

Data Platforms, Data Stores, and Security

Architecting the Data PlatformВидеоFactors for Selecting and Designing Data StoresВидеоSecurityВидеоViewpoints: Importance of Data SecurityВидеоSummary and HighlightsЧтениеPractice QuizЗадание

Data Collection and Data Wrangling

How to Gather and Import DataВидеоData WranglingВидеоTools for Data WranglingВидеоHands-On Lab: Load data into the Datasette from a CSV fileВнешний инструмент[Optional] Hands-on Lab: Load data into the Db2 Database from a CSV filePLUGINSummary and HighlightsЧтениеPractice QuizЗадание

Querying Data, Performance Tuning, and Troubleshooting

Querying and Analyzing DataВидеоPerformance Tuning and TroubleshootingВидеоLab: Explore your dataset using SQL queries using DatasetteВнешний инструмент[Optional] Hands-on Lab: Explore Your Dataset Using SQL Queries in DB2PLUGINSummary and HighlightsЧтениеPractice QuizЗадание

Governance and Compliance

Governance and ComplianceВидеоSummary and HighlightsЧтениеPractice QuizЗаданиеOptional: Overview of the DataOps MethodologyЧтение
08Assessment2 материалов

Lesson

Learner Expectations for Skill AssessmentЧтениеSkill AssessmentЗадание
PLUGIN