Prompt, RAG & Evaluation Operations

О курсе

Production generative AI systems require more than good prompts; they need systematic experimentation, rigorous evaluation, and managed model customization. This course teaches the core operational practices that make GenAI solutions reliable and continuously improvable. You'll design prompt versioning and variant experimentation frameworks, including A/B testing methodology, hyperparameter tracking, and rollback procedures using Microsoft Foundry. You'll run structured RAG optimization experiments across chunking strategies, embedding model variants, and retrieval configurations, evaluating each using groundedness, relevance, coherence, and fluency metrics via the Azure OpenAI Evaluation SDK. From there, you'll build automated evaluation frameworks with quality and safety thresholds, human-in-the-loop review triggers, and integration with CI/CD release decisions. You'll also design the fine-tuning operations lifecycle, including dataset versioning, LoRA configuration, model registry management, and evaluation against baseline models.

Навыки, которые вы освоите

Учитесь у экспертов

Microsoft

Преподаватель курса