Course Information
Registration period
Year-round Recruitment
Course Level
Study Mode
Duration
6 Hour(s) 32 Minute(s)
Language
English
Taught by
V2 Maestros, LLC
Rating
3 views
Course Overview
Analyze, Design and Build scalable, resilient and cost-effective Big Data pipelines with a methodical process
Big data technologies have been growing exponentially over the past few years and have penetrated into every domain and industry in software development. It has become a core skill for a software engineer. Robust and effective big data pipelines are needed to support the growing volume of data and applications in the big data world. These pipelines have become business critical and help increase revenues and reduce cost.
Do quality big data pipelines happen by magic? High quality designs that are scalable, reliable and cost effective are needed to build and maintain these pipelines.
How do you build an end-to-end big data pipeline that leverages big data technologies and practices effectively to solve business problems? How do you integrate them in a scalable and reliable manner? How do you deploy, secure and operate them? How do you look at the overall forest and not just the individual trees? This course focuses on this skill gap.
What are the topics covered in this course?
We start off by discussing the building blocks of big data pipelines, their functions and challenges.
We introduce a structured design process for building big data pipelines.
We then discuss individual building blocks, focusing on the design patterns available, their advantages, shortcomings, use cases and available technologies.
We recommend several best practices across the course.
We finally implement two use cases for illustration on how to apply the learnings in the course to a real world problem. One is a batch use case and another is a real time use case.
Course Content
- 10 section(s)
- 90 lecture(s)
- Section 1 Introduction & Expectations
- Section 2 Building Blocks for Big Data Pipelines
- Section 3 System Design Process
- Section 4 Scalable Pipelines - Design Principles
- Section 5 Data Acquisition Design
- Section 6 Data Transport Design
- Section 7 Data Processing & Transformation Design
- Section 8 Storage Design
- Section 9 Serving Design
- Section 10 Infrastructure and Deployments
What You’ll Learn
- Learn about the building blocks of a big data pipeline, their functions and challenges
- Adapt an end-to-end methodical approach to designing a big data pipeline
- Explore techniques to ensure overall scaling of a big data pipeline
- Study design patterns for building blocks, their advantages, shortcomings, applications and available technologies
- Focus additionally on Infrastructure, Operations and Security for Big Data deployments
- Exercise the learnings in the course with a Batch and Realtime use case study
Reviews
-
RRoosvelt Sanchez Oviedo
A collection of design recommendations, with a brief explanation
-
RRaimi Lawal Mustapha
This is an amazing course for anyone that is interested in having in-depth knowledge of how big data pipelines are designed, built and maintained. The guiding principles for big data pipelines design are taught in details, also, the tutor is very knowledgeable on the subject in question.
-
GGaurav Joshi
yes it is giving an insight into the end to end designing of a big data pipeline
