Course Information
Registration period
Year-round Recruitment
Course Level
Study Mode
Duration
4 Hour(s) 18 Minute(s)
Language
English
Taught by
Prashant Kumar Pandey, Learning Journal
Certificate
- Available
- *The delivery and distribution of the certificate are subject to the policies and arrangements of the course provider.
Rating
2 views
Course Overview
Learn to create Real-time Stream Processing applications using Apache Spark
About the Course
I am creating Apache Spark 3 - Real-time Stream Processing using the Scala course to help you understand the Real-time Stream processing using Apache Spark and apply that knowledge to build real-time stream processing solutions. This course is example-driven and follows a working session like approach. We will be taking a live coding approach and explain all the needed concepts along the way.
Who should take this Course?
I designed this course for software engineers willing to develop a Real-time Stream Processing Pipeline and application using the Apache Spark. I am also creating this course for data architects and data engineers who are responsible for designing and building the organization’s data-centric infrastructure. Another group of people is the managers and architects who do not directly work with Spark implementation. Still, they work with the people who implement Apache Spark at the ground level.
Spark Version used in the Course
This Course is using the Apache Spark 3.x. I have tested all the source code and examples used in this Course on Apache Spark 3.0.0 open-source distribution.
Course Content
- 7 section(s)
- 33 lecture(s)
- Section 1 Before you start
- Section 2 Setup your Environment
- Section 3 Getting started with Spark Structured Streaming
- Section 4 Spark Streaming with Kafka
- Section 5 Windowing and Aggregates
- Section 6 Stream Processing and Joins
- Section 7 Keep Learning
What You’ll Learn
- Real-time Stream Processing Concepts
- Spark Structured Streaming APIs and Architecture
- Working with File Streams
- Working With Kafka Source and Integrating Spark with Kafka
- State-less and State-full Streaming Transformations
- Windowing Aggregates using Spark Stream
- Watermarking and State Cleanup
- Streaming Joins and Aggregation
- Handling Memory Problems with Streaming Joins
- Creating Arbitrary Streaming Sinks
Reviews
-
LLoganayaki
Excellent Course!
-
UUdayan Biswas2
Yes
-
TThomas Valikodath
Short lessons with real use cases and covers a lot of apis.
-
JJaid Mirajkar
An excellent walkthrough of the entire spark structured streaming journey using hands-on examples and explanation of various concepts via well structured slides. Thanks a lot for the wonderful course Prashant.
