Overview
-
Explore how to build batch mode data pipelines with Apache Flink, the powerful and popular stream-processing platform.
Syllabus
-
Introduction
- Batch mode engineering
- What is Apache Flink?
- Apache Flink features
- Architecture of Apache Flink
- Flink program structure
- Flink execution flow
- Installing Flink standalone
- Creating a Flink project
- Build a sample Flink program
- Running jobs on the cluster
- Using the Flink web interface
- Setting up the exercise files
- DataSet API concepts
- Reading a CSV File
- Using Map
- Using FlatMap
- Using filters
- Using aggregates
- Using Reduce
- Using POJO classes
- Join operations
- Using MySQL with Flink
- Using broadcast variables
- Problem definition
- Computing total score
- Printing scores for physics
- Computing average scores across subjects
- Find the top student for each subject
- Next steps
