Overview
-
Learn the Apache Kafka basics. Learn how to use this enterprise-scale data streaming technology to read and write data from multiple sources—without managing multiple integrations.
Syllabus
-
Introduction
- Intro to Apache Kafka
- Apache Kafka in five minutes
- Kafka theory overview
- Topics, partitions, and offsets
- Brokers and topics
- Topic replication
- Producers and message keys
- Consumer and consumer group
- Consumer offsets and delivery semantics
- Kafka broker discovery
- ZooKeeper
- Kafka guarantees
- Theory roundup
- Important: Starting Kafka
- macOS: Download and set up Kafka in PATH
- macOS: Using brew
- macOS: Start ZooKeeper and Kafka
- Linux: Download and set up Kafka in PATH
- Linux: Start ZooKeeper and Kafka
- Windows: Download and set up Kafka in PATH
- Windows: Start ZooKeeper and Kafka
- CLI introduction
- Kafka topics CLI
- Kafka console producer CLI
- Kafka console consumer CLI
- Kafka consumers in groups
- Kafka consumer groups CLI
- Resetting offsets
- Kafka Tool UI
- Intro to Kafka programming
- Creating a Kafka project
- Java producer
- Java producer callbacks
- Java producer with keys
- Java consumer
- Java consumer inside consumer group
- Java consumer with threads
- Java consumer seek and assign
- Client bidirectional compatibility
- Real-world project overview
- Producer and advanced configurations overview
- Twitter setup
- Producer, part 1: Writing a Twitter client
- Producer, part 2: Writing the Kafka producer
- Producer configurations introduction
- Acks and min.insync.replicas
- Retries and max.in.flight.requests.per.connection
- Idempotent producer
- Producer, part 3: Safe producer
- Producer compression
- Producer batching
- Producer, part 4: High throughput producer
- Producer default partitions and key hashing
- Refactoring the project
- Consumer and advanced configuration overview
- Setting up Elasticsearch in the cloud
- Elasticsearch 101
- Consumer, part 1: Set up project
- Consumer, part 2: Write the consumer and send to Elasticsearch
- Consumer, part 3: Delivery semantics
- Delivery semantics for consumers
- Consumer, part 3: Idempotence
- Consumer poll behavior, part 1
- Consumer offset commit strategies
- Consumer, part 4: Manual commit of offsets
- Consumer, part 5: Performance improvement using batching
- Consumer offsets reset behavior
- Consumer, part 6: Replaying data
- Consumer internal threads
- Kafka in the real world
- Kafka Connect introduction
- Kafka Connect Twitter: Hands-on example
- Kafka Streams introduction
- Kafka Streams: Hands-on example
- Kafka Schema Registry introduction
- ZooKeeper
- Case study: MovieFlix
- Case study: GetTaxi
- Case study: MySocialMedia
- Case study: MyBank
- Case study: Big data ingestion
- Case study: Logging and metrics aggregation
- Kafka cluster setup, high-level architecture overview
- Kafka monitoring and operations
- Kafka security
- ZooKeeper
- Changing a topic configuration
- Segment and indexes
- Log cleanup policies
- Log cleanup delete
- Log compaction theory
- Log compaction practice
- min.insync.replicas reminder
- Unclean leader election
- What are annexes?
- Annex 1: Overview
- Starting Kafka with the Confluent CLI
- Starting a multibroker Kafka cluster using binaries
- Start Kafka development environment using Docker
- Starting a multibroker Kafka cluster using Docker
- Kafka advertised host setting
- Starting Kafka on a remote machine
