Skip to content

Category: Kafka

Data Engineering · 7 min read

Kafka Connect Concepts & Example

Kafka Connect is used to connect Kafka with external systems such as database, storage, and messaging systems for data import and export…

December 28, 2020Read →
Data Engineering · 3 min read

Key Concepts of Stream Processing

Stream processing is similar to any data processing where you read the data ,apply some transformation and then push it somewhere. However…

December 20, 2020Read →
Data Engineering · 6 min read

Kafka Streams

Kafka Streams is a client library (API) to build applications that analyze and process data stored in Kafka in real-time. Streams application…

December 15, 2020Read →
Data Engineering · 7 min read

Kafka & Reliability

Apache Kafka guarantees the following with respect to Reliability Kafka does not guarantee ordering of messages between partitions. It does provide ordering…

December 10, 2020Read →
Data Engineering · 4 min read

Kafka Producer API Internals

The Kafka Producer API allows applications to send messages to the Kafka cluster. Producer APIs are simple to use, but this post…

December 7, 2020Read →
System Design · 4 min read

Kafka Storage Architecture

In this article we will check how Kafka stores and organizes the data. Kafka records are organized and stored in a Topic.…

November 30, 2020Read →
Data Engineering · 3 min read

Apache Kafka & ZooKeeper

Apache ZooKeeper is an open source volunteer project under the Apache Software Foundation. It is a centralized service for maintaining configuration information,…

November 27, 2020Read →
Data Engineering · 2 min read

How to run a multi-node Kafka cluster in Local?

Download Kafka Confluent Community Edition https://www.confluent.io/download/ Unzip the downloaded file To run 3 clusters go to confluent-6.0.0/etc/kafka directory Copy server.properties 3 times…

November 24, 2020Read →
The BelowTheMalt newsletter

Stay curious. Stay inspired.

Fresh perspectives on AI, architecture and software engineering, delivered to your inbox.

Join 109 other subscribers