Kafka Storage Architecture

In this article we will check how Kafka stores and organizes the data. Kafka records are organized and stored in a Topic. Producer applications write data to a Topic and consumer applications read data from a Topic. Topics are similar to the folders in a file system and the messages that are sent by producers … Continue reading Kafka Storage Architecture

Apache Kafka & ZooKeeper

Apache ZooKeeper is an open source volunteer project under the Apache Software Foundation. It is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and group services. These services are used in some form or another by distributed applications. In Kafka also it acts like a centralized service that manages cluster memberships of … Continue reading Apache Kafka & ZooKeeper

How to create a consumer group and consume messages from a Multi Node Kafka Cluster in Local?

This example requires running a multi-node kafka cluster in local. For instructions on how to do this, please refer to my previous article. In this article we will send an insurance file with 36635 rows to Kafka with 3 clusters and have a group of 2 consumers consume this data. You can download the file … Continue reading How to create a consumer group and consume messages from a Multi Node Kafka Cluster in Local?

How to run a multi-node Kafka cluster in Local?

Download Kafka Confluent Community Editionhttps://www.confluent.io/download/ Unzip the downloaded file To run 3 clusters go to confluent-6.0.0/etc/kafka directory Copy server.properties 3 times and rename it as Next step is to change the configuration inside these property files. Properties to be changed are 3Broker id: Every cluster should have a unique broker id.Assign a unique id for … Continue reading How to run a multi-node Kafka cluster in Local?

BIG-O NOTATION & ALGORITHM ANALYSIS

Big O Notation is a mathematical notation, which is used to abstractly describe the behavior of an algorithm in computer science. It is used to express the efficiency or complexity of an algorithm by describing how a program will scale with growing input size. A program to search for a number in a data set … Continue reading BIG-O NOTATION & ALGORITHM ANALYSIS