Published on December 8, 2016 by Microsoft

Raghav Mohan stops by to announce the addition of Apache Kafka to the HDInsight family. Azure HDInsight is a managed cloud Hadoop and Spark service built for the enterprise. This is an exciting addition as it enables enterprises to build scalable, real-time streaming pipelines on the HDInsight platform within minutes and with just a few clicks.

Editing note: Tweet stream blurred for your protection. 😐

Leave a Reply

4 Comments on "Introducing Apache Kafka on Azure HDInsight"

Notify of
avatar

robcaron
Guest
robcaron
10 months 8 days ago

@Rohit:To replicate the above demo, you need a setup like the followingTwitterProducer –> Kafka <– Storm ConsumerFor a TwitterProducer, you can use a simple spark notebook as detailed below(doc for the setup)docs.microsoft.com/en-us/azure/hdinsight/hdinsight-apache-kafka-spark-structured-streaming(notebook for the actual producer)github.com/Azure-Samples/hdinsight-spark-kafka-structured-streamingOnce you have this working, you can use Storm to consume the messages from Kafka as shown heredocs.microsoft.com/en-us/azure/hdinsight/hdinsight-apache-storm-with-kafkaLet us know if you have more questions.

Rohit
Guest
Rohit
10 months 8 days ago

Can i get the code how to fetch the data from twitter to Kafka + Storm

mjda
Guest
mjda
1 year 2 months ago

I was waiting for this! What type of subscription is required? Tried a BizSpark and a "Pay as you go" to no avail :(See message: Please select a subscription that has access to HDInsight Kafka 

Paul Vetersneck
Guest
Paul Vetersneck
1 year 5 months ago

Interesting, with HIPAA compliance, could Hospitals & Primary Care Physicians import data and run analytics on patient data?

wpDiscuz