{"repo":"data-han/twitter_kafka_sentiment","free":true,"listed":false,"github":"https://github.com/data-han/twitter_kafka_sentiment","clone":"git clone https://github.com/data-han/twitter_kafka_sentiment.git","description":"Ingesting real-time Twitter API using tweepy into Kafka and process using Apache Spark Structured Streaming with Sentiment Analysis TextBlob before loading into time-series database, InfluxDB and monitoring dashboard, Grafana","language":"Python","stars":12,"topics":["grafana","influxdb","kafka","real-time","spark","streaming","structured-streaming"],"license":null,"category":"analytics","readme_excerpt":"Twitter Kafka Real-Time Streaming Background It's the time of the year and the NBA playoffs are happening right now. While I was watching the highlights on YouTube, I often find myself scrolling through the comments section. Interestingly, while some rooted for their favourite teams, there were also criticisms on certain players or plays. This made me wonder if I could gather general sentiments from Twitter in real-time on the games. Therefore, I wanted to build a pipeline to obtain real-time Tweets on #NBA while learning about a few technologies including Kafka and Spark streaming. Here, I managed to build a real-time dashboard using InfluxDB & Grafana, performing a simple sentiment analysis and showing the polarity (positive/negative/neutral) of each tweet over time. My focus here would be trying out the different technologies and looking at how people react when watching the games. For the sentiment analyses, I took references from other authors as credited below. Although some tweets may not be categorised accurately, it is still pretty interesting to see what people are saying especially during game-time. Introduction/ Set up In this set-up, I am using the following tools: 1. Twitter - tweepy python library 1. Purpose: to obtain streaming Tweets 2. Obtain Twitter Developer credentials including consumer key, consumer secret, access token, access token secret 2. Kafka 1. Kafka is a distributed event streaming platform that lets you read, write, store, and process events (","default_branch":null,"files":null,"tree":[],"storefront":"/r/data-han","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/data-han/twitter_kafka_sentiment/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}