Twitter API With Python On Linux TUT Example

View previous topic View next topic Go down

Twitter API With Python On Linux TUT Example

Post by jamied_uk on 17th September 2016, 22:42



adilmoujahid.com/posts/2014/07/twitter-analytics

Code:
python twitter_streaming.py


If you run the program from your terminal using the command: python twitter_streaming.py, you will see data flowing like the picture below.


Code:
sudo pip install tweepy


Code:
python twitter_streaming.py


example.py



2. Reading and Understanding the data



The data that we stored twitter_data.txt is in JSON format. JSON stands for JavaScript Object Notation. This format makes it easy to humans to read the data, and for machines to parse it. Below is an example for one tweet in JSON format. You can see that the tweet contains additional information in addition to the main text which in this example: "Yaayyy I learned some JavaScript today! #thatwasntsohard #yesitwas #stoptalkingtoyourself #hashbrown #hashtag".


example_2.py


3. Mining the tweets

Our main goals in these text mining tasks are: compare the popularity of Python, Ruby and Javascript programming languages and to retrieve programming tutorial links. We will do this in 3 steps:


    We will add tags to our tweets DataFrame in order to be able to manipualte the data easily.

    Target tweets that have "pogramming" or "tutorial" keywords.

    Extract links from the relevants tweets



Adding Python, Ruby, and Javascript tags


First, we will create a function that checks if a specific keyword is present in a text. We will do this by using regular expressions. Python provides a library for regular expression called re. We will start by importing this library


https://en.wikipedia.org/wiki/Regular_expression

example_3.py

4

Targeting relevant tweets


We are interested in targeting tweets that are related to programming languages. Such tweets often have one of the 2 keywords: "programming" or "tutorial". We will create 2 additional columns to our tweets DataFrame where we will add this information.


5.

example_4.py

Extracting links from the relevants tweets
Now that we extracted the relevant tweets, we want to retrieve links to programming tutorials. We will start by creating a function that uses regular expressions for retrieving link that start with "http://" or "https://" from a text. This function will return the url if found, otherwise it returns an empty string.

This returns 644 links for python, 413 links for javascript, and 136 for ruby. Below are some python related links

    http://t.co/WmTccp3rb1

    http://t.co/5qE3vPAy7N

    http://t.co/1rvmhqPsXD

    http://t.co/S9aq2AahjH

    http://t.co/ORg6IL8qXT

    http://t.co/EnK2UlDcJ8

    http://t.co/gtu9WVQCLK

    http://t.co/xvMTzqLGg0

    http://t.co/bgMZ0jIpA7

    http://t.co/O03VrRyEAb

    http://t.co/CfWYefZML7

    http://t.co/N3iU2ZYa2z

    http://t.co/S9aq2AahjH

    http://t.co/ytms7bcsQV


http://adilmoujahid.com/posts/2014/07/twitter-analytics/

https://en.wikipedia.org/wiki/Regular_expression

Requests

http://docs.python-requests.org/en/latest/index.html


AI

https://codingsec.net/2016/03/get-started-artificial-intelligence/

We want to capture this data into a file that we will use later for the analysis. You can do so by piping the output to a file using the following command:

Code:
python twitter_streaming.py > twitter_data.txt
avatar
jamied_uk
Admin

Posts : 2199
Join date : 2010-05-09
Age : 34
Location : UK

http://address-shortner.co.uk

Back to top Go down

Re: Twitter API With Python On Linux TUT Example

Post by jamied_uk on 17th September 2016, 22:45

More

Links

apps.twitter.com/app/new


adilmoujahid.com/posts/2014/07/twitter-analytics
avatar
jamied_uk
Admin

Posts : 2199
Join date : 2010-05-09
Age : 34
Location : UK

http://address-shortner.co.uk

Back to top Go down

View previous topic View next topic Back to top

- Similar topics

 
Permissions in this forum:
You cannot reply to topics in this forum