Hi maybe I'm missing something really obvious but I've been using tweepy to collect tweets from a keyword.
I cannot understand, and have been searching for most of the day, how to access details about the tweets that I have, for instance their retweet count or favorite count. This using the tweet ' s id number.
Any help to a to do this would be very helpful.
You can use the statuses/show endpoint (usable in Tweepy through api.get_status. Unfortunately, any data not provided by the endpoint can only be accessed by scraping the website - there's no way to have Twitter send more data.
import tweepy
consumer_key=""
consumer_secret=""
access_key = ""
access_secret = ""
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_key, access_secret)
api = tweepy.API(auth)
id_list = ['tweetidexample', 'tweetidexample',"....etc" ]
def get_retweet_count(tweet_id):
tweet = api.get_status(tweet_id)
return tweet.retweet_count
for id in id_list:
print get_retweet_count(id), id
This enables retweet count from ID at least.
Related
I have a Django site in which I want to use spotipy to look for statistics of the song like popularity and views. I have this code right now:
import spotipy
import spotipy.util as util #luxury
import json
import webbrowser
username = 'dgrqnco2rx8hdu58kv9if9eho'
scope = 'user-read-private user-read-playback-state user-modify-playback-state'
token = util.prompt_for_user_token(username, scope, client_id='08bb526962574a46b359bffc56048147',
client_secret='bf6d4184c8ae40aca207714e02153bad', redirect_uri='http://google.com/')
sp_obj = spotipy.Spotify(auth=token)
ss = 'name of song'
if ss.__contains__('('):
q = ss[0:ss.index('(')]
elif ss.__contains__('['):
q = ss[0:ss.index('[')]
elif ss.__contains__('['):
q = ss[0:ss.index('{')]
else:
q = ss
query = sp_obj.search(q, 1, 0, 'track')
#<<<<<<<<<<SONG>>>>>>>>>>
#FIND THE SONG URI
song_uri = query['tracks']['items'][0]['uri']
track = sp_obj.track(song_uri)
track_data = sp_obj.audio_features(song_uri)
song_popularity = track['popularity']
song_danceability = track_data[0]['danceability']
song_energy = track_data[0]['energy']
song_loudness = track_data[0]['loudness']
song_tempo = track_data[0]['tempo']
However spotipy redirects me to a page for authorization and I need to paste the url in the console. The regular user however does not have access to this console. So how can I do the authorization in an alternative way or even bypass it?
I was thinking about getting a spotify account in which every user will be getting logged in so that the user won't have to do the authorization and won't have to have a spotify account. Is this possible? If not what else can I try?
You can't use util.prompt_for_user_token because it's just a helper for local usage only.
You need to arrange your code as API endpoints so that multiple users can sign in. Here is a full working example that would allow multiple users to sign in https://github.com/plamere/spotipy/blob/master/examples/app.py.
It uses Flask but you can easily adapt it to Django.
I want to get every tweet of the HousingWire on Twitter (https://twitter.com/HousingWire). I understood how to authenticate into the twitter account but how I can get the tweet of HousingWire?
I know how to stream the data based on the keywords,but I want to stream the HousingWire tweet. how I can do that?
import time
from tweepy import Stream
from tweepy import OAuthHandler
from tweepy.streaming import StreamListener
ckey=''
csecret=''
atoken=''
asecret=''
class listener(StreamListener):
def on_data(self,data):
try:
print data
#tweet=data.split(',"text":"')[1].split('","source')[0]
#print tweet
#savethis=str(time.time())+'::'+tweet
savefile=open('tweetdb.txt','a')
savefile.write(data)
savefile.write('\n')
savefile.close()
return True
except BaseException,e:
print 'failed on data',str(e)
time.sleep(5)
def on_error(self,status):
print status
auth=OAuthHandler(ckey,csecret)
auth.set_access_token(atoken,asecret)
twitterStream=Stream(auth,listener())
twitterStream.filter(track=["stock"])
You can use the below Python script to grab the last 3,240 tweets from HousingWire (Twitter only allows access to that many tweets from a user - no way to grab the complete history). Usage: Simply put their twitter screen name in the script.
#!/usr/bin/env python
# encoding: utf-8
import tweepy #https://github.com/tweepy/tweepy
import csv
#Twitter API credentials
consumer_key = ""
consumer_secret = ""
access_key = ""
access_secret = ""
def get_all_tweets(screen_name):
#Twitter only allows access to a users most recent 3240 tweets with this method
#authorize twitter, initialize tweepy
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_key, access_secret)
api = tweepy.API(auth)
#initialize a list to hold all the tweepy Tweets
alltweets = []
#make initial request for most recent tweets (200 is the maximum allowed count)
new_tweets = api.user_timeline(screen_name = screen_name,count=200)
#save most recent tweets
alltweets.extend(new_tweets)
#save the id of the oldest tweet less one
oldest = alltweets[-1].id - 1
#keep grabbing tweets until there are no tweets left to grab
while len(new_tweets) > 0:
print "getting tweets before %s" % (oldest)
#all subsiquent requests use the max_id param to prevent duplicates
new_tweets = api.user_timeline(screen_name = screen_name,count=200,max_id=oldest)
#save most recent tweets
alltweets.extend(new_tweets)
#update the id of the oldest tweet less one
oldest = alltweets[-1].id - 1
print "...%s tweets downloaded so far" % (len(alltweets))
#transform the tweepy tweets into a 2D array that will populate the csv
outtweets = [[tweet.id_str, tweet.created_at, tweet.text.encode("utf-8")] for tweet in alltweets]
#write the csv
with open('%s_tweets.csv' % screen_name, 'wb') as f:
writer = csv.writer(f)
writer.writerow(["id","created_at","text"])
writer.writerows(outtweets)
pass
if __name__ == '__main__':
#pass in the username of the account you want to download
get_all_tweets("J_tsar")
Suppose user 'ABCD' has tweeted on topic, how to find the comments for that tweet ?
I am using tweepy to get twitter data.
It will also be good, If it possible using any other python packages.
Code for the following is:
Import tweepy
consumer_key = ""
consumer_secret = ""
access_key = ""
access_secret = ""
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_key, access_secret)
api = tweepy.API(auth)` # This is authentication process.
new_tweets = api.user_timeline(screen_name ='ABCD',count=20) # Code to download 20 timelines for user 'ABCD'
Till here i can get 20 timelines from user ABCD.
Q: How to get tweets for each timeline.
Thanks.
I am trying to extract the all tweets and timestamp of particular person. I am new in python and tweepy. I have the working code from internet search, but my desire is to print only all tweets of particular user.
import tweepy
# Authentication details. To obtain these visit dev.twitter.com
consumer_key = 'nWGEdfoaBt7d6wWhiAw5Tw'
consumer_secret = 'qM4QfDPqG9JQp6n0fqTCMrj6LJjES6vu2IzqpZLc'
access_token = '2284416938-JbD4F32m9xQPMxKoh6UikpCLoJm8F6xy8wDPS9P'
access_token_secret = 'XvJZQWa6zz5vHcHkUcYBacQKZJE9pcxbpxUUgNo9rN4AG'
if __name__ == '__main__':
# Create authentication token
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth)
print 'Getting statistics for #BarackObama:'
# Get information about the user
data = api.get_user('BarackObama')
print 'Followers: ' + str(data.followers_count)
print 'Tweets: ' + str(data.statuses_count)
print 'Favouries: ' + str(data.favourites_count)
print 'Friends: ' + str(data.friends_count)
print 'Appears on ' + str(data.listed_count) + ' lists'
print(data)
print(data) or print(status) only gives certain tweets(not all) along with other unwanted information in JSON format.
I found extracting only tweets from home and own timeline by use of following code
statuses = tweepy.Cursor(api.user_timeline).items(2)
data = [s.text.encode('utf8') for s in statuses]
print data
Anyway I got the way to download tweet at single window by passing screenname or twitter id, it works for multiple user at a same time as well
I have been up and down these pages looking for how to do this and there are many similar posts but I can't seem to get it to work, so I find myself having to ask specifically how to do this.
I am trying to gather metrics about my software project in git hub. For many of these metrics you can use the API. However, one of the most interesting items are the unique visitors and view count on the github graphs/traffic and unfortunately this info is not located in the Github API. So, to get this I am trying to log into my github account navigate to the site then get the numbers. Located below is my code. I can't seem to get logged into github to do anything however (my url request continues to show a login page rather then the traffic page). I think it probably has something to do with the variables that need to be posted but I'm not sure whats wrong with them.
from requests import session
from bs4 import BeautifulSoup as bs
USER = 'MYID'
PASSWORD = 'MYPASSWORD'
URL1 = 'https://github.com/login'
URL2 = 'https://github.com/MYPROJ/graphs/traffic'
with session() as s:
req = s.get(URL1).text
html = bs(req)
token = html.find("input", {"name": "authenticity_token"}).attrs['value']
com_val = html.find("input", {"name": "commit"}).attrs['value']
login_data = {'login_field': USER,
'password': PASSWORD,
'authenticity_token' : token,
'commit' : com_val}
r1 = s.post(URL1, data = login_data)
r2 = s.get(URL2)
print(r2.url)
print bs(r2.text).find('span', {'class':'num js-uniques uniques'})
Any help is appreciated.
Thanks,
-Jeff
Figured it out.
I was using the wrong address to post my login and username, as well as some other wrong bits.
This is the updated code that worked for me:
from requests import session
from bs4 import BeautifulSoup as bs
USER = 'MyUserName'
PASSWORD = 'Mypassword'
URL1 = 'https://github.com/session'
URL2 = 'https://github.com/MyProj/graphs/traffic-data'
with session() as s:
req = s.get(URL1).text
html = bs(req)
token = html.find("input", {"name": "authenticity_token"}).attrs['value']
com_val = html.find("input", {"name": "commit"}).attrs['value']
login_data = {'login': USER,
'password': PASSWORD,
'commit' : com_val,
'authenticity_token' : token}
r1 = s.post(URL1, data = login_data)
r2 = s.get(URL2)
Cut1 = r2.text.split(',"summary":{"total":',2)
ViewsTot = Cut1[1].split(',"unique":',1)
ViewsUnq = ViewsTot[1].split('}}',1)