I am working in the textwrap module that requires the outputted text to look like see picture of expected output formatting
However, my output looks like:
see my output
This is the code I am using for the text wrap:
wrap = textwrap.TextWrapper(initial_indent = ' '*4, subsequent_indent = ' '*4)
if name not in bus_name:
print 'This business is not found'
else:
count = 0
for id in bus_name[name]:
if id not in bus_review:
print 'No reviews for this business are found'
else:
for rev in bus_review[id]:
print 'Review %d' %(count+1)
string = textwrap.fill(rev)
string = string.replace(' ', '$')
string = wrap.fill(string)
string = string.replace('$', '\n'+' '*4)
print '%s' %string
print
count += 1
Related
I would like to count different universities from which the mail was sent for which i used the following code:
fname = raw_input('Enter the file name: ')
try:
fhan = open(fname)
except:
print 'File cannot be opened:', fname
count = 0
sum = 0
for i in fhan:
if i.startswith('From'):
x=i.find('#')
y=i.find(' ',x)
str1=i[x+1:y].strip()
print str1
count=count+1
print count
The final output gives me the handles but can i remove the repeated ones, if i print uct.ac.za it shouldnot print and count again
link for file: www.py4inf.com/code/mbox-short.txt
You can append the handles in a list instead of printing it. And then convert that list in a set. In a set there are no repeated elements so you will get the a set of unique universities. And Finally, you can iterate through the set and print the universities.
For count you can use the len function that will count the universities in the set.
This is the modified code:-
fname = raw_input('Enter the file name: ')
try:
fhan = open(fname)
except:
print 'File cannot be opened:', fname
universities = []
for i in fhan:
if i.startswith('From'):
x=i.find('#')
y=i.find(' ',x)
str1=i[x+1:y].strip()
universities.append(str1)
universities = set(universities)
for i in universities:
print i
print len(universities)
I have used the following url to fetch the historical data from yahoo finance. From last 16th May, 2017 the url is not working.
http://real-chart.finance.yahoo.com/table.csv?s=AAL&a=04&b=01&c=2017&d=04&e=02&f=2017&g=d&ignore=.csv
Seems like they have changed the url and the new url is:
https://query1.finance.yahoo.com/v7/finance/download/AAL?period1=1494873000&period2=1494959400&interval=1d&events=history&crumb=l0aEtuOKocj
In the above changed URL has a session cookie which is crumb. Is there any idea how to get this cookie programmatically(in JAVA)?
I recently wrote a simple python script to download the history of a single stock.
Here an example how to invoke it:
python get_quote_history.py --symbol=IBM --from=2017-01-01 --to=2017-05-25 -o IBM.csv
This will download IBM historical prices from 2017-01-01 to 2017-05-25 and save them in IBM.csv file.
import re
import urllib2
import calendar
import datetime
import getopt
import sys
import time
crumble_link = 'https://finance.yahoo.com/quote/{0}/history?p={0}'
crumble_regex = r'CrumbStore":{"crumb":"(.*?)"}'
cookie_regex = r'Set-Cookie: (.*?); '
quote_link = 'https://query1.finance.yahoo.com/v7/finance/download/{}?period1={}&period2={}&interval=1d&events=history&crumb={}'
def get_crumble_and_cookie(symbol):
link = crumble_link.format(symbol)
response = urllib2.urlopen(link)
match = re.search(cookie_regex, str(response.info()))
cookie_str = match.group(1)
text = response.read()
match = re.search(crumble_regex, text)
crumble_str = match.group(1)
return crumble_str, cookie_str
def download_quote(symbol, date_from, date_to):
time_stamp_from = calendar.timegm(datetime.datetime.strptime(date_from, "%Y-%m-%d").timetuple())
time_stamp_to = calendar.timegm(datetime.datetime.strptime(date_to, "%Y-%m-%d").timetuple())
attempts = 0
while attempts < 5:
crumble_str, cookie_str = get_crumble_and_cookie(symbol)
link = quote_link.format(symbol, time_stamp_from, time_stamp_to, crumble_str)
#print link
r = urllib2.Request(link, headers={'Cookie': cookie_str})
try:
response = urllib2.urlopen(r)
text = response.read()
print "{} downloaded".format(symbol)
return text
except urllib2.URLError:
print "{} failed at attempt # {}".format(symbol, attempts)
attempts += 1
time.sleep(2*attempts)
return ""
if __name__ == '__main__':
print get_crumble_and_cookie('KO')
from_arg = "from"
to_arg = "to"
symbol_arg = "symbol"
output_arg = "o"
opt_list = (from_arg+"=", to_arg+"=", symbol_arg+"=")
try:
options, args = getopt.getopt(sys.argv[1:],output_arg+":",opt_list)
except getopt.GetoptError as err:
print err
for opt, value in options:
if opt[2:] == from_arg:
from_val = value
elif opt[2:] == to_arg:
to_val = value
elif opt[2:] == symbol_arg:
symbol_val = value
elif opt[1:] == output_arg:
output_val = value
print "downloading {}".format(symbol_val)
text = download_quote(symbol_val, from_val, to_val)
with open(output_val, 'wb') as f:
f.write(text)
print "{} written to {}".format(symbol_val, output_val)
Andrea Galeazzi's excellent answer; with added options for splits and dividends, and twisted for python 3.
Also changed so "to:date" is included in the returned results, previously code returned up to but not including "to:date". Just different!
And be aware that Yahoo made minor changes in price rounding, column order, and split syntax.
## Downloaded from
## https://stackoverflow.com/questions/44044263/yahoo-finance-historical-data-downloader-url-is-not-working
## Modified for Python 3
## Added --event=history|div|split default = history
## changed so "to:date" is included in the returned results
## usage: download_quote(symbol, date_from, date_to, events).decode('utf-8')
import re
from urllib.request import urlopen, Request, URLError
import calendar
import datetime
import getopt
import sys
import time
crumble_link = 'https://finance.yahoo.com/quote/{0}/history?p={0}'
crumble_regex = r'CrumbStore":{"crumb":"(.*?)"}'
cookie_regex = r'Set-Cookie: (.*?); '
quote_link = 'https://query1.finance.yahoo.com/v7/finance/download/{}?period1={}&period2={}&interval=1d&events={}&crumb={}'
def get_crumble_and_cookie(symbol):
link = crumble_link.format(symbol)
response = urlopen(link)
match = re.search(cookie_regex, str(response.info()))
cookie_str = match.group(1)
text = response.read().decode("utf-8")
match = re.search(crumble_regex, text)
crumble_str = match.group(1)
return crumble_str , cookie_str
def download_quote(symbol, date_from, date_to,events):
time_stamp_from = calendar.timegm(datetime.datetime.strptime(date_from, "%Y-%m-%d").timetuple())
next_day = datetime.datetime.strptime(date_to, "%Y-%m-%d") + datetime.timedelta(days=1)
time_stamp_to = calendar.timegm(next_day.timetuple())
attempts = 0
while attempts < 5:
crumble_str, cookie_str = get_crumble_and_cookie(symbol)
link = quote_link.format(symbol, time_stamp_from, time_stamp_to, events,crumble_str)
#print link
r = Request(link, headers={'Cookie': cookie_str})
try:
response = urlopen(r)
text = response.read()
print ("{} downloaded".format(symbol))
return text
except URLError:
print ("{} failed at attempt # {}".format(symbol, attempts))
attempts += 1
time.sleep(2*attempts)
return b''
if __name__ == '__main__':
print (get_crumble_and_cookie('KO'))
from_arg = "from"
to_arg = "to"
symbol_arg = "symbol"
event_arg = "event"
output_arg = "o"
opt_list = (from_arg+"=", to_arg+"=", symbol_arg+"=", event_arg+"=")
try:
options, args = getopt.getopt(sys.argv[1:],output_arg+":",opt_list)
except getopt.GetoptError as err:
print (err)
symbol_val = ""
from_val = ""
to_val = ""
output_val = ""
event_val = "history"
for opt, value in options:
if opt[2:] == from_arg:
from_val = value
elif opt[2:] == to_arg:
to_val = value
elif opt[2:] == symbol_arg:
symbol_val = value
elif opt[2:] == event_arg:
event_val = value
elif opt[1:] == output_arg:
output_val = value
print ("downloading {}".format(symbol_val))
text = download_quote(symbol_val, from_val, to_val,event_val)
if text:
with open(output_val, 'wb') as f:
f.write(text)
print ("{} written to {}".format(symbol_val, output_val))
Got it to work, now I just have to parse the csv. Thought I'd share since I was having trouble with the syntax.
Dim crumb As String: crumb = "xxxx"
Dim cookie As String: cookie = "yyyy"
Dim urlStock As String: urlStock = "https://query1.finance.yahoo.com/v7/finance/download/SIRI?" & _
"period1=1274158800&" & _
"period2=1495059477&" & _
"interval=1d&events=history&crumb=" & crumb
Dim http As MSXML2.XMLHTTP: Set http = New MSXML2.ServerXMLHTTP
http.Open "GET", urlStock, False
http.setRequestHeader "Cookie", cookie
http.send
You can manually save the crumb/cookie pair in Chrome or you can use something like this to generate it. Then, just set the cookie header in java and pass the corresponding crumb in the URL
I have wrote a lightweight script that pulls together alot of the suggestions in this thread to fix this problem. https://github.com/AndrewRPorter/yahoo-historical
However, there are much better solutions such as, https://github.com/ranaroussi/fix-yahoo-finance
Hope these resources help!
I developed the following solution for this issue in Excel/VBA. The key challenge was the creation of the Crumb / Cookie pair. Once that is created you can re-use it for calls to Yahoo for the historical prices.
See here the key code for the Crumb / Cookie
Sub GetYahooRequest(strCrumb As String, strCookie As String)
'This routine will use a sample request to Yahoo to obtain a valid Cookie and Crumb
Dim strUrl As String: strUrl = "https://finance.yahoo.com/lookup?s=%7B0%7D"
Dim objRequest As WinHttp.WinHttpRequest
Set objRequest = New WinHttp.WinHttpRequest
With objRequest
.Open "GET", strUrl, True
.setRequestHeader "Content-Type", "application/x-www-form-urlencoded; charset=UTF-8"
.send
.waitForResponse
strCrumb = strExtractCrumb(.responseText)
strCookie = Split(.getResponseHeader("Set-Cookie"), ";")(0)
End With
End Sub
See the following Yahoo Historical Price Extract on my website for a Sample Excel workbook that demonstrates how to extract Yahoo Historical prices
Great answer Andrea, I have added to your code to allow for downloads of multiple stocks. (python 2.7)
file1: down.py
import os
myfile = open("ticker.csv", "r")
lines = myfile.readlines()
for line in lines:
ticker = line.strip();
cmd = "python get_quote_history.py --symbol=%s --from=2017-01-01 --to=2017-05-25 -o %s.csv" %(ticker,ticker)
os.system(cmd)
file2: ticker.csv
AAPL
MSFT
file3: get_quote_history.py
here my function . I have to do lower what ever coming in newfilename..i.e newfilename.lower()
def my_function(start, end):
sheetname = 'my-sheet'
filepath = "/myxl.xlsx"
try:
work_book=xlrd.open_workbook(filepath)
except:
print 'error'
try:
worksheet = work_book.sheet_by_name(sheetname)
except:
print 'error'
rows=worksheet.nrows
cols=worksheet.ncols
success = []
fail = []
for row in xrange(start,end):
print "row no. : ",row
state = '/home/myfolder/'
if os.path.exists(state):
print "state folder exits"
else:
os.makedirs(state)
district = state + worksheet.cell_value(row,0) + '/'
if os.path.exists(district):
print "district folder exits"
else:
os.makedirs(district)
city = district + worksheet.cell_value(row,2) + '/'
if os.path.exists(city):
print "city folder exits"
else:
os.makedirs(city)
newfilename = city + worksheet.cell_value(row,4).replace (" ", "-") + '.png'
if worksheet.cell_value(row,5) !="":
oldfilename = worksheet.cell_value(row,5)
else:
oldfilename="no-image"
newfullpath = newfilename
oldfullpath = '/home/old/folder/' + oldfilename
try:
os.rename(oldfullpath,newfullpath)
success.append(row)
except Exception as e:
fail.append(row)
print "Error",e
print 'renaming done for row #' ,row , ' file ', oldfilename , ' to ', newfilename
print 'SUCCESS ', success
print 'FAIL ', fail
newfilename.lower() not working
here when I am going to use unicode error coming...
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2019' in position 72: ordinal not in range(128)
I threw the 'Apple-code' in the trash, please look at the following code :
# Define processing inputted line
def inputfile(line):
linecontents = { 'item_0110': line[0:8],
'item_0111': line[8:16],
'item_0112': line[16:24] }
print 'In the function : ', linecontents
print 'In the function : ', len(linecontents)
# Set dictionary
linecontents = {}
# Pretend to open a file and read line for line
line = '010012343710203053525150'
# Start processing the read line
inputfile(line)
# Display end resultprint
print '\nOutside the function : ', linecontents
print 'Outside the function : ', len(linecontents)
Ok, first off : I'm an idiot for trying this with vars. In the original post I already stated I have more than thirty items (fields if you want) in the file. To make matters more complex, a line from the file could look like this :
010012343710203053525150
And not all lines have the same fields so depending on what type of field it is, I would like to call a different function.
The question now is : why is the output like this :
In the function : {'item_0112': '53525150', 'item_0111': '37102030', 'item_0110': '01001234'}
In the function : 3
Outside the function : {}
Outside the function : 0
I thought the dictionary is independent from functions and/or classes ?
There are some issues with your code. I don't really see the see for global variables here.
I reformatted and refactored your code (no uppercase except for classes). You intent is not clear so I tried my best. The function read_file actually reads you file line by line and returns customer_name and customer_item line by line.
def read_file(filepath):
with open(filepath) as customer_file:
for line in customer_file:
customer_name = line[:10]
customer_item = line[10:]
print('Name : ' + customer_name)
print('Item : ' + customer_item)
yield customer_name, customer_item
In the main() or whatever function, you can do what you want with the customer's variables.
What is important here, is that read_file actually reads a file and process the information for the file before returning them to the calling function.
def main():
myfile = 'CustomerFile.txt'
for customer_name, customer_item in read_file(myfile):
if customer_item == 'Apple':
print(customer_name)
else:
print(customer_name + ' is not eating an apple')
Instead of using globals, let the ReadFile function return the values
def ReadFile(line):
...
return CustomerName, CustomerItem
and assign them to variables after calling the function:
for line in CustomerFile:
CustomerName, CustomerItem = ReadFile(line)
def ReadFile(line):
CustomerName = line[0:10]
CustomerItem = line[10:]
return CustomerName, CustomerItem
def Report(CustomerName, CustomerItem):
# Try to keep all print statements in one place
print 'Name : ' + CustomerName
print 'Item : ' + CustomerItem
if CustomerItem == 'Apple':
print CustomerName
else:
print CustomerName + ' is not eating an apple'
with open(CustomerFile.txt) as CustomerFile:
for line in CustomerFile:
CustomerName, CustomerItem = ReadFile(line)
Report(CustomerName, CustomerItem)
Note that the PEP8 style guide recommends using lower_case for variable and function names and to reserve CamelCase for classes. While you are free to use your own style, use of PEP8 is quite prevalent and so by joining the PEP8-club your code will more naturally "fit in" with other people's code and vice-versa.
I found the (obvious) answer. As I said, my Python is a bit rusty :
## Define processing inputed line
#
def inputfile(line, linecontents):
linecontents['item_0110'] = line[0:8]
linecontents['item_0111'] = line[8:16]
linecontents['item_0112'] = line[16:24]
print 'In the function : ', linecontents
print 'In the function : ', len(linecontents)
## Define main script
#
def main():
# Set dict
linecontents = {}
# Pretend to open a file and read line for line
line = '010012343710203053525150'
# Start processing the read file
inputfile(line, linecontents)
# Display end resultprint
print '\nOutside the funtion : ', linecontents
print 'Outsude the function : ', len(linecontents)
## Start main script
#
main()
And this neatly returns :
In the function : {'item_0112': '53525150', 'item_0111': '37102030', 'item_0110': '01001234'}
In the function : 3
Outside the funtion : {'item_0112': '53525150', 'item_0111': '37102030', 'item_0110': '01001234'}
Outsude the function : 3
I have been using multiple code snippets to create a solution that will allow me to write a list of players in a football team to a csv file.
import csv
data = []
string = input("Team Name: ")
fName = string.replace(' ', '') + ".csv"
print("When you have entered all the players, press enter.")
# while loop that will continue allowing entering of players
done = False
while not done:
a = input("Name of player: ")
if a == "":
done = True
else:
string += a + ','
string += input("Age: ") + ','
string += input("Position: ")
print (string)
file = open(fName, 'w')
output = csv.writer(file)
for row in string:
tempRow = row
output.writerow(tempRow)
file.close()
print("Team written to file.")
I would like the exported csv file to look like this:
player1,25,striker
player2,27,midfielder
and so on. However, when I check the exported csv file it looks more like this:
p
l
a
y
e
r
,
2
5
and so on.
Does anyone have an idea of where i'm going wrong?
Many thanks
Karl
Your string is a single string. It is not a list of strings. You are expecting it to be a list of strings when you are doing this:
for row in string:
When you iterate over a string, you are iterating over its characters. Which is why you are seeing a character per line.
Declare a list of strings. And append every string to it like this:
done = False
strings_list = []
while not done:
string = ""
a = input("Name of player: ")
if a == "":
done = True
else:
string += a + ','
string += input("Age: ") + ','
string += input("Position: ") + '\n'
strings_list.append(string)
Now iterate over this strings_list and print to the output file. Since you are putting the delimiter (comma) yourself in the string, you do not need a csv writer.
a_file = open(fName, 'w')
for row in strings_list:
print(row)
a_file.write(row)
a_file.close()
Note:
string is a name of a standard module in Python. It is wise not to use this as a name of any variable in your program. Same goes for your variable file