I am using a csv iterator to go through a csv file, which contains data associated with time. I am sure the csv file is correct. I am using Jupyter, iPython notebook in python 3.x
When I try to iterate on the first row by using .next() method, I have an AttributeError: 'module' object has no attribute 'next'.
My code is divided in two parts, one part containing function and imports, one part calling them. The function I have a problem with is:
def get_durations(csvfile):
try:
csvfile = iter(csvfile)
except TypeError, te:
print csvfile, 'is not iterable'
print "data is", repr(csvfile)
first_time = csvfile.next()[5]
first_time = (first_time.replace(" ", ""));
for row in csvfile:
last_time = row[5]
last_time = (last_time.replace(" ", ""))
first_time = datetime.datetime.strptime(first_time, "%H:%M:%S")
last_time = datetime.datetime.strptime(last_time, "%H:%M:%S")
return first_time.replace(second = 0), last_time.replace(second = 0)
I make the call to the function here:
for el_mousefile in mousefiles:
os.chdir(el_mousefile)
print "data is", repr(csvfile)
csvfile = csv.reader(open("mouse.csv", "rU"), delimiter=';', quoting=csv.QUOTE_NONE)
print "data is", repr(csvfile)
try:
csvfile = iter(csvfile)
except TypeError, te:
print csvfile, 'is not iterable'
first_time, last_time = get_durations(csv)
I get this output when trying to run the program:
data is <_csv.reader object at 0x000000000A388D08>
data is <_csv.reader object at 0x000000000A388948>
module 'csv' from 'C:\Users\**\AppData\Local\Continuum\Anaconda\lib\csv.pyc' is not iterable
data is module 'csv' from 'C:\Users\**\AppData\Local\Continuum\Anaconda\lib\csv.pyc'
96------>first_time = csvfile.next()[5]
97 first_time = (first_time.replace(" ", ""));
98 for row in csvfile:
AttributeError: 'module' object has no attribute 'next'
I don't understand how can my csv be iterable in the second part, but then when being passed to the function, it is not iterable anymore which is responsible for my error.
first_time, last_time = get_durations(csv)
You are passing in the csv module, not a file.
Related
#!/usr/bin/py
from collections import defaultdict
"i am getting str object has no attribute appened"
class car:
def __init__(self,car_name):
self.car_name = car_name
class model(car):
def __init__(self,car_model,car_name):
self.car_model=car_model
car.__init__(self,car_name)
cars1=defaultdict(list)
if __name__ == "__main__":
i=1
while(i<=2):
car_name=raw_input("Enter the car name: ")
car_model=raw_input("Enter the car model: ")
brand=model(car_model,car_name)
if brand.car_name in cars1:
print "1"
print brand.car_name
#### here i am getting this error ######
cars1[brand.car_name].append(brand.car_model)
else:
print "2"
cars1[brand.car_name]=brand.car_model
i+=1
for key,values in cars1.items():
print key+":"+values
print cars1.items()
You are getting this error because in case the key does not exist you insert a string as the value (cars1[brand.car_name] = brand.car_model).
Since you are using defaultdict you don't even need to perform this check, just call append. If the key doesn't exist an empty list will be created, that's the point of using a defaultdict.
brand = model(car_model, car_name)
cars1[brand.car_name].append(brand.car_model)
I have the following function with a few print statements. In each print statement I would like to return it's value so I can use that and add it to my email code which will send each string text into an email.
I tried to concatenate each string into a variable and return it at the bottom of the function.
e.g.
p_text = p_start_time + p_duration + p_status
return p_text
I get the error:
File "E:/test_runners 2 edit project in progress add more tests/selenium_regression_test_5_1_1/Email/email_selenium_report.py", line 30, in <module>
report.extract_data_from_report_htmltestrunner()
File "E:\test_runners 2 edit project in progress add more tests\selenium_regression_test_5_1_1\Email\report.py", line 400, in extract_data_from_report_htmltestrunner
p_text = p_start_time + p_duration + p_status
TypeError: unsupported operand type(s) for +: 'Tag' and 'Tag'
My function implementation is:
def extract_data_from_report_htmltestrunner():
filename = (r"E:\test_runners 2 edit project\selenium_regression_test_5_1_1\TestReport\ClearCore501_Automated_GUI_TestReport.html")
html_report_part = open(filename,'r')
soup = BeautifulSoup(html_report_part, "html.parser")
div_heading = soup.find('div', {'class': 'heading'})
p_start_time = div_heading.find('strong', text='Start Time:').parent
p_start_time.find(text=True, recursive=False)
print p_start_time.text
p_duration = div_heading.find('strong', text='Duration:').parent
p_duration.find(text=True, recursive=False)
print p_duration.text
p_status = div_heading.find('strong', text='Status:').parent
p_status.find(text=True, recursive=False)
print p_status.text
#p_text = p_start_time + p_duration + p_status
table = soup.select_one("#result_table")
headers = [td.text for td in table.select_one("#header_row").find_all("td")[1:-1]]
print(" ".join(headers))
for row in table.select("tr.passClass"):
print(" ".join([td.text for td in row.find_all("td")[1:-1]]))
#return p_text
In each of my print statements how can i return it as a string variable?
Once I have it returned I can include it in the message part of my email code.
Even this print statement in teh for loop I would like to return it in a string variable somehow.
print(" ".join([td.text for td in row.find_all("td")[1:-1]]))
Thanks, Riaz
p_text = p_start_time + p_duration + p_status
In this expression, all the operands are Tag instances which you cannot glue together with +. What you can do is to cast them to strings and then concatenate:
p_text = "".join(map(str, [p_start_time, p_duration, p_status]))
How can I create a Dictionary from my while loop below? infile reads from a file called input, which has the following content:
min:1,23,62,256
max:24,672,5,23
sum:22,14,2,3,89
P90:23,30,45.23
P70:12,23,24,57,32
infile = open("input.txt", "r")
answers = open("output.txt", "w")
while True:
line = infile.readline()
if not line: break
opType = line[0:3]
numList = (line[4:len(line)])
numList = numList.split(',')
What I'm trying to do is basically 2 lists, one that has the operation name (opType) and the other that has the numbers. From there I want to create a dictionary that looks like this
myDictionary = {
'min': 1,23,62,256,
'max': 24,672,5,23,
'avg': 22,14,2,3,89,
'P90': 23,30,45.23,
'P70': 12,23,24,57,32,
}
The reason for this is that I need to call the operation type to a self-made function, which will then carry out the operation. I'll figure this part out. I currently just need help making the dictionary from the while loop.
I'm using python 2.7
Try the following code.
I believe, you would need the 'sum' also in the dictionary. If not, just add a condition to remove it.
myDictionary = {}
with open('input.txt','r') as f:
for line in f:
x = line.split(':')[1].rstrip().split(',')
for i in xrange(len(x)):
try:
x[i] = int(x[i])
except ValueError:
x[i] = float(x[i])
myDictionary[line.split(':')[0]] = x
print myDictionary
I have a CSV file and I am trying to populate them to a sqlite database. I have no error message and it works perfectly fine but loads only the last line of the file.
MD= MD()
database = options.get('database')
filename = options.get('filename')
dataReader = csv.reader(open(filename))
for row in dataReader:
if row[0] != 'ID':
bb= 1 if row[3] == 'YES' else 0
pro = 'YES' if row[4] == 'Pro' else 'NO'
MD.id = row[0]
MD.mol = row[1]
MD.phase = row[2]
MD.warning = black_box
MD.pro = pro
MD.status = Type.objects.get(description=row[5])
MD.name = row[6]
MD.stem = row[7]
MD.year = row[8]
MD.iname = row[9]
MD.iyear = row[10]
print row[1], row[2],row[3],row[4],row[5],row[6], row[0]
MD.save()
But the print statement prints all the lines in the CSV file. I have no idea what happens. Thanks
You are only creating one MD instance outside of the for loop, then saving to that same instance on each iteration of the loop. You need to create a new MD() instance for every iteration (per line of the file) if you want to create and save a new record for each line. This is why you are only saving the last line - you are over-writing the pre-existing instance you created. Good luck.
You are saving always the same object. Try with this:
Put the MD= MD() inside the for:
... # The same here
for row in dataReader:
if row[0] != 'ID':
MD= MD()
... # The same here
MD.id = row[0]
MD.mol = row[1]
...
MD.save()
I am attempting to alter a previous script that utilizes biopython to fetch information about a species phylum. This script was written to retrieve information one species at a time. I would like to modify the script so that I can do this for 100 organisms at a time.
Here is the initial code
import sys
from Bio import Entrez
def get_tax_id(species):
"""to get data from ncbi taxomomy, we need to have the taxid. we can
get that by passing the species name to esearch, which will return
the tax id"""
species = species.replace(" ", "+").strip()
search = Entrez.esearch(term = species, db = "taxonomy", retmode = "xml")
record = Entrez.read(search)
return record['IdList'][0]
def get_tax_data(taxid):
"""once we have the taxid, we can fetch the record"""
search = Entrez.efetch(id = taxid, db = "taxonomy", retmode = "xml")
return Entrez.read(search)
Entrez.email = ""
if not Entrez.email:
print "you must add your email address"
sys.exit(2)
taxid = get_tax_id("Erodium carvifolium")
data = get_tax_data(taxid)
lineage = {d['Rank']:d['ScientificName'] for d in
data[0]['LineageEx'] if d['Rank'] in ['family', 'order']}
I have managed to modify the script so that it accepts a local file that contains one of the organisms I am using. But I need to extend this to a 100 organisms.
So the idea was to generate a list from the file of my organisms and somehow separately fed each item generated from the list into the line taxid = get_tax_id("Erodium carvifolium") and replace "Erodium carvifolium" with my organisms name. But I have no idea how to do that.
Here is the sample version of the code with some of my adjustments
import sys
from Bio import Entrez
def get_tax_id(species):
"""to get data from ncbi taxomomy, we need to have the taxid. we can
get that by passing the species name to esearch, which will return
the tax id"""
species = species.replace(' ', "+").strip()
search = Entrez.esearch(term = species, db = "taxonomy", retmode = "xml")
record = Entrez.read(search)
return record['IdList'][0]
def get_tax_data(taxid):
"""once we have the taxid, we can fetch the record"""
search = Entrez.efetch(id = taxid, db = "taxonomy", retmode = "xml")
return Entrez.read(search)
Entrez.email = ""
if not Entrez.email:
print "you must add your email address"
sys.exit(2)
list = ['Helicobacter pylori 26695', 'Thermotoga maritima MSB8', 'Deinococcus radiodurans R1', 'Treponema pallidum subsp. pallidum str. Nichols', 'Aquifex aeolicus VF5', 'Archaeoglobus fulgidus DSM 4304']
i = iter(list)
item = i.next()
for item in list:
???
taxid = get_tax_id(?)
data = get_tax_data(taxid)
lineage = {d['Rank']:d['ScientificName'] for d in
data[0]['LineageEx'] if d['Rank'] in ['phylum']}
print lineage, taxid
The question marks refer to places where I am stumped as what to do next. I don't see how I can connect my loop to replace the ? in get_tax_id(?). Or do I need to somehow append each of the items in the list so that they are modified each time to contain get_tax_id(Helicobacter pylori 26695) and then find some way to place them in the line containing taxid =
Here's what you need, place this below your function definitions, i.e. after the line that says: sys.exit(2)
species_list = ['Helicobacter pylori 26695', 'Thermotoga maritima MSB8', 'Deinococcus radiodurans R1', 'Treponema pallidum subsp. pallidum str. Nichols', 'Aquifex aeolicus VF5', 'Archaeoglobus fulgidus DSM 4304']
taxid_list = [] # Initiate the lists to store the data to be parsed in
data_list = []
lineage_list = []
print('parsing taxonomic data...') # message declaring the parser has begun
for species in species_list:
print ('\t'+species) # progress messages
taxid = get_tax_id(species) # Apply your functions
data = get_tax_data(taxid)
lineage = {d['Rank']:d['ScientificName'] for d in data[0]['LineageEx'] if d['Rank'] in ['phylum']}
taxid_list.append(taxid) # Append the data to lists already initiated
data_list.append(data)
lineage_list.append(lineage)
print('complete!')