here is my code:
import csv
inp1 = raw_input('Enter your Hijjri year:')
intinp1 = int(inp1)
majmouaopen = open('Majmoua.csv')
majmouaread = csv.reader(majmouaopen)
majmouaread.next()
mabsoutaopen = open('Mabsouta.csv')
mabsoutaread = csv.reader(mabsoutaopen)
mabsoutaread.next()
hijrimiladimonthsopened = open('MiladiHijrimonths.csv')
hijrimiladimonthsread = csv.reader(hijrimiladimonthsopened)
yearslist = []
years = []
yearssection = []
monthssection = []
minutessection = []
def miladifromhijri(intinp1):#, inp2, intinp3):
fulyear = intinp1 - 1
n = 0
for row in majmouaread:
print row
introw = int(row[0])
if introw <= fulyear:
n += 1
years.append(introw)
continue
if n == len(years):
near = years[::-1][0]
nearlessyear = near
break
for row in majmouaread:
print row
my problem is with the last loop, it doesn't print all of the majmouaread files. for the first loop, which is the same, it does print all of the csv file rows.
What is causing the probblem, is it something in the code? or something happened to the csv file read? It looks fine with first loop?
Related
The error occurs in the last step of this code when I try to sort a list using SORTED. I get the error message that a "list object is not callable". I want to use the SORTED and not the SORT function for this.
from csv import reader
import datetime as dt
open_file = open("hacker_news.csv")
read_file = reader(open_file)
hn = list(read_file)
headers = hn[0]
hn = hn[1:]
def explore_data(dataset,start,finish,col_rows=True):
dataset_slice = dataset[start:finish]
for row in dataset_slice:
print(row)
print('\n')
if col_rows:
print('rows:' , len(dataset))
print('columns:' , len(dataset[0]))
ask_posts = []
show_posts = []
other_posts = []
for row in hn:
title = row[1]
if title.lower().startswith("ask hn"):
ask_posts.append(row)
elif title.lower().startswith("show hn"):
show_posts.append(row)
else:
other_posts.append(row)
total_ask_comments = 0
total_show_comments = 0
total = 0
for row in ask_posts:
total += 1
num_comments = int(row[4])
total_ask_comments += num_comments
avg_ask_comments = total_ask_comments/total
print(avg_ask_comments)
for row in show_posts:
total += 1
num_comments = int(row[4])
total_show_comments += num_comments
avg_show_comments = total_show_comments/total
print(avg_show_comments)
result_list = []
for row in ask_posts:
created_at = row[6]
num_comments = int(row[4])
result_list.append([created_at,num_comments])
counts_by_hour = {}
comments_by_hour = {}
for row in result_list:
comment = row[1]
date_time = row[0]
date_time = dt.datetime.strptime(date_time,'%m/%d/%Y %H:%M')
hour = date_time.strftime('%H')
if hour not in counts_by_hour:
counts_by_hour[hour] = 1
comments_by_hour[hour] = comment
else:
counts_by_hour[hour] += 1
comments_by_hour[hour] += comment
for hour in counts_by_hour:
if hour in comments_by_hour:
avg = round(comments_by_hour[hour]/counts_by_hour[hour],2)
avg_by_hour.append([hour,avg])
for row in avg_by_hour:
swap_avg_by_hour.append([row[1], row[0]])
print(swap_avg_by_hour)
sorted_swap = sorted(swap_avg_by_hour, reverse=True)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-49-307863a4b1cd> in <module>
6 print(swap_avg_by_hour)
7
----> 8 sorted_swap = sorted(swap_avg_by_hour, reverse=True)
9
10 print(sorted_swap)
TypeError: 'list' object is not callable
Not sure how to upload the csv as don't see an upload option. Is there an obvious error in the code? or can someone help with instructions to upload the csv file?
You need to declare these 2 lists before appnding avg_by_hour = [] swap_avg_by_hour= []
I'm relatively new to python (2.7) and need help looping through 2 CSV files. The first (outer loop) file is the row I want to write if certain conditions are met with the second (inner loop) file.
import csv
f = open('../CI Working Copy.csv')
with open('../first.csv', 'wb') as n:
theWriter = csv.writer(n)
csv_f = csv.reader(f)
g = open('../second.csv')
csv_g = csv.reader(g)
for row in csv_f:
cbd = row[3]
ced = row[4]
rbd = row[5]
red = row[6]
ciCn = row[10]
for iRow in csv_g:
cn = iRow[0]
startDate = iRow[1]
endDate = iRow[2]
iId = iRow[3]
writeRow = 'false'
if ciCn == cn:
if (cbd == startDate and ced == endDate) or (rbd == startDate and red == endDate):
theWriter.writerow(row)
g.close()
f.close()
It makes it into the second (inner loop) file, but never returns to the outer loop. I only need to write the row from the first file.
For each row of the first csv file, you consume all the second file, so you need to go back on the beginning of the second file on each iteration.
The solution is:
for row in csv_f:
g.seek(0) #go at the start of the second file
for iRow in csv_g:
do_smth(iRow,row)
g.close()
Hello I was hoping someone could help me with my college coursework, I have an issue with my code. I keep running into a memory error with my data export.
Is there any way I can reduce the memory that is being used or is there a different approach I can take?
For the course work I am given a file of 300 records about customer orders from a CSV file and then I have to export the Friday records to a new CSV file. Also I am required to print the most popular method for customer's orders and the total money raised from the orders but I have an easy plan for that.
This is my first time working with CSV so I'm not sure how to do it. When I run the program it tends to crash instantly or stop responding. Once it appeared with 'MEMORY ERROR' however that is all it appeared with. I'm using a college provided computer so I am not sure on the exact specs but I know it runs 4GB of memory.
defining count occurences predefined function
def countOccurences(target,array):
counter = 0
for element in array:
if element == target:
counter= counter + 1
print counter
return counter
creating user defined functions for the program
dataInput function used for collecting data from provided file
def dataInput():
import csv
recordArray = []
customerArray = []
f = open('E:\Portable Python 2.7.6.1\Choral Shield Data File(CSV).csv')
csv_f = csv.reader(f)
for row in csv_f:
customerArray.append(row[0])
ticketID = row[1]
day, area = datasplit(ticketID)
customerArray.append(day)
customerArray.append(area)
customerArray.append(row[2])
customerArray.append(row[3])
recordArray.append(customerArray)
f.close
return recordArray
def datasplit(variable):
day = variable[0]
area = variable[1]
return day,area
def dataProcessing(recordArray):
methodArray = []
wed_thursCost = 5
friCost = 10
record = 0
while record < 300:
method = recordArray[record][4]
methodArray.append(method)
record = record+1
school = countOccurences('S',methodArray)
website = countOccurences('W',methodArray)
if school > website:
school = True
elif school < website:
website = True
dayArray = []
record = 0
while record < 300:
day = recordArray[record][1]
dayArray.append(day)
record = record + 1
fridays = countOccurences('F',dayArray)
wednesdays = countOccurences('W',dayArray)
thursdays = countOccurences('T', dayArray)
totalFriCost = fridays * friCost
totalWedCost = wednesdays * wed_thursCost
totalThurCost = thursdays * wed_thursCost
totalCost = totalFriCost + totalWedCost + totalThurCost
return totalCost,school,website
My first attempt to writing to a csv file
def dataExport(recordArray):
import csv
fridayRecords = []
record = 0
customerIDArray = []
ticketIDArray = []
numberArray = []
methodArray = []
record = 0
while record < 300:
if recordArray[record][1] == 'F':
fridayRecords.append(recordArray[record])
record = record + 1
with open('\Courswork output.csv',"wb") as f:
writer = csv.writer(f)
for record in fridayRecords:
writer.writerows(fridayRecords)
f.close
My second attempt at writing to the CSV file
def write_file(recordArray): # write selected records to a new csv file
CustomerID = []
TicketID = []
Number = []
Method = []
counter = 0
while counter < 300:
if recordArray[counter][2] == 'F':
CustomerID.append(recordArray[counter][0])
TicketID.append(recordArray[counter][1]+recordArray[counter[2]])
Number.append(recordArray[counter][3])
Method.append(recordArray[counter][4])
fridayRecords = [] # a list to contain the lists before writing to file
for x in range(len(CustomerID)):
one_record = CustomerID[x],TicketID[x],Number[x],Method[x]
fridayRecords.append(one_record)
#open file for writing
with open("sample_output.csv", "wb") as f:
#create the csv writer object
writer = csv.writer(f)
#write one row (item) of data at a time
writer.writerows(recordArray)
f.close
counter = counter + 1
#Main Program
recordArray = dataInput()
totalCost,school,website = dataProcessing(recordArray)
write_file(recordArray)
In the function write_file(recordArray) in your second attempt the counter variable counter in the first while loop is never updated so the loop continues for ever until you run out of memory.
I create a list of dictionaries from excel. The code is listed below. What happens is the list contains the last row excel values for all the rows. I tried in python shell. It works fine. Why all the rows get updated with last row values?
d = {}
l = []
up = os.path.join(request.folder,'uploads')
workbook = xlrd.open_workbook(os.path.join(request.folder,'uploads','meas.xls'))
worksheet = workbook.sheet_by_name('Sheet1')
num_rows = worksheet.nrows - 1
num_cells = worksheet.ncols - 1
curr_row = -1
while curr_row < num_rows:
curr_row += 1
row = worksheet.row(curr_row)
#print 'Row:', curr_row
curr_cell = -1
while curr_cell < num_cells:
curr_cell += 1
# Cell Types: 0=Empty, 1=Text, 2=Number, 3=Date, 4=Boolean, 5=Error, 6=Blank
cell_type = worksheet.cell_type(curr_row, curr_cell)
cell_value = worksheet.cell_value(curr_row, curr_cell)
#print ' ', cell_type, ':', cell_value
if curr_cell == 0:
d['loc_of_work'] = cell_value
if curr_cell == 1:
d['n'] = cell_value
if curr_cell == 2:
d['t'] = cell_value
if curr_cell == 3:
d['l'] = cell_value
if curr_cell == 4:
d['b'] = cell_value
if curr_cell == 5:
d['d'] = cell_value
print 'dict'
print d.items()
l.append(d)
print 'len of list:'
print len(l)
print 'list:'
for i,j in enumerate(l):
print i,j
The issue is you are declaring d outside the while loop, which means within the loop you are simply overwriting the same dict with new values on every iteration. Your list simply contains multiple references to the same dict object, which contains the values from the last row because those are the last values to be written to the dict (all previous values are overwritten)
moveoing:
d = {}
inside the first while loop should fix your issue
Unsure of why I am getting this error. I'm reading from a file called columns_unsorted.txt, then trying to write to columns_unsorted.txt. There error is on fan_on = string_j[1], saying list index out of range. Here's my code:
#!/usr/bin/python
import fileinput
import collections
# open document to record results into
j = open('./columns_unsorted.txt', 'r')
# note this is a file of rows of space-delimited date in the format <1384055277275353 0 0 0 1 0 0 0 0 22:47:57> on each row, the first term being unix times, the last human time, the middle binary indicating which machine event happened
# open document to read from
l = open('./columns_sorted.txt', 'w')
# CREATE ARRAY CALLED EVENTS
events = collections.deque()
i = 1
# FILL ARRAY WITH "FACTS" ROWS; SPLIT INTO FIELDS, CHANGE TYPES AS APPROPRIATE
for line in j: # columns_unsorted
line = line.rstrip('\n')
string_j = line.split(' ')
time = str(string_j[0])
fan_on = int(string_j[1])
fan_off = int(string_j[2])
heater_on = int(string_j[3])
heater_off = int(string_j[4])
space_on = int(string_j[5])
space_off = int(string_j[6])
pump_on = int(string_j[7])
pump_off = int(string_j[8])
event_time = str(string_j[9])
row = time, fan_on, fan_off, heater_on, heater_off, space_on, space_off, pump_on, pump_off, event_time
events.append(row)
You are missing the readlines function, no?
You have to do:
j = open('./columns_unsorted.txt', 'r')
l = j.readlines()
for line in l:
# what you want to do with each line
In the future, you should print some of your variables, just to be sure the code is working as you want it to, and to help you identifying problems.
(for example, if in your code you would print string_j you would see what kind of problem you have)
Problem was an inconsistent line in the data file. Forgive my haste in posting