I have a folder where a store files from my fitting model in .txt format.
My question here is how to write a loop which will take e.g p1_cen 7.65782003 from this file and append it to a column in a .csv file?
The other thing with my question is that number of those files is equal to 288, because I store 5 minute long data from each day. And a loop what I need is to take from those 288 files a specifit data e.g like above, do You have any ideas how to do this?
For now, I have this code, which writes data in .txt files from my lmfit model.
with open('S:\Doc\Python\Results\DecompositionBx ' + "{0}".format(Station) + "{0}{1}".format(Start_time_hours_format, Start_time_minutes_format) + ".txt", 'w') as fh:
fh.write(result.fit_report(show_correl=False))
Btw. my files are named accordingly
DecompositionBxHylaty0000
...
DecompositionBxHylaty2355
UPDATE!!!
So the code from #bobrobbob works:
import csv
from datetime import timedelta
data = []
for i in range(288):
skip = i*timedelta(minutes=5)
hours, minutes, _ = str(skip).split(':')
filename = "S:\Dok\Python\Results\DecompositionBx Hylaty%02d%02d.txt" % (int(hours), int(minutes))
with open(filename) as f:
lines = f.readlines()
for line in lines:
if line.startswith(' p1_cen'):
data.append(line.split('+')[0])
break
with open('S:\Dok\Python\Results\data.csv', 'w') as f:
writer = csv.writer(f)
for line in data:
writer.writerow(line)
I get something like this, which is nearly perfect:
a bit ugly on the time handling, maybe someone will come with a cleaner solution. but it should work nonetheless
import csv
from datetime import timedelta
data = []
for i in range(288):
skip = i*timedelta(minutes=5)
hours, minutes, _ = str(skip).split(':')
filename = "DecompositionBxHylaty%02d%02d" % (int(hours), int(minutes))
with open(filename) as f:
lines = f.readlines()
for line in lines:
if line.startswith('p1_cen'):
data.append(line.split('+')[0].strip())
break
with open('data.csv', 'w', newline='') as f:
writer = csv.writer(f, delimiter=' ')
for line in data:
writer.writerow(line.split())
Related
So I wrote this code:
import csv
data = []
filename = "S:\Doc\Python\Data\Dekomp\Hth.txt"
with open(filename) as f:
lines = f.readlines()
for line in lines:
if line.startswith('%'):
data.append(line.split('+')[0].strip())
if line.endswith('%'):
break
with open('S:\Doc\Python\Data\Dekomp\Test.csv', 'w') as f:
writer = csv.writer(f, delimiter=' ')
for line in data:
writer.writerow(line.split())
And my data looks like this:
Headline starts with "%th=number", while number changes from 2 to 180 (each segment plus 2, so it goes (2,4,6... up to180).
Between those segments I have three columns of data, which I would like to append to a csv file. While using my code I save only headliners so (%th=2, %th=4... %th=180). Do you have any idea how to change my code so it will start reading headline, then append data below to a .txt or .csv file, and then starts loop again when it "sees" another headline and continue the process with saving next segment to another file, and that up to "%th=180"?
UPDATE:
Input:
Expected output:
That the program will append to another file all the data below "%th=number", and then when the following segment appears it will save to another file, and the process will continue till the end of this file.
In other words each segment starts with even number so (2, 4, 6, 8 ... 180) so I should get 90 files, each for every segment.
UPDATE 2:
So I have change my code:
with open("S:\Doc\Python\Data\Dekomp\Hth.txt", 'r') as f:
with open("S:\Doc\Python\Data\Dekomp\Hth2.txt", 'w') as g:
for line in f:
if line.startswith("%"):
g.write(line)
if line.endswith("%"):
break
But right now the problem is that if I put this startswith and endswith python will save only headliner, if I delete them, the obivous thing happens, it saves everything from input file.
data = []
filename = "S:\Doc\Python\Data\Dekomp\Hth.txt"
with open(filename) as f:
lines = f.readlines() # Reading file
def _get_all_starting_index(data): # Calculating index of all lines starting with %
return [data.index(line) for line in data if line.startswith("%")]
indices= _get_all_starting_index(lines)
data_info_to_write_in_file = {} # for storing data to write in each individual file
for i in range(len(indices)): # looping over number of indices
key = lines[indices[i]] # key value for starting of a segment.
end_point = indices[i+1] if len(indices) > i+1 else len(indices) # finding end point.
lines_to_get = lines[indices[i]+1 : end_point] # getting lines in between and storing it in dictionary
data_info_to_write_in_file[key] = lines_to_get
for key in data_info_to_write_in_file.keys(): # writing info in each individual file
filename = "S:\Doc\Python\Data\Dekomp\{}.txt".format(key.strip().split("=")[-1])
with open(filename, 'w') as f:
for line in data_info_to_write_in_file[key]:
f.write(line)
Hope it will help.
Feel free to get any info.
I have combined two lists using zip syntax. When I saved it in csv format, whole data are stayed in one cell of excell. what's that i want is: each element of zipped file should be stay on each row.
This is my code:
list_of_first_column=["banana","cat","brown"]
list_of_second_column=["fruit","animal","color"]
graph_zip_file=zip(list_of_first_column,list_of_second_column)
with open('graph.csv', 'w') as csv_file:
writer = csv.writer(csv_file)
writer.writerow(graph_zip_file)
what I want in csv format:
banana,fruit
cat,animal
brown,color
You've got two ways of doing this, assuming that you're using the csv module. You can either use writer.writerows:
list_of_first_column = ["banana", "cat", "brown"]
list_of_second_column = ["fruit", "animal", "color"]
graph_zip_file = zip(list_of_first_column, list_of_second_column)
with open('graph.csv', 'w') as csv_file:
writer = csv.writer(csv_file)
writer.writerows(graph_zip_file)
Or, you can use writer.writerow and a for-loop:
list_of_first_column = ["banana", "cat", "brown"]
list_of_second_column = ["fruit", "animal", "color"]
graph_zip_file = zip(list_of_first_column, list_of_second_column)
with open('graph.csv', 'w') as csv_file:
writer = csv.writer(csv_file)
for row in graph_zip_file
writer.writerow(row)
They both should return the same thing, which is what you've indicated as your desired output.
I hope this proves useful.
I have two parameters like filename and time and I want to write them in a column in a csv file. These two parameters are in a for-loop so their value is changed in each iteration.
My current python code is the one below but the resulting csv is not what I want:
import csv
import os
with open("txt/scalable_decoding_time.csv", "wb") as csv_file:
writer = csv.writer(csv_file, delimiter=',')
filename = ["one","two", "three"]
time = ["1","2", "3"]
zipped_lists = zip(filename,time)
for row in zipped_lists:
print row
writer.writerow(row)
My csv file must be like below. The , must be the delimeter. So I must get two columns.
one, 1
two, 2
three, 3
My csv file now reads as the following picture. The data are stored in one column.
Do you know how to fix this?
Well, the issue here is, you are using writerows instead of writerow
import csv
import os
with open("scalable_decoding_time.csv", "wb") as csv_file:
writer = csv.writer(csv_file, delimiter=',')
level_counter = 0
max_levels = 3
filename = ["one","two", "three"]
time = ["1","2", "3"]
while level_counter < max_levels:
writer.writerow((filename[level_counter], time[level_counter]))
level_counter = level_counter +1
This gave me the result:
one,1
two,2
three,3
Output:
This is another solution
Put the following code into a python script that we will call sc-123.py
filename = ["one","two", "three"]
time = ["1","2", "3"]
for a,b in zip(filename,time):
print('{}{}{}'.format(a,',',b))
Once the script is ready, run it like that
python2 sc-123.py > scalable_decoding_time.csv
You will have the results formatted the way you want
one,1
two,2
three,3
import csv
reader = csv.reader(post.text, quotechar="'")
with open('source91.csv', 'wb') as f:
writer = csv.writer(f)
writer.writerows(list(reader))
output is showing vertically i need to print the data horizantally in CSV
Simple Answer : if you have only one array
with open('source91.csv', 'wb') as f:
writer = csv.writer(f, delimiter='\n')
writer.writerows(list(reader))
Complicated answer:
you may need numpy to make is happen.
transpose will simply converts row to column
import numpy as np
a = np.array(list(reader))
a = np.append(a, list(reader)) # if you have multiple lines
a = np.transpose(a)
np.savetxt('source91.csv', a)
I'm writing a python executable script that does the following:
I want to gather information from a .csv file and read it into python as a dictionary. This .csv file contains several columns of information with headings, and I only want to extract particular columns (those columns with specific headings I want) , and print those columns out to another .csv file. I am using the functions DictReader and DictWriter.
I am reading in the .csv file as a dictionary (with the headings being the key and the column values being the items),and output the information as a dictionary to another .csv file.
After I read it in, I print out the items in the particular headings (so I can double check what I have read it). I then open up a new .csv file and want to write the data (which I have just read in) as a dictionary. I can write in the keys (column headings) but my code doesn't print any of the item values for some reason. The headings that I want in this case are 'Name' and 'DOB'.
Here is my code:
#!/usr/bin/python
import os
import os.path
import re
import sys
import pdb
import csv
csv_file = csv.DictReader(open(sys.argv[1],'rU'),delimiter = ',')
for line in csv_file:
print line['Name'] + ',' + line['DOB']
fieldnames = ['Name','DOB']
test_file = open('test2.csv','wr')
csvwriter = csv.DictWriter(test_file, delimiter=',', fieldnames=fieldnames)
csvwriter.writerow(dict((fn,fn) for fn in fieldnames))
for row in csv_file:
csvwriter.writerow(row)
test_file.close()
Any ideas of where I'm going wrong ? I want to print the item values under their their corresponding column headers in the output file.
I am using python 2.7.11 on a Mac machine. I am also printing values to the terminal.
You're unfortunately tricked by your own testing, that is, the printing of the individual rows. By looping through csv_file initially, you've exhausted the iterator and are at the end. Further iterations, as done in the bottom of your code, are not possible and will be ignored.
Your question is essentially a duplicate of various other question, such as how to read from a CSV file repeatedly. Albeit that the issue here comes up in a different way: you didn't realise what the problem was, while those questions do know the cause, but not the solution.
Answers to those questions tell you to simply reset the file pointer of the input file. Unfortunately, the input file gets closed promptly after reading, in your current code.
Thus, something like this should work:
infile = open(sys.argv[1], 'rU')
csv_file = csv.DictReader(infile ,delimiter = ',')
<all other code>
infile.seek(0)
for row in csv_file:
csvwriter.writerow(row)
test_file.close()
infile.close()
As an aside, just use the with statement when opening files:
with open(sys.argv[1], 'rU') as infile, open('test2.csv', 'wr') as outfile:
csv_file = csv.DictReader(infile ,delimiter = ',')
for line in csv_file:
print line['Name'] + ',' + line['DOB']
fieldnames = ['Name','DOB']
csvwriter = csv.DictWriter(outfile, delimiter=',', fieldnames=fieldnames)
infile.seek(0)
for row in csv_file:
csvwriter.writerow(row)
Note: DictWriter will take care of the header row. No need to write it yourself.