I have been searching for following Python solution to copy selectively lines from 1 txt file to another. I can copy the whole file, but with only a few lines I get an error.
My code:
f = open(from_file, "r")
g = open(to_file, "w")
#copy = open(to_file, "w") # this instruction copies whole file
rowcond2 = 'xxxx' # look for this string sequence in every line
for line in f:
if rowcond2 in f:
copy.write(line,"w") in g # write every corresponding line to destination
f.close()
# copy.close() # code receive error to close destination
g.close()
So without the rowcond2, I can copy the whole file. Yet with the condition nothing is written to destination file.
Thank you for your help.
Why not to put your condition inside the for loop?
for line in f:
if condition:
copy.write(line)
I have been able to solve this case searching on SO:
Using python to write specific lines from one file to another file
#Lukas Graf: thank you for your detailed step wise explanation.
Related
I am trying to run the script below. The intention of the script is to open different fasta files one after the other, and extract the geneID. The script works well if I don't use the glob.glob function. I get this message TypeError: coercing to Unicode: need string or buffer, list found
files='/home/pathtofiles/files'
#print files
#sys.exit()
for file in files:
fastas=sorted(glob.glob(files + '/*.fasta'))
#print fastas[0]
output_handle=(open(fastas, 'r+'))
genes_files=list(SeqIO.parse(output_handle, 'fasta'))
geneID=genes_files[0].id
print geneID
I am running of ideas on how to direct the script to open when file after another to give me the require information.
I see what you are trying to do, but let me first explain why your current approach is not working.
You have a path to a directory with fasta files and you want to loop over the files in that directory. But observe what happens if we do:
>>> files='/home/pathtofiles/files'
>>> for file in files:
>>> print file
/
h
o
m
e
/
p
a
t
h
t
o
f
i
l
e
s
/
f
i
l
e
s
Not the list of filenames you expected! files is a string and when you apply a for loop on a string you simply iterate over the characters in that string.
Also, as doctorlove correctly observed, in your code fastas is a list and open expects a path to a file as first argument. That's why you get the TypeError: ... need string, ... list found.
As an aside (and this is more a problem on Windows then on Linux or Mac), but it is good practice to always use raw string literals (prefix the string with an r) when working with pathnames to prevent the unwanted expansion of backslash escaped sequences like \n and \t to newline and tab.
>>> path = 'C:\Users\norah\temp'
>>> print path
C:\Users
orah emp
>>> path = r'C:\Users\norah\temp'
>>> print path
C:\Users\norah\temp
Another good practice is to use os.path.join() when combining pathnames and filenames. This prevents subtle bugs where your script works on your machine bug gives an error on the machine of your colleague who has a different operating system.
I would also recommend using the with statement when opening files. This assures that the filehandle gets properly closed when you're done with it.
As a final remark, file is a built-in function in Python and it is bad practice to use a variable with the same name as a built-in function because that can cause bugs or confusion later on.
Combing all of the above, I would rewrite your code like this:
import os
import glob
from Bio import SeqIO
path = r'/home/pathtofiles/files'
pattern = os.path.join(path, '*.fasta')
for fasta_path in sorted(glob.glob(pattern)):
print fasta_path
with open(fasta_path, 'r+') as output_handle:
genes_records = SeqIO.parse(output_handle, 'fasta')
for gene_record in genes_records:
print gene_record.id
This is way I solved the problem, and this script works.
import os,sys
import glob
from Bio import SeqIO
def extracting_information_gene_id():
#to extract geneID information and add the reference gene to each different file
files=sorted(glob.glob('/home/path_to_files/files/*.fasta'))
#print file
#sys.exit()
for file in files:
#print file
output_handle=open(file, 'r+')
ref_genes=list(SeqIO.parse(output_handle, 'fasta'))
geneID=ref_genes[0].id
#print geneID
#sys.exit()
#to extract the geneID as a reference record from the genes_files
query_genes=(SeqIO.index('/home/path_to_file/file.fa', 'fasta'))
#print query_genes[geneID].format('fasta') #check point
#sys.exit()
ref_gene=query_genes[geneID].format('fasta')
#print ref_gene #check point
#sys.exit()
output_handle.write(str(ref_gene))
output_handle.close()
query_genes.close()
extracting_information_gene_id()
print 'Reference gene sequence have been added'
I want to download multiple specific links(images´ urls) into a txt file(or any file where all links can be listed underneath each others).
I get them but the code wrtite each link on the top of the other one and at the end it stays only a link :(. Also I want not repeated urls
def dlink(self, image_url):
r = self.session.get(image_url, stream=True)
with open('Output.txt','w') as f:
f.write(image_url + '\n')
The issue is most simply that opening a file with mode 'w' truncates any existing file. You should change 'w' to 'a' instead. This will open an existing file for writing, but append instead of truncating.
More fundamentally, the problem may be that you are opening the file over and over in a loop. This is very inefficient. The only time the approach you use could be really useful is if your program is approaching the OS-imposed limit on number of open files. If this is not the case, I would recommended putting the loop inside the with block, keeping the mode as 'w' since you open the file just once now, and passing the open file to your dlink function.
Edit
Huge mistake of my part, as it is a method, and you will call it several times, if you open it in write mode ('w') or similar, it will Overwrites the existing file if the file exists.
So, if you use the 'a' way, you can see that:
Opens a file for appending. The file pointer is at the end of the file
if the file exists. That is, the file is in the append mode. If the
file does not exist, it creates a new file for writing.
The other problem radics in image_url is a list, so you need to write it line by line:
def dlink(self, image_url):
r = self.session.get(image_url, stream=True)
with open('Output.txt','a') as f:
for url in list(set(image_url)):
f.write(image_url + '\n')
another way to do it:
your_file = open('Output.txt', 'a')
r = self.session.get(image_url, stream=True)
for url in list(set(image_url)):
your_file.write("%s\n" % url)
your_file.close() #dont forget close it :)
the file open mode is wrong，'w' mode make this file was overwritten every time you open it,not appended to it. replace it to 'a' mode.
you can see this https://stackoverflow.com/a/23566951/8178794 for more detail
Open a file with option w overwrite the file if existring, use the mode a to append data to an existing file.
Try :
import requests
from os.path import splitext
# use mode='a' to append result without erasing filename
def dlink(url, filename, mode='w'):
r = requests.get(url)
if r.status_code != 200:
return
# here the link is valid
with open(filename, mode) as desc:
desc.write(url)
def dimg(img_url, img_name):
r = requests.get(img_url, stream=True)
if r.status_code != 200:
return
_, ext = splitext(img_url)
with open(img_name + ext, 'wb') as desc:
for chunk in r:
desc.write(chunk)
dlink('https://image.flaticon.com/teams/slug/freepik.jpg', 'links.txt')
dlink('https://image.flaticon.com/teams/slug/freepik.jpg', 'links.txt', 'a')
dimg('https://image.flaticon.com/teams/slug/freepik.jpg', 'freepik')
I'm building a little program that reads every line in a log file and if it finds a match it prints that line. The problem is, I have about 20 different log files and they all in the same folder. Is there a way I can parse through every single log file in a folder and print out the line that matches the searched word? Below is an example of what I have so far, but it prints nothing. The script needs to be able to incorporate readlines() and split()
What I have below doesn't work, but this is what I would expect it to look like. Any advice welcome.
def Preview():
path = ('C:Users/kev/Desktop/test/*.log')
files = glob.glob(path)
files.readlines()
for line in files:
if "test_word" in line:
print line
Preview()
This is how your code should look:
def Preview():
path = ('C:Users/kev/Desktop/test/*.log')
files = glob.glob(path)
for f in files:
f = open(f)
f = f.readlines():
for line in f:
if "test_word" in line:
print line
f.close()
Preview()
I have a for loop which creates a CSV of values of several files in a directory.
Within this loop I only want to create the file and write in the header once, currently I am doing this:
#name&path to table file
test = tablefile+"/"+str(cell[:-10])+"_Table.csv"
#write file
if not os.path.isfile(test):
csv.writer(open(test, "wt"))
with open(test, 'w') as output:
wr = csv.writer(output, lineterminator=',')
for val in header_note:
wr.writerow([val])
and to append data I have:
with open(test, 'a') as output:
wr = csv.writer(output, lineterminator=',')
for val in table_all:
wr.writerow([val])
Which works well, however, when I run the script over again another time it will append more data to the bottom of that same .csv. What I want is for the first time through the for-loop, is to just overwrite any existing .csv with a new one with a header then continue on appending data, and overwrite/re-write header once the script is run again. Thanks!
It look like you may have some code problems other than file handling, but here goes: You problem is basically that opening a file in 'w' mode will overwrite everything in the file, and opening in 'a' mode will not allow you to change the header line.
To get around this, you will have to get the contents of the file (if it already exists), then overwrite the file, including those lines that where there to begin with.
You will want something along the lines of:
if os.path.exists(file_name): # if file already exists
with open(file_name, 'r') as in_file: # open it
old_lines = in_file.readlines()[1:] # read all lines from file EXCEPT header line
with open(file_name, 'w') as out_file: # open file again, with 'w' to create/overwrite
out_file.write(new_header_line) # write new header line to file
for line in old_lines:
out_file.write(line) # write all preexisting lines back into file
# continue writing whatever you want.
I have an output file from a code which its name will ends to "_x.txt" and I want to connect two codes which second code will use this file as an input and will add more data into it. Finally, it will ends into "blabla_x_f.txt"
I am trying to work it out as below, but seems it is not correct and I could not solve it. Please help:
inf = str(raw_input(*+"_x.txt"))
with open(inf+'_x.txt') as fin, open(inf+'_x_f.txt','w') as fout:
....(other operations)
The main problem is that the "blabla" part of the file could change to any thing every time and will be random strings, so the code needs to be flexible and just search for whatever ends with "_x.txt".
Have a look at Python's glob module:
import glob
files = glob.glob('*_x.txt')
gives you a list of all files ending in _x.txt. Continue with
for path in files:
newpath = path[:-4] + '_f.txt'
with open(path) as in:
with open(newpath, 'w') as out:
# do something