Related
I don't know why I cant figure this out. But I have a column of numbers that I would like to turn into a list of strings. I should of mention this when i initially posted this but this isn't a DataFrame or did it come from a file this is a result of a some code, sorry wasn't trying to waste anybody's time, I just didn't want to add a bunch of clutter. This is exactly how it prints out.
Here is my column of numbers.
3,1,3
3,1,3
3,1,3
3,3,3
3,1,1
And I would like them to look like this.
['3,1,3', '3,1,3', '3,1,3', '3,3,3', '3,1,1']
I'm trying to find a way that is not dependent on how many numbers are in each row or how many sets of numbers are in the column.
Thanks, really appreciate it.
Assume you start with a DataFrame
df = pd.DataFrame([[3, 1, 3], [3, 1, 3], [3, 1, 3], [3, 3, 3], [3, 1, 1]])
df.astype(str).apply(lambda x: ','.join(x.values), axis=1).values.tolist()
Looks like:
['3,1,3', '3,1,3', '3,1,3', '3,3,3', '3,1,1']
def foo():
l = []
with open("file.asd", "r") as f:
for line in f:
l.append(line)
return l
To turn your dataframe in to strings, use the astype function:
df = pd.DataFrame([[3, 1, 3], [3, 1, 3], [3, 1, 3], [3, 3, 3], [3, 1, 1]])
df = df.astype('str')
Then manipulating your columns becomes easy, you can for instance create a new column:
In [29]:
df['temp'] = df[0] + ',' + df[1] + ',' + df[2]
df
Out[29]:
0 1 2 temp
0 3 1 3 3,1,3
1 3 1 3 3,1,3
2 3 1 3 3,1,3
3 3 3 3 3,3,3
4 3 1 1 3,1,1
And then compact it into a list:
In [30]:
list(df['temp'])
Out[30]:
['3,1,3', '3,1,3', '3,1,3', '3,3,3', '3,1,1']
# Done in Jupyter notebook
# add three quotes on each side of your column.
# The advantage to dataframe is the minimal number of operations for
# reformatting your column of numbers or column of text strings into
# a single string
a = """3,1,3
3,1,3
3,1,3
3,3,3
3,1,1"""
b = f'"{a}"'
print('String created with triple quotes:')
print(b)
c = a.split('\n')
print ("Use split() function on the string. Split on newline character:")
print(c)
print ("Use splitlines() function on the string:")
print(a.splitlines())
I'm trying to write a program that takes two functions:
count_word_lengths which takes the argument text, a string of text, and returns a default dictionary that records the count for each word length. An example call to this function:
top5_lengths which takes the same argument text and returns a list of the top 5 word lengths.
Note: that in the event that
two lengths have the same frequency, they should be sorted in descending order. Also, if there are fewer than 5 word lengths it should return a shorter list of the sorted word lengths.
Example calls to count_word_lengths:
count_word_lengths("one one was a racehorse two two was one too"):
defaultdict(<class 'int'>, {1: 1, 3: 8, 9: 1})
Example calls to top5_lengths:
top5_lengths("one one was a racehorse two two was one too")
[3, 9, 1]
top5_lengths("feather feather feather chicken feather")
[7]
top5_lengths("the swift green fox jumped over a cool cat")
[3, 5, 4, 6, 1]
My current code is this, and seems to output all these calls, however it is failing a hidden test. What type of input am I not considering? Is my code actually behaving correctly? If not, how could I fix this?
from collections import defaultdict
length_tally = defaultdict(int)
final_list = []
def count_word_lengths(text):
words = text.split(' ')
for word in words:
length_tally[len(word)] += 1
return length_tally
def top5_word_lengths(text):
frequencies = count_word_lengths(text)
list_of_frequencies = frequencies.items()
flipped = [(t[1], t[0]) for t in list_of_frequencies]
sorted_flipped = sorted(flipped)
reversed_sorted_flipped = sorted_flipped[::-1]
for item in reversed_sorted_flipped:
final_list.append(item[1])
return final_list
One thing to note is that you do not account for an empty string. That would cause count() to return null/undefined. Also you can use iteritems() during list comprehension to get the key and value from a dict like for k,v in dict.iteritems():
I'm not a Python guy, but I can see a few things that might cause issues.
You keep referring to top5_lengths, but your code has a function called top5_word_lengths.
You use a function called count_lengths that isn't defined anywhere.
Fix these and see what happens!
Edit:
This shouldn't impact your code, but it's not great practice for your functions to update variables outside their scope. You probably want to move the variable assignments at the top to functions where they're used.
Not really an answer, but an alternative way of tracking words instead of just lengths:
from collections import defaultdict
def count_words_by_length(text):
words = [(len(word),word) for word in text.split(" ")]
d = defaultdict(list)
for k, v in words:
d[k].append(v)
return d
def top_words(dict, how_many):
return [{"word_length": length, "num_words": len(words)} for length, words in dict.items()[-how_many:]]
Use as follows:
my_dict = count_words_by_length('hello sir this is a beautiful day right')
my_top_words = num_top_words_by_length(my_dict, 5)
print(my_top_words)
print(my_dict)
Output:
[{'word_length': 9, 'num_words': 1}]
defaultdict(<type 'list'>, {1: ['a'], 2: ['is'], 3: ['sir', 'day'], 4: ['this'], 5: ['hello', 'right'], 9: ['beautiful']})
Say I have two lists:
a=[1,2,3]
b=[4,5,6]
I want to write them into a text file such that I obtain a two column text file:
1 4
2 5
3 6
Simply zip the list, and write them to a csv file with tab as the delimiter:
>>> a=[1,2,3]
>>> b=[4,5,6]
>>> zip(a,b)
[(1, 4), (2, 5), (3, 6)]
>>> import csv
>>> with open('text.csv', 'w') as f:
... writer = csv.writer(f, delimiter='\t')
... writer.writerows(zip(a,b))
...
>>> quit()
$ cat text.csv
1 4
2 5
3 6
You can use numpy.savetxt(), which is a convenient tool from the numpy library.
A minimal example would be as follows:
import numpy as np
xarray = np.array([0, 1, 2, 3, 4, 5])
yarray = np.array([0, 10, 20, 30, 40, 50])
# here is your data, in two numpy arrays
data = np.column_stack([xarray, yarray])
datafile_path = "/your/data/output/directory/datafile.txt"
np.savetxt(datafile_path , data, fmt=['%d','%d'])
# here the ascii file is written.
The fmt field in np.savetxt() in the example specifies that the numbers are integers.
You can use a different format for each column.
E.g. to specify floating point format, with 2 decimal digits and 10 characters wide columns, you would use '%10.2f'.
Try this:
file = open("list.txt", "w")
for index in range(len(a)):
file.write(str(a[index]) + " " + str(b[index]) + "\n")
file.close()
A simple solution is to write columns of fixed-width text:
a=[1,2,3]
b=[4,5,6]
col_format = "{:<5}" * 2 + "\n" # 2 left-justfied columns with 5 character width
with open("foo.csv", 'w') as of:
for x in zip(a, b):
of.write(col_format.format(*x))
Then cat foo.csv produces:
1 4
2 5
3 6
The output is both human and machine readable, whereas tabs can generate messy looking output if the precision of the values varies along the column. It also avoids loading the separate csv and numpy libraries, but works with both lists and arrays.
You can write two lists into a text file that contains two columns.
a=[1,2,3]
b=[4,5,6]
c = [a, b]
with open("list1.txt", "w") as file:
for x in zip(*c):
file.write("{0}\t{1}\n".format(*x))
Output in the text file:
1 4
2 5
3 6
It exits a straightway to save and stack same vectors length in columns. To do so use the concatenate function, you can then stack 3,4 or N vectors in columns delimitered by a tab.
np.savetxt('experimental_data_%s_%.1fa_%dp.txt'%(signal,angle,p_range), np.c_[DCS_exp, p_exp], delimiter="\t")
I am getting an error and I'm not sure how to fix it.
The following seems to work:
def random(row):
return [1,2,3,4]
df = pandas.DataFrame(np.random.randn(5, 4), columns=list('ABCD'))
df.apply(func = random, axis = 1)
and my output is:
[1,2,3,4]
[1,2,3,4]
[1,2,3,4]
[1,2,3,4]
However, when I change one of the of the columns to a value such as 1 or None:
def random(row):
return [1,2,3,4]
df = pandas.DataFrame(np.random.randn(5, 4), columns=list('ABCD'))
df['E'] = 1
df.apply(func = random, axis = 1)
I get the the error:
ValueError: Shape of passed values is (5,), indices imply (5, 5)
I've been wrestling with this for a few days now and nothing seems to work. What is interesting is that when I change
def random(row):
return [1,2,3,4]
to
def random(row):
print [1,2,3,4]
everything seems to work normally.
This question is a clearer way of asking this question, which I feel may have been confusing.
My goal is to compute a list for each row and then create a column out of that.
EDIT: I originally start with a dataframe that hase one column. I add 4 columns in 4 difference apply steps, and then when I try to add another column I get this error.
If your goal is add new column to DataFrame, just write your function as function returning scalar value (not list), something like this:
>>> def random(row):
... return row.mean()
and then use apply:
>>> df['new'] = df.apply(func = random, axis = 1)
>>> df
A B C D new
0 0.201143 -2.345828 -2.186106 -0.784721 -1.278878
1 -0.198460 0.544879 0.554407 -0.161357 0.184867
2 0.269807 1.132344 0.120303 -0.116843 0.351403
3 -1.131396 1.278477 1.567599 0.483912 0.549648
4 0.288147 0.382764 -0.840972 0.838950 0.167222
I don't know if it possible for your new column to contain lists, but it deinitely possible to contain tuples ((...) instead of [...]):
>>> def random(row):
... return (1,2,3,4,5)
...
>>> df['new'] = df.apply(func = random, axis = 1)
>>> df
A B C D new
0 0.201143 -2.345828 -2.186106 -0.784721 (1, 2, 3, 4, 5)
1 -0.198460 0.544879 0.554407 -0.161357 (1, 2, 3, 4, 5)
2 0.269807 1.132344 0.120303 -0.116843 (1, 2, 3, 4, 5)
3 -1.131396 1.278477 1.567599 0.483912 (1, 2, 3, 4, 5)
4 0.288147 0.382764 -0.840972 0.838950 (1, 2, 3, 4, 5)
I use the code below it is just fine
import numpy as np
df = pd.DataFrame(np.array(your_data), columns=columns)
I'm trying to create a function that will take the input of the user to create a 3x4 matrix, then find the sum of each column individually. But I'm not sure how to set up the input so that I'll be given each number individually instead of a long string.
def testMatrixFunctions():
row0 = input("Enter a 3-by-4 matrix row for row 0: ")
row1 = input("Enter a 3-by-4 matrix row for row 1: ")
row2 = input("Enter a 3-by-4 matrix row for row 2: ")
I could also use some help with adding the columns, but my biggest concern is the input at the moment.
Use the str.split() method:
>>> "1 2 3 4 5".split()
['1', '2', '3', '4', '5']
And then convert each string into an integer:
>>> map(int, "1 2 3 4 5".split())
[1, 2, 3, 4, 5]
>>> [int(c) for c in "1 2 3 4 5".split()]
[1, 2, 3, 4, 5]
Use split() function, which will split the input and you can 'eval' each element.
[eval(eachItem) for eachItem in inputList]
That will accept input regardless of the datatype.