I have a list of images that have a shape of (3, 64, 64), read them and store them in a List images.
Then i applied stack to the List :
images = np.stack(images)
i got this error :
File "/usr/local/lib/python2.7/dist-packages/numpy/core/shape_base.py", line 350, in stack
raise ValueError('need at least one array to stack')
ValueError: need at least one array to stack'
I will be grateful to have anyone's idea about this.
I can reproduce your error with:
In [94]: images=[]
In [95]: np.stack(images)
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-95-adab3e1812bc> in <module>()
----> 1 np.stack(images)
/usr/local/lib/python3.5/dist-packages/numpy/core/shape_base.py in stack(arrays, axis, out)
347 arrays = [asanyarray(arr) for arr in arrays]
348 if not arrays:
--> 349 raise ValueError('need at least one array to stack')
350
351 shapes = set(arr.shape for arr in arrays)
ValueError: need at least one array to stack
It raises the error because this is True:
In [97]: not images
Out[97]: True
Related
I want to move an element from the training list to the validation list. Without torch tensors the method works, as in the following example:
test = [[2,1],[3,2],[4,4],[5,67]]
element = test[2]
test.remove(element)
print(test)
Out: [[2, 1], [3, 2], [5, 67]]
My question now is why it doesn't work when I do it from train_data to validation_data with a list of tensors? And is there a way to make it work?
I eventually could do it before converting to tensors, but I prefer afterward.
One element, which I got from the train_data has the dimension: 1 64 3 1080 1920
The error I get is:
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_22400/338233821.py in <module>
18 validation_data.append(element)
19 print(len(validation_data), len(validation_data[0]), len(validation_data[0][0]), len(validation_data[0][0][0]), len(validation_data[0][0][0][0]))
---> 20 train_data.remove(element)
21
RuntimeError: Boolean value of Tensor with more than one value is ambiguous
From this post I know that somewhere the tensor is casted into a boolean, but where?
I figured out that using train_data.pop(index) works. But it's still not clear to me why .remove() doesn't work. I assume there is somewhere an if case in the .remove() function which triggers the error.
I want to XOR the values in array using numpy. When using:
numpy.logical_xor([1,0,0,1,0,1,0,1])
I get:
Traceback (most recent call last):
File "test.py", line 44, in <module>
print np.logical_xor([1,0,0,1,0,1,0,1])
ValueError: invalid number of arguments
It seems numpy does not accept more than 2 elements in an array. Is there a workaround?
It seems you are looking for the reduction method. So, use -
np.logical_xor.reduce([1,0,0,1,0,1,0,1])
I am reading a csv and try to take make a linear regression model based on df['LSTAT'] (x/variable) v.s. df['MEDV'] (y/target). However, the error message " ValueError: Found arrays with inconsistent numbers of samples: [ 1 343]" keeps poping out during the model fitting stage.
I have shape/re-shape the data (not sure if I have done correctly) or transform the pd.DataFrame into numpy arrays and lists. None of them works. I still don't quite understand the issue after reading this post: sklearn: Found arrays with inconsistent numbers of samples when calling LinearRegression.fit() . The scripts and the error messages are below.
Could any guru offer some solutions with detailed explanations? Thank you!
import scipy.stats as stats
import pylab
import numpy as np
import matplotlib.pyplot as plt
import pylab as pl
import sklearn
from sklearn.cross_validation import train_test_split
from sklearn import datasets, linear_model
from sklearn.linear_model import LinearRegression
df=pd.read_csv("input.csv")
X_train1, X_test1, y_train1, y_test1 = train_test_split(df['LSTAT'],df['MEDV'],test_size=0.3,random_state=1)
lin=LinearRegression()
################## This line: " lin_train=lin.fit(X_train1,y_train1)" causes the trouble.
lin_train=lin.fit(X_train1,y_train1)
################## The followings are just the plotting lines after fitting the Linear regression
# The coefficients
print('Coefficients: \n', lin.coef_)
# The mean square error
print("Residual sum of squares: %.2f"
% np.mean((lin.predict(X_test1) - y_test1) ** 2))
# Explained variance score: 1 is perfect prediction
print('Variance score: %.2f' % lin.score(X_test1, y_test1))
# Plot outputs
plt.scatter(X_test1, y_test1, color='black')
plt.plot(X_test1, lin.predict(X_test1), color='blue',linewidth=3)
plt.xticks(())
plt.yticks(())
plt.show()
Here is the warning & error message:
Warning (from warnings module):
File "C:\Python27\Lib\site-packages\sklearn\utils\validation.py", line 386
DeprecationWarning)
DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and willraise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.
Traceback (most recent call last):
File "C:/Users/Pin-Chih/Google Drive/Real_estate_projects/test.py", line 36, in <module>
lin_train=lin.fit(X_train1,y_train1)
File "C:\Python27\Lib\site-packages\sklearn\linear_model\base.py", line 427, in fit
y_numeric=True, multi_output=True)
File "C:\Python27\Lib\site-packages\sklearn\utils\validation.py", line 520, in check_X_y
check_consistent_length(X, y)
File "C:\Python27\Lib\site-packages\sklearn\utils\validation.py", line 176, in check_consistent_length
"%s" % str(uniques))
ValueError: Found arrays with inconsistent numbers of samples: [ 1 343]>>>
If I print out the "x_train1":
X_train1:
61 26.82
294 12.86
39 29.29
458 4.85
412 8.05
Name: LSTAT, dtype: float64
If I print out the "y_train1":
y_train1:
61 13.4
294 22.5
39 11.8
458 35.1
412 29.0
Name: MEDV, dtype: float64
Certainly not a guru but I've had similar problems in the past because the model is expecting the X argument to have at least 2 dimensions, even if the second dimension is 1. The first thing I would try would be to replace
lin_train=lin.fit(X_train1,y_train1)
with
lin_train=lin.fit(X_train1.reshape(X_train1.shape[0], 1), y_train1)
which should give you data with shape (343, 1) rather than just 343.
I'm trying to implement a simple logistic regression model trained with my own set of images, but I am getting this error when I try to train the model:
Traceback (most recent call last):
File "main.py", line 26, in <module>
model.entrenar_modelo(sess, training_images, training_labels)
File "/home/jr/Desktop/Dropbox/Machine_Learning/TF/Míos/Hip/model_log_reg.py", line 24, in entrenar_modelo
train_step.run({x: batch_xs, y_: batch_ys})
File "/home/jr/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1267, in run
_run_using_default_session(self, feed_dict, self.graph, session)
File "/home/jr/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2763, in _run_using_default_session
session.run(operation, feed_dict)
File "/home/jr/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 334, in run
np_val = np.array(subfeed_val, dtype=subfeed_t.dtype.as_numpy_dtype)
ValueError: setting an array element with a sequence.
The data I'm feeding to train_step.run({x: batch_xs, y_: batch_ys}) is like this:
batch_xs: list of tensor objects representing images of 100x100 (10,000 long tensors)
batch_ys: list of labels as floats (1.0 or 0.0)
What am I doing wrong?
Edits
It seems the problem was that I had to evaluate the tensors in batch_xs before passing them to train_step.run(...). I thought the run method would take care of that, but I guess I was wrong?
Anyway, so once I did this before calling the function:
for i, x in enumerate(batch_xs):
batch_xs[i] = x.eval()
#print batch_xs[i].shape
#assert all(x.shape == (100, 100, 3) for x in batch_xs)
# Now I can call the function
I had several issues even after doing what is suggested in the answers below. I finally fixed everything by ditching tensors and using numpy arrays.
This particular error is coming out of numpy. Calling np.array on a sequence with a inconsistant dimensions can throw it.
>>> np.array([1,2,3,[4,5,6]])
ValueError: setting an array element with a sequence.
It looks like it's failing at the point where tf ensures that all the elements of the feed_dict are numpy.arrays.
Check your feed_dict.
The feed_dict argument to Operation.run() (also Session.run() and Tensor.eval()) accepts a dictionary mapping Tensor objects (usually tf.placeholder() tensors) to a numpy array (or objects that can be trivially converted to a numpy array).
In your case, you are passing batch_xs, which is a list of numpy arrays, and TensorFlow does not know how to convert this to a numpy array. Let's say that batch_xs is defined as follows:
batch_xs = [np.random.rand(100, 100),
np.random.rand(100, 100),
..., # 29 rows omitted.
np.random.rand(100, 100)] # len(batch_xs) == 32.
We can convert batch_xs into a 32 x 100 x 100 array using the following:
# Convert each 100 x 100 element to 1 x 100 x 100, then vstack to concatenate.
batch_xs = np.vstack([np.expand_dims(x, 0) for x in batch_xs])
print batch_xs.shape
# ==> (32, 100, 100)
Note that, if batch_ys is a list of floats, this will be transparently converted into a 1-D numpy array by TensorFlow, so you should not need to convert this argument.
EDIT: mdaoust makes a valid point in the comments: If you pass a list of arrays into np.array (and therefore as the value in a feed_dict), it will automatically be vstacked, so there should be no need to convert your input as I suggested. Instead, it sounds like you have a mismatch between the shapes of your list elements. Try adding the following:
assert all(x.shape == (100, 100) for x in batch_xs)
...before the call to train_step.run(), and this should reveal whether you have a mismatch.
I have a code that opens a file, calculates the median value and writes that value to a separate file. Some of the files maybe empty so I wrote the following loop to check it the file is empty and if so skip it, increment the count and go back to the loop. It does what is expected for the first empty file it finds ,but not the second. The loop is below
t = 15.2
while t>=11.4:
if os.stat(r'C:\Users\Khary\Documents\bin%.2f.txt'%t ).st_size > 0:
print("All good")
F= r'C:\Users\Documents\bin%.2f.txt'%t
print(t)
F= np.loadtxt(F,skiprows=0)
LogMass = F[:,0]
LogRed = F[:,1]
value = np.median(LogMass)
filesave(*find_nearest(LogMass,LogRed))
t -=0.2
else:
t -=0.2
print("empty file")
The output is as follows
All good
15.2
All good
15.0
All good
14.8
All good
14.600000000000001
All good
14.400000000000002
All good
14.200000000000003
All good
14.000000000000004
All good
13.800000000000004
All good
13.600000000000005
All good
13.400000000000006
empty file
All good
13.000000000000007
Traceback (most recent call last):
File "C:\Users\Documents\Codes\Calculate Bin Median.py", line 35, in <module>
LogMass = F[:,0]
IndexError: too many indices
A second issue is that t somehow goes from one decimal place to 15 and the last place seems to incrementing whats with that?
Thanks for any and all help
EDIT
The error IndexError: too many indices only seems to apply to files with only one line example...
12.9982324 0.004321374
If I add a second line I no longer get the error can someone explain why this is? Thanks
EDIT
I tried a little experiment and it seems numpy does not like extracting a column if the array only has one row.
In [8]: x = np.array([1,3])
In [9]: y=x[:,0]
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
<ipython-input-9-50e27cf81d21> in <module>()
----> 1 y=x[:,0]
IndexError: too many indices
In [10]: y=x[:,0].shape
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
<ipython-input-10-e8108cf30e9a> in <module>()
----> 1 y=x[:,0].shape
IndexError: too many indices
In [11]:
You should be using try/except blocks. Something like:
t = 15.2
while t >= 11.4:
F= r'C:\Users\Documents\bin%.2f.txt'%t
try:
F = np.loadtxt(F,skiprows=0)
LogMass = F[:,0]
LogRed = F[:,1]
value = np.median(LogMass)
filesave(*find_nearest(LogMass,LogRed))
except IndexError:
print("bad file: {}".format(F))
else:
print("file worked!")
finally:
t -=0.2
Please refer to the official tutorial for more details about exception handling.
The issue with the last digit is due to how floats work they can not represent base10 numbers exactly. This can lead to fun things like:
In [13]: .3 * 3 - .9
Out[13]: -1.1102230246251565e-16
To deal with the one line file case, add the ndmin parameter to np.loadtxt (review its doc):
np.loadtxt('test.npy',ndmin=2)
# array([[ 1., 2.]])
With the help of a user named ajcr, found the problem was that ndim=2 should have been used in numpy.loadtxt() to insure that the array always 2 has dimensions.
Python uses indentation to define if while and for blocks.
It doesn't look like your if else statement is fully indented from the while.
I usually use a full 'tab' keyboard key to indent instead of 'spaces'