I am reading images from google drive mounted to google colab. I have two folders, one with positive covid-19 chest x-rays and another with normal chest x-rays. I am trying to show these images side-by-side for comparison. Here are images of the code and error:
First Lines Of Code
Error Image
Here is also the written code:
Cimages = ('/content/drive/My Drive/Data/Covid')
Nimages = ('/content/drive/My Drive/Data/Normal')
import skimage
from skimage.transform import resize
def plot(i):
normal = cv2.imread(dataset +'Normal//' + Nimages[i])
normal = skimage.transform.resize(normal, (150,150,3))
covid = cv2.imread(dataset +'Covid//' + Cimages[i])
covid = skimage.transform.resize(normal, (150,150,3), mode = reflect)
pair = np.concatenate((normal, covid), axis = 1)
print('Normal vs. Covid')
plt.figure(figsize=(10,5))
plt.imshow(pair)
plt.show()
for i in range(0,3):
plot(i)
This gives me an error:
AttributeError Traceback (most recent call last)
<ipython-input-52-237aff042641> in <module>()
1 for i in range(0,3):
----> 2 plot(i)
<ipython-input-50-85bb2e03725c> in plot(i)
3 def plot(i):
4 normal = cv2.imread(dataset +'Normal//' + Nimages[i])
----> 5 normal = skimage.transform.resize(normal, (150,150,3))
6 covid = cv2.imread(dataset +'Covid//' + Cimages[i])
7 covid = skimage.transform.resize(normal, (150,150,3), mode = reflect)
/usr/local/lib/python3.6/dist-packages/skimage/transform/_warps.py in resize(image, output_shape, order, mode, cval, clip, preserve_range, anti_aliasing, anti_aliasing_sigma)
89 output_shape = tuple(output_shape)
90 output_ndim = len(output_shape)
---> 91 input_shape = image.shape
92 if output_ndim > image.ndim:
93 # append dimensions to input_shape
AttributeError: 'NoneType' object has no attribute 'shape'
So it seems to be occurring in the skimage.tranform.resize line of code. Please help.
The issue is not with the function
skimage.tranform.resize
but with the reading of the image
normal = cv2.imread(dataset +'Normal//' + Nimages[i])
not sure what are you trying to do there but Nimages[i] won't give you first file in a folder but it will yield first character of a string, in your case /. Then you will send dataset variable + Normal// + / which is basically in your case Normal/// and then you will try to read image on that path, but without doubt the is no image there in which case opencv will return to you None (which is basically nothing). and then you try to resize None with skimage which will fail.
better option would be to read the image directly or in a loop that could look somehow like this:
from os import listdir
from os.path import isfile, join
onlyfiles = [f for f in listdir(Nimages) if isfile(join(Nimages , f))]
for image_path in onlyfiles:
normal = cv2.imread(join(Nimages , image_path))
normal = skimage.transform.resize(normal, (150,150,3))
assuming that there are only images in your mentioned directory.
Related
Windows 10
Python 3.7
Anaconda 1.9.7
Spyder 3.3.3
PsychoPy for Python 2.7
I am coding an experiment that needs to present images in a random order for the participant to respond to. I am able to get the images in an array, but to present them one at a time I am using a while loop with a variable that increases by 1 every time it goes through the loop. It is not recognizing the variable as a number and therefore the array cannot call anything.
I've tried not randomizing the variable to see if that is the issue, but it just seems to be that my variable i is not being read as a number
#import packages
import random, os
from psychopy import core, visual, event
from PIL import Image
#setup screen with specs and draw
win = visual.Window([400, 300], monitor="testMonitor")
message = visual.TextStim(win, text="")
message.draw()
win.flip()
core.wait(3.0)
#set image size and populate array with images
stim_size = (0.8, 0.8)
image = [i for i in os.listdir('C:/Users/*/psychopy-tests')
if i.endswith('.bmp')]
#randomize image order
images = random.shuffle(image)
this is where my issue seems to be
i = 0
while i != 29: #there are only 28 images
new = images[i] #this is where the issue is
image_stim = Image.open(new)
stim = visual.ImageStim(win, image_stim, size = (stim_size))
stim.draw()
win.update()
output = []
if event.getKeys(keyList=['space']):
output[i] = 1
if event.getKeys(['escape']):
win.close()
core.quit()
if event.getKeys(keyList=None):
output[i] = 0
core.wait(5.0)
i = i + 1
The random.shuffle shuffles in place and doesn’t return anything i.e., It returns None.
Therefor images is None and not subscriptable.
source
I'm trying to use the TensorFlow's Boston housing price example to learn how to use TensorFlow/Keras for regressions, but I keep running into a problem using my own data, even when I make as small of changes as possible. After giving up on writing everything myself, I simply changed the two lines of the code that input the data:
boston_housing = keras.datasets.boston_housing
(train_data, train_labels), (test_data, test_labels) = boston_housing.load_data()
to something that, after looking online, should also create numpy arrays from my csv:
np_array = genfromtxt('trainingdata.csv', delimiter=',')
np_array = np.delete(np_array, (0), axis=0) # Remove header
test_np_array = np_array[:800,:]
tr_np_array = np_array[800:,:] # Separate out test and train data
train_labels = tr_np_array[:, 20] # Get the last row for the labels
test_labels = test_np_array[:, 20]
train_data = np.delete(tr_np_array, (20), axis=1)
test_data = np.delete(test_np_array, (20), axis=1) # Remove the last row so the data is only the features
Everything I can look at seems right – the shapes of the arrays are all correct, the arrays do seem to be correct-looking numpy arrays, the features do seem to become normalized, etc. and yet when I set verbose to 1 on model.fit(...), the very first lines of output show a problem with loss:
Epoch 1/500
32/2560 [..............................] - ETA: 18s - loss: nan - mean_absolute_error: nan
2016/2560 [======================>.......] - ETA: 0s - loss: nan - mean_absolute_error: nan
2560/2560 [==============================] - 0s 133us/step - loss: nan - mean_absolute_error: nan - val_loss: nan - val_mean_absolute_error: nan
I'm especially confused because every other place on stack overflow where I've seen the "TensorFlow loss is 'NaN'" error, it has generally a) been with a custom loss function, and b) once the model has trained for a while, not (as here) within the first 52 passes. Where that's not the case, it's because the data wasn't normalized, but I do that later in the code, and the normalization works for the housing pricing example and prints out numbers clustered around 0. At this point, my best guess is that it's a problem with the genfromtxt command, but if anyone can see what I'm doing wrong or where I might find my issue, I'd be incredibly appreciative.
Edit:
Here is the full code for the program. Commenting out lines 13 through 26 and uncommenting lines 10 and 11 make the program work perfectly. Commenting out lines 13 and 14 and uncommenting 16 and 17 was my attempt at using pandas, but that led to the same errors.
import tensorflow as tf
from tensorflow import keras
import numpy as np
from numpy import genfromtxt
import pandas as pd
print(tf.__version__)
# boston_housing = keras.datasets.boston_housing # Line 10
# (train_data, train_labels), (test_data, test_labels) = boston_housing.load_data()
np_array = genfromtxt('trainingdata.csv', delimiter=',') # Line 13
np_array = np.delete(np_array, (0), axis=0)
# df = pd.read_csv('trainingdata.csv') # Line 16
# np_array = df.get_values()
test_np_array = np_array[:800,:]
tr_np_array = np_array[800:,:]
train_labels = tr_np_array[:, 20]
test_labels = test_np_array[:, 20]
train_data = np.delete(tr_np_array, (20), axis=1)
test_data = np.delete(test_np_array, (20), axis=1) # Line 26
order = np.argsort(np.random.random(train_labels.shape))
train_data = train_data[order]
train_labels = train_labels[order]
mean = train_data.mean(axis=0)
std = train_data.std(axis=0)
train_data = (train_data - mean) / std
test_data = (test_data - mean) / std
labels_mean = train_labels.mean(axis=0)
labels_std = train_labels.std(axis=0)
train_labels = (train_labels - labels_mean) / labels_std
test_labels = (test_labels - labels_mean) / labels_std
def build_model():
model = keras.Sequential([
keras.layers.Dense(64, activation=tf.nn.relu,
input_shape=(train_data.shape[1],)),
keras.layers.Dense(64, activation=tf.nn.relu),
keras.layers.Dense(1)
])
optimizer = tf.train.RMSPropOptimizer(0.001)
model.compile(loss='mse',
optimizer=optimizer,
metrics=['mae'])
return model
model = build_model()
model.summary()
EPOCHS = 500
early_stop = keras.callbacks.EarlyStopping(monitor='val_loss', patience=20)
history = model.fit(train_data, train_labels, epochs=EPOCHS,
validation_split=0.2, verbose=1,
callbacks=[early_stop])
[loss, mae] = model.evaluate(test_data, test_labels, verbose=0)
print("Testing set Mean Abs Error: ${:7.2f}".format(mae * 1000 * labels_std))
I'm interested in augmenting my dataset with random image transformations. I'm using Keras ImageDataGenerator, and I'm getting the following error when trying to apply random_transform to a single image:
--> x = apply_transform(x, transform matrix, img_channel_axis, fill_mode, cval)
>>> RuntimeError: affine matrix has wrong number of rows.
I found the source code for the ImageDataGenerator here. However, I'm not sure how to debug the runtime error. Below is the code I have:
from keras.preprocessing.image import img_to_array, load_img
from keras.preprocessing.image import ImageDataGenerator
from keras.applications.inception_v3 import preprocess_input
image_path = './figures/zebra.jpg'
#data augmentation
train_datagen = ImageDataGenerator(
rotation_range=40,
width_shift_range=0.2,
height_shift_range=0.2,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True,
fill_mode='nearest')
print "\nloading image..."
image = load_img(image_path, target_size=(299, 299))
image = img_to_array(image)
image = np.expand_dims(image, axis=0) # 1 x input_shape
image = preprocess_input(image)
train_datagen.fit(image)
image = train_datagen.random_transform(image)
The error occurs at the last line when calling random_transform.
The problem is that random_transform expects a 3D-array.
See the docstring:
def random_transform(self, x, seed=None):
"""Randomly augment a single image tensor.
# Arguments
x: 3D tensor, single image.
seed: random seed.
# Returns
A randomly transformed version of the input (same shape).
"""
So you'll need to call it before np.expand_dims.
I'm using code from this github, following this tutorial.
There are few changes that I did, because I'm training the CNN on my data. But, probably there is a problem with the changes I performed in 'create_lmdb.py' file. the difference between the two databases are:
first: I'm training my network with 32x32 images.
second: my database includes only grayscale images.
but - I train my network for binary classification too.
After the modifications, this is my file:
import os
import glob
import random
import numpy as np
import cv2
import caffe
from caffe.proto import caffe_pb2
import lmdb
#Size of images
IMAGE_WIDTH = 32
IMAGE_HEIGHT = 32
def transform_img(img, img_width=IMAGE_WIDTH, img_height=IMAGE_HEIGHT):
#Histogram Equalization
img = cv2.equalizeHist(img)
#img[:, :, 1] = cv2.equalizeHist(img[:, :, 1]) not a RGB
#img[:, :, 2] = cv2.equalizeHist(img[:, :, 2])
#Image Resizing
img = cv2.resize(img, (img_width, img_height), interpolation = cv2.INTER_CUBIC) # make sure all the images are at the same size
return img
def make_datum(img, label):
#image is numpy.ndarray format. BGR instead of RGB
return caffe_pb2.Datum(
channels=1, #not an RGB image
width=IMAGE_WIDTH,
height=IMAGE_HEIGHT,
label=label,
data=img.tostring())
train_lmdb = '/home/roishik/Desktop/Thesis/Code/cafe_cnn/first/input/train_lmdb'
validation_lmdb = '/home/roishik/Desktop/Thesis/Code/cafe_cnn/first/input/validation_lmdb'
os.system('rm -rf ' + train_lmdb)
os.system('rm -rf ' + validation_lmdb)
train_data = [img for img in glob.glob("../input/train/*png")]
test_data = [img for img in glob.glob("../input/test1/*png")]
#Shuffle train_data
random.shuffle(train_data)
print 'Creating train_lmdb'
in_db = lmdb.open(train_lmdb, map_size=int(1e12))
with in_db.begin(write=True) as in_txn:
for in_idx, img_path in enumerate(train_data):
if in_idx % 6 == 0:
continue
img = cv2.imread(img_path, cv2.IMREAD_GRAYSCALE)
img = transform_img(img, img_width=IMAGE_WIDTH, img_height=IMAGE_HEIGHT)
if 'cat' in img_path:
label = 0
else:
label = 1
datum = make_datum(img, label)
in_txn.put('{:0>5d}'.format(in_idx), datum.SerializeToString())
print '{:0>5d}'.format(in_idx) + ':' + img_path
in_db.close()
print '\nCreating validation_lmdb'
in_db = lmdb.open(validation_lmdb, map_size=int(1e12))
with in_db.begin(write=True) as in_txn:
for in_idx, img_path in enumerate(train_data):
if in_idx % 6 != 0:
continue
img = cv2.imread(img_path, cv2.IMREAD_GRAYSCALE)
img = transform_img(img, img_width=IMAGE_WIDTH, img_height=IMAGE_HEIGHT)
prec=int(img_path[(img_path.index('prec_')+5):(img_path.index('prec_')+8)])
if prec>50:
label = 1
else:
label = 0
datum = make_datum(img, label)
in_txn.put('{:0>5d}'.format(in_idx), datum.SerializeToString())
print '{:0>5d}'.format(in_idx) + ':' + img_path
in_db.close()
print '\nFinished processing all images'
But I think that according to the training results: the .mdb output file is corrupted (maybe empty or something - even its weights 47MB).
Can anyone see something wrong with this file? or, alternatively, give me a link to a good tutorial about building lmdb files?
Really appreciate your help!
Thanks
Ok, I solved it!
After looking deeper into the code I noticed that I updated only the labels of the validation dataset (and skipped the training data) :P
it can be seeing in this piece of code:
img = transform_img(img, img_width=IMAGE_WIDTH, img_height=IMAGE_HEIGHT)
if 'cat' in img_path:
label = 0
else:
label = 1
that belongs to the original tutorial.
Conclusion: if you can't access your lmdb file, mabey it's because the function that creates it is broken.
If you want to create an 'lmdb' image dataset for training a classification net, don't sweat! Caffe already has a tool just for that purpose!
You are looking for $CAFFE_ROOT/build/tools/convert_imageset tool and you can find quite a detailed (if I may say so;) tutorial here.
I was trying to match two images using OpenCV ORB as explained in this tutorial.
Here is my code:
import numpy as np
import cv2
import six
import pyparsing
import dateutil
from matplotlib import pyplot as plt
import timeit
import os
import sys
img1_path = 'img1.jpg'
img2_path = 'img2.jpg'
img1 = cv2.imread(img1_path,0) # queryImage
img2 = cv2.imread(img2_path,0) # trainImage
orb = cv2.ORB()
kp1, des1 = orb.detectAndCompute(img1,None)
kp2, des2 = orb.detectAndCompute(img2,None)
FLANN_INDEX_LSH = 6
index_params= dict(algorithm = FLANN_INDEX_LSH,
table_number = 6, # 12
key_size = 12, # 20
multi_probe_level = 1) #2
search_params = dict(checks = 50)
flann = cv2.FlannBasedMatcher(index_params, search_params)
matches = flann.knnMatch(des1,des2,k=2)
if len(matches)>0:
print "%d total matches found" % (len(matches))
else:
print "No matches were found - %d" % (len(good))
sys.exit()
# store all the good matches as per Lowe's ratio test.
good = []
for m,n in matches:
if m.distance < 0.6*n.distance:
good.append(m)
I ran this script with two images that are quite similar. In most cases the script works fine and finds matching key-points.
However, in some cases I get this error (it refers to the last three lines of code):
Traceback (most recent call last):
for m,n in matches:
ValueError: need more than 1 value to unpack
It happens when img2 is significantly a smaller sub-image of img1.
(if img2 is the original image, and img1 is the modified images, it means that someone added details to the original image).
If I switch between the file names img1,img2 then the script runs with no problems.
Must be the query image (img1) smaller, or equal to the train image (img2)?
Each member of the matches list must be checked whether two neighbours really exist. This is independent of image sizes.
good = []
for m_n in matches:
if len(m_n) != 2:
continue
(m,n) = m_n
if m.distance < 0.6*n.distance:
good.append(m)