I want to augmente my data using data generator in keras as below:
datagen = ImageDataGenerator(
featurewise_center=True, # set input mean to 0 over the dataset
samplewise_center=True, # set each sample mean to 0
featurewise_std_normalization=True, # divide inputs by std of the dataset
samplewise_std_normalization=True, # divide each input by its std
zca_whitening=True, # apply ZCA whitening
rotation_range=0, # randomly rotate images in the range (degrees, 0 to 180)
rescale=1./255,
shear_range=0.2,
zoom_range=0,
width_shift_range=0, # randomly shift images horizontally (fraction of total width)
height_shift_range=0, # randomly shift images vertically (fraction of total height)
horizontal_flip=True, # randomly flip images
vertical_flip=True) # randomly flip images
but I use this network for regression not classification. I have my doubts that datagenerator set new output values or not? Because If I used 0 or 1 classification problem then datagenerator could flip or rotate data without changing output but in here it should change output like input. Does that do this?
Thanks in advance.
Related
Background: I am working with satellite observations of SST (time, lat, lon) from the CoRTAD SST dataset (netCDF file). I have a set of (lon,lat) coordinates following the coastal contour of Portugal (called below midshelf locations). I want to extract the SST time series at each of these midshelf locations, average them and subtract the SST at the same latitudes but a fixed longitude to give a coastal SST index.
The midshelf lon,lats were determined first from a nautical chart, which were then linearly interpolated to the lon,lats in the CoRTAD grid.
How can this be done using CDO?
The lon,lats from the nautical map are given below.
midshelf lon,lats from the nautical map:
-8.000 43.860
-9.000 43.420
-9.350 43.220
-9.388 42.893
-9.000 42.067
-8.935 41.308
-9.000 40.692
-9.278 40.000
-9.324 39.550
-9.518 39.387
-9.777 38.883
-9.285 38.378
-8.909 38.293
-8.951 38.000
-8.965 37.953
-8.917 37.833
-8.913 37.667
-8.915 37.500
-8.975 37.333
-9.017 37.167
-9.045 37.000
So here is my attempt to answer the question as it was stated in the comments (i.e. you wanted an index which was the midshelf locations averaged and then subtracting the same latitude SST sampled at Longitude=9E). I assume the locations are stored pair-wise in a text file called "locations.txt" as in your question above. The loop part of the answer is from one of this question's solutions.
# first loop over the pairs of indices in the text files.
while read -r -a fields; do
for ((i=0; i < ${#fields[#]}; i += 2)); do
# precise lon/lat for sampled mid-shelf
cdo remapnn,"lon=${fields[i]}/lat=${fields[i+1]}" in.nc pt_${i}.nc
# same lat but lon=9E (change as wanted)
cdo remapnn,"lon=9/lat=${fields[i+1]}" in.nc 9E_${i}.nc
done
done < example.txt
# now take the ensemble average over the points.
cdo ensmean pt_*.nc mid_shelf_sst.nc
cdo ensmean 9E_*.nc mid_shelf_9E.nc
# and calculate the index
cdo sub mid_shelf_sst.nc mid_shelf_9E.nc SST_index.nc
I'm trying to animate 2D vector with gnuplot. I want to show one line i.e, one vector at a time.
My Data Structure is as follows: They x,y,u,v
2.24448 0.270645 1.00 1.00
3.24448 0.270645 0.500 1.20
I'm able to create a static plot sing following command:
plot "datam.dat" using 1:2:3:4 with vectors filled head lw 3
Here is the output:
Here is my question: I would like to animate and show one row (i.e,) one vector at a time, how to accomplish this in GNU plot using GIF?
Thanks
Animated GIFs are created with set terminal gif animate. Check help gif for details.
Below is a simple example (tested with gnuplot 5.2). You have to make a new plot for each frame. So, put your plot command into a do for-loop. With every ::i::i you are plotting only the ith line (check help every). If you don't know the total number of lines of your datafile, do stats "YourFile.dat" and the variable STATS_records will tell you this number.
Code:
### animated graph with vectors
reset session
set term gif size 300,300 animate delay 12 loop 0 optimize
set output "AnimateVectors.gif"
# create some dummy data
set angle degrees
N = 60
set samples N
set table $Data
plot [0:360] '+' u (cos($1)):(sin($1)):(sin($1)):(cos($1)) w table
unset table
set xrange[-2.5:2.5]
set yrange[-2.5:2.5]
do for [i=0:N-1] {
plot $Data u 1:2:3:4 every ::i::i w vectors lw 2 lc rgb "red" notitle
}
set output
### end of code
Result:
Addition:
This would be the non-animated version, e.g. in a wxt-terminal.
Code:
### non-animated graph with vectors
reset session
set term wxt size 400,400
# create some dummy data
set angle degrees
N = 60
set samples N
set table $Data
plot [0:360] '+' u (cos($1)):(sin($1)):(sin($1)):(cos($1)) w table
unset table
set xrange[-2.5:2.5]
set yrange[-2.5:2.5]
plot $Data u 1:2:3:4 w vectors lw 1.5 lc rgb "red" notitle
### end of code
Result:
Addition2:
Do you maybe mean something like this? A "semi"-animated arrow? By the way, as you can see the arrow look quite different in gif and wxt terminal.
Code:
### "semi"-animated graph with vectors
reset session
set term gif size 300,300 animate delay 12 loop 0 optimize
set output "AnimateVectorsSemi.gif"
# create some dummy data
set angle degrees
N = 60
set samples N
set table $Data
plot [0:360] '+' u (cos($1)):(sin($1)):(sin($1)):(cos($1)) w table
unset table
set xrange[-2.5:2.5]
set yrange[-2.5:2.5]
do for [i=0:N-1] {
plot $Data u 1:2:3:4 every ::0::i w vectors lw 1.5 lc rgb "red" notitle
}
set output
### end of code
Result:
I am using OpenCV 3.4.1.
I am working on a video classification project and am trying to use KNearest to classify between 2 categories. I have 8 areas of interest in each video frame. To make decision on each frame, each KNearest is done on the pixel values on each area. The majority win (and favor to one category if it is a tie). So, I have 8 sets of training data (one for each area of interest).
Problem: The response generated from the knn model changed when I labelled the categories differently.
The training data sets are organized as rows of:
[category label], data0, data1, data2....etc. (different dimensions for each
training set)
where dataX = pixel data of a frame (1 row = 1 frame)
Then, I build the model by:
Ptr<TrainData> tdata = TrainData::loadFromCSV(filename, 0, 0, -1, String("cat"));
Mat raw = tdata->getTrainSamples();
Mat res = tdata->getResponses();
PCA pca(raw, noArray(), PCA::DATA_AS_ROW, 0.99);
Mat knnIn = pca.project(raw);
Ptr<ml::KNearest> knn = ml::KNearest::create();
knn->train(knnIn, ml::ROW_SAMPLE, res);
After that, testing data is passed to the pca and knn to get the response.
To testing it, I put 1 set of testing data to the 8 knn models.
If I use 0 & 1 as the [category label] in the training data sets, the 8 responses from KNN are 1,1,1,1,1,0,0,1.
If I change the label to 2 & 1 instead (replace all '0' by '2' in the first column of the training data), the 8 responses becomes 1,1,1,1,1,1,1,1, while I am expecting 1,1,1,1,1,2,2,1.
Some observations:
no editing error on the training data(while changing the category labels).
The KNN model isClassifier()=true. DefaultK=10. AlgorithmType=1 (BRUTE_FORCE).
The result is consistent with the same training data set and testing data set.
I don't see any pattern on the difference responses for the 2 label sets (after using different
training data sets and testing data sets).
Please shed some light. Thank you very much!
For instance, given the data in a text file:
10:37:18.459 1
10:37:18.659 0
10:37:19.559 1
How could this be displayed as an image that looked like a square wave that correctly represented the high time and low time? I am trying both gnuplot and scipy. The result should ultimately include more than one sensor, and all plots would have to be displayed above one another so as to show a time delta.
The code in the following link creates a square wave from the formulas listed,
link to waveforms. How can the lower waveform (pwm) be driven by the numbers above if they were in a file (to show a high state for 200 ms, then a low state for 100 ms, and finally a high state)?
If I understood your question correctly you want to plot a step function based on timedata. To avoid further guessing please specify in more detail.
In gnuplot there is the plotting style with steps. Check help steps.
Code:
### display waveform as steps
reset sesion
$Data <<EOD
10:37:18.459 1
10:37:18.659 0
10:37:19.559 1
10:37:19.789 0
10:37:20.123 1
10:37:20.456 0
10:37:20.789 1
EOD
set yrange [-0.05:1.2]
myTimeFmt = "%H:%M:%S" # input time format
set format x "%M:%.1S" time # output time format on x axis
plot $Data u (timecolumn(1,myTimeFmt)):2 w steps lc rgb "red" lw 2 ti "my square wave"
### end of code
Result:
The answer I ended up with was:
file_info = os.stat( self.__outfile)
if file_info.st_size:
x,y,z,a = np.genfromtxt( self.__outfile, delimiter=',',unpack=True )
fig = plt.figure(self.__outfile)
ax = fig.add_subplot(111)
fig.canvas.draw()
test_array = [(datetime.datetime.utcfromtimestamp(e2).strftime('%d_%H:%M:%S.%f')).rstrip('0') for e2 in x]
plt.xticks(x, test_array)
l1, = plt.plot(x,y, drawstyle='steps-post')
l2, = plt.plot(x,a-2, drawstyle='steps-post')
l3, = plt.plot(x,z-4, drawstyle='steps-post')
ax.grid()
ax.set_xlabel('Time (s)')
ax.set_ylabel('HIGH/LOW')
ax.set_ylim((-6.5,1.5))
ax.set_title('Sensor Sequence')
fig.autofmt_xdate()
ax.legend([l1,l2, l3],['sprinkler','lights', 'alarm'], loc='lower left')
plt.show()
I had a input file that had convertDateToFloat values in it. That was passed in to this function. The name is perhaps misleading (__outfile), but on the previous function, it was the output.
Assume that I have given only one test image.
I extract images,labels using mnist.test.next_batch(100).
When i give 1 test image, I am getting 2 images (same image duplicated)
When I give 2 test images, I am getting 4 (2 images duplicated).
Same problem exists for training images.
I consoled the length of test images and train images inside read_data_sets method(...tensorflow/contrib/learn/python/learn/datasets/mnist.py).
It is giving correct length.
Here is my code.
from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True, validation_size=0)
images, labels = mnist.test.next_batch(100)
print len(images) #double the actual length