Alternative ways to perform 'cv::flip(img, img, 1)' - c++

I am receiving an image in the form of raw data from a sensor insied ROS. Following fields are known:
Width: 1920
Height: 1080
Encoding: BGRA8
Bytes Per Pixel: 4
Image Data: 1-D byte array having 8294400 elements //1920 * 1080 * 4 = 8294400
I need to visualize this image in ROS hence I am composing ROS supported image as follows:
std::basic_string<char> color_pixels = color_frame.data();
//Step 1: Create OpenCV image
cv::Mat image(cv::Size(color_frame.width(), color_frame.height()), CV_8UC4, const_cast<char*>(color_pixels.data()), cv::Mat::AUTO_STEP);
//Step 2: Convert OpenCV image to CvBridge image
cv_bridge::CvImage cv_image;
cv_image.header.frame_id = frame_id;
cv_image.encoding = "bgra8";
cv_image.image = image;
//Step 3: Convert CvBridge image to ROS image
sensor_msgs::Image ros_image;
cv_image.toImageMsg(ros_image);
This looks fine. However, during visualization, I noticed that image is flipped. Hence in order to restore it back, I have to use cv::flip before step 2 in following way:
cv::flip(image, image, 1);
Since I have all the raw values and the above process seems long, I decided to compose sensor_msgs::Image directly. However, I am not able to perform cv::flip operation.
Below are my queries:
Why is flip required? Is the raw data captured incorrectly?
Is it possible to perform the operation similar to cv::flip directly in a byte array?
My approach
I tried to reverse the byte array but it didn't work. Please see below the code snippet:
std::basic_string<char> color_pixels = color_data.data();
std::vector<unsigned char> color_values(color_pixels.begin(), color_pixels.end());
std::reverse(color_values.begin(), color_values.end());
sensor_msgs::Image ros_image;
//width, height, encoding etc. are set but not shown here
ros_image.data = color_values;

probably you can set up your camera to flip the image. That is the proper place to specify this. (Sometimes, our cameras are mounted upside down)
since your byte array is {b0, g0, r0, a0, b1, g1, r1, a1, ...}, simply reversing it will result in a {aN, rN, gN, bN, ...}, and your format becomes argb. cv::flip already accounts for this. Just saying " the above process seems long" is not enough reason to do this by yourself: it will complicate your code, and will result in a poor replication of what the opencv guys already provided you with.
If you really want to write that yourself, maybe b = reinterpret_cast<BRGRA*>(&color_data.data().front()), and do the reverse on the reinterpreted 'structs'.
struct BRGRA { char b, g, r, a };
std::reverse(
reinterpret_cast<BRGRA*>( &data.front() ),
reinterpret_cast<BRGRA*>( &data.back() ) + 1,
reinterpret_cast<BRGRA*>( &data.front() ));
-- EDIT
The fact that this answer was accepted without additional comments to the suggested code proves the point that it's hard to provide better than a 'poor replication': above snippet will reverse into , turning it upside-down. (viewed 'only' 40 times, but still...). So it can only work for single-line images.

Related

grayscale image creation 16 bits

I am using openCV for the first time. I am using openCV3 and XCode to code it. I want to create a 16 bit grayscale image but I want to the data I have is defined such that 4000 is the pixel value for white and 0 for black. I have the information for these pixels in an array of type int. How can I create a Mat and assign the values in the array to the Mat?
short data[] = { 0,0,4000,4000,0,0,4000, ...};
Mat gray16 = Mat(h, w, CV_16S, data);
again, the types must match. for 16bit, you need CV_16S and a shortarray, for 8bit CV_8U and a uchar* array, for float CV_32S and a float* ....
You can create your Mat with
cv::Mat m(rows, cols, CV_16UC1);
but to my knowledge, there is no way to define a custom value for "white", you'll have to multiply m with std::numeric_limits::max / 4000. However, this is only necessary when displaying the image.
A lookup-table could do the same (potentially slower), see cv::LUT. However, it appearently only supports 8-bit images.
edit: OK, I missed the part about assigning existing array values; see berak's answer. I hope the answer is still useful.

C++ OpenCV SVM Predict Not Working

Trying to create a functional SVM. I have 114 training images, 60 Positive/54 Negative, and 386 testing images for the SVM to predict against.
I read in the training image features to float like this:
trainingDataFloat[i][0] = trainFeatures.rows;
trainingDataFloat[i][1] = trainFeatures.cols;
And the same for the testing images too:
testDataFloat[i][0] = testFeatures.rows;
testDataFloat[i][2] = testFeatures.cols;
Then, using Micka's answer to this question, I turn the testDataFloat into a 1 Dimensional Array, and feed it to a Mat like this so to predict on the SVM:
float* testData1D = (float*)testDataFloat;
Mat testDataMat1D(height*width, 1, CV_32FC1, testData1D);
float testPredict = SVMmodel.predict(testDataMat1D);
Once this was all in place, there is the Debug Error of:
Sizes of input arguments do not match (the sample size is different from what has been used for training) in cvPreparePredictData
Looking at this post I found (Thanks to berak) that:
"all images (used in training & prediction) have to be the same size"
So I included a re-size function that would re-size the images to be all square at whatever size you wished (100x100, 200x200, 1000, 1000 etc.)
Run it again with the images re-sized to a new directory that the program now loads the images in from, and I get the exact same error as before of:
Sizes of input arguments do not match (the sample size is different from what has been used for training) in cvPreparePredictData
I just have no idea anymore on what to do. Why is it still throwing that error?
EDIT
I changed
Mat testDataMat1D(TestDFheight*TestDFwidth, 1, CV_32FC1, testData1D);
to
Mat testDataMat1D(1, TestDFheight*TestDFwidth, CV_32FC1, testData1D);
and placed the .predict inside the loop that the features are given to the float so that each image is given to the .predict individually because of this question. With the to int swapped so that .cols = 1 and .rows = TestDFheight*TestDFwidth the program seems to actually run, but then stops on image 160 (.exe has stopped working)... So that's a new concern.
EDIT 2
Added a simple
std::cout << testPredict;
To view the determined output of the SVM, and it seems to be positively matching everything until Image 160, where it stops running:
Please check your training and test feature vector.
I'm assuming your feature data is some form of cv::Mat containing features on each row.
In which case you want your training matrix to be a concatenation of each feature matrix from each image.
These line doesn't look right:
trainingDataFloat[i][0] = trainFeatures.rows;
trainingDataFloat[i][1] = trainFeatures.cols;
This is setting an element of a 2d matrix to the number of rows and columns in trainFeatures. This has nothing to do with the actual data that is in the trainFeatures matrix.
What are you trying to detect? If each image is a positive and negative example, then are you trying to detect something in an image? What are your features?
If you're trying to detect an object in the image on a per image basis, then you need a feature vector describing the whole image in one vector. In which case you'd do something like this with your training data:
int N; // Set to number of images you plan on using for training
int feature_size; // Set to the number of features extracted in each image. Should be constant across all images.
cv::Mat X = cv::Mat::zeros(N, feature_size, CV_32F); // Feature matrix
cv::Mat Y = cv::Mat::zeros(N, 1, CV_32F); // Label vector
// Now use a for loop to copy data into X and Y, Y = +1 for positive examples and -1 for negative examples
for(int i = 0; i < trainImages.size(); ++i)
{
X.row(i) = trainImages[i].features; // Where features is a cv::Mat row vector of size N of the extracted features
Y.row(i) = trainImages[i].isPositive ? 1:-1;
}
// Now train your cv::SVM on X and Y.

Overlaying/merging two (and more) YUV images in OpenCV

I investigated and stripped down my previous question (Is there a way to avoid conversion from YUV to BGR?). I want to overlay few images (format is YUV) on the resulting, bigger image (think about it like it is a canvas) and send it via network library (OPAL) forward without converting it to to BGR.
Here is the code:
Mat tYUV;
Mat tClonedYUV;
Mat tBGR;
Mat tMergedFrame;
int tMergedFrameWidth = 1000;
int tMergedFrameHeight = 800;
int tMergedFrameHalfWidth = tMergedFrameWidth / 2;
tYUV = Mat(tHeader->height * 1.5f, tHeader->width, CV_8UC1, OPAL_VIDEO_FRAME_DATA_PTR(tHeader));
tClonedYUV = tYUV.clone();
tMergedFrame = Mat(Size(tMergedFrameWidth, tMergedFrameHeight), tYUV.type(), cv::Scalar(0, 0, 0));
tYUV.copyTo(tMergedFrame(cv::Rect(0, 0, tYUV.cols > tMergedFrameWidth ? tMergedFrameWidth : tYUV.cols, tYUV.rows > tMergedFrameHeight ? tMergedFrameHeight : tYUV.rows)));
tClonedYUV.copyTo(tMergedFrame(cv::Rect(tMergedFrameHalfWidth, 0, tYUV.cols > tMergedFrameHalfWidth ? tMergedFrameHalfWidth : tYUV.cols, tYUV.rows > tMergedFrameHeight ? tMergedFrameHeight : tYUV.rows)));
namedWindow("merged frame", 1);
imshow("merged frame", tMergedFrame);
waitKey(10);
The result of above code looks like this:
I guess the image is not correctly interpreted, so the pictures stay black/white (Y component) and below them, we can see the U and V component. There are images, which describes the problem well (http://en.wikipedia.org/wiki/YUV):
and: http://upload.wikimedia.org/wikipedia/en/0/0d/Yuv420.svg
Is there a way for these values to be correctly read? I guess I should not copy the whole images (their Y, U, V components) straight to the calculated positions. The U and V components should be below them and in the proper order, am I right?
First, there are several YUV formats, so you need to be clear about which one you are using.
According to your image, it seems your YUV format is Y'UV420p.
Regardless, it is a lot simpler to convert to BGR work there and then convert back.
If that is not an option, you pretty much have to manage the ROIs yourself. YUV is commonly a plane-format where the channels are not (completely) multiplexed - and some are of different sizes and depths. If you do not use the internal color conversions, then you will have to know the exact YUV format and manage the pixel copying ROIs yourself.
With a YUV image, the CV_8UC* format specifier does not mean much beyond the actual memory requirements. It certainly does not specify the pixel/channel muxing.
For example, if you wanted to only use the Y component, then the Y is often the first plane in the image so the first "half" of whole image can just be treated as a monochrome 8UC1 image. In this case using ROIs is easy.

obtaining cv::Scalar from cv::Mat of unknown type

after reading an image of unknown depth and channel number i want to access its pixels one by one.
on opencv 1.x the code goes:
IplImage * I = cvLoadImage( "myimage.tif" );
CvScalar pixel = cvGet2D( I, y, x );
but on opencv 2.x the cv::Mat.at() method demands that i know the image's type:
cv::Mat I = cv::imread( "myimage.tif" );
if( I.depth() == CV_8U && I.channels() == 3 )
cv::Vec3b pixel = I.at<cv::Vec3b>( x, y );
else if( I.depth() == CV_32F && I.channels() == 1 )
float pixel = I.at<cv::float>( x, y );
is there a function resembling cvGet2D that can receive cv::Mat and return cv::Scalar without knowing the image's type in compile time?
For someone who is really a beginner in C++ ...
... and/or a hacker who just need to save mere seconds of code typing to finish off the last project
cv::Mat mat = ...; // something
cv::Scalar value = cv::mean(mat(cv::Rect(x, y, 1, 1)));
(Disclaimer: This code is only slightly less wasteful than a young man dying for a revolutionary cause.)
The short answer is no. There's no such function in the C++ API.
The rationale behind this is performance. cv::Scalar (and CvScalar) is the same thing as cv::Vec<double,4>. So, for any Mat type other than CV_64FC4, you'll need a conversion to obtain cv::Scalar. Moreover, this method would be a giant switch, like in your example (you have only 2 branches).
But I suppose quite often this function would be convenient, so why not to have it? My guess is that people would tend to overuse it, resulting in really bad performance of their algorithms. So, OpenCV makes it just a tiny bit less convenient to access individual pixels, in order to force client code to use statically typed methods. This isn't such a big deal convenient-wise, since more often than not, you actually know the type statically and it's a really big deal performance-wise. So, I consider it a good trade-off.
I had the same issue, I just wanted to test something quickly and performance was not an issue. But all parts of the code uses cv::Mat(). What I did was the following
Mat img; // My input mat, initialized elsewhere
// Pretty fast operation, Will only create an iplHeader pointing to the data in the mat
// No data is copied and no memory is mallocated.
// The Header resides on the stack (note its type is "IplImage" not "IplImage*")
IplImage iplImg = (IplImage)img;
// Then you may use the old (slow converting) legacy-functions if you like
CvScalar s = cvGet2D( &iplImg, y, x );
Just a warning: you are using cvLoadImage and imread with default flags. This means that any image you read will be a 8-bit 3-channel image. Use appropriate flags (IMREAD_ANYDEPTH / IMREAD_ANYCOLOR) if you want to read image as is (which seems to be your intention).

changing the frame type without loosing data

I'm using Opencv and I have two frame : cv::frame1 and cv::frame2, when I check their type I get :
frame1.type() is 16
frame2.type() is 21
my Question is how can I change the type of frame2 to 16 without losing its data ?? I tried frame2.convertTo() it didn't work .
any Idea ?
From what I understand you want to convert an image of type CV_32FC3 (3 channel float) to CV_8UC3 (3 channel unsigned char). This cannot be done without losing some information: In the first case you have 4 bytes per pixel and in the second 1 byte per pixel, so as you understand there is going to be some loss.
This code is used to convert to a different type and works fine for me:
cv::Mat A = cv::Mat(480, 640, CV_32FC3, CV_RGB(1.0,1.0,1.0));
cv::Mat B;
A.convertTo(B, CV_8UC3);