Data in `std::vector` different when using `std::unique_ptr` - c++

I have written a custom class that stores images and eventually computes a calibration based on those images, but I am encountering an issue in the way that the images are stored. I have two overloaded functions that do this, with one reading the images from file using cv::imread, and the other using an intermediate Snapshot data structure for holding the data. The function using cv::imread works fine, but the one using the custom data structure does not. I am trying to store three images right now, and the issue is that as I push the images into a vector, the data for the second images is copied into the first one.
This is the working function:
bool CalibClass::AddImage(const std::string& snapshotPath) {
cv::Mat img = cv::imread(snapshotPath);
// _snapshots is a private member declared as a std::vector<cv::Mat>
_snapshots.push_back(img);
return true;
}
This is the function that is not working:
bool CalibClass::AddImage(const ImageSet& snapshot) {
RGBImage *rgb_image_ptr = snapshot.GetRGBImage();
std::vector<unsigned char> img_data(rgb_image_ptr->GetData());
cv::Mat img(rgb_image_ptr->GetHeight(), rgb_image_ptr->GetWidth(), CV_8UC3, img_data.data());
_snapshots.push_back(img);
return true;
}
The ImageSet class stores images as an std::unique_ptr<RGBImage>. The RGBImage class stores the image data as a std::vector<unsigned char>.
This is how the images are loaded into the class from the main:
cv::Mat img1 = cv::imread("img1.png");
cv::Mat img2 = cv::imread("img2.png");
cv::Mat img3 = cv::imread("img3.png");
int length = img1.total() * img1.elemSize();
std::vector<unsigned char> data1;
std::vector<unsigned char> data2;
std::vector<unsigned char> data3;
for (int i = 0; i < length; i++) {
data1.push_back(img1.data[i]);
}
for (int i = 0; i < length; i++) {
data2.push_back(img2.data[i]);
}
for (int i = 0; i < length; i++) {
data3.push_back(img3.data[i]);
}
CalibClass calib_test;
std::unique_ptr<RGBImage> rgb_image_ptr1(new RGBImage(img1.rows, img1.cols, data1));
ImageSet new_snap1(rgb_image_ptr1, nullptr, 0);
calib_test.AddImage(new_snap1);
std::unique_ptr<RGBImage> rgb_image_ptr2(new RGBImage(img2.rows, img2.cols, data2));
ImageSet new_snap2(rgb_image_ptr2, nullptr, 0);
calib_test.AddImage(new_snap2);
std::unique_ptr<RGBImage> rgb_image_ptr3(new RGBImage(img3.rows, img3.cols, data3));
ImageSet new_snap3(rgb_image_ptr3, nullptr, 0);
calib_test.AddImage(new_snap3);
When I put a break point inside the function and check the content of the _snapshots, the first element is the second image, and the second and third elements are the third image. When I set a break point after all the AddImage() calls, the content of _snapshots has the second image as the first element, the third image as the second element, and the third element has a cv::Mat with invalid data.
What is the reason why the two methods are storing the images differently? What would be the way to fix this issue?

Those symptoms sound a lot like there is a shallow copy going on, which would mean undefined behavior in the second approach since the cv::Mat in the vector outlives img_data. Let me see if I can find documentation for the constructor you used.
Found it here. Yes, it does a shallow copy (emphasis added):
Matrix constructors that take data and step parameters do not allocate matrix data. Instead, they just initialize the matrix header that points to the specified data, which means that no data is copied.
So when the second approach pushes an image onto _snapshots, that image's data lives in the local variable img_data. Then the function ends, making that data invalid. Thus you get undefined behavior when you look at the data.
To solve this, you would need to make sure the data gets copied. You would also want to make sure the data gets freed at some point to avoid a memory leak. One approach is to define a class consisting of a cv::Mat and something to store the data, perhaps a std::vector<unsigned char>. (Use the latter member instead of the local variable img_data.) A starting point might be the following:
class MatWrapper {
public:
explicit MatWrapper(const RGBImage & rgb_image) :
data(rgb_image.GetData()),
image(rgb_image.GetHeight(), rgb_image.GetWidth(), CV_8UC3, data.data())
{}
private:
std::vector<unsigned char> data; // Declaration order matters!
cv::Mat image;
};

Related

Raw pointer / cv::Mat data as shared_ptr

A library API takes an image raw pixel data as std::shared_ptr<uint8_t>.
The image files I have can be in multiple formats (e.g. .png, .jpg, .bmp), so I use cv::imread for practicality.
However, this returns the image as a cv::Mat object, which I still need to convert to std::shared_ptr<uint8_t> to comply with the library API signature.
I have tried similar solutions on the Internet, such as How to save image data in shared pointer and Creating shared_ptr from raw pointer, as well as std::make_shared, either directly from the cv::Mat
std::make_shared<uint8_t>(*mat.data);
or from a std::vector
data.assign(mat.data, mat.data + mat.total() * mat.channels());
std::make_shared<uint8_t>(*data.data());
But all attempts resulted in either double deletion exception, corrupted data or compilation error.
The only way I got it to work was to write the raw pixels data to a file then read it to a shared pointer with
std::shared_ptr<uint8_t> data;
std::ofstream output("temp", std::ios::out | std::ios::binary);
output.write(reinterpret_cast<char*>(mat.data), size);
std::ifstream input("temp", std::ios::in | std::ios::binary);
data.reset(new uint8_t[size], std::default_delete<uint8_t[]>());
input.read((char*)data.get(), size);
Although this works, it bothers me to have such terrible solution for this.
What is the proper way to convert cv::Mat to a shared pointer so that the data complies with the library API signature?
The full example that works, but looks terrible is
#include <opencv2/core.hpp>
#include <opencv2/imgcodecs.hpp>
#include <opencv2/highgui.hpp>
#include <fstream>
int main(int argc,char* argv[])
{
std::string image_path = cv::samples::findFile("starry_night.jpg");
cv::Mat mat = cv::imread(image_path, cv::IMREAD_COLOR);
size_t size = mat.cols * mat.rows * mat.channels();
std::shared_ptr<uint8_t> data;
std::ofstream output("temp", std::ios::out | std::ios::binary);
output.write(reinterpret_cast<char*>(mat.data), size);
std::ifstream input("temp", std::ios::in | std::ios::binary);
data.reset(new uint8_t[size], std::default_delete<uint8_t[]>());
input.read((char*)data.get(), size);
// function(std::shared_ptr<uint8_t> data);
return 0;
}
I use the following CMakeLists.txt to compile it (required OpenCV installed)
project(help)
find_package(OpenCV REQUIRED)
include_directories(${OpenCV_INCLUDE_DIRS})
add_executable(help help.cpp)
target_link_libraries(help ${OpenCV_LIBS})
Its impossible. Copy the matrix or read the image using another library than cv::imread to reverse the ownership semantics if it is a performance concern. OpenCV provides non-owning matrices (constructor from pointer, e.g. feed it with the output of stbi_load()) which you can use as simple view like std::span. However, you can't change that flag manually after the construction of the matrix.
Looking at the source code at modules/core/src/matrix.cpp, it may be possible to add manually the flag cv::UMatData::USER_ALLOCATED to prevent deallocation of memory.
Like that:
#include <opencv2/core.hpp>
int main()
{
cv::Mat img(10, 10, CV_8UC1);
auto ptr = std::shared_ptr<unsigned char>(img.data);
if(img.u)
{
img.u->flags |= cv::UMatData::USER_ALLOCATED;
}
// Single memory deallocation at the end of main()
}
~Mat() calls release() which calls deallocate() which calls u->unmap() and after a few function calls:
if( !(u->flags & UMatData::USER_ALLOCATED) )
{
fastFree(u->origdata);
u->origdata = 0;
}
However, it doesn't seem to work, maybe because OpenCV uses a custom allocator (fastFree()) which may not be compatible with delete called by the std::shared_ptr. This may be the reason why OpenCV don't let release the matrix.
As suggested in the comments, I will copy the data so that I can keep it after deleting cv::Mat
cv::Mat mat = cv::imread(image_path, cv::IMREAD_COLOR);
size_t size = mat.cols * mat.rows * mat.channels();
std::shared_ptr<uint8_t> data(new uint8_t[size]);
std::copy(mat.data, mat.data + size, data.get());
This already looks way better than what I had before, but a better solution would be to cv::Mat release its ownership of the data.

Initial value of reference to non-const must be an lvalue when passing opencv Mat to a function

I have an Mat. I want to update the mat based on certain regions defined by a Rect. When I pass it like the code shown below I get the error mentioned.
void temp2(Mat &A){
//this function updates the value of A in the region defined by the rect.
for(i = 0 to A.rows){
Vec2f *p = reinterpret_cast<Vec2f * >(A.ptr(i));
for(j = 0 to A.cols){
// p[j] = Vec2f(5,5);
}
}
}
void temp1(Mat &A){
Point a(0,0), b(10,10);
Rect t(a,b);
temp2(A(t));
}
void temp(Mat &A){
A = Mat(500, 500, CV_32FC2, Scalar(-1.0, -1.0));
temp1(A);
}
int main(){
Mat A;
temp(A);
}
I looked up the solution and it said to make the mat A const in the temp2 function.I cant make the mat A const in the temp2 function as I have to update the specific region of the mat defined by the rect in the temp2 function. How can I update the specific region in this manner?
Doesn't this work for you?
/* Renamed arg to reflect what's happening.
Let it be A if you so wish but it's not
the original A from temp().
*/
void temp2(Mat &extractedFromA){
// Do stuff
}
void temp1(Mat &A){
Point a(0,0), b(10,10);
Rect t(a,b);
Mat extracted = A(t);
temp2(extracted);
}
You are using this API
Mat cv::Mat::operator() (const Rect & roi)const
which means that on calling A(t), A is unmodified (since above API is a const method) and yields a new Mat object and this is what you need to operate on in temp2(), not the original object A passed into temp1(). Also the changes made to the extracted Mat shall be reflected back to the original Mat A since you're only sharing the headers between them.
Also, the error that you might be hitting into is that since A(t) was yielding a temporary object, and on passing it to temp2(), you were attempting to bind it to a non-const lvalue reference. Making it as a const lvalue reference would fix it but that obviously won't help you.

Destroying vectors of dynamic arrays via destructor in c++

I'm working on one of assignments to do with image manipulation (blending and zooming) and I've ran into a problem that I have hard time overcoming.
Structure of my application
Class Image
rgb struct: contains floats (they're flattened pixels) and overloaded operators.
pixels: a 1d array of pixels which is initialized via constructor to be h * w
Class destructor.
Destructor looks a little like this...
virtual ~Image()
{
if (pixels != NULL)
delete[] pixels;
}
Now I'm using another class called Filter which inherits from Image
class Filter: public class Image
std::vector of Image imgStack; Container for images that I'm going to blend
std::vector of Rgb pixelBuffer; A container for the pixels for each images one pixel. This is not dynamic so I'm not worried about deleting this.
Destructor for the derived class.
~Blend()
{
delete &imageStack;
}
what this part of my main.cpp looks like...
while (userInput != 5)
{
switch(userInput)
case 1:
{
Blend *meanImage = new Blend(3263, 2505, 13);
meanImage->writePPM("Mean_" + std::to_string(count) + ".ppm");//every iteration, the file name is unique
meanImage->~Blend();
}
}
In my main.cpp I'm basically running 13 images into the Blend object which stores the images in the vector container to do all my functionality on. During the run time the space used is around 1.3GB, but since my object is in a loop (i have a menu for multiple functionality) the object doesn't usually leave the scope so the destructor isn't automatically called, so I call it manually like this; medianImage->~Blend(); Now the all the error says is that my application "has triggered a breakpoint" and that's it... Note, no breakpoint is found anywhere. I'm aware that it's generally bad to use dynamic arrays because it causes all sorts of memory problems (if it's done by me), but I want to fix this just so I know how to solve these in the future.
If you have any questions of the code, I can post snippers.
Edit: here's an my Blend class.
#pragma once
#include "stdafx.h"
#include "Image.h"
#include <vector>
#include <string>
class Blend : public Image
{
private:
std::vector<Image> imageStack;
std::vector<Rgb*> pixelBuffer;//only works with pointers (no copying)
int m_noOfImages;
Rgb* getMedianFromSet();
Rgb getMeanFromSet();
Rgb getStdDev(Rgb meanPix);
public:
Blend(int width = 0, int height = 0, int noOfImages = 0):Image(width, height), m_noOfImages(noOfImages), imageStack(noOfImages), pixelBuffer(noOfImages)
{}
public:
void stack(bool push = true);
void meanBlend();
void medianBlend();
void sigmaClipping(float sigma = 0.5f);
~Blend()
{
delete &imageStack;
}
};
#pragma once
#include "stdafx.h"
#include "Image.h"
#include <vector>
#include <string>
#include <memory>
class Blend : public Image
{
private:
std::vector<Image> imageStack;
// Changed to shared_ptr<T> could use unique_ptr<T> depending on need.
std::vector<std::shared_ptr<Rgb>> pixelBuffer;//only works with pointers (no copying)
int m_noOfImages;
Rgb* getMedianFromSet();
Rgb getMeanFromSet();
Rgb getStdDev(Rgb meanPix);
public:
Blend(int width = 0, int height = 0, int noOfImages = 0):Image(width, height), m_noOfImages(noOfImages), imageStack(noOfImages), pixelBuffer(noOfImages)
{}
public:
void stack(bool push = true);
void meanBlend();
void medianBlend();
void sigmaClipping(float sigma = 0.5f);
// Clear Entire Buffer
void cleanup() {
// When using the heap with smart pointers
for ( auto item : containerVariable ) {
item.reset();
item = nullptr;
}
containerVariable.clear();
}
// Remove Single Element
void remove( unsigned idx ) {
// Write function to remove a single element from vector
}
~Blend()
{
// This is definitely wrong here: You are trying to delete a reference
// to a template container that is storing `Image` objects that
// are on the stack.
// delete &imageStack;
}
};
It is better to write a function to clean up memory, and to remove specific elements from containers when using dynamic memory than it is to use a class's destructor.

C++: how to alternate between types if they have to be hardcoded? [duplicate]

I am writing a function in which I need to access to a element of a Mat, but this function can receive Mat of differents types. So, if I have:
filtered.at<TypeofMat>(i) = (typeofmat) somevalue;
It only occurs to me to do something like this filtered.at<myMat.type()>(i), but obviously this does not work because type returns a int
I am stuck, Could someone give some light?
You can convert the source matrix to a matrix of double (type CV_64F). In this way you won't lose any data due to casting. Then you can work on this matrix as usual, since you know its type. Last step is to convert back the destination image to the source type.
You need to know the number of channels of your matrices, though. A CV_assert will make sure that you're working on the correct type.
#include <opencv2/opencv.hpp>
using namespace cv;
void foo(const Mat& src, Mat& dst)
{
// Assert number of channels
CV_Assert(src.channels() == 3);
// Convert to CV64F
Mat3d _src, _dst;
src.convertTo(_src, CV_64F);
_dst.create(_src.size());
// Work on _src and _dst (you know the type)
_dst(0,0) = _src(0,0) + Vec3d(1,2,3);
// Convert _dst to src type
_dst.convertTo(dst, src.type());
}
int main()
{
Mat3b img(10,10,Vec3b(0,0,0));
Mat res;
foo(img, res);
// res will be CV_8UC3
return 0;
}
There are also alternatives to this approach:
create a template function an call the appropriate specialization. See here
work on raw pointers. See here
use only OpenCV functions, that handle correctly every type. See here. This is usually the best option, if available.
If it is a possibility, make the function receiving the openCV Mat a template function:
void f<typename T>(const Mat& m)
{
(void) m.at<T>(0, 0);
}
use it like that:
Mat m1/* ... */;
m1.at<int>(0, 0) = 0;
f<int>(m);

flatbuffer c++: Is it possible to steal from flatbuffers::Vector

I'm new to flatbuffer and I want to know if it's possible to get full (not const*) access to data in flatbuffers::Vector. Looking at the example below, I want to steal the ownership of img2::mem::data to store it in an Img-struct and process it in any way I want. Is this somehow possible, without memcopy-ing?
struct Img
{
int iXCount;
int iYCount;
int iXOffset;
unsigned char *mem;
};
int _tmain(int argc, _TCHAR* argv[])
{
Img img;
//init img;
flatbuffers::FlatBufferBuilder fbb;
auto mem = fbb.CreateVector(img.mem, img.iXOffset * img.iYCount);
auto mloc = CreateImage(fbb, img.iXCount, img.iYCount, img.iXOffset, mem);
fbb.Finish(mloc);
//unpack
auto img2 = flatbuffers::GetRoot<Image>(fbb.GetBufferPointer());
const std::uint8_t*pMem = img2->mem()->data(); //unfortunately only const*
return 0;
}
Your pMem points to data that sits somewhere in the middle of the FlatBuffer you're using. So this means you can access it, but only as long as you can keep the parent buffer around.
Since these are bytes, you could const-cast them, and modify them in place without copying. Note that if you ever try this with things that are not bytes, you have to be aware that data in a FlatBuffer is always little-endian.
Alternatively to const-cast, you can compile your schema with --gen-mutable, which will give you additional accessors to modify data in-place starting from GetMutableRoot, and data() will also be non-const from that.