I tried to convert a WriteableBitmap to a cv::Mat in a c++/cx Microsoft universial App. But when I try to progress with the created Mat, I get the following error:
This is my Code:
void App1::MainPage::processImage(SoftwareBitmap^ bitmap)
{
WriteableBitmap^ wb = ref new WriteableBitmap(bitmap->PixelWidth, bitmap->PixelHeight);
bitmap->CopyToBuffer(wb->PixelBuffer);
Mat img_image(wb->PixelHeight, wb->PixelWidth, CV_8UC3,(void*)wb->PixelBuffer);
//next step results in error
cvtColor(img_image, img_image, CV_BGR2BGRA);
...
}
So my final question:
How to convert the SoftwareBitmap or the WriteableBitmap to a cv::Mat?
The problem is that PixelBuffer is not a void *, it is an IBuffer^.
To get at the raw data, you can either use the IBufferByteAccess interface if you're comfortable with COM programming, or you can initialize a DataReader with an IBuffer if you'd prefer to stay in WinRT (although this technique will make a copy of the data).
I used the DataReader to solve the problem:
void App1::MainPage::processImage(SoftwareBitmap^ bitmap)
{
WriteableBitmap^ wb = ref new WriteableBitmap(bitmap->PixelWidth, bitmap->PixelHeight);
bitmap->CopyToBuffer(wb->PixelBuffer);
IBuffer^ buffer = wb->PixelBuffer;
auto reader = ::Windows::Storage::Streams::DataReader::FromBuffer(buffer);
BYTE *extracted = new BYTE[buffer->Length];
reader->ReadBytes(Platform::ArrayReference<BYTE>(extracted, buffer->Length));
Mat img_image(wb->PixelHeight, wb->PixelWidth, CV_8UC4, extracted);
cvtColor(img_image, img_image, CV_RGBA2BGRA);
...
}
Thx to Peter Torr for the hint.
Related
I am really new to Flutter and C++. So I am struggling with this one. I want to do some transformation to an image through ffi->c++ (opencv). Here's my code on the Flutter side:
class Processvalue extends Struct {
Pointer<Utf8> image;
factory Processvalue.allocate(Pointer image) =>
malloc<Processvalue>().ref
..image = image;
}
I am basically accepting a pointer back from C++ side. Then I call C++ native function like so:
...
final processImage = nativeEdgeDetection
.lookup<NativeFunction<ProcessImageFunction>>("process_image")
.asFunction<ProcessImageNativeFunction>();
...
On the C++ side, in the process_image function I am using the following:
...
Mat dst = Mat::zeros(maxHeight, maxWidth, CV_8UC1);
Mat transformation_matrix = getPerspectiveTransform(img_pts, dst_pts);
warpPerspective(img, dst, transformation_matrix, dst.size());
uchar * arr = dst.isContinuous()? dst.data: dst.clone().data;
Basically, I am sending image path from Flutter to C++ through ffi package. And in C++ grab the image from this file and create opencv Mat variable dst, which will hold the new transformed image. Then I convert this Mat variable to uchar * arr, which I am returning back to Flutter (Pointer to arr). However in Flutter I am not able to convert this back to image data (Uint8List for example).
I have seen people use cv::imwrite(path, dst); to save transformation directly to the same file. But for me I want to keep the original file and only want to get the new image as data back in Flutter and show it through MemoryImage(...Uint8List variable...).
How do I send image data back to Flutter from C++? At the end I would like to have a Uint8List. Is uchar * arr even a good option for this?
I have tried the following but it gives me an invalid image data error:
int imageArrLength = _result.ref.image.length;
^- Pointer back from C++
Uint8List image = _result.ref.image.cast<Uint8>().asTypedList(imageArrLength);
I'm trying to show LiveView image in real time. I use EDSDK 2.14 + Qt5 + opencv+mingw32 under Windows. I'm not very sophisticated in image processing so now I have the following problem. I use example from Canon EDSDK and all was ok until this part of code:
//
// Display image
//
I googled a lot of examples but all of them was written on C# or MFC or VB. Also I found advise to use libjpegTurbo for decompressing image and then showing it using opencv. I tried to use libjpegTurbo but failed to undestand what to do :(. Maybe somebody here have code example of the conversion LiveView stream to opencv Mat or QImage (because I use Qt)?
Here is what worked for me after following the SAMPLE 10 from the Canon EDSDK Reference. It's a starting point for a more robust solution.
In the downloadEvfData function, I replaced the "Display Image" part by the code bellow:
unsigned char *data = NULL;
EdsUInt32 size = 0;
EdsSize coords ;
// get image coordinates
EdsGetPropertyData(evfImage, kEdsPropsID_Evf_CoordinateSystem, 0, sizeof(coords), &coords);
// get buffer pointer and size
EdsGetPointer(stream, (EdsVoid**)&data);
EdsGetLenth(stream, &size);
//
// release stream and evfImage
//
// create mat object
Mat img(coords.height, coords.width, CV_8U, data);
image = imdecode(img, CV_LOAD_IMAGE_COLOR);
I've also changed the function signature:
EdsError downloadEvfData(EdsCameraRef camera, Mat& image)
And in the main function:
Mat image;
namedWindow("main", WINDOW_NORMAL);
startLiveView(camera);
for(;;) {
dowloadEvfData(camera, image);
imshow("main", image);
if (waitkey(10) >= 0);
break;
}
Based on the Canon EDSDKs example, you may append your EdsStreamRef 'stream' data with its correct length into a QByteArray. Then, use for example the following to parse the raw data from the QByteArray as a JPG into a new QImage:
QImage my_image = QImage::fromData(limagedata,"JPG"); Once it's in a QImage you can convert it into a OpenCV cv::Mat (see How to convert QImage to opencv Mat)
Well it depends on the format of the liveview-stream.
There must be some kind of delimiter in it and you need then to convert each image and update your QImage with it.
Check out this tutorial for more information: Canon EDSDK Tutorial in C#
QImage img = QImage::fromData(data, length, "JPG");
m_image = QImageToMat(img);
// -----------------------------------------
cv::Mat MainWindow::QImageToMat(QImage& src)
{
cv::Mat tmp(src.height(),src.width(),CV_8UC4,(uchar*)src.bits(),src.bytesPerLine());
cv::Mat result = tmp.clone();
return result;
}
// -------------------------
void MainWindow::ShowVideo()
{
namedWindow("yunhu",WINDOW_NORMAL);
while(1)
{
requestLiveViewImage();
if(m_image.data != NULL)
{
imshow("yunhu", m_image);
cvWaitKey(50);
}
}
}
How do I convert a WriteableBitmap into a BitmapImage object in Silverlight5 ?
So far, here is my attempt in converting a WriteableBitmap into a BitmapImage; however, it doesn't work.
No exception is thrown, the resulting BitmapImage is simply an 'empty' object with no data; both PixelHeight and PixelWidth are zero:
public BitmapImage Convert(WriteableBitmap wb)
{
int[] p = wb.Pixels;
int len = p.Length * 4;
byte[] result = new byte[len];
Buffer.BlockCopy(p, 0, result, 0, len);
MemoryStream ms = new MemoryStream(result);
BitmapImage bi = new BitmapImage();
bi.SetSource(ms);
// bi.PixelHeight and bi.PixelWidth are zero.
return bi; // pretty much an empty image.
}
Cross-post here: http://social.msdn.microsoft.com/Forums/silverlight/en-US/0a0c833b-8e68-4c8d-871a-d0805d84ba16/writeablebitmap-to-bitmapimage?forum=silverlightpivotviewer
To summarize the discussion on your cross-post:
Yes, you have to convert your image to either png or jpg to create a BitmapImage.
The stream the BitmapImage expects is an encoded image file, but the bytes you get from WriteableBitmap is an uncompressed, raw ARGB pixel format.
In order to produce either a png or a jpg you have to use either link against a library that can do this, or use Stegman's one-page png encoder (that doesn't actually compress anything).
Since this was already suggested in the forum you asked but you noticed that the link is broken, I put a public gist on github from the copy I had locally.
Given the following
Bitmap raw image data in char array
Image width and height
Path wzAppDataDirectory in std::wstring generated using the following code
// Get a good path.
wchar_t wzAppDataDirectory[MAX_PATH];
wcscpy_s( wzAppDataDirectory, MAX_PATH, Windows::Storage::ApplicationData::Current->LocalFolder->Path->Data() );
wcscat_s( wzAppDataDirectory, MAX_PATH, (std::wstring(L"\\") + fileName).c_str() );
How can we save the image as JPG? (Include encoding as well as the char array is raw bitmap form)
Code example is very much appreciated.
You'll need to use a library to encode the JPEG. Some possibilities are the Independent JPEG Group's jpeglib, stb_image, or DevIL.
This is an example code which I obtained from my friend.
It uses OpenCV's Mat data structure. Note that, you need to ensure the unsigned char data array within cv::Mat is in continuous form. cv::cvtColor will do the trick (Or, cv::Mat.clone).
Take note, do not use OpenCV's imwrite. As at current time of writing, imwrite doesn't pass Windows Store Certification Test. It is using several APIs, which is prohibited in WinRT.
void SaveMatAsJPG(const cv::Mat& mat, const std::wstring fileName)
{
cv::Mat tempMat;
cv::cvtColor(mat, tempMat, CV_BGR2BGRA);
Platform::String^ pathName = ref new Platform::String(fileName.c_str());
task<StorageFile^>(ApplicationData::Current->LocalFolder->CreateFileAsync(pathName, CreationCollisionOption::ReplaceExisting)).
then([=](StorageFile^ file)
{
return file->OpenAsync(FileAccessMode::ReadWrite);
}).
then([=](IRandomAccessStream^ stream)
{
return BitmapEncoder::CreateAsync(BitmapEncoder::JpegEncoderId, stream);
}).
then([=](BitmapEncoder^ encoder)
{
const Platform::Array<unsigned char>^ pixels = ref new Platform::Array<unsigned char>(tempMat.data, tempMat.total() * tempMat.channels());
encoder->SetPixelData(BitmapPixelFormat::Bgra8, BitmapAlphaMode::Ignore, tempMat.cols , tempMat.rows, 96.0, 96.0, pixels);
encoder->FlushAsync();
});
}
I need to send an image into a byte array using c++/cli. The image is initially in Iplimage format.
int img_sz1 = img1->width * img1->height * img1->nChannels;
array <Byte>^ hh1 = gcnew array<Byte> (img_sz1);
Marshal::Copy( (IntPtr)img->imageData, hh1, 0, img_sz1 );
and it was working fine.
I added the encoding step to send it as jpeg
CvMat* buf1 = cvEncodeImage(".jpeg", img1, jpeg_params);
img_sz1=buf1->width*buf1->height
Marshal::Copy( (IntPtr)buf1, hh1, 0, img_sz1 );
and now it compiles fine but gives me the error at the marshal:copy line
An unhandled exception of type 'System.AccessViolationException' occurred in
mscorlib.dll. Additional information: Attempted to read or write protected memory.
Any help is very appreciated.
The return of cvEncodeImage is a single-row matrix, containing the encoded image data. What you're copying now is the struct itself, e.g., the width field, the height field, etc. I believe you need to copy from buf1->data instead.