I am new to WinRT c++. I am trying to pass an StorageFile image from C# and open the file and set it as source in BitmapImage in WinRT to extract height and width of image. I am using the following code.
auto openOperation = StorageImageFile->OpenAsync(FileAccessMode::Read); // from http://msdn.microsoft.com/en-us/library/windows/desktop/hh780393%28v=vs.85%29.aspx
openOperation->Completed = ref new
AsyncOperationCompletedHandler<IRandomAccessStream^>(
[=](IAsyncOperation<IRandomAccessStream^> ^operation, AsyncStatus status)
{
auto Imagestream = operation->GetResults();
BitmapImage^ bmp = ref new BitmapImage();
auto bmpOp = bmp->SetSourceAsync(Imagestream);
bmpOp->Completed = ref new
AsyncActionCompletedHandler (
[=](IAsyncAction^ action, AsyncStatus status)
{
action->GetResults();
UINT32 imageWidth = (UINT32)bmp->PixelWidth;
UINT32 imageHeight = (UINT32)bmp->PixelHeight;
});
});
This code does not seem to work. after the line BitmapImage^ bmp = ref new BitmapImage(); the debugger stops saying no source code is found.
Can you help me write the correct code?
I think you meant to write openOperation->Completed += ref new... and bmpOp->Completed += ref new.... I'm not an expert in C++, but from what I have seen - async operations are typically wrapped in create_task calls. Not really sure why - maybe to avoid subscribing to events without unsubscribing?
I think it should look roughly like this:
auto bmp = ref new BitmapImage();
create_task(storageImageFile->OpenAsync(FileAccessMode::Read)) // get the stream
.then([bmp](IRandomAccessStream^ ^stream) // continuation lambda
{
return create_task(bmp->SetSourceAsync(stream)); // needs to run on ASTA/Dispatcher thread
}, task_continuation_context::use_current()) // run on ASTA/Dispatcher thread
.then([bmp]() // continuation lambda
{
UINT32 imageWidth = (UINT32)bmp->PixelWidth; // needs to run on ASTA/Dispatcher thread
UINT32 imageHeight = (UINT32)bmp->PixelHeight; // needs to run on ASTA/Dispatcher thread
// TODO: use imageWidth and imageHeight
}, task_continuation_context::use_current()); // run on ASTA/Dispatcher thread
Related
I'm new to C++ and Windows and trying to write a WinRT/C++ app that can grab a file via path and run it through the built-in Media::Ocr model, but I can't see to even properly get the file into my application as a StorageFile. Here's how I've been trying to do it (to no avail):
runModel()
---------
IAsyncAction runModel() {
string _pth = "pic.png";
hstring pth = to_hstring(_pth);
StorageFile file = nullptr;
co_await LoadFileAsync(pth);
}
IAsyncAction LoadFileAsync (hstring pth) {
StorageFile file = co_await Windows::Storage::StorageFile::GetFileFromPathAsync(pth);
co_await LoadImageAsync(file);
}
IAsyncAction LoadImageAsync(StorageFile const& file) {
auto stream = co_await file.OpenAsync(Windows::Storage::FileAccessMode::Read);
auto decoder = co_await BitmapDecoder::CreateAsync(stream);
auto bitmap = co_await decoder.GetSoftwareBitmapAsync();
auto imgSource = WriteableBitmap(bitmap.PixelWidth(), bitmap.PixelHeight());
bitmap.CopyToBuffer(imgSource.PixelBuffer());
OcrEngine ocrEngine = nullptr;
ocrEngine = OcrEngine::TryCreateFromUserProfileLanguages();
auto ocrResult = co_await ocrEngine.RecognizeAsync(bitmap);
hstring hresult = ocrResult.Text();
std::cout << hresult.begin();
}
I'd appreciate any help of the following:
Loading the file in as a StorageFile
The right way to handle these async calls. I tried to follow https://learn.microsoft.com/en-us/windows/uwp/cpp-and-winrt-apis/concurrency but I couldn't seem to get it to work without this error: !isStaThread()
How to properly do the OCR
Thanks!
I am working on an off-screen render program and I use crate glium to do this. I have followed the example of screenshot.rs and this example worked well.
Then I made some change:
The orignal code was
fn main() {
// building the display, ie. the main object
let event_loop = glutin::EventsLoop::new();
let wb = glutin::WindowBuilder::new().with_visible(true);
let cb = glutin::ContextBuilder::new();
let display = glium::Display::new(wb, cb, &event_loop).unwrap();
// building the vertex buffer, which contains all the vertices that we will draw
I grouped these codes into a function:
fn main() {
// building the display, ie. the main object
let event_loop = glutin::EventsLoop::new();
let display = build_display((128,128), &event_loop);
// building the vertex buffer, which contains all the vertices that we will draw
pub fn build_display(size: (u32, u32), event_loop: &glutin::EventsLoop) -> glium::Display {
let version = parse_version(); //this will return `OpenGL 3.3`
let wb = glutin::WindowBuilder::new()
.with_visibility(false)
.with_dimensions(glutin::dpi::LogicalSize::from(size));
let cb = glutin::ContextBuilder::new()
.with_gl(version);
glium::Display::new(wb, cb, &event_loop).unwrap()
}
After this modification, the program still worked well. So I continued to add the headless-context:
fn main() {
// building the display, ie. the main object
let event_loop = glutin::EventsLoop::new();
let display = build_display_headless((128,128), &event_loop);
// building the vertex buffer, which contains all the vertices that we will draw
pub fn build_display_headless(size: (u32, u32), event_loop: &glutin::EventsLoop) -> glium::HeadlessRenderer {
let version = parse_version(); // this will return `OpenGL 3.3`
let ctx = glutin::ContextBuilder::new()
.with_gl(version)
.build_headless(&event_loop, glutin::dpi::PhysicalSize::from(size))
.expect("1");
//let ctx = unsafe { ctx.make_current().expect("3") };
glium::HeadlessRenderer::new(ctx).expect("4")
}
But this time, the program did not work. There was no panic during the running, but the output image was empty with black, and its size was not 128x128 but 800x600.
I have tried to remove the libEGL.dll so that, due to the doc of crate glutin, the function .build_headless will build a window and hide it, just as my function build_display does. However, this failed too. So what can cause this?
I have the following code which opens a file and it works most of the time for one time. After that I get exceptions thrown and I don't know where the problem is hiding. I tried to look for this for a couple of days already but no luck.
String^ xmlFile = "Assets\\TheXmlFile.xml";
xml = ref new XmlDocument();
StorageFolder^ InstallationFolder = Windows::ApplicationModel::Package::Current->InstalledLocation;
task<StorageFile^>(
InstallationFolder->GetFileAsync(xmlFile)).then([this](StorageFile^ file) {
if (nullptr != file) {
task<Streams::IRandomAccessStream^>(file->OpenAsync(FileAccessMode::Read)).then([this](Streams::IRandomAccessStream^ stream)
{
IInputStream^ deInputStream = stream->GetInputStreamAt(0);
DataReader^ reader = ref new DataReader(deInputStream);
reader->InputStreamOptions = InputStreamOptions::Partial;
reader->LoadAsync(stream->Size);
strXml = reader->ReadString(stream->Size);
MessageDialog^ dlg = ref new MessageDialog(strXml);
dlg->ShowAsync();
});
}
});
The error is triggered at this part of the code:
strXml = reader->ReadString(stream->Size);
I get the following error:
First-chance exception at 0x751F5B68 in XmlProject.exe: Microsoft C++ exception: Platform::OutOfBoundsException ^ at memory location 0x02FCD634. HRESULT:0x8000000B The operation attempted to access data outside the valid range
WinRT information: The operation attempted to access data outside the valid range
Just like I said, the first time it just works but after that I get the error. I tried detaching the stream and buffer of the datareader and tried to flush the stream but no results.
I've also asked this question on the Microsoft C++ forums and credits to the user "Viorel_" I managed to get it working. Viorel said the following:
Since LoadAsync does not perform the operation immediately, you should probably add a corresponding “.then”. See some code: https://social.msdn.microsoft.com/Forums/windowsapps/en-US/94fa9636-5cc7-4089-8dcf-7aa8465b8047. This sample uses “create_task” and “then”: https://code.msdn.microsoft.com/vstudio/StreamSocket-Sample-8c573931/sourcecode (file Scenario1.xaml.cpp, for example).
I have had to separate the content in the task<Streams::IRandomAccessStream^> and split it up in separate tasks.
I reconstructed my code and I have now the following:
String^ xmlFile = "Assets\\TheXmlFile.xml";
xml = ref new XmlDocument();
StorageFolder^ InstallationFolder = Windows::ApplicationModel::Package::Current->InstalledLocation;
task<StorageFile^>(
InstallationFolder->GetFileAsync(xmlFile)).then([this](StorageFile^ file) {
if (nullptr != file) {
task<Streams::IRandomAccessStream^>(file->OpenAsync(FileAccessMode::Read)).then([this](Streams::IRandomAccessStream^ stream)
{
IInputStream^ deInputStream = stream->GetInputStreamAt(0);
DataReader^ reader = ref new DataReader(deInputStream);
reader->InputStreamOptions = InputStreamOptions::Partial;
create_task(reader->LoadAsync(stream->Size)).then([reader, stream](unsigned int size){
strXml = reader->ReadString(stream->Size);
MessageDialog^ dlg = ref new MessageDialog(strXml);
dlg->ShowAsync();
});
});
}
});
I am trying to implement a screenshot functionality in a WinRT app that shows Video via a MediaElement. I have the following code, it saves a screenshot that's the size of the MediaElement but the image is empty (completely black). Tried with various types of Media files. If I do a Win Key + Vol Down on Surface RT, the screen shot includes the Media frame content, but if I use the following code, it's blackness all around :(
private async Task SaveCurrentFrame()
{
RenderTargetBitmap renderTargetBitmap = new RenderTargetBitmap();
await renderTargetBitmap.RenderAsync(Player);
var pixelBuffer = await renderTargetBitmap.GetPixelsAsync();
MultimediaItem currentItem = (MultimediaItem)this.DefaultViewModel["Group"];
StorageFolder currentFolder = Windows.Storage.ApplicationData.Current.LocalFolder;
var saveFile = await currentFolder.CreateFileAsync(currentItem.UniqueId + ".png", CreationCollisionOption.ReplaceExisting);
if (saveFile == null)
return;
// Encode the image to the selected file on disk
using (var fileStream = await saveFile.OpenAsync(FileAccessMode.ReadWrite))
{
var encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.PngEncoderId, fileStream);
encoder.SetPixelData(
BitmapPixelFormat.Bgra8,
BitmapAlphaMode.Ignore,
(uint)renderTargetBitmap.PixelWidth,
(uint)renderTargetBitmap.PixelHeight,
DisplayInformation.GetForCurrentView().LogicalDpi,
DisplayInformation.GetForCurrentView().LogicalDpi,
pixelBuffer.ToArray());
await encoder.FlushAsync();
}
}
Here MultimediaItem is my View Model class that among other things has a UniqueId property that's a string.
'Player' is the name of the Media Element.
Is there anything wrong with the code or this approach is wrong and I've to get in the trenches with C++?
P.S. I am interested in the WinRT API only.
Update 1 Looks like RenderTargetBitmap doesn't support this, the MSDN documentation clarifies it http://msdn.microsoft.com/en-us/library/windows/apps/windows.ui.xaml.media.imaging.rendertargetbitmap .
I'll appreciate any pointers on how to do it using DirectX C++. This is a major task for me so I'll crack this one way or the other and report back with the solution.
Yes, it is possible - little bit tricky, but working well.
You dont use mediaElement, but StorageFile itself.
You need to create writableBitmap with help of Windows.Media.Editing namespace.
Works in UWP (Windows 10)
This is complete example with file picking and getting video resolution and saving image to Picture Library
TimeSpan timeOfFrame = new TimeSpan(0, 0, 1);//one sec
//pick mp4 file
var picker = new Windows.Storage.Pickers.FileOpenPicker();
picker.SuggestedStartLocation = Windows.Storage.Pickers.PickerLocationId.VideosLibrary;
picker.FileTypeFilter.Add(".mp4");
StorageFile pickedFile = await picker.PickSingleFileAsync();
if (pickedFile == null)
{
return;
}
///
//Get video resolution
List<string> encodingPropertiesToRetrieve = new List<string>();
encodingPropertiesToRetrieve.Add("System.Video.FrameHeight");
encodingPropertiesToRetrieve.Add("System.Video.FrameWidth");
IDictionary<string, object> encodingProperties = await pickedFile.Properties.RetrievePropertiesAsync(encodingPropertiesToRetrieve);
uint frameHeight = (uint)encodingProperties["System.Video.FrameHeight"];
uint frameWidth = (uint)encodingProperties["System.Video.FrameWidth"];
///
//Use Windows.Media.Editing to get ImageStream
var clip = await MediaClip.CreateFromFileAsync(pickedFile);
var composition = new MediaComposition();
composition.Clips.Add(clip);
var imageStream = await composition.GetThumbnailAsync(timeOfFrame, (int)frameWidth, (int)frameHeight, VideoFramePrecision.NearestFrame);
///
//generate bitmap
var writableBitmap = new WriteableBitmap((int)frameWidth, (int)frameHeight);
writableBitmap.SetSource(imageStream);
//generate some random name for file in PicturesLibrary
var saveAsTarget = await KnownFolders.PicturesLibrary.CreateFileAsync("IMG" + Guid.NewGuid().ToString().Substring(0, 4) + ".jpg");
//get stream from bitmap
Stream stream = writableBitmap.PixelBuffer.AsStream();
byte[] pixels = new byte[(uint)stream.Length];
await stream.ReadAsync(pixels, 0, pixels.Length);
using (var writeStream = await saveAsTarget.OpenAsync(FileAccessMode.ReadWrite))
{
var encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.JpegEncoderId, writeStream);
encoder.SetPixelData(
BitmapPixelFormat.Bgra8,
BitmapAlphaMode.Premultiplied,
(uint)writableBitmap.PixelWidth,
(uint)writableBitmap.PixelHeight,
96,
96,
pixels);
await encoder.FlushAsync();
using (var outputStream = writeStream.GetOutputStreamAt(0))
{
await outputStream.FlushAsync();
}
}
Yeah...I spent lot of hours by this
Ok I have managed to get making snapshot from MediaElement on button press to work.
I am passing MediaStreamSource object to MediaElement using SetMediaStreamSource method. MediaStreamSource has event SampleRequested which is fired basicly everytime new frame is drawn. Then using boolean I control when to create bitmap
private async void MediaStream_SampleRequested(MediaStreamSource sender, MediaStreamSourceSampleRequestedEventArgs args)
{
if (!takeSnapshot)
{
return;
}
takeSnapshot = false;
Task.Run(() => DecodeAndSaveVideoFrame(args.Request.Sample));
}
After that what is left is to decode compressed image and convert it to WriteableBitmap. The image is (or at least was in my case) in YUV fromat. You can get the byte array using
byte[] yvuArray = sample.Buffer.ToArray();
and then get data from this array and convert it to RGB. Unfortunetly I cannot post entire code but I'm gonna give you a few more hints:
YUV to RGB wiki here you have wiki describing how does YUV to RGB conversion works.
Here I found python project which solution I have adapted (and works perfectly). To be more precise you have to analize how method NV12Converter works.
The last thing is to change takeSnapshot boolean to true after pressing button or doing other activity :).
I am Wrting code to capture image from camera.Below is the code I have written.
Here method CapturePhotoToStorageFileAsync doesnot return.
auto MediaCap = ref new Windows::Media::Capture::MediaCapture();
auto ImageProp = ref new Windows::Media::Capture::ImageEncodingProperties ();
ImageProp->Height = 240;
ImageProp->Width = 320;
ImageProp->Subtype = "JPEG";
Windows::Storage::StorageFile^ strFile;
auto res = MediaCap->CapturePhotoToStorageFileAsync(ImageProp,strFile);
res->Completed = ref new AsyncActionCompletedHandler([](IAsyncAction ^action)
{
//action->GetResults();
//action->Start();
///action->Close();
});
res->Start();
Am I missing something here??
Did you want to show UI to the user or just silently capture? The only C++ camera sample I've found uses CameraCaptureUI and CaptureFileAsync - then the operation is getting a StorageFile^ back.
If you're deliberately using CapturePhotoToStorageFileAsync, check your capabiities.
The Issue is resolved
I Added code for
InitializeAsync()
Created file to be used to store image
using
Windows::Storage::StorageFileRetrievalOperation^ CreateFileOp = Windows::Storage::KnownFolders::PicturesLibrary->CreateFileAsync("Test.jpg");
I found Java script article and implemented in c++.
http://code.msdn.microsoft.com/windowsdesktop/Media-Capture-Sample-adf87622/sourcecode?fileId=43837&pathId=1754477665