List device-names available for video capture from ksvideosrc in gstreamer 1.0 - c++

I am trying to query a list of available video capture devices (webcams) on windows using gstreamer 1.0 in c++.
I am using ksvideosrc as source and i am able to capture the video input but i can't query a list of available devices (and their caps).
On gstreamer 0.10 it has been possible through GstPropertyProbe which is removed in gstreamer 1.0. The documentation suggests using GstDeviceMonitor. But i have no luck using that either.
Has anyone succeeded in acquiring a list of device names? Or can you suggests another way of retrieving the available device names and their caps?

You can use GstDeviceMonitor and gst_device_monitor_get_devices () function.
First initialize GstDeviceMonitor by gst_device_monitor_new().
Second start the monitor by gst_device_monitor_start(pMonitor).
Third, get devices list by gst_device_monitor_get_devices(pMonitor).
Code would be like this:
GstDeviceMonitor* monitor= gst_device_monitor_new();
if(!gst_device_monitor_start(monitor)){
printf("WARNING: Monitor couldn't started!!\n");
}
GList* devices = gst_device_monitor_get_devices(monitor);
My references:
https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/gstreamer-GstDeviceMonitor.html#gst-device-monitor-get-devices

Although I haven't figured out how to enumerate the device names, I've come up with a workaround to at least get the available ksvideosrc device indexes. Below is the code in Python, but you should be able to port it to C++ fairly easily, thanks to the GObject introspection bindings.
from gi.repository import Gst
def get_ksvideosrc_device_indexes():
device_index = 0
video_src = Gst.ElementFactory.make('ksvideosrc')
state_change_code = None
while True:
video_src.set_state(Gst.State.NULL)
video_src.set_property('device-index', device_index)
state_change_code = video_src.set_state(Gst.State.READY)
if state_change_code != Gst.StateChangeReturn.SUCCESS:
video_src.set_state(Gst.State.NULL)
break
device_index += 1
return range(device_index)
if __name__ == '__main__':
Gst.init()
print get_ksvideosrc_device_indexes()
Note that the video source device-name property is None as of GStreamer version 1.4.5.0 on Windows for the ksvideosrc.

It's very late, but for the future...
The Gst.DeviceMonitor can be used to enumerate devices, and register an addition or removal of a device.
Here's how to get device names in C# with GStreamer 1.14
static class Devices
{
public static void Run(string[] args)
{
Application.Init(ref args);
GtkSharp.GstreamerSharp.ObjectManager.Initialize();
var devmon = new DeviceMonitor();
// to show only cameras
// var caps = new Caps("video/x-raw");
// var filtId = devmon.AddFilter("Video/Source", caps);
var bus = devmon.Bus;
bus.AddWatch(OnBusMessage);
if (!devmon.Start())
{
"Device monitor cannot start".PrintErr();
return;
}
Console.WriteLine("Video devices count = " + devmon.Devices.Length);
foreach (var dev in devmon.Devices)
DumpDevice(dev);
var loop = new GLib.MainLoop();
loop.Run();
}
static void DumpDevice(Device d)
{
Console.WriteLine($"{d.DeviceClass} : {d.DisplayName} : {d.Name} ");
}
static bool OnBusMessage(Bus bus, Message message)
{
switch (message.Type)
{
case MessageType.DeviceAdded:
{
var dev = message.ParseDeviceAdded();
Console.WriteLine("Device added: ");
DumpDevice(dev);
break;
}
case MessageType.DeviceRemoved:
{
var dev = message.ParseDeviceRemoved();
Console.WriteLine("Device removed: ");
DumpDevice(dev);
break;
}
}
return true;
}
}

Related

Saving a video using LIBVLC

HiI need to stream a video file and save it using LIBVLC. Here is what I have done so far:
libvlc_media_t* vlcMedia = nullptr;
libvlc_instance_t* vlcInstance = libvlc_new(0, nullptr);
vlcMedia = libvlc_media_new_location(vlcInstance, aUri);
if(nullptr != vlcMedia)
{
libvlc_media_player_t* vlcMediaPlayer = libvlc_media_player_new_from_media(vlcMedia);
if(nullptr != vlcMediaPlayer)
{
libvlc_media_release(vlcMedia);
libvlc_event_manager_t* vlcMediaManager = libvlc_media_player_event_manager(vlcMediaPlayer);
if(nullptr != vlcMediaManager)
libvlc_event_attach(vlcMediaManager, libvlc_MediaPlayerEndReached, OnStopped, this);
libvlc_media_player_set_hwnd(vlcMediaPlayer, Handle);
libvlc_media_player_play(vlcMediaPlayer);
}
}
This will connect to the remote media and starts playing the video. The question is how do I direct it to save the video? I could not find the API call for that.
Thank youSam
Thanks to #mtz the solution is to add:
libvlc_media_add_option(vlcMedia,":sout=#duplicate{dst=display,dst=std{access=file,mux=mp4,dst=xyz.mp4}");
after the call to libvlc_media_new_location.
Here's a C# version that you can easily adapt to C/C++
var currentDirectory = Path.GetDirectoryName(Assembly.GetEntryAssembly().Location);
var destination = Path.Combine(currentDirectory, "record.ts");
// Load native libvlc library
Core.Initialize();
using (var libvlc = new LibVLC())
using (var mediaPlayer = new MediaPlayer(libvlc))
{
// Redirect log output to the console
libvlc.Log += (sender, e) => Console.WriteLine($"[{e.Level}] {e.Module}:{e.Message}");
// Create new media with HLS link
var media = new Media(libvlc, "http://hls1.addictradio.net/addictrock_aac_hls/playlist.m3u8", FromType.FromLocation);
// Define stream output options.
// In this case stream to a file with the given path and play locally the stream while streaming it.
media.AddOption(":sout=#file{dst=" + destination + "}");
media.AddOption(":sout-keep");
// Start recording
mediaPlayer.Play(media);
Console.WriteLine($"Recording in {destination}");
Console.WriteLine("Press any key to exit");
Console.ReadKey();
}

Aspose.Barcode cannot read DecodeType.Code128 barcode

The aspose.barcode reader is unable to read the barcode of type DecodeType.Code128
Workflow Steps
1>Using Aspose.Barcode we have created a barcode using DecodeType.Code128 and put on PDF page ( our clients use this page as separator sheet)
2>Our client then insert this barcode page between several physical documents and scanned them all, which creates big single PDF
3>Our splitting process then, loop through all pages and check if any page is barcode page, and splits the big PDF into individual small PDF
Issue is some times the scanned quality of the barcode is not that great, and in such case ASPOSE.Barcode unable to read the barcode.
I have attached couple of barcode PDF with low scanned quality, and aspose is not able to read these barcodes. I have tried different combinations of RecognitionMode and ManualHints options without any luck
Below is my code to identity barcode page
using (var fs = new FileStream(file, FileMode.Open))
{
var pdfDocument = new Document(fs);
foreach (Page page in pdfDocument.Pages)
{
var isSeparator = splitter.IsSeparator(page);
Assert.IsTrue(isSeparator);
}
}
public bool IsSeparator(Page page)
{
if (page.Resources.Images != null && page.Resources.Images.Count >= 1)
{
var img = page.Resources.Images[1];
using (MemoryStream barcodeImage = new MemoryStream())
{
img.Save(barcodeImage, ImageFormat.Jpeg);
barcodeImage.Seek(0L, SeekOrigin.Begin);
using (BarCodeReader barcodeReader = new BarCodeReader(barcodeImage, _barcodeDecodeType))
{
barcodeReader.RecognitionMode = RecognitionMode.MaxQuality;
while (barcodeReader.Read())
{
var barcodeText = barcodeReader.GetCodeText();
if (barcodeText.ToLower() == "eof")
{
return true;
}
}
}
}
}
return false;
}
Unable to reproduce the issue at my end. I used the following sample code snippet to recognize the barcode along with latest version of the API. It is always recommended to use the latest version of the API as it contains new features and improvements.
CODE:
Aspose.Pdf.License licensePdf = new Aspose.Pdf.License();
licensePdf.SetLicense(#"Aspose.Total.lic");
// bind the pdf document
Aspose.Pdf.Facades.PdfExtractor pdfExtractor = new Aspose.Pdf.Facades.PdfExtractor();
pdfExtractor.BindPdf(#"173483_2.pdf");
// extract the images
pdfExtractor.ExtractImage();
// save images to stream in a loop
while (pdfExtractor.HasNextImage())
{
// save image to stream
System.IO.MemoryStream imageStream = new System.IO.MemoryStream();
pdfExtractor.GetNextImage(imageStream);
imageStream.Position = 0;
Aspose.BarCode.BarCodeRecognition.BarCodeReader barcodeReader =
new Aspose.BarCode.BarCodeRecognition.BarCodeReader(imageStream);
while (barcodeReader.Read())
{
Console.WriteLine("Codetext found: " + barcodeReader.GetCodeText() + ", Symbology: " + barcodeReader.GetCodeType().ToString());
}
// close the reader
barcodeReader.Close();
}
Further to update you that the same query has been post on Aspose.BarCode support forum. You may please visit the link for details.
I work as developer evangelist at Aspose.

Multiple USB to RS485 FTDI Device ID

I need some help. I'm programming a Win 10 App in C++/CX. I am using two USB to RS485 devices, both of which have the same VID number. In days of old, I could write a bit software and connect to ports using good old COMx etc.
I'm now following the example here Serial Sample which uses the approach gathering the device info so when looking for connected devices, what I see in the list of available devices is the following.
\?\FTDIBUS#VID_0403+PID_6001
Both devices have the same VID and PID. This leads to the problem of me being cable to connect to the correct USB device. I think my app is trying to connect to both devices at the same time? Does anyone have any ideas about how I can resolve this hitch?
void MainPage::Get_Serial_Devices() {
cancellationTokenSource_Port1 = new Concurrency::cancellation_token_source();
cancellationTokenSource_Port2 = new Concurrency::cancellation_token_source();
// THIS USES ASYNCRONOUS OPERATION. GET A LIST OF SERIAL DEVICES AND POPULATE THE COMBO BOX
Concurrency::create_task(ListAvailablePortsAsync()).then([this](DeviceInformationCollection^ serialDeviceCollectioin)
{
// serialDeviceCollection CONTAINS ALL SERIAL DEVICES FOUND, COPY INTO _deviceCollection
DeviceInformationCollection^ _deviceCollection = serialDeviceCollectioin;
// CLEAR EXISTING DEVICES FOR OUR OBJECT COLLECTION
_availableDevices->Clear();
// FOR EVERY DEVICE IN _deviceCollection
for (auto &&device : _deviceCollection) {
if (device->Name->Equals("USB-RS485 Cable")) {
// CREATE A NEW DEVICE TYPE AND APPEND TO OUR OBJECT COLLECTION
_availableDevices->Append(ref new Device(device->Id, device));
Total_Ports++;
this->DeviceLists->Items->Append(device->Id);
}
}
});
void MainPage::ConnectButton_Click(Object^ sender, RoutedEventArgs^ e) {
if (Port1_Connected == false) {
// CAST INDEX TO CORRELATING Device IN _availableDevices
Device^ selectedDevice = static_cast<Device^>(_availableDevices->GetAt(Port_1_ID));
// GET THE DEVICE INFO
DeviceInformation^ entry = selectedDevice->DeviceInfo;
Concurrency::create_task(ConnectToSerialDeviceAsync_Port1(entry, cancellationTokenSource_Port1->get_token())).then([this]( ) {
Get_Echo();
Waiting_For_Ack = true;
});
}
Concurrency::task<void> MainPage::ConnectToSerialDeviceAsync_Port1(DeviceInformation^ device, Concurrency::cancellation_token cancellationToken) {
// CREATE A LINKED TOKEN WHICH IS CANCELLED WHEN THE PROVIDED TOKEN IS CANCELLED
auto childTokenSource = Concurrency::cancellation_token_source::create_linked_source(cancellationToken);
// GET THE TOKEN
auto childToken = childTokenSource.get_token();
// CONNECT TO ARDUINO TASK
return Concurrency::create_task(SerialDevice::FromIdAsync(device->Id), childToken).then([this](SerialDevice^ serial_device) {
try {
_serialPort_Port1 = serial_device;
TimeSpan _timeOut; _timeOut.Duration = 10;
// CONFIGURE SERIAL PORT SETTINGS
_serialPort_Port1->WriteTimeout = _timeOut;
_serialPort_Port1->ReadTimeout = _timeOut;
_serialPort_Port1->BaudRate = 57600;
_serialPort_Port1->Parity = Windows::Devices::SerialCommunication::SerialParity::None;
_serialPort_Port1->StopBits = Windows::Devices::SerialCommunication::SerialStopBitCount::One;
_serialPort_Port1->DataBits = 8;
_serialPort_Port1->Handshake = Windows::Devices::SerialCommunication::SerialHandshake::None;
// CREATE OUR DATA READER OBJECT
_dataReaderObject_Port1 = ref new DataReader(_serialPort_Port1->InputStream);
_dataReaderObject_Port1->InputStreamOptions = InputStreamOptions::None;
// CREATE OUR DATA WRITE OBJECT
_dataWriterObject_Port1 = ref new DataWriter(_serialPort_Port1->OutputStream);
this->ConnectButton->IsEnabled = false;
this->DisconnectButton->IsEnabled = true;
// KICK OF THE SERIAL PORT LISTENING PROCESS
Listen_Port1();
}
catch (Platform::Exception^ ex) {
this->Error_Window->Text = (ex->Message);
CloseDevice(PORT_1);
}
});
FT_PROG is a free EEPROM programming utility for use with FTDI devices. It is used for modifying EEPROM contents that store the FTDI device descriptors to customize designs.
The full FT_PROG User Guide can be downloaded here.

Using multiple audio input devices with AVAudioEngine

I've got several usb audio input devices connected and I want to select more than 1 of them to connect to the mixer. The way I understand it, it will not be able to use AVAudioInputNode for this purpose.
I could accomplish this by using AVFoundation to read samples from each device and then schedule them to a player but it seems to me that this might be easier done with AVAudioUnit. Unfortunately, I have not been able to make that work.
AudioComponentDescription desc;
desc.componentType = kAudioUnitType_Output;
desc.componentSubType = kAudioUnitSubType_HALOutput;
desc.componentFlags = 0;
desc.componentFlagsMask = 0;
desc.componentManufacturer = kAudioUnitManufacturer_Apple;
[AVAudioUnit
instantiateWithComponentDescription:desc
options:kAudioComponentInstantiation_LoadInProcess
completionHandler:^(AVAudioUnit *xx, NSError *error) {
AudioUnitSetProperty(xx.audioUnit, kAudioOutputUnitProperty_CurrentDevice,
kAudioUnitScope_Global, 0, &inputDeviceID, sizeof(inputDeviceID));
[_engine attachNode:xx];
AVAudioMixerNode *mixer = [_engine mainMixerNode];
[_engine connect:xx to:mixer format:[xx outputFormatForBus:0]];
}];
[central] 54: ERROR: >avae> AVAudioEngineGraph.mm:1235: AddNode:
required condition is false: inImpl != nil && !IsIONode(inAVNode)
Any suggestions would be appreciated.

How do you capture current frame from a MediaElement in WinRT (8.1)?

I am trying to implement a screenshot functionality in a WinRT app that shows Video via a MediaElement. I have the following code, it saves a screenshot that's the size of the MediaElement but the image is empty (completely black). Tried with various types of Media files. If I do a Win Key + Vol Down on Surface RT, the screen shot includes the Media frame content, but if I use the following code, it's blackness all around :(
private async Task SaveCurrentFrame()
{
RenderTargetBitmap renderTargetBitmap = new RenderTargetBitmap();
await renderTargetBitmap.RenderAsync(Player);
var pixelBuffer = await renderTargetBitmap.GetPixelsAsync();
MultimediaItem currentItem = (MultimediaItem)this.DefaultViewModel["Group"];
StorageFolder currentFolder = Windows.Storage.ApplicationData.Current.LocalFolder;
var saveFile = await currentFolder.CreateFileAsync(currentItem.UniqueId + ".png", CreationCollisionOption.ReplaceExisting);
if (saveFile == null)
return;
// Encode the image to the selected file on disk
using (var fileStream = await saveFile.OpenAsync(FileAccessMode.ReadWrite))
{
var encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.PngEncoderId, fileStream);
encoder.SetPixelData(
BitmapPixelFormat.Bgra8,
BitmapAlphaMode.Ignore,
(uint)renderTargetBitmap.PixelWidth,
(uint)renderTargetBitmap.PixelHeight,
DisplayInformation.GetForCurrentView().LogicalDpi,
DisplayInformation.GetForCurrentView().LogicalDpi,
pixelBuffer.ToArray());
await encoder.FlushAsync();
}
}
Here MultimediaItem is my View Model class that among other things has a UniqueId property that's a string.
'Player' is the name of the Media Element.
Is there anything wrong with the code or this approach is wrong and I've to get in the trenches with C++?
P.S. I am interested in the WinRT API only.
Update 1 Looks like RenderTargetBitmap doesn't support this, the MSDN documentation clarifies it http://msdn.microsoft.com/en-us/library/windows/apps/windows.ui.xaml.media.imaging.rendertargetbitmap .
I'll appreciate any pointers on how to do it using DirectX C++. This is a major task for me so I'll crack this one way or the other and report back with the solution.
Yes, it is possible - little bit tricky, but working well.
You dont use mediaElement, but StorageFile itself.
You need to create writableBitmap with help of Windows.Media.Editing namespace.
Works in UWP (Windows 10)
This is complete example with file picking and getting video resolution and saving image to Picture Library
TimeSpan timeOfFrame = new TimeSpan(0, 0, 1);//one sec
//pick mp4 file
var picker = new Windows.Storage.Pickers.FileOpenPicker();
picker.SuggestedStartLocation = Windows.Storage.Pickers.PickerLocationId.VideosLibrary;
picker.FileTypeFilter.Add(".mp4");
StorageFile pickedFile = await picker.PickSingleFileAsync();
if (pickedFile == null)
{
return;
}
///
//Get video resolution
List<string> encodingPropertiesToRetrieve = new List<string>();
encodingPropertiesToRetrieve.Add("System.Video.FrameHeight");
encodingPropertiesToRetrieve.Add("System.Video.FrameWidth");
IDictionary<string, object> encodingProperties = await pickedFile.Properties.RetrievePropertiesAsync(encodingPropertiesToRetrieve);
uint frameHeight = (uint)encodingProperties["System.Video.FrameHeight"];
uint frameWidth = (uint)encodingProperties["System.Video.FrameWidth"];
///
//Use Windows.Media.Editing to get ImageStream
var clip = await MediaClip.CreateFromFileAsync(pickedFile);
var composition = new MediaComposition();
composition.Clips.Add(clip);
var imageStream = await composition.GetThumbnailAsync(timeOfFrame, (int)frameWidth, (int)frameHeight, VideoFramePrecision.NearestFrame);
///
//generate bitmap
var writableBitmap = new WriteableBitmap((int)frameWidth, (int)frameHeight);
writableBitmap.SetSource(imageStream);
//generate some random name for file in PicturesLibrary
var saveAsTarget = await KnownFolders.PicturesLibrary.CreateFileAsync("IMG" + Guid.NewGuid().ToString().Substring(0, 4) + ".jpg");
//get stream from bitmap
Stream stream = writableBitmap.PixelBuffer.AsStream();
byte[] pixels = new byte[(uint)stream.Length];
await stream.ReadAsync(pixels, 0, pixels.Length);
using (var writeStream = await saveAsTarget.OpenAsync(FileAccessMode.ReadWrite))
{
var encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.JpegEncoderId, writeStream);
encoder.SetPixelData(
BitmapPixelFormat.Bgra8,
BitmapAlphaMode.Premultiplied,
(uint)writableBitmap.PixelWidth,
(uint)writableBitmap.PixelHeight,
96,
96,
pixels);
await encoder.FlushAsync();
using (var outputStream = writeStream.GetOutputStreamAt(0))
{
await outputStream.FlushAsync();
}
}
Yeah...I spent lot of hours by this
Ok I have managed to get making snapshot from MediaElement on button press to work.
I am passing MediaStreamSource object to MediaElement using SetMediaStreamSource method. MediaStreamSource has event SampleRequested which is fired basicly everytime new frame is drawn. Then using boolean I control when to create bitmap
private async void MediaStream_SampleRequested(MediaStreamSource sender, MediaStreamSourceSampleRequestedEventArgs args)
{
if (!takeSnapshot)
{
return;
}
takeSnapshot = false;
Task.Run(() => DecodeAndSaveVideoFrame(args.Request.Sample));
}
After that what is left is to decode compressed image and convert it to WriteableBitmap. The image is (or at least was in my case) in YUV fromat. You can get the byte array using
byte[] yvuArray = sample.Buffer.ToArray();
and then get data from this array and convert it to RGB. Unfortunetly I cannot post entire code but I'm gonna give you a few more hints:
YUV to RGB wiki here you have wiki describing how does YUV to RGB conversion works.
Here I found python project which solution I have adapted (and works perfectly). To be more precise you have to analize how method NV12Converter works.
The last thing is to change takeSnapshot boolean to true after pressing button or doing other activity :).