I am newbie in media foundation programming and windows programing as well.
It might looks very silly question but i didn't get clear answer anywhere.
My application is to capture screen, scale, encode and send the data to network. I am looking to improve the performance of my pipeline. so i want to change some intermediate libraries like scaling or encoding libraries.
When i do a lot of search for better option of scaling and encoding, i end up with some MFT(media foundation transform) e.g.Video Processor MFT and H.264 Video Encoder MFT.
My application already implemented pipeline and i don't want to change complete architecture.
can we directly use MFT as a library and add in my project? or i have to build complete pipeline, source and sink.
As per architecture of Media foundation a MFT is intermediate block. It requires IMFTransform::GetInputStreamInfo and IMFTransform::GetOutputStreamInfo.
Is it any way to call direct API's of MFT to perform (scaling and encoding) with creating complete pipeline?
Please provide link if any similar question already asked.
Yes you can create this IMFTransform directly and use it in isolation from pipeline. It is very typical usage model for encoder MFT.
You will need to configure input / output media types, start streaming, feed input frames and grab output frames.
Depending on whether your transform is synchronous or asynchronous (which may differ depending on HW or SW implementation of your MFT) you may need use basic (https://msdn.microsoft.com/en-us/library/windows/desktop/aa965264(v=vs.85).aspx) or async (https://msdn.microsoft.com/en-us/library/windows/desktop/dd317909(v=vs.85).aspx) processing model.
Related
I'm trying to evaluate functionality in gstreamer for applicability in a new application.
The application should be able to dynamically play videos and images depending on a few criteria (user input, ...) not really relevant for this question. The main thing I was not able to figure out was how I can achieve seamless crossfading/blending between successive content.
I was thinking about using the videomixer plugin and programatically transition the sinks alpha values. However, I'm not sure if this would work nor if it is a good idea to do so.
A gstreamer solution would be prefered because of the availability on development and target platform. Furthermore, a custom videosink implementation may be used in the end for rendering the content to proprietary displays.
Edit: Was able to code up a prototype using two file-sources fed into a videomixer, using GstInterpolationControlSource and GstTimedValueControlSource to bind and interpolate the videomixer alpha control inputs. The fades look perfect, however, what I did not quite have on the radar was that I cannot dynamically change the file sources location while the pipeline is running. Furthermore, it feels like misusing functions not intended for the job at hand.
Any feedback on how to tackle this use case would still be very much appreachiated. Thanks!
So I'm trying to get a basic tool to output video/audio(s) to Twitch. I'm new to this side (AV) of programming so I'm not even sure what to look for. I'm trying to use mainly Windows infrastructure and third party where not available.
What are the steps of getting raw bitmap and wave data into a codec and then into a rtsp client and finally showing up on Twitch? I'm not looking for code. I'm looking for concepts so I can search for as I'm not absolutely sure what to search for. I'd rather not go through OBS source code to figure it out and use that as last resort.
So I capture the monitor via Output Duplication and also the Sound on the system as a wave and the microphone as another wave. I'm trying to push this to Twitch. I know that there's Media Foundation on Windows but I don't know how far to streaming it can get as I assume there no netcode integrated in it? And also the libav* collection in FFMPEG.
What are the basic steps of sending bitmap/wave to Twitch via any of thee above libraries or even others as long as they work on Windows. Please don't add code, I just need a not very long conceptual explanation and I'll take it from there. Try to cover also how bitrate and framerate gets regulated (do I have do it or the codec does it)?
Assume absolute noob level in this area (concept-wise not code-wise).
The Kinect OpenNI library uses a custom video file format to store videos that contain rgb+d information. These videos have the extension *.oni. I am unable to find any information or documentation whatsoever on the ONI video format.
I'm looking for a way to convert a conventional rgb video to a *.oni video. The depth channel can be left blank (ie zeroed out). For example purposes, I have a MPEG-4 encoded .mov file with audio and video channels.
There are no restrictions on how this conversion must be made, I just need to convert it somehow! Ie, imagemagick, ffmpeg, mencoder are all ok, as is custom conversion code in C/C++ etc.
So far, all I can find is one C++ conversion utility in the OpenNI sources. From the looks of it, I this converts from one *.oni file to another though. I've also managed to find a C++ script by a phd student that converts images from a academic database into a *.oni file. Unfortunately the code is in spanish, not one of my native languages.
Any help or pointers much appreciated!
EDIT: As my usecase is a little odd, some explanation may be in order. The OpenNI Drivers (in my case I'm using the excellent Kinect for Matlab library) allow you to specify a *.oni file when creating the Kinect context. This allows you to emulate having a real Kinect attached that is receiving video data - useful when you're testing / developing code (you don't need to have the Kinect attached to do this). In my particular case, we will be using a Kinect in the production environment (process control in a factory environment), but during development all I have is a video file :) Hence wanting to convert to a *.oni file. We aren't using the Depth channel at the moment, hence not caring about it.
I don't have a complete answer for you, but take a look at the NiRecordRaw and NiRecordSynthetic examples in OpenNI/Samples. They demonstrate how to create an ONI with arbitrary or modified data. See how MockDepthGenerator is used in NiRecordSynthetic -- in your case you will need MockImageGenerator.
For more details you may want to ask in the openni-dev google group.
Did you look into this command and its associated documentation
NiConvertXToONI --
NiConvertXToONI opens any recording, takes every node within it, and records it to a new ONI recording. It receives both the input file and the output file from the command line.
I am just starting to learn DirectShow with C++. I need to use DirectShow to record the audio and write it to a WAV file on the disk. I heard from other people that Win 7 does not allow for rendering audio using DirectShow.
In addition, I would like to know how should I start with recoding audio using DirectShow with C++? If there is sample source, it would be great.
Thanks in advance.
I think you may have misunderstood these other people. Windows Media Foundation is aimed to be the successor of DirectShow, but DirectShow is still a very valid technology on Windows 7.
The easiest thing to accomplish what you want to do, is to get it right using the GraphEdit tool first ( I assume you want to do this programmatically).
Create a graph that contains your audio device, a WavDestFilter, and a file writer.
Source -> WavDest -> File Writer
Play the graph. Stop the graph and you should have created a .wav file with the recorded audio. If you can get this right, then you need to do the whole thing programmatically.
There are a couple of samples in the SDK that show you how to programmatically add filters to a graph and connect them, that should enable you to get started.
WRT the WavDestFilter, IIRC it might not be in all versions of the SDK, you'll have to find an appropriate one. You also need to build it, and regsvr32 it, so that it will show up in your list of available filters in GraphEdit.
If this all seems a bit much, I would read through the DirectShow documentation on MSDN to at least get an overview of DirectShow.
I'm looking for build in library for converting videos/images. i heard something about DirectShow. Do you know any library you have used to convert videos/images?
For transcoding (converting one video format to another) using Directshow is bit tricky, you want to use Media Foundation for this job.
There is Transcode API available in Media Foundation to achieve this task. This link has more details on Transcode API, tutorials and samples to get you started.
You can use DirectShow for grabbing images from video stream. For it you must create your own filter node. It is complex task because of filter is COM object that will work within chain (DirectShow filter graph) of other filter nodes - codecs. So after creating you need register your filter in system. As for me i think you can try it because you can use all registered codecs in system and as result get decompressed/final image into your filter. As other solution i think that you can try to use modules from some open source media player. For example try VideoLAN but as i know it is big thing and not easy to use.
Good luck!