I have a raw h264 stream, and would like to decode it using directx11.
I'm following these docs, and am the section to actually start decoding frames. (ie Calling BeginDecodeFrame())
I'm also following some advice from this question:
Which references an older verson of the same docs I linked previously. Anyway, it's saying that I need to fill some buffers:
D3D11_VIDEO_DECODER_BUFFER_PICTURE_PARAMETERS
D3D11_VIDEO_DECODER_BUFFER_INVERSE_QUANTIZATION_MATRIX
D3D11_VIDEO_DECODER_BUFFER_BITSTREAM
D3D11_VIDEO_DECODER_BUFFER_SLICE_CONTROL
I'm relatively familiar with h264, and understand I can get this information from parsing the NALU's. That being said, I don't want to manually parse the h264 packets themselves, It seems like there should be some kind of convenience method to do this for me.. So is there??
Coming from the Mac world, I can just call: CMVideoFormatDescriptionCreateFromH264ParameterSets with the PPS and SPS data and that'll handle it for me.
A follow up on that, how exactly do I "Fill These Buffers" ?
HRESULT GetDecoderBuffer(
[in] ID3D11VideoDecoder *pDecoder,
[in] D3D11_VIDEO_DECODER_BUFFER_TYPE Type,
[out] UINT *pBufferSize,
[out] void **ppBuffer
);
The function signature expects an out param of (void**), so if I wanted to fill D3D11_VIDEO_DECODER_BUFFER_PICTURE_PARAMETERS, how do exactly to do I fill it with the parsed data? Do I need to assign a pointer to DXVA_PictureParameters and then cast it as a void** like: (void**)&myPictureParamets when I call the function?
The linked docs state:
Direct3D 11 uses the same data structures as DXVA 2.0 for decoding operations.
So I'm kind of guessing on DXVA_PictureParameters being the correct struct to use here.
I was expecting this to be easier than it has been! :P
Related
So... I'm dealing with a system that has input data coming in buffers (i.e. NOT a file). I want to determine which decoder to create to decompress an audio stream (MP3, WAV, OGG, ...) So obviously I do not know the input format.
I found out that I could determine the format using the av_probe_input_format[23]() functions. That part works great, I get a format pointer that matches the files that I use as input.
AVInputFormat * format(av_probe_input_format3(&pd, true, &score));
I can print the format->name and format->long_name and these are the correct type (so the detection is working as expected).
Now, I'm trying to understand how to convert that AVInputFormat * into a AVCodec * so I can call avcodec_alloc_context3(codec) to create the actual audio decoder.
I found a couple of functions, which I used like so:
AVCodecID const codec_id(av_codec_get_id(format->codec_tag, format->raw_codec_id));
AVCodec * codec(avcodec_find_decoder(codec_id));
Problem 1. the raw_codec_id field is marked as "private" (should not access/use anywhere in your client's code).
Problem 2. the first function always returns AV_CODEC_ID_NONE (0) so of course the second call fails each time.
Am I doing something wrong? Is there is way to instead create a generic decode that will automatically detect the type of audio I have as input? (that is, would that be the only way to make that work?)
Okay, so the fact is that trying to use these functions directly is pretty much futile. The problem I have with the design is that it forces me to actually have a callback and that callback forces me to have a thread (i.e. I have to somehow feed data from a stream, not a file or such!)
So I can use the avformat_open_input() as mentioned by Gyan, only I have to have my own AVIOContext. I was hoping I could just call functions with my incoming data and avoid the pipeline concept. The issue here is some background processes could be servers that use fork() and thus you need to be really careful (i.e. fork() is not friendly with threads).
I'm using Boosts Generic Image Library. I'm being given a string representation of an image. After decoding it, could I directly make an Image or View object with that data? Or would I need to write the data to the computer as example.png and use GIL's read_image functions? The documentation mentions dynamic images but still takes a filename as a parameter to the i/o functions.
I would ideally be looking for a function that takes a string or byte array as a parameter rather than the image name to be loaded from disk. Something like GDI+ FromStream. I see that the documentation says "All functions take the filename or a device as the first parameter. A device could be a FILE*, std::ifstream, and TIFF*." Maybe it is possible to edit the contents of an ifstream to have the image data, not sure if this is actually possible though.
I am having troubles filtering data that is passed to PR_Write. It is the Mozilla function that is used in passing all sorts of data sent to the server. I managed to hook it using a DLL( extremely basic ) using the code from Wikipedia on hooking.
The following is the declaration of the PR_Write function referenced from the Mozilla website.
PRInt32 PR_Write(PRFileDesc *fd, const void *buf, PRInt32 amount);
The second parameter buf is what I am logging by casting it to a const char*, it works fine but I don't know how I can filter the data since it logs everything from start to end.
The code below is what I tried but it is too heavy a loop and crashes Mozilla.
char *p=(char*)buf; // get pointer to beginning of the buffer
while (*p!='\00')
{
// do some data filtering
*p++;
}
The idea was from Grayhat Python book to iterate through the buffer and filter data as needed but the loop is too much since the buffer is always extremely large.
Overall, I need a way to filter the data that is passed to the second parameter.
Thanks for any suggestions in advance :)
C# Publisher is publishing continuos marketdata messages in custom protobuff format over the socket using "writeDelimitedTo" API. I have to read all messages in C++ and desearialize it. Below is my code. Since C++ don't have "parseDelimitedFrom", so have coded something like below after going through multiple suggestions in this forum.
Now my question is - Refering to the code below, If the first message size is less than 1024 then in the first iteration, i will have full stream of the 1st message and part of the stream from the 2nd message. After deserializing first message, How can i read remaining streams of the second message from socket and merge it with the stream which i read in the previous iteration ?
EDIT: Support for "delimited" format is now part of the official protobuf library. The post below predates it being added.
I've written optimally-efficient versions of parseDelimitedFrom and writeDelimitedTo in C++ here (the read and write methods of Uncompressed):
https://github.com/capnproto/capnproto/blob/06a7136708955d91f8ddc1fa3d54e620eacba13e/c%2B%2B/src/benchmark/protobuf-common.h#L101
Feel free to copy.
These implementations read from / write to a ZeroCopyInputStream / ZeroCopyOutputStream.(Hmm, for some reason my write is declared to use FileOutputStream, but you should be able to just change that to ZeroCopyOutputStream.)
So, you'll need to create a ZeroCopyInputStream which reads from your StreamSocket, then pass it to my read().
It looks like StreamSocket is a classic copying-read interface. You should therefore use CopyingInputStreamAdaptor as your ZeroCopyInputStream, wrapping an implementation of CopyingInputStream which reads from your StreamSocket.
https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.io.zero_copy_stream_impl_lite#CopyingInputStreamAdaptor
I am writing a c++ library that fetches and returns either image data or video data from a cloud server using libcurl. I've started writing some test code but still stuck at designing API because I'm not sure about what's best way to handle these media files. Storing it in a char/string variable as binary data seems to work, but I wonder if that would take up too much RAM memory if the files are too big. I'm new to this, so please suggest a solution.
You can use something like zlib to compress it in memory, and then uncompress it only when it needs to be used; however, most modern computers have quite a lot of memory, so you can handle quite a lot of images before you need to start compressing. With videos, which are effectively a LOT of images, it becomes a bit more important -- you tend to decompress as you go, and possibly even stream-from-disk as you go.
The usual way to handle this, from an API point of view, is to have something like an Image object and a Video object (classes). These objects would have functions to "get" the uncompressed image/frame. The "get" function would check to see if the data is currently compressed; if it is, it would decompress it before returning it; if it's not compressed, it can return it immediately. The way the data is actually stored (compressed/uncompressed/on disk/in memory) and the details of how to work with it are thus hidden behind the "get" function. Most importantly, this model lets you change your mind later, adding additional types of compression, adding disk-streaming support, etc., without changing how the code that calls the get() function is written.
The other challenge is how you return an Image or Video object from a function. You can do it like this:
Image getImageFromURL( const std::string &url );
But this has the interesting problem that the image is "copied" during the return process (sometimes; depends how the compiler optimizes things). This way is more memory efficient:
void getImageFromURL( const std::string &url, Image &result );
This way, you pass in the image object into which you want your image loaded. No copies are made. You can also change the 'void' return value into some kind of error/status code, if you aren't using exceptions.
If you're worried about what to do, code for both returning the data in an array and for writing the data in a file ... and pass the responsability to choose to the caller. Make your function something like
/* one of dst and outfile should be NULL */
/* if dst is not NULL, dstlen specifies the size of the array */
/* if outfile is not NULL, data is written to that file */
/* the return value indicates success (0) or reason for failure */
int getdata(unsigned char *dst, size_t dstlen,
const char *outfile,
const char *resource);