Before you say it: i know that audio decoding of Dolby Digital audio is not suitable for the market for free due to license limitations. But I only want to decode MKV video on my iPhone for study purposes.
Which libraries should I use for the audio and video? It's important to get the channels of the 5.1 audio separated.
Ffmpeg (https://ffmpeg.org) should do it. Ffmpeg should build on iOS.
Related
I have no experience in audio programming and I want to start with an audio player (C++, Qt, MacOS), that plays multi channel track on a multi channel audio card, multiplexing all input channels on all output.
Things, I need from a framework are:
Decode popular audio formats (FLAC) and get PCM streams for each channel
Query OS about currently installed audio cards and their capabilities (channel count)
Actually mix and transfer sound between these entities
VU Meter
Better cross-platform
From what I've learned, VLC is a powerful media framework. But I haven't found info neither if it is sufficient for my task nor any good tutorial about it.
Alternatively, I consider using Phonon (it's default media framework in Qt) or Apple CoreAudio API. What suits better for this task? Is there any good tutorials on audio programming in general and using VLC and other frameworks in particular?
I'm using Kinect v2 in Ubuntu 14.04 and trying to get a way to use it as a microphone using C++ as the programming language. I already have an application in C++ and Qt where the application redirects the audio streams from different audio input devices to some audio output devices.
In that application, it is possible to get a list of available audio input devices for that PC. Currently, the application already lists Xbox NUI Sensor Analog 4-channel Input as one of the audio input devices, as does ubuntu sound settings application. I have also checked if the Kinect Input audio device supports audio format of 44100 Hz sampling rate, sampling size of 16 bit, audio/pcm codec and 2 channel counts and apparently it does support.
The problem is that, in my application, I cannot hear any sound on the output when I use kinect xbox as the microphone, whereas in case of other audio input devices I can hear the sound just fine. I'm not sure what the solution could be.
I didn't find much about the microphone in libfreenect2 pages either. I know that libfreenect2 lists "audio transfer" as one of it's missing features, but on the other hand in the documentation it is also written in the Issues and Future Work section.
Audio. There is basic access to Kinect v2's audio via ALSA (Linux). However, this is directional audio with intricate calibration, which is probably beyond the scope of this image processing library.
Does this mean that it's still possible to get access to the audio stream from Kinect V2 microphone via ALSA in ubuntu 14.04 or only the fact, at most, that the Ubunut system can only detect Kinect v2 as an audio input device but cannot be used for actual recording purposes?
If it is the first case, could you suggest me how I can get an access to the audio stream of the Kinect microphone(I cannot find anything regarding the audio or the microphone in any of the docs of libfreenect2)? Do you have any other way to get kinect v2 microphone running apart from using libfreenect2?
Is it possible to use the iPhone's hardware accelerated decoding of mp3s and AAC when using the OpenAL library?
I suppose there are two possible approaches if this is possible.
iPhone specific OpenAL extensions.
iPhone APIs to decode audio into raw bytes.
I have two specific use cases.
Completely decode a short sound bite.
Piecewise decode a larger sound file so it can be streamed into OpenAL rather than loaded all at once.
update
Boy! no one's got an answer for this? Does Apple's NDA stiffle these kinds of questions? What's going on? Surely someone else using OpenAL has wanted better audio performance.
There is at least one hardware (or hardware assisted) decoder in all iPhone device models. It can be accessed to convert mp3 and AAC files into raw PCM bytes by using the Audio Queue Services API. From thence you can process those bytes or send them to OpenAL.
AFAIK, there is no hardware audio decoder in the iPhone, 3S, and 3GS. This might have changed in the iPhone 4, but I have not heard anything to make be believe so.
How to transcode RGB images into VP8 frames (Keyframe + some dependent frames)?
So I created some images how to turn tham into VP8 now?
The easiest way to go is to use ffmpeg.
The latest release of ffmpeg (0.6) now supports the VP8 codec, and building it is now easy.
Then, ffmpeg makes it simple to gather individual frames into a movive. Here is a tutorial, but you can google for more results.
First, you need a codec library for VP8:
http://www.webmproject.org/code/build-prerequisites/
Using libvpx API you can then encode your RGB frames into VP8 frames.
I'm developing a DirectShow filter which has 2 input pins (1 for audio, 1 for video). I'm using libavcodec/libavformat/libavutil of FFMpeg for encoding the video to H264, audio to AAC and mux it/stream using RTP. So far I was able to encode video and audio correctly using libavcodec but now I see that FFMpeg seems to support RTP muxing too. Unfortunatelly, I can't find any example code which shows how to perform H264 encoding and RTP muxing. Does anybody know good samples?
Try checking out the code in HandBrake. Specifically, this file muxmp4.c, which was a jem I found working with FFMpeg / RTP. Be sure and use av_interleaved_write_frame() and the extradata fields correctly. Those were some key differences I remember for RTP.
Still, I had some stability issues with RTP/RTSP with FFMpeg, (I'm sure it's getting better). I had much better luck with live555, and you can look at the code in VLC and MPlayer for good examples on how to use it.