I have a MOV file with two audio channels. I'd like to play both simultaneously using AVKits VideoPlayer (or really any other SwiftUI compatible view component). Looking over the docs on VideoPlayer there aren't mentions of multiple streams.
How could I create a player that allows me to play multiple audio streams at once, and select which ones I want playing?
Related
I'm researching options for creating a simple video player. What I'd like to do, is to apply some audio processing (e.g. low pass filter for simplicity) while playing back the video. I've looked at Qt multimedia API, so here's my main question:
How could I edit the audio output of a QMediaPlayer? Do I need some lower level APIs?
Additionally, if some other technologies would suit this purpose better or provide better open source libraries, feel free to suggest. I have experience with C# as well.
QMediaPlayer doesn't allow low-level access to the audio data.
I'd suggest to you to use the QAudioOutput and QAudioDecoder classes for your purpose.
The QAudioDecoder produces QAudioBuffer objects. You can access the data() of these objects, process it (modify it) and feed it to the QIODevice that is returned by the start() method of the QAudioOutput object.
This will be the audio playback path of your player.
For the video you'll still use a muted QMediaPlayer to decode the video frames from the same file and output them to a QAbstractVideoSurface. You'll then need an algorithm to sync the video and audio frames produced by the above two methods.
Recently I have tried to do some graphics on the top of VLC video using vlc-qt (which provides a video widget). The approach was trying to draw something on the widget. But it failed duo to the fact that vlc-qt's widget uses an internal widget to render video. (See more details here)
Now I'm trying to do something different. I want to try drawing text (or some rectangles) on the VLC media itself (not the widget). I suppose it's the way how VLC media player renders subtitles (isn't it?)
So the question is this: Having a vlc-qt interface, how can I access underlying vlc object and draw something on it [using libVLC API]?
I'm afraid the only way to do it with libvlc is to use libvlc_video_set_callbacks + libvlc_video_set_format_callbacks. It will decode media stream's frames to memory, which you could use as you wish.
I'm developing an application that loads frames from an ethernet camera and displays these frames in an element within a Qt QWebView.
So I would like to ask, which is the best or the most efficient way to display images in sequence from the camera, so that it would display as a live video for the user.
I'm working on a project that requires me to record webcam, microphone, and the screen. I have webcam recording, audio is a work in progress, and I stumbled across CMonitor wrapper (which I did some minor modifications to) to grab RGB images of the desktop on a specified monitor (if there are multiple monitors).
How do I go about pushing my raw RGB frames into windows media foundation to encode into a video file? My current video encoding is using a slightly modified version of this msdn sample, if that's easier to modify than it is to write a new class handler.
Or, perhaps there is some sort of media foundation route to recording the screen that I don't know of (which is possible, I'm not that great of a win32 programmer)?
Found PushSource in the Windows SDK samples, which does this.
Check Desktop Duplication API for capture desktop. Media Foundation provides two solution for encoding, MF Sink Writer for simple encoding, Media Session for a more flexible control of the media pipeline. Read this overview page first.
Background:
My background music is being played as a sound effect because I want to change the pitch between each repetition of the tune.
The Issue:
While the background music is playing there's a lot of other short sound effects happening. After a certain number of sound effects play, my background music (also a sound effect) cuts out. The queue seems to cycle so that after say 50 sound effects have played, when the 51st plays, the 1st will be released whether it has completed or not.
Request for direction:
There are two directions I can see possibly going with this issue.
1. Not play the background music as an effect and figure out how to change the pitch as background music instead of an effect
2. Some how make sure that the effect will be retained until it has completed.
Thanks
The SimpleAudioEngine doesn't allow you to control channels, so it'll just play your sound effects in one of the channels it has allocated internally. Eventually all channels are playing audio. In that case, when a new audio file is to be played and all channels are already playing, SimpleAudioEngine will cancel one of the existing audio files. On occasion this will be your background music. Dang.
What you normally do to fix this is to assign (allocate) one audio channel specifically for the background music. No other audio will ever attempt to play audio on that channel. You will have to use the "regular" CocosDenshion API. It looks like you either need to use CDAudioManager or CDSoundEngine, but I have no experience with using channels with CocosDenshion.
Personally I never used CocosDenshion, whenever I needed more control I found the ObjectAL API to be easier to understand and it has excellent documentation. I made the following helper function for ObjectAL that plays a sound on a particular channel. I simply instantiated a ALChannelSource for my background music, and only played background music through that channel. The returned ALSoundSource even allows you to change pitch, pan, etc. while the audio is playing.
+(id<ALSoundSource>) playEffect:(NSString*)effect channel:(ALChannelSource*)channel loop:(bool)loop
{
id<ALSoundSource> soundSource = nil;
if (channel)
{
ALBuffer* buffer = [[OALAudioSupport sharedInstance] bufferFromFile:effect];
soundSource = [channel play:buffer loop:loop];
}
return soundSource;
}
All other audio files I'm playing through the OALSimpleAudio class.