The project I'm currently working on, requires an addition to the already existing VoIP capabilities. The core for speech processing is in C, the remainder is in C++ with Qt - the audio is handled via portaudio. The connection between users is currently established via UDP, which I think has to be changed for the planned video connection. Developing platform is Windows on VS2012 - however, the system is cross-platform.
In a nutshell, what I want to do is: Grab the video signal from a webcam, synchronize audio coming from C core and video from webcam and use a library and codecs for (de-)coding/muxing the signals on the respective sides and sending via RTP. The system should be capable of multicast transmission.
I did some research for possible libraries and stumbled upon ffmpeg and libVLC. For the codec I thought about using x264. And if I'm correct, ffmpeg and libVLC should both be capable of what I'm looking for?
However I'm not sure which one to pick, and from their documentations I really can't extract, which library is the better fit. Has anybody had similar problems and can help me out - I'm quite a newbie, when it comes to video processing and encoding.
Extra question: Do you have any hints or approaches on syncing the video and audio signals?
If anyone is interested, this is what I ended up doing:
I am using the WebM container format, VP8 with Vorbis currently (but going to change to VP9 with Opus soon if out of beta), handled by ffmpeg/libav libraries for encoding/decoding/muxing etc. and SDL for displaying and threading. ffmpeg/libav was cross-compiled on Unix with LGPL support to keep our project closed source.
Related
I'm looking into coding a very simple LAN based home monitoring system using python 2.7 on Windows. I have a number of computers around the house each with a usb webcam attached. The cameras have built in microphones. I'm looking for the best way to capture and stream the audio and video over the network, then recieve and view/listen to it. I'm guessing I'd have to use PyAudio to get the audio from the microphone and CV2 to get the video, past that, I'm not sure how I'd stream that data to another computer, recieve it and then view/listen to it.
Not realy a question here, but still I guess I have a solution for you
(but there a millions of ways to solve this.)
My way is ROS (Robot Operating System), which is basically a TCP/IP Service Wrapper. Anyway you can simple broadcast and receive streams via your network.
ROS can be implemented in C++ or Python.
ROS usually applies OpenCV as CV-library, so it should suite your wishes.
Straight forward examples for video are (i.e.) here:
mjpeg_server
web_video_server
Audio streaming is introduced here:
audio_common (there might be solutions via PyAudio. Not sure if other solutions might come more easy and still suitable.)
I need to write a (portable) program that, amongst other things, is able to play a video file. The QMediaPlayer in Qt5 seems to be the obvious choice, but apparently it only uses codecs that are installed on the executing machine.
All I need for the video player is being able to play at least one video format on all (windows) machines without installing anything, and communicate the current status to the main program (currrent time in ms, playing/paused).
Specific question: Can someone give me some hints where I have to look? How can I implement a video codec into my program so that QMediaPlayer can reliably play videos on all machines? Alternatively (but less prefered): How to (easily) implement another video player into the program that is capable of doing the task?
I've already tried my luck with libVLC and one or two other options but it never worked so far and it seems to make my program way more complex than it has to be.
QMediaPlayer can play any format that the platform's media service supports. You should install the right codecs for a video type to be played successfully. So you have to install GStreamer codecs on Linux or if you are on Windows you should install something KLite Codecs.
May be it is good to consider using libraries like QtAV. It will work for most platforms and no additional codecs and plugins are required to install.
QtAV uses FFmpeg and has some interesting features like multiple renderers for 1 player, region of interest(video crop), custom filters, ... It also works fine for both Qt4 and Qt5.
QMediaPlayer uses the underlaying video framework of whatever machine you use, that means there's no way to portable write a player using such class. Your only solution, in my opinion, is switching to FFMpeg playback library, that uses native codecs instead.
I am actually working on a server-client multimedia player. This player can be a server to stream a MP3 file (or wma, wav, ogg, flac ...) over the network to another player (client).
I worked first on a basic network communication (client-server), that send and receive bits. But I have a problem : the audio encoding. I need a tool to encode the audio data to be able to send a little part of it through the network and let the client play it before the next part is coming.
I saw a few tools on internet such as BASS library, Live555 ... I used to work with PortAudio for student's projects but I hate it.
So basically, I need a tool to encode audio data (server side), (I can send it over lan), and decode data to play it (client-side).
Do you guys have some ideas about how to do it ? Which tool could be useful for me in that case?
PS : I am trying to use Qt library for the network interface (it is efficient, and it works on windows, linux, mac) ... Is there any audio streaming tool included in the Qt library ?
You can try FFMPEG. It can convert almost anything to anything (so it claims) and it is a widely used open source library.
We use it in our application mainly for decoding video/audio streams.
I'm trying to get mpg123 audio decoder to work with QT on windows. How do i play the decoded audio data at the right speed with Qmultimedia module in push mode. Currently i'm using simple timer to get it to play audio but it's not very efficient way to do it, if I do anything else at the same time audio get all distorted. Is there any better way to send the decoded data to audio output? It would be nice if anyone could point me to any nice examples using Qmultimedia module and Qaudiooutput class. I've tried to figure out QT example project "audiooutput" but it seems that it's also using timer to send audio to output in push mode.. Hope that I'm not too confusing.
I also had to figure that out and I would also suggest using the Phonon framework to do this.
It uses Windows Media Player as host on Windows, QuickTime on Mac and some KDE stuff on Linux.
So it's pretty platform independent.
If you need more low-level functionality, you should take a look into an open-source project called portaudio. It's very easy to use and you can manipulate or even fill buffers from code.
I used it to build an oscillator.
Hope that helps!
Best,
guitarflow
I want to create a Qt widget that can play incoming RTP streams where the video is encoded as H264 and contains no audio.
My basic plan for implementation is this:
Create a Phonon MediaSource object (Stream type).
Connect it with a QIODevice subclass that provides the data
Obtain the video data using either:
The JRTPLIB client library
The GStreamer gstrtpbin plugin. This plugin takes care depayloading the packages and decoding the video. Maybe this improves the chances that Phonon will recognize the data.
My environment:
Ubuntu 9.10
Qt 4.6
My questions:
Is my approach a good one? Perhaps I'm overlooking a more obvious or simple solution?
I'm currently experiencing this issue: when trying to play the video stream the state of the MediaObject turns to ErrorState with errorType FatalError. Can anyone tell me what I'm doing wrong?
Edit
One solution I found is using libVLC in combination with Qt, which I learned about in this thread. Here's a code sample for the interested.
I'm still looking for a Phonon-based solution.
Ideally I would only need to provide an SDP file and job is done.
I was able to get it to work using the libVLC solution. I can't garantuee that this is the best solution though as I simply stopped looking after that.
Here's a link to the libVLC sample.
The way I understand Phonon works at least in Windows is that QT provides a phonon backend plugin for DirectShow (\plugins\phonon_backend\phonon_ds94.dll) and GStreamer in your case. Then you would either obtain or write your own DirectShow filter which can accept RTP streams as a source. DirectShow takes care of the decoding, and Phonon will take care of the rendering.
So if the backend works, the application code is as simple as:
Phonon::MediaObject *media = new Phonon::MediaObject();
Phonon::VideoWidget *video = new Phonon::VideoWidget();
Phonon::createPath(media, video);
media->setCurrentSource(source);
media->play();
Seems that the problem lies with the GStreamer backend accepting RTP as a source. Can you playback that source in standalone GStreamer without any problems?