I am playing a wav file in Portaudio in C++ via Qt and libsndfile. This is working just fine. I need the sound to be played with a low latency, so I'm trying to optimize my code so that the actual playing time of the file equals the theoretical duration of the file.
I noticed that the sound does not immediately start playing after I call Pa_StartStream(stream); but there is a small delay. I would like to know the exact time when the sound starts playing, not loading, in millisecs, because I got some other things going on dependant of that time.
The obvious way would be:
myTimer.start();
Pa_StartStream(stream);
But I think Portaudio will first call my callback function, load the buffer, and then start to play, which will invalidate my measured time, right?
How can I find out the exact time when playback starts? And while we're at it, how can I find out when it ended (I do know when I write my last data into the buffer, but I don't know when it's being played..)
Thanks
I doubt you can know (precisely) when the audio stream starts. You can certainly find out when it's finished by using Pa_SetStreamFinishedCallback.
Related
I tried hard to find a solution for the follwing requirement under C++/Visual Studio:
It's simple: For purpose of alarming I have to play a number of wav sounds from files sequentially in an endless loop, each separated by a short delay:
sound-1 ...sound-2 ... sound-n ... sound-1 ... sound-2 ...
At any time it should be possible to cancel this sequence of sounds.
I tried to play around with PlaySound(), but the problem is, that I cannot stop the sounds properly once they are running.
https://msdn.microsoft.com/en-us/library/Dd743680(v=VS.85).aspx
When a sound is played asynchronously by flag, it can be stopped by specifying a NULL for parameter 1, but then I can't know when sound-1 is finished before sound-2 is started. In fact, starting sound-2 would interrupt sound-1.
In synchroneous mode however, interruption of a sound is not possible.
To be able to cancel sounds I tried to put PlaySound into a Poco::Thread, but how to cancel a thread? It is not foreseen to cancel a thread...Using Poco::Task instaed gives an unhandled exception at 0x01002d62 ... upon cancelling, so it doesn't work either.
What would be the preferred solution for that?
Is there a better solution?
Thanks!
They're all WAV files, you know how long they last. When you start playing sound1, store the time it will end. There's no need to guess.
Currently using the lib's from FFPMEG to stream some MPEG2 TS (h264 encoded) video. The streaming is done via UDP multicast.
The issue I am having currently is two main things. There is a long initial connection time / getting the video to show (the stream also contains metadata, and that stream is detected by my media tool immediately).
Once the video gets going things are fine but it is always delayed by that initial connection time.
I am trying to get as near to LIVE streaming as possible.
Currently using the av_dict_set(&dict, "tune", "zerolatency", 0) and "profile" -> "baseline" options.
GOP size = 12;
At first I thought the issue was an i frame issue, but the initial delay is there if gopsize is 12 or default 250. Sometimes the video will connect quickly, but it is immediately dropped, the delay occurs, then it starts back up and is good from that point on.
According to documentation the zero latency option should be sending many i frames, to limit initial syncing delays.
I am starting to think its a buffering type issue, as when I close the application and leave the media player up, it then fast forwards through the delay till it hits basically where the file stops streaming.
So while I don't completely understand what was wrong, I at least fixed the problem I was having.
The issue came from using the av_write_interleaved_frame() vs. the regular av_write_frame()(this one works for live streaming), when writing out the video frames. Ill have to dig into the differences a bit more to fully understand it, but its funny sometimes how you figure out the problem you are having on a total whim after bashing your face for a few days.
I can get pretty good live ish video streaming with the tune "zerolatency" option set.
I have a c++ object that accepts sound requests and plays them with ALSA. There is thread that processes the sound requests. Some sounds are periodic and are rescheduled after the wav file contents have been written to the ALSA library. Is there a way I can find out when all the data has been played? The function snd_pcm_writei is a blocking write function, but it does not necessarily mean that the file has been played.
One option that I am considering is to call snd_pcm_drain after playing each sound file, then call snd_pcm_prepare when I play the next file. Would this be an good solution? Or is this inefficient?
Update: The "drain solution" seems to work, but is not very efficient. The calls takes a while to return (maybe it cleans up some resources) and adds latency to the program. The latency is seen best when I play many small files consecutively. A few seconds of silence can be heard between each file; this is snd_pcm_drain executing.
Might not be correct (i've done very little work in this area), but from looking at the ALSA docs here: http://www.alsa-project.org/alsa-doc/alsa-lib/pcm.html
It looks like snd_pcm_status_t holds the status information that should give you an indication of whether the stream is currently processing data or not.
I am writing an application which is a kinda video streamer.The client is receiving a video stream using udp socket.Now as I am receiving the stream I want to play it simultaneous.It is different from playing local video file lying in your hard disk in which case it can be as simple as running the file using system("vlc filename").But here many issues are involved like there can be delay in receiving and player will have to wait for the incoming data.I have come to know about using vlc to run a video stream.Can you please elaborate the step for playing the stream using vlc.I am implementing my application in c++.
EDIT: Can somebody give me some idea regarding VLC API which can be used to stream a given video to particular destination and receive that stream at other end play it.
with regards,
Mawia
Well you can always take a look at VideoLan's own homepage
Other than that, streaming is quite straightforward:
Decide on a video codec that supports streaming. (ok obvious and probably already done)
Choose appropriate packet size.
Choose appropriate video quality.
At the client side: pre-buffer at least 2 secs of video and audio.
Number 2 and 3 sound strange, but they are worth thinking about:
If you have a broadband connection, you can afford to pump big packets over to the client. Note: Packets here means consistent units of data that the client needs to have completely to decode the next bit of video. If you send big packets, say 4 secs of video, you risk lag due to waiting for the complete data unit of, well, full 4 seconds, whilst small 0.5 sec packets would get you laggy but still recognizable and relatively fluent video on a bad connection.
Same goes for quality. Pixelated and artifact ridden videos are bad, stuttering video/sound desyncing videos are worse. Rather switch down to a lower quality/higher compression setting.
If your question is purely about the getting it done part, well, points 1 and 4 should do for you.
You might ask:
"If I want to do real time live video?"
All of the advice above still applies, but all of it has to be done smarter. First things first: You cannot do realtime over bad connections. It's a reality thing. If your connection is fat enough you can reach almost real time, just pump each image and a small sound sample out without much processing or any buffering at all. It is possible to get a good client experience from that, but connections like that are highly unlikely. The trick here usually is, transmit a video quality slightly lower than the connection would allow in theory and still wiggle caching and packet reordering in there... have fun. It is hard.
Unfortunately really the only API vlc has is the command line or equivalent of the command line (you can start player instances, passing them essentially what you would have on the command line). You can use libvlc if you need multiple instances or callbacks but it's pretty opaque still...
I am writing an application needs to use large audio multi-samples, usually around 50 mb in size. One file contains approximately 80 individual short sound recordings, which can get played back by my application at any time. For this reason all the audio data gets loaded into memory for quick access.
However, when loading one of these files, it can take many seconds to put into memory, meaning my program if temporarily frozen. What is a good way to avoid this happening? It must be compatible with Windows and OS X. It freezes at this : myMultiSampleClass->open(); which has to do a lot of dynamic memory allocation and reading from the file using ifstream.
I have thought of two possible options:
Open the file and load it into memory in another thread so my application process does not freeze. I have looked into the Boost library to do this but need to do quite a lot of reading before I am ready to implement. All I would need to do is call the open() function in the thread then destroy the thread afterwards.
Come up with a scheme to make sure I don't load the entire file into memory at any one time, I just load on the fly so to speak. The problem is any sample could be triggered at any time. I know some other software has this kind of system in place but I'm not sure how it works. It depends a lot on individual computer specifications, it could work great on my computer but someone with a slow HDD/Memory could get very bad results. One idea I had was to load x samples of each audio recording into memory, then if I need to play, begin playback of the samples that already exist whilst loading the rest of the audio into memory.
Any ideas or criticisms? Thanks in advance :-)
Use a memory mapped file. Loading time is initially "instant", and the overhead of I/O will be spread over time.
I like solution 1 as a first attempt -- simple & to the point.
If you are under Windows, you can do asynchronous file operations -- what they call OVERLAPPED -- to tell the OS to load a file & let you know when it's ready.
i think the best solution is to load a small chunk or single sample of wave data at a time during playback using asynchronous I/O (as John Dibling mentioned) to a fixed size of playback buffer.
the strategy will be fill the playback buffer first then play (this will add small amount of delay but guarantees continuous playback), while playing the buffer, you can re-fill another playback buffer on different thread (overlapped), at least you need to have two playback buffer, one for playing and one for refill in the background, then switch it in real-time
later you can set how large the playback buffer size based on client PC performance (it will be trade off between memory size and processing power, fastest CPU will require smaller buffer thus lower delay).
You might want to consider a producer-consumer approach. This basically involved reading the sound data into a buffer using one thread, and streaming the data from the buffer to your sound card using another thread.
The data reader is the producer, and streaming the data to the sound card is the consumer. You need high-water and low-water marks so that, if the buffer gets full, the producer stops reading, and if the buffer gets low, the producer starts reading again.
A C++ Producer-Consumer Concurrency Template Library
http://www.bayimage.com/code/pcpaper.html
EDIT: I should add that this sort of thing is tricky. If you are building a sample player, the load on the system varies continuously as a function of which keys are being played, how many sounds are playing at once, how long the duration of each sound is, whether the sustain pedal is being pressed, and other factors such as hard disk speed and buffering, and amount of processor horsepower available. Some programming optimizations that you eventually employ will not be obvious at first glance.