I'm currently working on simple VOD browser-based service, using mostly AWS technologies. HLS will be used as the streaming protocol, which is supported by Elastic Transcoder.
Currently, the source material is 720p (1280x720), and this is also the resolution I want to show to all devices that can handle it. I would like the videos to work on desktops, iPad's, and most smartphones. I'm using ViBlast with videojs, as the player.
I have the following questions:
the m3u8 playlist allows to specify multiple streams. Should each resolution get's it own playlist (with different source-streams on different bitrates), or can I put everything in one playlist (so one playlist can serve different resolutions and bitrates).
Seems desktops and most recent tablets can display 1280x720, I assume the same playlist can be used. I just need to specify bitrates. However, what is the best resolution for mobile phones? Seems every device has other dimensions (looking at Android here).
Which bitrate should I use for each device? I'm doing some research, but it seems every article has a different recommendation for the "best" setting, but never explain how they got those numbers.
If I use a playlist which contains different sources with different resolutions, does the order in the playlist matter? I've read somewhere that lowest bitrates should be listed first, but does this also apply to resolutions? Or does the player automatically picks the stream which best matches the screen?
I'm looking for a "good enough" solution that will fit most devices.
Hope this helps.
the m3u8 playlist allows to specify multiple streams. Should each
resolution get's it own playlist (with different source-streams on
different bitrates), or can I put everything in one playlist (so one
playlist can serve different resolutions and bitrates).
For reference, here is Apple's 'Technical Note TN2224' on the subject which is a good guideline for the info below.
https://developer.apple.com/library/content/technotes/tn2224/_index.html
Short answer: Each resolution should have its own variant playlist.
Typically there is one master playlist with references to the variant playlists (aka renditions). The variant playlists are different quality streams of the same video, varying in bitrate and resolution. But each variant only contains one bitrate level. Sample master playlist:
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=4648000,RESOLUTION=3840x2160
4648k/stream.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=2670000,RESOLUTION=1920x1080
2670k/stream.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1823000,RESOLUTION=1280x720
1823k/stream.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=975000,RESOLUTION=854x480
975k/stream.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=491000,RESOLUTION=640x360
491k/stream.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=186000,RESOLUTION=256x144
186k/stream.m3u8
"The bitrates are specified in the EXT-X-STREAM-INF tag using the BANDWIDTH attribute" (TN2224). And each descending bandwidth (bitrate) level has a corresponding lower resolution because there is less data available and usually expected to be viewed on smaller, mobile screens.
Seems desktops and most recent tablets can display 1280x720, I assume
the same playlist can be used. I just need to specify bitrates.
However, what is the best resolution for mobile phones? Seems every
device has other dimensions (looking at Android here).
Resolution and bitrate go together. A stream encoded with a 186K bitrate (very low) does not have enough data to fill a 1280x720 screen. But a mobile device on a cell network might not be able to download a high bitrate. So you need several variants options available, each with the appropriate resolution and bitrate.
Don't focus on a specific device or else you'll never finish. Build a ladder of bitrate/resolution variants using common 16:9 aspect ratios. E.g. 1280x720, 1024x576, 640x360,...
There are several things to consider though. Bitrate, resolution you are already considering. But are these videos encoded using H.264? If so you should consider the profile level. Here is a good article on the topic: http://www.streamingmedia.com/Articles/ReadArticle.aspx?ArticleID=94216&PageNum=1.
Which bitrate should I use for each device? I'm doing some research,
but it seems every article has a different recommendation for the
"best" setting, but never explain how they got those numbers.
Same answer as resolution. Don't focus on the actual device. Build a ladder of bitrate/resolution variants that allows the device to select the most appropriate based on available bandwidth, battery life, processing power, etc.
If I use a playlist which contains different sources with different
resolutions, does the order in the playlist matter? I've read
somewhere that lowest bitrates should be listed first, but does this
also apply to resolutions? Or does the player automatically picks the
stream which best matches the screen?
Each publisher or manufacturer might build their player differently. But this is what Apple recommends in TN2224.
"First bit rate should be one that most clients can sustain
The first entry in the master playlist will be played at the initiation of a stream and is used as part of a test to determine which stream is most appropriate. The order of the other streams is irrelevant. Therefore, the first bit rate in the playlist should be the one that most clients can sustain."
Hope that helps.
Ian
Related
I have a test.mp4 file (for example). I need to convert it so that there was an option to select the quality on the client-side in the player.
For example, if the video is in 4k resolution, then the client should be able to select the quality of auto, 4k, 1080p, 720p, and 480p.
If the video is 1080p, the choice should be auto, 1080p, 720p and 480p.
And so on.
I know I should choose to convert to Apple HLS and get an m3u8 file in the output.
Tried using ABR, but that's not what I need.
I use AWS MediaConvert to convert.
What you are describing sounds like an HLS bitrate stack. I'll answer based on that assumption.
It will be the responsibility of the playback software to present a menu of the available resolutions. If you want the player to disable its adaptive rendition selection logic and permit the viewer to stay on a specified rendition regardless of segment download times, that workflow needs to be configured within the video player object. In either case you will need an asset file group consisting of manifests and segments.
FYI, MediaConvert has both an automatic ABR mode (which determines the number of renditions & bitrate settings automatically) and a 'manual mode' where you provide the parameters of each child rendition. In this mode, each child rendition is added as a separate Output under the main Apple HLS Output Group. More information can be found here: https://docs.aws.amazon.com/mediaconvert/latest/ug/outputs-file-ABR.html.
I'm sure this question would have been asked before but I've searched and can't find anything specific to help a solution.
I'll start out outlining the initial concerns and if more indepth technical information is needed then I can give it. Hopefully there is enough information for the initial question(s).
I'm writing an app using c++ and directshow in visual studio 2010. The main project specification is for a live preview and, at any time of choosing, record the video to mpeg2 to harddrive then to dvd to be played in a standard dvd player, all the time the live preview is not to be interrupted.
The capturing seems a pretty trivial standard straight forward thing to do with directshow.
There are a couple of custom filters that i wrote. Nothing amazing but we wanted our own custom screen overlay information - time and date etc - this must be in the preview and the recorded file. I use the avi decompressor connected to the capture card video out pin, and connect the avi decompressor to my filter to give me an rgb image that i can manipulate. The output from this filter is then split via an inftee filter, one goes to the screen, the other goes into the ms mpeg2 encoder. The audio goes from the capture card audio out into the same mpeg2 encoder. Output from the mpeg2 encoder then goes to a file. That file then gets authored for dvd and burnt to dvd.
So my questions are...
Where and how would be the best place to allow starting and stopping of only mpeg2 file output, to be done via user action?
I have tried using smart tee filters - 1 for video and 1 for audio as the last filter BEFORE the mpeg2 encoder, then using the iamstreamcontrol interface to turn off the pins at the appropriate time. Should this cause any timing issues with the final mpeg2? as the output file will play via mplayer and vlc etc but doesnt get converted to be mpeg2 dvd compliant ( for testing - via any dvd authoring software - complaints of a broken file and somteimes gives time references ) - is it possible that time stamps in the file are a problem and giving an error? If the file is captured from the first moment that capture commences ( as opposed to say after 5 mins of streaming ) then everything is ok.
I did think of using the streambuffer route - http://msdn.microsoft.com/en-gb/library/windows/desktop/dd693041(v=vs.85).aspx - but I'm not sure on the best direction to takes things. It seems that are possibly a few choices for the best direction.
Any help and tips would be greatly appreciated. Especially if theres websites/books/information of DirectShow filters,pins,graphs and how they all flow together.
EDIT: I was thinking of making my own copy of the 'Smart Tee' filter that in that I would have 2 pins coming in - audio and video - and 4 out pins - 2 video ( 1 preview and 1 capture ) and 2 of the same for audio, but would I end up with the same issue? And what is the correct way to handle 'switching off' the capture pins of that custom filter. Would I be wasting my time to work on something like this? Is it a simple case of overriding the Active/Inactive methods of the output pin(s) and either send or not send the sample? I feel its not that easy?
Many thanks!
Where and how would be the best place to allow starting and stopping of only mpeg2 file output, to be done via user action?
For this kind of action I would recommend GMFBridge. Creating your own filter is not easy. GMFBridge allows you to use two separate graphs with a dynamic connection. Use the first graph for the preview and the second graph for the file output. And only connect the graphs after a user action.
I'm looking for a video library for Qt 4 (C++/Windows) that has:
1) Basic video playback functionality
It should play all the common video formats such as DVD VOB and MP4/MKV/AVI (h264, xvid, divx). It should also be able to deinterlace the video automatically and display it in Display Aspect Ratio.
2) Cropping
It should have some basic functionality to remove black bars (user supplied arguments).
3) Snapshots
It should have functionality to take snapshots in memory.
4) Frame-by-frame seeking
It should have some basic functionality to do frame-by-frame seeking, e.g. prevFrame(), nextFrame(), jumpTo(frame) and getNumFrames().
I have tried the following and from what I could find the functionality they support:
Qt Phonon:
Yes. Plays all the needed formats and displays them correctly.
No.
No. Not implemented (returns empty image).
No.
QtFFmpegWrapper:
Partial. Does not deinterlace DVD VOBs. Does not display DVD VOBs in DAR.
No.
Yes.
Partial. Broken for MKV (h264).
Qt VLC:
Yes. Plays all the needed formats and displays them correctly.
Yes. Have not tried if it works though.
Partial. Only to disk. edit: QPixmap::grabWindow(player->videoWidget()->winId()) works.
No. Only by milliseconds.
Now I'm looking at QVision, which seems to have all of those features except for cropping. Maybe implementing cropping isn't that difficult. But I'm wondering if there's any other libraries I should look into? Or perhaps I missed something and they're possible with one of these libraries. Thanks.
You can consider Movie Player Gold SDK ActiveX 3.6 from ViscomSoft. I don't see cropping mentioned on their site but memory snapshots and frame-by-frame steps are among supported features.
I used their VideoEdit and Screen2Video SDKs in Windows Qt software, worked quite well.
I need some help on software design. Let's say I have a camera that gets acquisitions, send them to a filter, and display the images one at a time.
Now, what I want is to wait for two images and after that send the two images to the filter and both of them to the screen.
I thought of two options and I'm wondering which one to choose:
In my Acquisitioner (or whatever) class, should I put a queue which waits for two images before sending them to the Filterer class?
Should I put an Accumulator class between the Acquisitionner & Filterer?
Both would work in the end, but which one do you think would be better?
Thanks!
To give a direct answer, I would implement the Accumulator policy in a separate object. Here's the why:
Working on similar designs in the past, I found it very helpful to think of different 'actors' in this model as sources and sinks. A source object would be capable of producing or outputting an image to the attached sink object. Filters or accumulators in this system would be designed as pipes -- in other words they would implement interfaces of both a sink and a source. Once you come up with a mechanism for connecting generic sources, pipes, and sinks, it's very easy to implement an accumulation policy as a pipe, which for every nth image received, would hold on to it if n is odd, and output both of them, if n is even.
Once you have this system, it will be trivial for you to change out sources (image file readers, movie decoders, camera capture interfaces), sinks (image file or movie encoders, display viewers, etc), and pipes (filters, accumulators, encoders, multiplexers) without disrupting the rest of your code.
It depends. But, if all that your queue does is waiting for a second image to come, i reckon you could simply implement it right in the Acquisitioner.
On the other hand, if you want to incorporate whatever additional functionality there, then added modularity and all the benefits that come hand in hand with it would not hurt one tiny bit.
I don't think it matters all that much in this particular case.
I'm taking a look at some random Icecast playlists (available here: http://dir.xiph.org/index.php) and I'm wondering why many seem to contain a list of the same mp3 file.
e.g.:
GET http://dir.xiph.org/listen/193289/listen.m3u
http://87.230.101.49:80/top100station.mp3
http://87.230.103.107:80/top100station.mp3
http://87.230.101.16:80/top100station.mp3
http://87.230.101.78:80/top100station.mp3
http://87.230.101.11:80/top100station.mp3
http://87.230.103.85:80/top100station.mp3
http://80.237.158.87:80/top100station.mp3
http://87.230.101.30:80/top100station.mp3
http://80.237.158.88:80/top100station.mp3
http://87.230.103.9:80/top100station.mp3
http://87.230.103.58:80/top100station.mp3
http://87.230.101.12:80/top100station.mp3
http://87.230.101.24:80/top100station.mp3
http://87.230.103.60:80/top100station.mp3
http://87.230.103.8:80/top100station.mp3
http://87.230.101.25:80/top100station.mp3
http://87.230.101.56:80/top100station.mp3
http://87.230.101.50:80/top100station.mp3
For what it's worth, Icecast streams are intended for playing those Shoutcast-type live streams (think live radio over internet) so it makes sense that there wouldn't be a list of different tracks..but what are those repeats? Different bitrates or just mirrors?
I'm asking all of this because I'm looking to stream one of those mp3s inside my mobile app so I'm wondering if there any need to somehow figure out which url to use...
Internet radio streams are typically mirrored across many servers. This balances the bandwidth load, and reduces points of failure.
In addition, it is common for servers to fill up as they get popular. Most players will go to the next track in the playlist when a track fails, so this allows autofailover when a client cannot connect to a specific server, or if it gets disconnected.