We have a web service that sends the video content in the response as binary (in different formats asx, asf, ram, mpeg, mpg, mpe, qt, mov, avi, movie, wmv, smil, mp4, mxf, gxf, flv, 3gp, f4v, mj2, omf, dv, vob).
Do you see any issue with performance, if I have an intermediate application which makes a request to web service to retrieve video content and render in browser?
Thanks
As long as the web service returns binary data directly, then there will be no performance hit. If this is an XML or SOAP web service that wraps the whole thing in a SOAP envelope and bae64 encodes it to make it all text, then you will not be able to play it directly and it will have a big impact on bandwidth, cpu, and memory.
Also note that by serving the video directly instead of using a true streaming protocol the user will only be able to seek within the portion downloaded so far. A streaming protocol like RTSP, RTMP, or the many varieties of HTTP Streaming allow seeking to any part of the file and only downloading the part seeked to.
Related
I am trying to loop audio from my Icecast server 24/7.
I have seen examples where people talk about storing their audio files on the EC2 instance or in an S3 bucket.
Do I also need a source client running on my EC2 Instance to be able to stream audio to the server? Or is there a way to play static files from Icecast?
Icecast and SHOUTcast servers work by passing a live audio stream from a source on to the users. You need something to produce a single audio stream in realtime from those source files.
The flow looks something like this:
Basically, you'll need to do everything you would in a normal radio studio, but automated. You'll stream the files from your bucket, play them to a raw audio stream, send that stream to your encoder to be compressed with the codec, and then sent to your streaming servers for distribution.
You can't simply push your audio files as-is to the Icecast server, for a few reasons:
Stream must be realtimeThe server doesn't really know or care about the timing of the stream. It takes the data its given and sends that off to the client. Therefore, if you push data faster than realtime, the server will attempt to deliver it to the client at this faster rate. Some clients will attempt to buffer this fast stream, but most will put backpressure on the stream, causing the TCP window to close, causing the client to eventually get far enough behind that the server drops the connection.
Consistent format is requiredChances are, your source files have varying sample rate, channel count, and even codec. Most clients are unable to take a change in sample rate or channel count mid-stream. I don't know of any client that supports a codec change mid-stream. (Theoretically possible with Ogg and Matroska/WebM, but yeah... not worth messing with.)
Stream should be free of ID3 tags and other file format cruftIf you simply PUT your files directly to your Icecast server, the output stream will contain more than just the audio data. At a minimum, you'd want to remove all that. Depending on your container format, you'll need to deal with timestamps as well.
Solutions
There are a handful of ways to solve this:
Radio automation softwareMany folks simply run something like RadioDJ on cloud-based servers. If you already have a radio station that uses automation, this might be a good solution. It can be expensive though, and not as flexible. You could even go as low as VLC or something for playout, but then you wouldn't have music transitions and what not.
Custom playout script (recommended)I use a browser engine, such as Chromium, and script my channels with normal JavaScript. From there, I take the output stream and pass it off to FFmpeg to encode and send to the streaming servers. This works really well, as I can do all my work in a language everybody knows, and I have easy access to data on cloud-hosted services. I can use the Web Audio API to mix and blend audio based on what's happening in realtime. As an alternative, there is Liquidsoap, but I do not recommend it these days as its language is difficult to deal with and it is not as flexible as a browser engine.
I am using Twilio Programmable video, and trying to pipe remote participant's audio in real time to Google Cloud Media Translation client.
There is a sample code on how to use Google Cloud Media Translation client via microphone on here.
What I am trying to accomplish is that instead of using a microphone and node-record-lpcm16, I want to pipe what I am getting from Twilio's AudioTrack to Google Cloud Media Translation client. According to
this doc,
Tracks represent the individual audio, data, and video media streams that are shared within a Room.
Also, according to this doc, AudioTrack contains an audio MediaStreamTrack. I am guessing this can be used to extract the audio and pipe it to somewhere else.
What's the best way of tackling this problem?
Twilio developer evangelist here.
With the MediaStreamTrack you can compose it back into a MediaStream object and then pass it to a MediaRecorder. When you start the MediaRecorder it will receive dataavailable events which will be a chunk of audio in the webm format. You can then pipe those chunks elsewhere to do the translation. I wrote a blog post on recording using the MediaRecorder, which should give you a better idea how the MediaRecorder works, but you will have to complete the work to stream the audio chunks to the server to be translated.
I want to build a live streaming app.
My thought process:
Get the Video/Audio data from the
navigator.mediaDevices.getUserMedia(constraints); [client-streamer]
create rooms using sockets(Socket.IO or WebSockets from flask) [backend]
Send the data in 1 to the room members using sockets.
display the media on the client-side.
Is that correct? How should I do it?
how do I broadcast data to specific room members and not to everyone? (flask)
How to consistently send data from the streamer -> server -> room members. the stream is given from 1 is an object, where is the data?
any other better ideas will be great! thanks.
I need to implement the server-side by myself without help from libraries that will do the work for me.
Implementing a streaming platform is not trivial. Unfortunately, it is not as simple as emitting chunks received from the MediaRecorder with onndatavailable and forwarding them to users using a WebSocket server - this is not scalable nor efficient nor reliable.
Below are some strategies you can try for different types of scenarios:
P2P: If you want to have simple peer-to-peer streaming, you can use WebRTC to achieve that with a simple socket.io server for signaling purposes.
Conference: Here things start to get more complicated. You will need a media server if you want to be somewhat scalable. One approach is to route your stream to the users using an SFU or MCU. This will take care of forwarding/processing media to different peers efficiently.
Broadcast: Here things are also non-trivial. Common WebRTC-based architectures include ingesting the WebRTC stream and forward that to an HLS server which will let your stream chunks available for clients through a CDN, or perform RTP forwarding of the WebRTC stream, convert it to RTMP using something like FFmpeg and deliver it through Youtube Live or Twitch to leverage from their infrastructure.
Be aware that the last 2 items are resource-intensive and will certainly not be cheap to maintain.
Below are some open source projects that could help you along the way:
Janus
MediaSoup
AntMedia
Jitsi
Good luck!
Explaining all this is far beyond the scope of a Stack Overflow answer.
Here are a few hints:
You need to use the MediaRecorder API to capture compressed data from your gUM (getUserMedia) stream. MediaRecorder support is inconsistent between makes and models of browser. though.
It kicks a Blob into its onndatavailable handler every so often.
They're compressed as a webm data stream.
You can push those Blobs to a server with socket.io, and the server can turn around and push them to whatever clients you want to.
Playing the webm on the clients is tricky. You may, on some makes and models of browsers, be able to feed the webm stream to the Media Source API using appendBuffer(). But some browsers cannot consume the webm streams.
These webm streams are useless to a player without all their Blob data in order. You can't just start sending a new client the Blobs of the stream when they sign in; you have to restart the MediaRecorder.
(You may be able to make it work without a MediaRecorder restart if you send the first few k bytes of the stream to each new client before sending the current Blob. Extracting those bytes is an intricate programming job involving the ebml package to parse the webm stream and extract the prologue. I have not proven this concept.)
Because getting all this to work -- originator -- server -- viewer is such a pain in the xxx neck, you may want to investigate using something like mediasoup instead. It uses WebRTC transport rather than socket.io, and works cross-platform.
I am working on a live-streaming prototype, I have been reading a lot about how live-streaming works and many different approaches but I still can't find a live-streaming stack that suits my needs...
These are the requirements for my prototype:
1)The video/audio recording must come from a web browser using the webcam, the idea is that the client preferably shouldn't need to install plugins or do anything complicated(maybe installing Flash player plugin is acceptable, only for recording the video, the viewers should be able to view the stream without plugins).
2)It can't be peer to peer since I also need to store the entire video in my server (or in Amazon s3 servers for example) for viewing later.
3)The viewers should also be able to watch the stream without the need of installing anything, from their web browsers, say Chrome and Firefox for example. We want to use the HTML5 video tag if possible.
4)The prototype should be constructed without expending money preferably. I have seen that AWS-Cloudfront and Wowza offer free trials so we are thinking about using these 2 services.
5)The prototype should be able to maintain 1 live stream at a time and 2 viewers, just that, so there are no restrictions regarding this.
Any suggestions?
I am specially stuck/confused with the uploading/encoding video part of the architecture(I am new to streaming and all the formats/codecs/protocols/technologies are making it really hard to digest).
As of right now, I came across WebRTC that apparently allows me to do what I want, record and encode video from the browser using the webcam, but this API only works with HTTPS sites. Are there any alternatives that work with HTTP sites?
The other part that I am not completely sure about is the need for an encoding server, for example Wowza Streaming Engine, why do I need it? Isn't it enough if I use for example WebRTC for encoding the video and then I just send it to the distribution service (AWS-Cloudfront for example)? I do understand that the encoding server would allow me to support many different devices since it will create lots of different encodings and serve many different HTTP protocols, but do I need it for this prototype? I just want to make a 1 format (MP4 for example) live-stream that can be viewed in 2 web browsers, that's all, I don't need variety of formats nor support for different bandwidths or devices.
Base on your requirement, WebRTC is good way.
API only works with HTTPS sites. Are there any alternatives that work
with HTTP sites?
No. Currently Firefox is only browser is allow WebRTC on HTTP, but finally it need HTTPS
For doing this prototype you need to go with the Wowza WebRTC.
While going with wowza all the streams are delivered from the wowza only.So it become a routed WebRTC.
Install Wowza - https://www.wowza.com/docs/how-to-install-and-configure-wowza-streaming-engine
Enable the WebRTC - https://www.wowza.com/docs/how-to-use-webrtc-with-wowza-streaming-engine
Downaload and configure the Streamlock. or Selfsigned JKS file - https://www.wowza.com/docs/how-to-request-an-ssl-certificate-from-a-certificate-authority
Download the sample WebRTC - https://www.wowza.com/_private/webrtc/
Publish stream using the Publish HTML and Play through the Play HTML ( Supported Chrome,Firefox & Opera Browsers)
For MP4 files in WebRTC : you need to enable the transcoder with h264 & aac. Also you need to enable the option Record all the incoming Streams in the properties of application which you are creating for the WebRTC ( Not the DVR ).Using the File writer module save all the recorded files in a custom location.By using a custom script(Bash,Python) Move all the Transcoded files to the s3 bucket, Deliver through cloudfront.
Is it possible to POST request data from browser to server in compressed format?
If yes, How can we do that?
Compressing data sent from the browser to the server is not natively supported in the browsers.
You'll have to find a workaround, using a clientside language (maybe a JavaScript GZip implementation, or a Java Applet, or ...). Be sure to visually display to the user what the browser is doing and why it is taking some time.
I don't know the scope of your application, but on company websites you could just restrict input to compressed files. Ask your users to upload .zip/.7z/.rar/... files.
The server->client responses can be gzip compressed automagically by the server.
Compressing the client->server messages is not standard, so will require some work by you. Take your very large POST data and compress it client-side, using JavaScript. Then decompress it manually on the server side.
This will usually not be a beneficial thing to do unless your bandwidth usage is a major bottleneck. Compression requires both time and CPU usage to perform.