Among the fragments I streamed into KVS via its gstreamer plugin, I see that hundreds of fragments - over 55% - have 0 in the field "FragmentLengthInMilliseconds" (fetched via the LIST_FRAGMENTS api). They all have the same timestamp. Another thing I observed is that in these cases, the value of "FragmentSizeInBytes" is almost always 12265.
I wonder why these fragments even exist? Am I missing something in my gstreamer plugin configuration - to get rid of them (if they are useless)? Here is how I configured it in C++:
g_object_set(kvs_sink,
"stream-name", stream_name.c_str(),
"storage-size", 1024,
"retention-period", 168, // in hours
"aws-region", AWS_REGION, NULL);
This is the plugin I'm using https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp
Your configuration looks OK. Fragment duration is calculated in the backend by adding the frame durations for the entire fragment. Some media pipelines have no way of specifying the frame durations and the fragment duration then evaluates to the timestamp of the last frame minus the timestamp of the first frame. In case of a single frame fragments and no duration specified, it will evaluate to 0. The 0 duration points to an issue with your media pipeline upstream. You might want to double check your configuration. If you are actually streaming video, then ensure your encoder produces a regular cadence I-frames. Take a look at the multiple samples in the repository and look through the docs.
Related
How should I validate pts/dts after demuxing and then after decoding?
For me it is significant to have valid pts all the time for days and
possibly weeks of continuous streaming.
After demuxing I check:
dts <= pts
prev_packet_dts < next_packet_pts
I also discard packets with AV_NOPTS_VALUE and wait for packets with
proper pts, because I don't know video duration at this case.
pts of packets can be not increasing because of I-P-B frames
Is it all right?
What about decoded AVFrames?
Should 'pts' be increasing all the time?
Why at some point 'pts' could lag behind 'dts'?
Why pict_type is a parameter of AVFrame? Should be at AVPacket, because
AVPacket is a compressed frame, not the opposite?
Ideally, yes. Unless if your format allows discontinuities, or wraps timestamps around due to overflow, like MPEG-TS.
Writing error.
It is an informational field, indicating the provenance of the frame. It can be used by filters or encoders, e.g. keyframe alignment during a re-encode.
At libav support I was advised to not rely on decoder output. It is more solid to produce pts/dts for encoding/muxing manually and I should search for ffmpeg tools sources to proper implementation. I will search for this approach.
For now I discard AVFrames only with AV_NOPTS_VALUE, and the rest of encoding/muxing works fine.
Validation of AVPackets after Demuxing remains the same, as described above.
"failureReason": "Job validation failed: Request field config is
invalid, expected an estimated total output size of at most 400 GB
(current value is 1194622697155 bytes).",
The actual input file was only 8 seconds long. It was created using the safari media recorder api on mac osx.
"failureReason": "Job validation failed: Request field
config.editList[0].startTimeOffset is 0s, expected start time less
than the minimum duration of all inputs for this atom (0s).",
The actual input file was 8 seconds long. It was created using the desktop Chrome media recorder api, with mimeType "webm; codecs=vp9" on mac osx.
Note that Stackoverlow wouldn't allow me to include the tag google-cloud-transcoder suggested by "Getting Support" https://cloud.google.com/transcoder/docs/getting-support?hl=sr
Like Faniel mentioned, your first issue is that your video was less than 10 seconds which is below the minimum 10 seconds for the API.
Your second issue is that the "Duration" information is likely missing from the EBML headers of your .webm file. When you record with MediaRecorder the duration of your video is set to N/A in the file headers as it is not known in advance. This means the Transcoder API will treat the length of your video is Infinity / 0. Some consider this a bug with Chromium.
To confirm this is your issue you can use ts-ebml or ffprobe to inspect the headers of your video. You can also use these tools to repair the headers. Read more about this here and here
Also just try running with the Transcoder API with this demo .webm which has its duration information set correctly.
This Google documentation states that the input file’s length must be at least 5 seconds in duration and should be stored in Cloud Storage (for example, gs://bucket/inputs/file.mp4). Job Validation error can occur when the inputs are not properly packaged and don't contain duration metadata or contain incorrect duration metadata. When the inputs are not properly packaged, we can explicitly specify startTimeOffset and endTimeOffset in the job config to set the correct duration. If the duration of the ffprobe output (in seconds) of the job config is more than 400 GB, it can result in a job validation error. We can use the following formula to estimate the output size.
estimatedTotalOutputSizeInBytes = bitrateBps * outputDurationInSec / 8;
Thanks for the question and feedback. The Transcoder API currently has a minimum duration of 10 seconds which may be why the job wasn't successful.
I am looking for a way to receive as an input any video (that is supported on iOS) and save on the device a new video with a new Frame Per Second rate. The motivation is to decrease the video size, and as well make it as lite weighted as possible.
Tried using ffmpeg library from command line (need it to run directly from application)
Tried working with SDAVAssetExportSessionDelegate, but managed only to change the bit per second (each frame quality is lower)
Though to work with OpenCV - but preferring something lighter and build in if possible
Objective C:
'''
compressionEncoder.videoSettings = #
{
AVVideoCodecKey: AVVideoCodecTypeH264,
AVVideoWidthKey: [NSNumber numberWithInt:width], //Set your resolution width here
AVVideoHeightKey: [NSNumber numberWithInt:height], //set your resolution height here
AVVideoCompressionPropertiesKey: #
{
AVVideoAverageBitRateKey: [NSNumber numberWithInt:bitRateKey], // Give bitrate for lower size low values
AVVideoProfileLevelKey: AVVideoProfileLevelH264High40,
// Does not change - quality setting and not reletaed to playback framerate!
//AVVideoMaxKeyFrameIntervalKey: #800,
},
};
compressionEncoder.audioSettings = #
{
AVFormatIDKey: #(kAudioFormatMPEG4AAC),
AVNumberOfChannelsKey: #2,
AVSampleRateKey: #44100,
AVEncoderBitRateKey: #128000,
};
'''
Expected a video with less Frame Per Second, each frame is in the same quality. Similar to a brief thumbnail summary of the video
The type of conversion you are doing will be time and power consuming on a mobile device, but I am guessing you are already aware of that.
Given your end goal is to reduce size, while presumably maintaining a reasonable quality, you may find you want to experiment with different settings etc in the encodings.
For this type of video manipulation, ffmpeg is a good choice as you probably saw from your command line usage. To use ffmpeg from an application, a common approach is to use a well supported 'ffmpeg wrapper' - this effectively runs the Ffmpeg command line commands from wihtin your application.
The advantage is that all the usual syntax should work and you can leverage the vast amount of info on ffmpeg command line syntax on the web. The downsides are that ffmpeg was not not designed to be wrapped like this so you may see some issues, although with a well supported wrapper you should find either help or that others have already worked around the issues.
Some examples of popular iOS ffmpeg wrappers:
https://github.com/tanersener/mobile-ffmpeg
https://github.com/sunlubo/SwiftFFmpeg
Get MobileFFMpeg up and running:
https://stackoverflow.com/a/59325680/1466453
Once you can make MobileFFMpeg calls in your IOS code then changing frame rate is pretty straightforward with this code:
[MobileFFmpeg execute: #"-i -filter:v fps=fps=30 "];
I would like to use Overtone to play a longer continuous audio file from disk.
I know Overtone has facilities for loading in samples into memory, but seeing as these files will be long and large (possibly on the order of hours), this is not the method I want to use.
SuperCollider - which Overtone uses as its audio engine - however, also has another way to load and stream files, namely using DiskIn, which Overtone also seems to have, but I wasn't able to find (docs, github) a corresponding Buffer.cueSoundFile() function.
Does Overtone have cueSoundFile at all? Is there another way I can use?
cueSoundFile is a fancier term for the equivalent osc message /b_read which you'll find in Overtone as overtone.sc.buffer/buffer-cue.
As a 5 second demo, this plays a 2-channel file from disk.
(demo (disk-in 2 (buffer-cue "~/Music/10mb.wav")))
And the doc for disk-in (SC DiskIn)
user=> (doc disk-in)
-------------------------
overtone.live/disk-in
([numChannels bufnum loop])
stream audio in from disk file
[numChannels :none, bufnum :none, loop 0]
numChannels - Number of channels in the incoming
audio.
bufnum - Id of buffer
loop - Soundfile will loop if 1 otherwise
not.
Continuously play a longer soundfile from disk. This
requires a buffer to be preloaded with one buffer size of
sound. If loop is set to 1, the soundfile will loop.
Categories: InOut, Buffer
Rates: [ :ar ]
Default rate: :ar
What is the primary difference between SetTime and SetMediaTime?
Right now in my directshow livesource I calculate the time it like this
REFERENCE_TIME rtStart = m_rtLastSampleTime;
m_rtLastSampleTime += pVih->AvgTimePerFrame;
pms->SetTime(&rtStart, &m_rtLastSampleTime);
pms->SetSyncPoint(TRUE);
pms->SetDiscontinuity(rtStart <= 1);
This doesn't work with some encoders.
I've noticed that source that do work with these encoders set mediatime and they seem to jump up.
Media Times:
Optionally, the filter can also specify a media time for the sample. In a video stream, media time represents the frame number. In an audio stream, media time represents the sample number in the packet. For example, if each packet contains one second of 44.1 kilohertz (kHz) audio, the first packet has a media start time of zero and a media stop time of 44100. In a seekable stream, the media time is always relative to the start time of the stream. For example, suppose you seek to 2 seconds from the start of a 15-fps video stream. The first media sample after the seek has a time stamp of zero but a media time of 30.
Renderer and mux filters can use the media time to determine whether frames or samples have been dropped, by checking for gaps. However, filters are not required to set the media time. To set the media time on a sample, call the IMediaSample::SetMediaTime method.
I don't think it is actually used anywhere. SetTime, on the contrary, is important.