Portaudio - Unable to play audio file - c++

I'm trying to implement a very simple API for audio playback using portaudio. I have minimal amount of code needed to play the audio file but I am not getting any errors and/or audio output.
Here is the code,
AudioStream::AudioStream()
{
PaError err = Pa_Initialize();
if (err != paNoError)
SH_LOG_ERROR("PAError: {}", Pa_GetErrorText(err));
}
AudioStream::~AudioStream()
{
PaError err = Pa_Terminate();
if (err != paNoError)
SH_LOG_ERROR("PAError: {}", Pa_GetErrorText(err));
}
int AudioStream::Callback(const void* inputBuffer, void* outputBuffer,
unsigned long framesPerBuffer,
const PaStreamCallbackTimeInfo* timeInfo,
PaStreamCallbackFlags statusFlags,
void* userData)
{
// Prevent warnings
(void)inputBuffer;
(void)timeInfo;
(void)statusFlags;
// an AudioFile gets passed as userData
AudioFile* file = (AudioFile*)userData;
float* out = (float*)outputBuffer;
sf_seek(file->file, file->readHead, SF_SEEK_SET);
auto data = std::make_unique<float[]>(framesPerBuffer * file->info.channels);
file->count = sf_read_float(file->file, data.get(), framesPerBuffer * file->info.channels);
for (int i = 0; i < framesPerBuffer * file->info.channels; i++)
*out++ = data[i];
file->readHead += file->buffer_size;
if (file->count > 0)
return paContinue;
else
return paComplete;
}
AudioFile AudioStream::Load(const char* path)
{
AudioFile file;
std::memset(&file.info, 0, sizeof(file.info));
file.file = sf_open(path, SFM_READ, &file.info);
return file;
}
bool AudioStream::Play(AudioFile* file)
{
m_OutputParameters.device = Pa_GetDefaultOutputDevice();
m_OutputParameters.channelCount = file->info.channels;
m_OutputParameters.sampleFormat = paFloat32;
m_OutputParameters.suggestedLatency = Pa_GetDeviceInfo(m_OutputParameters.device)->defaultLowOutputLatency;
m_OutputParameters.hostApiSpecificStreamInfo = nullptr;
// Check if m_OutputParameters work
PaError err = Pa_IsFormatSupported(nullptr, &m_OutputParameters, file->info.samplerate);
if (err != paFormatIsSupported)
{
SH_LOG_ERROR("PAError: {}", Pa_GetErrorText(err));
return false;
}
err = Pa_OpenStream(&m_pStream,
nullptr,
&m_OutputParameters,
file->info.samplerate,
file->buffer_size,
paClipOff,
&AudioStream::Callback,
file);
if (err != paNoError)
{
SH_LOG_ERROR("PAError: {}", Pa_GetErrorText(err));
return false;
}
if (m_pStream)
{
err = Pa_StartStream(m_pStream);
if (err != paNoError)
{
SH_LOG_ERROR("PAError: {}", Pa_GetErrorText(err));
return false;
}
SH_LOG_DEBUG("Waiting for playback to finish..");
while (IsStreamActive()) // <----- this works, but the application freezes until this ends
Pa_Sleep(500);
Stop();
if (IsStreamStopped())
{
SH_LOG_DEBUG("Stream stopped..");
err = Pa_CloseStream(m_pStream);
if (err != paNoError)
{
SH_LOG_ERROR("PAError: {}", Pa_GetErrorText(err));
return false;
}
}
SH_LOG_DEBUG("Done..");
}
return true;
}
bool AudioStream::Stop()
{
PaError err = Pa_StopStream(m_pStream);
if (err != paNoError)
{
SH_LOG_ERROR("PAError: {}", Pa_GetErrorText(err));
return false;
}
else
return true;
}
bool AudioStream::IsStreamStopped()
{
if (Pa_IsStreamStopped(m_pStream))
return true;
else
return false;
}
I noticed that if I add a print statement in the while loop I do get audio output.
// Wait until file finishes playing
while (file->count > 0) { SH_LOG_DEBUG(""); }
Why doesn't it play with a print statement or anything in while loop? Do I have to have something in the while loop?

You've got two flaws that I can see.
One is using what looks like a class method as the callback to PulseAudio. Since PA is a C API, it expects a C function pointer here. It won't call that function with the this pointer set, so you can't have a class method here. But maybe AudioStream::Callback is static? That will work.
Two is that you need to consider that the callback is called in another thread. The compiler does not take that into account when it optimizes the code. As far as it knows, there is nothing in your empty while loop that could possibly change the value of file->count.
Once you call the debug function, it brings in enough code, some probably in libraries that are already compiled, that the compiler can't be sure nothing has modified file->count. Maybe SH_LOG_DEBUG() calls prinf() and then printf() calls AudioStream::Load()? Of course it doesn't, but if the compiler doesn't see the code of printf() because it's already in a library, and your AudioStream object is global, then it's possible. So it actually works like you want.
But even when it works, it's really bad. Because the thread is sitting there busy waiting for the count to stop, hogging the/a CPU. It should block and sleep, then get notified when the playback finishes.
If you want to pass information between threads in a way that works, and also block instead of busy waiting too, look into the C++ concurrency support library. The C++20 std::latch and std::barrier would work well here. Or use a C++11 std::condition_variable.

Related

ZwSetIniformationFile and FileRenameInformation

I am trying to rename files from kernel.
with this api https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/nf-wdm-zwsetinformationfile
NTSTATUS Files::RenameFile(WCHAR* OriginalName, WCHAR* NewName)
{
// msdn says driver must be at IRQL PASSIVE_LEVEL to make calls to ZwSetInformationFile
if (KeGetCurrentIrql() != PASSIVE_LEVEL)
{
printf("IRQL invalid\n");
return STATUS_INVALID_LEVEL;
}
// Open handle to file , providing OriginalName with full path, example: "\\DosDevices\\C:\\named_file.txt"
// also to be able to rename files u delete the DELETE permission, but i assume with GENERIC_ALL is already giving me enough of them
auto FileHandle = Files(OriginalName, Files::OpenExisting, GENERIC_ALL, 0);
if (FileHandle.CreationStatus != STATUS_SUCCESS)
{
printf("Failed to open handle to file %ws. ERR: 0x%x\n", OriginalName, FileHandle.CreationStatus);
return FileHandle.CreationStatus;
}
IO_STATUS_BLOCK ioStatusBlock;
// msnd info for ZwSetInformatonFile when using FileRenameInformation class says that size must be the size of the
// structure + size of new name in bytes
const auto size = sizeof(FILE_RENAME_INFORMATION) + sizeof(NewName);
// allocate resources for struct
const auto rename_info = static_cast<PFILE_RENAME_INFORMATION>(ExAllocatePool(PagedPool, size));
if(rename_info == nullptr)
{
printf("Failed allocating rename info structure\n");
FileHandle.Close();
return STATUS_INSUFFICIENT_RESOURCES;
}
memset(rename_info, 0, size);
wcscpy(rename_info->FileName, NewName);
rename_info->FileNameLength = sizeof(NewName);// size in bytes
rename_info->RootDirectory = nullptr; // msnd: must be null if the filename is the absolute path
rename_info->ReplaceIfExists = false; // i dont want to replace if exissts
const auto status = ZwSetInformationFile(FileHandle.hFile, &ioStatusBlock, rename_info, size, FileRenameInformation);
// free resources
ExFreePool(rename_info);
FileHandle.Close();
// ZwSetInformationFile fails with 0xc0000034 STATUS_OBJECT_NAME_NOT_FOUND
if(status != STATUS_SUCCESS)
{
printf("0x%x : %ws\n", status, rename_info->FileName); // -> 0xc0000034 : "\\DosDevices\\C:\\renamed_file.txt"
return status;
}
printf("Renamed %ws to %ws\n", OriginalName, NewName);
return STATUS_SUCCESS;
}
i commented a bit of the code, but TLDR; it gives me 0xc0000034 STATUS_OBJECT_NAME_NOT_FOUND.
I followed everything as stated in the MSDN but im always with the same issue, the originalname file does exist in disk since it opens handle succesfully.
any help is appreciated, thanks

Why does adding audio stream to ffmpeg's libavcodec output container cause a crash?

As it stands, my project correctly uses libavcodec to decode a video, where each frame is manipulated (it doesn't matter how) and output to a new video. I've cobbled this together from examples found online, and it works. The result is a perfect .mp4 of the manipulated frames, minus the audio.
My problem is, when I try to add an audio stream to the output container, I get a crash in mux.c that I can't explain. It's in static int compute_muxer_pkt_fields(AVFormatContext *s, AVStream *st, AVPacket *pkt). Where st->internal->priv_pts->val = pkt->dts; is attempted, priv_pts is nullptr.
I don't recall the version number, but this is from a November 4, 2020 ffmpeg build from git.
My MediaContentMgr is much bigger than what I have here. I'm stripping out everything to do with the frame manipulation, so if I'm missing anything, please let me know and I'll edit.
The code that, when added, triggers the nullptr exception, is called out inline
The .h:
#ifndef _API_EXAMPLE_H
#define _API_EXAMPLE_H
#include <glad/glad.h>
#include <GLFW/glfw3.h>
#include "glm/glm.hpp"
extern "C" {
#include <libavcodec/avcodec.h>
#include <libavformat/avformat.h>
#include <libavutil/avutil.h>
#include <libavutil/opt.h>
#include <libswscale/swscale.h>
}
#include "shader_s.h"
class MediaContainerMgr {
public:
MediaContainerMgr(const std::string& infile, const std::string& vert, const std::string& frag,
const glm::vec3* extents);
~MediaContainerMgr();
void render();
bool recording() { return m_recording; }
// Major thanks to "shi-yan" who helped make this possible:
// https://github.com/shi-yan/videosamples/blob/master/libavmp4encoding/main.cpp
bool init_video_output(const std::string& video_file_name, unsigned int width, unsigned int height);
bool output_video_frame(uint8_t* buf);
bool finalize_output();
private:
AVFormatContext* m_format_context;
AVCodec* m_video_codec;
AVCodec* m_audio_codec;
AVCodecParameters* m_video_codec_parameters;
AVCodecParameters* m_audio_codec_parameters;
AVCodecContext* m_codec_context;
AVFrame* m_frame;
AVPacket* m_packet;
uint32_t m_video_stream_index;
uint32_t m_audio_stream_index;
void init_rendering(const glm::vec3* extents);
int decode_packet();
// For writing the output video:
void free_output_assets();
bool m_recording;
AVOutputFormat* m_output_format;
AVFormatContext* m_output_format_context;
AVCodec* m_output_video_codec;
AVCodecContext* m_output_video_codec_context;
AVFrame* m_output_video_frame;
SwsContext* m_output_scale_context;
AVStream* m_output_video_stream;
AVCodec* m_output_audio_codec;
AVStream* m_output_audio_stream;
AVCodecContext* m_output_audio_codec_context;
};
#endif
And, the hellish .cpp:
#include <stdio.h>
#include <stdarg.h>
#include <stdlib.h>
#include <string.h>
#include <inttypes.h>
#include "media_container_manager.h"
MediaContainerMgr::MediaContainerMgr(const std::string& infile, const std::string& vert, const std::string& frag,
const glm::vec3* extents) :
m_video_stream_index(-1),
m_audio_stream_index(-1),
m_recording(false),
m_output_format(nullptr),
m_output_format_context(nullptr),
m_output_video_codec(nullptr),
m_output_video_codec_context(nullptr),
m_output_video_frame(nullptr),
m_output_scale_context(nullptr),
m_output_video_stream(nullptr)
{
// AVFormatContext holds header info from the format specified in the container:
m_format_context = avformat_alloc_context();
if (!m_format_context) {
throw "ERROR could not allocate memory for Format Context";
}
// open the file and read its header. Codecs are not opened here.
if (avformat_open_input(&m_format_context, infile.c_str(), NULL, NULL) != 0) {
throw "ERROR could not open input file for reading";
}
printf("format %s, duration %lldus, bit_rate %lld\n", m_format_context->iformat->name, m_format_context->duration, m_format_context->bit_rate);
//read avPackets (?) from the avFormat (?) to get stream info. This populates format_context->streams.
if (avformat_find_stream_info(m_format_context, NULL) < 0) {
throw "ERROR could not get stream info";
}
for (unsigned int i = 0; i < m_format_context->nb_streams; i++) {
AVCodecParameters* local_codec_parameters = NULL;
local_codec_parameters = m_format_context->streams[i]->codecpar;
printf("AVStream->time base before open coded %d/%d\n", m_format_context->streams[i]->time_base.num, m_format_context->streams[i]->time_base.den);
printf("AVStream->r_frame_rate before open coded %d/%d\n", m_format_context->streams[i]->r_frame_rate.num, m_format_context->streams[i]->r_frame_rate.den);
printf("AVStream->start_time %" PRId64 "\n", m_format_context->streams[i]->start_time);
printf("AVStream->duration %" PRId64 "\n", m_format_context->streams[i]->duration);
printf("duration(s): %lf\n", (float)m_format_context->streams[i]->duration / m_format_context->streams[i]->time_base.den * m_format_context->streams[i]->time_base.num);
AVCodec* local_codec = NULL;
local_codec = avcodec_find_decoder(local_codec_parameters->codec_id);
if (local_codec == NULL) {
throw "ERROR unsupported codec!";
}
if (local_codec_parameters->codec_type == AVMEDIA_TYPE_VIDEO) {
if (m_video_stream_index == -1) {
m_video_stream_index = i;
m_video_codec = local_codec;
m_video_codec_parameters = local_codec_parameters;
}
m_height = local_codec_parameters->height;
m_width = local_codec_parameters->width;
printf("Video Codec: resolution %dx%d\n", m_width, m_height);
}
else if (local_codec_parameters->codec_type == AVMEDIA_TYPE_AUDIO) {
if (m_audio_stream_index == -1) {
m_audio_stream_index = i;
m_audio_codec = local_codec;
m_audio_codec_parameters = local_codec_parameters;
}
printf("Audio Codec: %d channels, sample rate %d\n", local_codec_parameters->channels, local_codec_parameters->sample_rate);
}
printf("\tCodec %s ID %d bit_rate %lld\n", local_codec->name, local_codec->id, local_codec_parameters->bit_rate);
}
m_codec_context = avcodec_alloc_context3(m_video_codec);
if (!m_codec_context) {
throw "ERROR failed to allocate memory for AVCodecContext";
}
if (avcodec_parameters_to_context(m_codec_context, m_video_codec_parameters) < 0) {
throw "ERROR failed to copy codec params to codec context";
}
if (avcodec_open2(m_codec_context, m_video_codec, NULL) < 0) {
throw "ERROR avcodec_open2 failed to open codec";
}
m_frame = av_frame_alloc();
if (!m_frame) {
throw "ERROR failed to allocate AVFrame memory";
}
m_packet = av_packet_alloc();
if (!m_packet) {
throw "ERROR failed to allocate AVPacket memory";
}
}
MediaContainerMgr::~MediaContainerMgr() {
avformat_close_input(&m_format_context);
av_packet_free(&m_packet);
av_frame_free(&m_frame);
avcodec_free_context(&m_codec_context);
glDeleteVertexArrays(1, &m_VAO);
glDeleteBuffers(1, &m_VBO);
}
bool MediaContainerMgr::advance_frame() {
while (true) {
if (av_read_frame(m_format_context, m_packet) < 0) {
// Do we actually need to unref the packet if it failed?
av_packet_unref(m_packet);
continue;
//return false;
}
else {
if (m_packet->stream_index == m_video_stream_index) {
//printf("AVPacket->pts %" PRId64 "\n", m_packet->pts);
int response = decode_packet();
av_packet_unref(m_packet);
if (response != 0) {
continue;
//return false;
}
return true;
}
else {
printf("m_packet->stream_index: %d\n", m_packet->stream_index);
printf(" m_packet->pts: %lld\n", m_packet->pts);
printf(" mpacket->size: %d\n", m_packet->size);
if (m_recording) {
int err = 0;
//err = avcodec_send_packet(m_output_video_codec_context, m_packet);
printf(" encoding error: %d\n", err);
}
}
}
// We're done with the packet (it's been unpacked to a frame), so deallocate & reset to defaults:
/*
if (m_frame == NULL)
return false;
if (m_frame->data[0] == NULL || m_frame->data[1] == NULL || m_frame->data[2] == NULL) {
printf("WARNING: null frame data");
continue;
}
*/
}
}
int MediaContainerMgr::decode_packet() {
// Supply raw packet data as input to a decoder
// https://ffmpeg.org/doxygen/trunk/group__lavc__decoding.html#ga58bc4bf1e0ac59e27362597e467efff3
int response = avcodec_send_packet(m_codec_context, m_packet);
if (response < 0) {
char buf[256];
av_strerror(response, buf, 256);
printf("Error while receiving a frame from the decoder: %s\n", buf);
return response;
}
// Return decoded output data (into a frame) from a decoder
// https://ffmpeg.org/doxygen/trunk/group__lavc__decoding.html#ga11e6542c4e66d3028668788a1a74217c
response = avcodec_receive_frame(m_codec_context, m_frame);
if (response == AVERROR(EAGAIN) || response == AVERROR_EOF) {
return response;
} else if (response < 0) {
char buf[256];
av_strerror(response, buf, 256);
printf("Error while receiving a frame from the decoder: %s\n", buf);
return response;
} else {
printf(
"Frame %d (type=%c, size=%d bytes) pts %lld key_frame %d [DTS %d]\n",
m_codec_context->frame_number,
av_get_picture_type_char(m_frame->pict_type),
m_frame->pkt_size,
m_frame->pts,
m_frame->key_frame,
m_frame->coded_picture_number
);
}
return 0;
}
bool MediaContainerMgr::init_video_output(const std::string& video_file_name, unsigned int width, unsigned int height) {
if (m_recording)
return true;
m_recording = true;
advance_to(0L); // I've deleted the implmentation. Just seeks to beginning of vid. Works fine.
if (!(m_output_format = av_guess_format(nullptr, video_file_name.c_str(), nullptr))) {
printf("Cannot guess output format.\n");
return false;
}
int err = avformat_alloc_output_context2(&m_output_format_context, m_output_format, nullptr, video_file_name.c_str());
if (err < 0) {
printf("Failed to allocate output context.\n");
return false;
}
//TODO(P0): Break out the video and audio inits into their own methods.
m_output_video_codec = avcodec_find_encoder(m_output_format->video_codec);
if (!m_output_video_codec) {
printf("Failed to create video codec.\n");
return false;
}
m_output_video_stream = avformat_new_stream(m_output_format_context, m_output_video_codec);
if (!m_output_video_stream) {
printf("Failed to find video format.\n");
return false;
}
m_output_video_codec_context = avcodec_alloc_context3(m_output_video_codec);
if (!m_output_video_codec_context) {
printf("Failed to create video codec context.\n");
return(false);
}
m_output_video_stream->codecpar->codec_id = m_output_format->video_codec;
m_output_video_stream->codecpar->codec_type = AVMEDIA_TYPE_VIDEO;
m_output_video_stream->codecpar->width = width;
m_output_video_stream->codecpar->height = height;
m_output_video_stream->codecpar->format = AV_PIX_FMT_YUV420P;
// Use the same bit rate as the input stream.
m_output_video_stream->codecpar->bit_rate = m_format_context->streams[m_video_stream_index]->codecpar->bit_rate;
m_output_video_stream->avg_frame_rate = m_format_context->streams[m_video_stream_index]->avg_frame_rate;
avcodec_parameters_to_context(m_output_video_codec_context, m_output_video_stream->codecpar);
m_output_video_codec_context->time_base = m_format_context->streams[m_video_stream_index]->time_base;
//TODO(P1): Set these to match the input stream?
m_output_video_codec_context->max_b_frames = 2;
m_output_video_codec_context->gop_size = 12;
m_output_video_codec_context->framerate = m_format_context->streams[m_video_stream_index]->r_frame_rate;
//m_output_codec_context->refcounted_frames = 0;
if (m_output_video_stream->codecpar->codec_id == AV_CODEC_ID_H264) {
av_opt_set(m_output_video_codec_context, "preset", "ultrafast", 0);
} else if (m_output_video_stream->codecpar->codec_id == AV_CODEC_ID_H265) {
av_opt_set(m_output_video_codec_context, "preset", "ultrafast", 0);
} else {
av_opt_set_int(m_output_video_codec_context, "lossless", 1, 0);
}
avcodec_parameters_from_context(m_output_video_stream->codecpar, m_output_video_codec_context);
m_output_audio_codec = avcodec_find_encoder(m_output_format->audio_codec);
if (!m_output_audio_codec) {
printf("Failed to create audio codec.\n");
return false;
}
I've commented out all of the audio stream init beyond this next line, because this is where
the trouble begins. Creating this output stream causes the null reference I mentioned. If I
uncomment everything below here, I still get the null deref. If I comment out this line, the
deref exception vanishes. (IOW, I commented out more and more code until I found that this
was the trigger that caused the problem.)
I assume that there's something I'm doing wrong in the rest of the commented out code, that,
when fixed, will fix the nullptr and give me a working audio stream.
m_output_audio_stream = avformat_new_stream(m_output_format_context, m_output_audio_codec);
if (!m_output_audio_stream) {
printf("Failed to find audio format.\n");
return false;
}
/*
m_output_audio_codec_context = avcodec_alloc_context3(m_output_audio_codec);
if (!m_output_audio_codec_context) {
printf("Failed to create audio codec context.\n");
return(false);
}
m_output_audio_stream->codecpar->codec_id = m_output_format->audio_codec;
m_output_audio_stream->codecpar->codec_type = AVMEDIA_TYPE_AUDIO;
m_output_audio_stream->codecpar->format = m_format_context->streams[m_audio_stream_index]->codecpar->format;
m_output_audio_stream->codecpar->bit_rate = m_format_context->streams[m_audio_stream_index]->codecpar->bit_rate;
m_output_audio_stream->avg_frame_rate = m_format_context->streams[m_audio_stream_index]->avg_frame_rate;
avcodec_parameters_to_context(m_output_audio_codec_context, m_output_audio_stream->codecpar);
m_output_audio_codec_context->time_base = m_format_context->streams[m_audio_stream_index]->time_base;
*/
//TODO(P2): Free assets that have been allocated.
err = avcodec_open2(m_output_video_codec_context, m_output_video_codec, nullptr);
if (err < 0) {
printf("Failed to open codec.\n");
return false;
}
if (!(m_output_format->flags & AVFMT_NOFILE)) {
err = avio_open(&m_output_format_context->pb, video_file_name.c_str(), AVIO_FLAG_WRITE);
if (err < 0) {
printf("Failed to open output file.");
return false;
}
}
err = avformat_write_header(m_output_format_context, NULL);
if (err < 0) {
printf("Failed to write header.\n");
return false;
}
av_dump_format(m_output_format_context, 0, video_file_name.c_str(), 1);
return true;
}
//TODO(P2): make this a member. (Thanks to https://emvlo.wordpress.com/2016/03/10/sws_scale/)
void PrepareFlipFrameJ420(AVFrame* pFrame) {
for (int i = 0; i < 4; i++) {
if (i)
pFrame->data[i] += pFrame->linesize[i] * ((pFrame->height >> 1) - 1);
else
pFrame->data[i] += pFrame->linesize[i] * (pFrame->height - 1);
pFrame->linesize[i] = -pFrame->linesize[i];
}
}
This is where we take an altered frame and write it to the output container. This works fine
as long as we haven't set up an audio stream in the output container.
bool MediaContainerMgr::output_video_frame(uint8_t* buf) {
int err;
if (!m_output_video_frame) {
m_output_video_frame = av_frame_alloc();
m_output_video_frame->format = AV_PIX_FMT_YUV420P;
m_output_video_frame->width = m_output_video_codec_context->width;
m_output_video_frame->height = m_output_video_codec_context->height;
err = av_frame_get_buffer(m_output_video_frame, 32);
if (err < 0) {
printf("Failed to allocate output frame.\n");
return false;
}
}
if (!m_output_scale_context) {
m_output_scale_context = sws_getContext(m_output_video_codec_context->width, m_output_video_codec_context->height,
AV_PIX_FMT_RGB24,
m_output_video_codec_context->width, m_output_video_codec_context->height,
AV_PIX_FMT_YUV420P, SWS_BICUBIC, nullptr, nullptr, nullptr);
}
int inLinesize[1] = { 3 * m_output_video_codec_context->width };
sws_scale(m_output_scale_context, (const uint8_t* const*)&buf, inLinesize, 0, m_output_video_codec_context->height,
m_output_video_frame->data, m_output_video_frame->linesize);
PrepareFlipFrameJ420(m_output_video_frame);
//TODO(P0): Switch m_frame to be m_input_video_frame so I don't end up using the presentation timestamp from
// an audio frame if I threadify the frame reading.
m_output_video_frame->pts = m_frame->pts;
printf("Output PTS: %d, time_base: %d/%d\n", m_output_video_frame->pts,
m_output_video_codec_context->time_base.num, m_output_video_codec_context->time_base.den);
err = avcodec_send_frame(m_output_video_codec_context, m_output_video_frame);
if (err < 0) {
printf(" ERROR sending new video frame output: ");
switch (err) {
case AVERROR(EAGAIN):
printf("AVERROR(EAGAIN): %d\n", err);
break;
case AVERROR_EOF:
printf("AVERROR_EOF: %d\n", err);
break;
case AVERROR(EINVAL):
printf("AVERROR(EINVAL): %d\n", err);
break;
case AVERROR(ENOMEM):
printf("AVERROR(ENOMEM): %d\n", err);
break;
}
return false;
}
AVPacket pkt;
av_init_packet(&pkt);
pkt.data = nullptr;
pkt.size = 0;
pkt.flags |= AV_PKT_FLAG_KEY;
int ret = 0;
if ((ret = avcodec_receive_packet(m_output_video_codec_context, &pkt)) == 0) {
static int counter = 0;
printf("pkt.key: 0x%08x, pkt.size: %d, counter:\n", pkt.flags & AV_PKT_FLAG_KEY, pkt.size, counter++);
uint8_t* size = ((uint8_t*)pkt.data);
printf("sizes: %d %d %d %d %d %d %d %d %d\n", size[0], size[1], size[2], size[2], size[3], size[4], size[5], size[6], size[7]);
av_interleaved_write_frame(m_output_format_context, &pkt);
}
printf("push: %d\n", ret);
av_packet_unref(&pkt);
return true;
}
bool MediaContainerMgr::finalize_output() {
if (!m_recording)
return true;
AVPacket pkt;
av_init_packet(&pkt);
pkt.data = nullptr;
pkt.size = 0;
for (;;) {
avcodec_send_frame(m_output_video_codec_context, nullptr);
if (avcodec_receive_packet(m_output_video_codec_context, &pkt) == 0) {
av_interleaved_write_frame(m_output_format_context, &pkt);
printf("final push:\n");
} else {
break;
}
}
av_packet_unref(&pkt);
av_write_trailer(m_output_format_context);
if (!(m_output_format->flags & AVFMT_NOFILE)) {
int err = avio_close(m_output_format_context->pb);
if (err < 0) {
printf("Failed to close file. err: %d\n", err);
return false;
}
}
return true;
}
EDIT
The call stack on the crash (which I should have included in the original question):
avformat-58.dll!compute_muxer_pkt_fields(AVFormatContext * s, AVStream * st, AVPacket * pkt) Line 630 C
avformat-58.dll!write_packet_common(AVFormatContext * s, AVStream * st, AVPacket * pkt, int interleaved) Line 1122 C
avformat-58.dll!write_packets_common(AVFormatContext * s, AVPacket * pkt, int interleaved) Line 1186 C
avformat-58.dll!av_interleaved_write_frame(AVFormatContext * s, AVPacket * pkt) Line 1241 C
CamBot.exe!MediaContainerMgr::output_video_frame(unsigned char * buf) Line 553 C++
CamBot.exe!main() Line 240 C++
If I move the call to avformat_write_header so it's immediately before the audio stream initialization, I still get a crash, but in a different place. The crash happens on line 6459 of movenc.c, where we have:
/* Non-seekable output is ok if using fragmentation. If ism_lookahead
* is enabled, we don't support non-seekable output at all. */
if (!(s->pb->seekable & AVIO_SEEKABLE_NORMAL) && // CRASH IS HERE
(!(mov->flags & FF_MOV_FLAG_FRAGMENT) || mov->ism_lookahead)) {
av_log(s, AV_LOG_ERROR, "muxer does not support non seekable output\n");
return AVERROR(EINVAL);
}
The exception is a nullptr exception, where s->pb is NULL. The call stack is:
avformat-58.dll!mov_init(AVFormatContext * s) Line 6459 C
avformat-58.dll!init_muxer(AVFormatContext * s, AVDictionary * * options) Line 407 C
[Inline Frame] avformat-58.dll!avformat_init_output(AVFormatContext *) Line 489 C
avformat-58.dll!avformat_write_header(AVFormatContext * s, AVDictionary * * options) Line 512 C
CamBot.exe!MediaContainerMgr::init_video_output(const std::string & video_file_name, unsigned int width, unsigned int height) Line 424 C++
CamBot.exe!main() Line 183 C++
Please note that you should always try to provide a self-contained minimal working example to make it easier for others to help. With the actual code, the matching FFmpeg version, and an input video that triggers the segmentation fault (to be sure), the issue would be a matter of analyzing the control flow to identify why st->internal->priv_pts was not allocated. Without the full scenario, I have to report to making assumptions that may or may not correspond to your actual code.
Based on your description, I attempted to reproduce the issue by cloning https://github.com/FFmpeg/FFmpeg.git and creating a new branch from commit b52e0d95 (November 4, 2020) to approximate your FFmpeg version.
I recreated your scenario using the provided code snippets by
including the avformat_new_stream() call for the audio stream
keeping the remaining audio initialization commented out
including the original avformat_write_header() call site (unchanged order)
With that scenario, the video write with MP4 video/audio input fails in avformat_write_header():
[mp4 # 0x2b39f40] sample rate not set 0
The call stack of the error location:
#0 0x00007ffff75253d7 in raise () from /lib64/libc.so.6
#1 0x00007ffff7526ac8 in abort () from /lib64/libc.so.6
#2 0x000000000094feca in init_muxer (s=0x2b39f40, options=0x0) at libavformat/mux.c:309
#3 0x00000000009508f4 in avformat_init_output (s=0x2b39f40, options=0x0) at libavformat/mux.c:490
#4 0x0000000000950a10 in avformat_write_header (s=0x2b39f40, options=0x0) at libavformat/mux.c:514
[...]
In init_muxer(), the sample rate in the stream parameters is checked unconditionally:
case AVMEDIA_TYPE_AUDIO:
if (par->sample_rate <= 0) {
av_log(s, AV_LOG_ERROR, "sample rate not set %d\n", par->sample_rate); abort();
ret = AVERROR(EINVAL);
goto fail;
}
That condition has been in effect since 2014-06-18 at the very least (didn't go back any further) and still exists. With a version from November 2020, the check must be active and the parameter must be set accordingly.
If I uncomment the remaining audio initialization, the situation remains unchanged (as expected). So, satisfy the condition, I added the missing parameter as follows:
m_output_audio_stream->codecpar->sample_rate =
m_format_context->streams[m_audio_stream_index]->codecpar->sample_rate;
With that, the check succeeds, avformat_write_header() succeeds, and the actual video write succeeds.
As you indicated in your question, the segmentation fault is caused by st->internal->priv_pts being NULL at this location:
#0 0x00000000009516db in compute_muxer_pkt_fields (s=0x2b39f40, st=0x2b3a580, pkt=0x7fffffffe2d0) at libavformat/mux.c:632
#1 0x0000000000953128 in write_packet_common (s=0x2b39f40, st=0x2b3a580, pkt=0x7fffffffe2d0, interleaved=1) at libavformat/mux.c:1125
#2 0x0000000000953473 in write_packets_common (s=0x2b39f40, pkt=0x7fffffffe2d0, interleaved=1) at libavformat/mux.c:1188
#3 0x0000000000953634 in av_interleaved_write_frame (s=0x2b39f40, pkt=0x7fffffffe2d0) at libavformat/mux.c:1243
[...]
In the FFmpeg code base, the allocation of priv_pts is handled by init_pts() for all streams referenced by the context. init_pts() has two call sites:
libavformat/mux.c:496:
if (s->oformat->init && ret) {
if ((ret = init_pts(s)) < 0)
return ret;
return AVSTREAM_INIT_IN_INIT_OUTPUT;
}
libavformat/mux.c:530:
if (!s->internal->streams_initialized) {
if ((ret = init_pts(s)) < 0)
goto fail;
}
In both cases, the calls are triggered by avformat_write_header() (indirectly via avformat_init_output() for the first, directly for the second). According to control flow analysis, there's no success case that would leave priv_pts unallocated.
Considering a high probability that our versions of FFmpeg are compatible in terms of behavior, I have to assume that 1) the sample rate must be provided for audio streams and 1) priv_pts is always allocated by avformat_write_header() in the absence of errors. Therefore, two possible root causes come to mind:
Your stream is not an audio stream (unlikely; the type is based on the codec, which in turn is based on the output file extension - assuming mp4)
You do not call avformat_write_header() (unlikely) or do not handle the error in the caller of your C++ member function (the return value of avformat_write_header() is checked but I do not have code corresponding to the caller of the C++ member function; your actual code might differ significantly from the code provided, so it's possible and the only plausible conclusion that can be drawn from available data)
The solution: Ensure that processing does not continue if avformat_write_header() fails. By adding the audio stream, avformat_write_header() starts to fail unless you set the stream sample rate. If the error is ignored, av_interleaved_write_frame() triggers a segmentation fault by accessing the unallocated st->internal->priv_pts.
As mentioned initially, scenario is incomplete. If you do call avformat_write_header() and stop processing in case of an error (meaning you do not call av_interleaved_write_frame()), more information is needed. As it stands now, that is unlikely. For further analysis, the executable output (stdout, stderr) is required to see your traces and FFmpeg log messages. If that does not reveal new information, a self-contained minimal working example and the video input are needed to get all the full picture.

Repeating ffmpeg stream (libavcodec/libavformat)

I am using the various APIs from ffmpeg to draw videos in my application. So far this works very well. Since I also have gifs I want to do looping without having to load the file over and over again.
In my code the decoder loop looks like this:
AVPacket packet = {};
av_init_packet(&packet);
while (mIsRunning) {
int error = av_read_frame(mContext, &packet);
if (error == AVERROR_EOF) {
if(mRepeat) {
logger.info("EOF-repeat");
auto stream = mContext->streams[mVideoStream];
av_seek_frame(mContext, mVideoStream, 0, 0);
continue;
}
if (mReadVideo) {
avcodec_send_packet(mVideoCodec, nullptr);
}
if (mReadAudio) {
avcodec_send_packet(mAudioCodec, nullptr);
}
break;
}
if (error < 0) {
char err[AV_ERROR_MAX_STRING_SIZE];
av_make_error_string(err, AV_ERROR_MAX_STRING_SIZE, error);
logger.error("Failed to read next frame from stream: ", err);
throw std::runtime_error("Stream reading failed");
}
if (packet.stream_index == mVideoStream && mReadVideo) {
int32 err;
{
std::lock_guard<std::mutex> l(mVideoLock);
err = avcodec_send_packet(mVideoCodec, &packet);
}
mImageEvent.notify_all();
while (err == AVERROR(EAGAIN) && mIsRunning) {
{
std::unique_lock<std::mutex> l(mReaderLock);
mReaderEvent.wait(l);
}
{
std::lock_guard<std::mutex> l(mVideoLock);
err = avcodec_send_packet(mVideoCodec, &packet);
}
}
}
av_packet_unref(&packet);
}
Reading a video to the end works perfectly well and if I dont set mRepeat to true it properly EOFs and stops parsing. However when I use looping the following happens:
The video ends
AVERROR_EOF happens at av_read_frame
EOF-repeat is printed
A random frame is read from the stream (and rendered)
AVERROR_EOF happens at av_read_frame
EOF-repeat is printed
A random frame is read from the stream (and rendered)
...
You can imagine it like I have a gif of a spinning globe and after one full turn it just starts randomly jumping around, sometimes for a fraction of a second correctly, sometimes backwards and sometimes just randomly everywhere.
I have also tried several versions with avformat_seek_file what other way would there be to reset everything to the beginning and start from scratch again?
I figured out that I also need to reset my IO context to the beginning:
if(mRepeat) {
auto stream = mContext->streams[mVideoStream];
avio_seek(mContext->pb, 0, SEEK_SET);
avformat_seek_file(mContext, mVideoStream, 0, 0, stream->duration, 0);
continue;
}
Now the video properly loops forever :)

sending data to bulk out pipe for prolific 2305 always immediately stalls

i'm trying to connect an old printer with Centronics interface to my Mac running OS X via a USB-to-IEEE1284 adapter containing a Prolific 2305 IC. I closely follow the example at Apple:USBDevInterfaces using IOKit and CF "classes". I find the device, successfully select the only configuration, find the only interface, get printer status byte, open Bulk Out Pipe #1 all successfully, but when i write a single byte to the bulk out pipe it always returns with Pipe stalled error. (I actually have 2 printers and 2 interfaces for testing, the printers behave identically but i excluded one interface from testing because it even stalled on reading the printer status byte.)
Q: Is there something more required to setup for a printing device than shown in the Apple example, e.g. for negotiating the port mode on the centronics interface?
is there a better way for doing this? Unluckily there is no /dev/lp* created for the parallel port.
Here are the most relevant parts from my source. In case more code is needed please ask or see UsbDevice.cpp (which contains some more superfluous stuff for my testing and understanding and the procedures below in reverse order.)
Thanks for help.
void testProlific2305()
{
// get device(s)
CFMutableDictionaryRef dict = newMatchingDictForService(service_USBDevice);
add(dict,key_VendorID,_prolific);
add(dict,key_ProductID,_ieee1284);
io_iterator_t iter = newIteratorForMatchingServices(dict);
// loop over all devices:
for(io_service_t device; (device = IOIteratorNext(iter)); IOObjectRelease(device))
{
cstr r = testProlific2305device(device);
if(r) LogLine("%s",r);
}
IOObjectRelease(iter);
}
cstr testProlific2305device(io_service_t device)
{
// note: general error codes in IOReturn.h
// usb error codes in USB.h line 360ff
IOReturn err;
// configure device to use configuration[0]:
MyIOUSBDeviceInterface** dev_if = newUSBDeviceInterfaceForDevice(device);
if(!dev_if) return "dev_if = NULL";
err = (*dev_if)->USBDeviceOpen(dev_if);
if(err) return usingstr("USBDeviceOpen: error 0x%08X",(uint)err);
err = ConfigureDevice(dev_if, 0/*configuration_idx*/);
if(err) return usingstr("ConfigureDevice: error 0x%08X",(uint)err);
// get interface(s):
IOUSBFindInterfaceRequest request;
request.bInterfaceClass = kIOUSBFindInterfaceDontCare;
request.bInterfaceSubClass = kIOUSBFindInterfaceDontCare;
request.bInterfaceProtocol = kIOUSBFindInterfaceDontCare;
request.bAlternateSetting = kIOUSBFindInterfaceDontCare;
io_iterator_t iterator;
err = (*dev_if)->CreateInterfaceIterator(dev_if, &request, &iterator);
// loop over all interfaces:
for(io_service_t interface; (interface = IOIteratorNext(iterator)); IOObjectRelease(interface))
{
MyIOUSBInterfaceInterface** if_if = newUSBDeviceInterfaceForInterface(interface);
if(!if_if) return "if_if = NULL";
cstr r = testProlific2305interface(dev_if,if_if);
if(r) LogLine("%s",r);
}
err = (*dev_if)->USBDeviceClose(dev_if);
if(err) return usingstr("USBDeviceClose: error 0x%08X",(uint)err);
//CFRelease(dev_if); dev_if=NULL;
return NULL; // ok
}
cstr testProlific2305interface(MyIOUSBDeviceInterface** dev_if, MyIOUSBInterfaceInterface** if_if)
{
IOReturn err;
//Get interface class and subclass: should be 7, 1
uint8 interfaceClass, interfaceSubClass;
err = (*if_if)->GetInterfaceClass(if_if, &interfaceClass);
err = (*if_if)->GetInterfaceSubClass(if_if, &interfaceSubClass);
LogLine("Interface class %d, subclass %d\n", interfaceClass, interfaceSubClass);
// open interface:
err = (*if_if)->USBInterfaceOpen(if_if);
if(err) return usingstr("USBDeviceOpen returned error 0x%08X", (uint)err);
// Get the number of endpoints associated with this interface
uint8 interfaceNumEndpoints;
err = (*if_if)->GetNumEndpoints(if_if, &interfaceNumEndpoints);
if(err) return usingstr("GetNumEndpoints returned error 0x%08X", (uint)err);
LogLine("interfaceNumEndpoints = %u",interfaceNumEndpoints);
// Access each pipe in turn, starting with the pipe at index 1
// The pipe at index 0 is the default control pipe and should be
// accessed using (*usbDevice)->DeviceRequest() instead
for(int pipeRef = 1; pipeRef <= interfaceNumEndpoints; pipeRef++)
{
cstr r = testProlific2305pipe(dev_if,if_if,pipeRef);
if(r) LogLine("%s",r);
}
return NULL; // ok
}
cstr testProlific2305pipe(MyIOUSBDeviceInterface** dev_if, MyIOUSBInterfaceInterface** if_if, int pipe_idx)
{
uint16 maxPacketSize;
uint8 direction, number, transferType, interval, status, interface_idx;
IOReturn err;
err = (*if_if)->GetPipeProperties(if_if,
pipe_idx, &direction,
&number, &transferType,
&maxPacketSize, &interval);
if(err) return usingstr("GetPipeProperties(%d) error: 0x%08X", pipe_idx, err);
LogLine("Pipe %d: direction %s, transferType %s, maxPacketSize %d",
pipe_idx,
usb_pipe_direction_str(direction),
usp_pipe_transfertype_str(transferType),
maxPacketSize);
err = (*if_if)->GetPipeStatus(if_if, pipe_idx);
if(err) return usingstr("GetPipeStatus error: 0x%08X", (uint)err);
err = printerGetPortStatus(dev_if, interface_idx, &status);
if(err) return usingstr("printerGetPortStatus error: 0x%08X",(uint)err);
if(status!=0x18) LogLine("Printer status = 0x%02X", status); // 0x18 = no error, selected, paper ok
cstr msg = "0"; // just 1 char
err = (*if_if)->WritePipe(if_if, pipe_idx, (void*)msg, strlen(msg));
// ****--> Here i always get error: 0xE00002ed device not responding
if(err)
{
LogLine("WritePipe: error 0x%08X", (uint)err);
err = printerGetPortStatus(dev_if, 0/*interface*/, &status);
if(err) LogLine("printerGetPortStatus error: 0x%08X",(uint)err);
return "bulk write to pipe failed";
}
return NULL; // ok
}
IOReturn printerGetPortStatus(MyIOUSBDeviceInterface **dev, uint interface_idx, uint8* result)
{
IOUSBDevRequest request;
request.bmRequestType = 0b10100001;
request.bRequest = 1;
request.wValue = 0;
request.wIndex = interface_idx;
request.wLength = 1;
request.pData = result;
IOReturn err = (*dev)->DeviceRequest(dev, &request);
XXXASSERT(err || request.wLenDone==1);
return err;
}
IOReturn ConfigureDevice(MyIOUSBDeviceInterface **dev, uint configuration_idx)
{
uint8 num_configurations;
IOReturn err;
IOUSBConfigurationDescriptorPtr configuration_descriptor;
// Get the number of configurations.
err = (*dev)->GetNumberOfConfigurations(dev, &num_configurations); XXXASSERT(!err);
if(configuration_idx >= num_configurations) return error;
// Get the configuration descriptor
err = (*dev)->GetConfigurationDescriptorPtr(dev, configuration_idx, &configuration_descriptor);
if(err) { LogLine("GetConfigurationDescriptorPtr: error = 0x%08X", err); return err; }
// Set the device’s configuration. The configuration value is found in
// the bConfigurationValue field of the configuration descriptor
err = (*dev)->SetConfiguration(dev, configuration_descriptor->bConfigurationValue);
if(err) { LogLine("SetConfiguration: error = 0x%08X", err); return err; }
return ok;
}

Losing data with GetOverlappedResult?

I have a thread that constantly looks for new data and if the data is not already in the serial buffer, ReadFile and GetOverlappedResult seem to tell me there's data, and that it read it, but not transfer it into my buffer...
func read()
{
if(state == 0)
{
memset(bytes, '\0', sizeof(amount_to_read));
readreturn = ReadFile(h, bytes, amount_to_read,NULL, osReader);
if(readreturn <= 0)
{
errorcode = GetLastError();
if(errorcode != ERROR_IO_PENDING)
{
SetEAIError(ERROR_INTERNALERROR);
return -1;
}
}
}
if (GetOverlappedResult(h, osReader, &dwRead, FALSE) == false)
{
errorcode = GetLastError();
if (errorcode == ERROR_IO_INCOMPLETE || errorcode == 0)
{
if(dwRead > 0)
{
return 1;
}
//timeout
SetEAIError(ERROR_EAITIMEOUT);
return -1;
}
else
{
//other error
SetEAIError(ERROR_WIN_ERROR);
return -1;
}
}
else
{
//read succeded, check if we read the amount required
if(dwRead != amount_to_read)
{
if(dwRead == 0)
{
//nothing read, treat as timeout
SetEAIError(ERROR_EAINOREAD);
return -1;
}
else
{
//memcpy_s(bytes, sizeof(bytes), readbuf, dwRead);
SetEAIError(ERROR_PARTIALREAD);
*_bytesRead = dwRead;
return -1;
}
}
else
{
if(strlen((char*)bytes) == 0)
{
//nothing read, treat as timeout
SetEAIError(ERROR_EAINOREAD);
return -1;
}
//memcpy_s(bytes, sizeof(bytes), readbuf, dwRead);
*_bytesRead = dwRead;
return 0;
}
}
}
This is what the error codes mean:
ERROR_TIMEOUT - switches the state to 1 so that it does not read again, which calls GetOverlappedResult again
INTERNALERROR,ERROR_EAINOREAD - it resets state to 0
ERROR_PARTIALREAD - starts a new read with new amount of bytes to read
If I swtich GetOverlappedResult to blocking (pass TRUE) it works every time.
If I switch my thread to only read when I know there is data there it works every time.
But if there is not data there, when there is data there it seems to "lose" the data, it my read amount parameter dwRead shows the correct number of bytes read (can see it read with a port monitor) but the bytes are not stored in my char*.
I constantly get ERROR_EAINOREAD
What am I doing wrong?
I do not want to use flags, I want to just use ReadFile and GetOverlappedResult, I should be able to accomplish this with the code I have....... I assume
The problem was exactly what was stated the data was getting lost... the REASON it was getting lost is because the bytes parameter passed into the readfile is a local variable in the parents thread. being local it gets re initialized each cycle so after I come into the read again, skip the readfile and go to the overlappedresults, I am now potentially working with a different area of memory