I just started using FFMPEG with C++ and try to code an audio decoder then write the decoded audio into a file.
However i'm not sure about which data to write to the output file. As far as i know from looking at the sample codes it seems to be the AVFrame -> data[0].
But when i try to print it on the consoles, i get some random numbers that are different each time i run the program. And when i try to write this AVFrame->data[0] into a file i keep getting an error.
So my question is how can I write the decoded audio after I call the function av_codec_decode_audio4 ?
Below I attached my code and I pass the argument "C:\02.mp3" which is a path for a valid mp3 file on my PC.
Thank you for your help.
// TestFFMPEG.cpp : Audio Decoder
//
#include "stdafx.h"
#include <iostream>
#include <fstream>
#include <sstream>
extern "C" {
#include <avcodec.h>
#include <avformat.h>
#include <swscale.h>
}
using namespace std;
int main(int argc, char* argv[])
{
int audioStream = -1;
AVCodec *aCodec;
AVPacket avPkt;
AVFrame *decode_frame = avcodec_alloc_frame();
AVCodecContext *aCodecCtxt;
AVFormatContext *pFormatCtxt = NULL;
if(argc != 2) { // Checking whether there is enough argument
return -1;
}
av_register_all(); //Initialize CODEC
avformat_network_init();
av_init_packet (&avPkt);
if (avformat_open_input (&pFormatCtxt, argv[1],NULL,NULL)!= 0 ){ //Opening File
return -2;
}
if(avformat_find_stream_info (pFormatCtxt,NULL) < 0){ //Get Streams Info
return -3;
}
AVStream *stream = NULL;
//av_read_play (pFormatCtxt); //open streams
for (int i = 0; i < pFormatCtxt->nb_streams ; i++) { //Find Audio Stream
if (pFormatCtxt->streams[i]->codec->codec_type == AVMEDIA_TYPE_AUDIO){
audioStream =i;
}
}
aCodecCtxt = pFormatCtxt ->streams [audioStream]->codec; // opening decoder
aCodec = avcodec_find_decoder( pFormatCtxt->streams [audioStream] ->codec->codec_id);
if (!aCodec) {
return -8;
}
if (avcodec_open2(aCodecCtxt,aCodec,NULL)!=0) {
return -9;
}
int cnt = 0;
while(av_read_frame(pFormatCtxt,&avPkt) >= 0 ){
if (avPkt.stream_index == audioStream){
int check = 0;
int result = avcodec_decode_audio4 (aCodecCtxt,decode_frame,&check, &avPkt);
cout << "Decoded : "<< (int) decode_frame->data[0] <<", "<< "Check : " << check << ", Format :" << decode_frame->format <<" " << decode_frame->linesize[0]<< " "<<cnt <<endl;
}
av_free_packet(&avPkt);
cnt++;
}
return aCodec ->id;
}
You're doing it right.
The data that you decode contain at the pointer decode_frame->data[0]. Data size in bytes is decode_frame->linesize[0], the number of audio samples is decode_frame->nb_samples.
Thus, you can copy the audio data into your own buffer as follows:
memcpy(OutputBuffer, decode_frame->data[0], decode_frame->linesize[0]);
You can try using ffms2 which will directly give you decoded audio sample. It internally used ffmpeg/libav. So you don't have to worry about decoding stuff.
https://code.google.com/p/ffmpegsource/
Related
In a project I'm working on, I'd like to use stb_vorbis to stream audio from an ogg file. However, after implementing the processes to do this, I found the audio was crackly. I feel the problem may be similar to this question, but I can't see where a problem could be.
Here is my code:
#include <SDL2/SDL.h>
#include <stb/vorbis.h>
#include <iostream>
void sdlAudioCallback(void* userData, Uint8* stream, int len)
{
stb_vorbis* myVorbis = (stb_vorbis*) userData;
SDL_memset(stream, 0, len);
stb_vorbis_get_frame_short_interleaved(myVorbis, 2, (short*) stream, len/sizeof(short));
}
int main()
{
if (SDL_Init(SDL_INIT_AUDIO) != 0)
return -1;
int error = 0;
stb_vorbis_alloc* alloc;
stb_vorbis* vorbis = stb_vorbis_open_filename("res/thingy.ogg", &error, alloc);
if (error != 0)
return -2;
stb_vorbis_info info = stb_vorbis_get_info(vorbis);
SDL_AudioSpec spec;
spec.freq = info.sample_rate;
spec.format = AUDIO_S16;
spec.channels = 2;
spec.samples = 1024;
spec.callback = sdlAudioCallback;
spec.userdata = vorbis;
if (SDL_OpenAudio(&spec, NULL) < 0) {
std::cout << "failed to open audio device: " << SDL_GetError() << std::endl;
SDL_Quit();
return -3;
}
SDL_PauseAudio(0);
SDL_Delay(5000);
}
More information:
thingy.ogg is from Sean Barrett's samples
building with g++ on an OSX machine
I have a source code in C++ using libavcodec for decoding h264 rtsp stream frames. This source code was written using ffmpeg 1.1. Now when I upgraded to ffmpeg 3.3, all things seem to work correctly except that decoding frames not work.
In old version, I was using avcodec_decode_video2. After upgrading, avcodec_decode_video2 always sets got_picture to 0 and return value is equal to the size of the input packet (which means all data is used). And never a frame is decoded.
I have also removed avcodec_decode_video2 and done decoding with avcodec_send_packet and avcodec_receive_frame, but avcodec_send_packet always returns 0 and avcodec_receive_frame always returns -11 (EAGAIN).
This is the code I use for decoding:
#include "stdafx.h"
#include <stdlib.h>
#include <string>
#include <iostream>
using namespace std;
extern "C"{
#include "libavformat/avformat.h"
#include "libavcodec/avcodec.h"
#include "libswscale/swscale.h"
#include "libavutil/pixfmt.h"
}
int extraDataSize;
static const int MaxExtraDataSize = 1024;
uint8_t extraDataBuffer[MaxExtraDataSize];
void AddExtraData(uint8_t* data, int size)
{
auto newSize = extraDataSize + size;
if (newSize > MaxExtraDataSize){
throw "extradata exceeds size limit";
}
memcpy(extraDataBuffer + extraDataSize, data, size);
extraDataSize = newSize;
}
int _tmain(int argc, _TCHAR* argv[])
{
std::string strFramesPath("g:\\frames\\");
AVCodec* avCodec;
AVCodecContext* avCodecContext;
AVFrame* avFrame;
AVCodecID codecId = AV_CODEC_ID_H264;
unsigned char sprops_part_1[9] = { 0x27, 0x42, 0x80, 0x1f, 0xda, 0x02, 0xd0, 0x49, 0x10 };
unsigned char sprops_part_2[4] = { 0x28, 0xce, 0x3c, 0x80 };
av_register_all();
avcodec_register_all();
avCodec = avcodec_find_decoder(codecId);
avCodecContext = avcodec_alloc_context3(avCodec);
if (!avCodecContext)
{
cout << "avcodec_alloc_context3 failed." << endl;
return 0;
}
uint8_t startCode[] = { 0x00, 0x00, 0x01 };
// sprops
{
// sprops 1
AddExtraData(startCode, sizeof(startCode));
AddExtraData(sprops_part_1, 9);
// sprops 2
AddExtraData(startCode, sizeof(startCode));
AddExtraData(sprops_part_2, 4);
avCodecContext->extradata = extraDataBuffer;
avCodecContext->extradata_size = extraDataSize;
}
AddExtraData(startCode, sizeof(startCode));
avCodecContext->flags = 0;
if (avcodec_open2(avCodecContext, avCodec, NULL) < 0)
{
cout << "failed to open codec" << endl;
return 0;
}
avFrame = av_frame_alloc();
if (!avFrame)
{
cout << "failed to alloc frame" << endl;
return 0;
}
void *buffer = malloc(100 * 1024); // 100 KB buffer - all frames fit in this buffer
for (int nFrameIndex = 0; nFrameIndex < 257; nFrameIndex++)
{
std::string strFilename = std::string("g:\\frames\\" + std::to_string(nFrameIndex));
FILE* f = fopen(strFilename.c_str(), "rb");
fseek(f, 0, SEEK_END);
long nFileSize = ftell(f);
fseek(f, 0, SEEK_SET);
size_t nReadSize = fread(buffer, 1, nFileSize, f);
// cout << strFilename << endl;
if (nReadSize != nFileSize)
{
cout << "Error reading file data" << endl;
continue;
}
AVPacket avpkt;
avpkt.data = (uint8_t*)buffer;
avpkt.size = nReadSize;
while (avpkt.size > 0)
{
int got_frame = 0;
auto len = avcodec_decode_video2(avCodecContext, avFrame, &got_frame, &avpkt);
if (len < 0) {
//TODO: log error
cout << "Error decoding - error code: " << len << endl;
break;
}
if (got_frame)
{
cout << "* Got 1 Decoded Frame" << endl;
}
avpkt.size -= len;
avpkt.data += len;
}
}
getchar();
return 0;
}
Test frames data can be downloaded from this link:
Frames.zip (~3.7MB)
I have used windows builds from Builds - Zeranoe FFmpeg
If you copy paste this code into your IDE, the code compiles successfully. Using libavcodec new versions, no frame is decoded. Using old version of libavcodec (20141216-git-92a596f), decoding starts when feed frame 2.
Any ideas?
FFmpeg/libAV description is quite complex and probably not the best one in terms of understandability (if not to say it's bad). You should read it carefully.
However, what you are missing is to init the AVPacket by using av_init_packet. Or use av_packet_alloc() to create the packet which will initialize the packet (and is better for duplicating it later for encoding).
// code ...
AVPacket avpkt;
av_init_packet(&avpkt); // initialize packet
avpkt.data = (uint8_t*)buffer;
avpkt.size = nReadSize;
// code ...
or
// code ...
AVPacket* avpkt = av_packet_alloc();
avpkt->data = (uint8_t*)buffer;
avpkt->size = nReadSize;
// use ...
auto len = avcodec_decode_video2(avCodecContext, avFrame, &got_frame, avpkt);
Using av_packet_alloc() is also more consistent within the code since you also use AVFrame as a pointer.
Remember to free the packets/frames/pictures as well, otherwise you will have memory leaks.
Also here is a (more or less) good/"up-to-date" tutorial for ffmpeg.
I am a beginner programmer trying to inflate text stream from pdfs. I have adopted and slightly altered some open source code which uses zlib, and generally it works very well. However, I have been testing on some different pdfs lately and some of the inflated streams are returning blank. Could anybody advise me as to why?
I have come across this question below which seems to address the same problem but does not really give a definitive answer
zLib inflate has empty result in some cases
#include <iostream>
#include <fstream>
#include <string>
#include "zlib.h"
int main()
{
//Discard existing output:
//Open the PDF source file:
std::ifstream filei("C:\\Users\\dpbowe\\Desktop\\PIDSearch\\P&ID.PDF", std::ios::in|std::ios::binary|std::ios::ate);
if (!filei) std::cout << "Error Opening Input File" << std::endl;
//decoded output
std::ofstream fileo;
fileo.open("C:\\Users\\dpbowe\\Desktop\\Decoded.txt", std::ios::binary | std::ofstream::out);
if (!fileother) std::cout << "Error opening output file" << std::endl;
if (filei && fileo)
{
//Get the file length:
long filelen = filei.tellg(); //fseek==0 if ok
filei.seekg(0, std::ios::beg);
//Read the entire file into memory (!):
char* buffer = new char [filelen];
if (buffer == NULL) {fputs("Memory error", stderr); exit(EXIT_FAILURE);}
filei.read(buffer,filelen);
if (buffer == '\0') {fputs("Reading error", stderr); exit(EXIT_FAILURE);}
bool morestreams = true;
//Now search the buffer repeated for streams of data
while (morestreams)
{
//Search for stream, endstream. Should check the filter of the object to make sure it if FlateDecode, but skip that for now!
size_t streamstart = FindStringInBuffer (buffer, "stream", filelen); //This is my own search function
size_t streamend = FindStringInBuffer (buffer, "endstream", filelen); //This is my own search function
if (streamstart>0 && streamend>streamstart)
{
//Skip to beginning and end of the data stream:
streamstart += 6;
if (buffer[streamstart]==0x0d && buffer[streamstart+1]==0x0a) streamstart+=2;
else if (buffer[streamstart]==0x0a) streamstart++;
if (buffer[streamend-2]==0x0d && buffer[streamend-1]==0x0a) streamend-=2;
else if (buffer[streamend-1]==0x0a) streamend--;
//Assume output will fit into 10 times input buffer:
size_t outsize = (streamend - streamstart)*10;
char* output = new char [outsize]; ZeroMemory(output, outsize);
//Now use zlib to inflate:
z_stream zstrm; ZeroMemory(&zstrm, sizeof(zstrm));
zstrm.avail_in = streamend - streamstart + 1;
zstrm.avail_out = outsize;
zstrm.next_in = (Bytef*)(buffer + streamstart);
zstrm.next_out = (Bytef*)output;
int rsti = inflateInit(&zstrm);
if (rsti == Z_OK)
{
int rst2 = inflate (&zstrm, Z_FINISH);
if (rst2 >= 0)
{
size_t totout = zstrm.total_out;
//Write inflated output to file "Decoded.txt"
fileother<<output;
fileother<<"\r\nStream End\r\n\r\n";
}
else std::cout<<"output uncompressed stream is blank"<<std::endl;
}
delete[] output; output=0;
buffer+= streamend + 7;
filelen = filelen - (streamend+7);
}
else
{
morestreams = false;
std::cout<<"End of File"<<std::endl;
}
}
filei.close();
}
else
{
std::cout << "File Could Not Be Accessed\n";
}
if (fileo) fileo.close();
}
Here is my link format to HTTP stream(user, password and address was changed to dummy):
http://username:password#192.168.0.104:8093/axis-cgi/mjpg/video.cgi
This stream works perfectly in VLC. However, I can't open it using OpenCV library.
Here is my code:
#include <iostream>
#include "opencv2/opencv.hpp"
using namespace std;
using namespace cv;
int main()
{
VideoCapture cap;
const string videoStreamAddress = "http://username:password#192.168.0.104:8093/axis-cgi/mjpg/video.cgi";
cap.open(videoStreamAddress);
if (!cap.isOpened())
{
cout << endl << "Videostream not found !" << endl;
system("pause");
return 0;
}
Mat frame;
while(1)
{
cap >> frame;
if (frame.empty())
break;
imshow("IPcamera", frame);
int c = waitKey(1);
if (c == 27)
{
break;
}
}
waitKey(0);
return 0;
}
This gives me an error:
warning: Error opening file (../../modules/highgui/src/cap_ffmpeg_impl.hpp:529)
which points to:
bool CvCapture_FFMPEG::open( const char* _filename )
{
unsigned i;
bool valid = false;
close();
#if LIBAVFORMAT_BUILD >= CALC_FFMPEG_VERSION(52, 111, 0)
int err = avformat_open_input(&ic, _filename, NULL, NULL);
#else
int err = av_open_input_file(&ic, _filename, NULL, 0, NULL);
#endif
if (err < 0)
{
CV_WARN("Error opening file");
goto exit_func;
}
...
What could be a problem?
Did you try opening a video file in your machine using Videocapture? (Just add the path to the video file to the place where you've put the URL) I assume that it fails in the same way.
So this is a problem with ffmpeg. You'll need to build OpenCV yourself with ffmpeg support. (Do some search on gstreamer as well. I'm not much familiar with that).
Also you can try using another software like ManyCam in the middle. It enables you to read the stream easily in the same way you are reading from a webcam.
I have been porting some video processing code to C++ using OpenCV 2.4.3. The following test program closely mimics how my code will read each frame from a video, operate on its contents, and then write new frames to a new video file.
Strangely, the output frames are entirely black when the pixels are set individually, but are written correctly when the entire frame is cloned.
In practice, I'd use the two macros to access and assign desired values, but the sequential scan used in the example shows the idea more clearly.
Does anyone know where I'm going wrong?
test.cpp:
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <iostream>
#include <string>
using namespace std;
using namespace cv;
#define RGB_REF(PR,NC,R,C,CH) (*((PR) + ((3*(NC)*(R)+(C))+(CH))))
#define GRAY_REF(PR,NC,R,C) (*((PR) + (NC)*(R)+(C)))
int main(int argc, char* argv[])
{
string video_path(argv[1]);
cerr << "Video path is " + video_path + "\n";
VideoCapture capture(video_path);
if ( !capture.isOpened() )
{
cerr << "Input file could not be opened\n";
return 1;
} else
{
string output_path(argv[2]);
VideoWriter output;
int ex = (int)capture.get(CV_CAP_PROP_FOURCC);
Size S = Size((int) capture.get(CV_CAP_PROP_FRAME_WIDTH),
(int) capture.get(CV_CAP_PROP_FRAME_HEIGHT));
output.open(output_path,ex,capture.get(CV_CAP_PROP_FPS),S,true);
if ( !output.isOpened() )
{
cerr << "Output file could not be opened\n";
return 1;
}
unsigned int numFrames = (unsigned int) capture.get(CV_CAP_PROP_FRAME_COUNT);
unsigned int m = (unsigned int) capture.get(CV_CAP_PROP_FRAME_HEIGHT);
unsigned int n = (unsigned int) capture.get(CV_CAP_PROP_FRAME_WIDTH);
unsigned char* im = (unsigned char*) malloc(m*n*3*sizeof(unsigned char));
unsigned char* bw = (unsigned char*) malloc(m*n*3*sizeof(unsigned char));
Mat frame(m,n,CV_8UC3,im);
Mat outputFrame(m,n,CV_8UC3,bw);
for (size_t i=0; i<numFrames; i++)
{
capture >> frame;
for (size_t x=0;x<(3*m*n);x++)
{
bw[x] = im[x];
}
output << outputFrame; // blank frames
// output << frame; // works
// output << (outputFrame = frame); // works
}
}
}
When you query a frame from VideoCapture as capture >> frame;, frame is modified. Say, it has a new data buffer. So im no longer points to the buffer of frame.
Try
bm[x] = frame.ptr()[x];