I'm trying to convert a cv::Mat to an AVFrame to encode it then in H.264 and wanted to start from a simple example, as I'm a newbie in both. So I first read in a JPEG file, and then do the pixel format conversion with sws_scale() from AV_PIX_FMT_BGR24 to AV_PIX_FMT_YUV420P keeping the dimensions the same, and it all goes fine until I call avcodec_encode_video2().
I read quite a few discussions regarding an AVFrame allocation and the question Segmentation fault while avcodec_encode_video2 seemed like a match but I just can't see what I'm missing or getting wrong.
Here is the minimal code that you can reproduce the crash, it should be compiled with,
g++ -o OpenCV2FFmpeg OpenCV2FFmpeg.cpp -lopencv_imgproc -lopencv_highgui -lopencv_core -lswscale -lavutil -lavcodec -lavformat
It's output on my system,
cv::Mat [width=420, height=315, depth=0, channels=3, step=1260]
I'll soon crash..
Segmentation fault
And that sample.jpg file's details by identify tool,
~temporary/sample.jpg JPEG 420x315 420x315+0+0 8-bit sRGB 38.3KB 0.000u 0:00.000
Please note that I'm trying to create a video out of a single image, just to keep things simple.
#include <iostream>
#include <cassert>
using namespace std;
extern "C" {
#include <libavcodec/avcodec.h>
#include <libswscale/swscale.h>
#include <libavformat/avformat.h>
}
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
const string TEST_IMAGE = "/home/baris/temporary/sample.jpg";
int main(int /*argc*/, char** argv)
{
av_register_all();
avcodec_register_all();
/**
* Initialise the encoder
*/
AVCodec *h264encoder = avcodec_find_encoder(AV_CODEC_ID_H264);
AVFormatContext *cv2avFormatContext = avformat_alloc_context();
/**
* Create a stream and allocate frames
*/
AVStream *h264outputstream = avformat_new_stream(cv2avFormatContext, h264encoder);
avcodec_get_context_defaults3(h264outputstream->codec, h264encoder);
AVFrame *sourceAvFrame = av_frame_alloc(), *destAvFrame = av_frame_alloc();
int got_frame;
/**
* Pixel formats for the input and the output
*/
AVPixelFormat sourcePixelFormat = AV_PIX_FMT_BGR24;
AVPixelFormat destPixelFormat = AV_PIX_FMT_YUV420P;
/**
* Create cv::Mat
*/
cv::Mat cvFrame = cv::imread(TEST_IMAGE, CV_LOAD_IMAGE_COLOR);
int width = cvFrame.size().width, height = cvFrame.size().height;
cerr << "cv::Mat [width=" << width << ", height=" << height << ", depth=" << cvFrame.depth() << ", channels=" << cvFrame.channels() << ", step=" << cvFrame.step << "]" << endl;
h264outputstream->codec->pix_fmt = destPixelFormat;
h264outputstream->codec->width = cvFrame.cols;
h264outputstream->codec->height = cvFrame.rows;
/**
* Prepare the conversion context
*/
SwsContext *bgr2yuvcontext = sws_getContext(width, height,
sourcePixelFormat,
h264outputstream->codec->width, h264outputstream->codec->height,
h264outputstream->codec->pix_fmt,
SWS_BICUBIC, NULL, NULL, NULL);
/**
* Convert and encode frames
*/
for (uint i=0; i < 250; i++)
{
/**
* Allocate source frame, i.e. input to sws_scale()
*/
avpicture_alloc((AVPicture*)sourceAvFrame, sourcePixelFormat, width, height);
for (int h = 0; h < height; h++)
memcpy(&(sourceAvFrame->data[0][h*sourceAvFrame->linesize[0]]), &(cvFrame.data[h*cvFrame.step]), width*3);
/**
* Allocate destination frame, i.e. output from sws_scale()
*/
avpicture_alloc((AVPicture *)destAvFrame, destPixelFormat, width, height);
sws_scale(bgr2yuvcontext, sourceAvFrame->data, sourceAvFrame->linesize,
0, height, destAvFrame->data, destAvFrame->linesize);
/**
* Prepare an AVPacket for encoded output
*/
AVPacket avEncodedPacket;
av_init_packet(&avEncodedPacket);
avEncodedPacket.data = NULL;
avEncodedPacket.size = 0;
// av_free_packet(&avEncodedPacket); w/ or w/o result doesn't change
cerr << "I'll soon crash.." << endl;
if (avcodec_encode_video2(h264outputstream->codec, &avEncodedPacket, destAvFrame, &got_frame) < 0)
exit(1);
cerr << "Checking if we have a frame" << endl;
if (got_frame)
av_write_frame(cv2avFormatContext, &avEncodedPacket);
av_free_packet(&avEncodedPacket);
av_frame_free(&sourceAvFrame);
av_frame_free(&destAvFrame);
}
}
Thanks in advance!
EDIT: And the stack trace after the crash,
Thread 2 (Thread 0x7fffe5506700 (LWP 10005)):
#0 0x00007ffff4bf6c5d in poll () at /lib64/libc.so.6
#1 0x00007fffe9073268 in () at /usr/lib64/libusb-1.0.so.0
#2 0x00007ffff47010a4 in start_thread () at /lib64/libpthread.so.0
#3 0x00007ffff4bff08d in clone () at /lib64/libc.so.6
Thread 1 (Thread 0x7ffff7f869c0 (LWP 10001)):
#0 0x00007ffff5ecc7dc in avcodec_encode_video2 () at /usr/lib64/libavcodec.so.56
#1 0x00000000004019b6 in main(int, char**) (argv=0x7fffffffd3d8) at ../src/OpenCV2FFmpeg.cpp:99
EDIT2: Problem was that I hadn't avcodec_open2() the codec as spotted by Ronald. Final version of the code is at https://github.com/barisdemiray/opencv2ffmpeg/, with leaks and probably other problems hoping that I'll improve it while learning both libraries.
Please provide a gdb backtrace after it crashes. I'm pretty sure it'll crash because your picture pointers are unaligned (avpicture_alloc() will allocate unaligned images, and avcodec_encode_video2() expects aligned data pointers), but there may be additional reasons why it would crash also.
To align data pointers in a AVFrame, use av_image_alloc() and related functions, with align=32.
Another problem is that you didn't call avcodec_open2(h264outputstream->codec, h264encoder).
Related
I have created with GIMP a C-Source image dump like the following:
/* GIMP RGBA C-Source image dump (example.c) */
static const struct {
guint width;
guint height;
guint bytes_per_pixel; /* 2:RGB16, 3:RGB, 4:RGBA */
guint8 pixel_data[304 * 98 * 2 + 1];
} example= {
304, 98, 2,
"\206\061\206\061..... }
Is there a way to read this in GIMP again in order to get back the original image? because it doesn't seem possible.
Or does it exist a tool that can do this back-conversion?
EDITED
Following some suggestion I tried to write a simple C programme to make the reverse coversion ending up with something very similar to another code found on internet but both dont work:
#include <stdlib.h>
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include "imgs_press.h"
#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>
using namespace std;
int main(int argc, char** argv) {
int fd;
char *name = "orignal_img.pnm";
fd = open(name, O_WRONLY | O_CREAT, 0644);
if (fd == -1) {
perror("open failed");
exit(1);
}
if (dup2(fd, 1) == -1) {
perror("dup2 failed");
exit(1);
}
// file descriptor 1, i.e. stdout, now points to the file
// "helloworld" which is open for writing
// You can now use printf which writes specifically to stdout
printf("P2\n");
printf("%d %d\n", press_high.width, press_high.height);
for(int x=0; x<press_high.width * press_high.height * 2; x++) {
printf("%d ", press_high.pixel_data[x]);
}
}
As suggested by n-1-8e9-wheres-my-share-m, maybe I need to manipulate the pixels usign the correct decode, but I have no idea how to do that, does anybody have other suggestions?
The image I got is indeed distorted:
Updated Answer
If you want to decode the RGB565 and write a NetPBM format PNM file without using ImageMagick, you can do this:
#include <stdint.h> /* for uint8_t */
#include <stdio.h> /* for printf */
/* tell compiler what those GIMP types are */
typedef int guint;
typedef uint8_t guint8;
#include <YOURGIMPIMAGE>
int main(){
int w = gimp_image.width;
int h = gimp_image.height;
int i;
uint16_t* RGB565p = (uint16_t*)&(gimp_image.pixel_data);
/* Print P3 PNM header on stdout */
printf("P3\n%d %d\n255\n",w, h);
/* Print RGB pixels, ASCII, one RGB pixel per line */
for(i=0;i<w*h;i++){
uint16_t RGB565 = *RGB565p++;
uint8_t r = (RGB565 & 0xf800) >> 8;
uint8_t g = (RGB565 & 0x07e0) >> 3;
uint8_t b = (RGB565 & 0x001f) << 3;
printf("%d %d %d\n", r, g ,b);
}
}
Compile with:
clang example.c
And run with:
./a.out > result.pnm
I have not tested it too extensively beyond your sample image, so you may want to make a test image with some reds, greens, blues and shades of grey to ensure that all my bit-twiddling is correct.
Original Answer
The easiest way to get your image back would be... to let ImageMagick do it.
So, take your C file and add a main() to it that simply writes the 304x98x2 bytes starting at &(example.pixel_data) to stdout:
Compile it with something like:
clang example.c -o program # or with GCC
gcc example.c -o program
Then run it, writing to a file for ImageMagick with:
./program > image.bin
And tell ImageMagick its size, type and where it is and what you want as a result:
magick -size 304x98 RGB565:image.bin result.png
I did a quick, not-too-thorough test of the following code and it worked fine for an image I generated with GIMP. Note it doesn't handle alpha/transparency but that could be added if necessary. Save it as program.c:
#include <unistd.h> /* for write() */
#include <stdint.h> /* for uint8_t */
/* tell compiler what those GIMP types are */
typedef int guint;
typedef uint8_t guint8;
<PASTE YOUR GIMP FILE HERE>
int main(){
/* Work out how many bytes to write */
int nbytes = example.width * example.height * 2;
/* Write on stdout for redirection to a file - may need to reopen in binary mode if on Windows */
write(1, &(example.pixel_data), nbytes);
}
If I run this with the file you provided via Google Drive I get:
Hello I am a beginner of C language for microprocessors. I want to read a ''.bmp'' file in order to apply line detection on it. I have declared a function to read the image. This error occurs when compile button is pushed:
#include "esp_camera.h"
#include "Arduino.h"
#include "FS.h" // SD Card ESP32
#include "SD_MMC.h" // SD Card ESP32
#include "soc/soc.h" // Disable brownour problems
#include "soc/rtc_cntl_reg.h" // Disable brownour problems
#include "driver/rtc_io.h"
#include <EEPROM.h> // read and write from flash memory
#include <SPI.h>
void imageReader(const char *imgName,
int *height,
int *width,
int *bitdepth,
unsigned char *header,
unsigned char *_colortable,
unsigned char *buf
) // READ AN IMAGE
{
int i;
fs::FS &fs = SD_MMC; //
FILE *file;
file = fopen(imgName,"rb"); // read imgName file ( it is a picture in .bmp format )
if(!file){
Serial.printf("Unable to read image");
}
for(i=0 ; i<54 ; i++){
header[i]=getc(file);
}
*width = *(int * )& header[18]; // width information of the image
*height = *(int * )& header[22]; // height information of image
*bitdepth = *(int *)& header[28];
if(*bitdepth<=8){
fread(_colortable,sizeof(unsigned char),1024,file);
}
fread(buf,sizeof(unsigned char),( 1600 * 1200 ) ,file);
fclose();
}
It gives this error. too few arguments to function 'int fclose(FILE*)'
The fclose() function needs to know which file to close. You need to tell it that by supplying "file" as an argument. You want to use fclose(file).
Hello right now i'm trying to use a thread to read some info about 2 images using OpenCV 2.3.1, while I can retrieve the sizes of the 2 images without a problem, an error occurs when I reach the end of the thread.
Unhandled exception at 0x00348A56 in OpenCvDemo.exe: An invalid parameter was passed to a function that considers invalid parameters fatal.
Since I can read the size of the images just fine I know they aren't empty or anything. In another test I detached the thread and put it to Sleep for 5 seconds, the main function executed fine and displayed the image, but once the thread woke up and reached the end it crashed the application and displayed the same error as above.
Sadly I need to use OpenCV 2.3.1 since this is just a test for a bigger application.
How can I solve this problem, or at least what is the cause of it?
#include "stdafx.h"
#include <windows.h>
#include <iostream>
#include <thread>
#include <opencv2/opencv.hpp>
void Loop(cv::Mat &image1, cv::Mat &image2) {
std::vector<uchar> buff1;
std::vector<uchar> buff2;
cv::imencode(".png", image1, buff1);
cv::imencode(".png", image2, buff2);
int imageSize1 = buff1.size();
int imageSize2 = buff2.size();
std::cout << "Size " << imageSize1 << "\n";
std::cout << "Size " << imageSize2 << "\n";
}
int main()
{
cv::Mat image1 = cv::imread("C:\\Users\\Cesar\\Pictures\\happyface.png", 1);
cv::Mat image2 = cv::imread("C:\\Users\\Cesar\\Pictures\\happyface.png", 1);
std::thread mythread(Loop, std::ref(image1), std::ref(image2));
mythread.join();
cv::imshow("name", image1);
cv::waitKey(0);
return 0;
}
The callstack
OpenCvDemo.exe!_invoke_watson(const wchar_t * expression, const wchar_t * function_name, const wchar_t * file_name, unsigned int line_number, unsigned int reserved) Line 224 C++
OpenCvDemo.exe!_invalid_parameter(const wchar_t * expression, const wchar_t * function_name, const wchar_t * file_name, unsigned int line_number, unsigned int reserved) Line 113 C++
[External Code]
OpenCvDemo.exe!Loop(cv::Mat & image1, cv::Mat & image2) Line 23 C++
[External Code]
OpenCvDemo.exe!invoke_thread_procedure(unsigned int(__stdcall*)(void *) procedure, void * const context) Line 92 C++
OpenCvDemo.exe!thread_start<unsigned int (__stdcall*)(void *)>(void * const parameter) Line 115 C++
[External Code]
[Frames below may be incorrect and/or missing, no symbols loaded for kernel32.dll]
This question is similar to cuModuleLoadDataEx options but I would like to bring the topic up again and in addition provide more information.
When loading a PTX string with the NV driver via cuModuleLoadDataEx it seems to ignore all options all together. I provide full working examples so that anyone interested can directly and with no effort reproduce this. First a small PTX kernel (save this as small.ptx) then the C++ program that loads the PTX kernel.
.version 3.1
.target sm_20, texmode_independent
.address_size 64
.entry main()
{
ret;
}
main.cc
#include<cstdlib>
#include<iostream>
#include<fstream>
#include<sstream>
#include<string>
#include<map>
#include "cuda.h"
int main(int argc,char *argv[])
{
CUdevice cuDevice;
CUcontext cuContext;
CUfunction func;
CUresult ret;
CUmodule cuModule;
cuInit(0);
std::cout << "trying to get device 0\n";
ret = cuDeviceGet(&cuDevice, 0);
if (ret != CUDA_SUCCESS) { exit(1);}
std::cout << "trying to create a context\n";
ret = cuCtxCreate(&cuContext, 0, cuDevice);
if (ret != CUDA_SUCCESS) { exit(1);}
std::cout << "loading PTX string from file " << argv[1] << "\n";
std::ifstream ptxfile( argv[1] );
std::stringstream buffer;
buffer << ptxfile.rdbuf();
ptxfile.close();
std::string ptx_kernel = buffer.str();
std::cout << "Loading PTX kernel with driver\n" << ptx_kernel;
const unsigned int jitNumOptions = 3;
CUjit_option *jitOptions = new CUjit_option[jitNumOptions];
void **jitOptVals = new void*[jitNumOptions];
// set up size of compilation log buffer
jitOptions[0] = CU_JIT_INFO_LOG_BUFFER_SIZE_BYTES;
int jitLogBufferSize = 1024*1024;
jitOptVals[0] = (void *)&jitLogBufferSize;
// set up pointer to the compilation log buffer
jitOptions[1] = CU_JIT_INFO_LOG_BUFFER;
char *jitLogBuffer = new char[jitLogBufferSize];
jitOptVals[1] = jitLogBuffer;
// set up wall clock time
jitOptions[2] = CU_JIT_WALL_TIME;
float jitTime = -2.0;
jitOptVals[2] = &jitTime;
ret = cuModuleLoadDataEx( &cuModule , ptx_kernel.c_str() , jitNumOptions, jitOptions, (void **)jitOptVals );
if (ret != CUDA_SUCCESS) { exit(1);}
std::cout << "walltime: " << jitTime << "\n";
std::cout << std::string(jitLogBuffer) << "\n";
}
Build (assuming CUDA is installed under /usr/local/cuda, I use CUDA 5.0):
g++ -I/usr/local/cuda/include -L/usr/local/cuda/lib64/ main.cc -o main -lcuda
If someone is able to extract any sensible information from the compilation process that would be great! The documentation of CUDA driver API where cuModuleLoadDataEx is explained (and which options it is supposed to accept) http://docs.nvidia.com/cuda/cuda-driver-api/index.html
If I run this, the log is empty and jitTime wasn't even touched by the NV driver:
./main small.ptx
trying to get device 0
trying to create a context
loading PTX string from file empty.ptx
Loading PTX kernel with driver
.version 3.1
.target sm_20, texmode_independent
.address_size 64
.entry main()
{
ret;
}
walltime: -2
EDIT:
I managed to get the JIT compile time. However it seems that the driver expects an array of 32bit values as OptVals. Not as stated in the manual as an array of pointers (void *) which are on my system 64 bits. So, this works:
const unsigned int jitNumOptions = 1;
CUjit_option *jitOptions = new CUjit_option[jitNumOptions];
int *jitOptVals = new int[jitNumOptions];
jitOptions[0] = CU_JIT_WALL_TIME;
// here the call to cuModuleLoadDataEx
std::cout << "walltime: " << (float)jitOptions[0] << "\n";
I believe that it is not possible to do the same with an array of void *. The following code does not work:
const unsigned int jitNumOptions = 1;
CUjit_option *jitOptions = new CUjit_option[jitNumOptions];
void **jitOptVals = new void*[jitNumOptions];
jitOptions[0] = CU_JIT_WALL_TIME;
// here the call to cuModuleLoadDataEx
// here I also would have a problem casting a 64 bit void * to a float (32 bit)
EDIT
Looking at the JIT compilation time jitOptVals[0] was misleading. As mentioned in the comments, the JIT compiler caches previous translations and won't update the JIT compile time if it finds a cached compilation. Since I was looking whether this value has changed or not I assumed that the call ignores the options all together. Which it doesn't. It's works fine.
Your jitOptVals should not contain pointers to your values, instead cast the values to void*:
// set up size of compilation log buffer
jitOptions[0] = CU_JIT_INFO_LOG_BUFFER_SIZE_BYTES;
int jitLogBufferSize = 1024*1024;
jitOptVals[0] = (void *)jitLogBufferSize;
// set up pointer to the compilation log buffer
jitOptions[1] = CU_JIT_INFO_LOG_BUFFER;
char *jitLogBuffer = new char[jitLogBufferSize];
jitOptVals[1] = jitLogBuffer;
// set up wall clock time
jitOptions[2] = CU_JIT_WALL_TIME;
float jitTime = -2.0;
//Keep jitOptVals[2] empty as it only an Output value:
//jitOptVals[2] = (void*)jitTime;
and after cuModuleLoadDataEx, you get your jitTime like jitTime = (float)jitOptions[2];
Deal all,
I need to decode an animated gif format picture into some bitmap files in MFC2010. Is there any library to decode a gif picture? I cannot use GDIPlus because the program has to run on windows XP. I do appreciate if someone provides me with a library, Activex, dll or anything similar.
Many Thanks,
Shervin Zargham
It's pretty simple using ImageMagick's C++ API (Magick++) :
/* list of Image to store the GIF's frames */
std::vector<Magick::Image> imageList;
/* read all the frames of the animated GIF */
Magick::readImages( &imageList, "animated.gif" );
/* optionnally coalesce the frame sequence depending on the expected result */
Magick::coalesceImages( &imageList, imageList.begin(), imageList.end());
/* store each frame in a separate BMP file */
for(unsigned int i = 0; i < imageList.size(); ++i) {
std::stringstream ss;
ss << "frame" << i << ".bmp";
imageList[i].write(ss.str());
}
WIC (included in Vista, available for XP) offers CLSID_WICGifDecoder, a COM component.
Try this using ImageMagick's C++ API (Magick++) ,tested on VS210:
#include <Magick++.h>
#include <string>
#include <iostream>
#include <list>
using namespace std;
using namespace Magick;
void kk(char * nombre, char *ext)
{
/* list of Image to store the GIF's frames */
std::list<Magick::Image> imageList;
/* read all the frames of the animated GIF */
Magick::readImages( &imageList, nombre );
/* compone las diferencias para obtener los cuadros reales */
Magick::coalesceImages(&imageList,imageList.begin( ),imageList.end( ));
/* store each frame in a separate BMP file */
list <Magick::Image>::iterator it;
int i=1;
for ( it = imageList.begin( ); it != imageList.end( ); it++ , i++)
{
std::string name = "frame" + to_string((_Longlong)(i)) + ext ;
it->write(name);
}
}
int main( int /*argc*/, char ** argv)
{
// Initialize ImageMagick install location for Windows
InitializeMagick(*argv);
try {
kk("luni0.gif", ".png"); // using ".bmp", ".jpg", ".png", OK
return 0;
}
catch( exception &error_ )
{
cout << "Caught exception: " << error_.what() << endl;
return 1;
}
}
It's been a long time, but I recall once using OleLoadPicture to open GIF and PNG files on old versions of Windows, though the documentation seems to suggest that it's only for BMP, ICO, and WMF.