OpenCV VideoWriter C++ not Writing to Output.avi - c++

I'm attempting to write code that takes a frames and writes to an output file .avi / .mp4 using cv VideoWriter in C++.
System setup is Linux Ubuntu 18.04.4 LTS, Qt 5.9.5 (GCC 7.3.0, 64 bit)
I'm getting frames from camera, convert them to QImage and display them on GUI well.
void FrameThread::run()
{
while (m_isWaitting)
{
if (SUCCESS == Buf_WaitForFrame(m_Cam, &m_frame))
{
int channels = m_frame.ucChannels;
int width = m_frame.usWidth;
int height = m_frame.usHeight;
int elementBytes = m_frame.ucElemBytes;
QImage img;
img = QImage(width, height, QImage::Format_Grayscale8);
uchar *pSrc = (uchar *)m_frame.pBuffer + m_frame.usHeader;
uchar *pDst = (uchar *)img.bits();
if (2 == elementBytes)
{
pSrc += (elementBytes / 2);
if (1 == channels)
{
int pixels = width * height * channels;
for (int i = 0; i < pixels; ++i)
{
*pDst++ = *pSrc;
pSrc += elementBytes;
}
}
}
emit signalUpdateImage(img);
if (m_isSaving)
{
saveImage();
}
if (m_isRecording)
{
if(!m_isCreateVideoWriter)
{
QString savedPath = "/home/nvidia/Pictures/";
CvSize size = cvSize(2048, 1148);
char cur[1024];
timespec time;
clock_gettime(CLOCK_REALTIME, &time);
tm nowtm;
localtime_r(&time.tv_sec, &nowtm);
sprintf(cur, "%s%04d%02d%02d%02d%02d%02d.avi", savedPath.toLocal8Bit().data(), nowtm.tm_year+1900, nowtm.tm_mon+1, nowtm.tm_mday, nowtm.tm_hour, nowtm.tm_min, nowtm.tm_sec);
video = cv::VideoWriter(cur,cv::VideoWriter::fourcc('M','J','P','G'),25.0, size, 1);
m_isCreateVideoWriter = true;
}
else
{
//Clicked 'Stop Record' button
//Close video writer
if (CamObject::getInstance()->m_bFinishRec)
{
video.release();
m_isRecording = false;
m_isCreateVideoWriter = false;
return;
}
// Recording...
else
{
videoFrame = QImageToCvMat(img);
video.write(videoFrame);
}
}
}
}
}
}
Also I created method QImageToCvMat because their is not write method with QImage :
cv::Mat QImageToCvMat(QImage &inImage)
{
cv::Mat mat(inImage.height(), inImage.width(),
CV_8UC1, inImage.bits(), inImage.bytesPerLine());
return mat;
}
The code failing to produce output and program collapse on line
video.write(videoFrame);
Any one familiar with this issue or can kindly give advice? Other way to create video file?

Related

How to resize an image from an rgb buffer using c++

I have an (char*)RGB buffer that has the data of actual image. Let's say that the actual image resolution is 720x576. Now I want to resize it to a resolution , say 120x90.
How can I do this using https://code.google.com/p/jpeg-compressor/ or libjpeg ?
Note: can use any other library, but should work in linux.
Edited: Video decoder decodes a frame in YUV, which I convert it into RGB. All these happen in a buffer.
I need to resize the RGB buffer to make a thumbnail out of it with variable size.
Thanks for the help in advance
I did the following to achieve my goal:
#define TN_WIDTH 240
#define TN_HEIGHT 180
#include "jpegcompressor/jpge.h"
#include "jpegcompressor/jpgd.h"
#include <ippi.h>
bool createThumnailJpeg(const uint8* pSrc, int srcwidth, int srcheight)
{
int req_comps = 3;
jpge::params params;
params.m_quality = 50;
params.m_subsampling = jpge::H2V2;
params.m_two_pass_flag = false;
FILE *fpJPEGTN = fopen("Resource\\jpegcompressor.jpeg","wb");
int dstWidth = TN_WIDTH;
int dstHeight = TN_HEIGHT;
int uiDstBufferSize = dstWidth * dstHeight * 3;
uint8 *pDstRGBBuffer = new uint8[uiDstBufferSize];
uint8 *pJPEGTNBuffer = new uint8[uiDstBufferSize];
int uiSrcBufferSize = srcwidth * srcheight * 3;
IppiSize srcSize = {srcwidth , srcheight};
IppiRect srcROI = {0, 0, srcwidth, srcheight};
IppiSize dstROISize = {dstWidth, dstHeight};
double xfactor = (double) dstWidth / srcwidth;
double yfactor = (double) dstHeight / srcheight;
IppStatus status = ippiResize_8u_C3R(pSrc, srcSize, srcwidth*3, srcROI,
pDstRGBBuffer, dstWidth*3, dstROISize, xfactor, yfactor, 1);
if (!jpge::compress_image_to_jpeg_file_in_memory(pJPEGTNBuffer, uiDstBufferSize, dstWidth, dstHeight, req_comps, pDstRGBBuffer, params))
{
cout << "failed!";
delete[] pDstRGBBuffer;
delete [] pJPEGTNBuffer;
return false;
}
if (fpJPEGTN)
{
fwrite(pJPEGTNBuffer, uiDstBufferSize, 1, fpJPEGTN);
fclose(fpJPEGTN);
}
delete [] pDstRGBBuffer;
delete [] pJPEGTNBuffer;
return true;
}

Get RGB values from AVPicture and change to grey-scale in FFMPEG

The main motive of my code is to change the RGB values from the AVPicture in FFMPEG.
I have been able to get the image data "data[0]" by following the article : http://blog.tomaka17.com/2012/03/libavcodeclibavformat-tutorial/
I would like to know that how can I access the 3 bytes of pic.data[0] which is in RGB format. I have been trying to access the pic.data[i][j] via for-loop in 2D matrix fashion but jth element>3.
Any guidance in this regard will be helpful.
Code is here :
AVPicture pic;
avpicture_alloc(&pic, PIX_FMT_RGB24, mpAVFrameInput->width,mpAVFrameInput->height);
auto ctxt = sws_getContext(mpAVFrameInput->width,mpAVFrameInput->height,static_cast<PixelFormat>(mpAVFrameInput->format),
mpAVFrameInput->width, mpAVFrameInput->height, PIX_FMT_RGB24, SWS_BILINEAR, nullptr, nullptr, nullptr);
if (ctxt == nullptr)
throw std::runtime_error("Error while calling sws_getContext");
sws_scale(ctxt, mpAVFrameInput->data, mpAVFrameInput->linesize, 0, mpAVFrameInput->height, pic.data,
pic.linesize);
for (int i = 0; i < (mpAVFrameInput->height-1); i++) {
for (int j = 0; j < (mpAVFrameInput->width-1); j++) {
printf("\n value: %d",pic.data[0][j]);
}
}
Pseudo code which is in my mind is :
For each pixel in image {
Red = pic.data[i][j].pixel.RED;
Green = pic.data[i][j].pixel.GREEN;
Blue = pic.data[i][j].pixel.BLUE;
GRAY = (Red+Green+Blue)/3;
Red = GRAY;
Green = GRAY;
Blue = GRAY;
Save Frame;}
I am quite new to FFMPEG therefore any guidance and help will be highly appreciable.
Many Thanks
First extract the row data row-by-row of each frame; iterate the loop keeping in view the frame's height.
Here's the sample:
int FrameHeight = FrameInput->height;
int FrameWidth = FrameInput->width;
for(int Counter=0; Counter<FrameHeight; Counter++)
{
int RowSize = FrameWidth*sizeof(uint8_t)*3;
uint8_t* RowData = (uint8_t*) malloc(RowSize);
memset(RowData, 0, RowSize);
memcpy(RowData, AVFrameInput->data[0]+Counter*AVFrameInput->linesize[0], RowSize);
for(int k=0;k<AVFrameInput->linesize[0];++k)
{
if(RowData[k]> 200)
{
RowData[k] = RowData[k]/3;
}
else
{
if(RowData[k] > 150)
{
RowData[k] = RowData[k]/3;
}
else
{
RowData[k] = RowData[k]/3;
}
}
}
memcpy(AVFrameInput->data[0]+Counter*AVFrameInput->linesize[0], RowData, RowSize);
}

RGB to x264 : Strange color render

i'm trying to make a video from an OpenGl context.
I'm Using glReadPixel, to be sure RGB buffer data is Ok i save it into a bmp file, wich i can read correctly.
My .h264 video is encoded but there are some artefact and i don't understand why.
I tried a lot of different parameters for the x264_param_t but anything better !
Bitmap saved (OpenGL real data) : Bitmap from OpenGl (1mo)
Raw h264 with error : Raw h264 video (1mo)
OpenGl ReadPixel :
int nSize = ClientHeight * ClientWidth * 3;
GLubyte *inBuff = new GLubyte[nSize];
glReadBuffer(GL_FRONT);
glReadPixels(0, 0, ldwidth, ldheight, GL_BGR, GL_UNSIGNED_BYTE, inBuff);
The params define :
x264_param_default(&mX264_param_t);
x264_param_default_preset(&mX264_param_t, "placebo", "film");
mX264_param_t.i_csp = X264_CSP_BGR;
mX264_param_t.i_threads = 6;
mX264_param_t.i_width = mWidth;
mX264_param_t.i_height = mHeight;
mX264_param_t.i_fps_num = mFps;
mX264_param_t.i_fps_den = 1;
// Intra refres:
mX264_param_t.i_keyint_max = mFps;
mX264_param_t.b_intra_refresh = 1;
//Rate control:
mX264_param_t.rc.i_rc_method = X264_RC_CRF;
mX264_param_t.rc.f_rf_constant = 25;
mX264_param_t.rc.f_rf_constant_max = 35;
int bps = 5000;
mX264_param_t.rc.i_bitrate = bps;
mX264_param_t.rc.i_vbv_max_bitrate = bps;
mX264_param_t.i_bframe = 2;
mX264_param_t.i_keyint_min = mFps / 4;
//For streaming:
mX264_param_t.b_repeat_headers = 1;
mX264_param_t.b_annexb = 1;
mX264_param_t.i_log_level = X264_LOG_DEBUG;
x264_param_apply_profile(&mX264_param_t, "baseline");
mpEncoder = x264_encoder_open(&mX264_param_t);
x264_encoder_parameters(mpEncoder, &mX264_param_t);
mpPictureOut = new x264_picture_t();
mpPictureIn = new x264_picture_t();
x264_picture_alloc(mpPictureIn, X264_CSP_BGR | X264_CSP_VFLIP, mWidth, mHeight);
Then the encoding loop :
mpPictureIn->img.i_csp = X264_CSP_BGR;
mpPictureIn->img.i_plane = 1;
mpPictureIn->img.i_stride[0] = 3 * mWidth;
mpPictureIn->img.plane[0] = rgbframe;
mpPictureIn->i_pts = mFrameCount;
mpPictureIn->i_type = X264_TYPE_AUTO;
mpPictureOut->i_pts = mFrameCount;
int i_nals;
x264_nal_t* nals;
int frame_size = x264_encoder_encode(mpEncoder, &nals, &i_nals, mpPictureIn, mpPictureOut);
if(frame_size > 0)
{
mpFileOut->write_frame(nals[0].p_payload, frame_size, mpPictureOut);
mFrameCount++;
}
The write frame :
int TVideoFileWriter::write_frame(uint8_t *p_nalu, int i_size, x264_picture_t *p_picture)
{
if(fwrite(p_nalu, i_size, 1, mFileHandle))
return i_size;
return -1;
}
You opened your output file in text mode (and not binary mode) and so all 0x0A bytes where replaced with 0x0D 0x0A bytes.
Here is your output with this replace reverted: out_fixed.h264
And it plays fine.

opencv open image without jerking

I am using opencv2 to open a series of images (On raspberry pi) I find the transition is very jerky (ie opening the image) is there any way to smooth this?
The code snippet below is what I am using atm (I guess I should use mat:)
void loadImage()
{
int nImages = 6;
for (int i = 0; i < nImages; ++i)
{
IplImage *image;
char filename[100];
strcpy(filename, "images/");
char frameNo[10];
sprintf(frameNo, "%03i", i);
strcat(filename, frameNo);
strcat(filename, ".jpg");
image = cvLoadImage(filename);
cvNamedWindow("pic");
cvShowImage("pic",image);
cvMoveWindow("pic", 0, 0);
cvWaitKey(1000);
}
}
You have to load your images either in other thread, or before the loop if you're lazy. It should be something like this:
void loadImages() {
IplImage *images[NUM_IMAGES];
images[0] = cvLoadImage("filename1.jpg");
images[1] = cvLoadImage("filename1.jpg");
images[2] = cvLoadImage("filename1.jpg");
.
.
. (etc)
}
void showImages(){
loadImages();
for (int i = 0; i < nImages; ++i){
cvNamedWindow("pic");
cvShowImage("pic",images[i]);
cvMoveWindow("pic", 0, 0);
cvWaitKey(1000);
}
}
Of course, array of pinters to images *images[] should be declared as global, or you should pass it from loadimages() function to showImages() (Better solution).

Convert Leptonica Pix Object to QPixmap ( or other image object )

I'm using the Leptonica Library to process some pictures. After that I want to show them in my QT GUI. Leptonica is using their own format Pix for the images, while QT is using their own format QPixmap. At the moment the only way for me is to save the pictures after processing as a file ( like bmp ) and then load them again with a QT function call. Now I want to convert them in my code, so I dont need to take the detour with saving them on the filesystem. Any ideas how to do this?
Best Regards
// edit:
Okay as already suggested I tried to convert the PIX* to a QImage.
The PIX* is defined like this:
http://tpgit.github.com/Leptonica/pix_8h_source.html
struct Pix
{
l_uint32 w; /* width in pixels */
l_uint32 h; /* height in pixels */
l_uint32 d; /* depth in bits */
l_uint32 wpl; /* 32-bit words/line */
l_uint32 refcount; /* reference count (1 if no clones) */
l_int32 xres; /* image res (ppi) in x direction */
/* (use 0 if unknown) */
l_int32 yres; /* image res (ppi) in y direction */
/* (use 0 if unknown) */
l_int32 informat; /* input file format, IFF_* */
char *text; /* text string associated with pix */
struct PixColormap *colormap; /* colormap (may be null) */
l_uint32 *data; /* the image data */
};
while QImage offers me a method like this:
http://developer.qt.nokia.com/doc/qt-4.8/qimage.html#QImage-7
QImage ( const uchar * data,
int width,
int height,
int bytesPerLine,
Format format )
I assume I cant just copy the data from the PIX to the QImage when calling the constructor. I guess I need to fill the QImage Pixel by Pixel, but actually I dont know how? Do I need to loop through all the coordinates? How do I regard the bit depth? Any ideas here?
I use this for conversion QImage to PIX:
PIX* TessTools::qImage2PIX(QImage& qImage) {
PIX * pixs;
l_uint32 *lines;
qImage = qImage.rgbSwapped();
int width = qImage.width();
int height = qImage.height();
int depth = qImage.depth();
int wpl = qImage.bytesPerLine() / 4;
pixs = pixCreate(width, height, depth);
pixSetWpl(pixs, wpl);
pixSetColormap(pixs, NULL);
l_uint32 *datas = pixs->data;
for (int y = 0; y < height; y++) {
lines = datas + y * wpl;
QByteArray a((const char*)qImage.scanLine(y), qImage.bytesPerLine());
for (int j = 0; j < a.size(); j++) {
*((l_uint8 *)lines + j) = a[j];
}
}
return pixEndianByteSwapNew(pixs);
}
And this for conversion PIX to QImage:
QImage TessTools::PIX2QImage(PIX *pixImage) {
int width = pixGetWidth(pixImage);
int height = pixGetHeight(pixImage);
int depth = pixGetDepth(pixImage);
int bytesPerLine = pixGetWpl(pixImage) * 4;
l_uint32 * s_data = pixGetData(pixEndianByteSwapNew(pixImage));
QImage::Format format;
if (depth == 1)
format = QImage::Format_Mono;
else if (depth == 8)
format = QImage::Format_Indexed8;
else
format = QImage::Format_RGB32;
QImage result((uchar*)s_data, width, height, bytesPerLine, format);
// Handle pallete
QVector<QRgb> _bwCT;
_bwCT.append(qRgb(255,255,255));
_bwCT.append(qRgb(0,0,0));
QVector<QRgb> _grayscaleCT(256);
for (int i = 0; i < 256; i++) {
_grayscaleCT.append(qRgb(i, i, i));
}
if (depth == 1) {
result.setColorTable(_bwCT);
} else if (depth == 8) {
result.setColorTable(_grayscaleCT);
} else {
result.setColorTable(_grayscaleCT);
}
if (result.isNull()) {
static QImage none(0,0,QImage::Format_Invalid);
qDebug() << "***Invalid format!!!";
return none;
}
return result.rgbSwapped();
}
This code accepts a const QImage& parameter.
static PIX* makePIXFromQImage(const QImage &image)
{
QByteArray ba;
QBuffer buf(&ba);
buf.open(QIODevice::WriteOnly);
image.save(&buf, "BMP");
return pixReadMemBmp(ba.constData(), ba.size());
}
I do not know the Leptonica Library, but I had a short look at the documentation and found the documentation about the PIX structure. You can create a QImage from the raw data and convert this to a QPixmap with convertFromImage.
Well I could solve the problem this way:
Leptonica offers a function
l_int32 pixWriteMemBmp (l_uint8 **pdata, size_t *psize, PIX *pix)
With this function you can write into the memory instead of a filestream. Still ( in this example ) the Bmp Header and format persists ( there are the same functions for other image formats too ).
The corresponding function from QT is this one:
bool QImage::loadFromData ( const uchar * data, int len, const char * format = 0 )
Since the the Header persits I just need to pass the data ptr and the size to the loadFromData function and QT does the rest.
So all together it would be like this:
PIX *m_pix;
FILE * pFile;
pFile = fopen( "PathToFile", "r" );
m_pix = pixReadStreamBmp(pFile); // If other file format use the according function
fclose(pFile);
// Now we have a Pix object from leptonica
l_uint8* ptr_memory;
size_t len;
pixWriteMemBmp(&ptr_memory, &size, m_pix);
// Now we have the picture somewhere in the memory
QImage testimage;
QPixmap pixmap;
testimage.loadFromData((uchar *)ptr_memory,len);
pixmap.convertFromImage(testimage);
// Now we have the image as a pixmap in Qt
This actually works for me, tho I don't know if there is a way to do this backwards so easy. ( If there is, please let me know )
Best Regards
You can save your pixmap to RAM instead of file (use QByteArray to store the data, and QBuffer as your I/O device).