opencv open image without jerking - c++

I am using opencv2 to open a series of images (On raspberry pi) I find the transition is very jerky (ie opening the image) is there any way to smooth this?
The code snippet below is what I am using atm (I guess I should use mat:)
void loadImage()
{
int nImages = 6;
for (int i = 0; i < nImages; ++i)
{
IplImage *image;
char filename[100];
strcpy(filename, "images/");
char frameNo[10];
sprintf(frameNo, "%03i", i);
strcat(filename, frameNo);
strcat(filename, ".jpg");
image = cvLoadImage(filename);
cvNamedWindow("pic");
cvShowImage("pic",image);
cvMoveWindow("pic", 0, 0);
cvWaitKey(1000);
}
}

You have to load your images either in other thread, or before the loop if you're lazy. It should be something like this:
void loadImages() {
IplImage *images[NUM_IMAGES];
images[0] = cvLoadImage("filename1.jpg");
images[1] = cvLoadImage("filename1.jpg");
images[2] = cvLoadImage("filename1.jpg");
.
.
. (etc)
}
void showImages(){
loadImages();
for (int i = 0; i < nImages; ++i){
cvNamedWindow("pic");
cvShowImage("pic",images[i]);
cvMoveWindow("pic", 0, 0);
cvWaitKey(1000);
}
}
Of course, array of pinters to images *images[] should be declared as global, or you should pass it from loadimages() function to showImages() (Better solution).

Related

OpenCV VideoWriter C++ not Writing to Output.avi

I'm attempting to write code that takes a frames and writes to an output file .avi / .mp4 using cv VideoWriter in C++.
System setup is Linux Ubuntu 18.04.4 LTS, Qt 5.9.5 (GCC 7.3.0, 64 bit)
I'm getting frames from camera, convert them to QImage and display them on GUI well.
void FrameThread::run()
{
while (m_isWaitting)
{
if (SUCCESS == Buf_WaitForFrame(m_Cam, &m_frame))
{
int channels = m_frame.ucChannels;
int width = m_frame.usWidth;
int height = m_frame.usHeight;
int elementBytes = m_frame.ucElemBytes;
QImage img;
img = QImage(width, height, QImage::Format_Grayscale8);
uchar *pSrc = (uchar *)m_frame.pBuffer + m_frame.usHeader;
uchar *pDst = (uchar *)img.bits();
if (2 == elementBytes)
{
pSrc += (elementBytes / 2);
if (1 == channels)
{
int pixels = width * height * channels;
for (int i = 0; i < pixels; ++i)
{
*pDst++ = *pSrc;
pSrc += elementBytes;
}
}
}
emit signalUpdateImage(img);
if (m_isSaving)
{
saveImage();
}
if (m_isRecording)
{
if(!m_isCreateVideoWriter)
{
QString savedPath = "/home/nvidia/Pictures/";
CvSize size = cvSize(2048, 1148);
char cur[1024];
timespec time;
clock_gettime(CLOCK_REALTIME, &time);
tm nowtm;
localtime_r(&time.tv_sec, &nowtm);
sprintf(cur, "%s%04d%02d%02d%02d%02d%02d.avi", savedPath.toLocal8Bit().data(), nowtm.tm_year+1900, nowtm.tm_mon+1, nowtm.tm_mday, nowtm.tm_hour, nowtm.tm_min, nowtm.tm_sec);
video = cv::VideoWriter(cur,cv::VideoWriter::fourcc('M','J','P','G'),25.0, size, 1);
m_isCreateVideoWriter = true;
}
else
{
//Clicked 'Stop Record' button
//Close video writer
if (CamObject::getInstance()->m_bFinishRec)
{
video.release();
m_isRecording = false;
m_isCreateVideoWriter = false;
return;
}
// Recording...
else
{
videoFrame = QImageToCvMat(img);
video.write(videoFrame);
}
}
}
}
}
}
Also I created method QImageToCvMat because their is not write method with QImage :
cv::Mat QImageToCvMat(QImage &inImage)
{
cv::Mat mat(inImage.height(), inImage.width(),
CV_8UC1, inImage.bits(), inImage.bytesPerLine());
return mat;
}
The code failing to produce output and program collapse on line
video.write(videoFrame);
Any one familiar with this issue or can kindly give advice? Other way to create video file?

Manipulating pixels of a cv::MAT just doesn't take effect

The following code is just supposed to load an image, fill it with a constant value and save it again.
Of course that doesn't have a purpose yet, but still it just doesn't work.
I can read the pixel values in the loop, but all changes are without effect and saves the file as it was loaded.
Think I followed the "efficient way" here accurately: http://docs.opencv.org/2.4/doc/tutorials/core/how_to_scan_images/how_to_scan_images.html
int main()
{
Mat im = imread("C:\\folder\\input.jpg");
int channels = im.channels();
int pixels = im.cols * channels;
if (!im.isContinuous())
{ return 0; } // Just to show that I've thought of that. It never exits here.
uchar* f = im.ptr<uchar>(0);
for (int i = 0; i < pixels; i++)
{
f[i] = (uchar)100;
}
imwrite("C:\\folder\\output.jpg", im);
return 0;
}
Normal cv functions like cvtColor() are taking effect as expected.
Are the changes through the array happening on a buffer somehow?
Huge thanks in advance!
The problem is that you are not looking at all pixels in the image. Your code only looks at im.cols*im.channels() which is a relatively small number as compared to the size of the image (im.cols*im.rows*im.channels()). When used in the for loop using the pointer, it only sets a value for couple of rows in an image ( if you look closely you will notice the saved image will have these set ).
Below is the corrected code:
int main()
{
Mat im = imread("C:\\folder\\input.jpg");
int channels = im.channels();
int pixels = im.cols * im.rows * channels;
if (!im.isContinuous())
{ return 0; } // Just to show that I've thought of that. It never exits here.
uchar* f = im.ptr<uchar>(0);
for (int i = 0; i < pixels; i++)
{
f[i] = (uchar)100;
}
imwrite("C:\\folder\\output.jpg", im);
return 0;
}

Convert a QImage to grayscale

I have a QImage and I need to convert it to grayscale, then later paint over that with colors. I found an allGray() and isGrayScale() function to check if an image is already grayscale, but no toGrayScale() or similarly-named function.
Right now I'm using this code, but it's does not have a very good performance:
for (int ii = 0; ii < image.width(); ii++) {
for (int jj = 0; jj < image.height(); jj++) {
int gray = qGray(image.pixel(ii, jj));
image.setPixel(ii, jj, QColor(gray, gray, gray).rgb());
}
}
What would be the best way, performance-wise, to convert a QImage to grayscale?
Since Qt 5.5, you can call QImage::convertToFormat() to convert a QImage to grayscale as follows:
QImage image = ...;
image.convertToFormat(QImage::Format_Grayscale8);
Rather than using the slow functions QImage::pixel and QImage::setPixel, use
QImage::scanline to access the data. Pixels on a scan (horizontal line ) are consecutive. Assuming you have a 32 bpp image, you can use QRgb to iterate over the scan. Finally always put the x coordinate in the inner loop. Which gives :
for (int ii = 0; ii < image.height(); ii++) {
uchar* scan = image.scanLine(ii);
int depth =4;
for (int jj = 0; jj < image.width(); jj++) {
QRgb* rgbpixel = reinterpret_cast<QRgb*>(scan + jj*depth);
int gray = qGray(*rgbpixel);
*rgbpixel = QColor(gray, gray, gray).rgba();
}
}
A quick test with an 3585 x 2386 image gave
********* Start testing of TestImage *********
Config: Using QTest library 4.7.4, Qt 4.7.4
PASS : TestImage::initTestCase()
RESULT : TestImage::grayscaleOp():
390 msecs per iteration (total: 390, iterations: 1)
PASS : TestImage::grayscaleOp()
RESULT : TestImage::grayscaleFast():
125 msecs per iteration (total: 125, iterations: 1)
PASS : TestImage::grayscaleFast()
PASS : TestImage::cleanupTestCase()
Totals: 4 passed, 0 failed, 0 skipped
********* Finished testing of TestImage *********
Source code:
testimage.h file:
#ifndef TESTIMAGE_H
#define TESTIMAGE_H
#include <QtTest/QtTest>
#include <QObject>
#include <QImage>
class TestImage : public QObject
{
Q_OBJECT
public:
explicit TestImage(QObject *parent = 0);
signals:
private slots:
void grayscaleOp();
void grayscaleFast();
private:
QImage imgop;
QImage imgfast;
};
#endif // TESTIMAGE_H
testimage.cpp file:
#include "testimage.h"
TestImage::TestImage(QObject *parent)
: QObject(parent)
, imgop("path_to_test_image.png")
, imgfast("path_to_test_image.png")
{
}
void TestImage::grayscaleOp()
{
QBENCHMARK
{
QImage& image = imgop;
for (int ii = 0; ii < image.width(); ii++) {
for (int jj = 0; jj < image.height(); jj++) {
int gray = qGray(image.pixel(ii, jj));
image.setPixel(ii, jj, QColor(gray, gray, gray).rgb());
}
}
}
}
void TestImage::grayscaleFast()
{
QBENCHMARK {
QImage& image = imgfast;
for (int ii = 0; ii < image.height(); ii++) {
uchar* scan = image.scanLine(ii);
int depth =4;
for (int jj = 0; jj < image.width(); jj++) {
QRgb* rgbpixel = reinterpret_cast<QRgb*>(scan + jj*depth);
int gray = qGray(*rgbpixel);
*rgbpixel = QColor(gray, gray, gray).rgba();
}
}
}
}
QTEST_MAIN(TestImage)
pro file:
QT += core gui
greaterThan(QT_MAJOR_VERSION, 4): QT += widgets
TARGET = QImageTest
TEMPLATE = app
CONFIG += qtestlib
SOURCES += testimage.cpp
HEADERS += testimage.h
Important note:
You already get an important performance boost just by inverting the loops. In this test case it was ~90ms.
You may use other libraries like opencv to make the grayscale conversion and then build the Qimage from an opencv buffer. I expect an even better performance improvement.
I'll post a slightly modified version of #UmNyobe's code. I just increment a pointer for the scan lines instead of calculating each pixel via an index.
// We assume the format to be RGB32!!!
Q_ASSERT(image.format() == QImage::Format_RGB32);
for (int ii = 0; ii < image.height(); ii++) {
QRgb *pixel = reinterpret_cast<QRgb*>(image.scanLine(ii));
QRgb *end = pixel + image.width();
for (; pixel != end; pixel++) {
int gray = qGray(*pixel);
*pixel = QColor(gray, gray, gray).rgb();
}
}
Internal qt class QPixmapColorizeFilter uses function grayscale that solves similar topic.
I derived following function from it, that should solve the problem.
Important part is converting image to 32-bit format, so you can consider each pixel as 32-bit value and you do not need to concern about bit alignment.
You can also use bits function directly and iterate over all pixels instead of iterating over lines and columns. With this trick you avoid multiplication performed in scanLine function.
QImage convertToGrayScale(const QImage &srcImage) {
// Convert to 32bit pixel format
QImage dstImage = srcImage.convertToFormat(srcImage.hasAlphaChannel() ?
QImage::Format_ARGB32 : QImage::Format_RGB32);
unsigned int *data = (unsigned int*)dstImage.bits();
int pixelCount = dstImage.width() * dstImage.height();
// Convert each pixel to grayscale
for(int i = 0; i < pixelCount; ++i) {
int val = qGray(*data);
*data = qRgba(val, val, val, qAlpha(*data));
++data;
}
return dstImage;
}

Get RGB values from AVPicture and change to grey-scale in FFMPEG

The main motive of my code is to change the RGB values from the AVPicture in FFMPEG.
I have been able to get the image data "data[0]" by following the article : http://blog.tomaka17.com/2012/03/libavcodeclibavformat-tutorial/
I would like to know that how can I access the 3 bytes of pic.data[0] which is in RGB format. I have been trying to access the pic.data[i][j] via for-loop in 2D matrix fashion but jth element>3.
Any guidance in this regard will be helpful.
Code is here :
AVPicture pic;
avpicture_alloc(&pic, PIX_FMT_RGB24, mpAVFrameInput->width,mpAVFrameInput->height);
auto ctxt = sws_getContext(mpAVFrameInput->width,mpAVFrameInput->height,static_cast<PixelFormat>(mpAVFrameInput->format),
mpAVFrameInput->width, mpAVFrameInput->height, PIX_FMT_RGB24, SWS_BILINEAR, nullptr, nullptr, nullptr);
if (ctxt == nullptr)
throw std::runtime_error("Error while calling sws_getContext");
sws_scale(ctxt, mpAVFrameInput->data, mpAVFrameInput->linesize, 0, mpAVFrameInput->height, pic.data,
pic.linesize);
for (int i = 0; i < (mpAVFrameInput->height-1); i++) {
for (int j = 0; j < (mpAVFrameInput->width-1); j++) {
printf("\n value: %d",pic.data[0][j]);
}
}
Pseudo code which is in my mind is :
For each pixel in image {
Red = pic.data[i][j].pixel.RED;
Green = pic.data[i][j].pixel.GREEN;
Blue = pic.data[i][j].pixel.BLUE;
GRAY = (Red+Green+Blue)/3;
Red = GRAY;
Green = GRAY;
Blue = GRAY;
Save Frame;}
I am quite new to FFMPEG therefore any guidance and help will be highly appreciable.
Many Thanks
First extract the row data row-by-row of each frame; iterate the loop keeping in view the frame's height.
Here's the sample:
int FrameHeight = FrameInput->height;
int FrameWidth = FrameInput->width;
for(int Counter=0; Counter<FrameHeight; Counter++)
{
int RowSize = FrameWidth*sizeof(uint8_t)*3;
uint8_t* RowData = (uint8_t*) malloc(RowSize);
memset(RowData, 0, RowSize);
memcpy(RowData, AVFrameInput->data[0]+Counter*AVFrameInput->linesize[0], RowSize);
for(int k=0;k<AVFrameInput->linesize[0];++k)
{
if(RowData[k]> 200)
{
RowData[k] = RowData[k]/3;
}
else
{
if(RowData[k] > 150)
{
RowData[k] = RowData[k]/3;
}
else
{
RowData[k] = RowData[k]/3;
}
}
}
memcpy(AVFrameInput->data[0]+Counter*AVFrameInput->linesize[0], RowData, RowSize);
}

DirectShow ISampleGrabber: samples are upside-down and color channels reverse

I have to use MS DirectShow to capture video frames from a camera (I just want the raw pixel data).
I was able to build the Graph/Filter network (capture device filter and ISampleGrabber) and implement the callback (ISampleGrabberCB). I receive samples of appropriate size.
However, they are always upside down (flipped vertically that is, not rotated) and the color channels are BGR order (not RGB).
I tried setting the biHeight field in the BITMAPINFOHEADER to both positive and negative values, but it doesn't have any effect. According to MSDN documentation, ISampleGrapper::SetMediaType() ignores the format block for video data anyways.
Here is what I see (recorded with a different camera, not DS), and what DirectShow ISampleGrabber gives me: The "RGB" is actually in red, green and blue respectively:
Sample of the code I'm using, slightly simplified:
// Setting the media type...
AM_MEDIA_TYPE* media_type = 0 ;
this->ds.device_streamconfig->GetFormat(&media_type); // The IAMStreamConfig of the capture device
// Find the BMI header in the media type struct
BITMAPINFOHEADER* bmi_header;
if (media_type->formattype != FORMAT_VideoInfo) {
bmi_header = &((VIDEOINFOHEADER*)media_type->pbFormat)->bmiHeader;
} else if (media_type->formattype != FORMAT_VideoInfo2) {
bmi_header = &((VIDEOINFOHEADER2*)media_type->pbFormat)->bmiHeader;
} else {
return false;
}
// Apply changes
media_type->subtype = MEDIASUBTYPE_RGB24;
bmi_header->biWidth = width;
bmi_header->biHeight = height;
// Set format to video device
this->ds.device_streamconfig->SetFormat(media_type);
// Set format for sample grabber
// bmi_header->biHeight = -(height); // tried this for either and both interfaces, no effect
this->ds.sample_grabber->SetMediaType(media_type);
// Connect filter pins
IPin* out_pin= getFilterPin(this->ds.device_filter, OUT, 0); // IBaseFilter interface for the capture device
IPin* in_pin = getFilterPin(this->ds.sample_grabber_filter, IN, 0); // IBaseFilter interface for the sample grabber filter
out_pin->Connect(in_pin, media_type);
// Start capturing by callback
this->ds.sample_grabber->SetBufferSamples(false);
this->ds.sample_grabber->SetOneShot(false);
this->ds.sample_grabber->SetCallback(this, 1);
// start recording
this->ds.media_control->Run(); // IMediaControl interface
I'm checking return types for every function and don't get any errors.
I'm thankful for any hint or idea.
Things I already tried:
Setting the biHeight field to a negative value for either the capture device filter or the sample grabber or for both or for neither - doesn't have any effect.
Using IGraphBuilder to connect the pins - same problem.
Connecting the pins before changing the media type - same problem.
Checking if the media type was actually applied by the filter by querying it again - but it apparently is applied or at least stored.
Interpreting the image as total byte reversed (last byte first, first byte last) - then it would be flipped horizontally.
Checking if it's a problem with the video camera - when I test it with VLC (DirectShow capture) it looks normal.
My quick hack for this:
void Camera::OutputCallback(unsigned char* data, int len, void *instance_)
{
Camera *instance = reinterpret_cast<Camera*>(instance_);
int j = 0;
for (int i = len-4; i > 0; i-=4)
{
instance->buffer[j] = data[i];
instance->buffer[j + 1] = data[i + 1];
instance->buffer[j + 2] = data[i + 2];
instance->buffer[j + 3] = data[i + 3];
j += 4;
}
Transport::RTPPacket packet;
packet.payload = instance->buffer;
packet.payloadSize = len;
instance->receiver->Send(packet);
}
It's correct on RGB32 color space, for other color spaces this code need to be corrected
I noticed that when using the I420 color space turning disappears.
In addition, most current codecs (VP8) is used as a format raw I/O I420 color space.
I wrote a simple mirroring frame function in color space I420.
void Camera::OutputCallback(unsigned char* data, int len, uint32_t timestamp, void *instance_)
{
Camera *instance = reinterpret_cast<Camera*>(instance_);
Transport::RTPPacket packet;
packet.rtpHeader.ts = timestamp;
packet.payload = data;
packet.payloadSize = len;
if (instance->mirror)
{
Video::ResolutionValues rv = Video::GetValues(instance->resolution);
int k = 0;
// Chroma values
for (int i = 0; i != rv.height; ++i)
{
for (int j = rv.width; j != 0; --j)
{
int l = ((rv.width * i) + j);
instance->buffer[k++] = data[l];
}
}
// U values
for (int i = 0; i != rv.height/2; ++i)
{
for (int j = (rv.width/2); j != 0; --j)
{
int l = (((rv.width / 2) * i) + j) + rv.height*rv.width;
instance->buffer[k++] = data[l];
}
}
// V values
for (int i = 0; i != rv.height / 2; ++i)
{
for (int j = (rv.width / 2); j != 0; --j)
{
int l = (((rv.width / 2) * i) + j) + rv.height*rv.width + (rv.width/2)*(rv.height/2);
if (l == len)
{
instance->buffer[k++] = 0;
}
else
{
instance->buffer[k++] = data[l];
}
}
}
packet.payload = instance->buffer;
}
instance->receiver->Send(packet);
}