I need to write a video from a sequence of images. I am using this code:
int main()
{
//Read sequence of images
Size frameSize(1360,1024);
//cout<<path<<endl;
VideoCapture sequence("path/fr%1d.jpg");
if (!sequence.isOpened())
{
cerr << "Failed to open Image Sequence!\n" << endl;
return -1;
}
//Write video
VideoWriter oVideoWriter ("path/MyVideo.avi",CV_FOURCC('8','B','P','S'), 15
, frameSize);
Mat imageGrey;
if(!oVideoWriter.isOpened())
{
cout<<"ERROR: Failed to write the video"<<endl;
return -1;
}
Mat Image;
int i=0;
while(true)
{
sequence>>Image;
if(Image.empty())
break;
cout<<i<<endl;
Mat imageArr[] = {Image, Image, Image};
merge(imageArr, 3, imageGrey);
//cvtColor(Image,imageGrey,CV_GRAY2BGR);
oVideoWriter.write(imageGrey);
i++;
}
cout<<"video written!"<<endl;
return 1;
}
I get a very dark video compared with one in Windows 7. I think it a problem of codec. What is the best codec that works on mac os x. thanks
As you said if you compared it to Windows, it's probably the codec problem. There is no best codec; some codecs are better for specific situations while worse for other cases. A common codec to use may be XVID using CV_FOURCC('X', 'V', 'I', 'D') but there is no guarantee that you have it.
A good practice is to pass -1 instead of CV_FOURCC() macro in the cv::VideoWriter constructor for your first run. A window will pop up and you can see what types of codecs you have available to use. When you are happy with one of them, find the 4CC code of that code and hardcode it in your program.
Also you may want to try Perian to have access to more codecs in Mac OS.
Related
I searched a lot but I can't find a decent library to solve to my problem. I receive a h264 stream from the network, and I want to decode it from memory and display it in real time. Something like this pseudocode below:
cv::Mat frame;
while (true)
{
if (newDataArrived)
{
void* h264Buffer; // address with current h264 stream
size_t h264BufferSize; // size of the stream
LibraryINeed::UpdateFrame(frame, h264Buffer, h264BufferSize); // !?
}
...
imshow("display", frame); // show most recent frame
}
Is there any library that provides this functionality?
I have to use Opus Codec to encode & decode audio datas in C++ and I have to encapsulate the functions.
So I try to send a floats array to try to encode it and I decode the result of the Opus encoding function. Unfortunately, the result is not the same and I get a table that contains no value from the initial table.
Here is my code.
Encapsulation:
std::vector<float> codec::OpusPlugin::decode(packet_t &packet) {
std::vector<float> out(BUFFER_SIZE * NB_CHANNELS);
int ret = 0;
if (!this->decoder)
throw Exception("Can't decode since there is no decoder.");
ret = opus_decode_float(this->decoder, packet.data.data(), packet.size, reinterpret_cast<float*>(out.data()), FRAME_SIZE, 0);
if (ret < 0)
throw Exception("Error while decoding compressed data.");
return out;
}
// ENCODER
packet_t codec::OpusPlugin::encode(std::vector<float> to_encode) {
std::vector<unsigned char> data(BUFFER_SIZE * NB_CHANNELS * 2);
packet_t packet;
int ret = 0;
if (!this->encoder)
throw Exception("Can't encode since there is no decoder.");
ret = opus_encode_float(this->encoder, reinterpret_cast<float const*>(to_encode.data()), FRAME_SIZE, data.data(), data.size());
if (ret < 0)
throw Exception("Error while encoding data.");
packet.size = ret;
packet.data = data;
return packet;
}
And there is the call of the functions:
packet_t packet;
std::vector<float> floats = {0.23, 0, -0.312, 0.401230, 0.1234, -0.1543};
packet = CodecPlugin->encode(floats);
std::cout << "packet size: " << packet.size << std::endl;
std::vector<float> output = CodecPlugin->decode(packet);
for (int i = 0; i < 10; i++) {
std::cout << output.data()[i] << " ";
}
Here is the packet_t structure, where I stock the return value of encode and the unsigned char array (encoded value)
typedef struct packet_s {
int size;
std::vector<unsigned char> data;
} packet_t;
The output of the program is
*-1.44487e-15 9.3872e-16 -1.42993e-14 7.31834e-15 -5.09662e-14 1.53629e-14 -8.36825e-14 3.9531e-14 -8.72754e-14 1.0791e-13 which is not the array I initialize at the beginning.
I read a lot of times the documentation and code examples but I don't know where I did a mistake.
I hope you will be able to help me.
Thanks :)
We don't see how you initialize your encoder and decoder so we don't know what their sample rate, complexity or number of channels is. No matter how you have initialized them you are still going to have the following problems:
First Opus encoding doesn't support arbitrary frame sizes but instead 2.5ms, 5ms, 10ms, 20, 40ms or 60ms RFC 6716 - Definition of the Opus Audio Codec relevant section 2.1.4. Moreover opus supports only 8kHz, 12kHz, 16kHz, 24kHz or 48kHz sample rates. No matter which of those you have chosen your array of 10 elements doesn't correspond to any of the supported frame sizes.
Secondly Opus codec is a lossy audio codec. This means that after you encode any signal you will never (probably except some edge cases) be able to reconstruct the original signal after decoding the encoded opus frame. The best way to test if your encoder and decoder work is with a real audio sample. Opus encoding preserves the perceptual quality of the audio files. Therefore if you try to test it with arbitrary data you might not get the expected results back even if you implemented the encoding and decoding functions correctly.
What you can easily do is to make a sine function of 2000Hz(there are multiple examples on the internet) for 20ms. This means 160 array elements at a sample rate of 8000Hz if you wish to use 8kHz. A sine wave of 2kHz is within the human hearing range so the encoder is going to preserve it. Then decode it back and see whether the elements of the input and output array are similar as we've already established that it is unlikely that they are the same.
I am not good in C++ so I can't help you with code examples but the problems above hold true no matter what language is used.
I have this OpenCV C++ code which is taking an image from the cam, encoding it and sending it to the STDOUT.
#include <unistd.h> //STDOUT_FILENO
#include "opencv2/opencv.hpp"
#include <iostream>
#include <fcntl.h>
using namespace std;
using namespace cv;
#define BUFLEN 4096
int main(int argc, char *argv[])
{
Mat frame;
std::vector<uchar> buf;
int bak, temp;
//read image as grayscale
namedWindow( "Camera", WINDOW_AUTOSIZE );
//redirect stdout to NULL in order to avoid printing to STDOUT undesired stuff
fflush(stdout);
bak = dup(1);
temp = open("/dev/null", O_WRONLY);
dup2(temp, 1);
close(temp );
VideoCapture cam(0 + CAP_V4L);
cam>>frame;
sleep(1);
if (!cam.isOpened())
{
cout << "\nCould not open reference " << 0 << endl;
return -1;
}
for (int i=0; i<5; i++)
{
cam>>frame;
}
/*Set the normal STDOUT back*/
fflush(stdout);
dup2(bak, 1);
close(bak);
//encode image and put data into the vector buf
imencode(".png",frame, buf);
//send the total size of vector to parent
cout<<buf.size()<<endl;
unsigned int written= 0;
int i = 0;
size_t toWrite = 0;
//send until all bytes have been sent
FILE * f = fdopen(STDOUT_FILENO, "w");
while (written<buf.size())
{
//send the current block of data
toWrite = BUFLEN < (buf.size()-written) ? BUFLEN : (buf.size()-written);
//written += write(STDOUT_FILENO, buf.data()+written, toWrite);
written += toWrite*fwrite ( buf.data()+written, toWrite, 1, f );
i++;
}
return 0;
}
Now instead of an image I would like to take an infinite continuous video from the cam. One solution would be to take a frame any given seconds, encode the frame and transmit it (print it to STDOUT), all actions inside an infinite loop.
Is there a better solution, more efficient than encoding and send each frame at each iteration?
Fundamentally, a video stream is a sequence of frames in a predefined order.
You could simply send the frames as images, one after another. There is nothing fundamentally wrong with that, but is not necessarily optimal (which also depends on your definition of optimal).
In communication, one of the aspects is to minimize the amount of data transferred. Simply sending the frames as images allows you to do some compression (e.g. jpeg). Better compression algorithms for video (e.g. mpeg) use the temporal properties of the sequence. If there is a (mostly) static frame, you can limit yourself sending data about the changing parts and assume the background is the same. This goes with some processing at both ends of the communication, but might increase the communication speed (assuming the link is the bottleneck). This would also add a lot of complexity to the system, so think about the potential advantages first (identify the bottlenecks).
I am not sure about the usage of your application, but sending the video stream to stdout might not necessarily be the best idea. Consider using a pipe or a socket instead (the latter allows you quite easily to transfer data over the network as well, which might be a nice outcome).
I am writing a program using OpenCV that shall work on Windows as well as on Linux. Now the problem with OpenCV is, that its cv::imread function can not handle filepaths that contain non-ASCII characters on Windows. A workaround is to first read the file into a buffer using other libraries (for example std-libraries or Qt) and then read the file from that buffer using the cv::imdecode function. This is what I currently do. However, it's not very fast and much slower than just using cv::imread. I have a TIF image that is about 1GB in size. Reading it with cv::imread takes approx. 1s, reading it with the buffer method takes about 14s. I assume that imread just reads those parts of the TIF that are necessary for displaying the image (no layers etc.). Either this, or my code for reading a file into a buffer is bad.
Now my question is if there is a better way to do it. Either a better way with regard to OpenCV or a better way with regard to reading a file into a buffer.
I tried two different methods for the buffering, one using the std libraries and one using Qt (actually they both use QT for some things). They both are equally slow.:
Method 1
std::shared_ptr<std::vector<char>> readFileIntoBuffer(QString const& path) {
#ifdef Q_OS_WIN
std::ifstream file(path.toStdWString(), std::iostream::binary);
#else
std::ifstream file(path.toStdString(), std::iostream::binary);
#endif
if (!file.good()) {
return std::shared_ptr<std::vector<char>>(new std::vector<char>());
}
file.exceptions(std::ifstream::badbit | std::ifstream::failbit | std::ifstream::eofbit);
file.seekg(0, std::ios::end);
std::streampos length(file.tellg());
std::shared_ptr<std::vector<char>> buffer(new std::vector<char>(static_cast<std::size_t>(length)));
if (static_cast<std::size_t>(length) == 0) {
return std::shared_ptr<std::vector<char>>(new std::vector<char>());
}
file.seekg(0, std::ios::beg);
try {
file.read(buffer->data(), static_cast<std::size_t>(length));
} catch (...) {
return std::shared_ptr<std::vector<char>>(new std::vector<char>());
}
file.close();
return buffer;
}
And then for reading the image from the buffer:
std::shared_ptr<std::vector<char>> buffer = utility::readFileIntoBuffer(path);
cv::Mat image = cv::imdecode(*buffer, cv::IMREAD_UNCHANGED);
Method 2
QByteArray readFileIntoBuffer(QString const & path) {
QFile file(path);
if (!file.open(QIODevice::ReadOnly)) {
return QByteArray();
}
return file.readAll();
}
And for decoding the image:
QByteArray buffer = utility::readFileIntoBuffer(path);
cv::Mat matBuffer(1, buffer.size(), CV_8U, buffer.data());
cv::Mat image = cv::imdecode(matBuffer, cv::IMREAD_UNCHANGED);
UPDATE
Method 3
This method maps the file into memory using QFileDevice::map and then uses cv::imdecode.
QFile file(path);
file.open(QIODevice::ReadOnly);
unsigned char * fileContent = file.map(0, file.size(), QFileDevice::MapPrivateOption);
cv::Mat matBuffer(1, file.size(), CV_8U, fileContent);
cv::Mat image = cv::imdecode(matBuffer, cv::IMREAD_UNCHANGED);
However, also this approach didn't result in a shorter time than the other two. I also did some time measurements and found that reading the file in the memory or mapping it to the memory is actually not the bottleneck. The operation that takes the majority of the time is the cv::imdecode. I don't know why this is the case, since using cv::imread with the same image only takes a fraction of the time.
Potential Workaround
I tried obtaining an 8.3 pathname on Windows for files that contain non-ascii characters using the following code:
QString getShortPathname(QString const & path) {
#ifndef Q_OS_WIN
return QString();
#else
long length = 0;
WCHAR* buffer = nullptr;
length = GetShortPathNameW(path.toStdWString().c_str(), nullptr, 0);
if (length == 0) return QString();
buffer = new WCHAR[length];
length = GetShortPathNameW(path.toStdWString().c_str(), buffer, length);
if (length == 0) {
delete[] buffer;
return QString();
}
QString result = QString::fromWCharArray(buffer);
delete[] buffer;
return result;
#endif
}
However, I had to find out that 8.3 pathname generation is disabled on my machine, so it potentially is on others as well. So I wasn't able to test this yet and it does not seem to provide a reliable workaround. I also have the problem that the function doesn't tell me that 8.3 pathname generation is disabled.
There is an open ticket on this in OpenCV GitHub: https://github.com/opencv/opencv/issues/4292
One of the comments there suggest a workaround without reading the whole file to memory by using memory-mapped file (with help from Boost):
mapped_file map(path(L"filename"), ios::in);
Mat file(1, numeric_cast<int>(map.size()), CV_8S, const_cast<char*>(map.const_data()), CV_AUTOSTEP);
Mat image(imdecode(file, 1));
I am doing my assignment to read a .rgb video file and display it in the window. I have only known how to read and display an image in C++. What should I do when reading the video and display it frame by frame. I don't want to use third party libraries, just pure C++ and windows programming.
My idea is: firstly load the whole video file into the program using fopen and allocate the buffer for it. Then just like display an image, I wanna treat the whole video as an array of frames, so after rendering the first frame, I will go to the next frame. In addition, how to keep the video display at a constant fps? If you have any learning resources or coding pieces, it would be very helpful!
Thanks
Since you haven't mentioned platform you are using.
But this snippet will help you to read file frame by frame.
#include <stdio.h>
int main()
{
FILE * fp = NULL;
int size = 800 * 600 * 2;
unsigned char * rawData = NULL;
fp = fopen("raw.rgb", "r+b");
rawData = (unsigned char *)malloc(size);
if (NULL == rawData)
return -1;
if (fp)
{
while(!feof(fp))
{
fread(rawData, size, 1, fp);
// GOT FRAME
}
fclose(fp);
fp = NULL;
}
}
Doing this without using any third-party library will be a lot of lot of works!
You may use the OpenCV library to do the work. Check http://opencv.org/