Retrieving video size using QtGStreamer returns ZERO - c++

I'm playing with QtGStreamer 0.10.0 and I'm trying to retrieve the video size but it's returning ZERO for height and width values.
However, I am able to play the video with no problems on a QImage.
QGst::init();
pipeline = QGst::Pipeline::create();
filesrc = QGst::ElementFactory::make("filesrc");
filesrc->setProperty("location", "sample.avi");
pipeline->add(filesrc);
decodebin = QGst::ElementFactory::make("decodebin2").dynamicCast<QGst::Bin>();
pipeline->add(decodebin);
QGlib::connect(decodebin, "pad-added", this, &MyMultimedia::onNewDecodedPad);
QGlib::connect(decodebin, "pad-removed", this, &MyMultimedia::onRemoveDecodedPad);
filesrc->link(decodebin);
// more code ...
The code above shows the begining of the pipeline setup. By connecting my method MyMultimedia::onNewDecodedPad on the signal "pad-added" I have access to the data of the video. At least that's what I think.
void MyMultimedia::onNewDecodedPad(QGst::PadPtr pad)
{
QGst::CapsPtr caps = pad->caps();
QGst::StructurePtr structure = caps->internalStructure(0);
if (structure->name().contains("video/x-raw"))
{
// Trying to print width and height using a couple of different ways,
// but all of them returns 0 for width/height.
qDebug() << "#1 Size: " << structure->value("width").get<int>() << "x" << structure->value("height").get<int>();
qDebug() << "#2 Size: " << structure->value("width").toInt() << "x" << structure->value("height").toInt();
qDebug() << "#3 Size: " << structure.data()->value("width").get<int>() << "x" << structure.data()->value("height").get<int>();
// numberOfFields also returns 0, which is very wierd.
qDebug() << "numberOfFields:" << structure->numberOfFields();
}
// some other code
}
I wonder what I might be doing wrong. Any tips? I was unable to find a relevant example on the web using this API.

Solved it. At onNewDecodedPad() you still don't have access to information about the video frames.
The class MyMultimedia inherits from QGst::Utils::ApplicationSink, so I had to implement a method named QGst::FlowReturn MyMultimedia::newBuffer() that is called by QtGstreamer whenever a new frame is ready.
In other words, use this method to copy the frame of the video to a QImage. What I didn't know is that pullBuffer() returns a QGst::BufferPtr, which has a QGst::CapsPtr. It's an internal structure from this var that holds the information I was looking for:
QGst::FlowReturn MyMultimedia::newBuffer()
{
QGst::BufferPtr buf_ptr = pullBuffer();
QGst::CapsPtr caps_ptr = buf_ptr->caps();
QGst::StructurePtr struct_ptr = caps_ptr->internalStructure(0);
qDebug() << struct_ptr->value("width").get<int>() <<
"x" <<
struct_ptr->value("height").get<int>();
// ...
}

Related

Receiving jpeg through socket and yet unable to display on Qt

I trying to send and receive jpeg images (image size around: 1kb) from my client socket. I have successfully send and receive the image data by checking the size of the data. However when I tried to past the image data to QByteArray and display on QImage, nothing was shown. The code below is a snippet of the receiving and displaying jpeg image.
Server.cpp
memcpy(&Rbuffer, ptr, msgCtl.iMsgLen - 8);
ptr += msgCtl.iMsgLen - 8;
cout << "Size of the image data recieve from the send buffer is " << sizeof(Rbuffer) << endl;
QByteArray ba = QByteArray::fromRawData(Rbuffer, sizeof(Rbuffer));
QBuffer qbuff(&ba);
qbuff.open(QIODevice::ReadOnly);
//QImageReader qimg ("C:\\test.jpg");
QImageReader qimg (&qbuff,"JPG");
qimg.setDecideFormatFromContent(true);
qimg.setDevice(&qbuff);
QImage img = qimg.read();
if (!img.isNull()){
cout << "no problem" << endl;
}
imageView->setPixmap(QPixmap::fromImage(img));
Hope someone could guide me on this. Thank you
On Windows, you have to copy the imageformats directory to your build directory.
eg.
C:\Qt\5.7\msvc2015_64\plugins\imageformats to my_build_dir\imageformats
Try this code:
QPixmap scaledPixmap = QPixmap::fromImage(image).scaled(
QSize(imageWidth, imageHeight),
Qt::IgnoreAspectRatio, Qt::SmoothTransformation);
imageView->setScaledContents(false);
imageView->setPixmap(scaledPixmap);

OpenCV videoCapture gives bad properties for file in iOS app documents directory

I am working on an iOS app that records video then does some processing on that video to create another video file. I'm getting really odd behavior from videoCapture though. It acts like the file has been opened but get() calls seem to return 1 for all the properties I care about (and I know they are not 1). My code looks like this:
VideoCapture cap(fileIn); // open the video file for reading
if ( !cap.isOpened() ) // if not success, exit program
{
cout << "Cannot open the video file" << endl;
return -1;
}
else{
cout << "File Open " << fileIn << endl;
}
double dFps = cap.get(CV_CAP_PROP_FPS); //get the frames per seconds of the video
cout << "Frames per second = " << dFps << endl;
double dWidth = cap.get(CV_CAP_PROP_FRAME_WIDTH); //get the width of frames of the video
double dHeight = cap.get(CV_CAP_PROP_FRAME_HEIGHT); //get the height of frames of the video
cout << "Frame Size = " << dWidth << "x" << dHeight << endl;
Size frameSize(static_cast<int>(dWidth), static_cast<int>(dHeight));
double dFrameCount = cap.get(CV_CAP_PROP_FRAME_COUNT);
cout << "Frame count = " << dFrameCount << endl;
VideoWriter oVideoWriter (fileOut, CV_FOURCC('m','p', '4', 'v'), dFps, frameSize, true); //initialize the VideoWriter object
if ( !oVideoWriter.isOpened() ) //if not initialize the VideoWriter successfully, exit the program
{
cout << "ERROR: Failed to write the video" << endl;
return -1;
}
else{
cout << "Output file open." << endl;
}
and the console output is:
File Open /var/mobile/Containers/Data/Application/7CEE8B87-BD50-4172-8F62-A74AD7F3C45A/Documents/15318B74-07D9-43EE-932F-224EDCA9CDA7.mov
Frames per second = 1
Frame Size = 1x1
Frame count = 1
Output file open.
I tried changing the file path to an invalid file in case it wasn't finding the file at all but that gave me "Cannot open the video file". Any ideas would be appreciated. I'm thinking it could possibly be a permissions issue because I saw similar behavior when I tried to access a video in the camera roll with this code but this file is in my app's documents directory so I should have full access to it.
It appears that this has been an issue for at least the last 2 years (OpenCV on iOS - VideoCapture properties always return 1) and that the iOS version of videocapture simply doesn't work for opening files.
I found an alternate way to do this (opening the video with apple apis then converting each frame to an opencv Mat) but in general it seems that the iOS port of opencv is far from complete and nearly completely undocumented.
First, get the URL of video file from UIImagePickerController object
func imagePickerController(_ picker: UIImagePickerController, didFinishPickingMediaWithInfo info: [UIImagePickerController.InfoKey : Any]) {
picker.dismiss(animated: true, completion: nil)
let url = info[UIImagePickerController.InfoKey.mediaURL] as? NSURL
let asset = AVURLAsset.init(url: url! as URL)
let mainstoryBoard = UIStoryboard(name:"Main", bundle: nil)
let viewcontroller = mainstoryBoard.instantiateViewController(withIdentifier:"ViewController") as! ViewController
viewcontroller.url = url
viewcontroller.asset = asset
self.navigationController?.pushViewController(viewcontroller, animated: true)
}
After that, use this url to send to cvwrapper.mm file as shown in openVideoFile function
- (void)openVideoFile:(NSURL *)fileName {
NSString *filename = fileName.path;
std::string filePath = [filename UTF8String];
cv::VideoCapture capture(filePath);
if (!capture.isOpened()) {
printf("#Error Opening video file");
// failed, print error message
}
else {
printf("File Opened");
}
}

WINDOWPLACEMENT's showCmd... always 1?

When I do a get GetWindowPlacement, the WINDOWPLACEMENT::showCmd seems to be always 1, which is SW_SHOWNORMAL.
Does anyone know why is this so and if it is updated? Does anyone know if this variable is maintained by the application itself or by the operating system?
I am running this on Windows 7.
I am using this to achieve the same purpose as mentioned in this thread: I am trying to undo hidden windows that were previously shown without storing the hidden windows in memory (hide/show will be called in different run sessions) or on disk.
void hide(const unsigned int pid){
std::list<HWND> windowList = getWindowbyPID(pid);
for(std::list<HWND>::iterator it = windowList.begin(); it != windowList.end(); it++){
if(IsWindowVisible(*it)){ std::cout << "Hid WIN#" << *it << std::endl; ShowWindow(*it,SW_HIDE); }
}
}
void show(const unsigned int pid){
std::list<HWND> windowList = getWindowbyPID(pid);
for(std::list<HWND>::iterator it = windowList.begin(); it != windowList.end(); it++){
//if(IsWindowVisible(*it)){ ShowWindow(*it,SW_SHOW); }
WINDOWPLACEMENT wp;
wp.length = sizeof(wp);
wp.showCmd = 0; // Just to clear showCmd before reading.
std::cout << *it << std::endl;
std::cout << "BEFORE: " << wp.showCmd << std::endl;
GetWindowPlacement(*it,&wp);
std::cout << "AFTER: " << wp.showCmd << std::endl;
}
}
Output of one example that I did (pid of notepad.exe) after hiding hwnd#00060CD0:
003D0642
BEFORE: 0
AFTER: 1
000B0682
BEFORE: 0
AFTER: 1
00060CD0
BEFORE: 0
AFTER: 1
I am trying to use GetWindowPlacement to differentiate the windows that were always hidden and the windows that were previously shown. It never seems to be 0 even for windows that were always hidden.
There are only three possible values of the showCmd after calling GetWindowPlacement.
From the MSDN documentation on GetWindowPlacement (emphasis mine):
The flags member of WINDOWPLACEMENT retrieved by this function is always zero. If the window identified by the hWnd parameter is maximized, the showCmd member is SW_SHOWMAXIMIZED. If the window is minimized, showCmd is SW_SHOWMINIMIZED. Otherwise, it is SW_SHOWNORMAL.
Therefore, it appears that the window you're asking for placement info on is in a state other than maximized or minimized when you're calling GetWindowPlacement.
I'd suspect what you're actually looking for is IsWindowVisible.

C++ OIS Segfault When Almost Identical Functions Work

See important edit below!
Hi all I'm having trouble figuring out why this segfault is happening. I'm using the Ogre and OIS library. Here is the code that causes it:
bool Troll::Application::keyPressed(const OIS::KeyEvent& event) {
//TODO: Segfault here!
Troll::State* state = mStateManager->peek();
state->key_pressed(event); //This causes the SEGFAULT!!!
return true;
};
And the key_pressed function:
void Troll::RootState::key_pressed(const OIS::KeyEvent& event) {
std::cout << "You got here" << std::endl; //this isnt printed!
std::cout << "Key Pressed: " << event.key << std::endl;
};
Because the segfault is happening on key_pressed but the first line of key_pressed isn't being executed, I can only guess that it is passing the const OIS::KeyEvent& that is causing it.
And the weird thing about this is I have three other functions that are almost identical (but for the mouse) which work perfectly.
bool Troll::Application::mouseMoved(const OIS::MouseEvent& event) {
mStateManager->peek()->mouse_moved(event);
return true;
};
void Troll::RootState::mouse_moved(const OIS::MouseEvent& event) {
std::cout << "Mouse Moved: rel x = " << event.state.X.rel << std::endl;
std::cout << " rel y = " << event.state.Y.rel << std::endl;
std::cout << " abs x = " << event.state.X.abs << std::endl;
std::cout << " abs y = " << event.state.Y.abs << std::endl;
};
I'm creating a basic state system so I can start writing applications for Ogre3D using the OIS library for input. I have an Application class which acts as an input listener for the mouse and keyboard. Here is how its setup...
void Troll::Application::setup_ois() {
//create a parameter list for holding the window handle data
OIS::ParamList pl;
size_t windowHnd = 0;
//we need the window handle to setup OIS
std::ostringstream windowHndStr;
mWindow->getCustomAttribute("WINDOW", &windowHnd);
windowHndStr << windowHnd;
//add the handle data into the parameter list
pl.insert(std::make_pair(std::string("WINDOW"), windowHndStr.str()));
//create the input system with the parameter list (containing handle data)
mInputManager = OIS::InputManager::createInputSystem(pl);
//true in createInputObject means we want buffered input
mKeyboard = static_cast<OIS::Keyboard*>(mInputManager->createInputObject( OIS::OISKeyboard, true ));
mMouse = static_cast<OIS::Mouse*>(mInputManager->createInputObject( OIS::OISMouse, true ));
//set this as an event handler
mKeyboard->setEventCallback(this);
mMouse->setEventCallback(this);
};
The application class relays the mouse moves, button pressed and key strokes to the Troll::State (the framework I'm making is called Troll) at the top of the state stack which is inside the Troll:: StateManager (which is merely a wrapper for an std::stack with memory allocation and startup() and shutdown() calls)
Sorry for any confusion the difference of the naming conventions causes for some reason I decided to use_underscores_for_some_reason and I haven't got round to changing it. Thanks in advance, ell. Hope you can solve my problem and please inform me if I haven't given enough detail.
EDIT:
After recently upgrading to Ubuntu Natty Narwhal I cannot get the debugger to work properly, it just crashes the computer. I use Code::Blocks and I don't have a clue how to use a debugger or compiler outside the IDE (sad I know, but I'll get round to learning someday). So sorry, I can't use a debugger.
EDIT:
In response to GMan's comment, even if I check for null, I still get segfaults
bool Troll::Application::keyPressed(const OIS::KeyEvent& event) {
//TODO: Segfault here!
Troll::State* state = mStateManager->peek();
if(state == 0) {
std::cout << "State is null!" << std::endl;
};
state->key_pressed(event);
return true;
};
Although I'm not sure thats the correct way to check for null? Also, other methods using peek() work correctly. Thanks again! :)
Important Edit:
It seems that it is in fact the peek function that is causing trouble, but only when called from the keyPressed function. I discovered this by adding a parameter to peek() so that it would print the address of the state object it return as well as the message. By setting the message parameter to the function from where the peek() function is called, I got these results.
Root state is: 0x8fdd470
Peeking state... 0x8fdd470 from: Application::frameRenderingQueued()
Peeking state... 0x8fdd470 from: Application::mouseMoved
Peeking state... 0x8fdd470 from: Application::frameRenderingQueued()
Peeking state... 0x8fdd470 from: Application::frameRenderingQueued()
Peeking state... 0x8fdd470 from: Application::frameRenderingQueued()
Peeking state... 0x936cf88 from: Application::keyPressed
Segmentation fault
Notice that when the keyPressed function calls the peek method, a different address is shown. I cannot see why a different address is being returned only when the keyPress function calls peek? Somebody please help me with this!
What happens when you check for mStateManager being NULL, and for NULL being returned from mStateManager->peek()?
bool Troll::Application::keyPressed(const OIS::KeyEvent& event) {
if (mStateManager == NULL) {
//! set breakpoint on next line
std::cout << "mStateManager is NULL, returning false" << std::endl;
return false;
}
std::cout << "about to call peek" << std::endl;
if (Troll::State* state = mStateManager->peek())
{
std::cout << "about to call key_pressed" << std::endl;
state->key_pressed(event); //Does this still cause a SEGFAULT?
std::cout << "back from key_pressed" << std::endl;
return true;
}
std::cout << "mStateManager->peek() returned NULL, returning false" << std::endl;
return false;
};
EDIT: I edited the code to print each branch, how it was traced through.

Problems capturing audio from the second sound card

I have written a program that captures sound via waveInOpen() in Wuindows. It works great on the michrophone-device on the main-board, but when I try to capture from the second sound-card, I get only [static] noise. Recording with SoundRecorder works great on both cards. Does any1 know if there are any known problems with waveInOpen() and multiple input-devices?
The code that opens the input-device looks like this:
void Audio::OpenDevice(const int device,
const Audio::SamplingRate samplingRate)
throw (Exception, std::exception)
{
switch(samplingRate)
{
...
case AUDIO_16BIT_44KHZ_STEREO:
bits_per_sample_ = 16;
hertz_ = 44100;
channels_ = 2;
break;
...
default:
throw Exception("Audio::OpenDevice(): Invalid enum value");
}
// Open the device
const UINT_PTR dev = (-1 == device) ? (UINT_PTR)WAVE_MAPPER : (UINT_PTR)device;
WAVEFORMATEX wf = {0};
wf.wFormatTag = WAVE_FORMAT_PCM;
wf.nChannels = channels_;
wf.wBitsPerSample = bits_per_sample_;
wf.nSamplesPerSec = hertz_;
wf.nBlockAlign = wf.nChannels * wf.wBitsPerSample / 8;
`
const MMRESULT result = waveInOpen(&hwi_, dev, &wf,
(DWORD_PTR)OnWaveEvent, (DWORD_PTR)this, CALLBACK_FUNCTION);
if (MMSYSERR_NOERROR != result)
throw Exception("waveInOpen()");
std::cout << "Audio: Sampling at " << hertz_ << " hertz from "
<< channels_ << " channel(s) with " << bits_per_sample_
<< " bits per sample. "
<< std::endl;
}
Did you check the microphone gain settings, mixer settings, that the microphone hardware you're using is compatible with the input you have it hooked to, etc? Hooking most microphones to a line in connection does not work well. The microphone doesn't have enough output voltage to drive that kind of input.
My guess (purely a guess) is that the bit depth or sample rate is somehow incorrect. If you are using 16/44100, then I would assume it is supported (pretty common). But maybe the sound card is not set for those rates. I have an external Edirol sound card that I have to physically turn on and off when I change bit depth (and adjust a separate switch on it).