I am trying to integrate a code already written in ROS with some basic Visp lines so as to display a camera feed using Visp functions. I am a beginner in visp and hence I am trying something basic.I am attaching the relevant code lines here
//Lots of lines of code above and blow this code block
cv::Mat src_gray;
cv::cvtColor(imageLeft, src_gray, CV_RGB2GRAY );//imageLeft is a colour image got from the camera through another node
vpImage<unsigned char> I;
vpImageConvert::convert(src_gray,I) ;
vpDisplayOpenCV display;
if(this->lt == false)//this if loop is to prevent from infinite windows coming out
{display.init(I, 100, 100, "Line tracking");
this->lt = true;}
vpDisplay::display(I);
vpDisplay::flush(I);
Let me ensure you that this piece of code is in a callback and hence it is equivalent to an infinte while loop unless the process is stopped.
I am not able to get the camera output in the window.When I run the node the window opens but no image.Any ideas?
The ViSP-ROS interfece has been changing recently. While ViSP Bridge provides low level interface between ROS and ViSP, Visp ROS is a better and higher level interface. From there you can reach to this tutorial where a regular ViSP code is modified to use ROS.
The ViSP code similar to yours:
#include <visp/vp1394TwoGrabber.h>
#include <visp/vpDisplayX.h>
#include <visp/vpImage.h>
int main()
{
#ifdef VISP_HAVE_DC1394_2
try {
vpImage<unsigned char> I; // Create a gray level image container
bool reset = true; // Enable bus reset during construction (default)
vp1394TwoGrabber g(reset); // Create a grabber based on libdc1394-2.x third party lib
g.setVideoMode(vp1394TwoGrabber::vpVIDEO_MODE_640x480_MONO8);
g.setFramerate(vp1394TwoGrabber::vpFRAMERATE_60);
g.open(I);
std::cout << "Image size: " << I.getWidth() << " " << I.getHeight() << std::endl;
#ifdef VISP_HAVE_X11
vpDisplayX d(I);
#else
std::cout << "No image viewer is available..." << std::endl;
#endif
while(1) {
g.acquire(I);
vpDisplay::display(I);
vpDisplay::flush(I);
if (vpDisplay::getClick(I, false))
break;
}
}
catch(vpException e) {
std::cout << "Catch an exception: " << e << std::endl;
}
#endif
}
And the ROS enabled code:
#include <visp/vpDisplayX.h>
#include <visp/vpImage.h>
#include <visp_ros/vpROSGrabber.h>
int main()
{
try {
//vpImage<unsigned char> I; // Create a gray level image container
vpImage<vpRGBa> I; // Create a color image container
vpROSGrabber g; // Create a grabber based on ROS
g.setCameraInfoTopic("/camera/camera_info");
g.setImageTopic("/camera/image_raw");
g.setRectify(true);
g.open(I);
std::cout << "Image size: " << I.getWidth() << " " << I.getHeight() << std::endl;
#ifdef VISP_HAVE_X11
vpDisplayX d(I);
#else
std::cout << "No image viewer is available..." << std::endl;
#endif
while(1) {
g.acquire(I);
vpDisplay::display(I);
vpDisplay::flush(I);
if (vpDisplay::getClick(I, false))
break;
}
}
catch(vpException e) {
std::cout << "Catch an exception: " << e << std::endl;
}
}
Hope this helps!
Related
I'm trying voice to text functions at Visual Studio 2019. I found this code on Microsoft website yet compiler says 'speechapi_cxx.h': No such file or directory.
........................................................................
#include <iostream> // cin, cout
#include <speechapi_cxx.h>
using namespace std;
using namespace Microsoft::CognitiveServices::Speech;
void recognizeSpeech() {
// Creates an instance of a speech config with specified subscription key and service region.
// Replace with your own subscription key and service region (e.g., "westus").
auto config = SpeechConfig::FromSubscription("YourSubscriptionKey", "YourServiceRegion");
// Creates a speech recognizer
auto recognizer = SpeechRecognizer::FromConfig(config);
cout << "Say something...\n";
// Starts speech recognition, and returns after a single utterance is recognized. The end of a
// single utterance is determined by listening for silence at the end or until a maximum of 15
// seconds of audio is processed. The task returns the recognition text as result.
// Note: Since RecognizeOnceAsync() returns only a single utterance, it is suitable only for single
// shot recognition like command or query.
// For long-running multi-utterance recognition, use StartContinuousRecognitionAsync() instead.
auto result = recognizer->RecognizeOnceAsync().get();
// Checks result.
if (result->Reason == ResultReason::RecognizedSpeech) {
cout << "We recognized: " << result->Text << std::endl;
}
else if (result->Reason == ResultReason::NoMatch) {
cout << "NOMATCH: Speech could not be recognized." << std::endl;
}
else if (result->Reason == ResultReason::Canceled) {
auto cancellation = CancellationDetails::FromResult(result);
cout << "CANCELED: Reason=" << (int)cancellation->Reason << std::endl;
if (cancellation->Reason == CancellationReason::Error) {
cout << "CANCELED: ErrorCode= " << (int)cancellation->ErrorCode << std::endl;
cout << "CANCELED: ErrorDetails=" << cancellation->ErrorDetails << std::endl;
cout << "CANCELED: Did you update the subscription info?" << std::endl;
}
}
}
int main(int argc, char** argv) {
setlocale(LC_ALL, "");
recognizeSpeech();
return 0;
}
Speech SDK is not installed on your machine. You may download it here https://learn.microsoft.com/en-us/azure/cognitive-services/speech-service/speech-sdk
I am trying to use a .caffemodel alongside OpenCV on iOS devices. I found this github repository, but it can only be built with Xcode 6. I am working with Xcode 7, but I also downloaded Xcode 6 and still have no success on building it.
How can I use a caffemodel with OpenCV on iOS 9?
PS: The alternative would be this but it's written with swift & metal and I need to be able to use it with OpenCV.
You can use OpenCV DNN contrib module.
You need first to build OpenCV with contrib modules, you can find the steps here.
Then you can import and use the .caffemodel following this tutorial.
Here is an updated version of the tutorial, since it's not working as is:
#include <opencv2/dnn.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/highgui.hpp>
using namespace cv;
using namespace cv::dnn;
#include <fstream>
#include <iostream>
#include <cstdlib>
using namespace std;
/* Find best class for the blob (i. e. class with maximal probability) */
void getMaxClass(dnn::Blob &probBlob, int *classId, double *classProb)
{
Mat probMat = probBlob.matRefConst().reshape(1, 1); //reshape the blob to 1x1000 matrix
Point classNumber;
minMaxLoc(probMat, NULL, classProb, NULL, &classNumber);
*classId = classNumber.x;
}
std::vector<String> readClassNames(const char *filename = "synset_words.txt")
{
std::vector<String> classNames;
std::ifstream fp(filename);
if (!fp.is_open())
{
std::cerr << "File with classes labels not found: " << filename << std::endl;
exit(-1);
}
std::string name;
while (!fp.eof())
{
std::getline(fp, name);
if (name.length())
classNames.push_back( name.substr(name.find(' ')+1) );
}
fp.close();
return classNames;
}
int main(int argc, char **argv)
{
cv::dnn::initModule();
String modelTxt = "bvlc_googlenet.prototxt";
String modelBin = "bvlc_googlenet.caffemodel";
String imageFile = (argc > 1) ? argv[1] : "space_shuttle.jpg";
Ptr<dnn::Importer> importer;
try //Try to import Caffe GoogleNet model
{
importer = dnn::createCaffeImporter(modelTxt, modelBin);
}
catch (const cv::Exception &err) //Importer can throw errors, we will catch them
{
std::cerr << err.msg << std::endl;
}
if (!importer)
{
std::cerr << "Can't load network by using the following files: " << std::endl;
std::cerr << "prototxt: " << modelTxt << std::endl;
std::cerr << "caffemodel: " << modelBin << std::endl;
std::cerr << "bvlc_googlenet.caffemodel can be downloaded here:" << std::endl;
std::cerr << "http://dl.caffe.berkeleyvision.org/bvlc_googlenet.caffemodel" << std::endl;
exit(-1);
}
dnn::Net net;
importer->populateNet(net);
importer.release(); //We don't need importer anymore
Mat img = imread(imageFile);
if (img.empty())
{
std::cerr << "Can't read image from the file: " << imageFile << std::endl;
exit(-1);
}
resize(img, img, Size(224, 224)); //GoogLeNet accepts only 224x224 RGB-images
dnn::Blob inputBlob = dnn::Blob(img); //Convert Mat to dnn::Blob batch of images
net.setBlob(".data", inputBlob); //set the network input
net.forward(); //compute output
dnn::Blob prob = net.getBlob("prob"); //gather output of "prob" layer
int classId;
double classProb;
getMaxClass(prob, &classId, &classProb);//find the best class
std::vector<String> classNames = readClassNames();
std::cout << "Best class: #" << classId << " '" << classNames.at(classId) << "'" << std::endl;
std::cout << "Probability: " << classProb * 100 << "%" << std::endl;
return 0;
} //main
I will post another answer because with recent versions there are some differences.
First of all now dnn is already inside the standard OpenCV library so you do not have to build it from contrib_modules.
The function to load the network is readNetFromCaffe.
For example the following code load the NN:
std::string modelName = "path/to/mymodel.caffemodel";
std::string protoName = "path/to/deploy.prototxt";
cv::dnn::Net net;
try
{
net = cv::dnn::readNetFromCaffe(protoName, modelName);
}
catch (cv::Exception& e)
{
std::cerr << "Exception: " << e.what() << std::endl;
if (net.empty())
{
std::cerr << "Can't load network by using the following files: " << std::endl;
std::cerr << "prototxt: " << protoName << std::endl;
std::cerr << "caffemodel: " << modelName << std::endl;
std::cerr << "bvlc_googlenet.caffemodel can be downloaded here:" << std::endl;
std::cerr << "http://dl.caffe.berkeleyvision.org/bvlc_googlenet.caffemodel" << std::endl;
exit(-1);
}
}
Then you can run the NN:
cv::Mat res_mat;
float res;
cv::Mat inputBlob = cv::dnn::blobFromImage(roi, 1.0f, cv::Size(227, 227),
cv::Scalar(0, 0, 0), false);
net.setInput(inputBlob);
//During the forward pass output of each network layer is computed,
//but in this example we need output from "prob" layer only.
res_mat = net.forward("score");
std::cout<<res_mat<<std::endl;
res_mat = res_mat.reshape(1, 1); //reshape the blob to 1x2 matrix
return res_mat.at<float>(0);
The function cv::dnn::blobFromImage resize the image to the input network size specified by the third argument (in my case cv::Size(227, 227)). The argument cv::Scalar(0, 0, 0) is to subtract the means from the three BGR channels.
score is the name of the output layer in the NN I used. You can see this information in the prototxt file.
I have an Asus XTION camera. On my PC there is Windows 7 with Visual Studio 2013 installed. I also got PCL, openCV compiled for VS2013 and the openNi Driver is intalled. Now I want to get a Point Cloud from the Camera. With the sample I can view the actual frames and save one:
// Original code by Geoffrey Biggs, taken from the PCL tutorial in
// http://pointclouds.org/documentation/tutorials/pcl_visualizer.php
// Simple OpenNI viewer that also allows to write the current scene to a .pcd
// when pressing SPACE.
#define _CRT_SERCURE_NO_WARNINGS
#include <pcl/io/openni2_grabber.h>
#include <pcl/io/pcd_io.h>
#include <pcl/visualization/cloud_viewer.h>
#include <pcl/console/parse.h>
#include <pcl/filters/passthrough.h>
#include <iostream>
using namespace std;
using namespace pcl;
PointCloud<PointXYZRGBA>::Ptr cloudptr(new PointCloud<PointXYZRGBA>); // A cloud that will store color info.
PointCloud<PointXYZ>::Ptr fallbackCloud(new PointCloud<PointXYZ>); // A fallback cloud with just depth data.
PointCloud<PointXYZRGBA>::Ptr filteredCloud(new PointCloud<PointXYZRGBA>);
boost::shared_ptr<visualization::CloudViewer> viewer; // Point cloud viewer object.
Grabber* openniGrabber; // OpenNI grabber that takes data from the device.
unsigned int filesSaved = 0; // For the numbering of the clouds saved to disk.
bool saveCloud(false), noColor(false); // Program control.
void
printUsage(const char* programName)
{
cout << "Usage: " << programName << " [options]"
<< endl
<< endl
<< "Options:\n"
<< endl
<< "\t<none> start capturing from an OpenNI device.\n"
<< "\t-v FILE visualize the given .pcd file.\n"
<< "\t-h shows this help.\n";
}
// This function is called every time the device has new data.
void
grabberCallback(const PointCloud<PointXYZRGBA>::ConstPtr& cloud)
{
if (!viewer->wasStopped())
viewer->showCloud(cloud);
if (saveCloud)
{
stringstream stream;
stream << "inputCloud" << filesSaved << ".pcd";
string filename = stream.str();
// Filter object.
PassThrough<pcl::PointXYZRGBA> filter;
filter.setInputCloud(cloud);
// Filter out all points with Z values not in the [0-2] range.
filter.setFilterFieldName("z");
filter.setFilterLimits(0.0, 1.5);
filter.filter(*filteredCloud);
if (io::savePCDFile(filename, *filteredCloud, true) == 0)
{
filesSaved++;
cout << "Saved " << filename << "." << endl;
}
else PCL_ERROR("Problem saving %s.\n", filename.c_str());
saveCloud = false;
}
}
// For detecting when SPACE is pressed.
void
keyboardEventOccurred(const visualization::KeyboardEvent& event,
void* nothing)
{
if (event.getKeySym() == "space" && event.keyDown())
saveCloud = true;
}
// Creates, initializes and returns a new viewer.
boost::shared_ptr<visualization::CloudViewer>
createViewer()
{
boost::shared_ptr<visualization::CloudViewer> v
(new visualization::CloudViewer("OpenNI viewer"));
v->registerKeyboardCallback(keyboardEventOccurred);
return (v);
}
int
main(int argc, char** argv)
{
if (console::find_argument(argc, argv, "-h") >= 0)
{
printUsage(argv[0]);
return -1;
}
bool justVisualize(false);
string filename;
if (console::find_argument(argc, argv, "-v") >= 0)
{
if (argc != 3)
{
printUsage(argv[0]);
return -1;
}
filename = argv[2];
justVisualize = true;
}
else if (argc != 1)
{
printUsage(argv[0]);
return -1;
}
// First mode, open and show a cloud from disk.
if (justVisualize)
{
// Try with color information...
try
{
io::loadPCDFile<PointXYZRGBA>(filename.c_str(), *cloudptr);
}
catch (PCLException e1)
{
try
{
// ...and if it fails, fall back to just depth.
io::loadPCDFile<PointXYZ>(filename.c_str(), *fallbackCloud);
}
catch (PCLException e2)
{
return -1;
}
noColor = true;
}
cout << "Loaded " << filename << "." << endl;
if (noColor)
cout << "This cloud has no RGBA color information present." << endl;
else cout << "This cloud has RGBA color information present." << endl;
}
// Second mode, start fetching and displaying frames from the device.
else
{
openniGrabber = new pcl::io::OpenNI2Grabber();
if (openniGrabber == 0)
return -1;
boost::function<void(const PointCloud<PointXYZRGBA>::ConstPtr&)> f =
boost::bind(&grabberCallback, _1);
openniGrabber->registerCallback(f);
}
viewer = createViewer();
if (justVisualize)
{
if (noColor)
viewer->showCloud(fallbackCloud);
else viewer->showCloud(cloudptr);
}
else openniGrabber->start();
// Main loop.
while (!viewer->wasStopped())
boost::this_thread::sleep(boost::posix_time::seconds(1));
if (!justVisualize)
openniGrabber->stop();
}
But with this code I need the viewer. How can I get the Point Cloud without the viewer? It should work without kinect SDK, only with PCL, openni.
I solved the problem like this:
#include <pcl/io/openni2_grabber.h>
#include <pcl/io/pcd_io.h>
#include <pcl/console/parse.h>
#include <iostream>
using namespace std;
using namespace pcl;
int main(int argc, char** argv)
{
PointCloud<PointXYZRGBA>::Ptr sourceCloud(new PointCloud<PointXYZRGBA>);
boost::function<void(const PointCloud<PointXYZRGBA>::ConstPtr&)> function = [&sourceCloud](const PointCloud<PointXYZRGBA>::ConstPtr &cloud)
{
copyPointCloud(*cloud, *sourceCloud);
};
// Create Kinect2Grabber
Grabber* grabber = new io::OpenNI2Grabber();
// Regist Callback Function
grabber->registerCallback(function);
// Start Retrieve Data
grabber->start();
boost::this_thread::sleep(boost::posix_time::seconds(1));
// Stop Retrieve Data
grabber->stop();
cout << "The Cloud size: " << sourceCloud->size() << " points ..." << endl;
}
I already wrote a program that detects different objects, and i'm now working on a tracking module to track the objects of interest. Because the detection isn't as fast, I'm hoping to pass in one frame with the detected objects about every 10-30 frames, and have the tracking module use camshift + kalman filtering to track the objects in the video stream until another new image with the detected objects is received from the detection module.
I'm pretty new to boost and c++ multi-threading in general and I wrote the following code just to see if I can be passing in frames from the detection module to the tracking module. For some reason, everything just freezes after the tracking module has received two imags. Any ideas why? Is there a better way to go about this? Thanks!
Main Thread (Detection module might work in a similar manner)
int main(int argc, char *argv[]){
cout << "main started" << endl;
Tracker tracker("Tracker");
tracker.start(imread("/home/cosyco/Desktop/images/lambo1.jpeg", CV_LOAD_IMAGE_COLOR));
boost::posix_time::seconds sleepTime(5);
cout << "main starting sleep" << endl;
boost::this_thread::sleep(sleepTime);
cout << "main just woke up, switching image" << endl;
tracker.resetImage(imread("/home/cosyco/Desktop/images/lambo2.jpeg", CV_LOAD_IMAGE_COLOR));
cout << "main sleeping second time" << endl;
boost::this_thread::sleep(sleepTime);
cout << "main just woke up, switching image" << endl;
tracker.resetImage(imread("/home/cosyco/Desktop/images/lambo3.jpeg", CV_LOAD_IMAGE_COLOR));
cout << "main sleeping third time" << endl;
boost::this_thread::sleep(sleepTime);
cout << "main just woke up, switching image" << endl;
tracker.resetImage(imread("/home/cosyco/Desktop/images/lambo4.jpeg", CV_LOAD_IMAGE_COLOR));
tracker.join();
cout << "main done" << endl;
return 0;
}
Tracker:
#include <iostream>
#include <boost/thread.hpp>
#include <boost/date_time/posix_time/posix_time.hpp>
#include <opencv2/opencv.hpp>
using namespace std;
using namespace cv;
class Tracker{
private:
string name;
Mat trackerImage;
boost::thread trackerThread;
public:
Tracker (string newName){
cout << "creating tracker : " << newName << "created" << endl;
name = newName;
}
void track(Mat image){
image.copyTo(trackerImage);
informUpdate();
// use this new image to generate a histogram for camshift on the next
//few images in the video stream before a new image with the detected object is received
for (; ; ){
// cout << "tracking" << endl;
}
}
void start(Mat image){
cout << "in tracker's start" << endl;
// trackerThread = boost::thread(&Tracker::track, this, trackLimit);
trackerThread = boost::thread(&Tracker::track, this, image);
}
void join(){
trackerThread.join();
}
void resetImage(Mat image){
track(image);
}
void informUpdate(){
cout << "\033[1;31m image updated \033[0m" << endl;
}
};
I am trying to read data from an industrial camera using the V4l linux driver and C++. I would like to display the result using the OpenCV. I read the buffer, create an Mat object, which actually contains values in range 0...255.
The problem seems to be the imshow() call. When commenting this line out, an actual window without an image is displayed. Once uncommented no window is diplayed and also no output in terminal after this line is shown. I am not able to find a solution on my own, all examples I found look the same as my code to me.
Here is the code:
#include <fcntl.h>
#include "opencv/cv.h"
#include "opencv/highgui.h"
#include <libv4l2.h>
#include <libv4l1.h>
#include <linux/videodev2.h>
#include <sys/ioctl.h>
#define BUFFERSIZE 357120 // 744 * 480
using namespace cv;
using namespace std;
int main(int argc, char **argv) {
int cameraHandle, i;
unsigned char pictureBuffer[BUFFERSIZE];
char cameraDevice[] = "/dev/video0";
struct v4l2_control V4L2_control;
/* open camera device */
if (( cameraHandle = v4l1_open(cameraDevice, O_RDONLY)) == -1 ){
printf("Unable to open the camera");
return -1;
}
// disable auto exposure
V4L2_control.id = V4L2_CID_EXPOSURE_AUTO;
V4L2_control.value = V4L2_EXPOSURE_SHUTTER_PRIORITY;
ioctl(cameraHandle, VIDIOC_S_CTRL, &V4L2_control);
// set exposure time
V4L2_control.id = V4L2_CID_EXPOSURE_ABSOLUTE;
V4L2_control.value = 2;
ioctl(cameraHandle, VIDIOC_S_CTRL, &V4L2_control);
// get 5 pictures to warm up the camera
for (i = 0; i <= 5; i++){
v4l1_read(cameraHandle, pictureBuffer, BUFFERSIZE);
}
// show pictures
Mat mat = Mat(744, 480, CV_8UC3, (void*)pictureBuffer);
cout << "M = " << endl << " " << mat << endl << endl; // display the image data
namedWindow("imagetest", CV_WINDOW_AUTOSIZE );
imshow("imagetest", mat);
waitKey(30);
cout << "test output" << endl;
//clenup
v4l1_close(cameraHandle);
destroyWindow("imagetest");
return 0;
}
EDIT:
Well, after running the code in terminal instead of ecipse I saw a segmentation fault Even commenting everything behind the
cout << "M = " << endl << " " << mat << endl << endl;
line gives me this error.
Solved. The problem lied in the wrong file format. CV_8UC1 or CV_8U instead of CV_8UC3 brought and an output. The difference between those formats is described here: In OpenCV, what's the difference between CV_8U and CV_8UC1?