Collecting pixel values in a line drawn using mouse - c++

I have an image (24 bit bmp) such as the following:
The user draws a line (shown here in red) using mouse. This line can be any where with any angle. Then he clicks right or left mouse button, and the image pixel values across the lines are stored in a file in addition to displaying on the console.
I have used setMouseCallback() for showing the position of the mouse (shown below ). But I need a little more help in understanding an elegant way for finding and storing the pixel values across the line. Kindly help!
void CallBackFunc(int event, int x, int y, int flags, void* userdata)
{
if ( event == EVENT_LBUTTONDOWN )
{
cout << "Left button of the mouse is clicked - position (" << x << ", " << y << ")" << endl;
}
else if ( event == EVENT_RBUTTONDOWN )
{
cout << "Right button of the mouse is clicked - position (" << x << ", " << y << ")" << endl;
}
else if ( event == EVENT_MOUSEMOVE )
{
cout << "Mouse move over the window - position (" << x << ", " << y << ")" << endl;
}
}
int main(int argc, char** argv)
{
Mat img = imread("C:\\Users\\Acme\\Desktop\\image-processing\\2.bmp");
namedWindow(" Window", 1);
setMouseCallback(" Window", CallBackFunc, NULL);
imshow(" Window", img);
waitKey(0);
return 0;
}

Extract the line by warping it to a 1 x (linelength) or (linelength) x 1, vertical or horizontal Mat. Then you can easily read down or across pixel values.

The specific details depend upon your program, but values is populated once two points are clicked. What you do after that is up to you.
cv::Point g_points[2];
int g_pointIndex;
std::vector<cv::Vec3b> values;
bool g_allGood = false;
void onMouse(int e, int x, int y, int d, void* ud)
{
if (e != CV_EVENT_LBUTTONDOWN || g_pointIndex >= 2)
return;
g_points[g_pointIndex++] = cv::Point(x, y);
}
void main()
{
// load image as greyscale
Mat img = imread("C:\\temp\\2.png", CV_8UC1);
namedWindow("img", 1);
setMouseCallback("img", onMouse);
while (1)
{
// all points collected
if (g_pointIndex == 2 && !g_allGood)
{
/*
to save processing, i suggest you remove the mouse callback once all points
are collected. do this with: setMouseCallback("img", NULL,NULL);
*/
// create line iterator, and add pixel values to values vector
cv::LineIterator it(img, g_points[0], g_points[1]);
for (int i = 0; i < it.count; i++, ++it)
values.push_back((Vec3b)*it);
// you now have all pixel values in values;
g_allGood = true;
}
imshow("img", img);
waitKey(100);
}
}

Related

Problem with getting pixel information using SetMouseCallback, c++, OpenCV

I've been trying to get information about pixels that I click on. I've been using function SetMouceCallback and honestly I don't know exactly how it works.
void startGUI()
{
VideoCapture video;
Mat frame1;
video.open("/Users/Dominik/Desktop/Legia22.mov");
video.read(frame1);
Mat img = frame1.clone();
namedWindow("My Window", 1);
setMouseCallback("My Window", onMouse, &img);
imshow("My Window", img);
//********* I want to use leftClicks[] here
waitKey(0);
}
static void onMouse(int event, int x, int y, int, void* param)
{
Mat &img = *((Mat*)param);
static int counter=0;
Point leftClicks[4];
if ( event == EVENT_LBUTTONDOWN && counter<4)
{
Vec3b val = img.at< Vec3b >(y,x);
cout << "left button clicked pos - (" << x << ", " << y << ")" <<"\t colour bgr: "<< val << endl;
leftClicks[counter].x = x;
leftClicks[counter].y = y;
circle(img, Point(x,y), 4, SCALAR_BLUE, -1);
imshow("My Window", img);
counter++;
}
}
The following code works fine. Window is shown and on the first frame I can draw 4 circles. The points are printed in the terminal.
So basically what I'm trying to do is to get the array leftClicks[4] out of the function for further processing in other functions, eg. where the ****** are.
The problem is that the type of function onMouse has to be void and the arguments can't be changed.
I noticed that the function setMouseCallback has pretty specified what onMouse function has to be and no changes to the function onMouse could be made...
So how can I get the array leftClicks[] out of the function onMouse?

How to draw a line on frame from live video feed in opencv c++

I want to draw a line on opencv frame in c++. For this I have below code where I am using setMouseCallback(winName, onMouse, NULL);. In below code I am using image:
Mat src;
const char* winName = "Image";
int start_x = 0;
int start_y = 0;
bool run_once = false;
void onMouse(int event, int x, int y, int f, void*)
{
if (cv::EVENT_LBUTTONDOWN)
{
if (f == 1)
{
if (run_once == false)
{
start_x = x;
start_y = y;
cout << "start x,y is : " << x << y << endl;
run_once = true;
}
}
if (f == 3)
{
cout << "start x,y is : " << start_x << start_y << endl;
int end_x = x;
int end_y = y;
cout << "end x,y is : " << end_x << end_y << endl;
cv::line(src, cv::Point(start_x, start_y), cv::Point(end_x, end_y), Scalar(255), 2, 8, 0);
imshow(winName, src);
run_once = false;
}
}
}
int main()
{
src = imread(<img path>, 1);
namedWindow(winName, WINDOW_NORMAL);
setMouseCallback(winName, onMouse, NULL);
imshow(winName, src);
while(1)
{
}
}
Using above code, if I left click using mouse on frame, it records start_x start_y. I drag my mouse to left/right and then right click and it records end_x end_y and simply draws a line and display it. This works fine but I want to achieve this functionality in live video frame.
The issue which I am facing in live video frame is that, in live video feed we are constantly displaying the frame in while(1) loop thus the line which is drawn is removed in next frame
void onMouse(int event, int x, int y, int f, void*)
{
/*
* SAME AS ABOVE
*/
}
int main(int, char**)
{
VideoCapture cap(1); // open the default camera
if (!cap.isOpened()) // check if we succeeded
return -1;
namedWindow(winName, WINDOW_NORMAL);
setMouseCallback(winName, onMouse, NULL);
for (;;)
{
cap >> src; // get a new frame from camera
imshow(winName, src);
if (waitKey(30) >= 0) break;
}
// the camera will be deinitialized automatically in VideoCapture destructor
return 0;
}
In above code, we have onMouse function where we are using imshow to show the line drawn on frame but we also have imshow in while(1) loop thus the drawn line is not shown.
Is there anyway I can draw line on live video feed frame. Please help. Thanks
As #Sunreef suggested, you can create separate cv::Mat to keep picture with lines only and display src combined with this picture
// #0 NEW - add declaration of lines here so this Mat is visible in both onMouse and main scope
cv::Mat src, lines;
void onMouse(int event, int x, int y, int f, void*)
{
if (f == 3)
{
cout << "start x,y is : " << start_x << start_y << endl;
int end_x = x;
int end_y = y;
cout << "end x,y is : " << end_x << end_y << endl;
// #1 NEW - draw line into lines instead of src
cv::line(lines, cv::Point(start_x, start_y), cv::Point(end_x, end_y), Scalar(255), 2, 8, 0);
// #2 NEW - remove unnecessary imshow here
run_once = false;
}
}
int main(int, char**)
{
for (;;)
{
cap >> src;
// #3 NEW - init lines once, to be the same size, same type as src filled with zeros
if(lines.empty()) lines = cv::Mat::zeros(src.size(), src.type());
// #4 NEW - show lines combined with lines
imshow(winName, lines + src);
if (waitKey(30) >= 0) break;
}
return 0;
}
This way only lines image is updated in onMouse (#1). No need to show it in onMouse event (#2) since it will be displayed anyway in the main loop (#4). But before you actualy display lines you overlay (add) them to the src image. The only tricky part is to initialize lines as soon as you know the size and type of src (#3). And remember to declare the lines first (#0) so that the image is visible globally, just like src.
I also suggest getting familiar with:
cv::add - this is basically what operator+ does
cv::Mat::type
cv::Mat::zeros
cv::Mat::empty
cv::line

I want to make setmousecallback function to pass the coordinate continuously when mouse left button is still pressed

My code works fine but it is not continuously pass the co-ordinates when the left button of mouse is still pressed. It does pass the co-ordinates when mouse is moving.
Sorry i didn't add the mouse_callback function, I now have added that.
Any help would be helpful.
code:
#include <stdio.h>
#include <opencv2/opencv.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/imgproc.hpp>
#include <iostream>
using namespace cv;
using namespace std;
Point pt(-1, -1);
bool newCoords = false;
void mouse_callback(int event, int x, int y, int flag, void *param)
{
if (event == EVENT_LBUTTONDOWN)
{
pt.x = x;
pt.y = y;
newCoords = true;
}
else if (flag == EVENT_FLAG_LBUTTON && event == EVENT_MOUSEMOVE) {
pt.x = x;
pt.y = y;
newCoords = true;
}
}
int main(int argc, char** argv)
{
String WindowName = "Original Feed";
namedWindow(WindowName, CV_WINDOW_AUTOSIZE);
Mat oriImg;
int Frame_Width = 720;
int Frame_Height = 540;
VideoCapture cap(0);
setMouseCallback(WindowName, mouse_callback);
cap.set(CV_CAP_PROP_FRAME_WIDTH, Frame_Width);
cap.set(CV_CAP_PROP_FRAME_HEIGHT, Frame_Height);
while (waitKey(30) != 27) {
cap.read(oriImg);
if (pt.x != -1 && pt.y != -1)
{
circle(oriImg, pt, 3, Scalar(0, 0, 255));
if (newCoords)
{
std::cout << "Clicked coordinates: " << pt << std::endl;
newCoords = false;
}
}
imshow(WindowName, oriImg);
}
return 0;
}
The callback is called whenever the OS/GUI generates a mouse event. Such events are generated whenever the state of the mouse (button, wheel, position, etc.) change -- if nothing changes, you already know (are can know) the current state.
If you want to somehow respond to a stable state of the mouse (e.g. button held down while stationary) in multiple iterations, you will have to handle it yourself.
The way I understand your problem is as follows: As long as the left button is held, draw the circle at the most recent position of the cursor on every frame displayed.
Since we're running at a decent frame rate, we can afford to keep things simple, and only update the current frame based on the click/position information from the previous frame. The user will not be able to observe this slight delay.
Just for posterity: the image is being displayed, and the mouse callback called only while the cv::waitKey function is running.
The following enumerates the situations we may encounter in each iteration and how we should react to them in the following frame:
LMB was not held at entry and exit of waitKey
LMB was never released (therefore never pressed) -> nothing to do
LMB was held and released at least once (and therefore pressed as many times) -> draw circle at position of last depress
LMB was not held at entry, but held at exit of waitKey
LMB is held down -> draw circle at last reported mouse position
LMB was held at entry, but not held at exit of waitKey
LMB was held and released at least once (and pressed one less time) -> draw circle at position of last depress
LMB was held at entry and exit of waitKey
LMB is held down -> draw circle at last reported mouse position
Based on this, we need to keep track of:
The last position where the LMB was either held down, or released.
Whether the LMB is currently being held down
Whether the LMB was released in the previous iteration (i.e. it was held down, even though it no longer is)
We can use a struct to hold this info (which we let the callback use thanks to the user data parameter):
struct mouse_state
{
mouse_state()
: position(-1, -1)
, left_button_held(false)
, left_button_clicked(false)
{}
void new_iteration()
{
left_button_clicked = false;
}
cv::Point2i position; // Last position where the LMB was down
bool left_button_held; // Is the LMB down right now?
bool left_button_clicked; // Was the LMB down in the last iteration?
};
The mouse callback will be responsible for updating the above struct.
When user presses LMB, record position and set button held
When user releases LMB, record position, set button clicked, and set button not held
When mouse moves and LMB held, record position
Sample implementation of the callback with some primitive debug tracing could look as such:
void mouse_callback(int event, int x, int y, int flag, void* param)
{
mouse_state* state(static_cast<mouse_state*>(param));
if (event == cv::EVENT_LBUTTONDOWN) {
std::cout << "LMB down # (" << x << "," << y << ")\n";
state->position = cv::Point2i(x, y);
state->left_button_held = true;
} else if (event == cv::EVENT_LBUTTONUP) {
std::cout << "LMB up #(" << x << "," << y << ")\n";
state->position = cv::Point2i(x, y);
state->left_button_held = false;
state->left_button_clicked = true;
} else if ((flag == cv::EVENT_FLAG_LBUTTON) && (event == cv::EVENT_MOUSEMOVE)) {
std::cout << "LMB held, mouse moved to (" << x << "," << y << ")\n";
state->position = cv::Point2i(x, y);
}
}
The callback then could be registered in the following manner:
mouse_state ms;
cv::setMouseCallback(WINDOW_NAME, mouse_callback, &ms);
The processing loop would then be quite simple:
Read new frame, exit loop on failure
Check last mouse state. If LMB is held, or was clicked, draw circle at recorded position.
Reset per-iteration mouse state flags
Show image, update mouse state info
This might look as follows:
for (;;) {
// We can have `image` here, since `cap.read` always gives us
// a reference to its internal buffer
cv::Mat image;
if (!cap.read(image)) {
std::cerr << "Failed to read image, exiting...\n";
break; // Failed to read image, nothing else to do
}
if (ms.left_button_clicked || ms.left_button_held) {
std::cout << "Current position: " << ms.position << "\n";
cv::circle(image, ms.position, 3, cv::Scalar(0, 0, 255));
cv::circle(image, ms.position, 10, cv::Scalar(0, 0, 255), 2);
}
ms.new_iteration();
cv::imshow(WINDOW_NAME, image);
if (cv::waitKey(30) == 27) {
std::cout << "Esc pressed, exiting...\n";
break;
}
}
Sample Program
#include <opencv2/opencv.hpp>
#include <iostream>
struct mouse_state
{
mouse_state()
: position(-1, -1)
, left_button_held(false)
, left_button_clicked(false)
{}
void new_iteration()
{
left_button_clicked = false;
}
cv::Point2i position; // Last position where the LMB was down
bool left_button_held; // Is the LMB down right now?
bool left_button_clicked; // Was the LMB down in the last iteration?
};
void mouse_callback(int event, int x, int y, int flag, void* param)
{
mouse_state* state(static_cast<mouse_state*>(param));
if (event == cv::EVENT_LBUTTONDOWN) {
std::cout << "LMB down # (" << x << "," << y << ")\n";
state->position = cv::Point2i(x, y);
state->left_button_held = true;
} else if (event == cv::EVENT_LBUTTONUP) {
std::cout << "LMB up #(" << x << "," << y << ")\n";
state->position = cv::Point2i(x, y);
state->left_button_held = false;
state->left_button_clicked = true;
} else if ((flag == cv::EVENT_FLAG_LBUTTON) && (event == cv::EVENT_MOUSEMOVE)) {
std::cout << "LMB held, mouse moved to (" << x << "," << y << ")\n";
state->position = cv::Point2i(x, y);
}
}
int main(int argc, char** argv)
{
cv::String const WINDOW_NAME("Original Feed");
cv::namedWindow(WINDOW_NAME, CV_WINDOW_AUTOSIZE);
mouse_state ms;
cv::setMouseCallback(WINDOW_NAME, mouse_callback, &ms);
int const FRAME_WIDTH(720);
int const FRAME_HEIGHT(540);
cv::VideoCapture cap(0);
if (!cap.isOpened()) {
std::cerr << "Unable to open camera, exiting...\n";
return -1;
}
cap.set(CV_CAP_PROP_FRAME_WIDTH, FRAME_WIDTH);
cap.set(CV_CAP_PROP_FRAME_HEIGHT, FRAME_HEIGHT);
for (;;) {
// We can have `image` here, since `cap.read` always gives us
// a reference to its internal buffer
cv::Mat image;
if (!cap.read(image)) {
std::cerr << "Failed to read image, exiting...\n";
break; // Failed to read image, nothing else to do
}
if (ms.left_button_clicked || ms.left_button_held) {
std::cout << "Current position: " << ms.position << "\n";
cv::circle(image, ms.position, 3, cv::Scalar(0, 0, 255));
cv::circle(image, ms.position, 10, cv::Scalar(0, 0, 255), 2);
}
ms.new_iteration();
cv::imshow(WINDOW_NAME, image);
if (cv::waitKey(30) == 27) {
std::cout << "Esc pressed, exiting...\n";
break;
}
}
return 0;
}

Create multiple ROI's using mouse click

My goal is to create a vector<Rect>, right now I'm starting with only a single Rect, containing any number of ROI's that the user creates within a frame, initially captured from the videocamera. The ROI is created by grabbing points when the mouse is pressed and lifted up.
I know that the code to populate the ROI is working in onMouse using commented out cout. If I try to print out the ROI in the while loop of the setup function it is empty. Did I mess up the pointers or is it something else?
using namespace std;
using namespace cv;
//Global Variables
Point P1(0,0);
Point P2(0,0);
bool complete = false;
struct MouseParams
{
Mat img;
Point pt;
int lucky;
};
void onMouse ( int event, int x, int y, int d, void *ptr ){
switch(event){
//mouse pressed (get coordinate of first point)
case CV_EVENT_LBUTTONDOWN:
cout << "press" << endl;
P1.x=x;
P1.y=y;
break;
//mouse lifted (get coordinate of final point)
case CV_EVENT_LBUTTONUP:
cout << "lift" << endl;
P2.x=x;
P2.y=y;
complete = true;
break;
default:
break;
}
//If lifted mosue and have two valid points()
if(complete && P1 != P2){
cout << "Building ROI" << endl;
Rect*ROI = (Rect*)ptr;
int w,h;
h= P2.y-P1.y;
w = P2.x-P1.x;
ROI->x=x;
ROI->y=y;
ROI->width=w;
ROI->height=h;
cout << ROI << endl;
complete = false;
}
}
Rect setup(Mat frame){
Rect ROI;
while(true){
namedWindow("ROI");
setMouseCallback("ROI",onMouse, &ROI);
cout << ROI << endl;
//draw ROI grabbed in onMouse
rectangle( frame, ROI, Scalar(0,255,0), 1, 8, 0 );
imshow("ROI",frame);
moveWindow("ROI", 0,50);
waitKey(0);
return ROI;
}
}
UPDATE: Modify the waitkey() as it was holding up the while loop waiting for an input to move forward. Now code works for retaining the latest ROI (can draw many in setup frame) but need to adapt to retain vector but feel like that is hard to do using pointers.
key = (char)waitKey(0); // explicit cast
if (key == 27) break; // break if `esc' key was pressed.
if (key == ' ') continue;;
}
destroyAllWindows();
return ROI;

using mouse as paint brush

So this is my attempt at setting MouseEvent in openCV, my point is to draw some thing at given point according to mouse event. Any idea what's wrong?
I followed this tutorials: https://docs.opencv.org/3.3.0/db/d5b/tutorial_py_mouse_handling.html
#include<opencv2\core\core.hpp>
#include<opencv2\imgproc\imgproc.hpp>
#include<opencv2\highgui\highgui.hpp>
#include<opencv2\ml\ml.hpp>
#include<iostream>
#include<conio.h>
using namespace std;
using namespace cv;
int radius = 5;
Mat img;
int ix = 1;
int iy = 1;
//mouse callback function
void drawCircle(int event, int x, int y, int, void* param) {
if (event == CV_EVENT_LBUTTONDOWN)
{
cout << "x = " << x
<< "\t y = " << y
<< endl;
circle(img, Point(x, y), 10, Scalar(255,255,255), 10);
}
//line(img, Point(10, 10), Point(x, y), Scalar(0, 0, 255), 3);
}
int main() {
img = imread("image.png");
if (img.empty()) {
cout << "\nerror reading image" << endl;
_getch();
return -1;
}
namedWindow("Image",1);
imshow("Image", img);
setMouseCallback("Image", drawCircle);
waitKey(0);
return 0;
}
Why doesn't it draw circle or line on my image? Thanks!
You probably need to force a redraw of the window to reflect the changes. I.e. replace your waitKey(0) by something like:
while (waitKey(20) != 27) // wait until ESC is pressed
{
imshow("Image", img);
}
Alternatively, you may be able to just add a call to imshow to the drawCircle callback.