Create Super Resolution image from multiple images using superres - c++

I have a sequence of images and I extract a card from it. After this process the card will be correctly aligned and projected back to a plane (warpPerspective). However the quality is too low to e.g. read text from that card. Thus I tried to use the superres module to increase the resolution, however the documentation is pretty shallow and I have yet to find out how I can pass multiple images to the algorithm.
I tried to implement a custom FrameSource which is basically an adapter to a std::vector but for some reason I get a segfault.
class InterFrameSource : public superres::FrameSource {
std::vector<cv::Mat> frames;
std::vector<cv::Mat>::iterator iter;
public:
InterFrameSource(std::vector<cv::Mat> _frames) : frames(_frames)
{
reset();
}
virtual void nextFrame(OutputArray _frame)
{
_frame.getMatRef().setTo(*iter);
++iter;
}
virtual void reset() {
iter = frames.begin();
}
};
Edit
The cv::Mat are all CPU-only.

OK, after two days I finally got it. I needed to inverse the copying logic:
virtual void nextFrame(OutputArray _frame)
{
if (iter == frames.end()) return;
iter->copyTo(_frame);
++iter;
}

Related

c++ Hold instructions for a later time

I am trying to design a UIDraw method. I want to declare what UI elements to draw in the main Draw method But then Have a separate UIDraw Method later in the code. So I need a way to store instructions to execute in this new function. I hope it makes sense.
Something like this:
Draw();
DrawUI();
But say what UI to draw in the Draw() function.
Any ideas on how to tackle this problem?
There are many ways to tackle this problem depending on what exactly you need. One approach popular in the OO world is the so called Command Pattern (similar approaches exist in other programming paradigms, they just have either different names or are considered so obvious they don't even get a specific name at all).
The basic idea is this: You want to execute some command, but the time you want to execute the command and the time you decide what command to execute are different. So the way to solve this problem is to simply create an object that contains the information you need to execute the command, pass that object to the place that decides when the execution should happen, and then that code can run the command as it pleases.
Here’s a mockup of what that might look like in C++ (note: didn't actually compile this code, might contain minor errors – just meant to convey the idea).
#include <memory>
#include <vector>
/// this is an abstract class that gives us an interface to use
class DrawCommand {
public:
virtual void Draw() = 0;
};
/// one kind of thing you might want to draw
class DrawTree : public DrawCommand {
public:
void Draw() override {
// tree drawing code
}
};
/// another kind of thing you might want to draw
class DrawCat : public DrawCommand {
public:
void Draw() override {
// cat drawing code
}
};
/// we can even come up with ways to combine these in interesting ways
class DrawABunchOfThings : public DrawCommand {
std::vector<std::unique_ptr<DrawCommand>> things;
public:
DrawABunchOfThings(std::vector<std::unique_ptr<DrawCommand>> things)
: things{std::move(things)}
{}
void Draw() override {
for(auto &thing : things) {
thing->Draw();
}
}
};
/// this is where we decide what we will draw
std::unique_ptr<DrawCommand> PrepareDraw() {
if(someCondition) {
// just a cat
return std::make_unique<DrawCat>();
} else if(someOtherCondition) {
// just a tree
return std::make_unique<DrawTree>();
} else {
// forest with a cat hidden inside
return std::make_unique<DrawABunchOfThings>(
std::vector<std::unique_ptr<DrawCommand>>{
std::make_unique<DrawTree>(),
std::make_unique<DrawTree>(),
std::make_unique<DrawCat>()
std::make_unique<DrawTree>(),
}
);
}
}
/// this is where we will do the actual drawing
/// note that any arbitrary amount of code can go between
/// PrepareDraw and ExecuteDraw
void ExecuteDraw(DrawCommand &command) {
// this can of course have a bunch of elaborate
// code here as well -- also, DrawCommand::Draw might
// take extra parameters here, like 2D or 3D transforms,
// time since we last drew something, or whatever
command.Draw();
}
Note that if you only need a single method on this thing C++ already has this in the form of std::function, so you could just say using DrawCommand = std::function<void()>; and be done with it, which would also immediately allow you to use it with lambdas:
int nTimes = 10;
DrawCommand drawNTimesCommand = [nTimes]() {
for(int i = 0; i < nTimes; ++i) {
// draw something
}
};
// --- any code you like here ---
// actually execute the draw command
drawNTimesCommand();

How to call a function for specific objects in a C++ Array?

I hope you can help me because I'm trying to improve my C++ inheritance concepts. First of all, I have the following object hierarchy:
A base class "Image" with two child classes: PNG and JPG (each one with their methods).
I wrote a method inside PNG class which (in a simulated way) deletes alpha channel (it just prints "Alpha channel deleted").
That being said, I want to write a method which takes an array of many pointers to image objects (can be either JPG or PNG) as input, and deletes the alpha channel of only those which are PNG. Since it isn't a good practice in C++ ask for the type of the object (or so I believe), which is the best way to loop through the image array (remembering that it could be filled with either PNG or JPG objects) and delete the alpha channel of only the PNG objects? In which class should I write that method? Parent or child?
I hope I've explained myself correctly, and thank you very much in advance.
The obvious implementation is that the base class has a virtual method for deleting the alpha channel that is a no-op. The PNG class inherits from Image and overrides this member function.
class Image
{
public:
virtual void DeleteAlphaChannel() {};
};
class PNG : public Image
{
void DeleteAlphaChannel() override
{
cout << "Alpha Channel deleted" << endl;
}
};
Then a helper function that takes an array of Image pointers, hence Image**
void DeleteAlphaChannelsFromArrayOfObjects(Image** imageList, int count)
{
for (int i = 0; i < count; i++)
{
imageList[i]->DeleteAlphaChannel();
}
}
And corresponding sample code.
{
PNG pngFile("foo.png");
JPG jpgFile("bar.jpg");
Image* imageList[2] = {&pngFile, &jpgFile};
DeleteAlphaChannelsFromArrayOfObjects(imageList, 2);
}
You could also do a vector style implementation:
void DeleteAlphaChannelsFromArrayOfObjects(std::vector<Image*>& imageList)
{
for (Image* pImage : images)
pImage->DeleteAlphaChannel();
}

Implementing abstract 'overlaps' method for different shapes?

I have an abstract base class called Shape, which looks something like this:
class Shape {
public:
Shape(Point center);
virtual bool overlaps(Shape *other) = 0;
private:
Point m_center; // has getter&setter
};
I'm having problems with the overlaps(Shape *other); method; I have no idea how to implement it in subclasses.
Let's take two examples, (I will probably have no more than two or three shapes) Circle and Rect.
Basically what I've tried is to create a two overloads in both classes after using forward declaration to allow Circle and Rect to "know" each other:
virtual bool Rect::overlaps(Circle *other);
virtual bool Rect::overlaps(Rect *other);
virtual bool Circle::overlaps(Circle *other);
virtual bool Circle::overlaps(Rect *other) { return other->overlaps(this); }
It's now easy to implement the maths inside all the overloads; however, I will get an error cannot allocate an object of abstract type 'Circle' and note: virtual bool Unit::overlaps(Unit *).
This is because my Circle and Rect classes only have methods with Circle * and Rect * as their parameters, but none with Unit *.
I also tried forward declarating Circle and Rect in my shape.h, but since forward declarations aren't the same classes as my actual Circle and Rect, I will only get the same error.
Without removing the common base class, is there a way to implement such behavior?
Or is there a workaround to make it work?
Additional Information
I have a 2D World class which contains vector<Shape *> m_shapes; and I will need to see if two shapes overlap each other;
for (unsigned int i = 0; i < m_shapes.size(); i++) {
if (certainShape->overlaps(m_shapes[i])) {
collapse();
}
}
Welcome to multiple dispatch! Essentially, you are asking for a method that is virtual with respect to the runtime type of more than one object - in your case, the types of two shapes being tested for overlap.
There are several common ways of implementing double dispatch in C++: for example, you could use the visitor pattern, or make a map based on RTTI. Selecting one or the other is up to you.
If you decide to go with the visitor pattern, you make the Shape "visitable" by adding the visit method.
Here is an example of the visitor-based approach. It is admittedly rather verbose, but it also addresses a complex task, so it is fair for it to require lots of code. I stripped the example below to the bare minimum - only two shapes with no data members, and methods that do not do anything except printing. This should be sufficient to get you started, though:
#include <iostream>
using namespace std;
class ShapeVisitor;
struct Shape {
virtual void accept(ShapeVisitor& v) = 0;
virtual bool overlaps(Shape& other) = 0;
};
class Circle;
class Square;
struct ShapeVisitor {
virtual void visitCircle(Circle& c) = 0;
virtual void visitSquare(Square& s) = 0;
};
// These three methods do the actual work
bool checkOverlap(Square& s, Circle& c) {
cout << "Checking if square overlaps circle" << endl;
return false;
}
bool checkOverlap(Square& a, Square& b) {
cout << "Checking if square overlaps square" << endl;
return false;
}
bool checkOverlap(Circle& a, Circle& b) {
cout << "Checking if circle overlaps circle" << endl;
return false;
}
class Square : public Shape {
struct OverlapVisitor : public ShapeVisitor {
OverlapVisitor(Square& _my) : result(false), my(_my) {}
virtual void visitCircle(Circle& c) {
result = checkOverlap(my, c);
}
virtual void visitSquare(Square& s) {
result = checkOverlap(my, s);
}
bool result;
Square& my;
};
public:
virtual void accept(ShapeVisitor& v) {
v.visitSquare(*this);
}
virtual bool overlaps(Shape& other) {
OverlapVisitor v(*this);
other.accept(v);
return v.result;
}
};
class Circle : public Shape {
struct OverlapVisitor : public ShapeVisitor {
OverlapVisitor(Circle& _my) : result(false), my(_my) {}
virtual void visitCircle(Circle& c) {
result = checkOverlap(my, c);
}
virtual void visitSquare(Square& s) {
// Important: note how I switched the order of arguments
// compared to Square::OverlapVisitor! There is only one
// square/circle overlap function checker, and it expects
// the square to be the first argument.
result = checkOverlap(s, my);
}
bool result;
Circle& my;
};
public:
virtual void accept(ShapeVisitor& v) {
v.visitCircle(*this);
}
virtual bool overlaps(Shape& other) {
OverlapVisitor v(*this);
other.accept(v);
return v.result;
}
};
Here is this running demo on ideone.
With RTTI approach you would make a map<pair<type_info,type_info>,checker> where checker is a type of a function that takes two pointers to Shape, and returns true or false depending on whether or not the shapes overlap. You make one such function for each pair of object types, populate the map with pointers to these functions based on type_info of their expected parameter types, and use this map at runtime to call the desired function.
Item 31 of the More Effective C++ book explains both these approaches in depth, with some great examples. In fact, the use case discussed in the book, detecting collisions between a pair of game objects, is similar to the one that you are implementing.
What you need is a "how big is other" type function. If we make it real simple, and just use a bounding box (a rectangle that is big enough to cover the entire shape), then we could do something like this:
(For simplicy, I'm using rect as a term for a rectangle)
class Shape
{
...
virtual rect BoundingBox() = 0;
bool overlaps(const Shape& other)
{
return BoundingBox.FitsInside(other.BoundingBox());
}
};
Obviously, you'll then have to write the function of fitsinside for two rectangles and BoundingBox for each shape, but it shouldn't be too hard.
To make a "is this Star completely covered by this Oval?" makes for a slightly more challenging solution [you will need to have a complete outline of both shapes, and an Oval outline may be quite a lot of points to be precisely oval].
Making the subclasses know about each other is a bad idea. If you want pixel-perfect collision, then you are going to have to iterate through every pixel in the shape and compare with the other shape's pixels. Create a virtual function to get a pixel N from the shape, where N is an index, and another function to return the number of pixels. For each pixel N in the current shape, compare with all pixels 0..Nmax in the other shape for collision.
The order of pixels from the index N can be any order. If you alternate pixels between different sides of the shape over nearby N, and start with the outer pixels first, you may be more likely to detect a collision on a lower N.
Now this simple approach is slow, especially if you have many shapes. The solution is to use a cheaper algorithm to check whether the perfect algorithm is necessary. A rectangle bounding box is the cheapest way. Work out the coordinates of a rectangle which is just large enough to hold your shape. I don't know how to work this out for a circle (geometry not my strong suit). You could even cache the bounding box sizes in the class to prevent recalculation for complex shapes. Checking whether two rectangles overlap is very quick and easy.
Only then move onto the costly algorithm if the bounding boxes overlap.
You can make faster checks between certain pairs of objects. For example, two rectangles overlap if their bounding boxes do. It's overkill to move onto pixel comparison. But you may not need this level of performance.

C++ Implementation of Race Game Tree

I am building a racing game in OpenGL using Glut, and I'm a bit lost in the details. First of all, any suggestions or a road map would be more than great.
So far what I am thinking is this:
Tree implementation for transformations.
Simulated dynamics.(*)
Octree implementation for collision detection.
Actual collision detection.(*)
Modelling in Maya and export them as .OBJs.
Polishing the game with GLSL or something like that for graphics quality.
(*): I am not sure the order of these two.
So I started with the simulated dynamics without a tree implementation, and it turned out to be a huge chaos for me. Is there any way you can think of something that could help me to build such a tree to use in racing game?
I thought of something like this but I have no idea how to implement it.
Reds are static, yellows are dynamic nodes
I would suggest the exact opposite of #bezad.
Start with a single car and an infinite road.
Split the problem of rendering and dynamics into two completely different things. The common Car updates and/or is the link between the CarRenderModel and the CarPhysicsModel.
What shapes the Car puts into the GL scene is up to the Car.
Among other things, this means you can have a really simple Car show up on screen, and attach a really simple physics model to it, and either make the Car prettier or make it behave physically better without having to tie the two together. And, ideally, at each stage you have something playable.
So, a car that is a rectangle, 5 long 3 wide and 1 unit high. A road that is 13 units wide, and goes on forever. A stationary camera. Maybe a horizon. The first physics model is a rocket ship, where every second you push down on an arrow key the car gains x units/second of velocity in that direction. Note that this car doesn't rotate -- it is axis aligned. If the car leaves the road, it explodes, and the "game" ends.
You now have something on the screen that responds to user input. You could spend time making a fancier car model (wheels, etc), or you could improve the car physics and control model (direction! Angle! Breaking != speeding up!), or you could make the environment more interesting (add black-and-white stripes so you can see velocity at the edge of the road. An off-road portion near the road, and maybe trees that blow up the car), or you could make the camera more interesting (say, it stays behind the car, and looks over its shoulder).
Now, for dynamics, I'd treat universe-car interaction using distinct code from car-car interaction, just to keep my sanity intact. The car doesn't get to modify the environment.
This means you can write a bunch of the car-universe interaction easier than you can the car-car interaction.
...
Building an arbitrary tree in C++ is easy.
#include <vector>
#include <memory>
#include <string>
struct MyTree;
typedef std::unique_ptr<MyTree> upTree; // punt on memory management!
struct MyBaseNode;
typedef std::unique_ptr<MyBaseNode> upNode;
struct MyTree {
std::vector<upTree> children;
upNode node;
MyTree( upNode node_ ):node(std::move(node_)) {}
private:
// if C++11 compiler, use these:
MyTree( MyTree const& ) = delete;
MyTree& operator=( MyTree const& ) = delete;
// if C++03, use these:
// MyTree( MyTree const& ); // no implementation
// MyTree& operator=( MyTree const& ); // no implementation
};
upTree make_tree(upNode node) { return upTree( new MyTree(std::move(node)) ); }
enum EOrder{ eFirst, eMiddle, eLast };
template<typename Functor>
void walk_tree( upTree const& tree, Functor f, bool bFirst = true, bool bLast = true) {
if (!tree) return;
f( tree, bFirst, bLast );
for (auto it = tree->children.begin(); it != tree->children.end(); ++it) {
bool bChildFirst = (it == tree->children.begin());
bool bChildLast = ((it+1) == tree->children.end());
walk_tree( *it, f, bChildFirst, bChildLast );
}
}
struct MyBaseNode {
virtual ~MyBaseNode() {};
// put things that your tree nodes have to be able to do here
// as pure virtual functions...
virtual std::string MyName() const = 0;
};
struct CarsNode : MyBaseNode {
// cars node implementation!
virtual std::string MyName() const /*override*/ { return "I am a bunch of CARS!"; }
};
upNode make_cars() { return upNode( new CarsNode() ); }
struct CarNode : MyBaseNode {
// car node implementation!
virtual std::string MyName() const /*override*/ { return "I am a CAR!"; }
};
upNode make_car() { return upNode( new CarNode() ); }
struct RoadNode : MyBaseNode {
// car node implementation!
virtual std::string MyName() const /*override*/ { return "I am a ROAD!"; }
};
upNode make_road() { return upNode( new RoadNode() ); }
#include <iostream>
void tree_printer_func( upTree const& tree, bool bFirst, bool bLast ) {
if (bFirst) std::cout << "[ ";
if (tree->node) {
std::cout << tree->node->MyName().c_str();
} else {
std::cout << "nullNode";
}
if (bLast) {
std::cout << " ]\n";
} else {
std::cout << ", ";
}
}
int main() {
upTree root = make_tree(upNode());
upTree carsTree = make_tree(make_cars());
carsTree->children.push_back( make_tree( make_car() ) );
carsTree->children.push_back( make_tree( make_car() ) );
root->children.push_back( std::move(carsTree) );
upTree roadTree = make_tree(make_road());
root->children.push_back( std::move(roadTree) );
walk_tree( root, tree_printer_func );
}
the above is pretty rough (I didn't get end-node handling quite right in the printer, for example), but it demonstrates a non-homogenious, non-leaking, n-ary tree structure in C++.
If I understand you right, you are asking for ideas on how to go about developing the project (your game). I would start with the smaller objects like the tire, body, other car parts and the car, then the pieces of the environment, the terrain and road track etc. Finally the world and putting all other objects together into the world. Hope it helps.
You might consider posting this on gamdev.stackexchange; you'd probably get better answers there.
That being said, I agree with Michael IV's suggestion in the comment. Your best bet would be to start with a free open source game engine. Even if you want to make your own engine, starting out by building it with an existing engine would give you valuable ideas about how it can be accomplished.
Here is a relevant answered question on gamedev regarding which engine to use.

Accessing an object from a different class - Design

I have three classes, TImageProcessingEngine, TImage and TProcessing
TImageProcessingEngine is the one which i am using to expose all my methods to the world.
TImage is the one i plan to use generic image read and image write functions.
TProcessing contains methods that will perform imaging operations.
class TImageProcessingEngine
{
public:
TImage* mpImageProcessingEngine;
};
class TImage
{
public:
int ReadImage();
int WriteImage();
private:
//a two dimensional array holding the pixel values
tImageMatrix* mpImageMatrix;
};
class TProcessing
{
public:
int ConvertToBinary();
int ConvertToGrayScale();
};
My question is how do i access the object mpImageMatrix in class TProcessing? So that my calling application can use the following
TImageProcessingEngine* vEngine = new TImageProcessingEngine;
//Converts an input gray scsale image to binary image
vEngine->ReadImage().ConvertToBinary();
//Write the converted image to disk
vEngine->WriteImage();
delete vEngine;
vEngine = NULL;
//During this whole processing internally,
//the image is read in to `mpImageMatrix`
//and will also be holding the binarised image data,
//till writing the image to disk.
Or Do you recommend any other approach to my class design?
I would certainly recommend a different implementation, but let's check the design first.
I don't really understand the added value of TImageProcessingEngine, it doesn't bring any functionality.
My advice would be quite simple in fact:
Image class, to hold the values
Processing class (interface), to apply operations
Encoder and Decoder classes (interfaces), to read and write to different formats
It does make sense for the Processing class to have access to the images internal only if you can get efficiency from it (which is likely), in this case you can simply makes Processing friend and having it unpack the values for its derived
class Image
{
public:
Image();
void Accept(Processing& p);
void Encode(Encoder& e) const; // Image is not modified by encoding
void Decode(Decoder& d); // This actually resets the image content
private:
friend class Processing;
size_t mHeight;
size_t mWidth;
std::vector<Pixel> mPixels; // 2D array of Pixels
};
class Processing
{
public:
void apply(Image& image)
{
this->applyImpl(image.mHeight, image.mWidth, image.mPixels);
}
private:
virtual void applyImpl(size_t h, size_t w, std::vector<Pixel>& pixels) = 0;
};
Encoder and Decoder follow the same principle.
Note how I never needed an explicit pointer, and the guaranteed correctness that results from it.
First off, based on your provided code there are no ReadImage() & WriteImage() functions in the TImageProcessingEngine class, so the later code where you use such functionality is flawed.
As for the solution, you can make a getter function for the tImageMatrix pointer like this:
tImageMatrix* GetImageMatrix() { return mpImageMatrix; }
Then just pass that pointer (or a pointer to the whole TImage instance) to the TProcessing function you want to call.
Why you want to have a separate TProcessing process, when it specifically has functions just accessing mpImageMatrix;
In OOP, you have to bind the data members and it's operations..
So, IMO, remove your TProcessing class and have both the functions within TImage..
Your TImage will be like,
class TImage
{
public:
int ReadImage();
int WriteImage();
int ConvertToBinary();
int ConvertToGrayScale();
private:
//a two dimensional array holding the pixel values
tImageMatrix* mpImageMatrix;
};
You could create an accessor TImage class:
byte * pixelAt(unsigned x, unsigned y);