for quite some time I've been working on an application. As programming is just a hobby this project is already taking way too long, but that's besides the point. I'm now at a point where every "problem" becomes terribly difficult to solve. And I'm thinking of refactoring the code, however that would result in a "complete" rewrite.
Let me explain the problem, and how I solved it currently. Basically I have data, and I let things to happen over this data (well I described about every program didn't I?). What happens is:
Data -> asks viewer to display -> viewer displays data based on actual data
viewer returns user input -> data -> asks "executor" to execute it -> new data
Now this used to work very well, and I was thinking originally "hey I might for example change command prompt by qt, or windows - or even take that external (C#) and simply call this program".
However as the program grew it became more and more tiresome. As the most important thing is that the data is displayed in different manners depending on what the data is and -more importantly- where it is located. So I went back to the tree & added someway to "track" what the parent-line is". Then the general viewer would search for the most specific actual widget.
It uses has a list with [location; widget] values, and finds the best matching location.
The problems starts when updating for new "data" - I have to go through all the assets - viewer, saver etc etc. Updating the check-mechanism gave me a lot of errors.. Things like "hey why is it displaying the wrong widget now again?".
Now I can completely swap this around. And instead of the tree datastructure calling to a generic viewer. I would use OO "internal" tree capabilities. The nodes would be childs (& when a new viewer or save-mechanism is needed a new child is formed).
This would remove the difficult checking mechanism, where I check the location in the tree. However it might open up a whole other can of worms.
And I'd like some comments on this? Should I keep the viewer completely separate - having difficulty checking for data? Or is the new approach better, yet it combines data & execution into a single node. (So if I wish to change from qt to say cli/C# it becomes almost impossible)
What method should I pursue in the end? Also is there something else I can do? To keep the viewer separate, yet prevent having to do checks to see what widget should be displayed?
EDIT, just to show some "code" and how my program works. Not sure if this is any good as I said already it has become quite a clusterfuck of methodologies.
It is meant to merge several "gamemaker projects" together (as GM:studio strangely lacks that feature). Gamemaker project files are simply sets of xml-files. (Main xml file with only links to other xml files, and an xml file for each resource -object, sprite, sound, room etc-). However there are some 'quirks' which make it not really possible to read with something like boost property trees or qt: 1) order of attributes/child nodes is very important at certain parts of the files. and 2) white space is often ignored however at other points it is very important to preserve it.
That being said there are also a lot of points where the node is exactly the same.. Like how a background can have <width>200</width> and a room too can have that. Yet for the user it is quite important which width he is talking about.
Anyways, so the "general viewer" (AskGUIFn) has the following typedefs to handle this:
typedef int (AskGUIFn::*MemberFn)(const GMProject::pTree& tOut, const GMProject::pTree& tIn, int) const;
typedef std::vector<std::pair<boost::regex, MemberFn> > DisplaySubMap_Ty;
typedef std::map<RESOURCE_TYPES, std::pair<DisplaySubMap_Ty, MemberFn> > DisplayMap_Ty;
Where "GMProject::pTree" is a tree node, RESOURCE_TYPES is an constant to keep track in what kind of resource I am at the moment (sprite, object etc). The "memberFn" will here simply be something that loads a widget. (Though AskGUIFn is not the only general viewer of course, this one is only opened if other "automatic" -overwrite, skip, rename- handlers have failed).
Now to show how these maps are initialized (everything in namespace "MW" is a qt widget):
AskGUIFn::DisplayMap_Ty AskGUIFn::DisplayFunctionMap_INIT() {
DisplayMap_Ty t;
DisplaySubMap_Ty tmp;
tmp.push_back(std::pair<boost::regex, AskGUIFn::MemberFn> (boost::regex("^instances "), &AskGUIFn::ExecuteFn<MW::RoomInstanceDialog>));
tmp.push_back(std::pair<boost::regex, AskGUIFn::MemberFn> (boost::regex("^code $"), &AskGUIFn::ExecuteFn<MW::RoomStringDialog>));
tmp.push_back(std::pair<boost::regex, AskGUIFn::MemberFn> (boost::regex("^(isometric|persistent|showcolour|enableViews|clearViewBackground) $"), &AskGUIFn::ExecuteFn<MW::ResourceBoolDialog>));
//etc etc etc
t[RT_ROOM] = std::pair<DisplaySubMap_Ty, MemberFn> (tmp, &AskGUIFn::ExecuteFn<MW::RoomStdDialog>);
tmp.clear();
//repeat above
t[RT_SPRITE] = std::pair<DisplaySubMap_Ty, MemberFn>(tmp, &AskGUIFn::ExecuteFn<MW::RoomStdDialog>);
//for each resource type.
Then when the tree datastructure tells the general viewer it wishes to be displayed the viewer executes the following function:
AskGUIFn::MemberFn AskGUIFn::FindFirstMatch() const {
auto map_loc(DisplayFunctionMap.find(res_type));
if (map_loc != DisplayFunctionMap.end()) {
std::string stack(CallStackSerialize());
for (auto iter(map_loc->second.first.begin()); iter != map_loc->second.first.end(); ++iter) {
if (boost::regex_search(stack, iter->first)) {
return iter->second;
}
}
return map_loc->second.second;
}
return BackupScreen;
}
And this is where the problems began to be frank. The CallStackSerialize() function depends on a call-stack.. However that call_stack is stored inside a "handler". I stored it there because everything starts FROM a handler. I'm not really sure where I ought to store this "call_stack". Introduce another object that keeps track of what's going on?
I tried going the route where I store the parent with the node itself. (Preventing the need for a call-stack). However that didn't go as well as I wished: each node simply has a vector containing its child nodes. So using pointers is out of the question to point to the parent note...
(PS: maybe I should reform this in another question..)
Refactoring/rewriting this complicated location checking mechanism out of the viewer into a dedicated class makes sense, so you can improve your solution without affecting the rest of your program. Lets call this NodeToWidgetMap.
Architecture
Seems your heading towards a Model-View-Controller architecture which is a good thing IMO. Your tree structure and its nodes are the models, where as the viewer and the "widgets" are views, and the logic selecting widgets depending on the node would be part of a controller.
The main question remains when and how you choose the widget wN for a given node N and how to store this choice.
NodeToWidgetMap: When to choose
If you can assume that wN does not change during its lifetime even though nodes are moved, you could choose it right when creating the node . Otherwise you'll need to know the location (or path through the XML) and, in consequence, find the parent of a node when requesting it.
Finding Parent Nodes
My solution would be to store pointers to instead of the node instances themselves, perhaps using boost::shared_ptr. This has drawbacks, for example copying nodes forces you to implement your own copy-constructors that uses recursion to create a deep-copy of your sub-tree. (Moving however will not affect the child nodes.)
Alternatives exist, such as keeping child nodes uptodate whenever touching the parent node respective the grandfathers vector. Or you can define a Node::findParentOf(node) function knowing that certain nodes can only (or frequently) be found as child of certain nodes. This is brute but will work reasonably well for small trees, just does not scale very well.
NodeToWidgetMap: How to choose
Try writing down the rules how to choose wN on piece of paper, perhaps just partially. Then try to translate these rules into C++. This might slightly longer in terms of code but will be easier to understand and maintain.
Your current approach is to use regular expressions for matching the XML path (stack).
My idea would be to create a lookup graph whose edges are labelled by the XML element names and whose nodes indicate which widget shall be used. This way your XML path (stack) describes a route through the graph. Then the question becomes whether to explicitly model a graph or whether a group of function calls could be used to mirror this graph.
NodeToWidgetMap: Where to store choice
Associating a unique, numeric id to each node, record the widget choice using a map from node id to widget inside the NodeToWidgetMap.
Rewriting vs Refactoring
If you rewrite you might get good leverage tieing to an existing framework such as Qt in order to focus on your program instead of rewriting the wheels. It can be easier to port a well-written program from on framework to another than to abstract around the pecularities of each platform. Qt is a nice framework for gaining experience and good understanding of the MVC-architectures.
A complete rewrite gives you a chance to rethink everything but implies the risk that you start from scratch and will be without a new version for a good amount of time. And who knows whether you will have enough time to finish? If you choose to refactor the existing structures you will improve it step by step, having a useable version after each step. But there is small risk to remain trapped in old ways of thinking, where as rewriting nearly forces you to rethink everything. So both approaches have their merits, if you enjoy programming I would rewrite. More programming, more joy.
Welcome to the world of programming!
What you describe is the typical life cycle of an application, starts as a small simple app, then it gets more and more features until it is no longer maintainable. You can't imagine how many projects I've seen in this last collapsing phase!
Do you need to refactor? Of course you do! All the time! Do you need to rewrite everything? Not so sure.
In fact the good solution is to work by cycles: you design what you need to code, you code it, you need more functionality, you design this new functionality, you refactor the code so you can integrate the new code, etc. If you don't do it like this then you will arrive to the point where its less expensive to rewrite then to refactor. Get this book: Refactoring - Martin Fowler. If you like it then get this one: Refactoring to Patterns.
As Pedro NF said, Martin Fowler "Refactoring" is the nice place to get familiar with it.
I recommend buying a copy of Robert Martins "Agile Principles, Patterns and Practices in C#" He goes over some very practical case studies that show how to overcome maintenance problems like this.
Related
I'm developing my own game engine with ECS framework.
This is the ECS framework I use:
Entity: Just an ID which connects its components.
Component: A struct which stores pure data, no methods at all (So I can write .xsd to describe components and auto-generate C++ struct code).
System: Handle game logic.
EventDispacher: Dispatch events to subscribers (Systems)
but I'm confused about how should systems update the members of components and inform other systems?
For example, I have a TransformComponent like this:
struct TransformComponent
{
Vec3 m_Position;
Float m_fScale;
Quaternion m_Quaternion;
};
Obviously, if any member of the TransformComponent of a renderable entity is changed, the RenderSystem should also update the shader uniform "worldMatrix" before render the next frame.
So if I do "comp->m_Position = ..." in a system, how should the RenderSystem "notice" the change of TransformComponent? I have come up with 3 solutions:
Send an UpdateEvent after updating the members, and handle the event in related System. This is ugly because once a system modify the component data, it must send an event like this:
{
...;
TransformComponent* comp = componentManager.GetComponent<TransformComponent>(entityId);
comp->m_Position = ...;
comp->m_Quaternion = ...;
eventDispatcher.Send<TransformUpdateEvent>(...);
...;
}
Make members private, and for each component class, write a relevant system with set/get methods (wrapping event sending in set methods). This will bring a lot of cumbersome codes.
Do not change anything, but add "Movable" component. RenderSystem will iteratively do update for renderable entities with "Movable" component in Update() method. This may not solve other similar problems, and I'm not sure about the performance.
I can't come up with an elegant way to solve this problem. Should I change my design?
I think that in this case, the simplest method will be the best: you could just keep a pointer to a Transform component in components that read/write it.
I don't think that using events (or some other indirection, like observers) solves any real problem in here.
Transform component is very simple - it's not something that will be changed during development. Abstracting access to it will actually make code more complex and harder to maintain.
Transform is a component that will be frequently changed for many objects, maybe even most of your objects will update it each frame. Sending events each time there's a change has a cost - probably much higher than simply copying matrix/vector/quaternion from one location to another.
I think that using events, or some other abstraction, won't solve other problems, like multiple components updating the same Transform component, or components using outdated transform data.
Typically, renderers just copy all matrices of rendered objects each frame. There's no point in caching them in a rendering system.
Components like Transform are used often. Making them overly complex might be a problem in many different parts of an engine, while using the simplest solution, a pointer, will give you greater freedom.
BTW, there's also very simple way to make sure that RenderComponent will read transform after it has been updated (e. g. by PhysicsComponent) - you can split work into two steps:
Update() in which systems may modify components, and
PostUpdate() in which systems can only read data from components
For example PhysicsSystem::Update() might copy transform data to respective TransformComponent components, and then RenderSystem::PostUpdate() could just read from TransformComponent, without the risk of using outdated data.
I think there are many things to consider here. I will go in parts, discussing first your soutions.
About your solution 1. Consider, that you could do the same with a boolean, or assigning an empty component acting as tag. Many times, using events in ECS is over complicating your system architecture. At least, I tend to avoid it, specially in smaller projects. Remember that a component acting as a tag, could be thought as basically, an event.
Your solution 2, follows up what we discussed in 1. But it reveals a problem about this general approach. If you are updating your TransformComponent in several Systems, you can't know if the TransformComponent has really changed until the last System has updated it, because one System could have moved it one direction, and another one could have moved it back, letting it as at the beginning of your tick. You could solve this by updating your TransformComponent just once, in one single System...
Which looks like your solution 3. But maybe the other way around. You could be updating the MovableComponent in several Systems, and later on in your ECS pipeline, have a single System, reading your MovableComponent and writting into your TransformComponent. In this case, is important that there is only one System allowed to write on the TransformComponents. At that time, having a boolean indicating if it has been moved or not, would do the job perfectly.
Until here, we have traded performance (because we avoid some processing on the RenderSystem when TransformComponent has not changed) for memory (because we are duplicating the content of TransformComponent in some way.
Another way to do the same, without having to add events, booleans, or components, is doing everything in the RenderSystem. Basically, in each RenderComponent, you could keep a copy (or a hash) of your TransformComponent from the last time you have updated it, and then compare it. If it's not the same, render it, and update your copy.
// In your RenderSystem...
if (renderComponent.lastTransformUpdate == transformComponent) {
continue;
}
renderComponent.lastTransformUpdate = transformComponent;
render(renderComponent);
This last one, would be my preferred solution. But it also depends on the characteristics of your system and your concerns. As always, don't try to optmize for performance blindly. First measure, and then compare.
Well, lately I have found a very interesting article about Map Hacks in online games.
After reading it I read they used a memory scanner to look for images in the memory.
How would they accomplish such a program, is there a solution for this freely available?
if not how would I code it in C++? How can I know a piece of memory is an "image"?
I can load my own DLL into the process so that shouldn't be a big issue..
To answer your questions:
A memory scanner uses OS apis to query memory from another process and perform searches for patterns or differences. A great tool for this is cheat engine.
The tool mentioned in the article visualizes the memory by coloring pixels according to the value of the bytes in memory. The alignment still needs to be done manually and could be very time consuming. I don't think the mentioned program was ever released.
The main problem is that you can't know that a particular piece of memory is supposed to be a map. Any big regular structure could look like one when colorized and aligned. Finding the actual piece of memory you are looking for is very hard.
Additional Info:
A property map in a game is very dynamic. If units or something moves the visibility has to update. So the actual format of a map like this is most likely a binary bitmap with no specific image format (png,jpg,...).
I personally find the approach to look for a map structure in memory is a very inefficient and time consuming approach. It's beatuful to show to people that have no idea about reverse engineering, but to me seems very impractical. The approach which is best totally depends on the game and your creativity.
I hope I can help you with the following example how I made a map hack for starcraft 2.
My idea was to load up a replay of a game, where I had full view of the map and find the difference to loading up a normal game where my vision is restricted. I switched a couple of times between replay and normal game and could indeed find a state variable that was 0 on normal game and 1 on replay (a common tool for finding memory like this is cheat engine).
Next I loaded the game up in a debugger and put a memory access breakpoint on this state variable. Now when loading up a normal game I would change the value when it is accessed while the map was loading. Through some trial and error I was able to find the correct location that was responsible for revealing the minimap and real map. The only task left was to create a dll that detours the code location and make sure the map is always revealed on every mapload.
Reply to typ1232: You mention that it's hard and impractical to find the map structure in memory. Heres a method I have had great success with: Load up a map in any game with fog of war, like StarCraft 2. Take a dump of the memory and save it. Send out troops/units and reveal as much of the previously undiscovered map and take another memory dump. Compare the two dumps and look closer at the areas in memory where there are a high frequency of changes. This is likely to be where the map is stored.
Sorry if I'm doing it wrong, new to stackoverflow :)
This might be a bit broader answer to the subject of "finding data" but there are binary analysis tools out there. For example, ..cantor.dust.. is a binary visualization tool (though its only in beta the idea remains the same). You can search for different patterns within a memory dump for "images" or structures. Youtube cantor dust and the creator did a presentation at DerbyCon of how he used it to find EFI structures to recreate an exploit of a PNG parser at the EFI level.
I also think the saving two memory states of visible map vs limited visibility map and search for the changes is viable, if not the best option, I just am trying to point out an alternative.
Basically as part of a team I have had to create a pacman like game for my university course, just zombies instead of ghosts.
We have built all of the game so far and it seems to work really well. Our current problem is that we have to Save a game (with a username and score), Load the game into the position it was once saved, with the correct username and score, and finally be able to offer a replay option where the user can see all the moves that they have previously made (as well as the moves the zombies have made). The zombies will always make the same moves that the user makes as they are designed to chase the user.
My question is what would be the best way to do the save, load and reload options? We cannot use vectors, stacks or queues. We can only really use strings, arrays and other basic variables.
We were thinking to do the reload first by adding everything onto the end of a string and then popping the last value off the string. We could then delay each one by a second and the user will be able to see his/her moves.
As for Saving we were unsure, there are also holes (0 symbols) and pills (* symbols) to take into account. So the position of character, zombies, pills and holes will need to be saved. The character can start from any random position and pretty much everything else is placed after.
The way we do the loading will depend on the way you suggest we do the saving.
Does anyone have any suggestions of the way we should do save, load and replay?
thanks
The simplest way I could think of is saving the user inputs.
This way you could easily replay the game by sending the inputs to the game engine (this may require a lot of restructuring depending on the design of the game engine). To accelerate the loading you could also save the game state at the time of the save (through serialization).
That's the idea, how to do it... you need an ever-expanding array to record the user-input, so let's use a linked-list.
struct Node {
T data;
Node* next_node;
};
//Google for the rest of the code, it is a reeeaaallly
// basic/fundamental data structure.
The data would be the user-inputs and the time they happened.
To save the data, you simply have to iterate through the linked-list and append it to a std::ostream& (to be generic, a std::ofstream& to be specific).
You may add some other useful information (such as the game state and the highscore) before or after the user inputs (or even in another file, which would really make sense for the highscores).
You'll need to read up on some serialization. I wrote some articles on it here, but this is going to be overkill for you guys: http://www.randygaul.net/2013/01/05/c-reflection-part-5-automated-serialization/
You can use some very simple serialization to write out the moves of each zombie into a file. Then when you want to reload this information you deserialize the information in the file. Each move will likely be stored in some form of a linked list so you'll have to come up with a way of recreating such lists upon deserialization.
Your question is really broad so my answer has to be quite broad as well. Really it's up to you to research a solution and implement it.
I can fill a TreeView control with particular Registry keys by enumerating registry keys recursively and put them in the TreeView control; then for performance reason, I attempt to
use a nonrecursive/iterative approach to enumerate registry keys, but how can I fill the TreeView since a "tree" is naturally recursive (at least, in my understanding)? Is recursion is the only way to achieve it? Would someone give some code snippets/examples or redirect me to webpages explain this matter?
BTW, I'm a Delphi/Free Pascal programmer, but C/C++ programming language explanation should be no problem at all. Cheers :-)
I have asked about this on the Free Pascal mailing list, too.
Thanks in advance.
First of all, worrying about the "performance" of recursion when you're reading the data from the registry and putting data into a tree control sounds a bit silly. The costs of recursion are in the nanosecond range, while the cost of reading from the registry is in the microsecond to millisecond range. The cost of inserting into the tree control will depend on things like visibility and how many items it contains, but it's typically closer to the range of the registry than of recursion. If you're going to insert a lot of items into a control, you usually want to lock the control so it's not updated during the insertion, then turn updating back on after you're done.
Second, yes, it can be done without recursion. The usual way is to have a container of some sort to hold data that will need to be processed, such as a queue or stack. When you're walking the registry tree, you retrieve data, and when/if you encounter a "subdirectory" in the tree, you push it on the stack. When you finish with the current "directory", you retrieve the next one from the stack/queue/whatever and process it the same way. When the collection is empty, you're done.
If performance is the issue, consider delayed filling of a TreeView. Start from creating top level of a tree. Fill each next level of a tree only when user expands it. This should solve a stack overflow problem as well.
Here you can find sources of the Native Registry Editor project which you could use as a sample.
I need to create a function that undoes the previous task/addition/change. How do I do this in Borland C++?
(The program stores strings of text in a text file using "list". It is stored and then erased unless I use the save-function I've created.)
I meant creating an undo function in a simple console application by the way.
I'll give yet another answer, but I think that the coverage has been insufficient so far.
The subject is far from trivial, and googling it returns a good number of results. Many applications implement a "undo" operation, and there are many variants.
There are 2 design patterns which can help us out here:
Command: it's a reification of an action
Memento: which consists in storing state (usually implies some form of serialization)
The Command pattern is heavily used in graphic environments because there is usually various ways to accomplish an action. Think of save in Microsoft Word for example:
you can click on the save icon
you can go into File menu and click on Save
you use the shortcut, typically CTRL+S
And of course save is probably implemented in term of save as.
The advantage of the Command pattern here is twofold:
you can create a stack of objects
you can ask every object to implement an undo operation
Now, there are various issues proper to undo:
some operations cannot be undone (for example, consider rm on Linux or the empty trash bin action on Windows)
some operations are difficult to undo, or it may not be natural (you need to store some state, the object is normally destroyed but here you would need to actually store it within the command for the undo action)
generally we think of undo/redo as a stack, some software (graphics mainly) propose to undo items without actually undoing what has been done afterward, this is much more difficult to achieve, especially when the newer actions have been built on top of the to-undo one...
Because there are various problems, there are various strategies:
For a simple Command, you might consider implementing an undo (for example, adding a character can be undone by removing it)
For a more complex Command, you might consider implementing the undo as restoring the previous state (that's where Memento kick in)
If you have lots of complex Command, that could mean lots of Mementos which consumes space, you can then use an approach which consists in only memorizing one Snapshot every 10 or 20 commands, and then redoing the commands from the latest snapshot up to the undone command
In fact, you can probably mix Command and Memento at leisure, depending on the specifics of your system and thus the complexity of either.
I would only considering undoing the last action executed to begin with (using a stack of action then). The functionality of undoing whatever action the user wishes is much more complicated.
To implement Undo, you need to create an "action stack" in your application. There are two basic approaches, though:
Knowing your baseline (the last time the file was saved, or since the file was created), remember every single change that was made so that when something needs to be undone you just throw away the "top-most" item and regenerate the current view from the baseline plus all of the changes. Clicking "Redo" then just puts that item back on the stack. This has a side benefit of being able to trivially remove items anywhere in the stack without messing up other undo/redo options, although there will be special care needed to make sure that the application of "higher" states is as the user intended.
For each action, store off the change that was made to the previous state as well as the change that would be necessary to restore that previous state if you were to undo. Now when the user clicks "Undo," just do the "undo" steps. When clicking "Redo," reapply the changes that were made. In some cases the "Undo" steps will be "here's what the thing looked like before," but that can cause havoc if you want to allow users to remove items that are not on the top of the stack and then need to remove something above it.
The proper choice depends on a lot of factors, including how much data you can/will carry around. Option #1 is in some sense easier but it could become very slow to undo anything if the action stack is large.
You should check out Command Pattern.
Another reference: Using the Command pattern for undo functionality
Also see the memento pattern. Sometimes the intelligence that must go into a command to undo an operation is pretty involved. Drawing objects in a paint program for example. It can be easier just to store a memento and then restore from that memento to implement undo.
You can store snapshots of state. State is the set of data that an action can modify. When you click undo, the current state is replaced by previous. Actually it is not a trivial task, especially if the state is complex.
I've been experimenting lately on that subject. In case you don't need binary compatibility, check out https://github.com/d-led/undoredo-cpp