I currently have a 1028*32 byte array of structures in shared memory using boost::interprocess. Each of these contains location and drawing information from players in a game that I am scraping from a DLL to be drawn in a remote overlay. My basic mechanism of controlling reading and writing is a bool at the beginning of the shared memory object that each process toggles on and off. This method works, and I can get information from each of the players ingame, but unfortunately this method is really slow. Is there any way to transfer this information quicker? If at all possible, I would like to have it so that I can call my drawing function once a particular member of the array gets updated (say member 0 gets updated, before my scraper dll updates the next member, start drawing the info of member 0, then continue).
Well, you could a have a flag per member and continuously process the array and check for newly set flags. If you need more event style notifications then you could have a monitor for each individual member and signal it once it got updated. I'm not sure how the performance will be affected by having a 1000 monitors though.
Related
The program I'm working on has the following features:
Display the incoming data (thru an ethernet UDP socket)
Manipulate and calculate some data in the header and display
Save a # of frames upon user input
The program has the following structure:
a Main UI thread running (provided by MFC)
a thread which takes the dialog itself, this, as a parameter and is in charge of receiving the data through the UDP socket
a thread which also takes the dialog pointer, this and is in charge of displaying the data
The data receiving thread is using a buffer to store data and the buffer is a member of the dialog class. What makes this possible is, in the display thread, I can access the data stored in the buffer with the use of the dialog pointer and display it; such as StretchBlt(....., pdlg->vbuf[0].data, ....);.
It really isn't much complicated. The program runs smooth until there is a user input (windows messages, or more). It crashes at:
Most frequent crash point
CEXAMPLEAPP::InitInstance()
{
...
CEXAMPLEDlg* pdlg = new CEXAMPLEDlg;
m_pMainWnd = pdlg;
INT_PTR nResponse = pdlg->DoModal(); <--------------------------------------------
if( pdlg!=NULL )
{
delete pdlg;
m_pMainWnd = NULL;
}
...
}
nResponse turns out to have a huuuuge negative value (something like 19 decimal degits) which I don't think is a normal behaviour.
It also crashes at a couple different locations. Most of them are reading violations or dll errors which are hard to interpret and track the cause.
I have done some research, the possible causes of the crashes are:
i) The dynamically allocated dialog object is a local. Therefore the allocated memory isn't enough to process the infinitely incoming data. That's why it's causing reading violations at random locations.
ii) Another possible cause is that most of the variables, including the frame buffers, are member variables - the program is using two separate worker threads and they all are taking the dialog pointer as a parameter. Even though the reading and the writing to the buffer routine are done within CCriticalSection lock(), and unlock() inside the thread functions (global), the variables' ownership could be the possible cause.
This is where I'm at right now. Any ideas to fix the crashes? Any thoughts on using the variables as member variables of the dialog class?
The threads are communicating through SetEvents() and WaitforSingleObject() btw.
When you display the data (StretchBlt(....., pdlg->vbuf[0].data, ....);) in the display thread and this is accessing the display DC, you have to make sure you are doing it in sync with the main thread, for example, by using atomic locks. Or as an alternately better solution, do the display in main thread and remove the display thread altogether.
I'm working on a data structure with moving devices. I already added geohashes (open location code). And I'm able to use where >= geohash_low and where <= geohash_high to search within an area.
If I look this up, I get results within (roughly) this area. The next step is to look back per device when they entered or left the area. If I look back, I can determine this easily, but the next step is to determine it at the moment that a new message is placed in the database.
My first idea would be to create a copy of every incoming message at device/latest. And then monitor with a onChange cloud function the changes. And then determine if the change was going from inside to outside an area or the otherway around.
But this architecture will add an extra write operation (to latest message) at every incoming message. And it will add an extra read operation (onChange).
Another approach would be to do an additional read at every incoming message to read a latest state of inside or outside an area. And compare that with the current position. That would 'cost' only an extra read at every incoming message. If the state needs to be changed, then perform an extra write.
Basically the problem is that I need to manage a state, while functions are stateless.
Any other thoughts...
Thanks a lot for your thoughts.
I currently have a multiplayer game that players are starting to use memory editing to to cancel attack animation making the attack packets come-in faster or making the attacks a lot faster than normal.
Yes a better design would be ideal but that could take a while. I wanted to get a temporary fix that can be done quick.
The ideas:
Check time difference between the last attack packet ignore everything that is too fast. (for server)
Use EnumWindows check for window classes and stop the game if a known memory editor is detected. EnumWindows will be executed each time an attack is made. (for client)
Use ReadProcessMemory to read running processes and find signitures for known memory editors.
Well the question really is if any of the following could work and how it would be done:
Detour ReadProcessMemory or OpenProcess and exit when called? (though I think this wont work because these functions gets called by the memory editor not my game).
ReadProcessMemory on my self(game) and check the addresses that they are changing. Check if the values are not within the normal range then exit.
Any suggestions?
I know that it is futile to do this because cheaters that knows what their doing can still get around all this. But my game has only about 600 active players, I believe they are just somewhat scriptkiddies. I think this simple countermeasures should be enough for small games like mine. But of course, the design will be corrected.
Detour ReadProcessMemory or OpenProcess and exit when called?
These are not being called in your process so hooking them locally wouldn't do anything. You would need to hook every running process, which is not recommended.
ReadProcessMemory on my self(game) and check the addresses that they are changing. Check if the values are not within the normal range then exit.
You don't need to ReadProcessMemory, you're inside your own process. Just check the value normally.
You should calculate these values on the server if you don't want the client's to be able to manipulate them, then just replicate this info to the clients and overwrite them.
You can also add an antidebug library to your client to prevent the majority of people from manipulating your process. Here is a decent one
I am trying to make some design decisions for an algorithm I am working on. I think that I want to use signals and slots to implement an observer pattern, but I am not sure of a few things.
Here is the algorithm I am working towards:
1.) Load tiles of an image from a large file
1a.) Copy the entire file to a new location
2.) Process the tiles as they are loaded
3.) If the copy has been created, copy the resulting data into the new file
so I envision having a class with functions like loadAllTiles() which would emit signals to tell a processTile() that another tile was ready to be processed, while moving on to load the next tile.
processTile() would perform some calculations, and when complete, signal to writeResults() that a new set of results data was ready to be written. writeResults() would verify that the copying was complete, and start writing the output data.
Does this sound reasonable? is there a way to make loadAllTiles() load in a tile, pass that data somehow to processTile() and then keep going and load the next tile? I was thinking about maybe setting up a list of some sort to store the tiles ready to be processed, and another list for result tiles ready to be written to disk. I guess the disadvantage there is I have to somehow keep those lists in tact, so that multiple threads arent trying to add/remove items from the list.
Thanks for any insight.
It's not completely clear in your question, but it seems that you want to split up the work into several threads, so that the processing of tiles can begin before you finishing loading the entire set.
Consider a multithreaded processing pipeline architecture. Assign one thread per task (loading, copying, processing), and pass around tiles between tasks via Producer-Consumer queues (aka BlockingQueue). To be more precise, pass around pointers (or shared pointers) to tiles to avoid needless copying.
There doesn't seem to be a ready-made thread-safe BlockingQueue class in Qt, but you can roll-up your own using QQueue, QWaitCondition, and QMutex. Here are some sources of inspiration:
Just Software Solutions' blog article.
Java's BlockingQueue
ZThreads's BlockingQueue
While there isn't a ready-made BlockingQueue within Qt, it seems that using signals & slots with the Qt::QueuedConnection option may serve the same purpose. This Qt blog article makes such use of signals and slots.
You may want to combine this pipeline approach with a memory pool or free list of tiles, so that already allocated tiles are recycled in your pipeline.
Here's a conceptual sketch of the pipeline:
TilePool -> TileLoader -> PCQ -> TileProcessor -> PCQ -> TileSaver -\
^ |
\----------------------------------------------------------------/
where PCQ represents a Producer-Consumer queue.
To exploit even more parallelism, you can try thread pools at each stage.
You can also consider checking out Intel's Threading Building Blocks. I haven't tried it myself. Be aware of the GPL licence for the open source version.
Keeping the lists from corruption should be possible with any kind of parallelization lock mechanisms, be it simple locks, semaphores etc etc.
Otherwise, the approach sounds reasonable, even though I would say that the files had to be large in order for this to make sense. As long as they easily fit into memory, I don't see the point of loading them piecewise. Also: How do you plan on extracting tiles without reading the entire image repeatedly?
I need to implement a statistics reporter - an object that prints to screen bunch of statistic.
This info is updated by 20 threads.
The reporter must be a thread itself that wakes up every 1 sec, read the info and prints it to screen.
My design so far: InfoReporterElement - one element of info. has two function, PrintInfo and UpdateData.
InfoReporterRow - one row on screen. A row holds vector of ReporterInfoElement.
InfoReporterModule - a module composed of a header and vector of rows.
InfoRporter - the reporter composed of a vector of modules and a header. The reporter exports the function 'PrintData' that goes over all modules\rows\basic elements and prints the data to screen.
I think that I should an Object responsible to receive updates from the threads and update the basic info elements.
The main problem is how to update the info - should I use one mutex for the object or use mutex per basic element?
Also, which object should be a threads - the reporter itself, or the one that received updates from the threads?
I would say that first of all, the Reporter itself should be a thread. It's basic in term of decoupling to isolate the drawing part from the active code (MVC).
The structure itself is of little use here. When you reason in term of Multithread it's not so much the structure as the flow of information that you should check.
Here you have 20 active threads that will update the information, and 1 passive thread that will display it.
The problem here is that you encounter the risk of introducing some delay in the work to be done because the active thread cannot acquire the lock (used for display). Reporting (or logging) should never block (or as little as possible).
I propose to introduce an intermediate structure (and thread), to separate the GUI and the work: a queuing thread.
active threads post event to the queue
the queuing thread update the structure above
the displaying thread shows the current state
You can avoid some synchronization issues by using the same idea that is used for Graphics. Use 2 buffers: the current one (that is displayed by the displaying thread) and the next one (updated by the queuing thread). When the queuing thread has processed a batch of events (up to you to decide what a batch is), it asks to swap the 2 buffers, so that next time the displaying thread will display fresh info.
Note: On a more personal note, I don't like your structure. The working thread has to know exactly where on the screen the element it should update is displayed, this is a clear breach of encapsulation.
Once again, look up MVC.
And since I am neck deep in patterns: look up Observer too ;)
The main problem is how to update the
info - should i use one mutex for the
object or use mutex per basic element?
Put a mutex around the basic unit of update action. If this is an InfoReporterElement object, you'd need a mutex per such object. Otherwise, if a row is updated at a time, by any one of the threads then put the mutex around the row and so on.
Also, which object should be a threads
- the reporter itself, or the one that received updates from the threads?
You can put all of them in separate threads -- multiple writer threads that update the information and one reader thread that reads the value.
You seem to have a pretty good grasp of the basics of concurrency.
My intial thought would be a queue which has a mutex which locks for writes and deletes. If you have the time then I would look at lock-free access.
For you second concern I would have just one reader thread.
A piece of code would be nice to operate on.
Attach a mutex to every InfoReporterElement. As you've written in a comment, not only you need getting and setting element value, but also increment it or probably do another stuff, so what I'd do is make a mutexed member function for every interlocked operation I'd need.