We currently have a series of variables that are loaded into the application scope that rarely change.
By rarely change, I mean that they are strings like phone numbers, or simple text values that appear on a website and may change once a week or once a month.
Since we are reading these variables and because they rarely change, is there any requirement to encapsulate these inside a cflock ?
I think it would be alot of coding overhead to wrap these variables inside a cflock as the template may contain upwards of 20 instances of these static variables.
Any advice on this greatly appreciated
Personally I would say you do not need to. These variables are essentially constants.
However, you need to assess this yourself. You need to answer the question, 'what would be the ramifications of these variables being read with stale data?'
This means, if as in your example the wrong phone number is used on a request is this a disaster? If that is a problem that you can live with then you can make no changes. If however there are variables that are used in calculations or ones that will cause unacceptable problems if they are stale, then you will need to lock access to these. In this way you can focus your efforts on where you need to and minimise the additional work.
As an aside if you do need to lock any variables then a good pattern to use is to store them inside a CFC instance that is stored in application scope. This way you can handle all the locking in the CFC and your calling code remains simple.
Depending on the version of ACF, Railo, etc... you are using I would suggest that data like this might be better stored in the cache and not in the application scope. The cache can have more persistences through restarts, etc... as well and could be a more efficient way to go.
Take a look at the cacheput, cacheget, cachedelete, etc... functions in the documentation. I believe this was functionality was added in CF9 and Railo 3.2.
Taking it one step further you could simply cache the entire output that uses them for X time as well, so that each time that part is loaded it only has to load one thing from the cache instead of the twenty or so times you mention.
If you are going to store them in the application scope then you only really need to have the cflock around the part of the code that updates them and lock it at the application level. That way anything wanting to read them will have to wait for it to finish updating them before it can read them anyway as the update thread will have a lock on the application scope.
Related
Is it possible to change addresses in an application such that the app still works fine, but hacks (based on memory read/write) to this app don't? Maybe move the stack or something?
#update
I don't looking for randomization base address. I'm looking for method for changing addresses in running app such that app could still work but "hacks and bots" could not read this part of memory. ASLR isn't that what am I looking for (it's to easy to bypass)
Moving the stack will be very hard unless you can return all the way to main before you do it. Any variable passed as a reference or pointer to a varíable on the stack will not be allowed to move. And before you say "well, then I'll just allocate everything dynamically", now you have exactly the same problem - your HEAP is located in one place that is predictable (at least somewhat predictable), and thus can be modified. And of course, even if the heap isn't predictably placed, you can't just move it around at random during execution, since your code would rely on pointers and references to other data in the heap - and if you move that, you'd end up having to rearrange all those references. Finally, you will still have some register or memory location that is known or possible to calculate from some other value (e.g. the stack, some global data value, or something) that can be used to figure out where your data is.
My best suggestion would be to generate code in the heap, and use that to parallel calculate results that are used in your game.
Also, one way to avoid persistent locations is to run code in threads that are dynamically created and destroyed - that way, the stack is only in one place for a short period of time. But of course, it doesn't really stop someone who is skilled and determined to find a way around you protection. And with millions and millions of people in the world that have access to computers and are able to "break into things", you can't really rely on "security through obscurity" [making things complicated is not security].
The proper secure way is to perform all essential calculations on a server which holds the code, and the code isn't available to the public! However, for a FPS game, that's probably not realistic. For a poker game, it's very much realistic, especially if you are going to win money if you play well!
Standard hacking case. Hack file type injects into a started process and writes over process memory using WriteProcessMemory call. In games this is not something you would want because it can provide the hacker to change the portion of the game and give himself an advantage.
There is a possibility to force a user to run a third-party program along with the game and I would need to know what would be the best way to prevent such injection. I already tried to use a function EnumProcessModules which lists all process DLLs with no success. It seems to me that the hacks inject directly into process memory (end of stack?), therefore it is undetected. At the moment I have came down to a few options.
Create a blacklist of files, file patterns, process names and memory patterns of most known public hacks and scan them with the program. The problem with this is that I would need to maintain the blacklist and also create an update of the program to hold all avalible hacks. I also found this usefull answer Detecting memory access to a process but it could be possible that some existing DLL is already using those calls so there could be false positives.
Using ReadProcessMemory to monitor the changes in well known memory offsets (hacks usually use the same offsets to achieve something). I would need to run a few hacks, monitor the behaviour and get samples of hack behaviour when comparing to normal run.
Would it be possible to somehow rearrange the process memory after it starts? Maybe just pushing the process memory down the stack could confuse the hack.
This is an example of the hack call:
WriteProcessMemory(phandler,0xsomeoffset,&datatowrite,...);
So unless the hack is a little more smarter to search for the actual start of the process it would already be a great success. I wonder if there is a system call that could rewrite the memory to another location or somehow insert some null data in front of the stack.
So, what would be the best way to go with this? It is a really interesting and dark area of the programming so I would like to hear as much interesting ideas as possible. The goal is to either prevent the hack from working or detect it.
Best regards
Time after time compute the hash or CRC of application's image stored in memory and compare it with known hash or CRC.
Our service http://activation-cloud.com provides the ability to check integrity of application against the signature stored in database.
I have a wizard class that gets used a lot in my program. Unfortunately, the wizard takes a while to load mostly because the GUI framework is very slow. I tried to redesign the wizard class multiple times (like making the object reusable so it only gets created once) but I always hit a brick wall somewhere. So, at this point is it a huge ugly hack to just load 50 instances of this beast into a vector and just pop them off as I use them? That way the delay will only be noticed on startup and run fine thereafter. Too much of a hack? Is such a construct common?
In games, we often first allocate and construct everything needed in a game session. Then we recycle the objects if they have short life-time, trying to get 0 allocations/deallocations while the game session is running.
So no it's not really a hack, it's just good sense to make the computer do less work to get faster. One strategy is "caching", that is, in general, first compute your non-variant data, then run with the dynamic ones. Memory allocation, object constructions, etc have to be prepared before use, where possible and necessary.
Unfortunately, the wizard takes a while to load mostly because the GUI framework is very slow.
Isn't a wizard just a form-based template? Shouldn't that carry essentially no overhead? Find what's slowing the framework down (uncompressed background image?) and fix the root cause.
As a stopgap, you could create the windows in the background and not display them until the user asks. But that's obviously just moving the problem somewhere else. Even if you create them in a background thread at startup, the user's first command might ask for the last wizard and then they have to wait 50x as long… which they'll probably interpret as a crash. At the very least, anticipate and test such corner cases. Also test on a low-RAM setup.
Yes it is bad practice, it breaks RFC2549 standard.
OK ok, I was just kidding. Do whatever is best for your application.
It isn't a matter of "hacks" or "standards".
Just make sure you have proper documentation about what isn't as straightforward as it should be (such as hacks).
Trust me, if a 5k investment produced a product with lots of hacks (such as windows), then they [hacks] must really help at some point.
This is about a C++ problem.
I have an object tracking program that takes images from 0,...,n in a loop. At current frame the computations are based on previous frames, therefore I need to hold those variables, matrices, etc for later use. This program has to be integrated now into another system which will provide an image and I have to return the tracking output. The system does later other processes, so my program has to become function to distribute as DLL.
I need to store my variables and matrices from previous images in order to use them again. I don't know if the best practice is to write them in hard drive and read them again in another instance. If this is the case what is the best way and data type/file to write/read. The systems aims to be real-time.
Thanks
One thing you could look into that IS NOT THREADSAFE is to keep the local variables as static. If you're not familiar with C/C++ static variables, they are stored in the global memory space and "remembered" between function calls. They're like global variables but can only be accessed by the function they're declared in. Run this a couple of times and see what happens.
void foo()
{
static int x=0;
x++;
cout << x << endl;
}
Remember, you cannot have multiple threads call foo because there's only one state now!
Alternatively you could do something where you create a struct that holds a copy of your local state and you pass that in.
struct state
{
int x
};
void bar(state& s)
{
s.x++;
cout << s.x << endl;
}
It depends on your platform, but these days rare is the platform that doesn't have oodles of memory to spare. So if you are just saving data from a previous pass, no matter how much, my first go at it would be to save it all in memory somewhere.
If you end up running out of space, my second go would be to look into getting more RAM for your system. If it costs an extra $100, and you aren't making thousands of units, then it may save you money in the long run over engineering hours.
If not, then you can worry about the extra complexity of trying to save and restore from disk in realtime.
Not really an answer, just a longer request for details.
Any kind of data persistence issue involves decisions such as:
required lifetime: app-controlled, thread, process, until host reboot, indefinite...?
how many concurrent readers/writers will there be for the repository
how are the readers/writers spaced across networks / hardware (e.g. endian issues, latencies)
You really haven't provided enough detail to make a serious start on this.
If your general hunch is right and a file is a suitable mechanism, you might consider whether memory mapping works well with your requirements... it tends to be faster than streamed file I/O.
Alternatives include a shared memory segment (can live longer than the creating process), heap...?
If your real interest is in serialisation mechanisms, you might have a look at boosts.
Anyway, I'm off home so it'll probably be someone else who answers....
Some ideas and suggestions:
Storing the images or attributes of the images?
In general, an image will take up more space than attributes or data calculated from the image. Perhaps you can store the attributes of the images rather than the whole image.
Cache the data
Put as much data in memory as possible, store the rest in a file. The issue to be answered is how much of information must be in memory (such as the last N items or perhaps the first N items).
Multi (task or thread)
Have one thread that caches the images on demand. As the images are received, it puts either the image into memory (or the attributes). When the fixed memory area fills up, it places the image (or attributes) onto external memory (e.g. files). The main thread would request images from the caching thread. After the caching thread removes an image, it replaces that image (or attributes) with one datum from the file. The thread can sleep until either a new image comes its way or an older image is requested.
Hey Guys thanks a lot, I got some ideas already from you:
STL
memory-mapped files
Multi-Threads
I will start working with these solutions and let's see what are the restrictions of each one with the requirement of real-time.
I will come back to post here later the final solution and in case I require more details I will go for an specific topic.
Thanks
My app keeps track of the state of about 1000 objects. Those objects are read from and written to a persistent store (serialized) in no particular order.
Right now the app uses the registry to store each object's state. This is nice because:
It is simple
It is very fast
Individual object's state can be read/written without needing to read some larger entity (like pulling out a snippet from a large XML file)
There is a decent editor (RegEdit) which allow easily manipulating individual items
Having said that, I'm wondering if there is a better way. SQLite seems like a possibility, but you don't have the same level of multiple-reader/multiple-writer that you get with the registry, and no simple way to edit existing entries.
Any better suggestions? A bunch of flat files?
If what you mean by 'multiple-reader/multiple-writer' is that you keep a lot of threads writing to the store concurrently, SQLite is threadsafe (you can have concurrent SELECTs and concurrent writes are handled transparently). See the [FAQ [1]] and grep for 'threadsafe'
[1]: http://www.sqlite.org/faq.html/ FAQ
If you do begin to experiment with SQLite, you should know that "out of the box" it might not seem as fast as you would like, but it can quickly be made to be much faster by applying some established optimization tips:
SQLite optimization
Depending on the size of the data and the amount of RAM available, one of the best performance gains will occur by setting sqlite to use an all-in-memory database rather than writing to disk.
For in-memory databases, pass NULL as the filename argument to sqlite3_open and make sure that TEMP_STORE is defined appropriately
On the other hand, if you tell sqlite to use the harddisk, then you will get a similar benefit to your current usage of RegEdit to manipulate the program's data "on the fly."
The way you could simulate your current RegEdit technique with sqlite would be to use the sqlite command-line tool to connect to the on-disk database. You can run UPDATE statements on the sql data from the command-line while your main program is running (and/or while it is paused in break mode).
I doubt any sane person would go this route these days, however some of what you describe could be done with Window's Structured/Compound Storage. I only mention this since you're asking about Windows - and this is/was an official Windows way to do this.
This is how DOC files were put together (but not the new DOCX format). From MSDN it'll appear really complicated, but I've used it, it isn't the worst API in Win32.
it is not simple
it is fast, I would guess it's faster then the registry.
Individual object's state can be read/written without needing to read some larger entity.
There is no decent editor, however there are some real basic stuff (VC++ 6.0 had the "DocFile Viewer" under Tools. (yeah, that's what that thing did) I found a few more online.
You get a file instead of registry keys.
You gain some old-school Windows developer geek-cred.
Other random thoughts:
I think XML is the way to go (despite the random access issue). Heck, INI files may work. The registry gives you very fine grain security if you need it - people seem to forget this when the claim using files are better. An embedded DB seems like overkill if I'm understanding what you're doing.
Do you need to persist the objects on each change event or just in memory and store on shutdown? If so, just load them up and serialize them at the end, assuming your app runs for a long time (and you don't share that state with another program) then in memory is going to be a winner.
If you've got fixed size structures then you could consider just using a memory mapped file and allocate memory from that?
If the only thing you do is serialize/deserialize individual objects (no fancy queries), then use a btree database, for example Berkeley DB. It is very fast at storing and retrieving chunks of data by key (I assume your objects have some id that can be used as a key) and access by multiple processes is supported.