My OS textbook says the following in a chapter discussing concurrency:
Concurrent processes come into conflict with each other when they are competing for the use of the same resource. In its pure form, we can describe the situation as follows. Two or more processes need to access a resource during the course of their execution. Each process is unaware of the existence of other processes, and each is to be unaffected by the execution of the other processes. It follows from this that each process should leave the state of any resource that it uses unaffected.
My question specifically concerns the last sentence:
It follows from this that each process should leave the state of any resource that it uses unaffected.
This does not make sense to me. If a process is using a resource, then it must necessarily affect the state of that resource. This seems obvious, but it sounds like the sentence is disagrees?
I would greatly appreciate it if the members of this site could please take the time to clarify this.
While it is not clear to me in what context this was said, as you mentioned a small portion of the quote. And didn't even bother to mention the book you quoted. However, I can shoot in the dark and assume that what they meant is: A process using resource X should, once done using it, leave it unaffected. That is, if processY decides to use logical resource, i.e. a file, it should not write or change the file as this might affect processZ which needs to use the file with its original data.
When it comes to physical resources, the statement above makes no sense... unless you provide the full quote.
Related
I have been working with GL_MAP_PERSISTENT_BIT and glBufferStorage/glMapBufferRange. I am curious if there is an improvement in performance possible using GL_MAP_UNSYNCHRONIZED_BIT.
I already found Opengl Unsynchronized/Non-blocking Map
But the answer seems to be a bit contradictory to me. It's said there that you need to sync or block when using this flag. What is the point of setting it unsynchronized if I have to sync it later then anyway? Also I tried this combination and was not able to see any performance difference. Does it even make sense together with persistent mapped buffers? I found literally no examples about such a usage.
The mentioned topic also says that you can
issue a barrier or flush that region of the buffer explicitly
But every attempt I made so far using these only resulted in garbage.
I am using currently triple buffering, but since I have to deal with very small chunks of data sometimes which I hardly can batch I had to find out that glBufferData is often faster in these cases and persistent buffers only of (huge) benefit if I can batch and reduce also the amount of drawcalls. Using GL_MAP_UNSYNCHRONIZED_BIT could be the key here.
Can anyone give me a working example, in case it even makes sense in this combination?
What is the point of setting it unsynchronized if I have to sync it later then anyway?
The point, as stated by that answer, is that OpenGL isn't doing the synchronization for you. You control when the synchronization happens. This means that you can ensure that it doesn't happen at an inappropriate time. By using your own synchronization, you can also ask the question, "are you finished using the buffer?" which is not a question you could ask without your own sync system.
By using unsynchronized mapping, you stop the implementation from having to check its own internal sync in addition to your synchronization.
However, that answer you linked to applies primarily to non-persistent mapping (since that's what the question was about). Unsynchronized mapping only applies to the map call itself. It prevents GL from issuing internal synchronization due to you calling glMapBufferRange.
But unsynchronized mapping doesn't really affect persistent mapping because... well, it's persistent. The whole point of the feature is that you keep the buffer mapped, so you're only going to call glMapBufferRange once. And the unsynchronized bit only applies at the moment you call glMapBufferRange.
So whether you use unsynchronized or not with your persistent mapping call is essentially irrelevant.
Asked on the grounds of: "...but if your question generally covers…
- a specific programming problem..." (Help center - asking)
Scope: This is not about how to use the file lock mechanisms on different platforms, but about how to mitigate the absence of mandatory file locks on the user's system. E.g.: I can't expect a user of a Linux system to modify the system, let alone know how to do it, so I have to assume advisory locks is all that is available. I have found a lot of info about how to use locks of both kinds, and what is available on some platforms, and even why they are not available on some systems. Portability would be great, but this is probably to much down to the bone for that.
I am a bit confused about how to safeguard my program's data if other processes don't cooperate, intentionally or not. Assuming that my program uses its own directory for the data, is there a way to make sure that my data will stay consistent while the program runs?
Would, for example, temporary hidden files be a practical solution (create a file, delete it from the OS' directory, so only my handle holds the inode to the file), copying all data at program start and overwriting the original at the end? It seems to be very platform specific, though.
Are there specific mechanisms or techniques to use that could help with this, or can I only "trust"?
Note: This is not specifically about Linux, it's just an example.
------ EDIT -------
I'm looking for a way to do this that works in C/C++, hence those tags, but am aware that it might involve system specific features. If possible, the solution would work regardless of platform and file locking mechanism.
While file locks is the mechanism referenced in the question, the real problem is how to prevent another process from trampling over the data my program relies on while running, even if that process runs as the same user as my program does, but does not care to check whether the files are locked. Also if it is using a mechanism that isn't working well with the one my program is using. (AFAICT, locks acquired with one of flock()/lockf() on Linux may not work when the other is used in the other process) (Another situation from Linux, but one that is outside the scope of the question)
As I tried to explain about the scope, this is not about how to use file locks on any platform, but what to do when you cannot assume anything about what mechanism is available/turned on, to achieve similar protection to what mandatory file locks would give.
I am a bit confused about how to safeguard my program's data if other processes don't cooperate, intentionally or not. Assuming that my program uses its own directory for the data, is there a way to make sure that my data will stay consistent while the program runs?
You cannot do so and should not even try to do so. How do you safeguard your data if the admin pulls the power cord? All things with access to your data must cooperate -- that is the precondition for safeguarding being possible.
Simply specify that your program requires a directory that is only touched by programs that cooperate with yours. That is a trivial requirement that many programs have any any competent administrator can provide.
It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 10 years ago.
I have been looking online at what a thread is and I do not feel like I understand it. Could someone shed some light on this? In terms of programming languages for relating to C++, objective-C would be nice.
In objective-c, I encountered
#property(nonatomic, strong) NSString *name;
the explanation for nonatomic was it means to not be worried about multiple threads trying to access the object at the same time, and objective-c does not have to synthesize thread safe code. So what does that exactly mean as well.
A process can consist of multiple threads of execution, which logically can be thought of as running simultaneously alongside each other. Each thread runs independently but shares the same memory and process state. A single thread can "do one thing": perform computation, interact with the network, update a UI, decode a video, etc. But, a single thread cannot do all of these at once without a significant amount of extra work from the programmer. Having multiple threads in a process enables the programmer to easily enable an application to do multiple things at once (multitasking).
Using multiple threads does introduce some new challenges, though. For example, if you have two threads that access the same variable, you can end up with a concurrency hazard in which the variable might not be completely updated by one thread before the other thread accesses it, leading to failures in the program. Objective-C will generate thread-safe code by default, to avoid this situation. nonatomic tells the compiler that you will never access it from multiple threads simultaneously, so the compiler can skip the thread-safe code and make faster code. This can be useful if you are going to supply your own synchronization, anyway (e.g. to keep a group of properties in sync, which Objective-C itself cannot help you with).
If you violate the core nonatomic assumption and access a nonatomic variable from multiple threads at the same time, all hell will break loose.
explanation for nonatomic was it means to not be worried about multiple threads trying to access the object at the same time, and objective-c does not have to synthesize thread safe code. So what does that exactly mean as well.
Imagine you are asked to write your name on a piece of paper. You're given a list of instructions someone thought would work just fine:
you find a line that's current empty,
move your pen over it,
write your name.
All good.
Now imagine you're given a new piece of paper, but both you and someone else are asked to write your names on the same piece of paper, and you're given the old instructions, perhaps:
1) You both look at the paper and determine to write on the first line.
2) You put your pens down (maybe you can both do it comfortably enough - one left / one right handed).
3) You start to write an I but the other person is writing a J and it comes out looking like a U.
4) gets worse from here....
But equally, it might be that you're paying more attention, and finish writing your name before they start looking for an empty line, or vice versa.
Threading is a lot like this... in the above example, each thread/person is keeping track of how they're progressing at the task, following their instructions very literally. Notice that if you complete only step 1, then the other person does step 1, you've already set yourselves up to write over each others' name regardless of the ordering or concurrency of the remaining steps.
In all this, you don't even have to be doing things at the same instant in time, it's just that the tracking of your tasks is independent - you're independent people with your own memory of where you are in your task. Same with threads - they're ways of tracking what to do independently, and it's optional whether they actually do things in your program at the same instant (which is possible with multi-core CPUs and multi-CPU systems).
"atomic" is used in the sense of indivisible (think: you can't cut an atom of gold in half and still have gold). Similarly, if you say write your name atomically, it means any observer is guaranteed to either witness the instant before - when no name is there - or the instant after - when your name is completely written - but they'll never see just half your name. An atomic update on a string variable is like that.
Atomic string updates don't solve the problem above... you might still clash in finding "an empty line" (in a computing context - say finding the next empty position in a container). If that process of finding an empty line is atomic, and the line is somehow marked "used" even before you've written anything on it yourself, then it means you'll never get the same line as someone else. At that stage, multiple people writing their names won't clash on the same line, but only when both the line finding and the name writing are atomic can people looking at the paper know that they're seeing completely written non-clashing names.
Making these kind of guarantees is very useful but expensive. It means that threads must communicate and coordinate amongst themselves, agreeing "who" will go first with others waiting as necessary.
I've never had formal training in this area so I'm wondering what do they teach in school (if they do).
Say you have two programs in written in two different languages: C++ and Python or some other combination and you want to share a constantly updated variable on the same machine, what would you use and why? The information need not be secured but must be isochronous should be reliable.
Eg. Program A will get a value from a hardware device and update variable X every 0.1ms, I'd like to be able to access this X from Program B as often as possible and obtain the latest values. Program A and B are written and compiled in two different (robust) languages. How do I access X from program B? Assume I have the source code from A and B and I do not want to completely rewrite or port either of them.
The method's I've seen used thus far include:
File Buffer - Read and write to a
single file (eg C:\temp.txt).
Create a wrapper - From A to B or B
to A.
Memory Buffer - Designate a specific
memory address (mutex?).
UDP packets via sockets - Haven't
tried it yet but looks good.
Firewall?
Sorry for just throwing this out there, I don't know what the name of this technique is so I have trouble searching.
Well you can write XML and use some basic message queuing (like rabbitMQ) to pass messages around
Don't know if this will be helpful, but I'm also a student, and this is what I think you mean.
I've used marshalling to get a java class and import it into a C# program.
With marshalling you use xml to transfer code in a way so that it can be read by other coding environments.
When asking particular questions, you should aim at providing as much information as possible. You have added a use case, but the use case is incomplete.
Your particular use case seems like a very small amount of data that has to be available at a high frequency 10kHz. I would first try to determine whether I can actually make both pieces of code part of a single process, rather than two different processes. Depending on the languages (missing from the question) it might even be simple, or turn the impossible into possible --depending on the OS (missing from the question), the scheduler might not be fast enough switching from one process to another, and it might impact the availability of the latest read. Switching between threads is usually much faster.
If you cannot turn them into a single process, then you will have to use some short of IPC (Inter Process Communication). Due to the frequency I would rule out most heavy weight protocols (avoid XML, CORBA) as the overhead will probably be too high. If the receiving end needs only access to the latest value, and that access may be less frequent than 0.1 ms, then you don't want to use any protocol that includes queueing as you do not want to read the next element in the queue, you only care about the last, if you did not read the element when it was good, avoid the cost of processing it when it is already stale --i.e. it does not make sense to loop extracting from the queue and discarding.
I would be inclined to use shared memory, or a memory mapped shared file (they are probably quite similar, depends on the platform missing from the question). Depending on the size of the element and the exact hardware architecture (missing from the question) you may be able to avoid locking with a mutex. As an example in current intel processors, read/write access to 32 bit integers from memory is guaranteed to be atomic if the variable is correctly aligned, so in that case you would not be locking.
At my school they teach CORBA. They shouldn't, it's an ancient hideous language from the eon of mainframes, it's a classic case of design-by-committee, every feature possible that you don't want is included, and some that you probably do (asynchronous calls?) aren't. If you think the c++ specification is big, think again.
Don't use it.
That said though, it does have a nice, easy-to-use interface for doing simple things.
But don't use it.
It almost always pass through C binding.
With my basic knowledge of C++, I've managed to whip together a simple program that reads some data from a program (using ReadProcessMemory) and sends it to my web server every five minutes, so I can see the status of said program while I'm not at home.
I found the memory addresses to read from using a program designed to hack games called "Memory Hacking Software." The problem is, the addresses change whenever I move the program to another machine.
My question is: is there a way to find a 'permanent' address that is the same on any machine? Or is this simply impossible. Excuse me if this is a dumb question, but I don't know a whole lot on the subject. Or perhaps another means to access information from a running program.
Thanks for any and all help!
There are ways to do it such as being able to recognise memory patterns around the thing you're looking for. Crackers can use this to find memory locations to patch even with software that "moves around", so to speak (as with operating systems that provide randomisation of address spaces).
For example, if you know that there are fixed character strings always located X bytes beyond the area of interest, you can scan the whole address space to find them, then calculate the area of interest from that.
However, it's not always as reliable as you might think.
I would instead be thinking of another way to achieve your ends, one that doesn't involve battling the features that are protecting such software from malicious behaviour.
Think of questions like:
Why exactly do you need access to the address space at all?
Does the program itself provide status information in a more workable manner?
If the program is yours, can you modify it to provide that information?
If you only need to know if the program is doing its job, can you simply "ping" the program (e.g., for a web page, send an HTML request and ensure you get a valid response)?
As a last resort, can you convince the OS to load your program without address space randomisation then continue using your (somewhat dubious) method?
Given your comment that:
I use the program on four machines and I have to "re-find" the addresses (8 of them) on all of them every time they update the program.
I would simply opt for automating this process. This is what some cracking software does. It scans files or in-memory code and data looking for markers that it can use for locating an area of interest.
If you can do it manually, you should be able to write a program that can do it. Have that program locate the areas of interest (by reading the process address space) and, once they're found, just read your required information from there. If the methods of finding them changes with each release (instead of just the actual locations), you'll probably need to update your locator routines with each release of their software but, unfortunately, that's the price you pay for the chosen method.
It's unlikely the program you're trying to read will be as secure as some - I've seen some move their areas of interest around as the program is running, to try and confuse crackers.
What you are asking for is impossible by design. ASLR is designed specifically to prevent this kind of snooping.
What kind of information are you getting from the remote process?
Sorry, this isn't possible. The memory layout of processes isn't going to be reliably consistent.
You can achieve your goal in a number of ways:
Add a client/server protocol that you can connect to and ask "what's your status?" (this also lends itself nicely to asking for more info).
Have the process periodically touch a file, the "monitor" can check the modification time of that file to see if the process is dead.