Is it possible to "memory protect" pointers so that it's actually impossible to change them in the code, so that attempting to change them in the code results in a bus error? I am not referring to const but some type of deeper immutability assurance on the OS level. The question applies to any OS.
(Carmack mentions something like that here: https://youtu.be/Uooh0Y9fC_M?t=1h41m)
Since your question is quite general, here are some general ideas.
From within your application, the language gives you a number of constructs to protect your data. For example, data that is not meant to be exposed should be flagged as private. This does not prevent clients from reverse engineering your class though, for example if you have
class C {
public: int a;
private: int b;
};
then usually you will be able to access b through int* pB = &(c.a) + 1. This is not standard by any definition, but on most systems this will probably work. In general, because C++ allows very low-level memory management you can basically access any part of your applications memory from anywhere within the application, though making sensible (ab)use of this requires some reverse engineering.
When you expose public data, you can return const references and pointers, but of course it is very easy to still change this memory:
const* T myImmutable = new T();
const_cast<T*>(myImmutable)->change(); // Oops
Again, this is not standard C++ as compilers will use the const qualifier to perform optimizations, and things may break when you circumvent that, but the language does not stop you from doing this.
If you want to protect your memory from external changes, things become a bit trickier. In general, the OS makes sure that the memory assigned to different processes is separated and you cannot just go and write in the memory space of other processes. However, there are some functions in the Windows API (Read/WriteProcessMemory) that one may use. Of course, this requires heavy reverse engineering to determine exactly where in memory the pointer to be changed is located, but it is possible. Some ways of protecting against this are VirtualProtect, as mentioned in Dani's answer. There are increasingly complex things that you can do, like keeping checksums of important data or
[writing a] driver that monitors the SSDT and then catches when WriteProcessMemory or ReadProcessMemory is executed and you can squash those calls if they're pointed at the game
but as the first answer on the page I found that on correctly points out:
Safety doesn't exist. The only thing you can do is make it as hard as possible to crack
which is a lesson that you should never forget!
Related
I was told recently in a job interview their project works on building the smallest size binary for their application (runs embedded) so I would not be able to use things such as templating or smart pointers as these would increase the binary size, they generally seemed to imply using things from std would be generally a no go (not all cases).
After the interview, I tried to do research online about coding and what features from standard lib caused large binary sizes and I could find basically nothing in regards to this. Is there a way to quantify using certain features and the size impact they would have (without needing to code 100 smart pointers in a code base vs self managed for example).
This question probably deserves more attention than it’s likely to get, especially for people trying to pursue a career in embedded systems. So far the discussion has gone about the way that I would expect, specifically a lot of conversation about the nuances of exactly how and when a project built with C++ might be more bloated than one written in plain C or a restricted C++ subset.
This is also why you can’t find a definitive answer from a good old fashioned google search. Because if you just ask the question “is C++ more bloated than X?”, the answer is always going to be “it depends.”
So let me approach this from a slightly different angle. I’ve both worked for, and interviewed at companies that enforced these kinds of restrictions, I’ve even voluntarily enforced them myself. It really comes down to this. When you’re running an engineering organization with more than one person with plans to keep hiring, it is wildly impractical to assume everyone on your team is going to fully understand the implications of using every feature of a language. Coding standards and language restrictions serve as a cheap way to prevent people from doing “bad things” without knowing they’re doing “bad things”.
How you define a “bad thing” is then also context specific. On a desktop platform, using lots of code space isn’t really a “bad” enough thing to rigorously enforce. On a tiny embedded system, it probably is.
C++ by design makes it very easy for an engineer to generate lots of code without having to type it out explicitly. I think that statement is pretty self-evident, it’s the whole point of meta-programming, and I doubt anyone would challenge it, in fact it’s one of the strengths of the language.
So then coming back to the organizational challenges, if your primary optimization variable is code space, you probably don’t want to allow people to use features that make it trivial to generate code that isn’t obvious. Some people will use that feature responsibly and some people won’t, but you have to standardize around the least common denominator. A C compiler is very simple. Yes you can write bloated code with it, but if you do, it will probably be pretty obvious from looking at it.
(Partially extracted from comments I wrote earlier)
I don't think there is a comprehensive answer. A lot also depends on the specific use case and needs to be judged on a case-by-case basis.
Templates
Templates may result in code bloat, yes, but they can also avoid it. If your alternative is introducing indirection through function pointers or virtual methods, then the templated function itself may become bigger in code size simply because function calls take several instructions and removes optimization potential.
Another aspect where they can at least not hurt is when used in conjunction with type erasure. The idea here is to write generic code, then put a small template wrapper around it that only provides type safety but does not actually emit any new code. Qt's QList is an example that does this to some extend.
This bare-bones vector type shows what I mean:
class VectorBase
{
protected:
void** start, *end, *capacity;
void push_back(void*);
void* at(std::size_t i);
void clear(void (*cleanup_function)(void*));
};
template<class T>
class Vector: public VectorBase
{
public:
void push_back(T* value)
{ this->VectorBase::push_back(value); }
T* at(std::size_t i)
{ return static_cast<T*>(this->VectorBase::at(i)); }
~Vector()
{ clear(+[](void* object) { delete static_cast<T*>(object); }); }
};
By carefully moving as much code as possible into the non-templated base, the template itself can focus on type-safety and to provide necessary indirections without emitting any code that wouldn't have been here anyway.
(Note: This is just meant as a demonstration of type erasure, not an actually good vector type)
Smart pointers
When written carefully, they won't generate much code that wouldn't be there anyway. Whether an inline function generates a delete statement or the programmer does it manually doesn't really matter.
The main issue that I see with those is that the programmer is better at reasoning about code and avoiding dead code. For example even after a unique_ptr has been moved away, the destructor of the pointer still has to emit code. A programmer knows that the value is NULL, the compiler often doesn't.
Another issue comes up with calling conventions. Objects with destructors are usually passed on the stack, even if you declare them pass-by-value. Same for return values. So a function unique_ptr<foo> bar(unique_ptr<foo> baz) will have higher overhead than foo* bar(foo* baz) simply because pointers have to be put on and off the stack.
Even more egregiously, the calling convention used for example on Linux makes the caller clean up parameters instead of the callee. That means if a function accepts a complex object like a smart pointer by value, a call to the destructor for that parameter is replicated at every call site, instead of putting it once inside the function. Especially with unique_ptr this is so stupid because the function itself may know that the object has been moved away and the destructor is superfluous; but the caller doesn't know this (unless you have LTO).
Shared pointers are a different beast altogether, simply because they allow a lot of different tradeoffs. Should they be atomic? Should they allow type casting, weak pointers, what indirection is used for destruction? Do you really need two raw pointers per shared pointer or can the reference counter be accessed through shared object?
Exceptions, RTTI
Generally avoided and removed via compiler flags.
Library components
On a bare-metal system, pulling in parts of the standard library can have a significant effect that can only be measured after the linker step. I suggest any such project use continuous integration and tracks the code size as a metric.
For example I once added a small feature, I don't remember which, and in its error handling it used std::stringstream. That pulled in the entire iostream library. The resulting code exceeded my entire RAM and ROM capacity. IIRC the issue was that even though exception handling was deactivated, the exception message was still being set up.
Move constructors and destructors
It's a shame that C++'s move semantics aren't the same as for example Rust's where objects can be moved with a simple memcpy and then "forgetting" their original location. In C++ the destructor for a moved object is still invoked, which requires more code in the move constructor / move assignment operator, and in the destructor.
Qt for example accounts for such simple cases in its meta type system.
I have a container of objects stored with unique_ptr, for simplicity say i have only one object :
class Container { std::unique_ptr<A> ptrA; }
I also have class that uses the object. These class take a raw pointer to these objects when they are constructed:
class B { A* a;
B(*A param) : a(param) }
They are created with : B b = B(Container.ptrA.get() );
The Container class is supposed to outlive the class B. However I'd like my whole program not to crash in the case there is an issue or a bug in my class Container and the unique_ptr goes out of scope and get deleted.
My question is about the design you would take to manage this 1% case so my program can try to reload the data and avoid crashing suddenly, would you use exceptions ? If so where would you do try/catch ?
Thanks !
When you use std::unique_ptr you're making a design decision: Container owns the pointer. Trying to work around that fact is only going to make your life harder.
But in fact you said Container outlives B. Why don't you just enforce that instead of being overly defensive against bugs that would probably break your program in several other ways?
I would say don't use shared_ptr to hide bugs. If your unique_ptr is designed to outlive the raw pointer then I would want the program to crash if there is a bug. Then I have something to fix. It's much worse when the bugs go undetected because they are hidden from you. Remember, a crash gives you a point of failure to investigate. But if the bugs go undetected you may not be able to find what's making things go wrong.
If you'd like your program not to crash, then use std::shared_ptr for both pointers.
That would be the easiest solution.
Otherwise, you will need to put in some kind of a mechanism by which the Container class tracks the number of instances of the B class, that use the same pointer, then throw an exception in the destructor if the Container is getting destroyed while there are still an instance of B somewhere. If its unique_ptr is getting blown away for some other reason, other than the destructor getting invoked, the same check would apply there, as well.
That's presuming that throwing an exception is what you would like to do to handle this edge case. It's not clear what you mean "can try to reload the data", but as then designer and the implementer of your application you need to decide how you are going to handle this situation. Nobody else can make the call for you, you know more about your overall application than anyone else. There is no universal, single answer here that will work best for every application in every situation.
But whatever you decide should be an appropriate course of action: throw an exception; or create a new instance of the object, stuff it into the unique_ptr and then update all native pointers in all the B classes that you're keeping track of, somehow; that would be your call to make. What's the best approach is a subjective call to make. There is no objective answer for that part.
Now, getting back to the technical aspects, keeping track of how many instances of the B class can be as simple as keeping a counter in the container, and have B's constructor and destructor update it accordingly. Or maybe have Container keep a container of pointers to all instances of B. In either case, don't forget to do the right thing in the copy constructor and the assignment operator.
But I think it's just easier to use use a std::shared_ptr in both classes, and not worry about any of this. Even though doing this kind of class bookkeeping is not rocket science, why bother when you can simply have std::shared_ptr do this for you.
Philosophically: this is not a great idea, at least in C++.
The Container class is supposed to outlive the class B. However I'd like my whole program not to crash in the case there is an issue or a bug ...
It sounds like you want a "safer" language.
The idea that you can write code that "should" work but is robust against ownership/lifetime errors is...pretty much anathema to the goals of low-level languages like C++ with explicit lifetime management, I think.
If you really want to write a program that simply doesn't crash, use a language with a runtime that manages memory and lifetimes for you—that is, a garbage-collected language like Java or Python. These languages are designed to "protect you from yourself," so to speak. In theory, they prevent you from encountering the sorts of errors you're describing by managing memory for you.
But part of the point of using low-level languages is to take advantage of explicit memory management. With C++ you can (in theory) write software that, in comparison to software written in managed languages, runs faster, has a smaller memory footprint, and releases system resources (such as filehandles) sooner.
The right approach in C++ is the one you're already using.
Explicitly letting your container class own the underlying objects and representing this ownership using unique_ptr is exactly correct in modern C++, and there is no reason to expect this approach not to work if your system is carefully engineered.
The key question, though, is how can you guarantee that your container class will stay alive and keep your owned objects alive throughout the entire lifetime of the "user" objects (in this case class B instances)? Your question doesn't provide enough details about your architecture for us to answer this, because different designs will require different approaches. But if you can explain how your system (in theory) provides this guarantee, then you are probably on the right track.
If you still have concerns, there are some approaches for dealing with them.
There are many reasons to have valid concerns about lifetime management in C++; a major one is if you are inheriting a legacy codebase and you're not sure it manages lifetimes appropriately.
This can happen even with modern C++ features such as unique_ptr. I'm working on a project that only got started last year, and we've been using C++14 features, including <memory>, since the beginning, and I definitely consider it a "legacy" project:
Multiple engineers who were on the project have now left; 60,000+ lines are "unowned" in the sense that their original author is no longer on the project
There are very few unit tests
There are occasional segfaults :D
Note that a bug in your lifetime management may not cause a crash; if it did, that would be fantastic, because, as Galik says in their answer, this would give you a point of failure to investigate. Unfortunately, there's no way to guarantee that dereferencing a stale pointer will cause a crash, because this is (obviously) undefined behavior. Thus your program could keep running and silently do something utterly disastrous.
Signal-catching
However, a crash—specifically, a segfault—is the most likely result of the error you describe, because a segfault is something you can (sort of) program around.
This is the weakest approach in terms of what kinds of fault-handling behavior you can implement: simply catch the SEGFAULT signal and try to recover from it. Signal-catching functions have some pretty severe limitations, and in general if your lifetime management is screwed up there's probably no way to make reasonable guarantees about what memory you can trust and what memory you can't, so your program might be doomed no matter what you do when you catch the signal.
This is not a good approach to "fixing" broken software; however, it is a very reasonable way to provide a clean exit path for unrecoverable errors (e.g. it will allow you to emulate the classic "memory error at " error messages). Additionally, if all you want to do is to restart your entire application and hope for the best, you can probably implement this using a signal-catcher, although a better approach may be to implement a second "watcher" application that restarts your software when it crashes.
std::shared_ptr
Joachim Pileborg is correct that a std::shared_ptr will work in this case, but (1) shared_ptr has some overhead compared to raw pointers (if you care about that) and (2) it requires changing your entire lifetime-management scheme.
Also, as pointed out by Galik in the comments, when there is a lifetime-management bug, the lifetime of the owned object will be extended; the object will still exist after the shared_ptr has been removed from the container if any shared_ptrs in your B class instances are still active.
std::weak_ptr
Your best bet might be a weak_ptr. This also requires changing your lifetime-management scheme to use shared_ptr, but it has the benefit of not keeping old objects around just because a shared_ptr to them exists somewhere outside of your lifetime-managing containers.
Not all low-level languages are this unforgiving, though.
I'm a bit biased because I love the philosophies behind the Rust language, so this is a bit of a plug. Rust enforces correct lifetime-management at compile-time. Rust is just as low-level as C++ in the sense that it gives full control over memory management, memory access, etc., but it's a "modern" high-level language in that it's closer to a redesigned version of C++ than it is to, say, C.
But the key point for our purposes is that the limitations Rust puts on you in terms of what it considers an "ownership" or lifetime-management error enable far better guarantees of program correctness than any possible static analysis of a C or C++ program ever could.
OK, so: we all know that generally the use of const_cast<>() anywhere is so bad it’s practically a programming war crime. So this is a hypothetical question about how bad it might be, exactly, in a specific case.
To wit: I ran across some code that did something like this:
std::string temporary = "/tmp/directory-XXXXX";
const char* dtemp = ::mkdtemp(const_cast<char*>(temporary.c_str()));
/// `temporary` is unused hereafter
… now, I have run across numerous descriptions about how to get writeable access to the underlying buffer of a std::string instance (q.v. https://stackoverflow.com/a/15863513/298171 for example) – all of them have the caveat that yes, these methods aren’t guaranteed to work by any C++ standard, but in practice they all do.
With this in mind, I am just curious on how using const_cast<char*>(string.c_str()) compares to other known methods (e.g. the aforementioned &string[0], &c)… I ask because the code in which I found this method in use seems to work fine in practice, and I thought I’d see what the experts thought before I attempt the inevitable const_cast<>()-free rewrite.
const cannot be enforced at hardware level because in practice, in non-hypothetical environment, you can set read-only attribute only to a full 4K memory page and there are huge pages on the way, which drastically reduce CPU's lookup misses in the TLB.
const doesn't affect code generation like __restrict from C99 does. In fact, const, roughly speaking, means "poison all write attempts to this data, I'd like to protect my invariants here"
Since std::string is a mutable string, its underlying buffer cannot be allocated in read-only memory. So const_cast<> shouldn't cause program crash here unless you're going to change some bytes outside of underlying buffer's bounds or trying to delete, free() or realloc() something. However, altering of chars in the buffer may be classified as invariant violation. Because you don't use std::string instance after that and simply throw it away this shouldn't provoke program crash unless some particular std::string implementation decide to check its invariants' integrity before destruction and force a crash if some of these are broken. Because such check couldn't be done in less than O(N) time and std::string is a performance critical class, it is unlikely to be done by anyone.
Another issue may come from Copy-on-Write strategy. So, by modifying the buffer directly you may break some other std::string's instance which shares the buffer with your string. But few years ago majority of C++ experts came to conclusion that COW is too fragile and too slow especially in multi-threaded environments, so modern C++ libraries shouldn't use it and instead adhere to using move construction where possible and avoiding heap traffic for small length strings where applicable.
This is definitely subjective, but I'd like to try to avoid it
becoming argumentative. I think it could be an interesting question if
people treat it appropriately.
In my several recent projects I used to implement architectures where long delegation chains are a common thing.
Dual delegation chains can be encountered very often:
bool Exists = Env->FileSystem->FileExists( "foo.txt" );
And triple delegation is not rare at all:
Env->Renderer->GetCanvas()->TextStr( ... );
Delegation chains of higher order exist but are really scarce.
In above mentioned examples no NULL run-time checks are performed since the objects used are always there and are vital to the functioning of the program and
explicitly constructed when execution starts. Basically I used to split a delegation chain in these cases:
1) I reuse the object obtained through a delegation chain:
{ // make C invisible to the parent scope
clCanvas* C = Env->Renderer->GetCanvas();
C->TextStr( ... );
C->TextStr( ... );
C->TextStr( ... );
}
2) An intermediate object somewhere in the middle of the delegation chain should be checked for NULL before usage. Eg.
clCanvas* C = Env->Renderer->GetCanvas();
if ( C ) C->TextStr( ... );
I used to fight the case (2) by providing proxy objects so that a method can be invoked on non-NULL object leading to an empty result.
My questions are:
Is either of cases (1) or (2) a pattern or an antipattern?
Is there a better way to deal with long delegation chains in C++?
Here are some pros and cons I considered while making my choice:
Pros:
it is very descriptive: it is clear out of 1 line of code where did the object came from
long delegation chains look nice
Cons:
interactive debugging is labored since it is hard to inspect more than one temporary object in the delegation chain
I would like to know other pros and cons of the long delegation chains. Please, present your reasoning and vote based on how well-argued opinion is and not how well you agree with it.
I wouldn't go so far to call either an anti-pattern. However, the first has the disadvantage that your variable C is visible even after it's logically relevant (too gratuitous scoping).
You can get around this by using this syntax:
if (clCanvas* C = Env->Renderer->GetCanvas()) {
C->TextStr( ... );
/* some more things with C */
}
This is allowed in C++ (while it's not in C) and allows you to keep proper scope (C is scoped as if it were inside the conditional's block) and check for NULL.
Asserting that something is not NULL is by all means better than getting killed by a SegFault. So I wouldn't recommend simply skipping these checks, unless you're a 100% sure that that pointer can never ever be NULL.
Additionally, you could encapsulate your checks in an extra free function, if you feel particularly dandy:
template <typename T>
T notNULL(T value) {
assert(value);
return value;
}
// e.g.
notNULL(notNULL(Env)->Renderer->GetCanvas())->TextStr();
In my experience, chains like that often contains getters that are less than trivial, leading to inefficiencies. I think that (1) is a reasonable approach. Using proxy objects seems like an overkill. I would rather see a crash on a NULL pointer rather than using a proxy objects.
Such long chain of delegation should not happens if you follow the Law of Demeter. I've often argued with some of its proponents that they where holding themselves to it too conscientiously, but if you come to the point to wonder how best to handle long delegation chains, you should probably be a little more compliant with its recommendations.
Interesting question, I think this is open to interpretation, but:
My Two Cents
Design patterns are just reusable solutions to common problems which are generic enough to be widely applied in object oriented (usually) programming. Many common patterns will start you out with interfaces, inheritance chains, and/or containment relationships that will result in you using chaining to call things to some extent. The patterns are not trying to solve a programming issue like this though - chaining is just a side effect of them solving the functional problems at hand. So, I wouldn't really consider it a pattern.
Equally, anti-patterns are approaches that (in my mind) counter-act the purpose of design patterns. For example, design patterns are all about structure and the adaptability of your code. People consider a singleton an anti-pattern because it (often, not always) results in spider-web like code due to the fact that it inherently creates a global, and when you have many, your design deteriorates fast.
So, again, your chaining problem doesn't necessarily indicate good or bad design - it's not related to the functional objectives of patterns or the drawbacks of anti-patterns. Some designs just have a lot of nested objects even when designed well.
What to do about it:
Long delegation chains can definitely be a pain in the butt after a while, and as long as your design dictates that the pointers in those chains won't be reassigned, I think saving a temporary pointer to the point in the chain you're interested in is completely fine (function scope or less preferably).
Personally though, I'm against saving a permanent pointer to a part of the chain as a class member as I've seen that end up in people having 30 pointers to sub objects permanently stored, and you lose all conception of how the objects are laid out in the pattern or architecture you're working with.
One other thought - I'm not sure if I like this or not, but I've seen some people create a private (for your sanity) function that navigates the chain so you can recall that and not deal with issues about whether or not your pointer changes under the covers, or whether or not you have nulls. It can be nice to wrap all that logic up once, put a nice comment at the top of the function stating which part of the chain it gets the pointer from, and then just use the function result directly in your code instead of using your delegation chain each time.
Performance
My last note would be that this wrap-in-function approach as well as your delegation chain approach both suffer from performance drawbacks. Saving a temporary pointer lets you avoid the extra two dereferences potentially many times if you're using these objects in a loop. Equally, storing the pointer from the function call will avoid the over head of an extra function call every loop cycle.
For bool Exists = Env->FileSystem->FileExists( "foo.txt" ); I'd rather go for an even more detailed breakdown of your chain, so in my ideal world, there are the following lines of code:
Environment* env = GetEnv();
FileSystem* fs = env->FileSystem;
bool exists = fs->FileExists( "foo.txt" );
and why? Some reasons:
readability: my attention gets lost till I have to read to the end of the line in case of bool Exists = Env->FileSystem->FileExists( "foo.txt" ); It's just too long for me.
validity: regardles that you mentioned the objects are, if your company tomorrow hires a new programmer and he starts writing code, the day after tomorrow the objects might not be there. These long lines are pretty unfriendly, new people might get scared of them and will do something interesting such as optimising them... which will take more experienced programmer extra time to fix.
debugging: if by any chance (and after you have hired the new programmer) the application throws a segmentation fault in the long list of chain it is pretty difficult to find out which object was the guilty one. The more detailed the breakdown the more easier to find the location of the bug.
speed: if you need to do lots of calls for getting the same chain elements, it might be faster to "pull out" a local variable from the chain instead of calling a "proper" getter function for it. I don't know if your code is production or not, but it seems to miss the "proper" getter function, instead it seems to use only the attribute.
Long delegation chains are a bit of a design smell to me.
What a delegation chain tells me is that one piece of code has deep access to an unrelated piece of code, which makes me think of high coupling, which goes against the SOLID design principles.
The main problem I have with this is maintainability. If you're reaching two levels deep, that is two independent pieces of code that could evolve on their own and break under you. This quickly compounds when you have functions inside the chain, because they can contain chains of their own - for example, Renderer->GetCanvas() could be choosing the canvas based on information from another hierarchy of objects and it is difficult to enforce a code path that does not end up reaching deep into objects over the life time of the code base.
The better way would be to create an architecture that obeyed the SOLID principles and used techniques like Dependency Injection and Inversion Of Control to guarantee your objects always have access to what they need to perform their duties. Such an approach also lends itself well to automated and unit testing.
Just my 2 cents.
If it is possible I would use references instead of pointers. So delegates are guaranteed to return valid objects or throw exception.
clCanvas & C = Env.Renderer().GetCanvas();
For objects which can not exist i will provide additional methods such as has, is, etc.
if ( Env.HasRenderer() ) clCanvas* C = Env.Renderer().GetCanvas();
If you can guarantee that all the objects exist, I don't really see a problem in what you're doing. As others have mentioned, even if you think that NULL will never happen, it may just happen anyway.
This being said, I see that you use bare pointers everywhere. What I would suggest is that you start using smart pointers instead. When you use the -> operator, a smart pointer will usually throw if the pointer is NULL. So you avoid a SegFault. Not only that, if you use smart pointers, you can keep copies and the objects don't just disappear under your feet. You have to explicitly reset each smart pointer before the pointer goes to NULL.
This being said, it wouldn't prevent the -> operator from throwing once in a while.
Otherwise I would rather use the approach proposed by AProgrammer. If object A needs a pointer to object C pointed by object B, then the work that object A is doing is probably something that object B should actually be doing. So A can guarantee that it has a pointer to B at all time (because it holds a shared pointer to B and thus it cannot go NULL) and thus it can always call a function on B to do action Z on object C. In function Z, B knows whether it always has a pointer to C or not. That's part of its B's implementation.
Note that with C++11 you have std::smart_ptr<>, so use it!
I'm writing a program that for performance reasons uses shared memory (sockets and pipes as alternatives have been evaluated, and they are not fast enough for my task, generally speaking any IPC method that involves copies is too slow). In the shared memory region I am writing many structs of a fixed size. There is one program responsible for writing the structs into shared memory, and many clients that read from it. However, there is one member of each struct that clients need to write to (a reference count, which they will update atomically). All of the other members should be read only to the clients.
Because clients need to change that one member, they can't map the shared memory region as read only. But they shouldn't be tinkering with the other members either, and since these programs are written in C++, memory corruption is possible. Ideally, it should be as difficult as possible for one client to crash another. I'm only worried about buggy clients, not malicious ones, so imperfect solutions are allowed.
I can try to stop clients from overwriting by declaring the members in the header they use as const, but that won't prevent memory corruption (buffer overflows, bad casts, etc.) from overwriting. I can insert canaries, but then I have to constantly pay the cost of checking them.
Instead of storing the reference count member directly, I could store a pointer to the actual data in a separate mapped write only page, while keeping the structs in read only mapped pages. This will work, the OS will force my application to crash if I try to write to the pointed to data, but indirect storage can be undesirable when trying to write lock free algorithms, because needing to follow another level of indirection can change whether something can be done atomically.
Is there any way to mark smaller areas of memory such that writing them will cause your app to blow up? Some platforms have hardware watchpoints, and maybe I could activate one of those with inline assembly, but I'd be limited to only 4 at a time on 32-bit x86 and each one could only cover part of the struct because they're limited to 4 bytes. It'd also make my program painful to debug ;)
Edit: I found this rather eye popping paper, but unfortunately it requires using ECC memory and a modified Linux kernel.
I don't think its possible to make a few bits read only like that at the OS level.
One thing that occurred to me just now is that you could put the reference counts in a different page like you suggested. If the structs are a common size, and are all in sequential memory locations you could use pointer arithmetic to locate a reference count from the structures pointer, rather than having a pointer within the structure. This might be better than having a pointer for your use case.
long *refCountersBase;//The start address of the ref counters page
MyStruct *structsBase;//The start address of your structures page
//get address to reference counter
long *getRefCounter(MyStruct *myStruct )
{
size_t n = myStruct - structsBase;
long *ref = refCountersBase + n;
return ref;
}
You would need to add a signal handler for SIGSEGV which recovers from the exception, but only for certain addresses. A starting point might be http://www.opengroup.org/onlinepubs/009695399/basedefs/signal.h.html and the corresponding documentation for your OS.
Edit: I believe what you want is to perform the write and return if the write address is actually OK, and tail-call the previous exception handler (the pointer you get when you install your exception handler) if you want to propagate the exception. I'm not experienced in these things though.
I have never heard of enforcing read-only at less than a page granularity, so you might be out of luck in that direction unless you can put each struct on two pages. If you can afford two pages per struct you can put the ref count on one of the pages and make the other read-only.
You could write an API rather than just use headers. Forcing clients to use the API would remove most corruption issues.
Keeping the data with the reference count rather than on a different page will help with locality of data and so improve cache performance.
You need to consider that a reader may have a problem and fail to properly update its ref count. Also that the writer may fail to complete an update. Coping with these things requires extra checks. You can combine such checks with the API. It may be worth experimenting to measure the performance implications of some kind of integrity checking. It may be fast enough to keep a checksum, something as simple as adler32.