Dynamic Linking ~ Limiting a DLL's system access - c++

I know the question might seem a little vague but I will try to explain as clearly as I can.
In C++ there is a way to dynamically link code to your already running program. I am thinking about creating my own plugin system (For learning/research purposes) but I'd like to limit the plugins to specific system access for security purposes.
I would like to give the plugins limited access to for example disk writing such that it can only call functions from API I pass from my application (and write through my predefined interface) Is there a way to enforce this kind of behaviour from the application side?
If not: Are there other language's that support secure dynamically linked modules?

You should think of writing a plugin container (or a sand-box), then coordinate everything through the container, also make sure to drop privileges that you do not need inside the container process before running the plugin. Being run in a process means, you can run the container also as a unique user and not the one who started the process, after that you can limit the user and automatically the process will be limited. Having a dedicated user for a process is the most common and easiest way, it is also the only cross-platform way to limit a process, even on Windows you can use this method to limit a process.
Limiting access to shared resources that OS provides, like disk or RAM or CPU depends heavily on the OS, and you have not specified what OS. While it is doable on most OSes, Linux is the prime choice because it is written with multi-seat and server-use-cases in mind. For example in Linux you can use cgroups here to limit CPU, or RAM easily for each process, then you will only need to apply it for your plugin container process. There is blkio to control disk access, but you can still use the traditional quote mechanism in Linux to limit per-process or per-user share of disk space.
Supporting plugins is an involved process, and the best way to start is reading code that does some of that, Chromium sand-boxing is best place I can suggest, it is very cleanly written, and has nice documentation. Fortunately the code is not very big.
If you prefer less involvement with actual cgroups, there is an even easier mechanism for limiting resources, docker is fairly new but abstracts away low level OS constructs to easily contain applications, without the need to run them in Virtual Machines.

To block some calls, a first idea may be to hook the system calls which are forbidden and others API call which you don't want. You can also hook the dynamic linking calls to prevent your plugins to load another DLLs. Hook disk read/write API to block read/write.
Take a look at this, it may give you an idea to how can you forbid function calls.
You can also try to sandbox your plugins, try to look some open source sandbox and understand how they work. It should help you.

In this case you really have to sandbox the environment in that the DLL runs. Building such a sandbox is not easy at all, and it is something you probably do not want to do at all. System calls can be hidden in strings, or generated through meta programming at execution time, so hard to detect by just analysing the binary. Luckyly people have already build solutions. For example google's project native client with the goal to generally allow C++ code to be run safely in the browser. And when it is safe enough for a browser, it is probably safe enough for you and it might work outside of the browser.

Related

How (and why?) do memory editors work

Memory editors such as Cheat Engine are able to read the memory of other processes and modify it.
How do they do it?(a code snippet would be interesting!) A process does typically not have the ability to access the memory of another one, the only cases that I've heard of are in sub-processes/threading, but memory editors are typically not related to the target process in any way.
Why do they work? In what scenario is this ability useful aside from using it to hack other processes, why wouldn't the operating system simply disallow unrelated processes from reading the memory of each other?
On Windows, the function typically used to alter the memory of another process is called WriteProcessMemory:
https://learn.microsoft.com/en-in/windows/win32/api/memoryapi/nf-memoryapi-writeprocessmemory
If you search the Cheat Engine source code for WriteProcessMemory you can find it both in their Pascal code and the C kernel code. It needs PROCESS_VM_WRITE and PROCESS_VM_OPERATION access to the process which basically means you need to run Cheat Engine as admin.
WriteProcessMemory is used any time you want to alter the runtime behavior of another process. There are legitimate uses, such as with Cheat Engine or ModOrganizer, and of course lots of illegitimate ones. It's worth mentioning that anti-virus software is typically trained to look for this API call (among others) so unless your application has been whitelisted it might get flagged because of it.
Operating systems typically expose system calls that allow a userspace program to inspect a target process's memory using said system calls.
For instance, the linux kernel supports the ptrace system call. This system call is primarily used by the well known debugger gdb and by popular system call tracing utilities such as strace.
The ptrace system call allows for the inspection of memory contents of the target process, code injection, register manipulation and much more. I would suggest this as a resource if you are interested in learning more.
On Linux, you can either run a binary from within gdb, or attach to a process. In case of the latter, you need elevated privileges. There are other protections that try to limit what you can do with ptrace, such as the one mentioned here.
On Windows you can achieve the same effect by using the following functions in order : OpenProcess, GetProcAddress, VirtualAllocEx, WriteProcessMemory and CreateRemoteThread. I would suggest this as a resource if you are interested in knowing more. You might need elevated privileges to do this on newer Windows versions.

How to have my program stops if its memory consumption exceeds a limit ?

I develop a C++ framework that is used to run user code in a well defined environment (Linux boxes under our supervision).
I would like to prevent badly written modules to start eating up all memory of a machine. As I develop the framework could I simply force the program to stop itself if its memory consumption is too high ? What api or tool should I use for this ?
A simple mechanism for controlling a process's resource limits is provided by setrlimit. But that doesn't isolate the process (as you should for untrusted third-party code), it only puts some restrictions on it. To properly isolate a process from the rest of the system, you should either use a VM, or make use of cgroups and kernel namespaces — preferrably not by hand, but via some existing library or framework (such as Docker).
How to have my program stops if its memory consumption exceeds a limit ?
When you define the interface between the application and it's modules, ensure that one of the first steps (probably the first) will be to pass an allocator-like class instance, from the application to the modules.
This instance should be used in the module to allocate and deallocate all necessary memory.
This will allow implementations of this allocator instance, to report memory allocations to the main application, which should be able to triggering an exception, if a limit (per module or per application) is reached.
You can directly provide your own operator new. However, that won't protect you from calls to malloc, or direct OS calls. This would require patching or wrapping glibc (since you're on Linux). Doable but not nice.
What's your desired security level? Are you protecting against Murphy or Machiavelli ? Might a plugin use a third-party library which allocates memory on bahalf of the plugin? Do you need to keep track of the plugin that allocated the memory?

How to create a sandbox environment in C++?

Is there a way to create a sandbox environment inside C++ where you can either:
- Run processes in
- Load Dynamic Libraries in (Linux)
Dynamic Libraries are preferred because the easy communication between the main process and the sand-boxed processes.
A sand-boxed process should not be able to put memory on the heap or interact with the Kernel / Operating System. Instead the main process will provide an interface to do these things.
Is there any way to do this? I could create a script interpreter but that'd take away a lot of the speed. I'd like to keep the speed loss minimal.
You can use some software which provides sandbox environment. Use can provide memory limit, time limit etc. and other parameters to the application. I used a sandbox file when I created some online judge and needed to execute c++ files of other users in a limited environment.

What is a Sandbox

When anti-viruses run some application in a virtual environment called a "sandbox", how does this sandbox precisely work from the Windows kernel point of view?
Is it hard to write such a sandbox?
At a high level such sandboxes are kernel drivers which intercept calls to APIs, and modify the results those APIs return using hooking. How an entire sandboxing solution works under the hood though, could easily fill several books.
As for difficulty, it's probably one of the harder things you could ever possibly write. Not only do you have to provide hooks for most everything the operating system kernel provides, but you have to prevent the application from accessing the memory space of other processes, you have to have a way to save the state of the changes a program makes so that the program does not realize it's running under a sandbox. You have to do all of this in Kernel mode, which effectively limits you to using C, and forces you to deal with different kinds of memory, e.g. paged pool and nonpaged pool. Oh, and you have to do all of this very fast, so that the user feels it's worthwhile to run applications inside your sandbox. 50+% performance hits won't be tolerated by most users.

how to check performance of a c++ api

my web server has a lot of dependencies for sending back data, when it gets a request. i am testing one of these dependency applications within the web server. the application is decoupled from the main web server, and only queries are going to it in the form of api's exposed.
my question is, if i wish to check these api's in a multithreaded environment (c++ functions with a 2 quadcore processor machine), what is the best wy to go about doing it?
do i call each api in a separate thread or process? if so, how do i implement such code? from what i can figure out, i would be duplicating the functioning of the web server, but i can find no other better way to figure out the performance improvements given by that component alone.
It depends on whether your app deails with data that's shared if it is run in parallel processes because that'll most likely determine where the speed bottleneck awaits.
E.g, if the app accesses a database or disk files, you'll probably have to simulate multiple threads/processes querying the app in order to see how they get along with each other, i.e. whether they have to wait for each other while accessing the shared resource.
But if the app only does some internal calculation, all by its own, then it may scale well, as long as all its data fits into memory (i.e. not virtual memory access, e.g. disk access, necessary). Then you can test the performance of just one instance and focus on optimizing its speed.
It also might help to state the OS you're planning to use. Mac OS X offers tools for performance testing and optimization that Windows and Linux may not, and vice versa.