Good Evening,
i would like to make use of a custom jump table in an current design.
As i dont know how to do this properly, i do like to ask for help.
Setup:
My project is going to run on an ARM Cortex-M4F Device.
The Device does have 256KB of Flash. I am going to allocate 128kB for System-Code and 128kB for User-Code. The Device does have 96kB of Ram. I am going to allocate 48kB for System-Code and 48kB for User-Code.
System-Code is code flashed to the device once via SWD. It does handle all the Peripherals and so on.
User-Code is code, which can be flashed via Bootloader (located in System-Code).
User-Code must not use any blocking calls or ISRs or so forth.
The goal of the project is not to provide a tasker or any sort of kernel.
My Goal:
Upon an external Event (Pin-Trigger ISR) the System-Code (declaring the associated ISR) does start a series of function calls. It does read external devices and performs some calculations. Afterwards, the System-Code does write some RAM data into the User-RAM section (fixed Adress and Size via Linker script). Then the System-Code should invoke the User-Code.
The User-Code does consist of an array of up to 2048 functions (Static in application, but unknown at compile time of System-Code as User-Code can be reflashed).
These functions must not return any values or take any arguments.
I would like to know, how to properly allocated and subsequently call these functions (Non-Blocking, no ISRs, shared Stack).
My requirements:
User-Functions do not block. They run to completion. No preemption and so forth is required.
User-Functions can be interrupted by ISRs.
The User-functions must not have any parameters and must not return any values.
They can use the common stack.
My approach:
Define a Section for the User-Functions-Table in both linker scripts (System and User - seperate projects and compilation units) at the Same adresses (1x Flash-Page e.g.)
Treat that page as an array of pointers to functions in the System-Code. Call as required (Handling max. index and so on seperatly).
In the user-code allocate an array of pointers to functions in this section as well. Implement the functions somewhere else and assign them to the array. Decalre them as attribute(isr) to make sure registers are preserved.
My questions:
Is tis approach possible at all?
Are there simpler approaches?
What issue can you see, which i cant? (Novice)
How would it be possible to pass parameters from system-code to these function calls? Is the signature of a pointer to a function like void DoX(char x, int z) the same as to void DoY(void)?
Thank you very much.
Related
I was trying to create a memory buffer in OpenCL with C++ binding. The sentence looks like
cl::Buffer buffer(context,CL_MEM_READ_ONLY,sizeof(float)*(100));
This sentence confuses me because it doesn't specify which device the memory is allocated on. In principle context contains all devices, including cpu and gpu, on the chosen platform. Is it true that the buffer is put in a common region shared by all the devices?
The spec does not define where the memory is. For the API user, it is "in the context".
If you have one device only, probably (99.99%) is going to be in the device. (In rare cases it may be in the host if the device does not have enough memory for the time being)
In case of many different devices, it will be in one of them at the creation. But it may move transparently to another device depending on the kernel launches.
This is the reason why the call clEnqueueMIgrateMemObjects (OpenCL 1.2 only) exists.
It allows the user to tell some hints to the API about where the memory will be needed, and prepare the copy in advance.
Here is the definition of what it does:
clEnqueueMIgrateMemObjects provides a mechanism for assigning which device an OpenCL memory object resides. A user may wish to have more explicit control over the location of their memory objects on creation. This could be used to:
Ensure that an object is allocated on a specific device prior to usage.
Preemptively migrate an object from one device to another.
Typically, memory objects are implicitly migrated to a device for which enqueued commands, using the memory object, are targeted
Can I allocate a block on the heap, set its bytes to values that correspond to a function call and its parameters, then use the function call and dereference operators to execute that sequence?
So if I read you right you want to dynamically create CPU assembly instructions on the heap and execute them. A bit like self-modifying code. In theory that's possible, but in practice maybe not.
The problem is that the heap is in a data segment, and CPU's/operating systems nowadays have measures to prevent exactly this kind of behavior (it's called the NX bit, or No-eXecute bit for x86 CPUs). If a segement is marked as NX, you can't execute code from it. This was invented to stop computer virusses from using buffer overflows to place exectuable code in data/heap/stack memory and then try the calling program to execute such code.
Note that DLL's and libraries are loaded in the code segment, which of course allows code execution.
Yes. How else could Dynamic loading and Linking work? Remembering that some (most?) Operating Systems, and some (most?) Linkers are also written in C/C++. For example,
#include <dlfcn.h>
void* initializer = dlsym(sdl_library,"SDL_Init");
if (initializer == NULL) {
// report error ...
} else {
// cast initializer to its proper type and use
}
Also, I believe that a JIT (e.g. GNU lightning and others) in general performs those operations.
In windows, for example, this is now very hard to do when it was once very easy. I used to be able to take an array of bytes in C and then cast it to a function pointer type to execute it... but not any more.
Now, you can do this if you can call Global or VirtualAlloc functions and specifically ask for executable memory. On most platforms its either completely open or massively locked down. Doing this sort of thing on iOS, for example, is a massive headache and it will cause a submission fail on the app store if discovered.
here is some fantastically out of date and crusty code where i did the original thing you described:
https://code.google.com/p/fridgescript/source/browse/trunk/src/w32/Code/Platform_FSCompiledCode.cpp
using bytes from https://code.google.com/p/fsassembler
you may notice in there that i need to provide platform (windows) specific allocation functions to get some executable memory:
https://code.google.com/p/fridgescript/source/browse/trunk/src/w32/Core/Platform_FSExecutableAlloc.cpp
Yes, but you must ensure that the memory is marked executable. How you do that depends on the architecture.
I'm trying to understand something about HGLOBALs, because I just found out that what I thought is simply wrong.
In app A I GlobalAlloc() data (with GMEM_SHARE|GMEM_MOVABLE) and place the string "Test" in it. Now, what can I give to another application to get to that data?
I though (wrongfully!) that HGLOBALs are valid in all the processes, which is obviously wrong, because HGLOBAL is a HANDLE to the global data, and not a pointer to the global data (that's where I said "OHHHH!").
So how can I pass the HGLOBAL to another application?
Notice: I want to pass just a "pointer" to the data, not the data itself, like in the clipboard.
Thanks a lot! :-)
(This is just a very long comment as others have already explained that Win32 takes different approach to memory sharing.)
I would say that you are reading into books (or tutorials) on Windows programming which are quite old and obsolete as Win16 is virtually dead for quite some time.
16-bit Windows (3.x) didn't have the concept of memory isolation (or virtual /flat/ address space) that 32-bit (and later) Windows versions provide. Memory there used to be divided into local (to the process) and global sections, both living in the same global address space. Descriptors like HGLOBAL were used to allow memory blocks to be moved around in physical memory and still accessed correctly despite their new location in the address space (after proper fixation with LocalLock()/GlobalLock()). Win32 uses pointers instead since physical memory pages can be moved without affecting their location in the virtual address space. It still provides all of the Global* and Local* API functions for compatibility reasons but they should not be used anymore and usual heap management should be used instead (e.g. malloc() in C or the new operator in C++). Also several different kind of pointers existed on Win16 in order to reflect on the several different addressing modes available on x86 - near (same segment), far (segment:offset) and huge (normalised segment:offset). You can still see things like FARPTR in legacy Win16 code that got ported to Win32 but they are defined to be empty strings as in flat mode only near pointers are used.
Read the documentation. With the introduction of 32-bit processing, GlobalAlloc() does not actually allocate global memory anymore.
To share a memory block with another process, you could allocate the block with GlobalAlloc() and put it on the clipboard, then have the other process retreive it. Or you can allocate a block of shared memory using CreateFileMapping() and MapViewOfFile() instead.
Each process "thinks" that it owns the full memory space available on the computer. No process can "see" the memory space of another process. As such, normally, nothing a process stores can be seen by another process.
Because it can be necessary to pass information between processess, certain mechanisms exists to provide this functionality.
One approach is message passing; one process issues a message to another, for example over a pipe, or a socket, or by a Windows message.
Another is shared memory, where a given block of memory is made available to two or more processes, such that whatever one process writes can be seen by the others.
Don't be confused with GMEM_SHARE flag. It does not work the way you possibly supposed. From MSDN:
The following values are obsolete, but are provided for compatibility
with 16-bit Windows. They are ignored.
GMEM_SHARE
GMEM_SHARE flag explained by Raymond Chen:
In 16-bit Windows, the GMEM_SHARE flag controlled whether the memory
should outlive the process that allocated it.
To share memory with another process/application you instead should take a look at File Mappings: Memory-mapped files and how they work.
I have been looking for a way to dynamically load functions into c++ for some time now, and I think I have finally figure it out. Here is the plan:
Pass the function as a string into C++ (via a socket connection, a file, or something).
Write the string into file.
Have the C++ program compile the file and execute it. If there are any errors, catch them and return it.
Have the newly executed program with the new function pass the memory location of the function to the currently running program.
Save the location of the function to a function pointer variable (the function will always have the same return type and arguments, so
this simplifies the declaration of the pointer).
Run the new function with the function pointer.
The issue is that after step 4, I do not want to keep the new program running since if I do this very often, many running programs will suck up threads. Is there some way to close the new program, but preserve the memory location where the new function is stored? I do not want it being overwritten or made available to other programs while it is still in use.
If you guys have any suggestions for the other steps as well, that would be appreciated as well. There might be other libraries that do things similar to this, and it is fine to recommend them, but this is the approach I want to look into — if not for the accomplishment of it, then for the knowledge of knowing how to do so.
Edit: I am aware of dynamically linked libraries. This is something I am largely looking into to gain a better understanding of how things work in C++.
I can't see how this can work. When you run the new program it'll be a separate process and so any addresses in its process space have no meaning in the original process.
And not just that, but the code you want to call doesn't even exist in the original process, so there's no way to call it in the original process.
As Nick says in his answer, you need either a DLL/shared library or you have to set up some form of interprocess communication so the original process can send data to the new process to be operated on by the function in question and then sent back to the original process.
How about a Dynamic Link Library?
These can be linked/unlinked/replaced at runtime.
Or, if you really want to communicated between processes, you could use a named pipe.
edit- you can also create named shared memory.
for the step 4. we can't directly pass the memory location(address) from one process to another process because the two process use the different virtual memory space. One process can't use memory in other process.
So you need create a shared memory through two processes. and copy your function to this memory, then you can close the newly process.
for shared memory, if in windows, looks Creating Named Shared Memory
http://msdn.microsoft.com/en-us/library/windows/desktop/aa366551(v=vs.85).aspx
after that, you still create another memory space to copy function to it again.
The idea is that the normal memory allocated only has read/write properties, if execute the programmer on it, the CPU will generate the exception.
So, if in windows, you need use VirtualAlloc to allocate the memory with the flag,PAGE_EXECUTE_READWRITE (http://msdn.microsoft.com/en-us/library/windows/desktop/aa366887(v=vs.85).aspx)
void* address = NULL;
address= VirtualAlloc(NULL,
sizeof(emitcode),
MEM_COMMIT|MEM_RESERVE,
PAGE_EXECUTE_READWRITE);
After copy the function to address, you can call the function in address, but need be very careful to keep the stack balance.
Dynamic library are best suited for your problem. Also forget about launching a different process, it's another problem by itself, but in addition to the post above, provided that you did the virtual alloc correctly, just call your function within the same "loadder", then you shouldn't have to worry since you will be running the same RAM size bound stack.
The real problems are:
1 - Compiling the function you want to load, offline from the main program.
2 - Extract the relevant code from the binary produced by the compiler.
3 - Load the string.
1 and 2 require deep understanding of the entire compiler suite, including compiler flag options, linker, etc ... not just the IDE's push buttons ...
If you are OK, with 1 and 2, you should know why using a std::string or anything but pure char *, is an harmfull.
I could continue the entire story but it definitely deserve it's book, since this is Hacker/Cracker way of doing things I strongly recommand to the normal user the use of dynamic library, this is why they exists.
Usually we call this code injection ...
Basically it is forbidden by any modern operating system to access something for exceution after the initial loading has been done for sake of security, so we must fall back to OS wide validated dynamic libraries.
That's said, one you have valid compiled code, if you realy want to achieve that effect you must load your function into memory then define it as executable ( clear the NX bit ) in a system specific way.
But let's be clear, your function must be code position independant and you have no help from the dynamic linker in order to resolve symbol ... that's the hard part of the job.
I'm aware that each process creates it's own memory address space, however I was wondering,
If Process A was to have a function like :
int DoStuff() { return 1; }
and a pointer typedef like :
typedef int(DoStuff_f*)();
and a getter function like :
DoStuff_f * getDoStuff() { return DoStuff; }
and a magical way to communicate with Process B via... say boost::interprocess
would it be possible to pass the function pointer to process B and call
Process A's DoStuff from Process B directly?
No. All a function pointer is is an address in your process's address space. It has no intrinsic marker that is unique to different processes. So, even if your function pointer just happened to still be valid once you've moved it over to B, it would call that function on behalf of process B.
For example, if you had
////PROCESS A////
int processA_myfun() { return 3; }
// get a pointer to pA_mf and pass it to process B
////PROCESS B////
int processB_myfun() { return 4; } // This happens to be at the same virtual address as pA_myfun
// get address from process A
int x = call_myfun(); // call via the pointer
x == 4; // x is 4, because we called process B's version!
If process A and B are running the same code, you might end up with identical functions at identical addresses - but you'll still be working with B's data structures and global memory! So the short answer is, no, this is not how you want to do this!
Also, security measures such as address space layout randomization could prevent these sort of "tricks" from ever working.
You're confusing IPC and RPC. IPC is for communicating data, such as your objects or a blob of text. RPC is for causing code to be executed in a remote process.
In short, you cannot use function pointer that passed to another process.
Codes of function are located in protected pages of memory, you cannot write to them. And each process has isolated virtual address space, so address of function is not valid in another process. In Windows you could use technique described in this article to inject your code in another process, but latest version of Windows rejects it.
Instead of passing function pointer, you should consider creating a library which will be used in both processes. In this case you could send message to another process when you need to call that function.
If you tried to use process A's function pointer from process B, you wouldn't be calling process A - you'd call whatever is at the same address in process B. If they are the same program you might get lucky and it will be the same code, but it won't have access to any of the data contained in process A.
A function pointer won't work for this, because it only contains the starting address for the code; if the code in question doesn't exist in the other process, or (due to something like address space randomization) is at a different location, the function pointer will be useless; in the second process, it will point to something, or nothing, but almost certainly not where you want it to.
You could, if you were insane^Wdaring, copy the actual instruction sequence onto the shared memory and then have the second process jump directly to it - but even if you could get this to work, the function would still run in Process B, not Process A.
It sounds like what you want is actually some sort of message-passing or RPC system.
This is why people have invented things like COM, RPC and CORBA. Each of them gives this general kind of capability. As you'd guess, each does so the job a bit differently from the others.
Boost IPC doesn't really support remote procedure calls. It will enable putting a variable in shared memory so its accessible to two processes, but if you want to use a getter/setter to access that variable, you'll have to do that yourself.
Those are all basically wrappers to produce a "palatable" version of something you can do without them though. In Windows, for example, you can put a variable in shared memory on your own. You can do the same in Linux. The Boost library is a fairly "thin" library around those, that lets you write the same code for Windows or Linux, but doesn't try to build a lot on top of that. CORBA (for one example) is a much thicker layer, providing a relatively complete distributed environment.
If both processes are in the same application, then this should work. If you are trying to send function pointers between applications then you are out of luck.
My original answer was correct if you assume a process and a thread are the same thing, which they're not. The other answers are correct - different processes cannot share function pointers (or any other kind of pointers, for that matter).