I am working on an OS, and need to load an INITRD (grub command "module") and can't find info on where (in RAM) the module is loaded to
Does grub have a way of telling me or is it a fixed address or none of the above?
Any help would be appreciated
This information is passed to the OS as part of the multiboot information structure data.
Although if you're working on a new OS, you may want to follow the (incompatible) Multiboot2 specification (using grub commands multiboot2/module2) instead.
Related
Question concerns only Windows for now - other OS's are not so relevant right now.
Just by quick googling - it's possible to load native .dll from RAM, there are for example following libraries:
https://www.joachim-bauch.de/tutorials/loading-a-dll-from-memory/
=>
https://github.com/fancycode/MemoryModule
https://forum.nim-lang.org/t/7943
But all of them requires:
in-depth knowledge of PE file format
mostly those approaches are not debugger friendly.
What I have checked - windows's LoadLibraryA / LoadLibraryW are directed to ntdll.dll / LdrLoadDll - and best picture of how things works can be found from here: https://github.com/hlldz/RefleXXion
And even thus I don't have windows source code - I've checked same functionality from Wine:
LdrLoadDll: https://source.winehq.org/source/dlls/ntdll/loader.c#3169
load_dll: https://source.winehq.org/source/dlls/ntdll/loader.c#3083
load_native_dll:
https://source.winehq.org/source/dlls/ntdll/loader.c#2564
NtMapViewOfSection: https://source.winehq.org/source/dlls/ntdll/unix/virtual.c#4469
find_dll_file: https://source.winehq.org/source/dlls/ntdll/loader.c#3021
open_dll_file: https://source.winehq.org/source/dlls/ntdll/loader.c#2467
Suspect loading dll happens via following function calls:
NtOpenFile, NtQueryAttributesFile, NtCreateSection/NtOpenSection, NtMapViewOfSection (*)
(More information could be found in
https://github.com/Hagrid29/PELoader
https://gist.github.com/bats3c/59932dfa1f5bb23dd36071119b91af0f
https://www.octawian.ro/fisiere/situri/asor/build/html/_downloads/122f95f9a032396603a837c53b125bb8/Russinovich_M_WinInternals_part1_7th_ed.pdf
)
I was also thinking if I could just override NtOpenFile and just redirect file open (in
https://github.com/SegaraRai/PathRedirector manner)
to different path - but main question what is the alternative location where to store file?
I was thinking if NtOpenFile can open even device, then maybe just replace file
with some sort of named pipe (https://learn.microsoft.com/en-us/windows/win32/ipc/named-pipe-client) - but then in maps on how well this will work with NtMapViewOfSection.
Since I was not able to find any working example of such hook or operation (E.g. LoadLibary("\\.\pipe\mynamedpipe_as_dll")) - there is always a risk that such combination is not simply supported.
Is it possible to load native .dll purely from RAM:
Without using file system (not to store .dll e.g. in temporary folder)
Without involving custom drivers (like Dokan) ?
So loaded .dll would be still debugger friendly ?
Not tightly bound to PE file format structures (or use PE structures as less as possible)
If you miss bit more information, check also my own experiments with native dll loading (maybe can give some hints on solving the issue):
https://github.com/tapika/test_native_dll_loading
https://github.com/tapika/test_native_dll_loading/discussions/2
Distinguish between debug and release use cases. In debug, save the DLL in a temp file and load with LoadLibrary, which will enable debugging. In release, run from memory with no capability for debugging.
Here's another idea, from considering the linked Guthub issue. If the purpose is to let the users provide their own compression/decompression logic while building a ReadyToRun executable, let them provide that as a static library (object) as opposed to a DLL. The larger project is already about packaging stuff into a single executable, might do some linking while at it.
Yet another idea would be to let the users provide the codec in some kind of interpreted language and optionally plug in the interpreter that supports debugging. Windows comes with a built-in JavaScript interpreter, look up Active Scripting, and debugging those is a free bonus. The performance probably won't be on par with a native code implementation, though.
I think you could probably do something similar with Frida. Hook the functions LoadLibraryA / LoadLibraryW and reimplement them in Frida. but I don't believe this is something that would be stable for production.
For some reference
By analyzing existing approaches (like PE Loader https://github.com/Hagrid29/PELoader) and using minhook library - I've managed to load .dll from RAM.
I've created git repository with example code on github:
https://github.com/tapika/dllloader
Could you create a ramdisk to put your DLL there? What exactly is the use-case for this? There are a couple ways to spin up a file in RAM, C#'s MemoryMappedFile for example. I'm not sure if this would be debugger friendly.
In Linux, I need to get the details(viz. service exit code,status,type,etc) of all the driver modules and I tried reading /proc/modules. But it gives only name, size and use count. I read that modinfo retrieves info from lib/modules/,but all modules doesn't have all info in it. From where can I get these details reliably. I am coding in C++.
Kernel modules are handled by struct module. You may write a module to get that information from in-kernel (and provide it in another /proc file) or use a debugger to read raw kernel memory from /proc/kcore .
But, the information Linux keeps per-module doesn't fit your needs:
service exit code is not saved by kernel but immediately returned to init_module() caller as error number (i.e. modprobe or insmod tools)
status -- there is no such thing. Closest is state, but it transitional and only used during loading
type -- Linux doesn't distinguish module types
Generally speaking, you cannot get that details from Linux. The most useful information is already provided in /proc/modules.
I have been looking into libraries for a file system that will allow path mounting on purely an application level. This may not be called just "path mounting" since that has the connotation of os level path mounting, but something else, I am not sure of the terminology. I was hoping to be able to find a few but were unable to find anything to what I am looking for (boost::filesystem was the closest I found). I wanted to be able to compare several different libraries in hopes of seeing what advantages and disadvantages they have.
What I mean by a file system with path mounting is so I would have a path such as
"SomeRoot:data\file.txt"
and the "SomeRoot" would be replaced with C:\SomeFolder", which would be set to the file mount system.
Does anyone know of a file system that will allow path mounting?
Edit:
Since it appears that there may not be many libraries for this, I would also be interested in how to construct one properly.
If you are looking for an "application level file system" then at the most basic level, you are going to need to do a string replace. On the most basic level there are two strings
MountPoint
Which will be used as the "mount point", such as your SomeRoot.
MountResolve
Which is the location to what mount point is pointed at for when "resolving" a file location. This is the same as your C:\SomeFolder.
Besides for the obvious accessor and getters for those variables, there is the need for a function to resolve the path, which is this case can be
bool ResolvePath(const String& mountPath, String& resolvedPath);
The contents of the ResolvePath are very simple, all you need to do is replace the current MountPoint string in mountPath and place the result into resolvedPath.
resolvedPath = mountPath;
resolvedPath.replace(0, mMountPoint.size() + 1, mMountResolve.c_str(), mMountResolve.size());
However, there is more that can be done in that function. The reason why I have it returning a bool is because the function should fail mountPath does not have the MountPoint. To check, just do a simple string::find.
if(mountPath.find(mMountPoint) == String::npos)
return false;
With this, you can now resolve SomeRoot:data\file.txt to C:\SomeFolder\data\file.txt if MountResolve is set to C:\SomeFolder\. However, you mentioned without the trailing slash at the end. Since there is nothing to be currently done to verify that slash, your result would be C:\SomeFolderdata\file.txt. This is wrong.
On your access for setting the mount resolve, you want to check to see if there is there is a trailing folder slash. If there is not, then add it.
void FileSystem::SetMountResolve(const String& mountResolve)
{
mMountResolve = mountResolve;
if(*(mMountResolve.end() - 1) != FOLDERSLASH)
mMountResolve += FOLDERSLASH;
}
This will allow a basic "FileSystem" class to have one MountPoint/MountResolve. It will not be very difficult to extend this to allow multiple mount points either.
I have been looking into libraries for a file system that will allow path mounting
You should forget about it. mounting a path/drive can be blocked on linux (administrator privilegies might be required), and on windows there is no built-in mechanism for that (there are directory junctions, though). On certain distros you have to be root to mount even cdrom drive. Manually.
Does anyone know of a file system that will allow path mounting?
ntfs, ext3, jfs. Operation might require root/administrator privilegies.
and the "SomeRoot" would be replaced with C:\SomeFolder"
You need to use something similar to environmental variables in your program. Use "${SomeDir}/path" and replace ${SomeDir} with whatever you want. That'll be much easier to implement than mounting.
--EDIT--
What I mean by a file system with path mounting is so I would have a path such as
"SomeRoot:data\file.txt"
Provide custom wrapper for fopen or whatever you use instead. Or make custom class that implements "File". In that class/wrapper add support for "mounting" by doing search/replace on provided file path - possibly using collection of variables stored within the program. That's the easiest solution. Of course, you'll also have to wrap other file functions you'll use, but that's still much easier than doing cross-platform mounting.
You might also want to consider PhysicsFS. In essence, it is a library that abstracts the location of files. You may define a "search path" and whenever you read a file it is searched in those locations, i.e., folders or archives. For example, if you want to read "logo.png", it might be searched in locations such as:
C:\mygame\data
E:\mygame\data (read-only CD-ROM)
C:\mygame\data.zip
When writing a file, it is always stored in a special "write path".
PhysicsFS also has a concept of application-level mounting. Citing from their webpage:
PhysicsFS 2.0 adds the concept of "mounting" archives to arbitrary
points in the search path. If a zipfile contains "maps/level.map" and
you mount that archive at "mods/mymod", then you would have to open
"mods/mymod/maps/level.map" to access the file, even though
"mods/mymod" isn't actually specified in the .zip file. Unlike the
Unix mentality of mounting a filesystem, "mods/mymod" doesn't actually
have to exist when mounting the zipfile. It's a "virtual" directory.
And, of course, the whole solution is 100% user-space and requires no kernel support nor administrative privileges.
FUSE on Unix, FUSE4X on MacOS X, our Callback File System with FUSE adapter on Windows. There's no single cross-platform solution because the architecture of file system drivers is different on those platforms.
Seems like you are looking for symbolic links functionality.
On Posix systems, e.g. Linux/Unix you can look into link()/symlink(). This functionality has been there forever and is pretty solid.
On Windows you may want to look into CreateSymbolicLink() and company. These were introduced with Windows 2000 and I'm not sure how robust they are.
Actual mounting of filesystems is trickier business and really depends on what you are mounting - NTFS, FAT, ext3, XFS, NFS, CIFS, WebDAV, etc, etc.
I am trying to obtain the memory working set value for a given PID in my C++ application running on LINUX. In Windows I can get this info using GetProcessWorkingSetSize function. Is there anything like this function I can call in LINUX?
The only sensible solution that comes to mind is accessing the relevant information via the /proc filesystem. It seems weird that a process would have to read out its own information from /proc, though, but I don't know about any other system calls that might make this easier.
The information you're probably most interested in is located in /proc/[pid]/statm, which includes :
total program size,
resident set size,
shared pages,
text (code) size,
library (unused in Linux 2.6),
data and stack size,
dirty pages (unused in Linux 2.6).
Keep in mind that all those measurements are given in the number of pages.
Is there a way to read the the import table of another process?
The function ImageNtHeader won't help me here, because it applies only to my process and not to the other process.
I know I can read the entire file and parse its PE header, but I'm afraid it will take a lot of time if the file is large.
Is there a way to do it using the process's memory directly and not reading from the file?
Any other easy and nice way to do it will be great as well :)
just use psapi or toolhlp32 to find the memory addresses of the processes modules you are interested in. once you have that you can use ReadProcessMemory get the required info. else just pretend to be a debugger and attach to the target process so you have full access to it's memory, then you can use the windows PE macros.