I am working on a project to replicate windows write requests from one hard disk to another in real time asynchronously.
I am new to kernel mode drivers development under windows, what I have in mind is to monitor IO requests.
I searched a lot but this area is not documented as it should.
I did my search and found 4 leads (but non of them is sure the best option to start with)
IRP
IO request packets used by windows drivers to communicate between them and the OS, but will I be able to monitor the IRP used by the hard disk driver ? and what about the security?
Windows filter drivers
there is a lot of system filter drivers levels from upper to lower, which one should be used? and Will I be able to filter the IOs used by the hard disk driver?
IO hooks
this technique is used by windows antiviruses to hook the IOs and check the files. Is it reliable to use in my situation ?
kernel event tracing
Used by Microsoft Diskmon tool to monitor hard disk activity.
The problem here is that I don't know where to start and what is the best option to start with in terms of performance and security.
you at first must perfect understand windows Storage, Volumes, and File System Stacks and use Device Tree
determinate which device you need filter, and as UpperFilter or Lowfilter. you really need replicate disk, or say partition(volume) on disk or file-systems read/write ?
for attach to device you can register self in registry, say for disk filter - under HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Class\{4d36e967-e325-11ce-bfc1-08002be10318} in UpperFilters or LowerFilters
read this storage stack example
your place is at (4) (LowFilter - you will be filter IRP_MJ_SCSI for read/write) or (7) (UpperFilter - you will be filter IRP_MJ_READ/IRP_MJ_WRITE for read/write)
driver must be or direct WDM (register AddDevice in Driver) or use KMDF (shell framework)
another solution use IoRegisterPlugPlayNotification(EventCategoryDeviceInterfaceChange, &GUID_DEVINTERFACE_DISK) for attach disk FDO - here you always will be UpperFilter
and the best resource for ask this kind of question, advice - is osronline - NTFSD or NTDEV forum
Related
I have a Windows 7 embedded device which is frequently power cycled like this: a local application writes an entry to the Application event log and a few seconds later it commands the custom power supply to cycle power. A clean Windows shutdown cannot be done. After the device boots back up, I check the Windows event log and notice the last entry missing.
Is there some way to flush the Windows event log to disk so that I don't miss that last entry? The application is written in C++.
Thanks,
Adnan
You have a hardware problem so the best solution is to resolve it in hardware. But moving on.
Use BIOS to ensure computer always starts after a power cycle
You have a hardware tool to perform the reboot. Could you reconfigure that so that rather than doing a hard power cycle while the computer is running, you:
notify the hardware power cycle tool,
perform an orderly shutdown, then
power the computer on again using the hardware power switch?
There is usually a bios setting which allows you to specify the action after a power cycle, e.g.
Leave the computer off,
Turn it on,
Turn it on only if it was on when power was lost.
If you can do this, then you can still do an orderly shutdown and wait for that to complete before powering on again.
Software Solution: Turn off disk write caching
It's not exactly clear what is causing your issue, but the following seem relevant.
Most likely, this is a result of disk write caching either by the disc controller hardware, or by the operating system.
You may be able to turn this off through the Disk Management tool:
http://www.thewindowsclub.com/enable-disable-disk-write-caching-windows-7-8
Software Solution: Flush the disk programmatically
Alternatively, you may be able to flush the disk cache programmatically. This tool may help:
https://superuser.com/questions/833552/manually-flushing-write-cache-on-window
Or indeed with WMI and PowerShell:
https://technet.microsoft.com/en-us/library/dn454975.aspx
When I used C# i was only able to access user-mode registry accesses.
Is it very difficult to access kernel-mode registry accesses using C++?
I recall reading somewhere I may have to create a dummy windows driver or something?
EDIT: Basically as a hobby project I wish to create a simple registry monitor. However, I do want to catch kernel mode (as well as user mode) registry accesses..... last time I did this, using C# I could not access the kernel mode activity.
There are two ways to achieve this:
Hook the relevant functions in the kernel - the traditional way - which requires a C/Kernel Driver. This is possible on x86 Windows, but on x64 Kernel Patch Protection will detect these modifications and shut down the system (with a bluescreen).
Build a registry filter driver - this is the now encouraged way to attack this problem and is the way process monitor works. You can also build file system filter drivers this way. Essentially, you simply need to pass the information back to userland which boils down to:
IoRegisterDevice(...somewhere in \Devices\YourDriverName...)
IoCreateSymbolicLink(\\DosDevices\Name -> \Devices\YourDriverName)
then a C, C++, C# application should be able to open the file \\.\YourDriverName and DeviceIoControl to it and receive responses.
It is possible to use C++ to write kernel drivers, but see this before you embark on doing so. To be clearer, you need to be really careful about memory in kernel mode (paged, nonpaged) and you're not going to have access to much of the standard library.
As an aside, you should be aware that:
Not all registry hives are accessible to kernel mode drivers, depending on context.
The paths are not common. So the kernel accesses \Registry\System whereas userland accesses HKLM.
i need to add some files to results returned by FindFirstFile and FindNextFile under windows. Is this possible by file system filter driver or what type of drivers?
Thank you
You can do this by File System Filter Driver. But you can do this by implementing a system wide API hook. I have not tried it before but you really don't need to take the pains of writing the drivers and making the system unstable in case of spoiling the driver stack.
System Wide API Hooking
API Hooking Revealed
As pointed out you can use a file system filter driver (legacy or mini-filter, based on fltmgr). However, I would strongly recommend against the system-wide API hooking. Simple reason: if you do it in usermode it's not really going to be system-wide and if you use an SSDT-hook or some hotpatching method you risk the system's stability. An alternative, albeit equally shady as system-wide hooking, would be entry-point stealing. In this case you use the device object of the volume (in which you're interested, just listen for the attach notifications or enumerate them at startup) to find the driver responsible for it and modify the major function entry points in the driver object (Ilho pointed you into the right direction already).
A file system filter driver is the supported method to do just that.
In the latest Windows 7 WDK the sample under 7600.16385.1\src\filesys\miniFilter\minispy provides a good starting point. Biggest problem with mini filters for a private person is to get assigned an altitude for the driver to load at. Because using just any altitude can well lead to BSODs - and in case of FSFDs you might even risk your data integrity (although the kernel steps in with the BSOD to prevent that). You only need to fake IRP_MN_QUERY_DIRECTORY - this is the minor control code you're looking for when you are handling the IRP_MJ_DIRECTORY_CONTROL major control code. All others you can pass through as long as you don't need to allow the file to be opened, read or written and such. How to do that can be seen in the 7600.16385.1\src\filesys\miniFilter\passThrough sample source.
I am trying to fix an Audacity bug that revolves around portmixer. The output/input level is settable using the mac version of portmixer, but not always in windows. I am debugging portmixer's window code to try to make it work there.
Using IAudioEndpointVolume::SetMasterVolumeLevelScalar to set the master volume works fine for onboard sound, but using pro external USB or firewire interfaces like the RME Fireface 400, the output volume won't change, although it is reflected in Window's sound control panel for that device, and also in the system mixer.
Also, outside of our program, changing the master slider for the system mixer (in the taskbar) there is no effect - the soundcard outputs the same (full) level regardless of the level the system says it is at. The only way to change the output level is using the custom app that the hardware developers give with the card.
The IAudioEndpointVolume::QueryHardwareSupport function gives back ENDPOINT_HARDWARE_SUPPORT_VOLUME so it should be able to do this.
This behavior exists for both input and output on many devices.
Is this possibly a Window's bug?
It is possible to workaround this by emulating (scaling) the output, but this is not preferred as it is not functionally identical - better to let the audio interface do the scaling (esp. for input if it involves a preamp).
The cards you talk about -like the RME- ones simply do not support setting the master or any other level through software, and there is not much you can do about it. This is not a Windows bug. One could argue that giving back ENDPOINT_HARDWARE_SUPPORT_VOLUME is a bug though, but that likely originates from the driver level, not Windows itself.
The only solution I found so far is hooking up a debugger (or adding a dll hook) to the vendor supplied software and looking at the DeviceIOControl calls it makes (those are the ones used to talk to the hardware) while setting the volume in the vendor software. Pretty hard to do this for every single card, but probably worth doing for a couple of pro cards. Especially for Audacity, for open source audio software it's actually not that bad so I can imagine some people being really happy if the volume on their card could be set by it. (at the time we were exclusively using an RME Multiface I spent quite some time in figuring out the DeviceIOControl calls, but in the end it was definitely worth it as I could set the volume in dB for any point in the matrix)
I don't have the money to build my own uber Blade system but I would like to get into concurrent and distributed programming (think CCR/DSS, Hadoop, Project Voldemort etc.).
I currently have a Q6600 with 4GB with some separate hdds but that's about it. While I can write multi-threaded programs I can not properly test distributed filesystems / key-value stores and look for associated bottlenecks (disk access, network, etc.).
Does anyone have some recommendations? Buying some small cheap boxes and setting up a mini network? Or maybe a single box with two i7's and ESX and a simulated network?
edit:
I'm currently using VirtualBox and VmWare and this does not look good enough for me, correct me if I'm wrong: The hard drives could lock for instance, either because two virtualized machines run on them, or because all hard drive access is channeled through the same hdd controller. The network is entirely virtual, so no real case test here either.
If I go the virtualization route, what would you recommend so I can get as near to 'real-life' as possible?
Virtualise for your distributed system tests. It's much easier to 'pull the plug' on a machine, disconnect the network cable etc.
Sun VirtualBox is an excellent free virtual machine which I've found extremely convenient for development purposes. Most of it is also Open Source, if you're into that.
As for the multi-threaded part, it's actually easier - always test with more software threads than your number of hardware threads. And then, just for fun, do something like writing a 10 GB file to your hard disk, plug/unplug hardware to throw the scheduler off. You'll get surprising results.