I was wondering how hard disk access works. Ex, how could I view/modify sectors? Im targeting Windows if that helps.
Thanks
This page seems to have some relevant information on the subject:
You can open a physical or logical
drive using the CreateFile()
application programming interface
(API) with these device names provided
that you have the appropriate access
rights to the drive (that is, you must
be an administrator). You must use
both the CreateFile() FILE_SHARE_READ
and FILE_SHARE_WRITE flags to gain
access to the drive.
Once the logical or physical drive has
been opened, you can then perform
direct I/O to the data on the entire
drive. When performing direct disk
I/O, you must seek, read, and write in
multiples of sector sizes of the
device and on sector boundaries. Call
DeviceIoControl() using
IOCTL_DISK_GET_DRIVE_GEOMETRY to get
the bytes per sector, number of
sectors, sectors per track, and so
forth, so that you can compute the
size of the buffer that you will need.
The documentation of CreateFile also offers some clues:
You can use the CreateFile function to open a physical disk drive or a volume,
which returns a direct access storage device (DASD) handle that can be
used with the DeviceIoControl function. This enables you to access the
disk or volume directly, for example such disk metadata as the partition
table. However, this type of access also exposes the disk drive or
volume to potential data loss, because an incorrect write to a disk
using this mechanism could make its contents inaccessible to the
operating system. To ensure data integrity, be sure to become
familiar with DeviceIoControl and how other APIs behave
differently with a direct access handle as opposed to a file system handle.
You can open a logical volume (e.g. c: drive) or a physical drive using win32's CreateFile() function. With the returned handle you can read and write sectors as needed.
This page at MSDN should get you started: CreateFile Function
I take no responsibility for damaged caused :-)
Related
I'm developing c++ WinAPI program to write data on my USB drive directly with CreateFile by the Volume{GUID} and WriteFile functions as dozens of examples do. As mentioned here in Remarks:
A write on a volume handle will succeed if the volume does not have a
mounted file system, or if one of the following conditions is true:
The sectors to be written to are boot sectors.
The sectors to be written to reside outside of file system space.
You have explicitly locked or dismounted the volume by using FSCTL_LOCK_VOLUME or FSCTL_DISMOUNT_VOLUME.
The volume has no actual file system. (In other words, it has a RAW file system mounted.)
I want write 100Mb of data on USB smoothly without any unmounting. So I've tried two of the cases from above.
The second case: writing outside of the file system (I've extended the number of sectors per partition without extending FAT32 table), but it doesn't work without unmounting!
The fourth case: write on the volume without any file system (unformatted). But it also doesn't work without unmounting!!!
Also I've tried to create the second partition (that is invisible for Windows) with/without file system and write directly there by the offset from the end of first partition, but also unsuccessfully: I can not read or write there.
So, if anybody knows ANSWER TO THE ONE OF THE QUESTIONS at least:
How can I create sectors inside the partition outside
the file system?
Is there an ability to write smoothly on USB directly?
Also can I write in Windows to the second partition on USB without
playing around with drivers, NTCreateFile and other deep functions?
I'm pretty sure it actually worked, but you don't realize that you're bypassing Windows. That means Windows will not have noticed what you did. So if you use Windows to check what you did, it will not report a change.
To address the individual sub-questions: A normal file system fills the entire partition, so you can't. Your functions write smoothly to USB, that's not the problem. And no, Windows normally treats USB as unpartitioned storage.
I have a large storage device (flash memory) plugged onto my computer via the PCIe bus, I want to access such device directly, i.e., without any file system (e.g., NTFS or ext4) on it.
How can I do this using C/C++? (on both Windows 7 and Linux)
I am wondering if I can 1) open the device just as a file, and then read and write binary data to it, or 2) allocate the whole device using some function like malloc, then each byte on the device have an address so that I can access them based on the addresses.
I prefer the second way if it possible, but I don't know if the OS supports this since it seems the address space needs to be shared with the main memory.
According to Microsoft documentation:
On Windows you can open a physical drive using CreateFile using a path of the form
\\.\PhysicalDriveN
where N is the device number or a logical drive using a path of the form
\\.\X:
You will need to seek, read and write in multiples of the sector size which can be retrieved using DeviceIoControl() with IOCTL_DISK_GET_DRIVE_GEOMETRY.
On Linux each storage device ends up getting a device entry in /dev. The first storage device is typically /dev/sda, the second storage device, if one is present, is /dev/sdb. Note that an optical disk is a storage device, so a CD-ROM or a DVD-ROM drive, if one is present, would get a device node entry.
Some Linux distributions may use a different naming convention, but this is what it usually is. So, you'll need to figure out which device corresponds to your flash disk, and just open the /dev/sdX device, and simply read and write from it. Your reads and writes must be for even block (sector) sizes, and seeking the opened file governs which disk blocks/sectors the subsequent read or write will affect.
Generally, /dev/sdX will be owned by root, but there are usually some Linux distribution-specific ways to fiddle the userid that owns a particular device node.
I would like to find out what physical media a file is on (either by filename or by handle).
I've read here that I can determine the physical drive of a given partition handle with a DeviceIoControl call.
But how do I get a partition handle from a filename? (Because NTFS allows to mount partitions in folders, I can't simply go by drive letter.)
The reason I'm asking is that copying a file to another physical drive is fastest when using asynchronous methods, but on the same drive it is fastest when done synchronously with a larger buffer. I want to write a function to determine the optimum copying algorithm for a given file and destination.
Volumes in Windows are generally mapped to drive letters (except on network shares).
You can create a volume file device handle on the source and destination using CreateFile() just passing in the drive letter path (ie: \\.\C:\). Use the returned handle with DeviceIoControl(). Compare the physical drives between the source and destination, and you should be good.
Discussion explained here: "To obtain the physical drive identifier for a volume, open ..."
http://msdn.microsoft.com/en-us/library/windows/desktop/aa363858%28v=vs.85%29.aspx
I searched a lot to get write - access to the boot sector (Master File Table ). I used CreateFile function with write access parameters like, GENERIC_WRITE. Then used SetFilePointer and WriteFile to write on a particular memory address. But what I always get is System.AccessViolationException (Windows 7). Am I doing something wrong here ?
I want to know if there is any alternative to CreateFile - WriteFile functions to get wrtie - access to boot sector ?
OR I was thinking if there is any way to use Interrup Service Routine to write on particular disk sectors in VC++ (C++/Cli) ?
According to Microsoft KB you have to call CreateFile with FILE_SHARE_READ and FILE_SHARE_WRITE on "\\.\PhysicalDriveN", where N is zero-based physical drive index. Then you can access the entire drive as one huge file. You have to be an administrator on your machine for this to work!
You can open a physical or logical drive using the CreateFile()
application programming interface (API) with these device names
provided that you have the appropriate access rights to the drive
(that is, you must be an administrator). You must use both the
CreateFile() FILE_SHARE_READ and FILE_SHARE_WRITE flags to gain access
to the drive.
Once the logical or physical drive has been opened, you can then
perform direct I/O to the data on the entire drive. When performing
direct disk I/O, you must seek, read, and write in multiples of sector
sizes of the device and on sector boundaries. Call DeviceIoControl()
using IOCTL_DISK_GET_DRIVE_GEOMETRY to get the bytes per sector,
number of sectors, sectors per track, and so forth, so that you can
compute the size of the buffer that you will need.
Update: I did some research and I found out that starting Vista you have to obtain the lock on the volume or dismount it. Otherwise the writes would fail. In the docs Microsoft says:
If you write directly to a volume that has a mounted file system, you
must first obtain exclusive access to the volume. Otherwise, you risk
causing data corruption or system instability, because your
application's writes may conflict with other changes coming from the
file system and leave the contents of the volume in an inconsistent
state. To prevent these problems, the following changes have been made
in Windows Vista and later:
A write on a volume handle will succeed if the volume does not have a mounted file system, or if one of the following conditions is true:
The sectors to be written to are boot sectors.
The sectors to be written to reside outside of file system space.
You have explicitly locked or dismounted the volume by using FSCTL_LOCK_VOLUME or FSCTL_DISMOUNT_VOLUME.
The volume has no actual file system. (In other words, it has a RAW file system mounted.)
A write on a disk handle will succeed if one of the following conditions is true:
The sectors to be written to do not fall within a volume's extents.
The sectors to be written to fall within a mounted volume, but you have explicitly locked or dismounted the volume by using FSCTL_LOCK_VOLUME or FSCTL_DISMOUNT_VOLUME.
The sectors to be written to fall within a volume that has no mounted file system other than RAW.
can I change the ECC code for a block of a file stored on a flash drive by any means ?
of a file stored on a HDD (though I don't think there would be a difference between the two)
Maybe , through some hardware interrupts or anything like that?
Also I need the solution to be in C/C++.
A NAND flash drive is composed of a number of data pages and a flash controller. The ECC code on the NAND flash is used by the flash controller. The controller uses it to determine if the associated data page has any errors. A filesystem (like fat32, NTFS, or ext3) is usually implemented on top of the raw data page structure. A file may be spread across one or more flash pages. Please note that there is an error code for each flash page, meaning that a file larger than one data page will have an error code for each page it uses. A hard disk drive is composed of one or more platters, heads which read and write data on the platters, and a disk controller. Each platter is divided into sectors. Hard disk drives also have error codes to check the integrity of sectors on the platters. Again, a filesystem, is typically implemented on top of the raw disk sectors. If a file is larger than a sector then there will be multiple sectors associated with the file, each sector having its own error code. To access a data page or sector error code you will need to interface with the flash or hard dive controller directly. This will require interfacing with the device driver for the device. You will need to read the documentation for the device driver in order to discover what functions it offers to allow you access to error codes. In either instance, unless a file occupies only one data page or sector it will have more than one error code associated with it. Some filesystems create error codes for files, regardless of there length, but accessing a filesystem level error code typically does not require hardware access.