the content of pointer changed when passed to the function - c++

I am using C++ to do my work. The following is the problem I am confulsed.
typedef struct netHandler {
int m_iFd;
poll_event_t* m_pPollEvent;
epoll_event m_Event;
void* m_pConn;
} netHandler_t;
Here is my struct. When the program running. I set a breakpoint in the following place:
connection_t* conn = geneConnection(idenfication, fd, handler);
And before step into this funcition, I record the handler's content.
p *(netHandler_t *) 0x2aaaac000960
$2 = {m_iFd = 9, m_pPollEvent = 0x2aaaac000930, m_Event =
{events = 1, data = {ptr = 0x2aaaac000960, fd = -1409283744, u32 = 2885683552, u64 = 46912518490464}},
m_pConn = 0x60d010}
Then I step into the geneConnection function. The content changed.
p *(netHandler_t *) 0x2aaaac000960
$3 = {m_iFd = 9, m_pPollEvent = 0xac00093000000000, m_Event =
{events = 10922, data = {ptr = 0xac00096000000001, fd = 1, u32 = 1, u64 = 12393916482445115393}},
m_pConn = 0x2aaa}
That's like the struct was shrifted.
connection_t* geneConnection(long iden, int sendfd, netHandler_t* handler);
This is the declare of geneConnection. I don't understand why this happened. When I change this function to inline type. The content of handler would not changed.

Related

vkQueueSubmit blocks when using timeline semaphores

I need to run a function on CPU between two GPU batches. For this I use timeline semaphores. As far as I know, vkQueueSubmit does not block. However, it blocks when I submit these GPU batches:
uint64_t host_wait = timeline;
uint64_t host_signal = ++timeline;
uint64_t wait0 = timeline;
uint64_t signal0 = ++timeline;
uint64_t wait1 = timeline;
uint64_t signal1 = ++timeline;
VkPipelineStageFlags wait_mask = VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT;
VkTimelineSemaphoreSubmitInfo sp_submit0 = {
.sType = VK_STRUCTURE_TYPE_TIMELINE_SEMAPHORE_SUBMIT_INFO,
.waitSemaphoreValueCount = 1,
.pWaitSemaphoreValues = &wait0,
.signalSemaphoreValueCount = 1,
.pSignalSemaphoreValues = &signal0,
};
VkSubmitInfo submit0 = {
.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO,
.pNext = &sp_submit0,
.waitSemaphoreCount = 1,
.pWaitSemaphores = &timeline_semaphore,
.pWaitDstStageMask = &wait_mask,
.signalSemaphoreCount = 1,
.pSignalSemaphores = &timeline_semaphore,
};
VkTimelineSemaphoreSubmitInfo sp_submit1 = {
.sType = VK_STRUCTURE_TYPE_TIMELINE_SEMAPHORE_SUBMIT_INFO,
.waitSemaphoreValueCount = 1,
.pWaitSemaphoreValues = &wait1,
.signalSemaphoreValueCount = 1,
.pSignalSemaphoreValues = &signal1,
};
VkSubmitInfo submit1 = {
.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO,
.pNext = &sp_submit1,
.waitSemaphoreCount = 1,
.pWaitSemaphores = &timeline_semaphore,
.pWaitDstStageMask = &wait_mask,
.commandBufferCount = 1,
.pCommandBuffers = &command_buffer,
.signalSemaphoreCount = 1,
.pSignalSemaphores = &timeline_semaphore,
};
VkSubmitInfo infos[2] = { submit0, submit1 };
vkQueueSubmit(queue, 2, infos, fence);
// here vkQueueSubmit blocks the thread
WaitSemaphore(timeline_semaphore, host_wait);
some_function();
SignalSemaphore(timeline_semaphore, host_signal);
It is blocking for seconds without return, I think this is something like a deadlock. In the debugger, I saw SleepEx function call from vkQueueSubmit: vk_icdGetInstanceProcAddrSG -> ... -> SleepEx.
But vkQueueSubmit does not block in this sample (combined batch):
uint64_t host_wait = timeline;
uint64_t host_signal = ++timeline;
uint64_t wait1 = timeline;
uint64_t signal1 = ++timeline;
VkPipelineStageFlags wait_mask = VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT;
VkTimelineSemaphoreSubmitInfo sp_submit1 = {
.sType = VK_STRUCTURE_TYPE_TIMELINE_SEMAPHORE_SUBMIT_INFO,
.waitSemaphoreValueCount = 1,
.pWaitSemaphoreValues = &wait1,
.signalSemaphoreValueCount = 1,
.pSignalSemaphoreValues = &signal1,
};
VkSubmitInfo submit1 = {
.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO,
.pNext = &sp_submit1,
.waitSemaphoreCount = 1,
.pWaitSemaphores = &timeline_semaphore,
.pWaitDstStageMask = &wait_mask,
.commandBufferCount = 1,
.pCommandBuffers = &command_buffer,
.signalSemaphoreCount = 1,
.pSignalSemaphores = &timeline_semaphore,
};
VkSubmitInfo infos[1] = { submit1 };
vkQueueSubmit(queue, 1, infos, fence);
WaitSemaphore(timeline_semaphore, host_wait);
some_function();
SignalSemaphore(timeline_semaphore, host_signal);
Why vkQueueSubmit blocks in the first code sample? What are the possible causes of this problem?
I use Vulkan 1.2 (SDK 1.2.135) on Windows 10 and Radeon RX 570 (driver 20.4.2).
EDIT: When I add a command buffer to submit0, vkQueueSubmit will not block. Is it a bug in the driver?
Doing something odd like submitting a batch with no work is far more likely to be the cause.
The spec doesn’t have performance requirements. The fact that something is legal does not make it a good idea. Broadly speaking, if there are two ways to do a thing, do it the obvious way. And sending an empty batch isn’t exactly obvious
https://community.khronos.org/t/vkqueuesubmit-blocks-when-using-timeline-semaphores/105704/2

Vulkan timeline semaphore extension cannot be enabled

I've been at this for a better part of today and I'm at the end of my wits.
I'm running Vulkan SDK 1.2.131.2
I have a RTX 2080 Ti.
I have Windows 10 Education, version 1909, build 18363.657.
I'm using Vulkan.hpp instead of Vulkan.h directly.
Here is where I specify the API version I use:
appInfo.apiVersion = VK_API_VERSION_1_2;
This is the relevant part of the code that creates the device:
// bla bla
const std::vector<const char*> deviceExtensions = {
VK_KHR_SWAPCHAIN_EXTENSION_NAME,
VK_KHR_TIMELINE_SEMAPHORE_EXTENSION_NAME,
VK_NV_RAY_TRACING_EXTENSION_NAME
};
deviceCreateInfo.enabledExtensionCount = static_cast<uint32_t>(deviceExtensions.size());
deviceCreateInfo.ppEnabledExtensionNames = deviceExtensions.data();
m_logicalDevice = m_physicalDevice.createDeviceUnique(deviceCreateInfo);
I use the following validation layers:
"VK_LAYER_LUNARG_api_dump"
"VK_LAYER_KHRONOS_validation"
This is how I later try to create a timeline semaphore:
vk::UniqueSemaphore VulkanContext::createTimelineSemaphore(const uint32_t initialValue) const {
vk::SemaphoreTypeCreateInfo timelineCreateInfo;
timelineCreateInfo.semaphoreType = vk::SemaphoreType::eTimeline;
timelineCreateInfo.initialValue = initialValue;
vk::SemaphoreCreateInfo createInfo;
createInfo.pNext = &timelineCreateInfo;
return m_logicalDevice->createSemaphoreUnique(createInfo);
}
I get the following error:
vkCreateSemaphore(device, pCreateInfo, pAllocator, pSemaphore) returns VkResultVkCreateSemaphore: timelineSemaphore feature is not enabled, can not create timeline semaphores The Vulkan spec states: If the timelineSemaphore feature is not enabled, semaphoreType must not equal VK_SEMAPHORE_TYPE_TIMELINE (https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/vkspec.html#VUID-VkSemaphoreTypeCreateInfo-timelineSemaphore-03252)
This is even more infuriating because timeline semaphores are supposed to be part of the core Vulkan 1.2, but I get the same error even if I ommit it from the extension list. Swapchain extension does work and I haven't had the time to verify that ray tracing extension is enabled.
It gets even more stupid because the next message tells me this:
VK_SUCCESS (0):
device: VkDevice = 0000023AA29BD8B0
pCreateInfo: const VkSemaphoreCreateInfo* = 0000008D145ED538:
sType: VkStructureType = VK_STRUCTURE_TYPE_SEMAPHORE_CREATE_INFO (9)
pNext: VkSemaphoreTypeCreateInfo = 0000008D145ED4F8:
sType: VkStructureType = VK_STRUCTURE_TYPE_SEMAPHORE_TYPE_CREATE_INFO (1000207002)
pNext: const void* = NULL
semaphoreType: VkSemaphoreType = VK_SEMAPHORE_TYPE_TIMELINE (1)
initialValue: uint64_t = 0
flags: VkSemaphoreCreateFlags = 0
pAllocator: const VkAllocationCallbacks* = NULL
pSemaphore: VkSemaphore* = AA989B000000001E
I have no idea if this creates the timeline semaphore or just creates a normal binary one.
When I later use it to submit to a transfer queue:
vk::CommandBufferBeginInfo beginInfo;
transferCmdBuffer->begin(beginInfo);
object->recordUploadToGPU(*transferCmdBuffer);
transferCmdBuffer->end();
vk::TimelineSemaphoreSubmitInfo timelineSubmitInfo;
timelineSubmitInfo.signalSemaphoreValueCount = 1;
timelineSubmitInfo.pSignalSemaphoreValues = &signalValue;
vk::SubmitInfo submitInfo;
submitInfo.pNext = &timelineSubmitInfo;
submitInfo.signalSemaphoreCount = 1;
submitInfo.pSignalSemaphores = &signalSemaphore;
submitInfo.commandBufferCount = 1;
submitInfo.pCommandBuffers = &*transferCmdBuffer;
vkCtx.m_transferQueue.submit(submitInfo, nullptr);
I get this error here:
vkQueueSubmit(queue, submitCount, pSubmits, fence) returns VkResultVkQueueSubmit: VkQueue 0x23aa2539500[] contains timeline sempahore VkSemaphore 0xaa989b000000001e[] that sets its wait value with a margin greater than maxTimelineSemaphoreValueDifference The Vulkan spec states: For each element of pSignalSemaphores created with a VkSemaphoreType of VK_SEMAPHORE_TYPE_TIMELINE the corresponding element of VkTimelineSemaphoreSubmitInfo::pSignalSemaphoreValues must have a value which does not differ from the current value of the semaphore or the value of any outstanding semaphore wait or signal operation on that semaphore by more than maxTimelineSemaphoreValueDifference. (https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/vkspec.html#VUID-VkSubmitInfo-pSignalSemaphores-03244)
And just to be futher mocked, this is the next line:
VK_SUCCESS (0):
queue: VkQueue = 0000023AA2539500
submitCount: uint32_t = 1
pSubmits: const VkSubmitInfo* = 0000008D145ED370
pSubmits[0]: const VkSubmitInfo = 0000008D145ED370:
sType: VkStructureType = VK_STRUCTURE_TYPE_SUBMIT_INFO (4)
pNext: VkTimelineSemaphoreSubmitInfo = 0000008D145ED318:
sType: VkStructureType = VK_STRUCTURE_TYPE_TIMELINE_SEMAPHORE_SUBMIT_INFO (1000207003)
pNext: const void* = NULL
waitSemaphoreValueCount: uint32_t = 0
pWaitSemaphoreValues: const uint64_t* = NULL
signalSemaphoreValueCount: uint32_t = 1
pSignalSemaphoreValues: const uint64_t* = 0000008D145ED740
pSignalSemaphoreValues[0]: const uint64_t = 1
waitSemaphoreCount: uint32_t = 0
pWaitSemaphores: const VkSemaphore* = NULL
pWaitDstStageMask: const VkPipelineStageFlags* = NULL
commandBufferCount: uint32_t = 1
pCommandBuffers: const VkCommandBuffer* = 0000008D145EF408
pCommandBuffers[0]: const VkCommandBuffer = 0000023AA9CEC8E0
signalSemaphoreCount: uint32_t = 1
pSignalSemaphores: const VkSemaphore* = 0000008D145EF430
pSignalSemaphores[0]: const VkSemaphore = AA989B000000001E
fence: VkFence = 0000000000000000
I've also tried with VK_API_VERSION_1_1 and VK_API_VERSION_1_0, both with and without enabling the extension explicitly, none of them work.
The dumps are from the VK_LAYER_LUNARG_api_dump validation layer, while VK_LAYER_KHRONOS_validation validation layer is the one spewing out the errors. They seem to disagree.
So, what gives?
In what way am I stupid today?
EDIT:
Here is an example that you should be able to run yourself. I think I made it as minimal as I can:
#include <vulkan/vulkan.hpp>
#include <iostream>
VKAPI_ATTR VkBool32 VKAPI_CALL debugCallback(
VkDebugUtilsMessageSeverityFlagBitsEXT messageSeverity,
VkDebugUtilsMessageTypeFlagsEXT messageType,
const VkDebugUtilsMessengerCallbackDataEXT* pCallbackData,
void* pUserData) {
std::cerr << pCallbackData->pMessage << std::endl;
return VK_FALSE;
};
int main() {
vk::ApplicationInfo appInfo;
appInfo.apiVersion = VK_API_VERSION_1_2;
vk::InstanceCreateInfo instanceCreateInfo;
instanceCreateInfo.pApplicationInfo = &appInfo;
std::vector<const char*> extensions;
extensions.push_back(VK_EXT_DEBUG_UTILS_EXTENSION_NAME);
instanceCreateInfo.enabledExtensionCount = static_cast<uint32_t>(extensions.size());
instanceCreateInfo.ppEnabledExtensionNames = extensions.data();
const std::vector<const char*> validationLayers = {
"VK_LAYER_LUNARG_api_dump",
"VK_LAYER_KHRONOS_validation"
};
instanceCreateInfo.enabledLayerCount = static_cast<uint32_t>(validationLayers.size());
instanceCreateInfo.ppEnabledLayerNames = validationLayers.data();
vk::DebugUtilsMessengerCreateInfoEXT debugCreateInfo;
debugCreateInfo.messageSeverity =
vk::DebugUtilsMessageSeverityFlagBitsEXT::eInfo |
vk::DebugUtilsMessageSeverityFlagBitsEXT::eVerbose |
vk::DebugUtilsMessageSeverityFlagBitsEXT::eWarning |
vk::DebugUtilsMessageSeverityFlagBitsEXT::eError;
debugCreateInfo.messageType =
vk::DebugUtilsMessageTypeFlagBitsEXT::eGeneral |
vk::DebugUtilsMessageTypeFlagBitsEXT::eValidation |
vk::DebugUtilsMessageTypeFlagBitsEXT::ePerformance;
debugCreateInfo.pfnUserCallback = debugCallback;
instanceCreateInfo.pNext = &debugCreateInfo;
vk::Instance m_instance = vk::createInstance(instanceCreateInfo);
vk::DispatchLoaderDynamic m_loader = vk::DispatchLoaderDynamic(m_instance, vkGetInstanceProcAddr);
vk::DebugUtilsMessengerEXT m_debugMessenger = m_instance.createDebugUtilsMessengerEXT(debugCreateInfo, nullptr, m_loader);
vk::PhysicalDevice m_physicalDevice = m_instance.enumeratePhysicalDevices()[0];
std::vector<vk::DeviceQueueCreateInfo> queueCreateInfos;
vk::DeviceQueueCreateInfo queueInfo;
queueInfo.queueFamilyIndex = 0;
queueInfo.queueCount = 1;
queueCreateInfos.push_back(queueInfo);
vk::PhysicalDeviceFeatures deviceFeatures;
vk::DeviceCreateInfo deviceCreateInfo;
deviceCreateInfo.pQueueCreateInfos = queueCreateInfos.data();
deviceCreateInfo.queueCreateInfoCount = static_cast<uint32_t>(queueCreateInfos.size());
deviceCreateInfo.pEnabledFeatures = &deviceFeatures;
// This part can be omitted from here...
const std::vector<const char*> deviceExtensions = {
VK_KHR_TIMELINE_SEMAPHORE_EXTENSION_NAME
};
deviceCreateInfo.enabledExtensionCount = static_cast<uint32_t>(deviceExtensions.size());
deviceCreateInfo.ppEnabledExtensionNames = deviceExtensions.data();
// ...to here. It doesn't work either way.
vk::Device m_logicalDevice = m_physicalDevice.createDevice(deviceCreateInfo);
vk::SemaphoreTypeCreateInfo timelineCreateInfo;
timelineCreateInfo.semaphoreType = vk::SemaphoreType::eTimeline;
timelineCreateInfo.initialValue = 0;
vk::SemaphoreCreateInfo semaphoreCreateInfo;
semaphoreCreateInfo.pNext = &timelineCreateInfo;
m_logicalDevice.createSemaphore(semaphoreCreateInfo);
}
The feature needs to be enabled explicitly as well:
vk::PhysicalDeviceVulkan12Features features;
features.timelineSemaphore = true;
vk::DeviceCreateInfo deviceCreateInfo;
deviceCreateInfo.pNext = &features;

C++ WINAPI Create Multiple Partitions on Mounted VHD

I managed to create a VHD and attached it. Afterwards, I created a disk(IOCTL CREATE_DISK) and set its layout using IOCTL_DISK_SET_DRIVE_LAYOUT_EX. Now, when I examine disk through Disk Management. I have a 14MB with a 7 MB partition, expectedly.
int sign = 80001;
CREATE_DISK disk;
disk.Mbr.Signature = sign;
disk.PartitionStyle = PARTITION_STYLE_MBR;
auto res = DeviceIoControl(device_handle, IOCTL_DISK_CREATE_DISK, &disk, sizeof(disk), NULL, 0, NULL, NULL);
res = DeviceIoControl(device_handle, IOCTL_DISK_UPDATE_PROPERTIES, 0, 0, NULL, 0, NULL, NULL);
LARGE_INTEGER partition_size;
partition_size.QuadPart = 0xF00;
DWORD driver_layout_ex_len = sizeof(DRIVE_LAYOUT_INFORMATION_EX);
DRIVE_LAYOUT_INFORMATION_EX driver_layout_info;
memset(&driver_layout_info, 0, sizeof(DRIVE_LAYOUT_INFORMATION_EX));
driver_layout_info.Mbr.Signature = sign;
driver_layout_info.PartitionCount = 1;
driver_layout_info.PartitionStyle = PARTITION_STYLE_MBR;
PARTITION_INFORMATION_EX part_info;
PARTITION_INFORMATION_MBR mbr_info;
part_info.StartingOffset.QuadPart = 32256;
part_info.RewritePartition = TRUE;
part_info.PartitionLength.QuadPart = partition_size.QuadPart/2 * 4096;
part_info.PartitionNumber = 1;
part_info.PartitionStyle = PARTITION_STYLE_MBR;
mbr_info.BootIndicator = TRUE;
mbr_info.HiddenSectors = 32256 / 512;
mbr_info.PartitionType = PARTITION_FAT32;
mbr_info.RecognizedPartition = 1;
part_info.Mbr = mbr_info;
driver_layout_info.PartitionEntry[0] = part_info;
auto res_layout = DeviceIoControl(device_handle, IOCTL_DISK_SET_DRIVE_LAYOUT_EX, &driver_layout_info, sizeof(driver_layout_info), NULL, 0, NULL, NULL);
Now, how do I partitionize this disk into two partitions? I want to create another partition out of the unpartitioned part of the disk(the other half basically). It says in the documentation is that PartitionEntry is an array of variable size(No, it is not it is an array of size 1.) Do I call set layout IOCTL for every partition I want to create? If so, how do you go about that? Is multi-partitioning possible through WINAPI interface?
P.S: I am aware that people usually invoke diskpart for this line of work.
Edit:
Adding second partition two layout was messing my stack up so I took another route (heap).
DWORD driver_layout_ex_len = sizeof(DRIVE_LAYOUT_INFORMATION_EX) + sizeof(PARTITION_INFORMATION_EX); // one layout+partition + partition
PDRIVE_LAYOUT_INFORMATION_EX driver_layout_info = (PDRIVE_LAYOUT_INFORMATION_EX) std::calloc(1, driver_layout_ex_len);
driver_layout_info->Mbr.Signature = sign;
driver_layout_info->PartitionCount = 2;
driver_layout_info->PartitionStyle = PARTITION_STYLE_MBR;
// omitted here..
PARTITION_INFORMATION_EX part_info2;
part_info2.StartingOffset.QuadPart = 32256 + part_info.PartitionLength.QuadPart;
part_info2.RewritePartition = TRUE;
part_info2.PartitionLength.QuadPart = partition_size.QuadPart / 2 * 4096;
part_info2.PartitionNumber = 2;
part_info2.PartitionStyle = PARTITION_STYLE_MBR;
part_info2.Mbr = mbr_info;
driver_layout_info->PartitionEntry[0] = part_info;
driver_layout_info->PartitionEntry[1] = part_info2;
auto res_layout = DeviceIoControl(device_handle, IOCTL_DISK_SET_DRIVE_LAYOUT_EX, driver_layout_info, driver_layout_ex_len, NULL, 0, NULL, NULL);
auto res_err = GetLastError();
Since it was overriding my device_handle I could not IOCTL at all. This improvement eliminated that. Do not forget to pass driver_layout_info instead of &driver_layout_info after this change.
It says in the documentation is that PartitionEntry is an array of
variable size(No, it is not it is an array of size 1.)
"Some Windows structures are variable-sized,
beginning with a fixed header, followed by
a variable-sized array. When these structures
are declared,
they often declare an array of size 1 where the
variable-sized array should be." Refer to #Raymond blog.
Here DRIVE_LAYOUT_INFORMATION_EX structure is an example:
typedef struct _DRIVE_LAYOUT_INFORMATION_EX {
DWORD PartitionStyle;
DWORD PartitionCount;
union {
DRIVE_LAYOUT_INFORMATION_MBR Mbr;
DRIVE_LAYOUT_INFORMATION_GPT Gpt;
} DUMMYUNIONNAME;
PARTITION_INFORMATION_EX PartitionEntry[1];
} DRIVE_LAYOUT_INFORMATION_EX, *PDRIVE_LAYOUT_INFORMATION_EX;
With this declaration, you would allocate memory for one such
variable-sized DRIVE_LAYOUT_INFORMATION_EX structure like this:
PDRIVE_LAYOUT_INFORMATION_EX driver_layout_info = (PDRIVE_LAYOUT_INFORMATION_EX)malloc(FIELD_OFFSET(DRIVE_LAYOUT_INFORMATION_EX, PartitionEntry[NumberOfPartitions]));
You would initialize the structure like this (Use 2 partitions as example):
DWORD NumberOfPartitions = 2;
LARGE_INTEGER partition_size;
partition_size.QuadPart = 0xF00;
PARTITION_INFORMATION_MBR mbr_info;
mbr_info.BootIndicator = TRUE;
mbr_info.HiddenSectors = 32256 / 512;
mbr_info.PartitionType = PARTITION_FAT32;
mbr_info.RecognizedPartition = TRUE;
PDRIVE_LAYOUT_INFORMATION_EX driver_layout_info = (PDRIVE_LAYOUT_INFORMATION_EX)malloc(FIELD_OFFSET(DRIVE_LAYOUT_INFORMATION_EX, PartitionEntry[NumberOfPartitions]));
for (DWORD Index = 0; Index < NumberOfPartitions ; Index++) {
driver_layout_info->PartitionEntry[Index].PartitionStyle = PARTITION_STYLE_MBR;
driver_layout_info->PartitionEntry[Index].PartitionNumber = Index + 1;
driver_layout_info->PartitionEntry[Index].RewritePartition = TRUE;
driver_layout_info->PartitionEntry[Index].PartitionLength.QuadPart = partition_size.QuadPart / 2 * 4096;
driver_layout_info->PartitionEntry[Index].Mbr = mbr_info;
}
driver_layout_info->Mbr.Signature = sign;
driver_layout_info->PartitionCount = 1;
driver_layout_info->PartitionStyle = PARTITION_STYLE_MBR;
driver_layout_info->PartitionEntry[0].StartingOffset.QuadPart = 32256;
driver_layout_info->PartitionEntry[1].StartingOffset.QuadPart = 32256 + driver_layout_info->PartitionEntry->StartingOffset.QuadPart;
DWORD driver_layout_ex_len = sizeof(DRIVE_LAYOUT_INFORMATION_EX) + sizeof(PARTITION_INFORMATION_EX);
Call free(driver_layout_info); after you use completely.

How much of a file has been read

I have a program which reads a 10 MByte file and processes the data as the data is being read in 4K chunks. The test usually takes 1 min - 2 min. But there are some instances when the program takes more than 10 min, at which point the test it killed and a core is generated. Following is the code that reads the file:
string filename("data.out");
ifstream ifs;
vector<char> buf(4096);
ifs.open(filename, ios::in | ios::binary);
if (!ifs.is_open()) {
cout << "ERROR : " << filename << "can't be opened." << endl;
VERIFY(ifs.is_open());
}
while (!ifs.eof()) {
ifs.read(buf.data(), buf.size()); <======== Line 1
process_data (buf.data(), ifs.gcount()); <======== Line 2
}
ifs.close();
I have two cores that show the program is stuck at Line 1 and Line 2.
Top of bt of core1 at Line 1:
#0 0x00007f942a462175 in std::istream::read (this=0x7fff4ce69de0,
__s=0x9120000 "\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324\324"..., __n=4096) at /home/packages/gcc/4.7/w/gcc-4.7-4.7.2/build/x86_64-linux-gnu/libstdc++-v3/include/bits/istream.tcc:651
Top of bt of core2 at Line 2:
#0 0x00000000004375f3 in std::__addressof<char> (__r=#0x7fa3176391a6: -128 '\200') at /usr/include/c++/4.7/bits/move.h:47
#1 0x0000000000436cd4 in std::vector<char, std::allocator<char> >::data (this=0x7fff346ad770)
at /usr/include/c++/4.7/bits/stl_vector.h:859
Initially, from core1, I thought the issue was with ifs.read() taking a long time. But after the second core, I am thinking the issue might be related to vector::data().
Is there a way I can tell if any part of the file has been read, by inspecting certain fields (e.g file offset) stored in ifstream.
I don't like posting dump of large structure, but here it is if someone can shed some light how I can figure out from this dump how much of the 10MB has been read.
(gdb) p ifs
$3 = warning: can't find linker symbol for virtual table for `std::basic_ifstream<char, std::char_traits<char> >' value
{
<std::basic_istream<char, std::char_traits<char> >> = {
<std::basic_ios<char, std::char_traits<char> >> = {
<std::ios_base> = {
_vptr.ios_base = 0xfbfcc0,
static boolalpha = std::_S_boolalpha,
static dec = std::_S_dec,
static fixed = std::_S_fixed,
static hex = std::_S_hex,
static internal = std::_S_internal,
static left = std::_S_left,
static oct = std::_S_oct,
static right = std::_S_right,
static scientific = std::_S_scientific,
static showbase = std::_S_showbase,
static showpoint = std::_S_showpoint,
static showpos = std::_S_showpos,
static skipws = std::_S_skipws,
static unitbuf = std::_S_unitbuf,
static uppercase = std::_S_uppercase,
static adjustfield = std::_S_adjustfield,
static basefield = std::_S_basefield,
static floatfield = std::_S_floatfield,
static badbit = std::_S_badbit,
static eofbit = std::_S_eofbit,
static failbit = std::_S_failbit,
static goodbit = std::_S_goodbit,
static app = std::_S_app,
static ate = std::_S_ate,
static binary = std::_S_bin,
static in = std::_S_in,
static out = std::_S_out,
static trunc = std::_S_trunc,
static beg = std::_S_beg,
static cur = std::_S_cur,
static end = std::_S_end,
_M_precision = 6,
_M_width = 0,
_M_flags = 4098,
_M_exception = std::_S_goodbit,
_M_streambuf_state = 5,
_M_callbacks = 0x0,
_M_word_zero = {
_M_pword = 0x0,
_M_iword = 0
},
_M_local_word = {{
_M_pword = 0x0,
_M_iword = 0
}, {
_M_pword = 0x0,
_M_iword = 0
}, {
_M_pword = 0x0,
_M_iword = 0
}, {
_M_pword = 0x0,
_M_iword = 0
}, {
_M_pword = 0x0,
_M_iword = 0
}, {
_M_pword = 0x0,
_M_iword = 0
}, {
_M_pword = 0x0,
_M_iword = 0
}, {
_M_pword = 0x0,
_M_iword = 0
}},
_M_word_size = 8,
_M_word = 0x7fff4ce69f20,
_M_ios_locale = {
static none = 0,
static ctype = 1,
static numeric = 2,
static collate = 4,
static time = 8,
static monetary = 16,
static messages = 32,
static all = 63,
_M_impl = 0x7f942a6e3aa0,
static _S_classic = 0x7f942a6e3aa0,
static _S_global = 0x7f942a6e3aa0,
static _S_categories = 0x7f942a6c86a0,
static _S_once = 2
}
},
members of std::basic_ios<char, std::char_traits<char> >:
_M_tie = 0x0,
_M_fill = 0 '\000',
_M_fill_init = false,
_M_streambuf = 0x7fff4ce69df0,
_M_ctype = 0x7f942a6e3d20,
_M_num_put = 0x7f942a6e4040,
_M_num_get = 0x7f942a6e4030
},
members of std::basic_istream<char, std::char_traits<char> >:
_vptr.basic_istream = 0xfbfc98,
_M_gcount = 0
},
members of std::basic_ifstream<char, std::char_traits<char> >:
_M_filebuf = warning: can't find linker symbol for virtual table for `std::basic_filebuf<char, std::char_traits<char> >' value
{
<std::basic_streambuf<char, std::char_traits<char> >> = {
_vptr.basic_streambuf = 0xfc0a70,
_M_in_beg = 0x6306000 "\317\317\317\......320\320\320\320"...,
_M_in_cur = 0x6307fff "",
_M_in_end = 0x6307fff "",
_M_out_beg = 0x0,
_M_out_cur = 0x0,
_M_out_end = 0x0,
_M_buf_locale = {
static none = 0,
static ctype = 1,
static numeric = 2,
static collate = 4,
static time = 8,
static monetary = 16,
static messages = 32,
static all = 63,
_M_impl = 0x7f942a6e3aa0,
static _S_classic = 0x7f942a6e3aa0,
static _S_global = 0x7f942a6e3aa0,
static _S_categories = 0x7f942a6c86a0,
static _S_once = 2
}
},
members of std::basic_filebuf<char, std::char_traits<char> >:
_M_lock = {
__data = {
__lock = 0,
__count = 0,
__owner = 0,
__nusers = 0,
__kind = 0,
__spins = 0,
__list = {
__prev = 0x0,
__next = 0x0
}
},
__size = '\000' <repeats 39 times>,
__align = 0
},
_M_file = {
_M_cfile = 0x70186c0,
_M_cfile_created = true
},
_M_mode = 12,
_M_state_beg = {
__count = 0,
__value = {
__wch = 0,
__wchb = "\000\000\000"
}
},
_M_state_cur = {
__count = 0,
__value = {
__wch = 0,
__wchb = "\000\000\000"
}
},
_M_state_last = {
__count = 0,
__value = {
__wch = 0,
__wchb = "\000\000\000"
}
},
_M_buf = 0x6306000 "\317\317\317\317\317\......320\320\320\320\320"...,
_M_buf_size = 8192,
_M_buf_allocated = true,
_M_reading = true,
_M_writing = false,
_M_pback = 0 '\000',
_M_pback_cur_save = 0x0,
_M_pback_end_save = 0x0,
_M_pback_init = false,
_M_codecvt = 0x7f942a6e3f60,
_M_ext_buf = 0x0,
_M_ext_buf_size = 0,
_M_ext_next = 0x0,
_M_ext_end = 0x0
}
}
(gdb)
Thank you,
Ahmed.
Do not loop on eof.
while (ifs.read(buf.data(), buf.size())) {
size_t read = ifs.gcount();
if(read==0) break; // don't trust passing `0` to `process_data`:
process_data(buf.data(), read);
if (read<buf.size()) break; // if we finished, end.
}
Finding the end of input is best done by attempting io, and noticing something went wrong. In this case, we read, count how many bytes we read, and when we read 0 bytes or have read fewer bytes than we expected to read, we decide there isn't any more data to come.
We also end if any failbit has been set on ifs by the IO operation.

Byte layout of a struct in C vs. C++

I need to write a struct to a filedescriptor. I have code in C that works, however when I try to create a struct of the same layout in memory, the result is different.
In C, the struct is initialized using designated initializers. In C++, in the header file I am defining the structure and initialize every member in an init function in the implementation.
Below are both code snippets. Thank you!
struct in C:
static const struct {
struct usb_functionfs_descs_head header;
struct {
struct usb_interface_descriptor intf;
struct usb_endpoint_descriptor_no_audio sink;
struct usb_endpoint_descriptor_no_audio source;
} __attribute__((packed)) fs_descs, hs_descs;
} __attribute__((packed)) descriptors = {
.header = {
.magic = cpu_to_le32(FUNCTIONFS_DESCRIPTORS_MAGIC),
.length = cpu_to_le32(sizeof descriptors),
.fs_count = 3,
.hs_count = 3,
},
.fs_descs = {
.intf = {
.bLength = sizeof descriptors.fs_descs.intf,
.bDescriptorType = USB_DT_INTERFACE,
.bNumEndpoints = 2,
.bInterfaceClass = USB_CLASS_VENDOR_SPEC,
.iInterface = 1,
},
.sink = {
.bLength = sizeof descriptors.fs_descs.sink,
.bDescriptorType = USB_DT_ENDPOINT,
.bEndpointAddress = 1 | USB_DIR_IN,
.bmAttributes = USB_ENDPOINT_XFER_BULK,
/* .wMaxPacketSize = autoconfiguration (kernel) */
},
.source = {
.bLength = sizeof descriptors.fs_descs.source,
.bDescriptorType = USB_DT_ENDPOINT,
.bEndpointAddress = 2 | USB_DIR_OUT,
.bmAttributes = USB_ENDPOINT_XFER_BULK,
/* .wMaxPacketSize = autoconfiguration (kernel) */
},
},
.hs_descs = {
.intf = {
.bLength = sizeof descriptors.fs_descs.intf,
.bDescriptorType = USB_DT_INTERFACE,
.bNumEndpoints = 2,
.bInterfaceClass = USB_CLASS_VENDOR_SPEC,
.iInterface = 1,
},
.sink = {
.bLength = sizeof descriptors.hs_descs.sink,
.bDescriptorType = USB_DT_ENDPOINT,
.bEndpointAddress = 1 | USB_DIR_IN,
.bmAttributes = USB_ENDPOINT_XFER_BULK,
.wMaxPacketSize = cpu_to_le16(512),
},
.source = {
.bLength = sizeof descriptors.hs_descs.source,
.bDescriptorType = USB_DT_ENDPOINT,
.bEndpointAddress = 2 | USB_DIR_OUT,
.bmAttributes = USB_ENDPOINT_XFER_BULK,
.wMaxPacketSize = cpu_to_le16(512),
.bInterval = 1, /* NAK every 1 uframe */
},
},
};
In c++ header file:
struct stDescriptorBody {
usb_interface_descriptor source_sink_intf;
usb_endpoint_descriptor_no_audio sink;
usb_endpoint_descriptor_no_audio source;
};__attribute__((packed))
struct stDescriptor {
usb_functionfs_descs_head header;
stDescriptorBody fs_descs;
stDescriptorBody hs_descs;
}__attribute__((packed)) ;
init function in implementation file:
stDescriptor initdescriptor(){
stDescriptor descriptor;
descriptor.header.magic = cpu_to_le32(FUNCTIONFS_DESCRIPTORS_MAGIC);
descriptor.header.length = cpu_to_le32(sizeof descriptor);
descriptor.header.fs_count = 2;
descriptor.header.hs_count = 2;
descriptor.fs_descs.source_sink_intf.bLength = sizeof descriptor.fs_descs.source_sink_intf;
descriptor.fs_descs.source_sink_intf.bDescriptorType = USB_DT_INTERFACE;
descriptor.fs_descs.source_sink_intf.bNumEndpoints = 2;
descriptor.fs_descs.source_sink_intf.bInterfaceClass = USB_CLASS_VENDOR_SPEC;
descriptor.fs_descs.source_sink_intf.iInterface = 1;
descriptor.fs_descs.source.bDescriptorType = USB_DT_ENDPOINT;
descriptor.fs_descs.source.bEndpointAddress = 1 | USB_DIR_OUT;
descriptor.fs_descs.source.bLength = sizeof descriptor.fs_descs.source;
descriptor.fs_descs.source.bmAttributes = USB_ENDPOINT_XFER_BULK;
/* .wMaxPacketSize = autoconfiguration (kernel) */
descriptor.fs_descs.sink.bDescriptorType = USB_DT_ENDPOINT;
descriptor.fs_descs.sink.bEndpointAddress = 1 | USB_DIR_IN;
descriptor.fs_descs.sink.bLength = sizeof descriptor.fs_descs.source;
descriptor.fs_descs.sink.bmAttributes = USB_ENDPOINT_XFER_BULK;
/* .wMaxPacketSize = autoconfiguration (kernel) */
descriptor.hs_descs.source_sink_intf.bLength = sizeof descriptor.hs_descs.source_sink_intf;
descriptor.hs_descs.source_sink_intf.bDescriptorType = USB_DT_INTERFACE;
descriptor.hs_descs.source_sink_intf.bNumEndpoints = 2;
descriptor.hs_descs.source_sink_intf.bInterfaceClass = USB_CLASS_VENDOR_SPEC;
descriptor.hs_descs.source_sink_intf.iInterface = 1;
descriptor.hs_descs.source.bDescriptorType = USB_DT_ENDPOINT;
descriptor.hs_descs.source.bEndpointAddress = 1 | USB_DIR_OUT;
descriptor.hs_descs.source.bLength = sizeof descriptor.hs_descs.source;
descriptor.hs_descs.source.bmAttributes = USB_ENDPOINT_XFER_BULK;
/* .wMaxPacketSize = autoconfiguration (kernel) */
descriptor.hs_descs.sink.bDescriptorType = USB_DT_ENDPOINT;
descriptor.hs_descs.sink.bEndpointAddress = 1 | USB_DIR_IN;
descriptor.hs_descs.sink.bLength = sizeof descriptor.hs_descs.source;
descriptor.hs_descs.sink.bmAttributes = USB_ENDPOINT_XFER_BULK;
/* .wMaxPacketSize = autoconfiguration (kernel) */
return descriptor;
}