Disable LFH in Windows? - c++

Question

I'm looking to disable the LFH for an application I'm trying to debug. I'm able to rebuild and redeploy the application, but I cannot attach a debugger or set any gflags.
What's a good way to disable the LFH with these constraints? Is there maybe an attribute I can modify on the executable itself? Or some startup code I can add to the program?

Answer 1

On Vista and Win7, I think you can disable the Low-Fragmentation Heap on a per-executable basis with the Application Compatibility Toolkit.
On XP, the documentation suggests you don't get a LFH by default. So probably it is your C++ runtime library (which you haven't named) that is turning it on. And it cannot be disabled once enabled. So check the documentation for your particular runtime library to see if you can tell it not to enable LFH, or if there's another version of the runtime library you can link with that doesn't enable it.
See also this thread on the Microsoft forums

Answer 2

Based on Michael Burr's comment above about the IMAGE_LOAD_CONFIG_DIRECTORY containing GlobalFlagSet I wrote the following code to demonstrate enabling the correct GlobalFlag to disable the Low Fragmentation Heap. One caveat about writing your own IMAGE_LOAD_CONFIG_DIRECTORY at compile time is that it disables SafeSEH.
// editloadconfig.cpp : Defines the entry point for the console application.
//
#include "stdafx.h"
#include <windows.h>
#include <tchar.h>
#include <stdio.h>
/*
typedef struct {
DWORD Size;
DWORD TimeDateStamp;
WORD MajorVersion;
WORD MinorVersion;
DWORD GlobalFlagsClear;
DWORD GlobalFlagsSet;
DWORD CriticalSectionDefaultTimeout;
DWORD DeCommitFreeBlockThreshold;
DWORD DeCommitTotalFreeThreshold;
DWORD LockPrefixTable; // VA
DWORD MaximumAllocationSize;
DWORD VirtualMemoryThreshold;
DWORD ProcessHeapFlags;
DWORD ProcessAffinityMask;
WORD CSDVersion;
WORD Reserved1;
DWORD EditList; // VA
DWORD SecurityCookie; // VA
DWORD SEHandlerTable; // VA
DWORD SEHandlerCount;
} IMAGE_LOAD_CONFIG_DIRECTORY32, *PIMAGE_LOAD_CONFIG_DIRECTORY32;
*/
extern "C"
IMAGE_LOAD_CONFIG_DIRECTORY _load_config_used = { 0x48, 0, 0, 0,0, 0x00000020/*enable heap free checking*/};
// change the last value to 0 to not enable any globalflags
#define HEAP_STANDARD 0
#define HEAP_LAL 1
#define HEAP_LFH 2
#define SIZE 100
int _tmain(int argc, _TCHAR* argv[])
{
BOOL bResult;
HANDLE hHeap;
ULONG HeapInformation;
void* allocb[0x12+1];
// based on "Understanding the LFH" paper at
// http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=5&ved=0CE0QFjAE&url=http%3A%2F%2Fillmatics.com%2FUnderstanding_the_LFH.pdf&ei=GlBvT9yrMKHy0gGHpLnaBg&usg=AFQjCNGsvVtl54X7MWGyWYqiSrsdTBrbXQ
int i = 0;
for(i = 0; i < 0x12; i++)
{
printf("Allocation 0x%02x for 0x%02x bytes\n", i, SIZE);
allocb[i] = HeapAlloc(GetProcessHeap(), 0x0, SIZE);
}
printf("Allocation 0x%02x for 0x%02x bytes\n", i++, SIZE);
printf("\tFirst serviced by the LFH\n");
allocb[i] = HeapAlloc(GetProcessHeap(), 0x0, SIZE);
// LFH is now activated so the query below will return 0 or 2.
// sample code from MSDN for querying heap information
//
// Get a handle to the default process heap.
//
hHeap = GetProcessHeap();
if (hHeap == NULL) {
_tprintf(TEXT("Failed to retrieve default process heap with LastError %d.\n"),
GetLastError());
return 1;
}
//
// Query heap features that are enabled.
//
bResult = HeapQueryInformation(hHeap,
HeapCompatibilityInformation,
&HeapInformation,
sizeof(HeapInformation),
NULL);
if (bResult == FALSE) {
_tprintf(TEXT("Failed to retrieve heap features with LastError %d.\n"),
GetLastError());
return 1;
}
//
// Print results of the query.
//
_tprintf(TEXT("HeapCompatibilityInformation is %d.\n"), HeapInformation);
switch(HeapInformation)
{
case HEAP_STANDARD:
_tprintf(TEXT("The default process heap is a standard heap.\n"));
break;
case HEAP_LAL:
_tprintf(TEXT("The default process heap supports look-aside lists.\n"));
break;
case HEAP_LFH:
_tprintf(TEXT("The default process heap has the low-fragmentation ") \
TEXT("heap enabled.\n"));
break;
default:
_tprintf(TEXT("Unrecognized HeapInformation reported for the default ") \
TEXT("process heap.\n"));
break;
}
return 0;
}

Answer 3

You can use the gflags.exe tool that's included with the WDK (maybe also the SDK via the Debugging Tools for Windows package) to manipulate a subset of the gflags in the executable image's PE header. Just go to the "Image File" tab in 'gflags.exe'.
As pointed out by jcopenha in a comment, it looks like gflags.exe does not manipulate the PE file header (I was relying on information from "Windows Internals, Fifth Edition" in Chapter 9's "Heap Debugging Features" section) - apparently it only manipulates the "Image File Execution Options" registry key.
However, it may still be possible possible to set (or clear) the gflags bits for a particular executable in the image - see the docs for the IMAGE_LOAD_CONFIG_DIRECTORY structure; in particular the GlobalFlagsClear and GlobalFlagsSet fields:
GlobalFlagsClear - The global flags that control system behavior. For more information, see Gflags.exe.
GlobalFlagsSet - The global flags that control system behavior. For more information, see Gflags.exe.
You can dump these fields with dumpbin (or link /dump) using the /loadconfig option:
C:\temp>dumpbin /loadconfig test.exe
Microsoft (R) COFF/PE Dumper Version 10.00.40219.01
Copyright (C) Microsoft Corporation. All rights reserved.
Dump of file test.exe
File Type: EXECUTABLE IMAGE
Section contains the following load config:
00000048 size
0 time date stamp
0.00 Version
0 GlobalFlags Clear
0 GlobalFlags Set // <=======
0 Critical Section Default Timeout
// remainder of dump snipped...
You can get the RVA of the "Load Configuration Directory" using dumpbin /headers:
C:\temp>dumpbin /headers test.exe
Microsoft (R) COFF/PE Dumper Version 10.00.40219.01
Copyright (C) Microsoft Corporation. All rights reserved.
Dump of file test.exe
// ...
OPTIONAL HEADER VALUES
// ...
142B0 [ 40] RVA [size] of Load Configuration Directory
// ^^^^^ ^^^
// ...
As a point of interest, the /loadconfig and /headers option disagree on the size of the structure (for the record, it looks like the /header info isn't right)
Unfortunately, I'm unaware of PE editor that directly supports these fields - you'll probably have to use a hex editor (or the hex editing feature of a PE editor) to change those fields. The RVA of the IMAGE_LOAD_CONFIG_DIRECTORY structure should help you find it in the hex editor.
I believe that setting one or more of the heap debugging flags in the image header (maybe any of them, but you might have to experiment) will disable the low fragmentation heap. But I haven't tested whether or not setting bits in these fields actually works. If you try this, please let us know how it fares.

Answer 4

The simplest way if you can't change the configuration on the machine, is to set the heap information.
http://msdn.microsoft.com/en-us/library/windows/desktop/aa366705(v=vs.85).aspx
I believe you can disable the LFH heap programmatically this way, although I haven't tried it.