Anti-debugging trick - Checking for the Low Fragmentation Heap

Hi everyone,

I'll introduce you today a Anti-debugging trick which the idea came across my mind while debugging Windows Heap, I don't know if it was used before anywhere but here I am showing it today.

Check the C/C++ source code : http://pastebin.com/pacrKh0z

Short introduction to the Windows front end allocator :

Yesterday I promised writing something related to the windows heap manager and an opportunity came now to discuss some of the front end allocator aspects.
First of all let me define what a LFH (low fragmentation heap) is :
The LFH was introduced in Windows XP and Windows Server 2003 but it wasn't used as a default front end allocator until Windows Vista. The default front end allocator were the lookaside lists (LAL) , each of these 2 is a singly linked list with 128 lists.

The LFH as its name describes is implemented to guarantee that heap fragmentation will be reduced and it's strongly recommended to use for application that allocate a big number of small size blocks.
When the LFH is created first, predetermined sizes of memory will be allocated and put into buckets (LFH lists), when the application will call for an allocation the LFH will provide the smallest available block to satisfy the allocation , otherwise the request will be passed into the heap manager then to the Freelists (check explanation later).

When LAL is used as a front end allocator, a block won't reside in a list of its list until it was allocated either from the FreeLists or by committing memory then freed. All that won't apply until a list of the lookaside table can handle the freed block otherwise it will passed to the Heap manager to perform coalescing if two adjacent blocks are free, update the FreeList bitmap values (if necessary), invalidate coalesced blocks entries then insert the coalesced block into its valid list in the FreeLists. If no block coalescing is possible the block is inserted directly in the FreeLists.

Anti-Debugging Trick :

I noticed that when the executable is run under a debugger no Low Fragmentation Heap (LFH) is created for it so the pointer to the LFH equals NULL.
So we'll just have to check if the pointer to LFH is null to detect if the process was created inside a debugger.
I tried after to run the process outside the debugger then attach it and I noticed that a LFH is created for the heap and the pointer to the LFH is valid.
 The pointer to the LFH is located at "heap_handle+0xd4" under Windows 7 for WOW64 executables and at "heap_handle+0x178" for 64-bit executable.

I attached the debugger to the process :
0:001> dt _HEAP 00460000
ntdll!_HEAP
   +0x000 Entry            : _HEAP_ENTRY
   +0x008 SegmentSignature : 0xffeeffee
   +0x00c SegmentFlags     : 0
[...]
   +0x0d0 CommitRoutine    : 0x5b16148e
   +0x0d4 FrontEndHeap     : 0x00468cf0 Void
   +0x0d8 FrontHeapLockCount : 0
   +0x0da FrontEndHeapType : 0x2 <-- Type : LFH
   +0x0dc Counters         : _HEAP_COUNTERS
   +0x130 TuningParameters : _HEAP_TUNING_PARAMETERS

When running the process from the debugger LFH won't be enabled :
0:001> dt _HEAP 00320000
ntdll!_HEAP
   +0x000 Entry            : _HEAP_ENTRY
   +0x008 SegmentSignature : 0xffeeffee
   +0x00c SegmentFlags     : 0
[...] 
   +0x0cc LockVariable     : 0x00320138 _HEAP_LOCK
   +0x0d0 CommitRoutine    : 0x6d58ec0e     long  +6d58ec0e
   +0x0d4 FrontEndHeap     : (null)
   +0x0d8 FrontHeapLockCount : 0
   +0x0da FrontEndHeapType : 0
   +0x0dc Counters         : _HEAP_COUNTERS
   +0x130 TuningParameters : _HEAP_TUNING_PARAMETERS

Remember that, The LFH isn't used by default until Windows Vista and posterior versions , so to implement the anti-debugging technique under Windows XP we'll need to enable the LFH as it's not used by default, to do so you'll simply need to call HeapSetInformation with the HEAP_INFORMATION_CLASS set to 0 and with the pointer to the information buffer pointing to "0x2" which will enable the LFH for the heap passed as the first argument.

A simple way to bypass this technique is simply by attaching the debugger to the application instead of running it from a debugger :) .

More details on the LFH : here

Thanks for your time :)
Souhail Hammou. @Dark_Puzzle