Anti-debugging trick - Checking for the Low Fragmentation Heap

Hi everyone,

I'll introduce you today a Anti-debugging trick which the idea came across my mind while debugging Windows Heap, I don't know if it was used before anywhere but here I am showing it today.

Check the C/C++ source code : http://pastebin.com/pacrKh0z

Short introduction to the Windows front end allocator :

Yesterday I promised writing something related to the windows heap manager and an opportunity came now to discuss some of the front end allocator aspects.
First of all let me define what a LFH (low fragmentation heap) is :
The LFH was introduced in Windows XP and Windows Server 2003 but it wasn't used as a default front end allocator until Windows Vista. The default front end allocator were the lookaside lists (LAL) , each of these 2 is a singly linked list with 128 lists.

The LFH as its name describes is implemented to guarantee that heap fragmentation will be reduced and it's strongly recommended to use for application that allocate a big number of small size blocks.
When the LFH is created first, predetermined sizes of memory will be allocated and put into buckets (LFH lists), when the application will call for an allocation the LFH will provide the smallest available block to satisfy the allocation , otherwise the request will be passed into the heap manager then to the Freelists (check explanation later).

When LAL is used as a front end allocator, a block won't reside in a list of its list until it was allocated either from the FreeLists or by committing memory then freed. All that won't apply until a list of the lookaside table can handle the freed block otherwise it will passed to the Heap manager to perform coalescing if two adjacent blocks are free, update the FreeList bitmap values (if necessary), invalidate coalesced blocks entries then insert the coalesced block into its valid list in the FreeLists. If no block coalescing is possible the block is inserted directly in the FreeLists.

Anti-Debugging Trick :

I noticed that when the executable is run under a debugger no Low Fragmentation Heap (LFH) is created for it so the pointer to the LFH equals NULL.
So we'll just have to check if the pointer to LFH is null to detect if the process was created inside a debugger.
I tried after to run the process outside the debugger then attach it and I noticed that a LFH is created for the heap and the pointer to the LFH is valid.
 The pointer to the LFH is located at "heap_handle+0xd4" under Windows 7 for WOW64 executables and at "heap_handle+0x178" for 64-bit executable.

I attached the debugger to the process :
0:001> dt _HEAP 00460000
ntdll!_HEAP
   +0x000 Entry            : _HEAP_ENTRY
   +0x008 SegmentSignature : 0xffeeffee
   +0x00c SegmentFlags     : 0
[...]
   +0x0d0 CommitRoutine    : 0x5b16148e
   +0x0d4 FrontEndHeap     : 0x00468cf0 Void
   +0x0d8 FrontHeapLockCount : 0
   +0x0da FrontEndHeapType : 0x2 <-- Type : LFH
   +0x0dc Counters         : _HEAP_COUNTERS
   +0x130 TuningParameters : _HEAP_TUNING_PARAMETERS

When running the process from the debugger LFH won't be enabled :
0:001> dt _HEAP 00320000
ntdll!_HEAP
   +0x000 Entry            : _HEAP_ENTRY
   +0x008 SegmentSignature : 0xffeeffee
   +0x00c SegmentFlags     : 0
[...] 
   +0x0cc LockVariable     : 0x00320138 _HEAP_LOCK
   +0x0d0 CommitRoutine    : 0x6d58ec0e     long  +6d58ec0e
   +0x0d4 FrontEndHeap     : (null)
   +0x0d8 FrontHeapLockCount : 0
   +0x0da FrontEndHeapType : 0
   +0x0dc Counters         : _HEAP_COUNTERS
   +0x130 TuningParameters : _HEAP_TUNING_PARAMETERS

Remember that, The LFH isn't used by default until Windows Vista and posterior versions , so to implement the anti-debugging technique under Windows XP we'll need to enable the LFH as it's not used by default, to do so you'll simply need to call HeapSetInformation with the HEAP_INFORMATION_CLASS set to 0 and with the pointer to the information buffer pointing to "0x2" which will enable the LFH for the heap passed as the first argument.

A simple way to bypass this technique is simply by attaching the debugger to the application instead of running it from a debugger :) .

More details on the LFH : here

Thanks for your time :)
Souhail Hammou. @Dark_Puzzle

Creating and using your own 'heap' manager

Hi,
First of all, I didn't want to write a long title so I've put word heap between apostrophes, simply because in this example we're not creating or managing a heap, but a (zeroed) chunk of  memory which we reserve from the virtual address space of our application. (we'll commit from the reserved chunk as needed)

The differences between a heap and our custom heap are huge.
The only thing that you have to keep in mind for now is that a Heap is divided into Segments each segment contains heap blocks within the committed memory range in which the user/application accessible part resides.
Another thing that I want to mention is that the Windows heap manager keeps track of free blocks using the BEA (back-end-allocator or FreeLists) and the LFH/LAL (low fragmentation heap or lookaside lists).
In addition each block in use by the application or that is in the LAL is considered busy (A Flag withing each block describes its status) and each block within a free list is considered Free.
Maybe I'll write a detailed post soon about Windows Heap Management.

Back to our topic : I spent much time in the previous months trying to understand the concepts behind the Windows Heap Manager and also some of its exploitation techniques. Thus, I also added a small security check in the custom manager that does safe unlinking from the FreeList.
Our custom manager will clone only a small part of Windows heap management which is implementing a circular FreeList structure and will also use a singly linked list to keep track of all allocated and free blocks. I programmed it so no new portions will be committed from reserved memory until there are no free blocks to allocate from.
You will be able also to display details about the MAP (singly linked list) and the FreeList used while allocating and freeing to see the changes.

Full commented C++ source here : http://pastebin.com/2LgcByyC

Execution example :
I've allocated some blocks then freed some of them and displayed the map :

And after that displayed the FreeList :

 

P.S : 0x002A4D10 is the FreeList's head.

You can play with allocations and frees as you like then display details :)

See you soon,

Souhail Hammou. @Dark_Puzzle .

HackIM CTF 2014 - Reverse100 write-up

Hey,
The challenge is a 32-bit executable : when executing it displays the following :
   Flag : )T(+,*'))$&T(Y)*#(+&#+)$%'T+&#(T
I found that  ")T(+,*'))$&T(Y)*#(+&#+)$%'T+&#(T" is passed to 3 functions, only one of them is executed when running the program (the one that will concatenate 'Flag :' and the encrypted flag then pass them to WriteFile which will write the output to the console) .
So there might be a piece of code that may decrypt the encrypted flag somewhere and was keeped there but never refered to anywhere.
When I saw where the other pushes are it turned out to be the decryption stub which is never executed.
Disassembly here : http://pastebin.com/sw6DkfPN

As you can see it takes every character from  ")T(+,*'))$&T(Y)*#(+&#+)$%'T+&#(T" then adds "0xD" to it , after that it concatenates the result with another string "5F 41 4E 44 5F 4D 4F 4F 4F 4F" which is  "_AND_MOOOO" in ASCII.

Thus the flag would be : 6a589746613a5f670583086124a8305a_AND_MOOOO
Easy right ? :)
Cheers,

Souhail Hammou.

Introduction To Windows API Hooking.

Hey,
Today I wrote a piece of code that hooks MessageBoxA API to redirect execution to another function that will call MessageBoxA with different arguments.
This is the traditional way to hook APIs , there are other methods used for instance IAT hooking.

C++ Visual Studio Express 2010 : http://pastebin.com/ny3NGn93

The 1st step was to Hotpatch MessageBoxA function. In other words some functions in Windows DLLs are ready to be hooked that's why a "mov edi,edi" (which represent a 2 bytes NOP) which will be patched to a short jump is the first instruction executed in each of these functions.

The question that you may ask is why not simply using 2 NOPs instead of "mov edi,edi" , the answer is related to performance : each NOP takes 1 clock cycle so 2 NOPs will take 2 clock cycles. But "mov edi,edi" instruction will take only 1 clock cycle.

Each "mov edi,edi" is preceded by 5 NOPs which are never being executed. Hotpatching consists of overwriting "mov edi,edi" by a short 5 bytes jump back. And then inserting our hook by overwriting the 5 NOPs with a long jump instruction that will execute our desired function, this requires us to calculate a relative address.

When the hook is no longer needed it is sufficient to restore only "mov edi,edi" instruction as the 5 bytes before it are never executed.

See you soon :)

Souhail Hammou.

A Quick Dive Into Windows Kernel-Mode Debugging - Protected Processes.

Hey :) ,
This is maybe a first write-up of many which will discuss windows kernel debugging.
I've mentioned in my previous post "protected processes" and I decided to share with you this writeup discussing them and showing some applied kernel debugging.
Protected processes are digitally signed processes that were first introduced in Windows Vista in order to protect Media contents (HD/Blu-ray).
The use of Protected processes wasn't hindered by Media use only . Processes such as System and Werfault.exe (Displays information about crashes and needs access to protected processes in case one of them crashes) are protected. So you can see that protected processes can interact with each other normally. In addition, even a normal process has the ability to create a Protected process BUT it will certainly fail because a special digital signature isn't present in the created process
Another point is that Protected Processes forbid users even with an administrator account to debug them,dump memory,edit memory. Furthermore, access rights to a protected process are limited for instance PROCESS_QUERY_LIMITED_INFORMATION / PROCESS_SUSPEND_RESUME / PROCESS_TERMINATE accesses are granted while PROCESS_ALL_ACCESS is not.

As you may know, each process has an executive process (EPROCESS) structure that resides in Kernel mode and which can be accessed only from it. The EPROCESS structure contains detailed information about the process and points to many other structures like KTHREAD structures list, and it also contains structures like pcb which is a KPROCESS structure and the first element of EPROCESS.

Speaking about Protected Processes , EPROCESS has a special bit that defines the nature of a process (protected or not).
In our example we will try to attach (audiodg.exe) which is a protected process to OllyDbg. The result is the following:

We have 2 choices here , the first is to make Ollydbg a protected process and then Attach it to audiodg.exe or zeroing the bit in audiodg.exe EPROCESS structure to make it unprotected then attach it normally.

We'll do the first one, we can similarly do this with "Task Manager" to create a dump of audiodg.exe or any protected process then examine that dump.
So let's set up the kernel debugger and display the EPROCESS structure for ollydbg.exe :

lkd> dt _EPROCESS 846d8030
nt!_EPROCESS
   +0x000 Pcb              : _KPROCESS
   +0x098 ProcessLock      : _EX_PUSH_LOCK
   +0x0a0 CreateTime       : _LARGE_INTEGER 0x01cf1a21`239824ec
   +0x0a8 ExitTime         : _LARGE_INTEGER 0x0
   +0x0b0 RundownProtect   : _EX_RUNDOWN_REF
   [...CUT __ CUT...]
   +0x26c Flags2           : 0x2d014
   +0x26c JobNotReallyActive : 0y0
   +0x26c AccountingFolded : 0y0
   +0x26c NewProcessReported : 0y1
   +0x26c ExitProcessReported : 0y0
   +0x26c ReportCommitChanges : 0y1
   +0x26c LastReportMemory : 0y0
   +0x26c ReportPhysicalPageChanges : 0y0
   +0x26c HandleTableRundown : 0y0
   +0x26c NeedsHandleRundown : 0y0
   +0x26c RefTraceEnabled  : 0y0
   +0x26c NumaAware        : 0y0
   +0x26c ProtectedProcess : 0y0        <-- bit in which we are interested.
   +0x26c DefaultPagePriority : 0y101
   +0x26c PrimaryTokenFrozen : 0y1
   +0x26c ProcessVerifierTarget : 0y0
   +0x26c StackRandomizationDisabled : 0y1
   +0x26c AffinityPermanent : 0y0
   +0x26c AffinityUpdateEnable : 0y0
   +0x26c PropagateNode    : 0y0
   +0x26c ExplicitAffinity : 0y0
   [...CUT __ CUT...]
   +0x2b0 RequestedTimerResolution : 0
   +0x2b4 ActiveThreadsHighWatermark : 5
   +0x2b8 SmallestTimerResolution : 0
   +0x2bc TimerResolutionStackRecord : (null)

So all we need to do now is set that bit , in order to so we'll need to edit Flags2.
You can see that the "Protected Process" bit is the 12th bit and we'll need to set it to 1.
0x2d014 -> 101101  0  00000010100
Setting this bit will need us to set Flags2 to 0x2d814 ->101101  1  00000010100 .

lkd> ed 846d8030+0x26c 0x2d814
lkd> dt _EPROCESS 846d8030
nt!_EPROCESS
   +0x000 Pcb              : _KPROCESS
   +0x098 ProcessLock      : _EX_PUSH_LOCK
   [...]
   +0x26c Flags2           : 0x2d814
   +0x26c JobNotReallyActive : 0y0
   +0x26c AccountingFolded : 0y0
   +0x26c NewProcessReported : 0y1
   +0x26c ExitProcessReported : 0y0
   +0x26c ReportCommitChanges : 0y1
   +0x26c LastReportMemory : 0y0
   +0x26c ReportPhysicalPageChanges : 0y0
   +0x26c HandleTableRundown : 0y0
   +0x26c NeedsHandleRundown : 0y0
   +0x26c RefTraceEnabled  : 0y0
   +0x26c NumaAware        : 0y0
   +0x26c ProtectedProcess : 0y1        <-- Ollydbg.exe is a protected process now
   +0x26c DefaultPagePriority : 0y101
   +0x26c PrimaryTokenFrozen : 0y1
   +0x26c ProcessVerifierTarget : 0y0
   +0x26c StackRandomizationDisabled : 0y1
   +0x26c AffinityPermanent : 0y0
   +0x26c AffinityUpdateEnable : 0y0
   +0x26c PropagateNode    : 0y0
   +0x26c ExplicitAffinity : 0y0

Now we'll simply try to attach audiodg.exe and it will work as expected :

I hope you enjoyed reading this short writeup.
See you later :).

Regards,

Souhail Hammou . @Dark_Puzzle

DLL injection - Injecting a DLL into a running process.

Hey,

Today I had some free time and decided to write a piece of code that will inject a DLL into a running process. In this blog entry I'll be briefly explaining the injection process.

DLL injection is wildly used by normal software, malware , game cheats. For example , game trainers will inject a DLL in order to hook certain functions inside the game or simply patch them in order to get a certain cheat to work. There are many techniques to inject a DLL into a process and today I'll be introducing one of them to you.

The C++ source code can be found here : http://pastebin.com/wW81nApT
(Tested on win 7 64-bit)

The concept is simple , we will walk through all running processes list until finding the target process in which we want to inject the DLL.
After getting the process id we have to get a Handle to it , that's why we'll use the OpenProcess function.
The only important thing that you need to be aware of is trying to inject a DLL into a protected process which will certainly fail .
Protected processes are processes which are required to be signed .Injecting a dll , a thread , accessing their virtual memory or debugging them is prohibited. So the PROCESS_ALL_ACCESS won't work in this case. Only few Accesses are allowed like SYNCHRONIZE.

After getting the HANDLE we'll try to allocate some memory in the target process and make write/read access possible to it. Thus, we'll be able to copy the dll name into the process's virtual address space (VAS) using WriteProcessMemory.
We're copying the DLL name into the target process for the simple reason that a process can't load a DLL if its name/directory isn't in its VAS.
The next thing is injecting the DLL so we'll need the address of LoadLibraryA function which we'll pass to CreateRemoteThread and as an argument to it we'll pass the pointer to where our DLL name is in the target process VAS.

Now our DLL is successfully injected into the process , we'll be in our DllMain as soon as the CALL to LoadLibrary is made.After that I used CloseHandle to close the handle to the process.

Cheers :) and see you soon.

Souhail Hammou.

64-bit calc.exe Stack Overflow Root Cause Analysis

Introduction :
I was surfing the net when I found by chance a blog post about a bug found in calc.exe under Win7 64bit.
http://marcoramilli.blogspot.com/2013/08/bug-in-wincalcexe.html
I've tried to reproduce right away in the same environment and then calc.exe crashed. I hadn't any stuff going on so I said to myself why not do a root cause analysis of the bug ? and as I started taking notes it turned out to be an article (Just gathered the notes here)...

This article is about my analysis of the bug so if you noticed something that appears to be wrong or unclear don't mind e-mailing me about it or leaving a comment below. I'll be glad to learn from you :) .

Reproducing the crash :
The crash can be simply reproduced by following these steps :
- Open calc.exe and calculate 1/255.
- Choose the [F-E] button.
- Crash !! .
I reproduced with calc.exe attached to Windbg and the following was displayed :

(2f0c.2618): Stack overflow - code c00000fd (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
ntdll!RtlpAllocateHeap+0x30:
00000000`77c23520 89542434        mov     dword ptr [rsp+34h],edx

The status code shows us that this is a stack overflow , and the exception is triggered when an instruction will try to write to an address that is believed to belong to the stack but it's not. This is because the stack has run out of memory and can't commit anymore space.
The stack committed and reserved memory can be examined by dumping the PE header using "!dh" command under Windbg :
0000000000080000 size of stack reserve
0000000000002000 size of stack commit

size of stack reserve : The total stack size used by the application.
size of stack commit : The stack space that can be used by the application until hitting the guarded page (a page from reserved memory). The access to that page is now granted and the page guard is the next page.This process will continue until "size of stack reserve" is reached which will result in a STACK OVERFLOW EXCEPTION, for the simple reason that the application will try to write out of the stack bound.

Now we need to know at least what has caused the Stack Overflow Exception , so all we have to do is check the call stack.


RETADDR                       CALL SITE
00000000`77c234d8 :     ntdll!RtlpAllocateHeap+0x30
00000000`77ca70dd :     ntdll!RtlAllocateHeap+0x16c
00000000`77c6b5aa : ntdll!RtlDebugAllocateHeap+0xcd
00000000`77c234d8 : ntdll! ?? ::FNODOBFM::`string'+0x18b42
000007fe`fdb81635 : ntdll!RtlAllocateHeap+0x16c
00000000`ff5d1d89 : KERNELBASE!LocalAlloc+0x71
00000000`ff5d5ed4 :       calc!_createnum+0x2d
00000000`ff5d5c4d :   calc!_addnum+0x64
00000000`ff5d7514 : calc!addnum+0x67
00000000`ff5d27da :   calc!_divnum+0x154
00000000`ff5d5aa8 :   calc!putnum+0x14d
00000000`ff5d5aa8 : calc!putnum+0x22d
00000000`ff5d5aa8 : calc!putnum+0x22d
00000000`ff5d5aa8 : calc!putnum+0x22d
00000000`ff5d5aa8 : calc!putnum+0x22d
00000000`ff5d5aa8 : calc!putnum+0x22d
00000000`ff5d5aa8 : calc!putnum+0x22d
[...]
00000000`ff5d5aa8 : calc!putnum+0x22d

Oops, the cause of the problem is a recursive call . It has written its arguments , return address and local variable and arguments to other function calls so many times until the stack has been exhausted.
But why ? that's the question we'll be answering until the last sentence of this article.
For some of you , it appears useless to look for a root cause of this kind of bug and maybe it is. But I found working on this enjoyable cause this is the first bug I encountred in calc.exe.

The first thing I started with is the calc!putnum function itself , what's its prototype and what are its parameters.
In our case the putnum function is called by calc!putrat :
Caller : (calc!Putrat)
00000000`ff29571c 488d542448      lea     rdx,[rsp+48h]
00000000`ff295721 448bc5              mov     r8d,ebp
00000000`ff295724 498bcc              mov     rcx,r12
00000000`ff295727 e834010000      call    calc!putnum (00000000`ff295860)
00000000`ff29572c 488b4c2448      mov     rcx,qword ptr [rsp+48h]
Callee : (calc!Putnum)
00000000`ffd55860 488bc4            mov     rax,rsp
00000000`ffd55863 44894018        mov     dword ptr [rax+18h],r8d (argument_3)
00000000`ffd55867 48895010        mov     qword ptr [rax+10h],rdx (argument_2)
00000000`ffd5586b 48894808        mov     qword ptr [rax+8],rcx (argument_1)
00000000`ffd5586f 53                     push    rbx
00000000`ffd55870 55                    push    rbp
00000000`ffd55873 4883ec48        sub     rsp,48h

Keep in mind that these arguments will be accessed from now on using rax , and rsp will be used to store the functions' local variables.

In x64 , argument passing consists the use of rcx,rdx,r8 and r9 , any other additional parameters passed will be pushed on stack.
Here in our case the callee will place arguments passed to it (in rcx,rdx,r8d) on stack in the same way the caller would have pushed them.

The image above shows  : 

- calc!putrat stack frame .

- the return address for the calc!putnum (return to calc!putrat+0xb8) being just pushed.

- the arguments freshly placed on stack by the callee.

Full disassembly of calc!putnum : http://pastebin.com/imDN8DUa

After some analysis here's what I came to concerning the putnum function :

Argument_1 appears to be a pointer to a stack address which has the value "00000000`00000001" (was set originally as a dword so it is of type int).

Argument_2 appears to be a pointer also to a stack address which has a pointer also.Thus,Argument_2 is a pointer to a pointer.

Argument_3 this appears to be of type int :  mov     dword ptr [rax+18h],r8d.

Argument_2 is a pointer to a pointer to an Array of integers that we will see in detail soon.

The function returns a pointer to a unicode string . The string is nothing but the "valid" array integers converted to be ready for the display.

C/C++ Function prototype :

wchar_t* putnum(int* arg1,int** arr,int arg3);

We will be mainly interested in argument_2 (int** arr).

Now all we need to do is spot the recursive call and see how it's reached following conditional and unconditional jumps.

cmp edi,edx :  In the case when the call to putnum isn't made : edi = edx which means that edi value is NULL. EDX always equals 0 before the comparison , thus now only EDI now is the problem.

I found that EDI switches its value between 1 and 0.

When triggering the bug , EDI value is always 1 before the comparison. (Putnum is called everytime).

This value of edi is returned from calc!stripzeroesnum in eax.

The Array that I talked about is something like this :

arr[2] has a negative value , we add to it the difference between the previous and current total length elements.

This will increase its value , because zeroe(s) have been stripped.

The value (arr[2] - 1) is the length of the whole array. In my opinion , the negative value describes that the integers are stored in the inverted way.

More information about calc!stripzeroesnum :

Full disassembly of the function : http://pastebin.com/jSt2Ufh0

I worked also on manually decompiling the function because it's important for us: 

A part of this function compares an element of the array to NULL , if it is : it simply increments the index and set the return value to 1 then compare again again until finding a value that is different from NULL. Besides, it calls a function calc!memmove which will Strip all the zeros from the array so the starting of the array (where the integers to convert are stored , however the first element of the array is at (first_integer_index - 3)) will be the first element different from NULL. This function also uses ebx as a counter for the elements in the array so it decrements whenever the next element is accessed.

P.S : The function is supplied a Maximum value that the valid integers in the array don't have to exceed , this value is supplied through edx . The array contains more than just the integers for example the array[1] contains the total numbers of the integers to process . 

If total_int > MAX then the array will be accessed from the element array[total_int-MAX] to make the number of integers in the wanted interval and then the Max value itself will be used as a counter. If total_int < MAX then the counter will still ebx by default and the array will be accessed from index 3.

The compare instruction is : cmp  dword ptr [rdx],0.

RCX is supplied to the function and RCX+0xC is the first element of the array which holds the integers . These integers are nothing but the result of the arithmetic operation. The integers are stored in the reverse order in memory so the last ones are the first ones listed in memory.

calc!stripzeroesnum+0x19: mov ebx,dword ptr [rsi+4] , ebx holds now the number of the elements.The elements are only the Xs "0.XXX..." where X is different from NULL. This is the second element of the array which holds the total number of valid integers (total_int). 

e.i : for 0.000005 , ebx will hold the value 1.

PS : the screen can display 34 characters max for integers that start with 0,XXXX (0 and the point are counted)  and 33 characters for numbers that start with X,XXXXX... (the point is counted).

In this article we will be interested in 2 calls to calc!stripzeroesnum , conditions will control each execution flow of the execution . 

the first call is at :  "calc!putnum+0x34".

The second one is at : "calc!putnum+0x203". 

The main condition that controls the access to the array is a comparison between ebx and r8d : 

cmp  ebx,r8d

jg  calc!stripzeroesnum+0x26

we dicussed the value of ebx earlier (total_int), and the value of r8d is supplied by the caller in edx.So we will be studying 2 cases .

Concerning the call at "calc!putnum+0x34" : The caller supplies as an argument edx with a value of 0x20+2. calc!g_maxout == 0x20 the value 2 is added in case the number to display contains a point and an additional number (0.) for example.

Concerning the call to stripzeroesnum at "calc!putnum+0x203": In this call the array is accessed using an index in RAX the comparison starts with the first element accessed.

This isn't done until the result is bigger to be displayed , to fix this , the array is accessed using an index which simply is the result of the substraction between the total integers to process in the array and the max value allowed.

 sub     ebx,r8d

 movsxd  rax,ebx

 mov     ebx,r8d

 lea     rdx,[rcx+rax*4]

Now the question is Why does the crash happen only when pressing [F-E] button and not when clicking "=" button ??!

Actually ,sometimes, there's a majoration when trying to display the result using the "=" button when the value is long enough , a majoration would simply make the following value 0.009009009009009009009009009009... , this one  0.00900900900900901.

The function putnum calls a function (calc!addnum) which does that , and suprisingly this function does a majoration just in case we want to display the number using "=" button and it does not do it when we're trying to display the scientific notation using [F-E].

The Crash Cause :

A Quick Resumé :

- As was mentioned earlier, the addnum function will do a majoration if needed to decrease the length and precision. Sometimes , in a normal case , the recursive call will be accessed but the MAX will be greater than total_int so the execution will not reach calc!stripzeroesnum.

- Surprisingly , when choosing the [F-E] mode , the addnum will not do any majoration and will sometimes add more precision to the number.

- Remember when I said that the array ignores the zeroes after the point and before the first number different from zero ?

"0.0016316168515"

     ^^----Ignored.

Those will be added later to the total_int and then compared with calc!g_maxout global variable which is equal 20.

calc!putnum+0x186:

mov     eax,ecx

sub     eax,esi  <-- ESI has a negative value so it is added to eax.

cmp     eax,r8d

jg      calc!putnum+0x1b3

Conditions To Reproduce The Crash :

The difference is that when clicking "=" button a majoration of the number will be made to stop getting more precision , then the zeroes at the end will be stripped off. However , when choosing [F-E] mode in special cases (1/255 , 1/111 , 1/999) the addnum will always add precision, and total_int+zeros_after_the_point > g_maxout , which will take us to stripzeroesnum at putnum+0x203 again .This one will access the array from the index (total_int-MAX) which points always to a NULL elements then strip off zeros. When the recursive call is made the stripzeroesnum+0x34 will get an array that is stripped off of zeroes. However total_int > calc!g_maxout or total_int+zeros_after_the_point > calc!g_maxout.

Then As soon as the call to addnum is made : more precision will be added making the array larger. The array should exceed the max value and making (total_int - MAX) poiting to a NULL element again which will result in returning 1 (stripping the zero and numbers before in the array) and calling putnum again which will call addnum  ....etc until the stack is fully overflown.

Well , That's all for this article. See you again soon.

Best regards, Souhail.
Twitter : @Dark_Puzzle