IDA & MASM Confusions

**nishant** · 03-01-2012, 04:36 PM

Greetings to all,

I have been trying to brush my ASM skills offlate and started to understand the internals of MASM and IDA. I coded a very simple program with MASM. The code is as follows:

Code:

.386
.model flat,stdcall
option casemap:none

include windows.inc
include kernel32.inc
includelib kernel32.lib
include user32.inc
includelib user32.lib

.data
szMsg db "Beep",0
szCaption db "Windows Beep",0
szOK db "You pressed OK",0
szCancel db "You pressed Cancel",0

.code
main:
invoke Beep,750,500
invoke MessageBox,NULL,addr szMsg,addr szCaption,MB_OKCANCEL
.IF eax==IDOK
    invoke MessageBeep,MB_ICONEXCLAMATION
    invoke MessageBox,NULL,addr szOK,addr szCaption,MB_OK
.ELSE
    invoke MessageBeep,MB_ICONEXCLAMATION
    invoke MessageBox,NULL,addr szCancel,addr szCaption,MB_OK
.ENDIF
xor eax,eax
invoke ExitProcess,eax
end main

Now when I assemble this code and do a static analysis in IDA, I see the following dis-assembly:

Now my question is where do the INT 3 in the last but one line of IDA analysis came from? Because the IDA analysis of the following code:

Code:

.386
.model flat,stdcall
option casemap:none

include windows.inc
include kernel32.inc
includelib kernel32.lib
include user32.inc
includelib user32.lib

.data
szCaption db "CommandLine",0

.data?
hInstance HINSTANCE ?
CmdLine LPSTR ? 
szStr dd ?

.code
main:
invoke GetCommandLine
mov CmdLine,eax
invoke MessageBox,NULL,CmdLine,addr szCaption, MB_OK
xor eax,eax
invoke ExitProcess,eax
end main

gives the following output in IDA:

Name: cmdline.jpg
Views: 231
Size: 24.2 KB

Anyone can help me understand, what is the concept behind this? Any help would be greatly appreciated.

Thanks
Nishant

**nishant** · 03-01-2012, 04:45 PM

Sorry for the poor resolution of images. Better resolution version are below:

Image#1 > https://docs.google.com/open?id=0B0H...WGlpMEx4Z1pjdw
Image#2 > https://docs.google.com/open?id=0B0H...SGxlVHAwalU0dw

**fb1h2s** · 03-01-2012, 05:04 PM

Hi,

Since we already had this discussion, and I got more doubts am putting my explanation + my doubts .

#3495222 - Pastie

And the IDA generated alternate of this code is the following,

#3495228 - Pastie

[Question ]

Her if we check the IDA code on line 68 we could see

int 3 ; Trap to Debugger

Which is not there in the actual MASM code, and a sensible explanation for this is ,

[Answer]

Since a call is made to exit Process and not a JMP,

call ExitProcess

A CALL on completion of a module would return back to caller, in order to stop that form happening since its an ExitProcess call , an Intrupt [int 3] is raised and the program halts and won't let it return back to the MAIN, coz it's pointless.

So this is what I believe is happening here , and in the following case,

MASM Code: #3495340 - Pastie

IDA Code: #3495336 - Pastie

Here in the following code since the code conversation is in the main module a direct JMP is made which dsn't have to return back to caller, so the INT 3 call is omitted.

47 jmp ds:ExitProcess

Now this explains what Nishant has asked, but my confusion now is how does , MASM compiler treat IF loops, does anyone got a documentation| reference on code conversation on MASM IF : ELSE clauses . Is If statement treated as separate function if so, then what data does each IF , ELSE, module return etc etc.

You always have cool brain teasers Man \m/

**nishant** · 03-02-2012, 10:46 AM

Hi Rahul,

Thanks for your reply. But question still remains: That in both the cases I have done

Code:

xor eax,eax
invoke ExitProcess,eax
end main

in the main block (not inside any IF..ELSE or any secondary function.) Then why did MASM convert one to Call ExitProcess and other to as JMP ds:ExitProcess. Hope you get my questions.

Thread: IDA & MASM Confusions

LinkBack

Thread Tools

Search Thread

Display