gbadev.org forum archive

This is a read-only mirror of the content originally found on forum.gbadev.org (now offline), salvaged from Wayback machine copies. A new forum can be found here.

ASM > Malloc in x86/ARM assembly(And IDA Pro Free things)

#158951 - yellowstar - Mon Jun 23, 2008 3:34 am

Could somebody explain this?

IDA Pro Free disassembly of a exported function in my Wmb Asm module:
Code:

                 public _Z17pcap_open_offlinePKcPc
.text:10001360 _Z17pcap_open_offlinePKcPc proc near
.text:10001360
.text:10001360 var_28          = dword ptr -28h
.text:10001360 var_24          = dword ptr -24h
.text:10001360 var_20          = dword ptr -20h
.text:10001360 var_1C          = dword ptr -1Ch
.text:10001360 arg_0           = dword ptr  8
.text:10001360
.text:10001360                 push    ebp             ; 1C number is size of capture header structure
.text:10001361                 mov     ebp, esp
.text:10001363                 push    edi
.text:10001364                 push    esi
.text:10001365                 push    ebx
.text:10001366                 sub     esp, 1Ch        ; void *
.text:10001369                 mov     [esp+28h+var_28], 19h
.text:10001370                 call    malloc
.text:10001375                 mov     ds:lvarpcaplibpcap_open_offline_cap, eax

...

mov     esi, ds:lvarpcaplibpcap_open_offline_cap
.text:10001438                 mov     [esp+28h+var_28], esi
.text:1000143B
.text:1000143B loc_1000143B:                           ; CODE XREF: _Z17pcap_open_offlinePKcPc+22Fj
.text:1000143B                 call    free
.text:10001440                 xor     ecx, ecx
.text:10001442
.text:10001442 loc_10001442:                           ; CODE XREF: _Z17pcap_open_offlinePKcPc+20j
.text:10001442                                         ; _Z17pcap_open_offlinePKcPc+8Fj ...
.text:10001442                 add     esp, 1Ch
.text:10001445                 mov     eax, ecx
.text:10001447                 pop     ebx
.text:10001448                 pop     esi
.text:10001449                 pop     edi
.text:1000144A                 pop     ebp
.text:1000144B                 retn                    ; void *



This would most likely be very similar in ARM assembly, instead of x86. (Register names aside)Anyway: I know the esp register points to the stack. I know with x86/Intel CPUs, the stack grows backwards. So the sub mnemonic would make the stack grow, will add will make the stack shrink. The size that's it's changing the stack with these mnemonics, that's 0x1C. The size of the capture header. Then after the add mnemonic, it does a mov, putting 0x19 in the stack, were esp points. How does malloc know how much to allocate? 0x19 is 3 bytes smaller than 0x1C. Does malloc word-align the value passed to it via stack?(parameter, I think)
Does it use the values subtracted from esp? So would subtract ebp by esp to get the amount to allocate? I tried to look at the dissassembly for malloc, but I couldn't - only got something similar to a header. )-:

I'd much rather RE/disassemble NDS/ARM code, but with IDA Pro Free, the only processors it supports is x86 processors on PC. And the only loaders available are PC only. The commercial has these things, but that's $500. It seems to be possible to add this processors myself with the Free version and the IDA Pro SDK, but on the web site they only have the SDK for IDA 5.2. And the free version is 4.9. I need SDK version 4.9, for the commercial version.(The SDK is free)
Unfortunately, the SDK can't be used with the free IDA Pro without patching. I have that patch, just not the SDK. Does anybody know where I can download it? Unfortunately, loaders/processor modules have to be made specifically for the free version with that patch. Plugins that aren't built for that, and aren't made with SDK version 4.9 too probably, won't work with the free version.(But there seems to be a tool to patch plugins so they work with the free version)
Quote:

Processor modules are blocked by the free version.

)-: But I heard that the processor modules, and even the the loaders, are encrypted. But that might be only in the commercial version, that they would be encrypted. Encrypting modules in the commercial version would make a lot of sense. There's this private key unique to each copy of IDA Pro. The modules are encrypted with this key.(However there might be more things done for this, I don't remember. I read this in a tutorial about this)

EDIT:
I guess there is not encryption in IDA Pro Free. I checked one processor module in an hex editor, and the strings weren't incomplete or anything. That tutorial said if there incomplete/scrambled/and such, the file is encrypted.

#158954 - tepples - Mon Jun 23, 2008 4:38 am

Have you checked GNU Binutils to see if it has a disassembler? If so, you might be able to disassemble devkitARM's libc that way.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#158955 - eKid - Mon Jun 23, 2008 4:50 am

arm-eabi-objdump can disassemble things, I haven't played with it much though.

#158974 - yellowstar - Mon Jun 23, 2008 4:33 pm

Strange... When I ran objdump with cmd.exe for the first time, it worked fine, but after that, Windows kept reporting the program was not a valid Win32 application... And cmd.exe reports "access denied".

I know ndstool can disassemble, but I'd prefer a tool with a GUI, and supports ARM, and maybe even PPC.

#159085 - gladius - Wed Jun 25, 2008 6:03 am

What is the real goal of disassembling malloc? It would be easier to help if we knew the real problem :). You probably don't want to try to map what is happening in x86 to what would happen in ARM. The calling conventions are completely different, which are how arguments are passed to functions. That has a major impact on how the stack is handled and the way functions are written.

arm-eabi-objdump is a pretty effective disassembler (text wise) actually. It sounds like you might have overwritten the exe when you used it the first time, so I'd re-download it.

#159087 - yellowstar - Wed Jun 25, 2008 6:19 am

What I want to know is, how does malloc know how much memory the program wants to allocate? I'll try that sometime gladius.

A few days ago I found the IDA Pro 4.9 SDK, but I couldn't get the samples to compile... )-: I read that the commercial plugins aren't compatible with the free version, unless they are built with a certain patch. And that patch is only for Visual Studio. Is this because there's only problems with VS, or is this a problem with all compilers?

#159097 - tepples - Wed Jun 25, 2008 12:56 pm

yellowstar wrote:
What I want to know is, how does malloc know how much memory the program wants to allocate?

In the ARM procedure call standard, the first four arguments to a function are passed in r0 through r3. This means malloc() looks at r0 to see how many bytes to allocate.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#159098 - Dwedit - Wed Jun 25, 2008 1:05 pm

(post being edited)
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

#159108 - yellowstar - Wed Jun 25, 2008 4:23 pm

tepples wrote:
yellowstar wrote:
What I want to know is, how does malloc know how much memory the program wants to allocate?

In the ARM procedure call standard, the first four arguments to a function are passed in r0 through r3. This means malloc() looks at r0 to see how many bytes to allocate.

So what would the x86 version be?(Yes, I know this forum is for mainly ARM assembly...)
The stack is used for for function parameters in x86, but the value passed isn't the same as the the value that I'm allocating in my C++ code...

#159109 - silent_code - Wed Jun 25, 2008 4:47 pm

Random thought: Maybe that's due to paging or something simillar?
_________________
July 5th 08: "Volumetric Shadow Demo" 1.6.0 (final) source released
June 5th 08: "Zombie NDS" WIP released!
It's all on my page, just click WWW below.

#159117 - tepples - Wed Jun 25, 2008 7:25 pm

Are you using C++? An implementation of operator new MAY[1] just pass the object's size down to malloc(), but it is not REQUIRED to. To help fight memory fragmentation, some implementations of operator new use malloc() to allocate pools of objects with similar sizes and then allocate chunks within those pools.

[1] RFC 2119 defines "MAY".
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#159119 - Maxxie - Wed Jun 25, 2008 8:15 pm

yellowstar wrote:

So what would the x86 version be?(Yes, I know this forum is for mainly ARM assembly...)
The stack is used for for function parameters in x86, but the value passed isn't the same as the the value that I'm allocating in my C++ code...


There are some different calling standards.

The c-calling standard for x86 would be to push arguments from right to left and let the caller clean up afterwards. Return values are passed in EAX.

stdcall would require the callee to clean up.

So a malloc call (clearly a c call) to allocate 10 bytes under flat windows 32bit memory layout, with stack growing down (configureable via d-bit in flags) would look like
Code:

PUSH 0x0000000A
CALL @malloc
ADD esp,4

The pointer is returned in EAX.

#159120 - Dwedit - Wed Jun 25, 2008 8:31 pm

You can also just go to newlib's website and read their code to malloc if you're interested in how it works.
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

#159180 - yellowstar - Thu Jun 26, 2008 4:10 pm

Yes, I'm using C++, but I'm mainly using only malloc, not new.(New is used in the assembly stage, I guess I should switch it back to malloc...)

I need a disassembly of malloc, not C code, as this is assembly I'm working with.

#159199 - tepples - Thu Jun 26, 2008 9:13 pm

You can get the assembly code corresponding to any C or C++ code by using gcc -S where you would otherwise use gcc -c.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#159211 - yellowstar - Fri Jun 27, 2008 4:29 am

I can't get g++ to output the assembly...(With Dev-Cpp)
I tried using g++.exe -c -s, but that didn't work.

#159219 - Dwedit - Fri Jun 27, 2008 12:35 pm

gcc -S -O2 -c file.c -o output.s

must be a captial -S, not a small -s.
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

#159223 - yellowstar - Fri Jun 27, 2008 2:32 pm

That worked, kind of. I got the assembly for dllmain, but it won't work correctly for the wmb cpp code...(Compiling errors. )-: )

#159229 - tepples - Fri Jun 27, 2008 7:30 pm

yellowstar wrote:
(Compiling errors. )-: )

Please paste them.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#159231 - yellowstar - Fri Jun 27, 2008 9:02 pm

Hmm... Apparently it did work at some point, I just didn't notice the assembly file... But I don't see any malloc dissembly anywhere...

#159232 - tepples - Fri Jun 27, 2008 9:34 pm

When you use gcc -S, you should specify the name of an assembly language file, not an object file, after -o.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#159236 - Dwedit - Fri Jun 27, 2008 10:40 pm

You'll only see the assembly of malloc if you're actually compiling malloc. Otherwise, you need to disassemble it from somewhere else. You can find the malloc which devkitarm uses in the CVS browser of newlib.

I think the code for malloc is in this file:
http://sourceware.org/cgi-bin/cvsweb.cgi/~checkout~/src/newlib/libc/stdlib/mallocr.c?rev=1.16&content-type=text/plain&cvsroot=src
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."