gbadev.org forum archive

This is a read-only mirror of the content originally found on forum.gbadev.org (now offline), salvaged from Wayback machine copies. A new forum can be found here.

DS development > libmem (alpha memory handler)

#167299 - hacker013 - Sun Mar 08, 2009 3:31 pm

hey everybody,

I have written this library more for debug purpose the for real use. But I thought maybe you guys/girls can use it. It handles memory allocating. It shows you much your using. How much you can totaly use (defined by yourself). And how much memory is free totaly on the nds. If you find bugs pm me or post it here in this forum. If you have requests ask me :)

Download Url + Documentation.

gr,

newbie013
_________________
Website / Blog

Let the nds be with you.

#167304 - kusma - Sun Mar 08, 2009 4:41 pm

What's the purpose of this library, apart from making false assumptions on how much memory is used and slowing down memory allocations / deallocations? A call to malloc might use more memory than you requested, due to internal memory fragmentation. Also, why does it try to handle arrays differently?

#167310 - hacker013 - Sun Mar 08, 2009 7:01 pm

I handle arrays this way so when you free everything also the arrays can be freed. The purpose of this library was for debugging so you can track down some things in your memory handling.
_________________
Website / Blog

Let the nds be with you.

#167311 - kusma - Sun Mar 08, 2009 7:31 pm

hacker013 wrote:
I handle arrays this way so when you free everything also the arrays can be freed.

Uhm, what? Care to explain that a bit more?

#167312 - hacker013 - Sun Mar 08, 2009 8:19 pm

If I define array like this
Code:

void mem[MAX_MEM_POINTERS];


Now this memory can't be freed anymore.

Code:

void ** mem;


This memory can be freed :)
Like this:
Code:

int i=0;
for(i=0;i<MAX_MEM_POINTERS;i++)
{
      free(mem[0]);
      mem[0]=NULL;
}
free(mem);


I hope this explains it. :)
_________________
Website / Blog

Let the nds be with you.

#167315 - Griffin - Sun Mar 08, 2009 9:06 pm

hacker013 wrote:
If I define array like this
Code:

int i=0;
for(i=0;i<MAX_MEM_POINTERS;i++)
{
      free(mem[0]);
      mem[0]=NULL;
}
free(mem);


I suppose the loop should read
Code:

        free(mem[i]);
        mem[i] = NULL;

Even so, wouldn't it be better to create some sort of array pseudo-class that frees its contents automatically?

#167317 - albinofrenchy - Sun Mar 08, 2009 11:29 pm

You are better off having functions that handle deallocating structs for you.

Also, the built in memory management tool has mallinfo if you haven't checked that out.

#167326 - hacker013 - Mon Mar 09, 2009 8:16 am

Griffin wrote:
hacker013 wrote:
If I define array like this
Code:

int i=0;
for(i=0;i<MAX_MEM_POINTERS;i++)
{
      free(mem[0]);
      mem[0]=NULL;
}
free(mem);


I suppose the loop should read
Code:

        free(mem[i]);
        mem[i] = NULL;

Even so, wouldn't it be better to create some sort of array pseudo-class that frees its contents automatically?


Yes, srry, It was typo.
Classes in c??????????

@albinofrenchy
Yes i've checked it out but it was a little confusing and I don't need it because it doesn't what I want
_________________
Website / Blog

Let the nds be with you.

#167327 - kusma - Mon Mar 09, 2009 8:36 am

hacker013 wrote:
I hope this explains it. :)

It doesn't even begin to explain it, because your library (at least the version I downloaded) doesn't work with double pointers, as your explanation is using.

#167329 - albinofrenchy - Mon Mar 09, 2009 9:24 am

The memory management stuff is a bit confusing, but its good to know.

(BTW, if I fuck any of this up people please correct me.)

Basically the C memory manager sits on top of a low level memory allocator and requests heap space to fit the demand of the memory you request. The function that gives malloc this space is called sbrk ( I don't now why).

So if you look at mallinfo you can see how much memory the C memory manager is taking up in the 'arena' field.

You can view total allocated bytes via 'uordblks'. This is literally the total of all malloc-ish calls minus whatever you've freed (or realloced back, etc etc).

The principle problem with this I think is that you completely do not account for fragmentations in your code. Lets say we have 100 kb of free space. Now I allocate a 20 kb struct, and then a 30 kb struct. Then I free the 20 kb struct. In theory, we have 70 kb of memory left, but if you try to allocate more than 50, you will get a out of memory exception because of fragmentation: that 30 kb structure is fragmenting the memory so that the largest continuous part is only 50 kb.

Kusma is also correct to point out that you are adding a level of complexity on top of memory management that can drastically slow down your app. The only time you want to handle memory management on your own is a) To find memory leaks, which you could probably retool this library to do and b) in cases when you can optimize memory management because of something you know about the data structure. I don't think I've ever seen a case where b was really justified though.

Bluntly, If you tell us exactly what you wanted to track with this tool, we can tell you how to track it in mallinfo more accurately and without slowing down your app.

#167348 - hacker013 - Mon Mar 09, 2009 5:01 pm

I'm using Nitro Engine for my 3d game and it doesn't show some graphics. First I thought I was doing some thing wrong in the code. But that wasn't so I thought maybe because NE allocates all its arrays with the way this library does but without checking if it has enough memory, so with this library I thought to trackdown if the library was using to much memory.
_________________
Website / Blog

Let the nds be with you.

#167350 - albinofrenchy - Mon Mar 09, 2009 6:02 pm

With this library you'd have to go through and replace all malloc's with your code anyway, so you might as well just start putting error handling code in for malloc failures.

Malloc returns 0 when it is out of memory. If I were you, I'd track down where most of the mallocs occur and make sure they are checking that the value they are getting back is not zero.

In general though, if they were abusing memory that bad and not checking for out of memory exceptions the rom would either crash or lock up.

You can check this another way though too. Call mallinfo() and print out the uordblks field every so often. You can be sure that if they have an active memory leak, that number will start to skyrocket until the thing crashes.

Check your premise about your graphics. Graphics on the DS are a pain in the ass in that the most innocuous of programmer error can f up all video output.

#167352 - elhobbs - Mon Mar 09, 2009 6:08 pm

hacker013 wrote:
I'm using Nitro Engine for my 3d game and it doesn't show some graphics. First I thought I was doing some thing wrong in the code. But that wasn't so I thought maybe because NE allocates all its arrays with the way this library does but without checking if it has enough memory, so with this library I thought to trackdown if the library was using to much memory.
are you exceeding the polylimit?

#167354 - Griffin - Mon Mar 09, 2009 6:16 pm

albinofrenchy wrote:
With this library you'd have to go through and replace all malloc's with your code anyway, so you might as well just start putting error handling code in for malloc failures.

That could be fixed with malloc hooks assuming that you can do those on the gba/ds. (I'm just guessing that we're using glibc)

#167356 - kusma - Mon Mar 09, 2009 7:02 pm

Griffin wrote:
That could be fixed with malloc hooks assuming that you can do those on the gba/ds. (I'm just guessing that we're using glibc)

Nope, we're using uclibc.

#167357 - albinofrenchy - Mon Mar 09, 2009 7:53 pm

Quote:
Nope, we're using uclibc.


I'm pretty sure we actually use newlib, unless I missed something.

I looked into it, this is from deep within the newlib malloc source:

Code:
   
* No user-definable hooks for callbacks and the like.


Depending on your affinity for black magic though, you could always do the -finstrument-functions thing and scan for free's and malloc's pointers.

#167358 - kusma - Mon Mar 09, 2009 8:19 pm

albinofrenchy wrote:
Quote:
Nope, we're using uclibc.


I'm pretty sure we actually use newlib, unless I missed something.

Indeed, my bad :)

#167377 - hacker013 - Tue Mar 10, 2009 3:24 pm

elhobbs wrote:
hacker013 wrote:
I'm using Nitro Engine for my 3d game and it doesn't show some graphics. First I thought I was doing some thing wrong in the code. But that wasn't so I thought maybe because NE allocates all its arrays with the way this library does but without checking if it has enough memory, so with this library I thought to trackdown if the library was using to much memory.
are you exceeding the polylimit?


I was using the 2d mode :)
_________________
Website / Blog

Let the nds be with you.

#167379 - elhobbs - Tue Mar 10, 2009 3:47 pm

2d mode? that is your problem right there ;)
Quote:
I'm using Nitro Engine for my 3d game

#167382 - hacker013 - Tue Mar 10, 2009 5:17 pm

No, NE has a option to draw in 2d >.>
_________________
Website / Blog

Let the nds be with you.

#167384 - elhobbs - Tue Mar 10, 2009 5:51 pm

hacker013 wrote:
No, NE has a option to draw in 2d >.>
I was being sarcastic. though, depending on the implementation, it is possible for a 2d mode to use the 3d hardware. in which case it would still be a valid issue. I have no idea how 3d is implemented in this library though.

#167385 - DiscoStew - Tue Mar 10, 2009 5:55 pm

Are you drawing via triangles/quads, or by sprites/backgrounds? If the former, then you are drawing with 3D, even if it is to look 2D.
_________________
DS - It's all about DiscoStew

#167386 - hacker013 - Tue Mar 10, 2009 6:20 pm

DiscoStew wrote:
Are you drawing via triangles/quads, or by sprites/backgrounds? If the former, then you are drawing with 3D, even if it is to look 2D.


Yes, it is 3d, it looks like 2d. And my debugging screen is saying that i'm using 0 polygons oO. I use a NE function to get how many there are used.
_________________
Website / Blog

Let the nds be with you.

#167387 - DiscoStew - Tue Mar 10, 2009 6:50 pm

At what point are you checking the polygon count in your code? If it is after the scene gets flushed, but before you draw anything for the next scene, then expect the result to be zero (unless I'm wrong, and it checks the RAM that holds the polygons/vertices that just got flushed).

Also, are you checking this in an emulator? I could be wrong, but last time I tried retrieving the polygon/vertex count of my program while using an emulator (No$GBA specifically), it returned zero because that feature was unsupported. It may be now, I'm not sure.
_________________
DS - It's all about DiscoStew

#167392 - hacker013 - Tue Mar 10, 2009 7:30 pm

I'm just running it on hardware and I don't know if it before or after it get flushed because NE handles mine drawing to the screen. I have just to use:
NE_ProcessDual(DrawUpperScreen,DrawDownScreen);

DrawUpperScreen and DrawDownScreen are functions.
_________________
Website / Blog

Let the nds be with you.

#167395 - Kath - Tue Mar 10, 2009 7:59 pm

hacker013 wrote:
I'm just running it on hardware and I don't know if it before or after it get flushed because NE handles mine drawing to the screen. I have just to use:
NE_ProcessDual(DrawUpperScreen,DrawDownScreen);

DrawUpperScreen and DrawDownScreen are functions.


NE_ProcessDual() calls the functions you pass then right afterwards calls a flush:

Code:
if(NE_Screen == 1) { topscreen(); }
else { downscreen(); }

GFX_FLUSH = (1<<0); //GL_TRANS_MANUALSORT


I would try reading the poly/vert count near the end of the function you pass in.

#167448 - hacker013 - Thu Mar 12, 2009 2:19 pm

That is what i'm doing but it is always showing 0. :(
_________________
Website / Blog

Let the nds be with you.