gbadev.org forum archive

This is a read-only mirror of the content originally found on forum.gbadev.org (now offline), salvaged from Wayback machine copies. A new forum can be found here.

DS development > Inefficient texture layout in glTexImage2D

#123684 - 3D_geek - Fri Mar 30, 2007 7:15 am

Suppose for sake of argument that I allocated RAM banks A and B for textures (they are 128kbytes each). Now assume I'm using the 16 bits per texel mode and I have the following set of textures:

Seven maps each of: 128x128x2bytes = 32kbytes each
Two maps each of 8x8x2 bytes = 128 bytes each

That's a total of 32k*7+128*2 = 224.25kbytes. So theoretically, I have plenty of texture memory (2 banks==256kbytes - I only need 224.25kbytes).

However - if I say:

glTexImage2D ( .... 8x8 map ....);
glTexImage2D ( .... 128x128 map ....);
glTexImage2D ( .... 128x128 map ....);
glTexImage2D ( .... 128x128 map ....);
glTexImage2D ( .... 128x128 map ....);
glTexImage2D ( .... 8x8 map ....);
glTexImage2D ( .... 128x128 map ....);
glTexImage2D ( .... 128x128 map ....);

...then, finally:

glTexImage2D ( .... 128x128 map ....);

...the last call fails because it claims to have run out of texture map memory. The reason being (as far as I can see from the libnds code) that the system simply stuffs textures into the first bank in the order they are presented - and when there is not enough room left in that bank for the next texture, it moves on to the next bank and starts allocating from there. However, in the case above, this means that the first 8x8 map and the next three 128x128's go into bank A, then the fourth 128x128 won't quite fit - so it goes onto bank B, then in goes the 8x8 and then two more 128x128's - and finds that there is not quite enough room left for the final 128x128 map.

The problem is that there was quite a bit of space left (just short of 32kbytes) at the end of bank A that never got used. If the second 8x8 map had gone into there instead of into bank B, there would have been enough memory left over for the last map.

This is a classic memory fragmentation problem - the best heuristic for which is always to put each new object into the smallest space into which it'll fit.

Hence, the algorithm needs to remember how much memory there is left at the end of each bank when it opens a new one. Then it can fit each new texture into whichever bank has the least amount of room left into which the map will fit. If that were the case then the second of the tiny 8x8 maps would have been stuck into the end of bank A instead of taking a teeny bit out of bank B and thereby preventing the last large map from just sneaking in.

In the absence of that change, it's stongly advised that if your applications need more than 128kbytes of textures that you follow the rule of allocating your maps in some very carefully pre-planned order in order to give yourself the best chance of fitting everything in, RIght now, I think the simplest strategy is to allocate your largest maps first, moving gradually down to doing the smallest ones last.

...but it really needs to be fixed properly in libnds.

#123719 - 3D_geek - Fri Mar 30, 2007 4:06 pm

I'm using banks A, B and D for texture and bank C for the bottom screen. In theory I have just under 256kbytes of texture so it should fit into banks A & B - but due to the problem I described above, it probable actually uses a little of bank D too. That's inefficient - but it's not "wrong" as such.

When I test the results of 'glTexImage2D', it says that all of my textures fit into memory...which is what I'd expect because I've allocated three banks for texture which is WAY more than I need.

But the odd thing is that the bottom display is all screwed up. This suggests that there is something wrong with the bank selection code and instead of using bank D for my 'overflow' textures, it's actually (illegally) using bank C.

Anyway - when I discovered the problem in texture bank allocation, I changed my texture loaded to sort my textures by size and carefully allocate things in big-to-small order. That should allow my textures to fit into banks A & B without overflowing.....and lo and behold...just by changing the order of texture loading - the problem with the bottom screen has 'gone away'.

Do I have something wrong with my startup code? (I freely confess I have no clue what most of this actually does).
Code:

  powerON ( POWER_ALL ) ;
  lcdMainOnTop    () ;
  videoSetMode ( MODE_0_3D ) ;
  videoSetModeSub(MODE_0_2D | DISPLAY_BG0_ACTIVE);
  vramSetBankC(VRAM_C_SUB_BG);
  SUB_BG0_CR = BG_MAP_BASE(31);
  BG_PALETTE_SUB[ 0 ] = RGB15(0,0,0); // black backdrop
  BG_PALETTE_SUB[255] = RGB15(31,31,31); // white foreground
  consoleInitDefault((u16*)SCREEN_BASE_BLOCK_SUB(31),
                                (u16*)CHAR_BASE_BLOCK_SUB(0), 16);
  irqInit();
  irqEnable(IRQ_VBLANK);
  vramSetBankA(VRAM_A_TEXTURE_SLOT0);
  vramSetBankB(VRAM_B_TEXTURE_SLOT1);
  vramSetBankD(VRAM_D_TEXTURE_SLOT2);
  vramSetBankC(VRAM_C_SUB_BG);

  glReset () ;

....then I start calling glGenTextures/glBindTexture/glTexImage2D - and as I do so, the bottom display is corrupted.

#124535 - simonjhall - Fri Apr 06, 2007 6:09 pm

Did you ever sort out these problems? I've never had the problem you're describing, but I AM writing my own VRAM allocator at the moment and I've been staring at those libnds vram/texture functions for the last half an hour!

Thing is, I'm having trouble following the code and matching it to what you're saying... Let's say you assign A, B and D for textures, but C for other stuff. I can see that if a texture hangs off the end of B (into the unassigned C section) then the code will instead upload the texture to the beginning of D (and waste the end of B). But if you upload a texture and it crosses the A/B boundary then I don't think the code uploads the texture to the beginning of B...
I can't see anything on gbatek which says that textures can't straddle banks...

It's not the easiest piece of code, so I could be mistaken!
_________________
Big thanks to everyone who donated for Quake2

#124556 - 3D_geek - Fri Apr 06, 2007 9:23 pm

simonjhall wrote:
Did you ever sort out these problems?


I have a work-around...but it's not ideal.

Quote:
Thing is, I'm having trouble following the code and matching it to what you're saying...


Yes - it's a little tricky to follow - and I could be wrong too.

Quote:
Let's say you assign A, B and D for textures, but C for other stuff. I can see that if a texture hangs off the end of B (into the unassigned C section) then the code will instead upload the texture to the beginning of D (and waste the end of B). But if you upload a texture and it crosses the A/B boundary then I don't think the code uploads the texture to the beginning of B...
I can't see anything on gbatek which says that textures can't straddle banks...

I assumed they couldn't...but either way, there is still a problem if we have a big texture that doesn't fit in B - it can't straddle into C because that's used for something else - so, yes, it has to go into D. But from what I understand of the code, if I then give it a small texture, it doesn't check to see if there was space left at the end of B - it mindlessly sticks it onto the end of D.

So space is frequently wasted at the ends of RAM banks if you allocate a large texture that has to be bumped off onto the next bank.

Until/unless this gets fixed, the best advice I have (and what my code currently does) is to allocate large maps first - working down to the smallest maps last. That way, this problem is least likely to happen (although it still can) - and if it does happen, the penalties are minimised.

What really needs to happen is for the code to keep track of how much space is left at the end of each bank that's allocated to texture and to assign new textures to whichever bank has the least available space (into which the new texture none-the-less fits).