gbadev.org forum archive

This is a read-only mirror of the content originally found on forum.gbadev.org (now offline), salvaged from Wayback machine copies. A new forum can be found here.

DS development > Background Transparency and 3D Edge Antialiasing

#125061 - rhaegar - Wed Apr 11, 2007 2:29 am

A bit of a noob here so bear with me.

I seems like you can 'blend' more than 2 backgrounds together using BLEND_CR.
Code:
BLEND_CR = BLEND_ALPHA | BLEND_DST_BG0 | BLEND_DST_BG1  | BLEND_DST_SPRITE | BLEND_SRC_BG2;
BLEND_AB = 8 | 8<<8;
This will render BG0, BG1 and sprites normally and then blend BG2 over this rendered image. The reverse however (swapping DST and SRC and inverting the background priorities) does not seem to work.

Situation:
I'm using 2 background layers, BG0 and BG3 under mode 5. BG3 is a full screen tilled image. I'm using the 3D on BG0 to simulate 2D vector graphics using antialiasing and layers of flat shaded polygons along with one large polygon at the back for the background.
Code:
BG0_CR = BG_PRIORITY(BG_PRIORITY_1);
BG3_CR = BG_COLOR_16 | BG_RS_32x32 | BG_MAP_BASE(0) | BG_TILE_BASE(1) | BG_PRIORITY(BG_PRIORITY_0);
BLEND_CR = BLEND_ALPHA | BLEND_DST_BG0 | BLEND_DST_SPRITE | BLEND_SRC_BG3;
BLEND_AB = 8 | 8<<8;
This will work properly, drawing the 3D and the priority 1 sprites, then blending this with BG3, and finally drawing the priority 0 sprites over that.

Problem:
I want the transparent sprites to blend with the 3D background and not BG3. As i can't seem to simply replace BLEND_DST_SPRITE with BLEND_SRC_SPRITE, i revese the role of BG0 and BG3.
Code:
BG0_CR = BG_PRIORITY(BG_PRIORITY_0);
BG3_CR = BG_COLOR_16 | BG_RS_32x32 | BG_MAP_BASE(0) | BG_TILE_BASE(1) | BG_PRIORITY(BG_PRIORITY_1);
BLEND_CR = BLEND_ALPHA | BLEND_SRC_BG0 | BLEND_DST_SPRITE | BLEND_DST_BG3;
BLEND_AB = 8 | 8<<8;
However, only the 3D and priority 0 sprites appear (no blending).
Changing POLY_ALPHA under glPolyFmt will seem to fix the problem though i seem need a new POLY_ID for each polygon layer or they disappear completely (havent tried changing for every layer as it will require a significant recode). From what i can tell with different IDs though, i seem to loose antialiasing.

So my question is, how can i blend the 3D layer into the rest of my screen without loosing antialiasing?

#125064 - DekuTree64 - Wed Apr 11, 2007 2:53 am

Yeah, anti-aliasing only works for opaque polygons, so you'll have to use BLEND_CR if you want transparency+AA.

I'm not sure if I understand the effect you're going for though. So you want a solid tile background in the back, a 50% transparent 3D layer above it, and then 50% transparent sprites on top of all that? And you need to be able to see the 3D stuff through the sprites?
_________________
___________
The best optimization is to do nothing at all.
Therefore a fully optimized program doesn't exist.
-Deku

#125066 - rhaegar - Wed Apr 11, 2007 3:15 am

I want a solid tile background in the back, sprites ontop of this background, and 50% transparent 3D layer ontop of all of it. I can do 3D layer at the back, sprites above it, and 50% tile background on the top. The difference being i want the sprites to blend with the 3D as opposed to the tile layer.

#125069 - DekuTree64 - Wed Apr 11, 2007 3:32 am

Oh ok, so the sprites themselves aren't transparent? That should be no problem then. Try this:

BG0: Priority 0
Sprites: Priority 1
BG3: Priority 1

BLEND_CR: SRC_BG0 | DST_BG3 | DST_SPRITE
_________________
___________
The best optimization is to do nothing at all.
Therefore a fully optimized program doesn't exist.
-Deku

#125073 - rhaegar - Wed Apr 11, 2007 4:06 am

This is what i had initially. The problem is that the sprites blend with BG3 but i want them to blend with the 3D...
Will have a look at screen capture and see if that is of any use, though its probably more effort that its worth.
Might just stick with what i have.

#125075 - DekuTree64 - Wed Apr 11, 2007 4:45 am

rhaegar wrote:
The problem is that the sprites blend with BG3 but i want them to blend with the 3D...

Hmm, that's strange. With dest set to BG3+sprites, those two should be flattened together before blending takes place. Are you testing on hardware, or an emulator? If you can't run on hardware, send me a .nds and I'll try it. My email is my nickname here, at hotmail.com.
_________________
___________
The best optimization is to do nothing at all.
Therefore a fully optimized program doesn't exist.
-Deku

#125076 - rhaegar - Wed Apr 11, 2007 5:03 am

Sorry, read ur post wrong. Thought u had BG0 as DST and BG3 as SRC.
I can put the BG3 on top (flattening out 3D and sprites), but putting 3D on top (BG0 as DST) means only the 3D is displayed for some reason.
I'm running it on hardware.