gbadev.org forum archive

This is a read-only mirror of the content originally found on forum.gbadev.org (now offline), salvaged from Wayback machine copies. A new forum can be found here.

DS development > Stencil Shadow Casting

#159631 - zeruda - Thu Jul 03, 2008 6:33 am

I'm trying to get shadow casting using the hardware working but it eludes me. Here is what I am doing so far:

Code:

    glPushMatrix();
        BoxPos = CharacterPosition;
        glColor3f(1,0,0);
        glPolyFmt(POLY_ALPHA(1) | POLY_SHADOW | (1<<14));
        DrawOutCube(0, 0.03, BoxPos);
    glPopMatrix(1);

    glPolyFmt(POLY_ALPHA(30) | CULL_BACK | POLY_ID(8) | LIGHT_0);
    glPushMatrix();
        BoxPos.z -= 0.1;
        glColor3f(0,1,0);
        DrawOutCube(0, 0.03, BoxPos); // <draw quad>
    glPopMatrix(1);

    glPushMatrix();
        glColor3f(0.9,0.9,0.9);
        BoxPos = CVector(4.0, 0.0, 0.0);
        glPolyFmt(SETALPHA(30) | CULL_BACK | POLY_ID(12) | LIGHT_0);
        DrawOutCube(0, 3, BoxPos); // 100 times the other cubes size
    glPopMatrix(1);

    glColor3f(1,1,1);
    glPolyFmt(SETALPHA(30) | POLY_FOG | CULL_BACK | LIGHT_0 | POLY_ID(17));
    glPushMatrix();
        Texture.Bind(Texture.texture[0]);
        DrawLandscape()
    glPopMatrix(1);

    Texture.Bind(Texture.texture[1]);
    glPushMatrix();     // Draw Kazir
        glPolyFmt(SETALPHA(30) | POLY_FOG | CULL_BACK | LIGHT_0 | POLY_ID(11));
        DrawCharacter();
    glPopMatrix(1);

    glFlush(GL_TRANS_MANUALSORT | GL_WBUFFERING);
    // or glFlush(GL_TRANS_MANUALSORT);


So theres a character that moves around. A shadow box that surrounds the character. A normal box offset on the z axis to the character. A terrain, and a giant box.

I get some strange results.
The character and terrain show as normal.
The shadow box is invisible and does nothing.
The giant box which should be white is normally invisible, but if I move so that it is in between the camera and the terrain, it is clipped to the terrain by the stencil. So where there is no terrain the box is invisible, where there is terrain I can see white.
The same happens with the character.
The other green box is stencilled to the terrain. So if in the sky it is invisible, any parts overlapping the terrain green shows. If both the white and green box overlap, they are invisible, and if they both overlap the terrain the white box wins the fight and white shows.

Any suggestions on what I'm doing wrong? I get some sort of stencilling, but no projection of any kind of shadows. Is the algorithm/approach correct?

Thanks

#159641 - ritz - Thu Jul 03, 2008 2:25 pm

You need to draw the shadow twice, culled differently in the following order:

Code:
... some code ...

GFX_POLY_FORMAT = POLY_SHADOW | POLY_CULL_FRONT | V_POLY_ALPHA(plya) | BIT(12) | BIT(13) | POLY_FOG | V_POLY_ID(0);
v_drawvoid(pso->svdp);

GFX_POLY_FORMAT = POLY_SHADOW | POLY_CULL_BACK | V_POLY_ALPHA(plya) | BIT(12) | BIT(13) | POLY_FOG | V_POLY_ID(scpid);
v_drawvoid(pso->svdp);

... more code ...

This is just a copy and paste from my stuff. You probably don't need the other things depending on your program.

Oh, and you need to use glFlush(GL_TRANS_MANUALSORT);

#159649 - silent_code - Thu Jul 03, 2008 2:58 pm

Did you see my "Volumetric Shadow Demo"? It's on my website, sources and builds are available, along with a little tutorial. I hope that helps. :^)

Could you post any images? It sounds like you are setting the attributes wrong, but I am not sure.
_________________
July 5th 08: "Volumetric Shadow Demo" 1.6.0 (final) source released
June 5th 08: "Zombie NDS" WIP released!
It's all on my page, just click WWW below.

#159659 - zeruda - Thu Jul 03, 2008 7:28 pm

I've already got predefined volumetric meshes working. I was misunderstanding while experimenting. What I want to do is project the shadow of a character onto a 3D terrain. Once I've done that then the stencil would clip it onto the terrain accurately. So what I'm missing is dynamically generating a volumetric mesh.

Would I have to do it in "software" as in calculate the individual vertices of a shadow volume?

Or is it possible to change some status settings, maybe load a light matrix of some kind and then fire my normal character mesh and let the DS do it? (if that makes sense)

#159662 - silent_code - Thu Jul 03, 2008 9:20 pm

You could render the same mesh, but with a "squish" matrix. ;^)
That means the matrix would either (simple case) "eliminate" the Y coordinate by setting all (local model space) Ys to 0 (and then displace the mesh accordingly, so that it appears on the *flat* ground) or (more complex case) the matrix would procect all vertices to the plane defined by the ground's normal...
Effectively, you would have to define some oriented "flat space" with some "cool" base vectors and all that maps a volume onto a plane. ;^)

You just have to look up shadow projection on the web and you'll find a ton of implementations with free sources.

Good luck. :^)
_________________
July 5th 08: "Volumetric Shadow Demo" 1.6.0 (final) source released
June 5th 08: "Zombie NDS" WIP released!
It's all on my page, just click WWW below.