gbadev.org forum archive

This is a read-only mirror of the content originally found on forum.gbadev.org (now offline), salvaged from Wayback machine copies. A new forum can be found here.

DS development > VOLUMETRIC SHADOW Mini-Tutorial

#143065 - silent_code - Tue Oct 16, 2007 2:29 pm

VOLUMETRIC SHADOW Mini-Tutorial
by silent_code (http://www.code-basement.net), 2007
Updated on July 7th 2008 (2013: Moved to a new domain.)

There are three basic steps you need to follow to render volumetric shadows, but first, lets see what we need to get started:
    - Some geometry to receive shadows

    - Some geometry to cast shadows (just regular geometry, like the "receivers" above)

    - Some shadow volume(s) (also regular geometry)

    - A working code base, that can load and render geometry, as well as "do other stuff" (input etc.) - or at least one of the examples available on the net (like the "Volumetric Shadow Demo" on my web site.)

You can create all of the geometry with your favorite 3D modelling application and convert it to a format, that your base code can load and display on the NDS.
Alternatively, you could also compute any shadow geometry, but that is beyond the scope of this mini-tutorial.

Prepare volumetric shadow rendering by setting the "flush mode" to "manual transparency sorting" and "w-depth-sorting" and sort your translutient geometry - not on a per primitive base, just the translutient objects - as needed. Note, that this also includes shadow geometry!

To set the right flush mode, use the following setting in the frame before any shadow geometry is rendered. That means, whenever shadows are involved, use this:

Code:
glFlush(GL_TRANS_MANUALSORT | GL_WBUFFERING);


Then do the following in each frame:
    1) Render all shadow casting and receiving geometry as usually, then for each and all shadow volumes do the next two steps in exactly that order:

    2) Shadow geometry must be translutient, so make sure you enable blending. Then render the shadow geometry's front (the "Stencil Mask") using the following polygon format. Note, that for the mask, alpha must be between 1 and 30 and the polygon ID must be 0!

Code:
glPolyFmt(POLY_ALPHA(1) | POLY_CULL_FRONT | POLY_ID(0) | POLY_SHADOW);

3) Finally, draw the shadow geometry's back (the "Stencil Shadow") with the next polygon format. Again, alpha must to be between 1 and 30, though this time the polygon ID must not be 0!

Code:
glPolyFmt(POLY_ALPHA(20) | POLY_CULL_BACK | POLY_ID(63) | POLY_SHADOW);

Now you should have volumetric shadowing on screen - all hardware accelerated and rendered. :^)


ADDITIONAL NOTES
    - This is NOT your usual PC stencil buffer! Actually, there is just a single line stencil buffer, which is not accessible by software. Shadow geometry must allways be transparent, as that's part of the "shadow polygon mode trigger" for the graphics hardware!

    - If you're using hardware fogging, you also need to enable it in step 3's polygon format or you will get unfogged shadow areas.
    You can do that on purpose, though. Just like with the flashlight in "Dementium - The Ward".

    - Setting different polygon IDs for different shadow geometry will result in "overlapping" shadows, which create even darker areas. Of course, the that depends on the amount of blending used.

    - Shadow geometry counts just like "regular" geometry, so watch the hardware limits!

    - Shadow geometry can be colored (real world shadows are rarely black, too!)

    - Use closed volumes! The algorithm the hardware is using (the only one available on the NDS), is called "z-pass". Penetrating shadow geometry with the "camera" (so the geometry gets clipped by the near plane) will break it and result in "shadow holes"! Thank you Creative Labs, for that damn "z-Fail" software patent! Really! X^C

    - If the shadows are flickering or the scene isn't rendered at all, go back to the beginning. You probably forgot to set manual transparency sorting and w-sorting correctly.

    - For an object to shadow itself, simply set the polygon ID for it's shadow geometry to something different, as it's regular geometry's polygon ID.

    - ALLWAYS follow the three steps and especially step two and three for each shadow geometry! DO NOT draw all masks first and then all shadows! That's a bad idea!

That's everything!

Happy programming! :^)

PS: If this helped you, feel free to mention me and / or the URL to my web site (http://www.robs-basement.de) somewhere in your program or its documentation. :^)

Disclaimer: Use this information at your own risk. I can't be held liable for any damage, direct or indirect, of any sort, that might result from using this information.

Hereby I grant everyone the right to freely use (commercially or not, it doesn't matter) and distribute (in it's current form, not altered without the author's permission) this information.
Selling this information is strictly prohibited.
_________________
July 5th 08: "Volumetric Shadow Demo" 1.6.0 (final) source released
June 5th 08: "Zombie NDS" WIP released!
It's all on my page, just click WWW below.

Last edited by silent_code on Wed Jan 16, 2013 2:55 am; edited 15 times in total

#143071 - M3d10n - Tue Oct 16, 2007 3:31 pm

silent_code wrote:

- If you're using hardware fogging, you also need to enable it in the latter poly format or you will get unfogged areas where shadows are (though you can do that on purpose).


Ah! So this is how they implemented the flashlight in Dementium: The Ward. I thought it was something way, way more complicated, but it's just clever abuse of the hardware. :)

#143085 - Noda - Tue Oct 16, 2007 7:04 pm

Thanks for the tutorial ;)

Hmm that is exactly what I was doing, but I think I got the missing point in my program: the shadow volume g?ometry itself.

I just thought that the shadow volume would be generated by step 2-3, and the DS just needed the object volume that is casting shadow to calculate the shadow volume...

Then my question is: how to calculate/define the shadow volume geomtery?

Thanks :)

#143089 - silent_code - Tue Oct 16, 2007 7:24 pm

if the hw did it, how would it know the shadow's orientation? ;^D

well... there are some algorithms out there, if you don't want to build (static) volumes by hand (like i did for the sake of simplicity for the demo).

basically you have to build an edge list for your caster's regular geometry. you can do that once, offline.

you then use that edge list ("online", in the game engine) to check if the two connected triangles, relative to a specific light's position, create a silhuette edge. that is, if one triangle normal points "away" from the light and the other points "towards" the light.

after you have all silhuette edges, you construct the shadow volume. this depends on what shadow algorithm you're using.
for the nds, as it's only capable of "z-pass"ing, you'll create a front cap (you can get away without it, but miss it in some situation and the weird artifacts will ruin the whole image!) - which needs to take the near plane into account for "penatratable" shadow volumes. then you "stretch" the silhuette edges into "infinity" (just so the volume penetrates the to be shadowed geometry - commonly read: far away).

you can also use an infinity projection matrix, where everything you send to the hw will be projected close to the far plane (just make sure it doesn't surpass it!). though that's impractical here - half of the vertices need to be pretty much *far* away from the far plane and into the view frutum. (any other opinions on how to use an ipm for volume stretching?)

you don't need a back cap here.

if your edgelist has flaws, you'll see shadow artifacts ("shadow holes") mostly noticable due to the extreme flickering when moving *anything* that affects the volume's computation (thus recomputing the volume).

look for more specific information on the internet, this goes beyond the time i have to share with you guys. sorry!

there's a very good and valuable article series by some id software programmer on intel's developper site. it's about how to optimize shadow volume creation and skinned meshes (also a good read for matrix palette skinning) with sse on pentium processors (and of course AMDs! ;^p ), but it also features "less" and "not" optimized implementations (in fact it goes all the way from "naive" to sse) and is a great inspiraton if you're into "optimizing" and all. ;^p it also explains the algorithm and data structures used and all is downloadable as .pdf.

yippie-kay-yay [...]

#143115 - Ishi - Wed Oct 17, 2007 12:01 am

Nice tutorial, silent_code. If only I'd had this a couple of weeks ago ;)

Something else I don't think was mentioned, the shadow volumes have to be convex for them to always work correctly. That limits you to mostly circular shadows and stuff like that unfortunately.

You can get away with a bit of concave-ness without people noticing though. Go on the kart selection in Mario Kart DS, and Bowser's plane-kart's shadow can be seen breaking. Areas that shouldn't be shadowed end up getting shadowed sometimes.

#159650 - silent_code - Thu Jul 03, 2008 3:38 pm

I did a little update and made a little fix to one statement, that was misleading. Thanks to zeruda for pointing it out to me. ;^)
_________________
July 5th 08: "Volumetric Shadow Demo" 1.6.0 (final) source released
June 5th 08: "Zombie NDS" WIP released!
It's all on my page, just click WWW below.