#157458 - Sora3087 - Sun May 25, 2008 12:20 am
So I'm trying to make a 3d fighter that is possibly going to use a lot of textures.
Is the best format pcx or a paletted format
All I know is that pcx is a raster type image and it is used for low performance systems, but is it the best for loading a lot of textures on screen at the same time?
Any clarification is appreciated thank you
#157462 - tepples - Sun May 25, 2008 1:08 am
| Sora3087 wrote: |
So I'm trying to make a 3d fighter that is possibly going to use a lot of textures.
Is the best format pcx or a paletted format |
It depends. PCX can be a paletted format, but PCX in 16 colors is designed for planar video architectures (EGA, early VGA, Sega Master System, Game Gear), not the packed-pixel architectures of typical PCs. Do you plan to make your textures 16-color or 256-color?
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#157467 - M3d10n - Sun May 25, 2008 3:26 am
The PCX files will be uncompressed to be displayed in VRAM, so in the end they're only a container format, not a "texture format".
The texture formats are:
4 colors palette
16 colors palette
256 colors palette
32768 colors direct (15-bit color +1-bit alpha)
8 color palette + 32 alpha levels
31 color palette + 8 alpha levels
Compressed texture (2-bits per pixel)
Using PCX you can store the 16, 256 and 32k color formats (but I never managed to save a true 4-bit, 16 color PCX out from photoshop). For the other formats (except the compressed one) you can a tool like texmex.
#157483 - kusma - Sun May 25, 2008 12:59 pm
The compressed texture format really kick ass, but unfortunately it's a bit awkward to use since it dominates VRAM layout a bit. It's not a disaster, but you can't really expect to have one compressed texture and the rest in other formats. I've written a (very naive, but still functional) compressor, perhaps I should release it since I haven't found any publicly available compressors.
edit: Oh, and in case I wasn't clear: Generally, the compressed texture format seems to be the superior if you want to squeeze the most high-color textures (ie not cartoony stuff) into VRAM.
#157484 - silent_code - Sun May 25, 2008 1:35 pm
@ kusma: could you elaborate a bit on the vram dominating thing or post an url?
EDIT: thanks a lot! :^)
_________________
July 5th 08: "Volumetric Shadow Demo" 1.6.0 (final) source released
June 5th 08: "Zombie NDS" WIP released!
It's all on my page, just click WWW below.
Last edited by silent_code on Sun May 25, 2008 3:57 pm; edited 1 time in total
#157486 - kusma - Sun May 25, 2008 2:08 pm
| silent_code wrote: |
| @ kusma: could you elaborate a bit on the vram dominating thing or post an url? |
Each texture is divided up in 4x4 blocks, and the blocks are divided up in a header and a 4x4 bitmap. The problem is that the headers and the bitmaps must be stored in separate, specific VRAM banks, and at fixed offsets relative to each other. I suspect this has to do with the amount of read-ports on the physical RAMs.
#157489 - M3d10n - Sun May 25, 2008 6:52 pm
I'd love to see a working tool for generating compressed textures. Due to the header/bitmap issue all compressed textures' bitmaps should get a whole bank for themselves, right? It isn't that much of a problem if you're using 2 banks for textures (keep one for compressed, and another one for other formats).
#157491 - kusma - Sun May 25, 2008 8:10 pm
| M3d10n wrote: |
| I'd love to see a working tool for generating compressed textures. |
I'll clean up the code a tad and release it asap.
| M3d10n wrote: |
| Due to the header/bitmap issue all compressed textures' bitmaps should get a whole bank for themselves, right? It isn't that much of a problem if you're using 2 banks for textures (keep one for compressed, and another one for other formats). |
No, they require at least two banks. One to put bitmaps in, and one for the block-headers. Bitmaps goes in bank A or C, and headers goes in bank B. As the headers are half the size of the bitmaps, the lower half of bank B is used for headers for bank A bitmaps, and the upper half of bank B is used for bank C bitmaps.
By the way, people seem to think that the compressed format is 2bpp - that's not exactly true. The bitmap is 2bpp, but each 4x4 block needs a header (as mentioned), and that header is 16bits bit. So 3bpp is really a more accurate way to look at it as. And then comes the palette as well ;)
#157492 - simonjhall - Sun May 25, 2008 9:02 pm
Can the compression be done at run time? Or should it really be a one-shot thing?
_________________
Big thanks to everyone who donated for Quake2
#157493 - kusma - Sun May 25, 2008 9:27 pm
| simonjhall wrote: |
| Can the compression be done at run time? Or should it really be a one-shot thing? |
Absolutely not - at least not with my hacky solution :)
It could possibly be doable if it wasn't for the need to optimize the palettes (since they can easily become bigger than the rest of the image if you're not careful). I'm using a bit of brute-force to try to find a kinda-optimal palette-reusing.
#157501 - M3d10n - Mon May 26, 2008 1:32 am
Reading gbatek on compressed textures...
So a 32x32 texture would need 256 bytes of data in bank A or C and 128 bytes in bank B? So in order to use only two texture banks (A and B), bank A would be compressed-texture only, the first half of bank B would be the header (index data) and the rest of bank B could be used for other textures.
One thing... I'm reading the S3TC specs and the DS compressed texture's mode 1 and mode 3 seem to use the exact same color calculation method as S3TC.
I think it might be possible to convert DXT1 textures to DS compressed textures and take advantage of the amazing quantization performed by Nvidia's DDS tools.
#157526 - kusma - Mon May 26, 2008 9:13 am
| M3d10n wrote: |
Reading gbatek on compressed textures...
So a 32x32 texture would need 256 bytes of data in bank A or C and 128 bytes in bank B? So in order to use only two texture banks (A and B), bank A would be compressed-texture only, the first half of bank B would be the header (index data) and the rest of bank B could be used for other textures.
|
Yeah. You don't even need to use the entire bank A if you don't want to either. I'd upload all compressed textures first, and then try to fit in uncompressed ones in the space left.
| M3d10n wrote: |
One thing... I'm reading the S3TC specs and the DS compressed texture's mode 1 and mode 3 seem to use the exact same color calculation method as S3TC.
I think it might be possible to convert DXT1 textures to DS compressed textures and take advantage of the amazing quantization performed by Nvidia's DDS tools. |
The formats and algorithms are pretty similar, the only big difference I can see myself is that the palette isn't stored inside the block-header like in DXT. The block-sizes are different IIRC (isn't DXT 8x4?), but that could easily be handled by just splitting the bitmaps for each block. But yes, This would guarantee only two colors per block, and hopefully the palettes will still be able to be shared (possibly with some block-remapping if needed), and at least between two "macro blocks". I think it's worth looking into, yeah.
DXT is also not that uncommon to do at run-time, but then again, they don't need to merge the palettes. But maybe we don't really need to do that after all? If only two colors are used per block in DXT, and two blocks re-use the same 2-color palette, we end at 16 bits of palette data per block - this is similar to the results currently get with my palette-merging. I suspect I can make a lot smaller palettes by tweaking my compressor a bit, though.
#157540 - JanoSicek - Mon May 26, 2008 2:03 pm
| kusma wrote: |
| M3d10n wrote: | Reading gbatek on compressed textures...
So a 32x32 texture would need 256 bytes of data in bank A or C and 128 bytes in bank B? So in order to use only two texture banks (A and B), bank A would be compressed-texture only, the first half of bank B would be the header (index data) and the rest of bank B could be used for other textures.
|
Yeah. You don't even need to use the entire bank A if you don't want to either. I'd upload all compressed textures first, and then try to fit in uncompressed ones in the space left.
| M3d10n wrote: |
One thing... I'm reading the S3TC specs and the DS compressed texture's mode 1 and mode 3 seem to use the exact same color calculation method as S3TC.
I think it might be possible to convert DXT1 textures to DS compressed textures and take advantage of the amazing quantization performed by Nvidia's DDS tools. |
The formats and algorithms are pretty similar, the only big difference I can see myself is that the palette isn't stored inside the block-header like in DXT. The block-sizes are different IIRC (isn't DXT 8x4?), but that could easily be handled by just splitting the bitmaps for each block. But yes, This would guarantee only two colors per block, and hopefully the palettes will still be able to be shared (possibly with some block-remapping if needed), and at least between two "macro blocks". I think it's worth looking into, yeah.
DXT is also not that uncommon to do at run-time, but then again, they don't need to merge the palettes. But maybe we don't really need to do that after all? If only two colors are used per block in DXT, and two blocks re-use the same 2-color palette, we end at 16 bits of palette data per block - this is similar to the results currently get with my palette-merging. I suspect I can make a lot smaller palettes by tweaking my compressor a bit, though. |
This tool to create compressed textures sounds very interesting to me!
Generally a 4x4 block can consume:
32 bits for texture data
16 bits for palette offset
0-64 bits for palette data.
That is 3-7 bits per pixel. Obviously most interesting would be the ability to compress this to 3-4 bpp, but for that, one needs a lot of palette reuse.
Can you describe an algorithm you used for finding a good reuse of palettes?
JanoS
#157541 - kusma - Mon May 26, 2008 2:58 pm
| JanoSicek wrote: |
| Can you describe an algorithm you used for finding a good reuse of palettes? |
I'm just generating a new sorted, palette for each block, and searching for some pre-existing palette within some error-range. It works okay-ish, leaving me with around 16 bits of palette per block (a bit smaller on some texture, a bit more on some other). Very small textures (32x32 etc) usually gives at least 32bits of palette per block.
I'm planning on changing this to trying to palettizing a bigger block at the time to get some basis-palettes for bigger areas, and trying to re-quantisize each block to see how good a match it is. My hope is that there will be less palettes with really small differences.
I'm finished cleaning up the code, and I'll try to get a release out the door tonight when I come home.
#157555 - kusma - Mon May 26, 2008 6:40 pm
Here's source for the texture compressor. Builds with MinGW. Any bugs and/or suggestions are welcome. I'll try to hack up an example on how to use compressed textures next.
edit: updated the archive with exe and an example
#157569 - silent_code - Mon May 26, 2008 9:28 pm
i'd like to quote a german translation of something eric cartman said in south park, but instead i'll just stick with: that kicks ass! ;^)
_________________
July 5th 08: "Volumetric Shadow Demo" 1.6.0 (final) source released
June 5th 08: "Zombie NDS" WIP released!
It's all on my page, just click WWW below.
#157632 - JanoSicek - Tue May 27, 2008 1:32 pm
| kusma wrote: |
Here's source for the texture compressor. Builds with MinGW. Any bugs and/or suggestions are welcome. I'll try to hack up an example on how to use compressed textures next.
edit: updated the archive with exe and an example |
Great! I haven't had the time to check the source, but it sounds good! I was thinking for few days about some solution, which would be pretty computationally expensive. With constant shrinking of the palettes and trying to make the RMS as small as possible. However the biggest Texture that can fit into VRAM is 1024x1024, which gives you 65536 4x4 blocks. And if you iterate over this in O(N^2) time, you will wait quite a lot for your results...
My idea was to generate first 65536 quantitised blocks, each with different palette (thus consuming up to 256kB of palette VRAM), and then trying to kick some of the palettes out, until the total number of palettes can fit into palette VRAM (96kB) or into a limit given by user...
I'll have to think about the strategy for this for some time...
#157705 - TwentySeven - Tue May 27, 2008 10:23 pm
Microsoft provide a really nice C# implementation of an octree based palette quantizer. I know because I use it for my current stuff.
I mention it because the palette reduction problem you guys are mentioning sounds like an expanded version of the traditional true-color to 256 color quantization issue, except for the standard 3 rgb triplets per color, you have (I think) 12 triplets per 'palette' that you need reduced.
#157716 - kusma - Wed May 28, 2008 12:54 am
| TwentySeven wrote: |
Microsoft provide a really nice C# implementation of an octree based palette quantizer. I know because I use it for my current stuff.
I mention it because the palette reduction problem you guys are mentioning sounds like an expanded version of the traditional true-color to 256 color quantization issue, except for the standard 3 rgb triplets per color, you have (I think) 12 triplets per 'palette' that you need reduced. |
Luckily, I'm already using a good one, one of the routines provided in FreeImage. Quantisizing palettes isn't my primary concern. Being able to reuse the memory is.
edit: Thanks for the suggestion anyway!
Last edited by kusma on Wed May 28, 2008 9:17 am; edited 1 time in total
#157726 - Sausage Boy - Wed May 28, 2008 8:50 am
Hey kusma, take a look at the squish library! That guy has put some really nice work into this kind of compression.
_________________
"no offense, but this is the gayest game ever"
#157727 - kusma - Wed May 28, 2008 9:16 am
| Sausage Boy wrote: |
| Hey kusma, take a look at the squish library! That guy has put some really nice work into this kind of compression. |
Hey, that's awesome! Thanks a lot for the link!