#16426 - headspin - Mon Feb 16, 2004 1:25 am
If I have lz77 compressed tiles I find I can't just uncompress straight to a char base address, instead I need to uncompress using the BIOS call LZ77UnCompWRAM to EWRAM and then write it to character memory (0x060040000). Because the BIOS decompression function only has a source and destination address and no size I'm assuming the lz77 compressed data has a header indicating the size of the file. Which means it has to uncompress the entire file all in one hit.
This raises questions about compressing audio using a differential pass then a huffman. Where do you compress to and do you have to have a buffer the size of the entire file to cater for all the data to be decompressed. And using multiple compression on one file seems to mean you have to have two buffers.
Someone mentioned something about the ability to decompress back over the original data. Any ideas?
What is the best way to use buffers with decompression?
#16428 - Miked0801 - Mon Feb 16, 2004 1:50 am
Write your own decompressors that do a better job than the Nintendo ones :)
#16431 - DekuTree64 - Mon Feb 16, 2004 3:42 am
That's what the UnCompLZ77VRAM BIOS function is for. I haven't done any tests myself, but I've heard it's a little slower, and possibly not quite as good of compression rates as the WRAM version (if you're using GBACrusher to compress, be sure to use -LV to work with the VRAM function), but I use it for all sorts of stuff and it works great.
Also, the compressed data does contain a header telling the size and such.
I'm not sure about writing to the buffer you're decompressing from. Having a general idea of how the LZ77 algorithm works (actually I think the BIOS routines are LZSS, but it's the same idea), it seems like it would be ok, but I guess it depends on how the BIOS function works.
Lastly, as Mike said, you can write your own compression routines too. The BIOS is just normal code put into a 0-waitstate ROM chip, so if you put the same code in IWRAM, it will be just as fast. Not much point wasting your time writing them though, since LZSS is probably the best for GBA-style graphics anyway.
The only thing I decided to write one for was sprites. Since LZSS requires the whole file to be decompressed, it's not too practical for sprites where you only need one frame at a time, so I made a simple RLE compressor that compresses in whatever size chunks you want. If anyone wants it, I'd be happy to release it. The only reason I haven't is because it doesn't get very good compression rates unless you have a lot of blank area around your sprites. My game's battle sprites usually only use about 24x24 out of their 32x32 area, so it works good on them (actually that's what I made it for), but for most things it doesn't really make any difference.
_________________
___________
The best optimization is to do nothing at all.
Therefore a fully optimized program doesn't exist.
-Deku
#16434 - tom - Mon Feb 16, 2004 10:19 am
nah, compression rate is the same, but decompression is slower, which is mainly due to the way lz77/lzss work (references to already decompressed data in the output buffer) and the way the vram can be accessed (not bytewise).
#16443 - Miked0801 - Mon Feb 16, 2004 5:39 pm
We use PUCrunch for most things and an in-house LZ77/RLE byte aligned hybrid for speed critical things (or RLE for things that beat the other 2 in compression.) They both beat the built in stuff for compression (PU) and speed (hybrid). That's why I suggested it.
#16448 - tom - Mon Feb 16, 2004 7:27 pm
jorgen ibsen's aplib and brieflz are neat too. gfx2gba can output aplib compressed data.
the aplib decompressor isn't exactly fast, but the compression ratio is very good.
http://www.ibsensoftware.com
#16545 - headspin - Thu Feb 19, 2004 5:09 am
Thanks for all the replies :)
| DekuTree64 wrote: |
That's what the UnCompLZ77VRAM BIOS function is for. I haven't done any tests myself, but I've heard it's a little slower, and possibly not quite as good of compression rates as the WRAM version (if you're using GBACrusher to compress, be sure to use -LV to work with the VRAM function), but I use it for all sorts of stuff and it works great.
Also, the compressed data does contain a header telling the size and such. |
Never occured to me to try the VRAM function for character data! I'll give it a go. If that works I don't need a buffer and should be great for interchanging tiles from ROM to VRAM.
| DekuTree64 wrote: |
| I'm not sure about writing to the buffer you're decompressing from. Having a general idea of how the LZ77 algorithm works (actually I think the BIOS routines are LZSS, but it's the same idea), it seems like it would be ok, but I guess it depends on how the BIOS function works.. |
A buffer is not something I want either. I'm experimenting with these functions and will get back to you with my results.
| DekuTree64 wrote: |
Lastly, as Mike said, you can write your own compression routines too. The BIOS is just normal code put into a 0-waitstate ROM chip, so if you put the same code in IWRAM, it will be just as fast. Not much point wasting your time writing them though, since LZSS is probably the best for GBA-style graphics anyway.
The only thing I decided to write one for was sprites. Since LZSS requires the whole file to be decompressed, it's not too practical for sprites where you only need one frame at a time, so I made a simple RLE compressor that compresses in whatever size chunks you want. If anyone wants it, I'd be happy to release it. The only reason I haven't is because it doesn't get very good compression rates unless you have a lot of blank area around your sprites. My game's battle sprites usually only use about 24x24 out of their 32x32 area, so it works good on them (actually that's what I made it for), but for most things it doesn't really make any difference. |
Seems a bit silly to have to write your own decompression routine when the GBA has functions to do this in it's BIOS! But it's also annoying that one can't specify the size of the chunks like you say, for stuff like sprites and things like that. I would appreciate if you release your RLE compressor just out of interest sake. At the moment compressing tiles is my main concern, but sprites will be coming up soon!
#16565 - Miked0801 - Thu Feb 19, 2004 7:25 pm
What's more silly is how poorly the Nintendo decompressor perform. They were written with minimum code-size on ROM as their priority, not execution speed. You can easily outperform their routines with your own in IWRAM. This is the other reason to write your own.
Mike
#16570 - headspin - Thu Feb 19, 2004 9:33 pm
Well, as a commercial developer (as I've gathered you are), writing your own decompression algs would be the way to go. Any chance you could slip some of that code my way? Hmmm I smell the sweet essence of NDA. ;)
But seriously, as a home dev'r I like to get my stuff compressed to minimise the size of my ROM and thats it. The speed of decompression is not a problem because I can blank the screen and make someone wait while the tiles/gfx is decompressed to VRAM. I'm not in a situation where quick decompression is necessary. If I wrote games for a living then I know I would. Any chance you could shed some light on your current project? Just out of interest.
Well, I tried UnCompLZS77VRAM and it worked, that was until I tried my tiles where it changed one of the colours to black. The palette was copied ok, looks like the data got corrupted through the decompression some how. I used Marcus's gfx2gba with his built in lz77 compression, so perhaps I need to try GBA Cruncher with the VRAM safe option or something. I'll give it a try tomorrow.
#16574 - DekuTree64 - Thu Feb 19, 2004 10:11 pm
Check my post about sprite compresasion, Mike was kind enough to explain how that hybrid RLE/LZ77 works.
I don't know about gfx2gba's LZ77, I've only used GBA Cruncher.
Also, the BIOS functions are fine for home projects. It's not really worth the trouble of writing your own unless you just want to.
_________________
___________
The best optimization is to do nothing at all.
Therefore a fully optimized program doesn't exist.
-Deku
#16575 - Miked0801 - Thu Feb 19, 2004 10:14 pm
NDAs aren't fun - nor is finding a new job in the current market. I have to be real careful about what I say here - though I do enjoy helping people whenever possible. :)
I'm not sure if my current game has been annouced yet, so I'll be quiet there. Basically, look on the net for Griptonite Games. Any GBA/GBC game that is listed, I've either been lead or done some sort of coding for. I've been at this for quite a while now.
And you guessed it, I can't give code, only advice. The sprite compression thread that is currenty on going is something you should look at though. It's been pretty well spelled out how to do a efficient (size and speed) compressor there.
#16577 - headspin - Thu Feb 19, 2004 10:35 pm
Thanks alot, I really appreciate it :)
Am I alone in saying that I understand code much better than pseudo code? I read C like english, but pseudo like.. um bad english?? hehe
Thanks, I'll take a really good look at what you've got there and convert it to C..
Cheers!
#16599 - tom - Fri Feb 20, 2004 9:28 am
| headspin wrote: |
| I like to get my stuff compressed to minimise the size of my ROM and thats it. The speed of decompression is not a problem because I can blank the screen and make someone wait while the tiles/gfx is decompressed to VRAM. I'm not in a situation where quick decompression is necessary. |
so you want to have a good packer and don't care about decompression speed -> definitely a case for aplib =)
#16614 - headspin - Sat Feb 21, 2004 1:28 am
Checked out aPLib and am very impressed. I'm getting better compression than lz77.
I have to use a buffer again for some reason cos unpacking straight to VRAM just won't work. Even using Marcus' gfx2gba -aps (safe) option, I get a corrupted image.
#16618 - Miked0801 - Sat Feb 21, 2004 2:58 am
You can get the PUCrunch compressor/decompressor off of Jeff's site
www.devrs.org/gba
Very strong compression - beats PKZip on occasion with our files :)
#16622 - tom - Sat Feb 21, 2004 1:33 pm
| headspin wrote: |
| I have to use a buffer again for some reason cos unpacking straight to VRAM just won't work. Even using Marcus' gfx2gba -aps (safe) option, I get a corrupted image. |
the supplied aplib depacker won't work with vram, you'll have to modify it yourself.
the safe option doesn't have to do anything with vram. it adds a header to the compressed data, but i can't remember what's exactly in that header.
#16624 - headspin - Sat Feb 21, 2004 2:28 pm
That would explain things, so I need to modify the depacker code for VRAM. Is that cos you need to output 16 bits at a time to VRAM?
As for Pucrunch, had a look and that depacker code on devrs.com/gba and it's in GB asm, so I'd have to re-write it for GBA and I'm not that good at asm yet.
I'm getting obsessed by compression here! Very addictive stuff! ;)
#16630 - tom - Sat Feb 21, 2004 5:37 pm
| headspin wrote: |
| That would explain things, so I need to modify the depacker code for VRAM. Is that cos you need to output 16 bits at a time to VRAM? |
yes, and also because you can only do halfword/word *reads*. anyway, adjusting the depacker is easy: find every place in the code where the output buffer is accessed (read or write) and make sure it's a 16bit transfer.
#16631 - tepples - Sat Feb 21, 2004 5:47 pm
| tom wrote: |
| yes, and also because you can only do halfword/word *reads*. |
Evidence? Can you whip up a short code sample that proves that 8-bit VRAM reads don't work as one would naively expect?
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#16632 - tom - Sat Feb 21, 2004 6:01 pm
| tom wrote: |
| yes, and also because you can only do halfword/word *reads*. |
oops, of course, that's wrong. you can do 8 bit reads on vram =)
#16639 - Miked0801 - Sat Feb 21, 2004 8:04 pm
Actually, there is a GBA version of the decompressor on his site as well. Just poke around till you find it :)
#17309 - dagamer34 - Sat Mar 06, 2004 3:24 am
headspin, you're tutorial is really good and has helped me out a lot in trying to figure out compression but there is one thing I don't know how to do. How would I go about compressing stuff like sprites and tile data? Because bimbo doesn't convert those file types.
_________________
Little kids and Playstation 2's don't mix. :(
#17315 - headspin - Sat Mar 06, 2004 5:21 am
dagamer34: If you take a look at the new sections for aPLib & PuCrunch you will see a demonstration using 16 colour tiles.
As for sprites, havn't had a look into that yet, but converting tile data I use markus' gfx2gba (there is a link on my website).