#35670 - headspin - Wed Feb 09, 2005 8:40 am
I've been looking around the net and there are so many different types of lz77 out there, I'm looking for a struct of the lz77 header so I can get the size of the uncompressed data.
_________________
Warhawk DS | Manic Miner: The Lost Levels | The Detective Game
#35673 - DekuTree64 - Wed Feb 09, 2005 9:33 am
You mean the LZ (SS, I think) that the BIOS decompression function uses? I think GBATEK has some info on it, in the section for the decompression SWI.
_________________
___________
The best optimization is to do nothing at all.
Therefore a fully optimized program doesn't exist.
-Deku
#35702 - headspin - Thu Feb 10, 2005 12:27 am
Hmm, couldn't find anything about the header though. ALthough gbatek do call it LZ77, but from memory, I do remember LZSS being the correct term. Perhaps it's a variation of the LZ77 so maybe thats why they call it that.
_________________
Warhawk DS | Manic Miner: The Lost Levels | The Detective Game
#35705 - DekuTree64 - Thu Feb 10, 2005 12:49 am
Hmm, it does call it LZ77. Oh well, here's the header info I was talking about:
| Code: |
r0 Source address, pointing to data as such:
Data header (32bit)
Bit 0-3 Reserved
Bit 4-7 Compressed type (must be 1 for LZ77)
Bit 8-31 Size of decompressed data
Repeat below. Each Flag Byte followed by eight Blocks.
Flag data (8bit)
Bit 0-7 Type Flags for next 8 Blocks, MSB first
Block Type 0 - Uncompressed - Copy 1 Byte from Source to Dest
Bit 0-7 One data byte to be copied to dest
Block Type 1 - Compressed - Copy N+3 Bytes from Dest-Disp-1 to Dest
Bit 0-3 Disp MSBs
Bit 4-7 Number of bytes to copy (minus 3)
Bit 8-15 Disp LSBs
r1 Destination address |
Looks like what you're after is the upper 24 bits of the first word.
_________________
___________
The best optimization is to do nothing at all.
Therefore a fully optimized program doesn't exist.
-Deku
#35815 - headspin - Fri Feb 11, 2005 2:01 am
Is there a relative easy way to modify my LZ77UnCompVRAM function to return the size of the decompressed data?
| Code: |
inline void LZ77UnCompVRAM(u32 source, u32 dest) {
asm("mov r0, %0\n"
"mov r1, %1\n"
"swi 0x12\n"
:
:"r" (source), "r" (dest)
:"r0", "r1" );
} |
_________________
Warhawk DS | Manic Miner: The Lost Levels | The Detective Game
#35830 - tepples - Fri Feb 11, 2005 6:10 am
Push R0 on the stack, call the SWI, pull R0, and then look in the header for the decompressed data size. Put that in R0, and it'll be the return value.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#35838 - headspin - Fri Feb 11, 2005 8:39 am
Because r0 points directly to the header can't I just return the 32 bit header shifted right 8 bits to isolate the size of the uncompressed data?
Would the following achieve that?
| Code: |
mov r0, %0 // put first parameter into r0
mov r1, %1 // put second parameter into r1
swi 0x12 // call BIOS function 0x12
ldr r2, [r0] // place the header into r2
mov r0, r2, LSR#8 // right shift r2 8 bits to return the size of uncompressed data |
_________________
Warhawk DS | Manic Miner: The Lost Levels | The Detective Game
#35860 - tepples - Fri Feb 11, 2005 4:46 pm
| headspin wrote: |
| Because r0 points directly to the header |
It does before you call the SWI but not after. Calling an SWI trashes r0-r3 and r12. Therefore, you'll need to push r0, call the SWI, pull r0, and then read the header.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#35901 - headspin - Sat Feb 12, 2005 4:32 am
Ahhh, I see, thanks for that tepples - always appreciate your advice!
_________________
Warhawk DS | Manic Miner: The Lost Levels | The Detective Game
#36447 - headspin - Sun Feb 20, 2005 5:36 pm
Out of interest, here is the modified function to return the size of the uncompressed data
| Code: |
// UnCompress LZ77 Data to WRAM
u32 LZ77UnCompWRAM(u32 source, u32 dest) {
u32 size;
asm volatile (
"mov r0, %1 \n" // put first parameter into r0
"mov r1, %2 \n" // put second parameter into r1
"push {r0} \n" // push r0 onto the stack (pointer to header)
"swi 0x11 \n" // call the BIOS function (r0 gets trashed)
"pop {r0} \n" // pull r0 off the stack
"ldr r0, [r0]\n" // load the value r0 (header) into r0
"lsr r0, #8 \n" // right shift 8 bits
"mov %0, r0 \n" // move r0 into size (to return)
:"=r" (size)
:"r" (source), "r" (dest)
:"r0", "r1", "r2" );
return size;
} |
_________________
Warhawk DS | Manic Miner: The Lost Levels | The Detective Game
#36560 - mentz - Thu Feb 24, 2005 1:56 pm
There is a method to know the size of the compressed data ??
(the header stores the type of compression and the lenght of decompressed data)
This compression got an "end" byte or the decompression stops when is reached the size of decompressed data ?
_________________
xXx
#36561 - tepples - Thu Feb 24, 2005 4:59 pm
| mentz wrote: |
| There is a method to know the size of the compressed data ?? |
How are you storing compressed data in your program? Usually, such as with my tools, either the binary-to-object converter makes a variable called something like tiles_chr_size, or the data appending tool will return a size along with the address of the data.
| Quote: |
| This compression got an "end" byte or the decompression stops when is reached the size of decompressed data ? |
I'm pretty sure that the latter is true, that the BIOS uses the decompressed size in the header as a bytes_left variable when writing out decompressed data.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#36585 - headspin - Fri Feb 25, 2005 12:48 pm
| mentz wrote: |
| There is a method to know the size of the compressed data ?? |
I use tepples gbfs so gbfs_get_obj() will return the size of the compressed data. I can't imagine why you would need this value since the bios function will take care of the decompression, the only question left is 'what is the size of my graphic data so I can DMA it to charbase?'
Although if you use GBA Crusher's LZ77 'VRAM safe' mode you can decompress directly to VRAM. gfx2gba013 won't compress in safe mode so if you use that you would have to decompress to a buffer in EWRAM before you DMA it back into VRAM. Although I don't know which method is quicker...
_________________
Warhawk DS | Manic Miner: The Lost Levels | The Detective Game
#36712 - Cearn - Mon Feb 28, 2005 2:39 pm
| headspin wrote: |
Although I don't know which method is quicker...
|
Ran a test for the speed of LZ decompression a few days ago because I had been wondering about this too. Using a random image (original 30720 bytes, compressed to 3924), no$gba (sorry no hardware test, but no$gba has given me reliable readings in the past so these should be worth something) gives me
| Code: |
LZ77UncompVram 757k cycles
LZ77UncompWram (EWRAM) 459k cycles
+= DMA to VRAM 521k cycles |
For reference, a normal u16 loop-copy of 30720 bytes would be about 445k. So it looks like the WRAM+DMA method is a good deal faster.
#36916 - mentz - Thu Mar 03, 2005 4:26 pm
| headspin wrote: |
| I can't imagine why you would need this value since the bios function will take care of the decompression |
I would like to extract compressed part from gba roms...For translation aim...
_________________
xXx
#36918 - tepples - Thu Mar 03, 2005 4:37 pm
A GBA-LZSS compressed object will never be more than 9/8 the size of the decompressed version plus 5 bytes. You could extract more than that (or the whole ROM), then write a custom PC-side decompressor that returns the source address at the end of decompression, giving the address of the end of the object. Subtract the start address, and you have your size.
However, further detailed discussion of hacking proprietary ROMs is off-topic here.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#36965 - mentz - Fri Mar 04, 2005 10:07 am
Thank you...
(Sorry for Little OT)
_________________
xXx
#37267 - batblaster - Wed Mar 09, 2005 7:03 pm
Hi,
all the info iread are really good but anyone is sure the data compressed are stored in asequential mode ??? i have the same problem on my engine i have a very large map and i want to unpack only the small part i need the only problemi don't know where i can point in the packed data anyone have an idea ??? i'm not sure the compressed data can be unpacked only in a part...
If someone know different way tell me please...
Thanks...
_________________
Batblaster / 7 Raven Studios Co. Ltd
------------------------------------------
#37271 - Miked0801 - Wed Mar 09, 2005 7:43 pm
Huffman compression (and arithmetic if you like slow) will allow you to do this. In Huffman, each byte of data in converted to a bit-code. As such, you can count your way through data to get to where you want (slow!), or store an offset table to key areas of data (we use this for strings.) Not the best compression, but fast to decompress and some is better than none!