#170304 - sverx - Wed Sep 16, 2009 2:46 pm
Hi!
is there anybody who's using libz (zlib port to NDS) uncompress() function? I'm experiencing strange problems with it... (the decompression seems to work ok but the function gives error instead...)
I'm using version 1.2.3, which I guess is the latest.
Thanks!
#170310 - kusma - Wed Sep 16, 2009 3:34 pm
What's wrong with using plain old zlib and it's deflate()-function?
#170311 - sverx - Wed Sep 16, 2009 3:42 pm
I was searching the easier way to decompress a small amount of data (some KB) that it's already in memory, and I've read this about uncompress() function.
Do you suggest using another way? I'm a complete newbie on that...
#170313 - kusma - Wed Sep 16, 2009 4:07 pm
Nah, it should probably be OK. But my guess would be that some of the parameters are wrong - what error do you get?
#170316 - FluBBa - Wed Sep 16, 2009 5:00 pm
I used "puff" (as I also wanted to decompress from RAM to RAM) which is some kind of school example on how zlib compression works.
_________________
I probably suck, my not is a programmer.
#170337 - sverx - Thu Sep 17, 2009 9:01 am
kusma: I'm getting Z_DATA_ERROR, which should mean that my input stream (the compressed data) is 'bad'. But now I'm thinking it's a CRC problem, going to run some tests today...
About 'puff'... well, I didn't know that. But it says it's slower... thanks fluBBa for the advice anyway :)
#170339 - sverx - Thu Sep 17, 2009 11:56 am
well, it seems to me that the problem is that the uncompress() functions uses inflate() functions that compute a so called 'adler32' crc but my data ends with a 'normal' crc32.
I think I'm going to modify the library source... or is that something important I am missing? I've got a strange feeling...
#170341 - sverx - Thu Sep 17, 2009 2:17 pm
Ok, I'm really lacking ideas. Maybe I can borrow someone else ideas?
I've got a .gz file already loaded in RAM (I don't want to use filesystems for this project) and I want to uncompress the file that there's in it.
zlib gives functions to access a .gz file, but on filesystems.
zlib gives functions to uncompress from memory to memory, but it doesn't work with my .gz file as it is. But if I remove the .gz header (the first 10 bytes) and I add a custom header (two bytes) then I can uncompress the file that there's in it: it's correct and it's also fast. But the function complains that it's failing (I guess it's because of the different type of CRC at the end of the data)
Any help?
#170353 - sverx - Fri Sep 18, 2009 9:26 am
Ok, it turns out that maybe there's a bug somewhere in the zlib code... or in the port maybe?...
I've read the sources and I understood uncompress() should be able to work on a gz file too: it uses inflate() which checks for gz headers... but it's failing at this. So I changed one line (in inflate.c) from
| Code: |
| if ((state->wrap & 2) && hold == 0x8b1f) { /* gzip header */ |
to
| Code: |
| if (hold == 0x8b1f) { /* gzip header */ |
and now it works correctly. Decodes the .gz file perfectly and returns no error. BTW I stil don't know why it works now and what was the meaning of that code I removed.
So I'm leaving it here, maybe somebody has a comment to add...
#170369 - headspin - Sat Sep 19, 2009 7:07 pm
I think there must be a problem with this lib on the DS or there is something else were missing. Your right though changing that line and it works.
_________________
Warhawk DS | Manic Miner: The Lost Levels | The Detective Game
#170418 - wintermute - Tue Sep 22, 2009 10:12 am
Are you sure you're not missing something here?
That sounds rather like you're not preparing the compressed data in a supported manner.
_________________
devkitPro - professional toolchains at amateur prices
devkitPro IRC support
Personal Blog
#170419 - sverx - Tue Sep 22, 2009 11:14 am
| wintermute wrote: |
| Are you sure you're not missing something here? |
No, I'm not sure unfortunatly.
| wintermute wrote: |
| That sounds rather like you're not preparing the compressed data in a supported manner. |
zlib documentation also says, about uncompress():
| Quote: |
| This function can be used to decompress a whole file at once if the input file is mmap'ed. |
and I think this means something like "if you've got a gz file somewhere in your memory you can use this function to decompress it".
Of course maybe I'm not following the correct procedure... in fact for instance I'm not calling gzopen() or gzread() since I'm not using a filesystem... maybe inflate() was expecting me to do that before using it for decompressing a gz file?
I don't know. What I do know is that in inflate() function source there's the code needed for decoding all the gz headers... but it won't be executed if you don't remove that
check.
Finally, I'm also sure that the gz file I prepare (using -nv9 command line options) it's correct: I checked his header with an hex editor comparing it to what's expected (RFC 1952) and, most important, I get it decompressed perfectly after all!
No more ideas, personally.
#170421 - SteveH - Tue Sep 22, 2009 11:48 am
| sverx wrote: |
| wintermute wrote: | | Are you sure you're not missing something here? |
Of course maybe I'm not following the correct procedure... in fact for instance I'm not calling gzopen() or gzread() since I'm not using a filesystem... maybe inflate() was expecting me to do that before using it for decompressing a gz file?
I don't know. What I do know is that in inflate() function source there's the code needed for decoding all the gz headers... but it won't be executed if you don't remove that
check.
Finally, I'm also sure that the gz file I prepare (using -nv9 command line options) it's correct: I checked his header with an hex editor comparing it to what's expected (RFC 1952) and, most important, I get it decompressed perfectly after all!
No more ideas, personally. |
I know that in the zip library I used on a project on windows you had to call the zipOpen() functions even on a memory mapped file as it setup some initial pointers in a handler with offsets to the zip file list, etc. Try going though the correct process from open to close, if your still experiencing issues then something is wrong with your zip file or your zlib implementation.
#170422 - sverx - Tue Sep 22, 2009 12:04 pm
| SteveH wrote: |
| Try going though the correct process from open to close, if your still experiencing issues then something is wrong with your zip file or your zlib implementation. |
How can I open my gz file which is already in memory? zlib has got gzopen() function, which wants a path to the file (so I think this requires a filesystem) and gzdopen() which requires a file descriptor, that I do not have of course. Is there some function I'm missing?
Thanks!