gbadev.org forum archive

This is a read-only mirror of the content originally found on forum.gbadev.org (now offline), salvaged from Wayback machine copies. A new forum can be found here.

DS development > GCC problems

#109150 - Cthulhu - Wed Nov 15, 2006 4:52 pm

Don't know where to post it, so if this is the wrong place, please feel free to move or delete this post :)

I have some structures in a binary file that i want to read, the structures are like this:
Code:

#pragma pack(push, 1)
struct MulCell
{
   u16 TileID;
   u8 Z;
};

struct MulBlock
{
   MulCell cells[8][8];
};
#pragma pack


In a win32 test that i did all worked fine and the data was loaded without problems, but trying to read it from the DS gives "corrupted" data, the first (and ONLY the first) cell (cell[0][0]) gets the correct data, but the other cells won't.
The code i use to read:

Code:

   x %= width;
   y %= height;

   u32 blockID = (x / 8) * (height / 8) + (y / 8);
   u32 offset = blockID * 192;

   fseek(fmap, offset, SEEK_SET);
   fread(&block->cells, sizeof(MulCell), 64, fmap);


The compiler throws this warning:

Quote:
warning: #pragma pack(push[, id], <n>) is not supported on this target


Any ideas? :S

#109151 - Mighty Max - Wed Nov 15, 2006 5:02 pm

use "__attribute__(__packed__)" on the structs instead of the pragma

the corruption you observed should be a 1 byte shifting per entry, as the normal alignment will add a 1 byte space between the 3 byte structures
_________________
GBAMP Multiboot

#109152 - Cthulhu - Wed Nov 15, 2006 5:13 pm

I tried that and it gives the same "corrupted" value
Win32 right value = 174
NDS value = 17636

The code:
Code:
struct MulCell
{
   u16 TileID;
   u8 Z;
}__attribute__((__packed__));

struct MulBlock
{
   MulCell cells[8][8];
}__attribute__((__packed__));

#109153 - Mighty Max - Wed Nov 15, 2006 5:27 pm

Add a packed to the array too. the aray is a structure that might exist packed and unpacked too.

If i understand it correc, packed on the struct can only remove the alignment fillers at the structures level, not from substructures as the array.
_________________
GBAMP Multiboot

#109161 - Cthulhu - Wed Nov 15, 2006 6:12 pm

I've found that this only works if the two variables are of the same size, so i had to change it to:
Code:

struct MulCell
{
   u16 TileID;
   u16 Z;
}__attribute__((__packed__));


And change the file format to fit this... increasing its file size. Don't like this solution at all... anyone knows a more elegant way to do this?

#109163 - Mighty Max - Wed Nov 15, 2006 6:29 pm

Yes, undo the change (that works only because you now fit axactly the alignment) and do 64 single reads into that struct of sizeof(MulCell) = 3.
_________________
GBAMP Multiboot

#109165 - tepples - Wed Nov 15, 2006 6:53 pm

Cthulhu wrote:
I have some structures in a binary file that i want to read, the structures are like this:
Code:

#pragma pack(push, 1)
struct MulCell
{
   u16 TileID;
   u8 Z;
};

struct MulBlock
{
   MulCell cells[8][8];
};
#pragma pack

The most portable way to deserialize a struct is to read it a byte at a time into RAM.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#109290 - Cthulhu - Thu Nov 16, 2006 11:03 pm

Mighty Max wrote:
Yes, undo the change (that works only because you now fit axactly the alignment) and do 64 single reads into that struct of sizeof(MulCell) = 3.


Its the same... the data gets all corrupted or shifted or whatever... and don't like at all the "solution" of making all the variables of the same type.

Quote:
The most portable way to deserialize a struct is to read it a byte at a time into RAM.


You mean this?

Code:

   fseek(fmap, offset, SEEK_SET);
   //fread(&block->cells, sizeof(MulCell), 64, fmap);
   for(int i = 0; i < 8; i++)
   {
      for(int j = 0; j < 8; j++)
      {
         fread(&block->cells[i][j].TileID, sizeof(u16), 1, fmap);
         fread(&block->cells[i][j].Z, sizeof(s8), 1, fmap);
      }
   }


Its exactly the same, it reads wrong data... any ideas? :(

(In Win32 reads the data fine... it must be some DS or libfat thing :S)

#109291 - Mighty Max - Thu Nov 16, 2006 11:34 pm

That should work just fine, regardless of the packed settings. libfat should be alignment secure too (only the very early versions had a problem here).

Are you sure you have opened them on both systems as binary?
_________________
GBAMP Multiboot

#109294 - Cthulhu - Thu Nov 16, 2006 11:50 pm

Yes, in both systems the files are opened in binary, i double checked this.

I downloaded the libfat form the first post here. I really dont know what to do, because change the type of the variables to fit "well" isn't really an option (maybe for one file, but not for the rest).

#109298 - DekuTree64 - Fri Nov 17, 2006 1:11 am

You could try explicitly splitting the u16 member into two u8s, and combine them manually when reading:
Code:
struct MulCell
{
   u8 TileID_lo;
   u8 TileID_hi;
   u8 Z;
};

struct MulBlock
{
   MulCell cells[8][8];
};

void Foo()
{
    MulCell *cell = something;
    u32 tileID = cell->TileID_lo | (cell->TileID_hi << 8);

    // Do stuff with tileID
}

A bit uglier, but no compiler funkiness to worry about. That's basically what the compiler would do with a packed u16 anyway.
_________________
___________
The best optimization is to do nothing at all.
Therefore a fully optimized program doesn't exist.
-Deku

#109317 - tepples - Fri Nov 17, 2006 4:44 am

Cthulhu wrote:
Yes, in both systems the files are opened in binary, i double checked this.

"Binary" varies between platforms. The big differences are endianness and alignment. Safest is not to use fwrite() on one platform and fread() on another unless you're reading something that has been formatted as an array of u8 or s8.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#109352 - Cthulhu - Fri Nov 17, 2006 3:14 pm

I tried to go with fprintf & fscanf, fwrite & fread, and nothing... even if i read byte per byte only the first struct gets readed fine, the others get incorrect data, here is my code for write the file in win32, i read another file and write the new one without the header information, the only info i need is the mulblocks.

Code:
   FILE * nuevo;
   fopen_s(&nuevo,"map2_n.mul","wb");

   while(!feof(fmap))
   {
      MulBlock block;

      unsigned int header;
      int h = fread(&header, sizeof(unsigned int),1,fmap);
      int h1 = fread(&block.cells, sizeof(MulCell), 64, fmap);
      int h2 = fwrite(&block.cells,sizeof(MulCell), 64, nuevo);
   }
   fclose(nuevo);



And here is the code for reading in the DS:

Code:

   x %= width;
   y %= height;

   u32 blockID = (x / 8) * (height / 8) + (y / 8);
   u32 offset = blockID * 192;

   fseek(fmap, offset, SEEK_SET);
   int h = fread(&block->cells, sizeof(MulCell), 64, fmap);


So... what can i do?

#109354 - KoshNi - Fri Nov 17, 2006 3:30 pm

fae ujlagf nyarlathotep jgjklg daukjg !!
adjklhad yoggsototh f kl lh iophe guik a!
sgiz !! sgiz !!


[I'm sorry. I couldn't resist. :) ]
_________________
Sunshine. Beauty. Love. Happiness.

#109358 - OOPMan - Fri Nov 17, 2006 4:18 pm

Shouldn't that be...

Ia! Ia! Cthulhu Ftaghn!
_________________
"My boot, your face..." - Attributed to OOPMan, Emperor of Eroticon VI

You can find my NDS homebrew projects here...

#109413 - tepples - Sat Nov 18, 2006 1:58 am

Cthulhu wrote:
I tried to go with fprintf & fscanf, fwrite & fread, and nothing... even if i read byte per byte only the first struct gets readed fine, the others get incorrect data

Have you looked at the file that your win32 code generates in a hex editor? And have you made a simple hex viewer for the DS? The learning experience from those should teach you about file handling.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#109578 - Cthulhu - Sun Nov 19, 2006 12:20 pm

tepples wrote:
Have you looked at the file that your win32 code generates in a hex editor?


Yes, i've looked at the original and the new file and all the data is correct (at least for what i understood from the struct viewer in Hex Workshop), i even tried to read the original file from the DS and it's the same, the first 64 (8x8 MulCell = 1 Block) structs are readed fine, then all the data readed is wrong. I'll keep looking at the files in a hex editor, maybe i missed something...
By the way, as i dont have the ds right now to test things on it (girlfriend have it until thursday) i use libfat-20060709.tar.bz2 because its the one that works with Dualis without too much trouble, is there some other emu that supports fat?