gbadev.org forum archive

This is a read-only mirror of the content originally found on forum.gbadev.org (now offline), salvaged from Wayback machine copies. A new forum can be found here.

DS development > libfat / fatlib and FCSR

#97803 - josath - Fri Aug 11, 2006 5:29 pm

This is mainly for chishm, but perhaps someone else with experience knows some answers.

Hello, I've been doing some experimentation with FCSR, so I can create a FAT image builder (crossplatform), that way people can use the same code for both media-based carts, and flash carts.

When using the old gba_nds_fat (chishm's latest, with sata's drivers, plus wintermute's bugfixes), FCSR seems to work as expected.

When using libfat, it seems very buggy: (note, in order to test FCSR on my SC CF, I had to disable the other flashcard drivers. I did this by removing everything except &_io_fcsr from ioInterfaces[] in disc.c. Is this the right way to do it?)
1. fatInitDefault() works, returning 1
2. fopen() works on a file which exists in the FAT image, and fails on a file that doesn't exist (as expected)
3. fread() always reads only 0 bytes, when file opened successfully.
4. the 4th time I call fopen (of course I called fclose after the other ones), it locks up (doesn't return to my code)
I tried different sizes of fat images, fat 12 and fat 16, everything i could think of. it always failed.

Back to gba_nds_fat: It seems to work great, including the SRAM overlay. My only problem, is how do I know which sectors to set to the overlay? If I make a 64KB FAT image, then it's easy, I simply tell it to overlay the whole thing. Generally, what I want to do, is have a FAT image with 64KB of free space, and place the overlay there, and force all writes to go into that area only. If writes go into some other area which is not in the overlay, the writes work, and I can read it back again (I'm assuming due to cache), but it does not persist over powercycles.

*EDIT*: See my other post where I compiled dsdoom with a FAT image

#97890 - chishm - Sat Aug 12, 2006 4:37 am

The overlays are a bit complicated to get working manually.

You'll need to fill the FAT with 0x0FFFFFF7 (bad sector) masked off to the appropriate number of bits for the type of FAT (FAT12 uses 12 bits, etc). Then find out the number of bytes per cluster.

Divide the SRAM size (64KiB) by this, then subtract at least 2 (one cluster for the dir entries, one or more sectors for the FAT). Choose a place in the FAT that is filled with bad sector markers and change them to 0x00000000, based on the number of sectors available. Write down the start sector and length of the FAT where you did this. This is the first overlay, with the size being the number of sectors filled with 0.

Next, fill the root directory (or a sub directory where you want to save) with 0x41 as the first byte and 0x0F as the attribute every 32 bytes in the empty entries, until it is aligned to a sector. The next sector within this directory will be the start of the 2nd SRAM overlay, and the number of sectors will be enough to fill it to a cluster aligned sector (or for the minimum number of entries you want, but this is where things get really complicated).

The 3rd (and final) SRAM overlay is used for the data. The start is calculated as the first free cluster (as chosen in the FAT step) multiplied by sectors per cluster and added to the data start sector. The size is simply sectors per cluster multiplied by the number of writable clusters.


I'm not sure what is causing it to fail libfat. My usual advice is upgrade to DKP r19b. If it still fails, check errno for the reason it failed.
_________________
http://chishm.drunkencoders.com
http://dldi.drunkencoders.com

#97897 - josath - Sat Aug 12, 2006 7:16 am

chishm wrote:
The overlays are a bit complicated to get working manually.

You'll need to fill the FAT with 0x0FFFFFF7 (bad sector) masked off to the appropriate number of bits for the type of FAT (FAT12 uses 12 bits, etc). Then find out the number of bytes per cluster.


I can handle that...when copying stuff into an empty FAT image, it puts all my data at the front, and the rest of the image is zeroed out. I can just fill the empty space with the bad sector marker.

Quote:

Divide the SRAM size (64KiB) by this, then subtract at least 2 (one cluster for the dir entries, one or more sectors for the FAT). Choose a place in the FAT that is filled with bad sector markers and change them to 0x00000000, based on the number of sectors available. Write down the start sector and length of the FAT where you did this. This is the first overlay, with the size being the number of sectors filled with 0.

Ok, I think I can do that out too

Quote:

Next, fill the root directory (or a sub directory where you want to save) with 0x41 as the first byte and 0x0F as the attribute every 32 bytes in the empty entries, until it is aligned to a sector. The next sector within this directory will be the start of the 2nd SRAM overlay, and the number of sectors will be enough to fill it to a cluster aligned sector (or for the minimum number of entries you want, but this is where things get really complicated).

Here's where I have trouble...I'm not sure how to find the root directory, or sub dir, in the image, in order to fill it. I don't really know anything about FAT itself

Quote:

The 3rd (and final) SRAM overlay is used for the data. The start is calculated as the first free cluster (as chosen in the FAT step) multiplied by sectors per cluster and added to the data start sector. The size is simply sectors per cluster multiplied by the number of writable clusters.

I'm not clear what this overlay is for. The first (large) one is for the file data itself, and 2nd one is for directory entries, right? what else is there?

Quote:

I'm not sure what is causing it to fail libfat. My usual advice is upgrade to DKP r19b. If it still fails, check errno for the reason it failed.

ok, i'll poke around with it.

But as far as SRAM overlays, they seem a lot more involved than I had hoped. I think I might look into some other way of doing it, that doesn't involve me having to learn how FAT works. That would involve me poking into the lib itself. Perhaps some sort of 'dynamic' overlay, where I let you modify ANY sector, and I just keep track of which sectors are modified, and store that table into SRAM, along with the modified sectors.

#97899 - chishm - Sat Aug 12, 2006 8:01 am

As I said, it is fairly complicated to create the overlays manually. The first overlay is not within the data region of the partition, but in the FAT -- the File Allocation Table. This is a table where the index is the current cluster and the value is the next cluster within a file's cluster chain. So by filling the FAT with the bad cluster value (I accidentally called this bad sector before), you are saying that there are no more usable clusters, so it shouldn't try writing to them.

To get the offsets and value required (data area start, sectors per cluster), you'll need some sort of disk image editting tool. Alternatively, you can look at the raw disk image and interpret the values in the BSB (Boot Sector Block), but that requires learning about FAT.

Keeping track of written sectors, rather than using an overlay may work. However, it would need a lot more comparisons to work. 64KiB of SRAM will give 126 writable sectors (1KiB reserved for a header), so each time a sector is read or written it would need to be checked against 126 possible previously written sectors. I suppose I could do this and write an FCS2 driver. That won't be for a while though.
_________________
http://chishm.drunkencoders.com
http://dldi.drunkencoders.com

#97981 - josath - Sat Aug 12, 2006 5:45 pm

chishm wrote:
To get the offsets and value required (data area start, sectors per cluster), you'll need some sort of disk image editting tool.

Is there any tool in particular you recommend?

#98074 - chishm - Sun Aug 13, 2006 2:14 am

josath wrote:
Is there any tool in particular you recommend?

The evaluation version of WinHex can open binary images, then you can apply a template to get it to interpret the data as though it were the BSB of a disc. You won't be able to save large files, but it should do for your purposes (information extaction).
_________________
http://chishm.drunkencoders.com
http://dldi.drunkencoders.com

#109008 - GPFerror - Tue Nov 14, 2006 12:14 am

using davr's(josath) source and shell scripts for linux FCSR image creation, I have created a windows commandline version using his code and someothers I found on the internet.

sram overlay doesn't work in my version either, as I am not familiar with FAT layout either.

This is from the readme.

Code:
Windows(command prompt) version is by Troy Davis(GPF) http://gpf.dcemu.co.uk
uses mkdosfs.exe(edited to build from commandline) source included, original source from http://www.frontiernet.net/~fys/mtools.htm
and Bfi 1.1 (Build floppy image) by Bart Lagerweij (patched by Erwin Veermans)

usage: build.bat name.img dirname

should work with directories from 10000 bytes+ to the limits of the fat12 filesystem



download fcsrimage.zip from my left nav bar on my site under utilities.
http://gpf.dcemu.co.uk

Troy(GPF)

#109010 - josath - Tue Nov 14, 2006 12:31 am

I'd just like to voice my request for either a 'FCS2' driver, or for chishm himself to release a tool which will create the necessary SRAM overlay info in a FAT image (now that we have working fat image builders for both windows and linux)

#109012 - Lazy1 - Tue Nov 14, 2006 1:04 am

You could try using romdiskfs instead, I found it to be faster and with some changes I made it can be used with the standard C I/O functions like fopen, fread, ect.
Images are also easy to make, genromfs does a nice job.

It doesn't support sram, but I'm sure a filesystem could be designed to fit in such a small amount of space.

#109013 - josath - Tue Nov 14, 2006 1:11 am

Lazy1 wrote:
You could try using romdiskfs instead, I found it to be faster and with some changes I made it can be used with the standard C I/O functions like fopen, fread, ect.
Images are also easy to make, genromfs does a nice job.

It doesn't support sram, but I'm sure a filesystem could be designed to fit in such a small amount of space.


Yes, I saw that, and it comes very close to what I want to do, which is this: Have the same exact code run on libfat-supported devices, and non-supported devices (with GBAROM space).

This could probably be done pretty easily with romfs, with a simple wrapper, like
Code:
FSInit() {
  if(!fatInitDefault()) {
     romfsInit();
  }
}


Though I haven't tried that yet.

The main part which would take more work is the writable FS, in order to keep it compatible with libfat (so I don't need to duplicate code every time i want to write, like if(libfat) fwrite() else romfswrite(), and so on)

#109016 - Lazy1 - Tue Nov 14, 2006 1:30 am

What I'm doing is this: first check if romfs initializes, if it does then stop looking otherwise try to initialize libfat.

That's as far as it goes, since after you call romfsInitDefault all the I/O functions are the same.

#129785 - knight0fdragon - Sun May 27, 2007 5:39 am

is it possible to get a script to place into a makefile? I do not know too much about makefiles, but I do know I cant get either script to work lol (in the makefile)
_________________
http://www.myspace.com/knight0fdragonds

MK DS FC: Dragon 330772 075464
AC WW FC: Anthony SamsClub 1933-3433-9458
MPFH: Dragon 0215 4231 1206