gbadev.org forum archive

Hi everybody,
I have spent hours on this, but I think the effort has been worth it. After countless attempts I have finally successfully worked out how to use the GBAMP to read CF cards from within our own programs. Here is the code. Please note, it needs libgba (or some other gba defines) to function correctly. The driver has been updated to work on the NDS without modification.

EDIT: The driver has been released. It is available here.

libCF.h

Code:

/*
libCF.h
By chishm (Michael Chisholm)

Routines for reading a compact flash card
using the GBA Movie Player
*/

#include "gba_types.h"
//---------------------------------------------------------------------------------
#ifdef __cplusplus
extern "C" {
#endif
//---------------------------------------------------------------------------------

/*-----------------------------------------------------------------
CF_IsInserted
Returns true if a CF card is inserted
-----------------------------------------------------------------*/
bool CF_IsInserted ();

/*-----------------------------------------------------------------
CF_ReadSector
Read 512 byte sector numbered "sector" into "buffer"
sector: IN u32: address of 512 byte sector on CF card to read
buffer: IN pointer: pointer to 512 byte buffer to store data in
-----------------------------------------------------------------*/
void CF_ReadSector (u32 sector, void* buffer);

//---------------------------------------------------------------------------------
#ifdef __cplusplus
} // extern "C"
#endif
//---------------------------------------------------------------------------------

Last edited by chishm on Tue Jul 26, 2005 10:00 am; edited 7 times in total

libCF.c

Code:

/*
libCF.c
By chishm (Michael Chisholm)

Routines for reading a compact flash card
using the GBA Movie Player
*/

#include "libCF.h"

#include "gba_types.h"
#include "gba_dma.h"

//---------------------------------------------------------------
// Addresses
#define CF_REG_STS    *(vu16*)(0x098C0000) // Status of the CF Card
#define CF_REG_CNT1    *(vu16*)(0x090C0000) // Commands sent to control chip
#define CF_REG_CNT2    *(vu16*)(0x090E0000) // Commands sent to control chip
#define CF_REG_CNT3    *(vu16*)(0x09040000) // Commands sent to control chip

#define CF_REG_ADD1    *(vu16*)(0x09060000) // Low byte of sector address
#define CF_REG_ADD2    *(vu16*)(0x09080000) // 2nd lowest byte of sector address
#define CF_REG_ADD3    *(vu16*)(0x090A0000) // 3rd lowest byte of sector address

#define CF_DATA       (void*)0x09000000    // Pointer to buffer of CF data transered from card

// Card status
#define CF_STS_INSERTED    0x50
#define CF_STS_REMOVED    0x00
#define CF_STS_READY    0x58

// Card commands
#define CF_CMD_1       0xA0
#define CF_CMD_2       0x0E
#define CF_CMD_3       0x0A
#define CF_CMD_4       0x01
#define CF_CMD_5       0xE0
#define CF_CMD_6       0x20

//-----------------------------------------------------------------

/*-----------------------------------------------------------------
CF_IsInserted
Returns true if a CF card is inserted
-----------------------------------------------------------------*/
bool CF_IsInserted ()
{
// Change register, then check if value did change
CF_REG_STS = CF_STS_INSERTED;
return (CF_REG_STS == CF_STS_INSERTED);
}

/*-----------------------------------------------------------------
CF_ReadSector
Read 512 byte sector numbered "sector" into "buffer"
sector: IN u32: address of 512 byte sector on CF card to read
buffer: IN pointer: pointer to 512 byte buffer to store data in
-----------------------------------------------------------------*/
void CF_ReadSector (u32 sector, void* buffer)
{
int i;

CF_REG_CNT1 = CF_CMD_1;
CF_REG_STS = CF_CMD_2;
for (i = 0; i < 2400; i++);

CF_REG_STS = CF_CMD_3;
for (i = 0; i < 2400; i++);

CF_REG_CNT1 = CF_CMD_1;

CF_REG_CNT3 = CF_CMD_4;

CF_REG_ADD1 = sector & 0xFF;       // 1st byte of sector number
CF_REG_ADD2 = (sector >> 8) & 0xFF;    // 2nd byte of sector number
CF_REG_ADD3 = (sector >> 16) & 0xFF; // 3rd byte of sector number

CF_REG_CNT1 = CF_CMD_5;

CF_REG_CNT2 = CF_CMD_6;

// Wait until card is ready for reading
while (CF_REG_STS != CF_STS_READY)
{
   for (i = 0; i < 2400; i++)
   {
   }
}

DMA3COPY( CF_DATA, buffer, 128 | DMA32 | DMA_ENABLE);
}

Provided you can get to "read sector", here are some useful links:

_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

I was wondering, chishm, how did you test and run your code? Did you program the Flash Rom on the device directly or were you able to update the firmware with your code from the CP Flash card? I guess I may need to do some more reading.

vorpalarrow wrote:

I was wondering, chishm, how did you test and run your code? Did you program the Flash Rom on the device directly or were you able to update the firmware with your code from the CP Flash card? I guess I may need to do some more reading.

Actually the GBA Movie Player has the ability to load multiboot ROMs directly from a compact flash card. I just compile into a multiboot ROM and load it that way.

Quick Update:
I have the following functions working now:

Code:

bool CF_IsInserted ();
void CF_ReadSector (u32 sector, void* buffer);
u32 CF_LastSector ();
bool FAT_InitFiles ();
DIR_ENT FAT_GetDirEntry ( u32 dirCluster, int entry);
bool FAT_GetFilename (DIR_ENT dirEntry, char* filename);
DIR_ENT FAT_DirEntFromPath (char* path);
bool FAT_FindNextFile(char* filename);
bool FAT_FindFirstFile(char* filename);
bool FAT_CWD (char* path);

//-----------------------------------------------------------------
// File functions
FAT_FILE* FAT_fopen(char* filename);
void FAT_fclose (FAT_FILE* file);
u32 FAT_ftell (FAT_FILE* file);
bool FAT_fseek(FAT_FILE* file, u32 position);

Unfortunately, the most important function, FAT_fread, is not working yet. I am trying to make it work for any read length, but it is broken. Is there any particular reason why I shouldn't make it align to 512 byte boundaries (taking care of end of file, of course)? This would increase speed and be easier to implement, but is not as good.

Any comments, suggestions or help is appreciated. I'd post code, but its a little messy and difficult to follow at the moment.

If you want to have a byte-stream interface, which is more familiar to most developers, you'll just need to keep a 512-byte buffer inside the FAT_FILE struct and make partial reads from that whenever a read ends in the middle of a sector. The speed problem could be fixed with 'zero copy' techniques: whenever reading whole chunks to a word-aligned destination address, read directly to that address.

Curious: How are you allocating FAT_FILE structures?
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

Good idea about the buffer, thanks. I was originally reading a whole 512 byte then copying only the bytes i needed.

FAT_FILE structs are allocated using malloc within the FAT_fopen method. They are freed in FAT_fclose. I am not sure if this is a good thing to do, as most of the stuff I know is self taught so I am not sure of the theory behind some of the more advanced techniques. If there's a better way to do it, please tell me.

Also, should I be using typedef structs or just structs? What is the difference?

I have gotten my FAT16 driver working! As proof, I have modified the libGBA pcx viewer to load the splash from the compact flash card.

Download this and extract the two files to the root dir of your CF card. Using the GBAMP open PCX_mb.gba. You should see a picture of Pacman.

I will release the driver when I have cleaned up the code a bit.

Last edited by chishm on Tue Jun 28, 2005 6:22 am; edited 1 time in total

Oh... someone else is working on the Movie Player too.
I haven't checked out this thread because I'm mainly concerned about the DS :). You should've informed me sooner ;)
Anyway... this is some bit of src I came up. No FAT routines yet though...

Code:

/*
0 data
1 error/features
2 sector count
3 LBA 0-7 / sector no.
4 LBA 8-15 / cyl low
5 LBA 16-23 / cyl high
6 LBA 24-27 | 0xE0 / head
7 status/command
*/

void CF_Reset();
void CF_SetReg(int reg, unsigned short data);
unsigned short CF_GetReg(int reg);
void CF_SetLBA(unsigned int lba);

#define CF_STATUS_BUSY    0x80 // busy. other status bits are invalid
#define CF_STATUS_RDY    0x40 // ready to accept command
#define CF_STATUS_DWF    0x20 // write fault
#define CF_STATUS_DSC    0x10 // CompactFlash card is ready
#define CF_STATUS_DRQ    0x08 // data request in either direction
#define CF_STATUS_CORR    0x04 // correctable error
#define CF_STATUS_IDX    0x02 // always 0
#define CF_STATUS_ERR    0x01 // see error bits

#define CF_SECTOR_SIZE    0x200

int CF_WaitStatus(unsigned char mask);
int CF_WriteSectors(void *data, unsigned int lba, unsigned int num = 1);
int CF_ReadSectors(void *data, unsigned int lba, unsigned int num = 1);

Code:

#include "cf.h"

#define HW_CTRDG_ROM 0x08000000
#define HW_CTRDG_RAM 0x0a000000

inline void CF_SetReg(int reg, unsigned short data)
{
*(volatile unsigned short *)(HW_CTRDG_ROM | 0x01000000 | reg<<17) = data;
}

inline unsigned short CF_GetReg(int reg)
{
return *(volatile unsigned short *)(HW_CTRDG_ROM | 0x01000000 | reg<<17);
}

inline void CF_SetLBA(unsigned int lba)
{
CF_SetReg(3, lba >> 0);
CF_SetReg(4, lba >> 8);
CF_SetReg(5, lba >> 16);
CF_SetReg(6, lba >> 24 | 0xE0);
}

inline unsigned int CF_GetLBA()
{
return
   CF_GetReg(3) << 0 |
   CF_GetReg(4) << 8 |
   CF_GetReg(5) << 16 |
   (CF_GetReg(6) & 0x0F) << 24;
}

int CF_WaitStatus(unsigned char mask)
{
volatile unsigned int timeout = 1000000; // hmm...
while (--timeout)
{
   unsigned int s = CF_GetReg(7);
   if (s & CF_STATUS_BUSY) continue;
   if (s & CF_STATUS_ERR) return -1;
   if (s & mask) break;
}
return timeout ? 0 : -1;
}

int CF_WriteSectors(void *data, unsigned int lba, unsigned int num)
{
if (CF_WaitStatus(CF_STATUS_RDY) < 0) return -1;
CF_SetReg(2, num);
CF_SetLBA(lba);
CF_SetReg(7, 0x30);
if (CF_WaitStatus(CF_STATUS_DRQ) < 0) return -1;
for (unsigned int i=0; i<CF_SECTOR_SIZE/sizeof(short); i++)
{
   CF_SetReg(0, ((unsigned short *)data)[i]);
}
return 0;
}

int CF_ReadSectors(void *data, unsigned int lba, unsigned int num)
{
if (CF_WaitStatus(CF_STATUS_RDY) < 0) return -1;
CF_SetReg(2, num);
CF_SetLBA(lba);
CF_SetReg(7, 0x20);
if (CF_WaitStatus(CF_STATUS_DRQ) < 0) return -1;
for (unsigned int i=0; i<CF_SECTOR_SIZE/sizeof(short); i++)
{
   ((unsigned short *)data)[i] = CF_GetReg(0);
}
return 0;
}

These routines work also for my homebrew CF adapter.
I got the pinout and can identify the flashchip. So, I'll just have to write a flasher now :)
Hmm.. I never tried to run multiboot apps from inside movie player.
Btw.. I can't seem to run that PCX_mb.gba somehow.

I have decided to release the FAT routines I have writtenso far. I have modifed the reading a bit thanks to darkfader's info. Writing files is not supported yet, and I strongly advise against trying to write individual sectors. I repeat, DO NOT USE CF_WriteSector. It has huge potential to ruin the file system on your card.

This is good for reading files. Read the enclosed text file for details.

EDIT: FAT routines no longer available. They need more work.

Last edited by chishm on Sun Jun 19, 2005 11:40 am; edited 1 time in total

ah... I didn't know about 0x018C0000 yet. My own CF adapter doesn't support it, but it's not strictly required.
I read 0x5000 from this address. Does writing really work? I can't get any other value than this one and 0x0000. Lower byte always 0x00 :/

Last edited by darkfader on Wed Jun 15, 2005 9:17 pm; edited 1 time in total

chishm wrote:

FAT_FILE structs are allocated using malloc within the FAT_fopen method. They are freed in FAT_fclose. I am not sure if this is a good thing to do, as most of the stuff I know is self taught so I am not sure of the theory behind some of the more advanced techniques.

There are some situations in which malloc() does not work. For instance, if you're using data (e.g. GBFS) appended to the .mb file, malloc() will not work unless you use some clever trick to relocate the appended data into some other region of memory.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

Darkfader:
18c0000 seems to be a status register. I think it is equivalent to 1020000 (error/features, as you called it). When the card is idle, it reads 0x50, when the card is in use it reads 0x58. All my code was based on what I reverse engineered from the firmware, so I guess I am not entirely sure why it does that :-(. I write 0x50 to it mainly because the firmware does and I assumed it cleared any previous status. I get 0x00 from it when no card is inserted, and 0xd8 (from memory) when the GBAMP is not inserted.

About your problem with reading 0x5000, make sure you read it as a volatile unsigned 16 bit number.

Tepples:
Am I right in thinking that malloc gets its address range during compile and linking and that this is based on the size of the ROM binary. GBFS support shouldn't be needed if this works properly, as assets can be stored in separate files on the CF card.
Hmm... But I suppose it also wouldn't work if the programmer tries to use a predefined pointer to a location in EWRAM for data storage.

Are there any alternatives to malloc that do not have this problem? Or should I forget about it entirely and set aside a certain amount of memory and limit the number of files open at a time? The problem with this is I am either wasting memory by having more FAT_FILEs than needed, or there may not be enough and users would run out and not be able to open any more files.

It seems like they have connected 2nd chip select signal too. So they're accessing the alternate status register.

01000000..01020000 0x7341 cs0, reg0 1F0 (Read and Write): Data Register
01020000..01040000 0xXX01 cs0, reg1 1F1 (Read): Error Register / 1F1 (Write): Features Register
01040000..01060000 0xXX01 cs0, reg2 1F2 (Read and Write): Sector Count Register
01060000..01080000 0xXX01 cs0, reg3 1F3 (Read and Write): LBA Low Register
01080000..010A0000 0xXX00 cs0, reg4 1F4 (Read and Write): LBA Mid Register
010A0000..010C0000 0xXX00 cs0, reg5 1F5 (Read and Write): LBA High Register
010C0000..010E0000 0xXX00 cs0, reg6 1F6 (Read and Write): Drive/Head Register
010E0000..01100000 0xXX50 cs0, reg7 1F7 (Read): Status Register / 1F7 (Write): Command Register

01800000..01820000 cs1, reg0
01820000..01840000 cs1, reg1
01840000..01860000 cs1, reg2
01860000..01880000 cs1, reg3
01880000..018A0000 cs1, reg4
018A0000..018C0000 cs1, reg5
018C0000..018E0000 cs1, reg6 3F6 (Read): Alternate Status Register / 3F6 (Write): Device Control Register
018E0000..01900000 cs1, reg7

This information looks very exciting and useful! Thanks!

Will long file names be possible?
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

I was writing a FAT driver too. Trying to do it all by correct specs.

(never mind, I just checked the code, and it does properly check directories...)
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

Last edited by Dwedit on Fri Jun 17, 2005 3:32 am; edited 2 times in total

Are you properly handling volume labels and hidden files? If not, then you'll get phantom directories for LFNs.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

So far I have gotten FAT16 working properly for reads and will get writes done next week. It uses short file names but properly skips over long filenames by ignoring any dir entries with the volume flag set. After writes are working I will try to implement FAT32.

Can you even run .mb files from a GBA Movie player with a FAT32 formatted card? Or is this where overwriting the flash comes into play?

By the way, I'd *really* like to see long filenames supported. Maybe store pieces of the filename into a 256 byte buffer (convert any non-ascii to ? or something) as it traverses over the LFN directory entries, then verify it with the checksum?
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

Dwedit wrote:

Can you even run .mb files from a GBA Movie player with a FAT32 formatted card? Or is this where overwriting the flash comes into play?

By the way, I'd *really* like to see long filenames supported. Maybe store pieces of the filename into a 256 byte buffer (convert any non-ascii to ? or something) as it traverses over the LFN directory entries, then verify it with the checksum?

I haven't tried a FAT32 card yet, but only supporting FAT16 is not what I'd call the best way to do things. That is why I have also gone through and rewritten the file fuctions to behave as expected (i.e. same input and output). The new functions aren't available yet, as I forgot to upload them yesterday and can't access my computer now until sunday night.

VFAT (long filenames) is not a high priority, as 8.3 filenames work perfectly fine across all OSs and are widely used. Then again, Darkfader might just one up me and incroporate them into his FAT driver. There's nothing like a little healthy competition to get things moving along.

Oh, and Dwedit, once I release the final version of the code, would you mind incorporating it into Pocketnes. That might be enough incentive to get long filenames working. probably not though - reading VFAT is a pain, writing it is a nightmare.

What about a supercard adaptation ?
Are there screenshot ? Thanks :)

wow I read through the whole thread and understood about 2% of it....oh well.
I just wanna ask: Are you working on something that lets you load stuff from CF-card(via GBAMP) similar to GBAflash-card usage?

would it be even possible(because GBAMP lack on RAM)?

and if it isn't.....would it be possible with the upcoming "M3"?

It's basicly a GBAMP but with DS fullscreen-support(AFAIK) and RAM.

just curious to know.

10x in advance for answer
_________________
Abusing Cube and DS with all sorts of homebrew and hacks.

Zubiac:
What I am working on is a way for homebrew games to load files from the CF card, so they are no longer limited to 256 KiB. There is no way this can be used for commercial ROMs, the M3 will do that natively. And who needs commercial ROM support anyway?

Kouray: If the supercard works the same way as the GBAMP it should work. Then again, it may disable CF access when running ROMs. There are no screen shots, as this is filesystem code and so it is not related to any particular game.

Dwedit:
I've been thinking - I could probably do VFAT, but only for directory listings. Internally everything would still be 8.3. That is the user would see long file names but everything would be accessed using short filenames. Is that okay?

chishm wrote:

Zubiac:
What I am working on is a way for homebrew games to load files from the CF card, so they are no longer limited to 256 KiB.

Which would lead to something almost like an "Advance Disc System" as discussed on the Yahoo! Group.

Quote:

There is no way this can be used for commercial ROMs, the M3 will do that natively.

You're talking about commercial .gba ROMs. The commercial .mb files (e.g. from Animal Crossing and Nintendo Puzzle Collection) and commercial 8-bit ROMs run in emulation (e.g. from a version of PocketNES designed to use EWRAM as a cache for the ROM) would still work, no?

Quote:

There are no screen shots, as this is filesystem code and so it is not related to any particular game.

Yet I still managed to make a screenshot of GBFS ;-) The screenshot of a directory listing (consider AGBTTY to speed up text output) would probably work as evidence of how far your VFAT implementation has come.

Quote:

I've been thinking - I could probably do VFAT, but only for directory listings. Internally everything would still be 8.3. That is the user would see long file names but everything would be accessed using short filenames. Is that okay?

If I were implementing it, I'd allow for read-only LFN support. I could live with a system incapable of fopen("Paranoia Survivor.gsm", "rb"), but I'd strongly recommend against it. If the file name has to be precompiled into the program, the short name is not always predictable from the long name. Besides, one of the most common file name suffixes in GBA homebrew development (".gbfs") has four characters. And if you can read the LFNs, is it that much harder to match an LFN? Or is it a matter of storing the case-folding tables for scripts other than Latin?

Speaking of other scripts, how will your VFAT implementation handle file names in scripts other than Latin? Will it convert to UTF-8, or will it assume ISO Latin-1?
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

chishm wrote:

Zubiac:
What I am working on is a way for homebrew games to load files from the CF card, so they are no longer limited to 256 KiB. There is no way this can be used for commercial ROMs, the M3 will do that natively. And who needs commercial ROM support anyway?

oh thanx for the answer and to make it clear: I wasn't talking about pirating.I'm a strict fighter against piracy in any form.
I'm still curious why there actualy IS a 256kb limit on GBAMP?
I didn't even know that the M3 will play commercial roms.All I knew is that it has RAM and DS support.

anyways....thank you(and all ther others) for working on this.
Darn Flashcards are really expensive and loading stuff from CF card would really help.
^_^
_________________
Abusing Cube and DS with all sorts of homebrew and hacks.

Questions about why the GBAMP's 256k limit probably should be moved by a moderator into another thread, but the answer is simply that the GBAMP has 512k of ROM to hold its own movie playing software, a CF interface, and nothing else. Everything except the movie player's own functionality has to run from the GBA's internal 256k+ of RAM.
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

I don't see the picture of pac man, it just freezes!
And LFNs for display only is fine, until I need to start creating .sav files with the same LFN as the parent .nes file...
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

Right now I'm getting filenames like:
B
[
(garbage character)
(garbage character)

[text removed]
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

Tepples:
From what I've read about VFAT in windows (check the Maverick OS link you gave me) it would seem that only the first 255 Unicode characters are used, i.e. the characters are just ASCII. This may not be correct as I am sure I have seen chinese character filenames. However the GBA has only limited resources to work within and storing the full 16 bit unicode character set in something that will only be used for reading files will bloat ROMs unnecessarilly. I think the best thing to do in this situation is to assume ISO Latin-1 and limit filenames to this character set, with all other characters being converted to underscores.

Dwedit:
Those problems are probably caused by the buggy Boot sector finding code in my first release. It would seem that your CF card does not have a partition table, but my code is assuming one. This has been fixed in my new code, but as I said before, I can't upload for at least 10 more hours.

If you know how, could you analyse your card and tell me if there are in fact any partitions (i.e. it uses a hard drive style layout) or if the boot sector is the first sector (i.e. a floppy disk style layout)

No, it is finding the boot sector (sector #32) correctly. Ignore the old deleted text, I was looking at the wrong place (inside the boot sector...heh) when I noticed the bad values. Winhex was editing a partition's boot sector instead of the MBR, and I didn't notice at the time. There is a MBR and partition table.

Can you just call FAT_InitFiles() then call FAT_FindFirstFile(filename) and FAT_FindNextFile(filename), or am I using this wrong? Right now I'm instead getting lots of directories with garbled 4-length names.
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

Dwedit:
I think I found the problem. I'm going to take a stab in the dark and say you aren't using a card with 16 sectors per cluster (eg 512MB). I have such a card, and 16 sectors per cluster * 2 clusters not available at start of disk * 512 bytes per sector == 32 bytes per dir entry * 512 entries in root dir. In other words I was (un)lucky in that assuming the root dir was cluster 0, when in my case it actually was. Should be fixed now. No source code is available, but try out these two demos.

PCX viewer
Root directory listing

Last edited by chishm on Tue Jun 28, 2005 6:24 am; edited 1 time in total

My 128MB (122MB) card has 2048 bytes per cluster (4 sectors per cluster).

The pcx viewer crashes with no display (it still displays the GBA Move Player file listing), while the directory listing test program only shows two lines at the bottom.

Before, I also had lots of compiler errors. Mainly caused by having variable declarations AFTER lines of executable code, which is unsupported by pocketnes's compiler. Mostly those functions that check if the card is inserted before declaring variables, which is illegal. Also, __attribute__ was unsupported (so I just commented those out).
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

Dwedit wrote:

The pcx viewer crashes with no display (it still displays the GBA Move Player file listing), while the directory listing test program only shows two lines at the bottom.

But is it better than what it was? I realised last night that the fix only worked for files in the first cluster. Will do better this time.

EDIT: Okay I am sure I have fixed the bug now... Can you please download the files again and test them out. Make sure the file date is the 20th of June for the latest version.

Dwedit wrote:

Before, I also had lots of compiler errors. Mainly caused by having variable declarations AFTER lines of executable code, which is unsupported by pocketnes's compiler. Mostly those functions that check if the card is inserted before declaring variables, which is illegal. Also, __attribute__ was unsupported (so I just commented those out).

Attribute is required to ensure that the structures are packed correctly. If your compiler doesn't automatically compact structs you will need to tell it to do so, otherwise it won't read the boot sector correctly. If its possible, I could precompile the source into object files ready for linking. Does anyone know if this works or how to do it?
I will move the variable declarations to the top for you.

Dwedit wrote:

Before, I also had lots of compiler errors. Mainly caused by having variable declarations AFTER lines of executable code, which is unsupported by pocketnes's compiler. Mostly those functions that check if the card is inserted before declaring variables, which is illegal.

It's legal in C99 and (as an extension) in GCC 3.4 or later. Is there a reason that PocketNES still uses another compiler toolchain (presumably ARM SDT/ADS)?

Quote:

Also, __attribute__ was unsupported (so I just commented those out).

Many of the attributes specify a section (EWRAM, etc) that a given function is supposed to go into. For instance, code dealing with I/O devices on the cart (SRAM, flash, or EEPROM on a Game Pak, CF on a GBA Movie Player, etc) should ideally go in EWRAM or IWRAM for the most stable results.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

chishm wrote:

Darkfader:
18c0000 seems to be a status register. I think it is equivalent to 1020000 (error/features, as you called it). When the card is idle, it reads 0x50, when the card is in use it reads 0x58. All my code was based on what I reverse engineered from the firmware, so I guess I am not entirely sure why it does that :-(. I write 0x50 to it mainly because the firmware does and I assumed it cleared any previous status. I get 0x00 from it when no card is inserted, and 0xd8 (from memory) when the GBAMP is not inserted.

About your problem with reading 0x5000, make sure you read it as a volatile unsigned 16 bit number.

Tepples:
Am I right in thinking that malloc gets its address range during compile and linking and that this is based on the size of the ROM binary. GBFS support shouldn't be needed if this works properly, as assets can be stored in separate files on the CF card.
Hmm... But I suppose it also wouldn't work if the programmer tries to use a predefined pointer to a location in EWRAM for data storage.

Are there any alternatives to malloc that do not have this problem? Or should I forget about it entirely and set aside a certain amount of memory and limit the number of files open at a time? The problem with this is I am either wasting memory by having more FAT_FILEs than needed, or there may not be enough and users would run out and not be able to open any more files.

with devkitARM you can search for the gbfs assets, find the end address & then write that to fake_heap_start *before* the first call to malloc. define fake_heap_start as

Code:

extern char *fake_heap_start;

the address should be rounded up to the nearest 32bit value.

This will set the start address of the heap and obviously applies only to multiboot binaries and ds arm9 code.

The sample programs work great now!
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

chishm wrote:

Am I right in thinking that malloc gets its address range during compile and linking and that this is based on the size of the ROM binary. GBFS support shouldn't be needed if this works properly, as assets can be stored in separate files on the CF card.

That is, unless (say for a .txt reader) you're storing a font in an appended GBFS and a .txt file in CF. But if you run into problems with the heap or the .sbss segment overlapping the GBFS file, then you could always store the font and other assets in a GBFS file statically linked into your program using bin2s or objcopy.

Quote:

Hmm... But I suppose it also wouldn't work if the programmer tries to use a predefined pointer to a location in EWRAM for data storage.

Programs that use static allocation of EWRAM typically are simple enough to place the scratchpad area at the end of EWRAM. For example, Tetanus On Drugs M3 uses the last 64 KiB for decompression of assets, knowing that the program+assets will never be larger than 192 KiB. (TOD M2 and M3 were both around 160 KB.) A lot of these were written by developers who started out before linker scripts began to support the .sbss segment.

wintermute wrote:

with devkitARM you can search for the gbfs assets, find the end address

Specifically, using skip_gbfs_file().

Quote:

then write that to fake_heap_start *before* the first call to malloc.

Which doesn't help if a C++ constructor for a static object uses operator new (which calls malloc()) before your code has a chance to set up the heap. Is there a way to guarantee that a constructor that enumerates GBFS files will be called before a constructor that (directly or indirectly) triggers heap initialization, other than by modifying crt0 or replacing malloc()? And even in C, it still doesn't seem to fix the .sbss problem. But thanks anyway for the tip.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

Quick status update:
I have now started work on writing files. So far I am able to delete files properly, which is harder than it sounds. First I have to scan the entire file's cluster chain and free cluster as I go. Only then can I remove the directory entry.

I have thought about the use of malloc. I think I will stop using it. Instead I will have an array of files in memory. This means only a fixed number of files can be openned at one time and space is taken up even if no files are open. However it is also more compatible with other code, which is what I hope to aim for.

I won't release any more code until I have cleaned it up some more.

n00b alert
As I said at Acmlm's Board:

Once chishm perfects writing to the CF card, it would be great if it was incorporated into PocketNES. You could eliminate the 'SRAM Cannibalism' (in the PocketNES hack by Dwedit) feature and replace it with saving the SRAM, and maybe even savestates to the flash card.
Also, it would be great to incorperate into Goomba as well. You wouldn't have to prepare the GBA roms beforehand, you could load the Goomba emulator and load the GB Roms inside of Goomba...

I'd like to see the fixed read code for now. Looks like sorting the files (directories first, all files/dirs in alphabetical order) is going to be a necessary challenge.

Any hopes of a PWD function?
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

I have gotten writing working as far as I have tested (write, append). I have also started implementing long filenames. Demos have been updated (22 June 2005), see link above.

Be warned THIS IS BETA CODE. IT HAS THE POTENTIAL TO CORRUPT YOUR CF CARD. BACKUP EVERYTHING ON IT BEFORE USING THESE DEMOS.

The PCX demo will now copy the file SPLASH.PCX into SPLASH2.PCX. It also appends to the file TEST.TXT. Wait until the picture displays before turning off your GBA / DS.

The other demo (GBAMP_CF.GBA) still displays a directory listing, but now it also displays long filenames. This doesn't write to the card at all so should be safe.

Yep, it created lots of garbage directories with garbage filenames. It definately corrupted the card.
It didn't delete anything or mess up any existing files, just created lots of new junk.
TEST.TXT created fine. SPLASH2.PCX is unreadable. Only Winhex will open it.

I REALLY don't care about whether the code supports writing or not, I would like to see the fixed read code anyway.

How about overwrite support if the target file's size doesn't change? I'm sure that could be safely added, and would work well for NES SRAM files. But I'd really like the working read code.

I also tested it out on a FAT32 partition. Looks like the GBAMP supports fat32, but the test program doesn't.
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

I flashed my movie player v2. But it's still locked in DS mode, so I only get a contiguous 512 bytes. I still have to try GBA mode.
It'd be nice if I could put FAT support in it, so it could load a multiboot app in GBA mode and NDS file in DS mode.
My FAT driver hasn't progressed much though.

Dwedit:
Can you do a test for me please? This will probably corrupt your card. First, format your card. Then put PCX_MB.gba and SPLASH.pcx onto the card. Run PCX_MB.gba once, as soon as you see the picture, turn off you GBA and check the CF card in your PC to see if it is corrupted. Then, run PCX_MB again without erasing any more files. Again check the card in your computer and see if it is corrupted or not, trying to view SPALSH2 and Test.txt. SPLASH2 should be a copy of SPLASH, while Test.txt should have two sets of the same text if you ran PCX_MB twice.

Here is a link to the source so far. It is broken, so DO NOT use it in anything that will be released. Reading should work if you open files in "r" mode.

EDIT: Get proper source on first page

Last edited by chishm on Tue Jun 28, 2005 6:25 am; edited 1 time in total

After the first run, Splash.pcx and PCM_mb.gba are missing, and splash2.pcx cannot be opened. Test.txt is unreadable. Both file sizes are 0 bytes.

In winhex, the files hex dumps appear to be correct, but winhex is a lot more forgiving. Their sizes are zero bytes large, so only winhex can view them since it directly reads the disk.

Can't really run it a second time, since the gba file went missing. I'll extract it and run it again.

Okay, I ran it again after dumping the files back on the disk. The movie player software sees lots of crazy directory entries containing strange characters, but windows explorer doesn't see them. I ran it again anyway. The first ~60 bytes of the .pcx file became "This is a test..." so Irfanview detected a bad header on the pcx file. The other pcx file is unreadable.
The text file is readable, and only contains one line of text. Turns out the text file is actually crosslinked to the .pcx file now.

Then I just ran chkdsk for no reason, and it was able to recover the first 4k of SPLASH2.PCX, test.txt became a 10k file containing the entirety of splash1.pcx. It also dumped a couple of .chk files, one of which looked like a .gba rom.

Hope it will be safe to use only the reading functions.
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

darkfader wrote:

I flashed my movie player v2. But it's still locked in DS mode, so I only get a contiguous 512 bytes. I still have to try GBA mode.
It'd be nice if I could put FAT support in it, so it could load a multiboot app in GBA mode and NDS file in DS mode.
My FAT driver hasn't progressed much though.

Have you tried writing to 0x50 to the status and/or alternate status registers before doing anything else. This should initialise the card.

PCX demo has been updated again. Both demo links here. Just extract the zips to the root of your CF card. As always, these are very likely to corrupt your card. Keep a backup of all files. I am not responsible for any damage.
Changes to PCX demo: Now supports long file names! Make sure "temp folder" exists on the root of your card. It will copy splash.pcx into temp folder.
PCX viewer
Root directory listing

Last edited by chishm on Tue Jun 28, 2005 6:26 am; edited 2 times in total

I have finished writing the FAT16 driver.

Features:
* FAT16 only - no FAT32
* Long filenames
* File reading and writing
* File creation and deletion
* No longer uses malloc - uses a global array instead

Get it here.

Last edited by chishm on Tue Jun 28, 2005 6:26 am; edited 2 times in total

Wow. Great work! Cant wait for the first applications to use this.

Yatta! I got it working with ARM SDT.
Arm SDT is weird. It complains about using the dot operator on a function call, even if that function returns a struct. Had to stick it into a temporary variable.
I also had to kill off __attribute__, I think it's GCC specific. Had to declare the structs as "typedef __packed struct", then finally, I had to cast every second parameter to readsector as (void*) to get rid of compiler errors resulting from use of __packed. It still gives out lots of warnings about how dividing by sizeof(xxxx) might destroy __packed, and lots of warnings about how the functions that take in no parameters are missing the word (void).
It also didn't like the boot sector that was declared in the middle of executable code, an easy fix.

So far, I've only tested getting a list of files though. Haven't tested reading or writing yet. Soon I'll need to write a file sorter and picker, so a user can select from a list of long file names, and traverse over different directories.

How hard would it be to modify visualboyadvance to emulate the movie player, and be able to directly access a the sectors of a connected compact flash card?
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

Dwedit:
Good to hear you have it compiling - I have changed the function declarations to include void now and moved the bootSector declaration to the top of the function. You may want to redownload the files - see edit above.
Have you tried out the new demos yet? If so, did they work correctly?
I am not too sure how hard it would be to modify VBA, but it would be harder than adding a completely new save type to it (eg incorporating flash saves into a non-flash save compatible version). It's a bit annoying not having GBAMP support in an emulator as I can't take any screenshots.

With the file sorter/picker you don't need to remember the parent directory if you go into subdirectories, as the .. entry will link back for you. PWD is not incorporated, as it would mean having to keep track of the parent directories of the current sub directory.

Sorry if this has already been discussed, but I'm a little confused.

Will this code ever be released as a firmware upgrade for the GBAMP? I'm wonderinig if there's a way to just throw a .nds.gba file on the flash card and boot it with a PassMe. Or is it already possible to execute DS code from the GBAMP in some way?

Thanks for clearing this up,

MrNull
http://ndswire.blogspot.com

Well... I'm doing FAT stuff for use on NDS. If chishm keeps focussing on GBA, it's fine with me :)
I already updated my GBAMP. Current loader is 384 bytes and can load START.NDS from FAT16 (possibly FAT12 too) but the file cannot be fragmented.
I think I'm going to write a multi stage loader. This small loader will load a nicer loader with menu etc.
The main problem is upgrading the device for the second time. I think that's solved by also making it GBA mode compatible. Oh hey... all I need is to add a Nintendo logo, I guess :)
When starting in DS mode, the GBAMP is somewhat locked so after first 512 bytes, it repeats itself.

Does anything different happen if you have the DS code mimic the movie player's boot process and do its bootup DMA copy?
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

darkfader wrote:

Well... I'm doing FAT stuff for use on NDS. If chishm keeps focussing on GBA, it's fine with me :)
I already updated my GBAMP. Current loader is 384 bytes and can load START.NDS from FAT16 (possibly FAT12 too) but the file cannot be fragmented.
I think I'm going to write a multi stage loader. This small loader will load a nicer loader with menu etc.
The main problem is upgrading the device for the second time. I think that's solved by also making it GBA mode compatible. Oh hey... all I need is to add a Nintendo logo, I guess :)
When starting in DS mode, the GBAMP is somewhat locked so after first 512 bytes, it repeats itself.

My FAT routines should work on NDS too, just need to change the gamepak start address. I haven't tried it in DS mode, as I don't have a *me, but the code doesn't assume anything about the platform except for the start address and use of DMA.

Anyway, after I add FAT32 (next week sometime) my FAT routines will be complete. It is still good that you are writing FAT drivers, as competition leads to progress (theoretically). Your routines will probably be faster than mine anyway.

Using VBA Dev version with logging enabled I see 10 halfword writes in a row of 0 to 0x08000000 (gamepak start address), a DMA of 16 bytes from 0x08000000 then another halfword write of 0 to the same address. This may be involved in unlocking the GBAMP. Then again, I could be wrong.

Can't you use "#pragma packed(1)" for your structs?
And don't forget to add partition support :)
Anyway, mine isn't near finished. Got also other things to do.

Running into the same problem dwedit had when I tey it my my gbamp i get the same problems. I reformatted my compact flash card from FAT32 to FAT I assumed XP would make it FAT16, but I was wrong. Is there a tool to reformat the compact card to FAT 16 from FAT 32?

Oh yeah the problem I had was the two files equalling 0 bypes under GBAMP with the dump gba program.

gbaplaya05 wrote:

Oh yeah the problem I had was the two files equalling 0 bypes under GBAMP with the dump gba program.

It has formatted as FAT16. If you can see a picture then the CF card must have been FAT16. Run scandisk on the card and see if the file sizes change or if file fragments are found. You may need to show hidden and system files to see the file fragments. After you have done this, report back what size the files are.

darkfader wrote:

Can't you use "#pragma packed(1)" for your structs?
And don't forget to add partition support :)
Anyway, mine isn't near finished. Got also other things to do.

Apparently #pragma pack() doesn't work in GCC. I have added it for non-GCC compilers though.
It does have partition support - it reads the first partition. Multiple partitions on the one CF card is not too common an occurence, and not worth supporting.

EDIT: New demo - A save game copier. Warning - this could ruin your CF card or your game save.
Instructions to use:
1. Download from here and extract the file to your CF card.
2. Put the GBAMP in your GBA (or NDS) and run MP_Save.gba.
3. Remove your GBAMP, insert your game then press A once.
4. Remove your game, reinsert your GBAMP then press A again.
5. Turn off your GBA, put your CF card in the computer and then use the save file for whatever purpose you want.

Last edited by chishm on Tue Jun 28, 2005 6:27 am; edited 1 time in total

could you a ds version of your save game copier but for ds save ? would be usefull for ones who flashed tehir gbamp for running homebrew for it :)

YodaNC wrote:

could you a ds version of your save game copier but for ds save ? would be usefull for ones who flashed tehir gbamp for running homebrew for it :)

You flashed your GBAMP? I thought only Darkfader had done that. I haven't got a *me of any kind, so I can't test any DS code and hence can't write any DS programs. If you haven't flashed your GBAMP, do all of the other demos work? I just need feedback to make sure the driver works correctly, as I only have one CF card.

chishm wrote:

YodaNC wrote:

could you a ds version of your save game copier but for ds save ? would be usefull for ones who flashed tehir gbamp for running homebrew for it :)

You flashed your GBAMP? I thought only Darkfader had done that. I haven't got a *me of any kind, so I can't test any DS code and hence can't write any DS programs. If you haven't flashed your GBAMP, do all of the other demos work? I just need feedback to make sure the driver works correctly, as I only have one CF card.

I flashed it with Darkfader's flasher ;)

How can i get this flasher for the GBAMP????

chishm wrote:

EDIT: New demo - A save game copier. Warning - this could ruin your CF card or your game save.
Instructions to use:
1. Download from here and extract the file to your CF card.
2. Put the GBAMP in your GBA (or NDS) and run MP_Save.gba.
3. Remove your GBAMP, insert your game then press A once.
4. Remove your game, reinsert your GBAMP then press A again.
5. Turn off your GBA, put your CF card in the computer and then use the save file for whatever purpose you want.

I get as far as #4. When I re-insert my GBAMP, it resets. (I was trying to use my Mario Kart gba game, just to test. It's the only one I don't care about losing the save files.

EDIT: After about the fifth try, i got it to work... sorry...
EDIT2: Maybe not... It went through all the steps, BUT all I get is an emtpy (0 byte) file named 'MARIO KART' (no file-extension).
If it matters, I'm using a 16 MB (Kodak brand) FAT-formatted card.
EDIT3: Tried the PCX viewer. That doesn't quite work either. I see half of the image, the Text file appears ok, but the copied file appears to be corrupted.

jmr:
Try getting the new save demo (same link) and see what happens. Before turning off your GBA, copy down the text and post it here, starting with the line that says AddDir:. Include the numbers / letters when copying the text. This is just to help me work out what is happening. Also, if you have a larger card, you may want to try it.

After backing up my 512 MB card, I tried it again. Success! I managed to load my SAV file in VBA. It was kinda cool to watch my recorded 'Ghost' races on my PC. Now that I know it works, I think I'll try copying my e-Levels from SMA4... which reminds me: Do you think you can make a similar program that reads the SAV file on the Flash Card and writes it to the actual cartridge? I know it can been done with the GBAMP and certain types of Saves, but it doesn't work with all save types (as far as I know). (I did it with SMA4 before I had an eReader). (See HERE.)

And here is the info you asked for:
16 MB Card
AddDir: 50 9
DEFP: 50 9
fopen: 50 9
fclose: 50 9
Done
---
512 MB Card
AddDir: 233 8
DEFP: 233 8
fopen: 233 8
fclose: 233 8
Done

jmr:
From what I can see it should have worked with your 16MB card too. I think I have figured out the other problem with the PCX viewer too. It should only have that problem on small cards. I will fix it later today.

Moving saves from GBAMP to cartridge is just as easy as the other way around, except I have to make a file select menu, which I haven't done yet. That save demo was whipped up in less than an hour and was based on SendSave (see my website for details). SendSave can write everything except 256KiB flash saves. It should work with nearly all games as I have yet to see a game with a 256KiB save.

I have added FAT32 support. The FAT code is now complete and the only changes will be bug fixes. I have also updated the demos with the new code.

Get the FAT driver here.

Last edited by chishm on Mon Mar 20, 2006 3:41 am; edited 2 times in total

Hi,

GCC works with attributes, not pragmas :

Code:

__attribute__ ((packed))

for the packing stuff. See http://gcc.gnu.org/onlinedocs/gcc-3.4.3/gcc/Variable-Attributes.html#Variable-Attributes for more info about attributes on variables.

HTH,
Vinz
_________________
Reclaim control of your F2A/F2AU with if2a !!

Vince wrote:

Hi,

GCC works with attributes, not pragmas :

Code:

__attribute__ ((packed))

for the packing stuff. See http://gcc.gnu.org/onlinedocs/gcc-3.4.3/gcc/Variable-Attributes.html#Variable-Attributes for more info about attributes on variables.

HTH,
Vinz

Yes I knew that already, although I don't know if Darkfader did. That is why the source uses __atribute__((__packed__)). It also uses pragma for compatibility with other compilers, such as the official ARM one.
This is shown here:

Code:

#ifdef __GNUC__
#define __PACKED __attribute__ ((__packed__))
#else
#define __PACKED
#pragma pack(1)
#endif

The structures are then declared with __PACKED, which will be interpreted as __attribute__((__packed__)) if GCC is being used. If not, they will be ignored by the compiler and pragma will be used instead.

gcc understands #pragma pack fine

http://gcc.gnu.org/onlinedocs/gcc/Structure_002dPacking-Pragmas.html#Structure_002dPacking-Pragmas

Just tried the latest demo GBAMP_CF.gba and it seems to have worked. The file size is 188 bytes, but Windows says the file is corrupt.

Ran scan disk and tried again I also opened the filein GBAMP and it worked with a few garbage characters in the text. Then I opened it in Windows and it worked flawlessly.

Also the picture program worked on the top screen.

Just saw a program that uses the GBAMP to dump commercial DS roms. Did you guys have something to do with that?

gbaplaya05 wrote:

Just tried the latest demo GBAMP_CF.gba and it seems to have worked. The file size is 188 bytes, but Windows says the file is corrupt.

Ran scan disk and tried again I also opened the filein GBAMP and it worked with a few garbage characters in the text. Then I opened it in Windows and it worked flawlessly.

Also the picture program worked on the top screen.

Was that GBAMP_PCX.gba you are referring to? I take it it was test.txt which had the garbage characters. This is actually because of the way the GBAMP treats text files. Was it the copied pcx which was corrupt? Or was it the text file? If it was the text file, this is actually a quirk in the program. 188 bytes tells me it was the text file, and you viewed the picture 4 times.

mrnull wrote:

Just saw a program that uses the GBAMP to dump commercial DS roms. Did you guys have something to do with that?

I had nothing to do with it - can't speak for Darkfader. Although it seems like they used our code.

mrnull wrote:

Just saw a program that uses the GBAMP to dump commercial DS roms. Did you guys have something to do with that?

According to the post on http://forum.gbadev.org/viewtopic.php?t=6103

chishm wrote:

Yeah, and somehow I think this dumper is based in someway on Darkfader and my flash read / write sector routines. Notice how it came out just after Darkfader posted code for writing? I think the least they could do to thank us is update it to use FAT so it doesn't corrupt the CF card. That and release their source code.

_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

chishm: As far as I can tell, all of the demos work fine (atleast on my 512 MB card). The PCX viewer now displays and copies the image properly.
Keep up the good work!

Any word on incorporating this into pocketnes? Do I see save states in the future possibly?

chism: Are you sure it's a good idea to use pointer comparison on strings?
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

Dwedit wrote:

chism: Are you sure it's a good idea to use pointer comparison on strings?

Can you please elaborate. Which function (or if I did it in all of them) and what do you mean by pointer comparison?
I'm sorry, I am not too sure on this issue. If it is bad programming style then I will fix it. It just means I don't need to include the string library.

I mean doing string comparisons with ==. Looks like they're all in fopen.
Stuff like 'if (mode=="w" || mode=="w+")'
This kind of code requires the compiler to combine identical const strings to the exact same address, and will fail otherwise.
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

Dwedit wrote:

I mean doing string comparisons with ==. Looks like they're all in fopen.
Stuff like 'if (mode=="w" || mode=="w+")'
This kind of code requires the compiler to combine identical const strings to the exact same address, and will fail otherwise.

Whoops. I now realise that was wrong. I blame it on the java course I did, where that was a perfectly good way of doing things. I will fix that today.

uhhh... I might try to grab the IDE/FAT stuff from Linux..
Dunno which is more compact.

Dwedit wrote:

I mean doing string comparisons with ==. Looks like they're all in fopen.
Stuff like 'if (mode=="w" || mode=="w+")'

The Right Way(tm) to handle fopen() mode strings is to parse the mode string and set separate variables in the FILE struct: access mode (one of 'r', 'w', 'a'), random access (whether '+' is present), and transformation between \r\n in the file and \n in memory (whether 't' is present).

chishm wrote:

Dwedit wrote:

This kind of code requires the compiler to combine identical const strings to the exact same address, and will fail otherwise.

Whoops. I now realise that was wrong. I blame it on the java course I did, where that was a perfectly good way of doing things. I will fix that today.

For other string comparisons, drop this in a header file somewhere:

Code:

#include <string.h>
static inline int streq(const char *a, const char *b)
{
return !strcmp(a, b);
}

It'll return nonzero iff the strings are equal.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

The supplied code is very stack unfriendly. I have about 1200 bytes of stack space, and I'm modifying the code to be less stack dependent.
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

If someone wants to put this in CVS it would be good. That way anybody can fix my mistakes. I will supply the latest code I have. Even if it needs heavy modification or a full rewrite, at least it is a starting point.

EDIT:
Ok, this makes me sound like I can't be bothered cleaning up my code, which is not what I meant. I actually want to get this working well, but am not sure on some points, so I have a few questions:
1) With string comparisons is it better to use the string library or my own custom code? Will the string library significantly slow down or bloat my code, or is it well optimised and perfect for the task?
2) I am currently using large 512 byte local variables as a sector buffers in functions that need it. This requires a lot of stack space, as Dwedit mentioned. Keeping in mind that the functions will be used quite a bit, should I malloc this space instead? Or should I use one global sector buffer and cast it as required?
Thanks in advance for any help.

The 512 byte buffer should be kept as part of the FILE struct, for various reasons.

Ideally you should also provide a way to redirect calls to fputc(), fputs(), and fwrite() for a given FILE * to some other code so that, say, I could point stdout and stderr to AGBTTY.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

tepples wrote:

The 512 byte buffer should be kept as part of the FILE struct, for various reasons.

I do have a 512 byte buffer as part of the file structs. However I also need to read sectors that aren't part of a file (FAT, directories, MBR) but these reads must be buffered, due to the way CF cards work. That means I need to either allocate a buffer in each function that performs one of these reads, or I need to keep a global generic sector buffer that is cast as needed to the correct type (DIR entry, FAT entry).

At the moment I am doing the former, but I am thinking that the later is better in terms of stack usage at the cost of a 512 bytes of EWRAM being constantly used. Is this right, do globals not use the stack?

tepples wrote:

Ideally you should also provide a way to redirect calls to fputc(), fputs(), and fwrite() for a given FILE * to some other code so that, say, I could point stdout and stderr to AGBTTY.

This CF file library, although it properly implements FAT_fwrite() and FAT_fread() (for binary files) is only meant for use with files on the CF card. In this regard it behaves a lot like the file functions in libgba as used for Xboo Communicator. FAT_fwrite() is heavily tailored for use with FAT files running on standard LBA addressed ATA devices (ie a CF card). This is why all the function names are prepended with FAT. I have tried to implement basic functionality for using the GBAMP without bloating the code with unnecessary functions. Then again, I should probably add an fputc() and fgetc() function, as they will not require too much code. fputs() will need the inclusion of stdarg, again bloating code.

Sorry if this sounds like I'm flamming you, I am not. I am only trying to explain my reasoning for the way the code is, so that everyone knows exactly what I'm trying to do.

chishm wrote:

I do have a 512 byte buffer as part of the file structs. However I also need to read sectors that aren't part of a file (FAT, directories, MBR) but these reads must be buffered, due to the way CF cards work.

Then you'll probably need to keep a 512-byte buffer in EWRAM. By default, global variables use IWRAM (shared with the stack), but they can be put into EWRAM by putting them in section ".sbss" (or whatever the ARM SDT link script uses).

Quote:

Then again, I should probably add an fputc() and fgetc() function, as they will not require too much code. fputs() will need the inclusion of stdarg, again bloating code.

Actually, it's fprintf() and fiprintf() that need stdarg.h. fputs() just writes a string:

Code:

int fputs(const char *str, FILE *out)
{
int c;

for(c = *str; c != 0; c = *++str)
fputc(str, out);
return 0;
}

_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

The real stack killer is the path[] string. Either use 1024 bytes for that, or just make the input non-const and put a null character where it separates path from filename.

tepples wrote:

Actually, it's fprintf() and fiprintf() that need stdarg.h. fputs() just writes a string:

Sorry, you're right about fputs().

Dwedit wrote:

The real stack killer is the path[] string. Either use 1024 bytes for that, or just make the input non-const and put a null character where it separates path from filename.

I will use a 512 byte global sector buffer. For the path I can still do it with a const input and without the path[] string. It will just take a bit more effort to get going. I will work on it sometime today and post the results today or tomorrow.

EDIT: Stack usage reduced. Stack usage of any function should be at most half a KiB now. Still doesn't use malloc, instead just one global sector buffer.

Having problems. Using a FAT32 file system with lots of files causes it to mess up the file system, even when only doing directory listings and read only fopens.
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

Dwedit wrote:

Having problems. Using a FAT32 file system with lots of files causes it to mess up the file system, even when only doing directory listings and read only fopens.

If it is happening even when you are not writing, then it is probably a problem with the fatWriteBuffer.
In the FAT_NextCluster function, comment out the line that reads

Code:

CF_WriteSector(fatCurSector, fatWriteBuffer);

This should stop it corrupting the file system as a temporary fix until I work out what is wrong. You may also want to try formatting as FAT16 before changing this line, just to see what happens. It may take a little while for a fix to this problem, as I only have the one CF card to test with, and it works perfectly.

I replaced the first line of writesector to return false, but listing directories returns the same ~23 files repeating, and corrupts the file system. Maybe my replacement readsector code is bad? (unlikely, but possible?) (this code replaces the DMA3 macro)

Code:

   u16 *dest=(u16*)buffer;
   u16 *src=(u16*)(CF_DATA);
   int count=256;
   for (i=0;i<count;i++)
   {
      dest[i]=*src;
   }

On FAT32, there are about 6 repeating files. Something tells me it's not advancing to the next cluster correctly or something.
Oddly enough, it works perfectly with no corruption in the root directory only. Subdirectories mess up, displaying repeating files, and corrupting the directory.
The directory list demo works fine. It seems to display all the files twice, is that normal? I don't know if the dir listing demo does subdirectories though. Have you tested a subdirectory with lots of files in it yet?

new: test version of pocketnes that exhibits the bug. http://home.comcast.net/~alanweiss3/dwedit/files/pocketnes_mp_test.zip
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

Dwedit:
I tried out the test and it works fine, even for sub directories. Restarting worked a few times, then froze it with a white/black flashing screen. It has not corrupted my card.
The dir demo lists twice because it tries to change to "/music" then lists whatever is there. If there is no music directory then it stays in the root dir and shows whatever is there, again. My card uses 4 KiB clusters, even with FAT32, so I have not had a directory filled enough for it to span a cluster. I will look into it though.
It should not be writing to the card if CF_WriteSector is disabled. That is the only function that directly writes to the card. Would you mind if I took a look at your source to see what I can do? PM a link to it to me if you wish to keep it closed (I won't distribute it).

Just tried the test version of PocketNES... Everything worked a-okay on my card. I'm glad that progress is being made!

Edit: Here's something interesting... NES Open Golf loads with the new GBAMP Pocketnes, but doesn't when loaded using Compy r8. However, it still runs too slow to be played properly.
So technically you've already increased compatibility! Nice work!

I tried it with subdirectories, did'nt corrupt my card, but would'nt load any roms either. Also, even though I'm pretty sure FAT32 is'nt supposed to be supported (since this is a FAT16 r/w thread) I did try it with a FAT32 formatted card and it did some, 'interesting' things, nothing that worked though.

I finally got my hands on a small (32MB) CF card and tried it out. It wasn't working... so I went on a major bug hunt today. I found a few && where there should have been &, which would cause problems with files away from the beginning of FAT32 formatted cards. I also found a few FAT16 problems with the root directory. Both bugs are fixed now. The updated source is available on the first post. Let me know if it works or not.

Other than a few writes in FAT_AddDirEntry, are there any times where bytes are written to globalBuffer? I'd like to stick that in VRAM, where writes must be at least 16-bit.
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

Dwedit wrote:

Other than a few writes in FAT_AddDirEntry, are there any times where bytes are written to globalBuffer? I'd like to stick that in VRAM, where writes must be at least 16-bit.

Just it and FAT_DeleteFile. Both of these can be done as a 16 bit write, as it simply sets the first character of the filename to either the end of directory marker (0x00) or the deleted file marker (0xE5). If either of these are set, the rest of that particular directory entry will not matter. Most other writes are copies of DIR_ENT structs, which are 32 bytes in size.

So do the bug fixes work for your card now?

Fseek is broken!
I called it after reading 0x34010 bytes, called fseek(0,16,SEEK_SET), then read data. It read the data from offset 0x34010, not 0x10.
It read 0x1F0 bytes from file offset 0x34010, then 0x10 bytes from file offset 0x170. (I only looked at 512 bytes, but it probably read the rest from that point on)
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

Dwedit wrote:

Fseek is broken!
I called it after reading 0x34010 bytes, called fseek(0,16,SEEK_SET), then read data. It read the data from offset 0x34010, not 0x10.
It read 0x1F0 bytes from file offset 0x34010, then 0x10 bytes from file offset 0x170. (I only looked at 512 bytes, but it probably read the rest from that point on)

That is caused by buffered reads. Insert this code just before the last return statement in FAT_fseek() to reload the buffer:

Code:

// Reload sector buffer for new position in file
CF_ReadSector( openFiles[file].curSect + FAT_ClustToSect(openFiles[file].curClus), openFiles[file].readBuffer);

Is it possible to run homebrew gba games that are larger than 256k with your driver?
Sorry if this was answered earlier in the thread, I did read through the whole thing but cannot remember if it was and I don't really want to look again :)

If it is possible, what would need to be done inorder to get it working ( even if it's an ugly hack ).
I'd really like to try some of the games/demos out there but most are either closed source, too large or are not multiboot.

Lazy1 wrote:

Is it possible to run homebrew gba games that are larger than 256k with your driver?
Sorry if this was answered earlier in the thread, I did read through the whole thing but cannot remember if it was and I don't really want to look again :)

If it is possible, what would need to be done inorder to get it working ( even if it's an ugly hack ).
I'd really like to try some of the games/demos out there but most are either closed source, too large or are not multiboot.

You need the source to the game. The game has to be written specifically for the GBAMP in order to use it. It is near impossible to hack a game that you don't have the source to.

Sorry, I mean about this qoute...

Quote:

What I am working on is a way for homebrew games to load files from the CF card, so they are no longer limited to 256 KiB. There is no way this can be used for commercial ROMs, the M3 will do that natively. And who needs commercial ROM support anyway?

What I wondered is if that is really possible to do, and if so what is needed to make it work.
Unless I didn't understand what you said ( could be, but thats what I get for browsing forums at 3am ( can't sleep ) ).
[/quote]

Yes it is possible to do so. What you need to make it work is to have the source for the game. You then need to include the FAT driver. The game then needs to load files that it needs as it needs them.
Normally all the files that make up a game are compiled into the ROM, so the ROM is made up of not just the code, but things like graphics and level data. Using the driver it is possible for the game to, say, load the level off the CF card rather than from the ROM.

Ah, thanks for clearing that up.
I'll have to see if any open source games can be modified to use this :)

I've just added sram saving support to movie player pocketnes... and now it corrupts the card. Fun fun...
Among the weirdness witnessed was files disappearing, two files with identical names but different case, and a weird folder spontaneously appearing after creating a new text file in windows explorer.
Any ideas on where it's failing this time?

Here's the aforementioned new test version of pocketnes GBAMP:
http://home.comcast.net/~alanweiss3/dwedit/files/pocketnes_test2.zip

I'm working on support for big roms. Right now the PRG only big roms work fine, but some are incredibly slow (Battletoads, Dragon Warrior IV...).
Compressed 256k PRG roms are failing now for some unknown reason. The beginning of each bank decompresses fine, but the data is corrupt by the end of the bank. This didn't happen before when the entire file was first loaded into memory.
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

The SRAM saving works for me, but I guess it would, since its my card that I used when writing the driver.

Does file writing consistantly corrupt the card, or does it work sometimes and fail others? One option you may want to try is creating a dummy file with the same name as the one you intend to save, so that it doesn't need to create the directory entry for it. Another test is to open a new file for writing, but don't write anything and close it again.

Reading is probably never going to be fast enough to support constant streaming of game data (eg repeatedly changing and reloading banks). I don't know if you thought of this, but I think your best option would be to store the ROMs uncompressed, and simply load banks into memory as they are needed, but keep a buffer. When a new bank is loaded into the buffer, the last used one is removed.

Are the corrupted banks being overlapped? This may explain the corruption.

Saving worked fine for me. I have a SanDisk 256mb cf card.

skabio wrote:

Saving worked fine for me.

Same here.
MEGAMAN 4 WORKS!! (well, sort of)... YAHOO!!
I'm also impressed that some other games load (at least to some extent), like TMNT3 & SMB3. Even Kirby's Adventure loads.
Keep up the good work, Dwedit.

EDIT: Maybe I'm not so impressed... I just tried loading SMB 1... it's A LOT harder when half of the objects are invisible! Some other games are plagued by this problem too.

Fixed the typo that killed mario...

http://dwedit.users.tlgaming.net:6080/pocketnes_test2.zip
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

Hmm, I just got a blank screen with that pocketnes version :/
Tried:
RBI Baseball 2
RBI Baseball 3
Baseball stars
Super Mario 3

I could still access the menu but nothing happened, I know its a test version but it looks like other people atleast got it to load :)

I tested the first test2 with zelda and it worked with mp save but it corrupted all the other files on the same folder.
Zelda continued to work fine after a long time with loads and saves, and only the other files on the same folder got corrupted (it's a 256mb cf).
Keep the good work!

Dwedit wrote:

Fixed the typo that killed mario...

Much better!
Like CoolkcaH, I too ended up corrupting a few files on my flash card (because I tried Zelda), but it was restricted to the folder. I had to replace a few of the NES roms, but that's about it.

Looks like ladyada's MintyMP3 project also has some code for reading and writing FAT16 through a compactflash interface. Seems to be based on calling readbyte functions lots of times instead of using 512 byte buffers. also looks like a lot less code than Chism's. No LFNs, no FAT32. I wonder whether it corrupts cards or actually works :)

http://www.ladyada.net/make/minty/download.html (look for the firmware link)
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

Looks like less code, but looks can be deceiving. The code is spread across multiple files, and doesn't implement standard fread, fwrite or fseek. However it does look fairly easy to adapt to the GBAMP. It also won't support CF cards without an MBR. Probably is a lot less buggy than my code, though.

You still haven't replied as to whether the dummy files prevent corruption. Can you please test it out and tell me if it works. If it does, it narrows down the bug.

I've managed sometimes get it working by isolating a single file with its corresponding existing 8k SRAM file into a single folder and use only short names, but still it corrupts the directory sometimes.
And I just tested it after overwriting an 8k file named "Final Fantasy 3.sav"... it rendered the directory unreadable, crashing the directory lister. The movie player simply refused to open the directory.
The first time I ran the firmware dumper, it created a zero byte file, then I ran chkdsk, which fixed it back up to 512k. I ran the firmware updater twice (by accident...) afterwards. Various random files throughout the card became crosslinked and corrupted.

So I guess dummy files aren't eliminating the problem.
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

Alright, now what happens if you open a file for writing then close it again without writing anything?

Also, what mode are you openning the files in?

Quote:

Various random files throughout the card became crosslinked and corrupted.

Did this happen when you ran the firmware updater? Because it should not have written to the card at all. In fact, it shouldn't have read from the card either.

Oopsie, I meant dumper, not updater.

Off Topic:
I think someone needs to relax the restictions on editing your posts immediatly after posting.
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

I've committed my current work to sourceforge devkitpro CVS.
Directory [buildscripts\]tools\nds\bootstrap.
There some README file in there that explains very shortly what is what.
Currently only targeted for NDS, but should allow GBA in the future too.
Suggestions are welcome.

Ah, very good. If your driver works how you said it does it should make GBAMP support even easier. I have just one suggestion. You might not want to use the bootsector of the CF card as a loader. Many people won't know how to install the boot loader if you do. But then again, it is more flexible. The decision is yours.

Currently it doesn't use the MBR for code and it might never use it. I just don't want to remove that stuff from CVS yet. Perhaps it's usable for something else.

Ok, so I've updated my firmware to boot from the CF card using a PassMe, but we are trying to read the CF card in NDS mode. What changes to we need to make? We keep getting "CF_IsInserted: no" when we check. I saw you mentioned changing the gamepak start address, but didn't explain what to change it to.

Ignore that start address rubbish - I didn't know what I was talking about :-). Either run it from the ARM7 or give control of the gampak, with both read and write access, to the ARM9. You will also need to supply the libgba path in your makefile list of lib dirs for the variable types and DMA to work properly.

Last edited by chishm on Sun Jul 24, 2005 3:09 am; edited 1 time in total

chishm wrote:

Ignore that start address rubbish - I didn't know what I was talking about :-). Either run it from the ARM7 or give control of the gampak, with both read and write access, to the ARM9.

Running from the ARM7 would be highly inconvient. What's involved in giving control of the gamepak, with read/write access, to the ARM9?

Check this thread.

damn my movie player is now...dead......black screen......

code did not work...................
hamilton.
Melbourne Australia

Your talking about the firmware update, not the CF Drivers. Anyway, take a look a this page.

ndshb.com page 26

Chishm, I added a pic at the top of the REAL GBA MP.. Are their recovery instructions I could add to the page for people that have either updated the wrong GBA MP, or just didn't follow directions?

Chism:

After a bit of hackery I got your lib working on DS directly from the arm9. I'd appreciate if it didn't need any external libraries though, for type defines etc. I had to remove the two dma calls you have aswell, since it generated header confusion if I tried using the libnds headers together with the other required ones.

Kudos to Chishm and DF.

I'm really liking using the GBAMP for development. However, I have run into some problems with the FAT32 routines.

I'm developing on the Nintendo DS, and am having difficulty reading the directory.

CF_IsInserted returns true now, and FAT_InitFiles returns true as well. I FAT_CWD("/"); and that is successful. However, when I use the Find first/find next, I get the following as short names:

Xmkdos.fs
"tV.
any.to

I've tried formatting the CF from WinXP and Linux, and using Fat 16 and Fat 32. This output was from a CF formatted in Linux for Fat 32. But all results have been similar.

Any ideas of how to proceed? How to debug?

Looks like the RockBox project also has a FAT16/32 driver, and it's GPLed.
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

DesktopMan:
I am going to have to make a few changes to it anyway, so I will try to remove the external dependancies.

PhoenixRising:
I am going to be working on my own project, so I will see if I run into the same problems. As I said before, it turns out a few things need changing to make it compatible with the DS. Make sure the global variables are going in the right memory locations. Also check the values of the disc* global variables to make sure they are correct. If they are not it will cause the sort of trouble you are experiencing. Use WinHex (if you can) to check what the proper values should be.
For example, discBytePerClus should always be 512 when initialised. Only discRootDirCluster can have a value of 0.

Dwedit:
I had a look. There seems to be a lot of assembly use, and not ARM ASM either. Probably not worth porting.

Looks like C to me... Don't mistake the big unicode conversion table for asm.
Looks like everything's in fat.c, dir.c, and file.c. Only supports fully qualified paths though.
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

I was refering to ata.c, but that probably needs replacing anyway to use the CF card. Apart from that, the rest is easy to port. Shouldn't be too hard for you to do :-P .

EDIT: I have moddified gbamp_cf to work in NDS mode. Make sure you #define NDS if you use it on a DS. Download from here.

I have also been running a few tests with it. It is reading at approximately 1.5 MB per second using the full power of the ARM9. This seems to be plenty for a music player, but maybe not for video.

Here's the hacked up project for use with DS.

http://www.auby.no/files/ds/cf.rar

everything works fine so far, I've changed directories, read directories, read files etc.

Chishm: yeah, would be great if it was independent. The above project is pretty dirty :P

I'm still working on it. I want to make sure it works before I release it. (Hehe, first time for everything). Anyway, it is not totally independant, but it can be tailored to work on the NDS with the native ndslib / libnds.

chishm: I really wanted to keep filesystem away from user binaries.
But I think that it won't happen because of your great lib everyone wants to use.
PSP is much better in that respect. Don't forget DS has upgradeable firmware too! (or MoviePlayer or whatever loader).
I got tired already of this MP stuff and feel like I've doing it all for nothing.
And I got loads of other projects too. I'm so slow :(

Darkfader:
I agree that keeping the file system out of the programs is a good idea, allowing upgrades and everything without recompilation & rerelease. However, I am not capable enough to make it as a TSR (old DOS term) binary. If you still want to make it, you can just use whatever code of mine you like in your driver.

Don't worry about being too slow, I had 6 weeks of unforeseen vacation (I was meant to be working, but didn't, long story) that ended only two weeks ago. I needed something to do, and this was that something. Updates will probably slow down now that I'm back at Uni. Don't feel bad about doing it all for nothing. Just remember, you're the Darkfader, the first to run any homebrew code on the DS. Plus, without your help, I wouldn't have gotten file writting or firmware flashing working. So, I guess I just want to say thanks for all you're help so far.

EDIT:
The driver has been updated to work on the NDS without modification. Download from the usual place.

Once a FAT driver is completely bug free and stable, then I'm all for burning it into some kind of rom.
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

Wouldn't be rom, just firmware. So you can update it later on. The frontend from user code needs to be done though.

Chishm:

With any sort of compression 1.5mbyte a sec is alot of bandwidth. Even uncompressed that's about 128 frames a second without sound.

But does the read/write code block execution while seeking/reading the CF card? Or can I run stuff in the background while streaming data from CF?
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

It runs in the foreground, but nothing is stopping you from using interrupts or dma.
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

The question is whether I'll be able to run an audio decoder and game logic alongside the CF access without the audio or game stuttering. Did you mean "using interrupts or dma" for the CF access, or "using interrupts or dma" for my own code? Should I put the entire game logic into a vblank interrupt handler like Super Mario Bros. (NES) does?
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

Read/writes block execution of everything else while they are occuring, which is why I said 1.5MB/s is probably too slow for video. On the DS it would probably be a good idea to use the ARM7 for the driver, leaving the ARM9 free. Communication between the user code and the driver could be handled through the IPC, with an interrupt generated or flag set when it is done.

Unfortunately on the GBA it is too slow to stream constantly from the CF card. In this situation it is probably better to load most of the data between levels or during slow parts (such as a simple cutscene - data could be loaded in the background while it is playing). Although Nintendo consoles are well known for their lack of load times, I'm sure 1-2 seconds between levels is not too bad.

Sorry to bump, but I think this is important enough to warrant it. I have updated the driver with a few bug fixes. If you are using this driver to write files, I advise you to update. Reading is unaffected.

You can get the update from the main page of this thread.

Until all the bugs are fixed, I suggest anyone using this filesystem make this change to the bottom of FAT_WriteFatEntry:

Replace the line at the bottom (before the return)

Code:

fatWriteUpdate = true;

with this:

Code:

if (fatWriteUpdate && (fatCurSector > 0))
{
CF_WriteSector(fatCurSector, fatWriteBuffer);
}
fatCurSector=0;
fatWriteUpdate=false;

It worked for me to prevent any FAT corruption from writes. But I might have been having strange problems. If anyone gets their CF card corrupted from writing, try that change.
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

chishm wrote:

Unfortunately on the GBA it is too slow to stream constantly from the CF card. In this situation it is probably better to load most of the data between levels or during slow parts (such as a simple cutscene - data could be loaded in the background while it is playing). Although Nintendo consoles are well known for their lack of load times, I'm sure 1-2 seconds between levels is not too bad.

But would a 1-2 second freeze within a level be acceptable? I was considering using something built around GSM Player for a game's soundtrack, which would require streaming about 4 KB per second from the cart.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

tepples wrote:

But would a 1-2 second freeze within a level be acceptable? I was considering using something built around GSM Player for a game's soundtrack, which would require streaming about 4 KB per second from the cart.

4KB per second shouldn't take too much CPU away to cause a freeze. Also, a mid level freeze can be unacceptable or unnoticable, depending on the type of game. Eg in an RPG it wouldn't really matter if done at the appropriate time, or even in a platformer, if you had a doorway to separate areas you could get away with it. However, in something like F-Zero (high speed, quick reflex type game) it would kill the experience. That's just my opinion though.

Line 27 of gbamp_cf.h:

Code:

#include <NDS/jtypes.h>

Should be:

Code:

#include <nds/jtypes.h>

For it to compile on linux.
_________________
If a tree falls in the forest and no one is there to hear it, why the heck do you care?

LunarCrisis:
Thanks for the tip. Is this only for libnds, or ndslib too? I'm asking because ndslib has a "NDS" directory for includes, whereas libnds has a "nds" directory.

Quote:

Thanks for the tip. Is this only for libnds, or ndslib too? I'm asking because ndslib has a "NDS" directory for includes, whereas libnds has a "nds" directory.

I'm using libnds, I'm not sure about ndslib.

I ran into another problem compiling, which is that bool is typedef'd both in gbamp_cf.h and in the nds/jtypes.h in libnds. Other than these problems it seems to compile fine.

On another note, I can't seem to get a true return value from FAT_InitFiles(), should I post the source code in this thread?
EDIT: It's conceivable that my CF card is formatted with FAT12, I'm gonna investigate that.
_________________
If a tree falls in the forest and no one is there to hear it, why the heck do you care?

Make sure you set WAIT_CR correctly. For ARM9 I think it is something like

Code:

WAIT_CR &= ~(0x8080)

For ARM7:

Code:

WAIT_CR |= (0x8080)

I have put this in the version I'm currently working on, but it is not released yet.

Looks like libnds is moving further along than ndslib, since the later doesn't define bool while the former now does. The capital/lower case NDS is going to be a pain to workaround, so I guess I'll have to add that to the readme.

chishm wrote:

Make sure you set WAIT_CR correctly. For ARM9 I think it is something like

Code:

WAIT_CR &= ~(0x8080)

For ARM7:

Code:

WAIT_CR |= (0x8080)

I have put this in the version I'm currently working on, but it is not released yet.

Awesome, this fixed it =D.

chishm wrote:

Looks like libnds is moving further along than ndslib, since the later doesn't define bool while the former now does. The capital/lower case NDS is going to be a pain to workaround, so I guess I'll have to add that to the readme.

Yeah, http://www.drunkencoders.com/ lists ndslib as being outdated, so maybe the changes are going straight to libnds instead.
_________________
If a tree falls in the forest and no one is there to hear it, why the heck do you care?

I believe they are. Not only is it a properly named lib (libnds) but it also has cross platform control. Meaning, changes made to it are tested on multiple platforms, as NDSlib were not. And version control, where NDSlib was CVS only, everyone was using a different "version" of NDSlib because it was being updated all the time.

@ Chishm:
SaTa has released a "NDS Memo".You can write via stylus notes and it seems that you can save them to CF card.Now I assume that proper saving means "writing" to CF card.
What do you think?Does it work or am I talking bulls*it as usual?
_________________
Abusing Cube and DS with all sorts of homebrew and hacks.

I haven't had a look, but it should work. My driver can write to the CF card and his is based off of mine.

LunarCrisis wrote:

chishm wrote:

Looks like libnds is moving further along than ndslib, since the later doesn't define bool while the former now does. The capital/lower case NDS is going to be a pain to workaround, so I guess I'll have to add that to the readme.

Yeah, http://www.drunkencoders.com/ lists ndslib as being outdated, so maybe the changes are going straight to libnds instead.

ndslib is now deprecated and all the developers have moved to the devkitPro CVS.

It should be noted that end users should not use CVS and stick with the released stable versions. The CVS code is constantly evolving and will be using the next iteration of the toolchain at some points.

I'm currently working on stdio support which will require devkitARM release 16 and CVS HEAD presently reflects that. I hope to provide a standardised method for inserting device drivers which will ease the use of filesystems like the gbamp CF. This will eventually allow extremely portable file IO.

Okay, I will work with libnds from now on. Did you want this added to the CVS, or is the code too incorrect, nonstandard, etc?

chishm wrote:

Okay, I will work with libnds from now on. Did you want this added to the CVS, or is the code too incorrect, nonstandard, etc?

I'd like to add it in at some point. I'm still researching some aspects of the stdio interface and figuring out how some of the standard filesystem functions are supposed to work.

Updated, changes are:
* Added FAT_GetFileSize - Returns the size of the last file accessed (idea by MoonShine)
* Included automatic memory access control setting for the NDS
* Removed typedef of bool (not needed with libnds)
* Added FAT_GetFileCluster - Returns the start cluster of the last file accessed

Sorry, I keep hearing conflicting reports, are the write functions stable now, or are they still buggy?
_________________
If a tree falls in the forest and no one is there to hear it, why the heck do you care?

Still slightly buggy.

Any part of them in particular, or just keep away from them?
_________________
If a tree falls in the forest and no one is there to hear it, why the heck do you care?

It seems it has problems if it tries to enlarge a fat16 root directory. Also, The FAT cache seemed buggy when I tried it, so I disabled it.
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

FAT16 root directories are not meant to be enlarged. That is the problem. They are limited to 512 entries. FAT32 is fine though. I will have to fix the cache, but I am busy with another project at the moment.

I noticed that the RockBox's FAT driver used a single unified disk cache for everything. Would that be a good idea?
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."

Not with such a low amount of memory and many other things needing to go into it. The FAT cache is currently set up to reduce the number of writes to the FAT area, specifically not to ruin that section too quick. I know that CF cards are meant to spread writes over the entire chip, but I don't want to take that chance.

I currently use 3 caches / buffers:
- A global sector buffer for general filesystem reads.
- File specific sector buffers for unaligned reads and
- A FAT cache for reduced wear and tear, as well as speed increases on disks with small cluster sizes.

Okay, small Errata:
In GBAMP_CF.c, there is a line that reads:

Code:

#define CARD_TIMEOUT 50000

This is too small a timeout, and results in write errors on slow cards. SaTa poitned this out to me a while ago, but I foolishly ignored him. Now I look like an idiot. You should increase this to something like:

Code:

#define CARD_TIMEOUT 10000000 // Updated due to suggestion from SaTa, otherwise card will timeout sometimes on a write

What unit is CARD_TIMEOUT measured in? Perhaps a 200-fold increase might not be as appropriate as maybe a 10-fold increase.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

tepples wrote:

What unit is CARD_TIMEOUT measured in? Perhaps a 200-fold increase might not be as appropriate as maybe a 10-fold increase.

It's basically cycles (or multiples of cycles) as it is just used to break out of a wait loop if it takes too long.

Code:

// Wait until CF card is finished previous commands
i=0;
while ((CF_REG_CMD & CF_STS_BUSY) && (i < CARD_TIMEOUT))
{
i++;
}

A 200 fold increase is probably better, since I have a fairly fast card, and people with slow cards would experience more problems. Don't forget CF cards range from 1x to over 150x, with mine about 60x (I can't remember what it is a multiple of).
Increasing the value won't affect the performance of the program, except to make it wait longer before assuming something is wrong with the CF card.

Thank you for the useful APIs, chishm!
---
I compiled gba_cf.c with devkitARM r16a, and I was warned:

Quote:

gbamp_cf.c: In function 'FAT_GetDirEntry':
gbamp_cf.c:945: warning: 'dir.attrib' is used uninitialized in this function

To avoid warning, some variables (used for return value) should be defined as static.
In the 'FAT_GetDirEntry()', 'DIR_ENT dir' should be 'static DIR_ENT dir'.
In the 'FAT_DirEntFromPath()', 'DIR_ENT dirEntry' should be 'static DIR_ENT dirEntry'.
---
IMHO, these APIs should have an argument for returning DIR_ENT.
If an error has occurred, the function should return 'false' (and don't touch DIR_ENT passed by new arg).
For example,
bool FAT_GetDirEntry ( u32 dirCluster, int entry, int origin, DIR_ENT *dirEntry);

How to use:
DIR_ENT dir;
bool bSuccess;
bSuccess = FAT_GetDirEntry(dirCluster, entry, origin, &dir);
if (!bSuccess) {
// error
}

wwcube wrote:

To avoid warning, some variables (used for return value) should be defined as static.
In the 'FAT_GetDirEntry()', 'DIR_ENT dir' should be 'static DIR_ENT dir'.
In the 'FAT_DirEntFromPath()', 'DIR_ENT dirEntry' should be 'static DIR_ENT dirEntry'.

I'm sorry. I made a mistake in solution.
(I misunderstood that the return value is the pointer to DIR_ENT.)
To solve the problem, just add initialization code to FAT_GetDirEntry().

Code:

*** gbamp_cf.c.orig Sun Sep 4 14:56:26 2005
--- gbamp_cf.c Fri Sep 16 03:10:07 2005
***************
*** 938,943 ****
--- 938,944 ----
u8 lfnChkSum, chkSum;

dir.name[0] = FILE_FREE; // default to no file found
+ dir.attrib = 0x00;

// Check if fat has been initialised
if (discBytePerSec == 0)

Hey Chishm,

made the fgets and fputs for it (needed them for another feature in multiboot) so just grab them if you can need them for the driver.

Code:

/*********************************************************
* char *FAT_fgets(char *tgtBuffer,int num,int file) *
* *
* Implements fgets on the GBAMP *
* CAUTION: does not do strictly streaming. I.e. it's *
* reading more then needed bytes and seeking back. *
* shouldn't matter for random access devices like GBAMP *
*********************************************************/

char *FAT_fgets(char *tgtBuffer,int num,int file) {
// invalid filehandle
if (file<0) return NULL ;
// end of file
if (FAT_feof(file)==true) return NULL ;
// save current position
long curPos = FAT_ftell(file) ;
// read the full buffer (max string chars is num-1 and one end of string \0
int readLength = FAT_fread(tgtBuffer,1,num-1,file) ;
// mark least possible end of string
tgtBuffer[readLength] = 0 ;
if (readLength==0) {
   // return error
   return NULL ;
} ;
// get position of first return '\r'
char *returnChar = strchr(tgtBuffer,'\r') ;
// and mark it, if existant, as end of line/string
if (returnChar!=NULL) {
   *returnChar++ = 0 ;
   if (*returnChar=='\n') { // catch newline too when jumping over the end
      // return to location after \r\n (strlen+2)
      FAT_fseek(file,curPos+strlen(tgtBuffer)+2,SEEK_SET) ;
      return tgtBuffer ;
   } else {
      // return to location after \r (strlen+1)
      FAT_fseek(file,curPos+strlen(tgtBuffer)+1,SEEK_SET) ;
      return tgtBuffer ;
   } ;
} ;

// return to location after \0 (strlen+1)
FAT_fseek(file,curPos+strlen(tgtBuffer)+1,SEEK_SET) ;
return tgtBuffer ;
} ;

/*********************************************************
* int FAT_fputs(const char *string,int file) *
* *
* Implements fputs on GBAMP *
*********************************************************/

int FAT_fputs(const char *string,int file) {
// save string except end of string '\0'
int writtenBytes = FAT_fwrite((void *)string,1,strlen(string),file) ;
// check if we had an error
if (writtenBytes != (int)strlen(string)) {
   // return EOF error
   return -1 ;
} ;
// return the charcount written
return writtenBytes ;
} ;

PS: i did not test it much yet (too less time), but it should catch all errors
_________________
GBAMP Multiboot

Last edited by Mighty Max on Fri Sep 16, 2005 4:48 pm; edited 1 time in total

@ wwcube
Thanks for the help, but I already noticed that. It's corrected in the version I am currently working on, but I want to test it a bit more before releasing it. FAT_GetDirEntry is an internal function used only by the API / driver (whatever it's called), so I have not used poiners since the return value is passed straight to other functions.

@ Mighty Max
I will add those functions when I get time. I am thinking that I should get this put into CVS as soon as I can, so anyone can add useful functions like that. It's too bad I don't know how to set up CVS. Maybe wintermute can help.

As i said, i am writing some code that excessive uses the read & seek functions.

It helped a lot (speedwise) to modify the fseek to not do the full work where it is not needed. I.e.

- seeking short distances (within same sector - check for pos & 0xFFFFFE00 and target & 0xFFFFFE00 are the same) does not need to look for cluster / sector and reread them into buffer.

- within one cluster you don't need to lookup the cluster sequence

- seeking forward in cluster queue does not need to seek from clusterindex0 but from where you already are

That saved me about 20% of time while running the fseek heavy demultiplexer code. I'll bring on the optimizations once they are "clean" to use *g*

PS: A (very) lil' bug: fseek should accept signed position instead of u32 one.
_________________
GBAMP Multiboot

I think I've found a bug in the FAT_fread function. It seems that when I read the first 81 bytes of a file, then read another 8192 bytes, every byte after byte 510 is actually the byte after the one it should be. I wrote a short demo showing the problem, and I'll link it. Just copy the arm9/data/test_png.bin to the root directory of the flash cart, and run the nds file from the flash cart. It shows the values it gets from it's copy in memory and compares them to the values it gets from the CF card.

Here it is: http://topoi.pooq.com/lunarcrisis/gbampbug.tar.bz
_________________
If a tree falls in the forest and no one is there to hear it, why the heck do you care?

Okay, after a lot of bug hunting, I have concluded this "bug" is actually caused by unaligned memory. When you create buffer1 inside the function without using malloc (as a local variable), I am pretty sure it is put on the stack (correct me if I'm wrong). This has caused all manner of problems, not only because it is big, but also because it is an uneven number of bytes long. To fix this, either malloc the space instead or declare it global so its space is assigned at compile time, making it aligned.

The lesson here: Any buffer passed to FAT_fread or FAT_fwrite must either be < 512 bytes big, or aligned to a 16 bit boundary.

How about this patch ?

Code:

*** gbamp_cf.c.orig Mon Sep 19 10:18:20 2005
--- gbamp_cf.c Wed Sep 21 22:13:34 2005
***************
*** 389,398 ****
_CODE_IN_RAM bool CF_ReadSector (u32 sector, void* buffer)
{
int i;
! #ifndef _CF_USE_DMA
! u16 *buff;
! #endif
!
// Wait until CF card is finished previous commands
i=0;
while ((CF_REG_CMD & CF_STS_BUSY) && (i < CARD_TIMEOUT))
--- 389,395 ----
_CODE_IN_RAM bool CF_ReadSector (u32 sector, void* buffer)
{
int i;
!
// Wait until CF card is finished previous commands
i=0;
while ((CF_REG_CMD & CF_STS_BUSY) && (i < CARD_TIMEOUT))
***************
*** 431,444 ****
   return false;

// Read data
#ifdef _CF_USE_DMA
! DMA3COPY ( CF_DATA, buffer, 256 | DMA16 | DMA_ENABLE | DMA_SRC_FIXED);
#else
! i=256;
! buff= (u16*)buffer;
!    while(i--)
!    *buff++ = *CF_DATA;
#endif
return true;
}

--- 428,451 ----
   return false;

// Read data
+ if ((u32)buffer & 1) {
+    u8 *p = buffer;
+    i = 256;
+    while (i--) {
+       u16 d = *CF_DATA;
+       *p++ = d & 0xff; *p++ = (d >> 8) & 0xff;
+    }
+ } else {
#ifdef _CF_USE_DMA
!    DMA3COPY ( CF_DATA, buffer, 256 | DMA16 | DMA_ENABLE | DMA_SRC_FIXED);
#else
!    u16 *buff;
!    i = 256;
!    buff= (u16*)buffer;
!    while(i--)
!       *buff++ = *CF_DATA;
#endif
+ }
return true;
}

***************
*** 453,462 ****
_CODE_IN_RAM bool CF_WriteSector (u32 sector, void* buffer)
{
int i;
! #ifndef _CF_USE_DMA
! u16 *buff;
! #endif
!
// Wait until CF card is finished previous commands
i=0;
while ((CF_REG_CMD & CF_STS_BUSY) && (i < CARD_TIMEOUT))
--- 460,466 ----
_CODE_IN_RAM bool CF_WriteSector (u32 sector, void* buffer)
{
int i;
!
// Wait until CF card is finished previous commands
i=0;
while ((CF_REG_CMD & CF_STS_BUSY) && (i < CARD_TIMEOUT))
***************
*** 495,508 ****
   return false;

// Write data
#ifdef _CF_USE_DMA
! DMA3COPY( buffer, CF_DATA, 256 | DMA16 | DMA_ENABLE | DMA_DST_FIXED);
#else
! i=256;
! buff= (u16*)buffer;
!    while(i--)
!    *CF_DATA = *buff++;
#endif

return true;
}
--- 499,523 ----
   return false;

// Write data
+ if ((u32)buffer & 1) {
+    u8 *p = buffer;
+    i = 256;
+    while (i--) {
+       u16 d;
+       d = *p++; d |= (*p++ << 8);
+       *CF_DATA = d;
+    }
+ } else {
#ifdef _CF_USE_DMA
!    DMA3COPY( buffer, CF_DATA, 256 | DMA16 | DMA_ENABLE | DMA_DST_FIXED);
#else
!    u16 *buff;
!    i = 256;
!    buff= (u16*)buffer;
!    while(i--)
!       *CF_DATA = *buff++;
#endif
+ }

return true;
}

Sorry about the long delay in replying, I forgot. Anyway, I will add an optional patch to allow unaligned buffers. It will probably be similar to wwcube's method.
_________________
http://chishm.drunkencoders.com
http://dldi.drunkencoders.com

Ok, I've added MightyMax' fgets and fputs code. I made a few changes, commented in the source.
_________________
http://chishm.drunkencoders.com
http://dldi.drunkencoders.com

gbadev.org forum archive

Hardware > GBAMP CF FAT file driver

#45595 - chishm - Sun Jun 12, 2005 3:07 pm

#45596 - chishm - Sun Jun 12, 2005 3:08 pm

#45598 - tepples - Sun Jun 12, 2005 4:14 pm

#45675 - vorpalarrow - Mon Jun 13, 2005 7:06 pm

#45709 - chishm - Mon Jun 13, 2005 11:55 pm

#45740 - chishm - Tue Jun 14, 2005 3:15 pm

#45745 - tepples - Tue Jun 14, 2005 3:43 pm

#45792 - chishm - Tue Jun 14, 2005 11:56 pm

#45809 - chishm - Wed Jun 15, 2005 9:17 am

#45812 - darkfader - Wed Jun 15, 2005 12:53 pm

#45820 - chishm - Wed Jun 15, 2005 2:54 pm

#45825 - darkfader - Wed Jun 15, 2005 4:07 pm

#45848 - tepples - Wed Jun 15, 2005 8:30 pm

#45887 - chishm - Thu Jun 16, 2005 12:17 am

#45898 - darkfader - Thu Jun 16, 2005 8:45 am

#45913 - Dwedit - Thu Jun 16, 2005 5:14 pm

#45935 - darkfader - Thu Jun 16, 2005 11:56 pm

#45943 - Dwedit - Fri Jun 17, 2005 12:47 am

#45948 - tepples - Fri Jun 17, 2005 12:57 am

#45957 - chishm - Fri Jun 17, 2005 4:27 am

#45958 - Dwedit - Fri Jun 17, 2005 4:30 am

#45971 - chishm - Fri Jun 17, 2005 11:47 am

#46009 - kouray - Sat Jun 18, 2005 2:08 am

#46024 - zubiac - Sat Jun 18, 2005 10:05 am

#46026 - chishm - Sat Jun 18, 2005 10:32 am

#46029 - tepples - Sat Jun 18, 2005 3:16 pm

#46045 - zubiac - Sat Jun 18, 2005 9:56 pm

#46046 - Dwedit - Sat Jun 18, 2005 9:59 pm

#46047 - Dwedit - Sat Jun 18, 2005 10:09 pm

#46052 - Dwedit - Sat Jun 18, 2005 11:53 pm

#46056 - chishm - Sun Jun 19, 2005 1:55 am

#46058 - Dwedit - Sun Jun 19, 2005 3:22 am

#46067 - chishm - Sun Jun 19, 2005 11:50 am

#46074 - Dwedit - Sun Jun 19, 2005 4:13 pm

#46096 - chishm - Mon Jun 20, 2005 12:04 am

#46100 - tepples - Mon Jun 20, 2005 3:02 am

#46102 - wintermute - Mon Jun 20, 2005 4:09 am

#46103 - Dwedit - Mon Jun 20, 2005 4:18 am

#46105 - tepples - Mon Jun 20, 2005 6:04 am

#46118 - chishm - Mon Jun 20, 2005 1:51 pm

#46134 - jmr - Mon Jun 20, 2005 5:57 pm

#46140 - Dwedit - Mon Jun 20, 2005 6:56 pm

#46258 - chishm - Wed Jun 22, 2005 12:22 pm

#46271 - Dwedit - Wed Jun 22, 2005 5:43 pm

#46313 - darkfader - Wed Jun 22, 2005 11:46 pm

#46315 - chishm - Thu Jun 23, 2005 12:43 am

#46316 - Dwedit - Thu Jun 23, 2005 2:37 am

#46333 - chishm - Thu Jun 23, 2005 11:35 am

#46394 - chishm - Fri Jun 24, 2005 3:03 am

#46395 - skabio - Fri Jun 24, 2005 3:18 am

#46400 - Dwedit - Fri Jun 24, 2005 6:45 am

#46403 - chishm - Fri Jun 24, 2005 7:31 am

#46437 - mrnull - Fri Jun 24, 2005 7:28 pm

#46452 - darkfader - Sat Jun 25, 2005 2:12 am

#46454 - Dwedit - Sat Jun 25, 2005 3:43 am

#46459 - chishm - Sat Jun 25, 2005 8:59 am

#46464 - darkfader - Sat Jun 25, 2005 3:53 pm

#46467 - gbaplaya05 - Sat Jun 25, 2005 4:32 pm

#46468 - gbaplaya05 - Sat Jun 25, 2005 5:14 pm

#46486 - chishm - Sun Jun 26, 2005 1:15 am

#46502 - YodaNC - Sun Jun 26, 2005 11:14 am

#46503 - chishm - Sun Jun 26, 2005 11:24 am

#46504 - YodaNC - Sun Jun 26, 2005 11:26 am

#46519 - king yoda123 - Sun Jun 26, 2005 5:58 pm

#46524 - jmr - Sun Jun 26, 2005 7:50 pm

#46536 - chishm - Mon Jun 27, 2005 12:39 am

#46539 - jmr - Mon Jun 27, 2005 1:17 am

#46540 - chishm - Mon Jun 27, 2005 1:52 am

#46554 - chishm - Mon Jun 27, 2005 9:21 am

#46556 - Vince - Mon Jun 27, 2005 10:01 am

#46557 - chishm - Mon Jun 27, 2005 11:32 am

#46575 - wintermute - Mon Jun 27, 2005 4:24 pm

#46606 - gbaplaya05 - Tue Jun 28, 2005 12:12 am

#46616 - mrnull - Tue Jun 28, 2005 4:25 am

#46617 - chishm - Tue Jun 28, 2005 4:41 am

#46622 - Dwedit - Tue Jun 28, 2005 7:37 am

#46657 - jmr - Tue Jun 28, 2005 6:17 pm

#46670 - skabio - Tue Jun 28, 2005 10:50 pm