#101230 - MrD - Sun Sep 03, 2006 11:39 pm
I keep reading that all of the data and program code should fit into the DS' 4MB of RAM.
Why?
_________________
Not active on this forum. For Lemmings DS help see its website.
#101231 - tepples - Sun Sep 03, 2006 11:40 pm
Where would you store things, if not in RAM?
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#101233 - MrD - Sun Sep 03, 2006 11:43 pm
Well... the first guess would be 'in the ROM'. :)
But I guess it's not that simple...?
Edit - 'in the ROM', loaded from SLOT-1 or 2.
_________________
Not active on this forum. For Lemmings DS help see its website.
#101234 - gmiller - Sun Sep 03, 2006 11:57 pm
Well, code can be read-only (unless the code is self modifying) so being on a ROM would be ok for code but not for read/write variables. Read only variables could be in ROM. One thing you need to know is that ROM generally is not very fast for non-sequencial reading so any jumping around is VERY slow but sequencial is faster. Generally standard read-write memory is MUCH faster than ROM. Of course these are general comments and the hardware architecture significantly impacts the way you use the memory. Each of the procesors (ARM7 and the ARM9) has ways of accessing memory and has to obey those rules. The ARM9 processor that follows the Winchester model, not the Von Neumann model as the ARM7 does, has more instructions and supported data type than the ARM7. The way the DS is built and it's ability to execute GBA code on the ARM7 only impact the way each of the CPU's is designed to work. There is a lot more but the other documents around could describe this better.
#101235 - tepples - Mon Sep 04, 2006 12:06 am
You can load chunks of data called "files" from ROM (or flash or whatever) into RAM while your program is running using a library called libfat. This will let your program access more than 4 MB of data, but no more than 4 MB can fit in RAM at once.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#101237 - MrD - Mon Sep 04, 2006 12:12 am
Hmm... I got the impression that the entire rom file kaboodle had to be less than 4MB in size because it all goes in the RAM (I don't know why). (Contrast with GBA style lumping it altogether into a single < 32Mbyte file (ignoring bankswitching))
So, I guess I got the wrong impression then?
_________________
Not active on this forum. For Lemmings DS help see its website.
#101238 - tepples - Mon Sep 04, 2006 12:14 am
If you're using FlashMe + WMB, or you're using a single .nds file without libfat, then yes, it has to be under 4 MB. But if you're using a CF or SD card, you can read other files on the card into RAM while the program is running.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#101239 - MrD - Mon Sep 04, 2006 12:15 am
Right, gotcha. So why is that?
Is it purely because it's faster, as gmiller said?
_________________
Not active on this forum. For Lemmings DS help see its website.
#101246 - tepples - Mon Sep 04, 2006 12:36 am
Your program storage (a DS Game Card, a CF card, or an SD card) is a lot like a disk. A PC can't execute code straight from a hard disk; it has to load the code into RAM first.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#101248 - MrD - Mon Sep 04, 2006 12:38 am
Hmm... so basically 'it's just the DS way', then.
Okie doke.
_________________
Not active on this forum. For Lemmings DS help see its website.
#101250 - AnalogMan - Mon Sep 04, 2006 1:01 am
No.... it's the Computer way. :)
When using libfat, you can pull pieces from the storage card (like a hard drive for the DS) and put that in RAM. Exactly like a computer.
Now if you don't have libfat, you need to load the whole thing, so it's limited to 4MB of RAM since that's all the DS has, just like if you were to load a picture on the computer that was 1 gig and you only have 256MB you'll have issues.
So the DS follows the same laws as a PC and the PSP.
#101255 - MrD - Mon Sep 04, 2006 1:36 am
Well, why does the GBA have maximum single-file ROM size of 32MBytes?
_________________
Not active on this forum. For Lemmings DS help see its website.
#101261 - dantheman - Mon Sep 04, 2006 2:11 am
I apologize in advance if this post contains inaccurate information.
From what I understand, the 32 MB limit is just a limit on the amount of space the 16-bit GBA can address without bankswitching. Theoretically, a GBA binary could be larger than 32 MB, but it would be specific to a single type of flash cart, since each flash cart handles bankswitching differently.
I believe that it would be possible for a commercial game to use more than 32 MB through bankswitching as well, but no game has ever exceeded that limit due to the fact that the NOR memory the GBA uses is more expensive than the cheaper NAND memory used for DS game carts and devices like the GBAMP, M3, and Supercard.
#101262 - tepples - Mon Sep 04, 2006 2:12 am
I wrote a very nice write-up on the difference between a block device and a word-addressed memory, but the administrators of this forum don't want anybody to directly link to the site that it was posted on because other pages on the same domain describe how to copy software illegally. I'll summarize here and then copy my writeup to my own web space where it may be more useful to others:
Word-addressed memory
The hardware must seek to a given 2-byte block (called a "word") and the CPU starts reading instructions or data from there. RAM is word-addressed memory, and so is ROM in many cartridge-based video game systems.
Block device
The hardware can take arbitrarily long to seek to a given 512-byte block and sends the whole block to the I/O chips in a spurt. Disk is a block device, and so are CompactFlash, SmartMedia, Memory Stick, Secure Digital, and DS Game Card.
The advantage of word-addressed memory is that the CPU can seek to and read individual instructions and individual bytes of data. The disadvantage is higher price, which is why many architectures have shifted from word-addressed mass storage to copying between RAM and a block device. Ever notice how GBA Game Paks never broke 256 Mbit during the life of the system, but DS Game Cards are already up to 1024 Mbit less than two years in? That's a block device. Ever notice how a 1 gigabyte (8000 Mbit) CF or SD memory card costs $40 at an office supply store, but a 32 megabyte (256 megabit) GBA card costs at least $60? CF and SD are block devices.
Wikipedia has a thing or two to say about [[block storage]].
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#101266 - MrD - Mon Sep 04, 2006 2:38 am
Aha. Thanks, tepples. I think I understand now.
_________________
Not active on this forum. For Lemmings DS help see its website.
#101267 - HyperHacker - Mon Sep 04, 2006 2:54 am
Technically you can store your program in the ROM if you use a GBA flash cart. You might get it working on Supercard and M3 as well since they use 32MB of RAM to emulate this ROM. It won't work on any slot-1 adapter or any slot-2 adapter that lacks this RAM (GBAMP for example) though (nor WMB), so it's not a popular method.
_________________
I'm a PSP hacker now, but I still <3 DS.
#101284 - chishm - Mon Sep 04, 2006 6:39 am
| dantheman wrote: |
| I believe that it would be possible for a commercial game to use more than 32 MB through bankswitching as well, but no game has ever exceeded that limit due to the fact that the NOR memory the GBA uses is more expensive than the cheaper NAND memory used for DS game carts and devices like the GBAMP, M3, and Supercard. |
Commercial GBA carts use a Mask ROM, not NOR nor NAND memory. Flash cartridges use NOR (or RAM) for the primary storage and NAND for secondary storage.
_________________
http://chishm.drunkencoders.com
http://dldi.drunkencoders.com
#101285 - dantheman - Mon Sep 04, 2006 6:50 am
Ah, see I knew I was spouting incorrect info there. Thank you for the correction.
#101952 - sajiimori - Sat Sep 09, 2006 5:09 am
Here is a more proximate reason to complement the more ultimate reasons already given (speed and cost).
The ROM on the DS is not memory mapped, which means there are no memory addresses that correspond to locations in the ROM. On the GBA, ROM addresses start at 0x08000000, but on the DS, there is no such corresponding address for the card's ROM.
Are you familiar with the concept of a program counter, or equivalently, an instruction pointer? It's an important register in the CPU that holds the address of the next instruction to be executed (or sometimes a little ahead, depending on the architecture). A "goto" command essentially means storing a new address in the program counter.
So, the reason the DS has to execute from RAM is that the ROM is not memory mapped, so you can't set the program counter to point at it.
#101958 - Turambar - Sat Sep 09, 2006 6:05 am
| sajiimori wrote: |
Here is a more proximate reason to complement the more ultimate reasons already given (speed and cost).
The ROM on the DS is not memory mapped, which means there are no memory addresses that correspond to locations in the ROM. On the GBA, ROM addresses start at 0x08000000, but on the DS, there is no such corresponding address for the card's ROM.
|
How do commercial games access the ROM, if it's not memory mapped?
#101959 - chishm - Sat Sep 09, 2006 6:07 am
| Turambar wrote: |
| How do commercial games access the ROM, if it's not memory mapped? |
The same way a PC program accesses from the HDD -- using provided routines to read files from the block device.
_________________
http://chishm.drunkencoders.com
http://dldi.drunkencoders.com
#101960 - HyperHacker - Sat Sep 09, 2006 6:14 am
...GBA ROM space is still mapped to 0x08000000 on the DS... O_o
_________________
I'm a PSP hacker now, but I still <3 DS.
#101965 - chishm - Sat Sep 09, 2006 8:26 am
| HyperHacker wrote: |
| ...GBA ROM space is still mapped to 0x08000000 on the DS... O_o |
sajimori said "there is no such corresponding address for the card's ROM", implying the DS card and not the GBA cart
_________________
http://chishm.drunkencoders.com
http://dldi.drunkencoders.com
#101972 - PypeBros - Sat Sep 09, 2006 9:59 am
so, if i try to sum it up:
* the GBA cartridges (official ones) have some ROM directly mapped (accessible by both ARM7 and ARM9 on a DS), located at 0800:0000h (GBA Slot ROM, max. 32MB)
* ARM7 could run code from there but it's much more advised to run code from the ARM7 ram instead (0A00:0000, that's 64K, right?) for performance reason.
* ARM9 has 4MB of RAM where code & data can be placed.
* official DS cartridges that contain more ROM still have to move data in RAM (or in VRAM, if textures, i guess) before they can use it, though i haven't read of the specific mechanism anywhere (is that a BIOS function?)...
* homebrew DS code running from the GBA slot may access the faked-gba ROM directly, but not for running code (restrictions on PC values?), though using Supercard/GBAMP/M3/whatever-specific code, they can read additional dataset from the flash storage (e.g. using chishm's libfat driver), a bit like official DS cartridges load stuff from the DS ROM.
4MiB of RAM as "resident set" ... coming from PC programming, that sounds terribly challenging...
_________________
SEDS: Sprite Edition on DS :: modplayer
#101984 - gmiller - Sat Sep 09, 2006 2:33 pm
From a typical embedded system this is pretty simple and the tools you have are good at helping you follow some of these 'rules'. Suffice it to say the CPU has a Fetch-Decode-Execute cycle that requires the code/data be addressable within the address space the CPU can access.
#101990 - tepples - Sat Sep 09, 2006 4:37 pm
| HyperHacker wrote: |
| ...GBA ROM space is still mapped to 0x08000000 on the DS... O_o |
And apart from Nintendo DS Browser, which will appear in Europe four weeks from yesterday, which DS games come with anything to put in the GBA slot?
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#101999 - Turambar - Sat Sep 09, 2006 6:14 pm
| PypeBros wrote: |
* homebrew DS code running from the GBA slot may access the faked-gba ROM directly, but not for running code (restrictions on PC values?), though using Supercard/GBAMP/M3/whatever-specific code, they can read additional dataset from the flash storage (e.g. using chishm's libfat driver), a bit like official DS cartridges load stuff from the DS ROM. |
Correct me if I'm wrong, but I think you can run code from the GBA ROM, since it is memmory mapped. Isn't this why you can have programs up to 32MB if you have a flashcart with 32MB of RAM?
#102000 - tepples - Sat Sep 09, 2006 6:41 pm
| Turambar wrote: |
| Correct me if I'm wrong, but I think you can run code from the GBA ROM, since it is memmory mapped. |
Yes, and ds.gba binaries run the ndsloader from GBA ROM. But for reasons of speed (internal RAM is faster than GBA ROM) and universality (execution from internal RAM works with both ROM and FAT configurations), most DS programs copy the executable code into RAM first.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#102001 - sajiimori - Sat Sep 09, 2006 7:04 pm
On a PC with similar processing power (say, a 486 DX2/66), I might feel restricted with less than 8 MB. It's not so bad on the DS, though.
One reason is that PC games at the time ran mostly in 8 bit video modes, and it was desirable to store all graphics in that format for fast drawing. On the DS, it's possible to mix formats, and there are compressed formats, so the savings can be significant.
And of course, textures only have to be stored in VRAM, so you really have closer to 5 MB. (Not sure of the exact number off the top of my head.)
But maybe most importantly, the card may be slow, but it's nowhere near as slow as a hard drive. On my last game, animations were the biggest assets and they couldn't all fit in RAM, so we loaded them as they were needed. It dropped a frame now and then if several characters changed animations at the same time, but performance was acceptable.
(To diffuse the cost of switching animations, our next game will stream animations a frame at a time.)
#102681 - MrD - Sat Sep 16, 2006 1:59 am
So... Here's my next question:
What would be the most compatible way of organising a homebrew game with lots of large loadable resources?
I'd like it to be able to run off a standard GBA flashcart, and I'd guess that compatibility with all other DS specialist gadgets like the Supercard, M3 and G6 would just follow on from this. I'd like to avoid having to use seperate final ROMs for the different devices, but that seems like it might be unavoidable.
Currently I've got large const arrays compiled in, and various other bin2o'd knick-knacks linked to the arm9 binary. For after-compilation addition of extra resources, I've appended them to the end of the final .ds binary. (although I have read that this is a very bad idea as although it will appear in the 0x08000000 range and work, it will react strangely with some global variables...? Forgive me... that was some time ago, and it was very vague)
_________________
Not active on this forum. For Lemmings DS help see its website.
#102696 - DekuTree64 - Sat Sep 16, 2006 7:40 am
I'd recommend designing it for the block devices (GBAMP, supercard, etc.) rather than GBA flash carts, since you can't access the data on a block device directly the way you can with a flash cart, but you can copy data from the flash cart to RAM similarly to loading data from a block device.
So...basically, store your data as files that you put on the CF/SD card for the block device, and load those when you need them. No giant const arrays in the binary.
Or if you want to keep the card less cluttered, you can string all your small files together into one big file, and store an offset table at the start of it. Then when you want to load a file, you just read a section of the big one.
_________________
___________
The best optimization is to do nothing at all.
Therefore a fully optimized program doesn't exist.
-Deku
#102749 - sajiimori - Sat Sep 16, 2006 5:55 pm
Making a binary that runs on both a GBA cart and a DS card is a slight hack. You have to run different startup code depending on what you're running on.
#102819 - MrD - Sun Sep 17, 2006 7:18 am
| sajiimori wrote: |
| Making a binary that runs on both a GBA cart and a DS card is a slight hack. You have to run different startup code depending on what you're running on. |
I'm confused: when you say 'DS card', are you referring to running the game from SLOT-1?
It seems to me that the DS card block rules only apply to SLOT-1, so I don't really need to worry about it if I only use GBA style code and such-such devices. I think it's best that I keep on as I was, using a single GBA style ROM.
_________________
Not active on this forum. For Lemmings DS help see its website.
#102840 - sajiimori - Sun Sep 17, 2006 9:56 am
If SLOT-1 is the card slot, then yeah. :P
I got the impression that you wanted to make a ROM that would run from either a GBA cartridge or a DS card. If you only want to support GBA cartridges, then by all means, take advantage of the capabilities that allows, namely executing code and using data straight from ROM.
#102864 - tepples - Sun Sep 17, 2006 3:11 pm
I can see a few reasons why we have standardized on copying code from GBA ROM to RAM: - The GBA Movie Player v2 CF card has been hacked and is more affordable than M3.
- FlashMe + WMB was popular for a while before the new firmware (late Q3 2005) discouraged the purchase of Ralink cards as PassMe replacements.
- Many of us want to work under similar restrictions to those of official developers so that if we get hired, we don't have to re-learn everything. Randomly accessing the GBA ROM isn't something that official games, which run from a DS card, are allowed to do.
But if these don't apply, go ahead and make a program that executes from or otherwise randomly accesses GBA ROM, but prepare for whining from GBAMP owners.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#102869 - MrD - Sun Sep 17, 2006 3:56 pm
Groovy, mission complete. ^_^
_________________
Not active on this forum. For Lemmings DS help see its website.
#112311 - MrD - Fri Dec 15, 2006 4:21 am
So... past self,
If you're going to try to do this, don't use const arrays in the ARM9 side. That doesn't work.
You can use GBFS, but that does mean changing... everything.
_________________
Not active on this forum. For Lemmings DS help see its website.
#112374 - Quirky - Fri Dec 15, 2006 8:45 pm
GBFS used pointers into ROM on the GBA, so it would have similar problems to using const arrays in code, wouldn't it? Or does the NDS version of GBFS use libfat under the hood?
Using libfat on old GBA carts - e.g. a Flash 2 Advance from cerca 2001 - will code that loads assets using libfat on these carts also run on GBAMP, CF etc? Assuming read-only loading files here.
Finally: is SRAM memory mapped for all cards? i.e. SRAM[0] = 1; does what it did on the GBA and writes a 1 to the first byte of SRAM? Or does one have to use libfat here too, writing files to the CF or SD card?
#112376 - tepples - Fri Dec 15, 2006 8:54 pm
Libfat and GBFS are completely separate libraries, but they can be made to work together: - Use bin2o to convert your .gbfs file to an object file and link it into your .nds file, and use that instead of find_first_gbfs_file().
- Load an entire .gbfs file from FAT into RAM, and use that instead of find_first_gbfs_file().
- Make an extension that uses gba_nds_fat or libfat to read objects from .gbfs files into RAM. Classic Mac OS's "resource fork" system uses this.
- Make an extension that mounts a .gbfs file in ROM as a devoptab file system, just as libfat mounts the CF or SD card. I might do this once R20 comes out, as only R20 would let me gbfs_get_objn().
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#112384 - Sunray - Fri Dec 15, 2006 9:25 pm
I use GBFS as a resource/archive file (game.bin). But I also support external files. Basically, it first check for a file in the archive file, if it wasn't found it tries to open an external file. It's completely abstract too, very nice too use.
| Code: |
ScopedPointer<InputStream> Stream = FileSystem::OpenFile("foo.file");
if (Stream.IsValid())
Stream->Read(buffer, 1024);
|