#175837 - brave_orakio - Thu Feb 17, 2011 9:03 am
Out of curiosity has anyone ever encountered a situation where the 4MB of the NDS wasn't enough?
Specifically I mean that you were required to manually copy(Don't know if that should be the term to use there?) game code to RAM because there's so much game code around (Or you use the RAM as a buffer leaving only a little space for game code to run) and NDS requires game code to run from RAM.
On that note can NDS execute code from ROM like the GBA? Or even if its possible is it ridiculously slow to even bother thinking about it?
_________________
help me
#175838 - sverx - Thu Feb 17, 2011 11:55 am
4MB of CODE? What can be so big? ;)
Seriously... I remember once I read on this forum considerations about moving code in/out from RAM, so you could try searching...
#175839 - elhobbs - Thu Feb 17, 2011 2:30 pm
If you have 4MB of code then you have no room for data. If this is the case then your project is not a good fit for the ds.
in regards to your second question - where is this rom that you want to run from? if you are making homebrew then you are most likely loading something through dldi which is not rom nor is it directly addressable as memory. the only "extra" place to put code that I am aware of is on a slot 2 ram expansion card. not many people have these devices and it is tricky getting the code onto the device but is technically possible. quake2 ds makes use of these devices. it is not an option for dsi since it is lacking a slot 2.
however, dsi slot 1 device are starting to appear so you could make it dsi only. aside from that, if you are in this situation and you are asking the question then you probably need to consider a different platform for your project. or scale it back to fit on the ds.
#175840 - Miked0801 - Thu Feb 17, 2011 5:30 pm
Code Overlays loaded from ROM to RAM happen quite often in commerical development. And no, I would never want to attempt to execute code from ROM. A 1-bit serial interface is not a good way to run code. You don't even want to load data too much from ROM for that matter - too slow. We cache heavily used data or our framerate tanks.
#175841 - brave_orakio - Fri Feb 18, 2011 2:00 am
| Quote: |
| 4MB of CODE? What can be so big? ;) |
Hahaha! No not necessarily pure code, like I put in the parenthesis it might be used for buffering as well and the buffered data might be big. Or maybe some awesome self aware AI? >:-D
I think I was the one who asked that question as well back then, but that was for GBA.
| Quote: |
| in regards to your second question |
Nothing special, just from slot 1 like a normal DS game. But I guess Mike already answered my question.
Thanks Mike! So in DS development, the 4MB RAM seems to be heavily used(Code and data cache) as compared to the 256KB of the GBA. For single player GBA games at least anyway.
Its just a data gathering question, my project isn't that big yet. I thought I might be able to plan around while I update my tools for DS development.[/quote]
_________________
help me
#175843 - gameporter - Fri Feb 18, 2011 2:33 am
You could go DSi and get 16 MB!
#175846 - brave_orakio - Fri Feb 18, 2011 5:43 am
Hahahaha! I guess that is one way to go around that!
_________________
help me
#175848 - sverx - Fri Feb 18, 2011 9:57 am
BTW you can have .nds files bigger than 4MB, if you meant that. (WarHawk DS is an example). Just you won't load everything into RAM, just the code and some data, the rest will be loaded when needed and discarded when no longer needed, or even streamed -if it's music- (again I'm thinking at WarHawk DS).
Of course if your code plus all your data can be easily fit into the 4MB RAM, it's easier to link your data directly into the program (and many homebrew game do this, so this won't even need any kind of filesystem access)
#175866 - brave_orakio - Mon Feb 21, 2011 1:56 am
I see thanks. But I'll have to do those manually right? Meaning I have to tell the program when to load this piece of code or this piece of data? I think I'll do some research on that here, I remember asking something the same about the GBA a few months back.
_________________
help me
#175869 - sverx - Mon Feb 21, 2011 11:23 am
'Manually', yes, speaking about data. You have to access the filesystem [fopen(), fread(), fclose()] and read data to RAM. Of course you could write functions to 'automate' that when needed...
#175871 - wintermute - Mon Feb 21, 2011 5:12 pm
| Miked0801 wrote: |
| Code Overlays loaded from ROM to RAM happen quite often in commerical development. And no, I would never want to attempt to execute code from ROM. A 1-bit serial interface is not a good way to run code. You don't even want to load data too much from ROM for that matter - too slow. We cache heavily used data or our framerate tanks. |
Surely you mean 8bit here?
Cheeky question, not sure if an answer might run foul of NDAs but ... do you guys have some easy way to generate overlays? ELF support was added to ndstool a while ago which should allow us to pull sections out of the ELF and use them as overlays.
Currently I'm at a bit of a loss as to how to make generating the sections easy for the programmer. I guess this could be done by naming the source files in a particular way - for instance an entire object file can be placed in itcm by naming the input source as .itcm.c/cpp but it seems a bit awkward to do this for overlays.
Some ideas would be much appreciated.
_________________
devkitPro - professional toolchains at amateur prices
devkitPro IRC support
Personal Blog
#175873 - brave_orakio - Tue Feb 22, 2011 5:47 am
Yes, the normal way we do it around here would probably be not applicable for overlays.
I can't find my topic about this before though(for the GBA but it should be similar), and I think Mike was the one who answered it too.
edit: Sorry it was kusma who had an example for code overlays for the GBA.
The example can be found here http://forum.gbadev.org/viewtopic.php?t=14619 for those interested.
edit2: I have very little idea of whats happening in that piece of code that kusma gave. Can anyone give an explanation on whats happening there?
_________________
help me
#175874 - kusma - Tue Feb 22, 2011 9:13 am
| brave_orakio wrote: |
| I have very little idea of whats happening in that piece of code that kusma gave. Can anyone give an explanation on whats happening there? |
There's three iwram overlay-sections, "iwram0", "iwram1" and "iwram2", with "iwram0" being the section that is loaded by default. The symbols __load_start_iwram0, __load_stop_iwram0 etc are automatically output by the linker (thanks to the link-script) when these sections exist. The main-function is in rom, so I can use memcpy from it to switch which iwram section is the currently "valid" one.
__iwram_overlay_start defines where the overlay-area starts. You can use the normal "iwram"-section to put things that should always be in iwram; __iwram_overlay_start should point to somewhere right after the "iwram"-section.
Does this explanation make it a bit clearer?
#175875 - brave_orakio - Tue Feb 22, 2011 9:57 am
I see. The __load_start stuff is understood automatically by the compiler so long as the sections iwram1, 2... etc is declared and start stop will determine the size of the code to copied. But how about the allocated size for the code though? How would that be handled? Say when overlay1_test and overlay2_test are not the same size.
_________________
help me
#175876 - kusma - Tue Feb 22, 2011 10:08 am
| brave_orakio wrote: |
| I see. The __load_start stuff is understood automatically by the compiler so long as the sections iwram1, 2... etc is declared and start stop will determine the size of the code to copied. |
Close. It's understood automatically by the linker.
| Quote: |
| But how about the allocated size for the code though? How would that be handled? Say when overlay1_test and overlay2_test are not the same size. |
Exactly the size of each function in each section does not matter, it's the maximum size of all overlay-sections that matters. IWRAM is of fixed size. If sizeof(iwram) + max(sizeof(iwram1), sizeof(iwram2), sizeof(iwram3), ...) is above that size, the linker will give you an error in the same way as if sizeof(iwram) was above the size of IWRAM.
#175877 - Miked0801 - Tue Feb 22, 2011 5:23 pm
Yes - a 16-bit (or was it 8?) serial interface for GBA. The DS pipeline is the thinner bus at 8-bit - ish. Nintendo has been on a mission in recent years to make their card interfaces and slow as possible. I actually miss the days of GBC/DMG or even GBA where reading from the cart wasn't much of a penalty and ROM could be accessed for code without too much penalty.
And the Nintendo stuff did allow some code overlays without too much problem on the GBA, though it's been years since I even looked at that stuff. Spyro Eternal Night was my last GBA project and it's starting to fade from memory quickly. The DS also has support for code overlays, but we've mostly been able to work around that as debugging within overlays is a real b*tch.
As to how it works? Not really sure. Didn't have to deal with that area of magic. I'd have been happy loading asm into a local buffer and just running it as needed. In recent years, we've been much more careful about seperating data from code and loading the data as needed form the cart. You'd be amazed how much code can be saved by dropping every day static const tables into an external loading system and using a universal method for reading it. Swap the tables as needed and leave the code be in RAM. Makes it so non programmers can easily modify the data as well - a huge plus.
#175878 - Exophase - Tue Feb 22, 2011 11:29 pm
GBA/slot 2 was indeed 16-bit: http://www.devrs.com/gb/files/gameboy1.gif
Note that this makes it, by definition, not a serial but a parallel interface.
On GBA there wasn't much benefit of running code in EWRAM instead of RAM, since under normal settings the EWRAM is actually slower by not offering faster sequential accesses. Some small number of games put code in EWRAM anyway for some reason. It could be useful if you're dynamically generating a large amount of code, like with a recompiler.
Some kernel code in IWRAM was very common, but most game code ran straight off the cart, and most games didn't bother with overlaying IWRAM stuff too aggressively. Some were pretty severe about it though.
#175880 - brave_orakio - Wed Feb 23, 2011 1:35 am
@kusma - Thanks! That cleared thing up a lot!
@Mike - For DS, declaring data as const doesn't go to ROM anymore(Technically it does of course but we know what were talking about here) but goes to RAM instead?
_________________
help me
#175881 - wintermute - Wed Feb 23, 2011 1:48 am
@brave_orakio, yes, everything on the DS goes in RAM, we don't have memory mapped ROM like the GBA - it's 8bit serial access.
@exophase, did you just pop in to Well Actually us?
_________________
devkitPro - professional toolchains at amateur prices
devkitPro IRC support
Personal Blog
#175882 - brave_orakio - Wed Feb 23, 2011 3:08 am
@wintermute - Any declaration to make them go to ROM by the way?
Other than that it probably is good practice for me to make my games small too like dsiware games. Which is why I'm trying to make a relatively fast LZSS decompression algorithm.
_________________
help me
#175883 - TwentySeven - Wed Feb 23, 2011 6:11 am
I don't think well-actually counts in a dev forum in the middle of a tech discussion :P
#175885 - sverx - Wed Feb 23, 2011 11:42 am
I'm not sure I'm really able to understand such a schema, but isn't it an 8 bit data parallel interface? I see ports D0 through D7 only... it seems it only has 16 bit addressing... (I think we're speaking about that Game Cartridge Socket in the upper right corner of the gif, right?)
#175888 - Dwedit - Wed Feb 23, 2011 3:13 pm
The big secret of the GBA is that video ram is almost as fast as IWRAM when you use it as data storage for bytes or halfwords. You're making an emulator which runs on the GBA, and you get a big performance improvement from moving the most-accessed part of the ROM file into VRAM.
Obviously, then you can't use that memory for displaying stuff, and you have that whole "Don't write bytes" thing going on, but reading bytes is fine, and far faster than EWRAM.
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."
#175903 - Exophase - Fri Feb 25, 2011 11:38 pm
| sverx wrote: |
I'm not sure I'm really able to understand such a schema, but isn't it an 8 bit data parallel interface? I see ports D0 through D7 only... it seems it only has 16 bit addressing... (I think we're speaking about that Game Cartridge Socket in the upper right corner of the gif, right?) |
Whoops, I linked the wrong thing. That schematic is for Gameboy, not GBA >_>
Here's the right one:
http://darkfader.net/gba/files/cartridge.txt
Note that data and address are muxed (at least partially), which is one of the reasons why sequential accesses are faster than non-sequential accesses.
#175904 - Miked0801 - Sat Feb 26, 2011 12:59 am
brave_orakio: Yes on the DS, const stuff lives in RAM. It's in its own .section or two, but RAM none the less. If you want a good laugh sometime, go look at the way the DS is forced to read info from the cartridge 1 bit at a time, usually polled. I thought that was bad, and then I moved into new Nintendo territory lately and wish I had it this good. :P
#175908 - tepples - Sat Feb 26, 2011 6:48 pm
| sverx wrote: |
| BTW you can have .nds files bigger than 4MB, if you meant that. (WarHawk DS is an example). |
Do modern card firmwares support the argv[0] needed to see this data?
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#175913 - Sektor - Sun Feb 27, 2011 11:59 pm
#175914 - brave_orakio - Mon Feb 28, 2011 2:38 am
@Mike - Hahahaha! Wonder why they're gimping the read from cartridge speed? A matter of cutting on hardware costs maybe?
_________________
help me
#175916 - sverx - Mon Feb 28, 2011 1:45 pm
| tepples wrote: |
| sverx wrote: | | BTW you can have .nds files bigger than 4MB, if you meant that. (WarHawk DS is an example). |
Do modern card firmwares support the argv[0] needed to see this data? |
I'm not sure it's really needed. In fact WarHawkDS runs fine on carts/firmware without argv[0] support...
#175920 - wintermute - Mon Feb 28, 2011 9:44 pm
argv support is absolutely essential for homebrew which uses nitrofs.
That EFS scan the card bullshit takes a *long* time to start. Bear in mind that a user with many files on their SD card is the normal scenario, not the exception.
http://devkitpro.org/viewtopic.php?f=24&t=2546&p=6931#p6931
_________________
devkitPro - professional toolchains at amateur prices
devkitPro IRC support
Personal Blog
#175921 - sverx - Tue Mar 01, 2011 9:36 am
I never meant to promote EFS... just saying that the NDS file doesn't need to fit in RAM, answering the original question.
OT: argv support is indeed the best option, but I guess people who code homebrew would like their game/app to run even on firmware that doesn't support it, one way or another.
#175922 - wintermute - Tue Mar 01, 2011 2:47 pm
With argv support things just work, all other methods have a negative impact on the user experience. I really don't understand why people keep wanting to justify doing things that ultimately make the experience of homebrew worse than it needs to be.
If the firmware doesn't support argv it needs fixed. Make your users aware of that fact. Help make the world a better place instead of compromising to help support lazy rip off merchants.
_________________
devkitPro - professional toolchains at amateur prices
devkitPro IRC support
Personal Blog
#175923 - Pate - Wed Mar 02, 2011 9:01 pm
In the topic of needing over 4MB, I'm in the process of adding 286 protected mode support for DSx86 (when running in DSi mode).
Wintermute, can I ask how the extra RAM is supposed to be used with libnds? I did some quick testing and it looks like the heap area is still limited to 4MB also in DSi mode. Is there an error in my method of checking for free RAM, or is it meant to be like this?
For my tests I next moved my EMS memory out of the heap area and made it start at the 4MB RAM boundary, with a size of 4MB, and everything worked fine. I'm just worried that I'll corrupt the heap when switching to some later libnds version if it begins to use all of the 16MB for heap RAM. Any plans/recommendations regarding this?
Thanks!
Pate
_________________
#175924 - elhobbs - Wed Mar 02, 2011 11:03 pm
I think you are looking at it wrong - search this forum for mallinfo for a thread on this subject.
the dsi heap does include the whole 16 MB. just use malloc or you will corrupt the heap.
EDIT: I should say that I did not test this method in dsi mode. but, I have used malloc to allocate 15MB without a problem in dsi mode.
#175926 - Pate - Thu Mar 03, 2011 6:37 am
Hi!
Thanks for the reply! I used the code from this thread: http://forum.gbadev.org/viewtopic.php?t=14438&highlight=mallinfo
The problem is that getMemFree() returns something around 2MB in DSx86, both in DS mode and in DSi mode. Not all that surprisingly, as it seems to be (at least partially) based on static linker addresses, which obviously can not change in DSi mode.
Thanks for the info about the heap actually using the whole 16MB, so I won't go around corrupting the heap!
Pate
_________________
#175928 - wintermute - Thu Mar 03, 2011 4:01 pm
That code is *really* old and uses variables in the linkscript that aren't actually used for anything any more.
| Code: |
#include <malloc.h> // for mallinfo()
#include <unistd.h> // for sbrk()
extern u8 *fake_heap_end; // current heap start
extern u8 *fake_heap_start; // current heap end
u8* getHeapStart() {
return fake_heap_start;
}
u8* getHeapEnd() {
return (u8*)sbrk(0);
}
u8* getHeapLimit() {
return fake_heap_end;
}
int getMemUsed() { // returns the amount of used memory in bytes
struct mallinfo mi = mallinfo();
return mi.uordblks;
}
int getMemFree() { // returns the amount of free memory in bytes
struct mallinfo mi = mallinfo();
return mi.fordblks + (getHeapLimit() - getHeapEnd());
}
|
malloc has had access to all 16meg on DSi since late 2009.
_________________
devkitPro - professional toolchains at amateur prices
devkitPro IRC support
Personal Blog
#175930 - Pate - Thu Mar 03, 2011 5:03 pm
Thanks Wintermute!
Pate
_________________
#175944 - wintermute - Mon Mar 07, 2011 12:08 pm
This still seems really hacky to me though. Software for systems with a full blown OS doesn't usually involve detecting installed RAM does it?
I can't help feeling that code like this should be avoided if possible.
_________________
devkitPro - professional toolchains at amateur prices
devkitPro IRC support
Personal Blog