gbadev.org forum archive

I have a little (maybe) problem, but do not know how to solve it. I have been thinking about it for about a week now, and still have not come to any logical solution. I am trying to separate my program, which has many "modules" built into it. I would like these modules to be spread among different files to be accessed via a filesystem. The main program would control these modules, but I do not know how the main program and the module would interact. For example, I do not want to have the GUI functions compiled both in the main program and in the module. This would waste lots of space and would be inefficent, and would also cause problems because there would be two conflicting buffers which would both try to write to the screen. What I am trying to figure out is how these two programs, the main program and the module, can interact. One might think of this as a Windows executable and a DLL. The main program is the EXE, and the modules are DLLs. Each module is recognized by the EXE through a config. file, or by some other method, and is run whenever the user activates that module. My question is, how can this be applied to my application on the DS? Would either the main program or the module have to be specially configured?

Thanks in advance,
strager

Dynamic linkers are a tricky business. It will involve parsing .o files. Write a static linker for practice.

Overlays are a simpler but more limited approach. You have to know in advance what address each module will be loaded at. Modules that are never loaded at the same time can share the same address, and the coder must be careful to not call functions that aren't loaded.

I've never done overlays by hand, but off the top of my head, here's how I might do it without special compiler or linker support:

- Create a .c file with some functions that you'd like to dynamically load.

- Create another .c file with an array of pointers to the functions.

- Compile the 2 .c files and link them together as a flat binary. Tell the linker that the starting address is wherever you are going to load the module. Link in the array file first so it will be the first data object in the binary.

- In the main program:

Code:

#define OVERLAY_ADDRESS 0x...
#define OVERLAY_FUNCTION(i) (*((void(**)())OVERLAY_ADDRESS + i))

void loadFile(u8* dest, const char* name);

void test()
{
loadFile(OVERLAY_ADDRESS, "module.bin");
OVERLAY_FUNCTION(12)();
}

Add #defines of all the overlay functions for clarity and typesafety.

Last edited by sajiimori on Mon Jan 09, 2006 4:19 am; edited 1 time in total

Oh yeah, I guess you wanted to call outside functions from within the modules. To do that, I'd probably put an extra pointer right before the array of the module's function pointers (so that this new pointer is the first object in the module's binary). It will point to an array of pointers to all the functions that the module needs, and the main program will set the pointer just after loading the module.

Code:

// Add one to skip the reserved pointer that the module uses.
#define OVERLAY_FUNCTION(i) ((void(*)())OVERLAY_ADDRESS + i + 1)

void* exportedFunctions[] =
{
...
};

void test()
{
loadFile(OVERLAY_ADDRESS, "module.bin");
*(void**)OVERLAY_ADDRESS = exportedFunctions;
OVERLAY_FUNCTION(12)();
}

I was just reading about the -fPIC option for GCC. Does anyone here have experience with the position-independent code that arm-gcc produces? Would it mean modules could be loaded at any address, letting its array of function pointers just contain offsets?

sajiimori wrote:

I was just reading about the -fPIC option for GCC. Does anyone here have experience with the position-independent code that arm-gcc produces? Would it mean modules could be loaded at any address, letting its array of function pointers just contain offsets?

You can create .so files on linux with that option. .so files are libaries, the same as .dll files for windows. A special program handles the loading, so you can use it for DS libs, but you have to write your own loader, which would be an interesting task, but would be non trivial on the other hand! A simple plugin API /lib would be nice, but you have to do it yourself.

But the DS is not Linux. ;) I might try it out later and see what it does.

sajiimori wrote:

But the DS is not Linux. ;) I might try it out later and see what it does.

As i said, you have to write your own loader ;-)

I'm mostly wondering if the simple mechanism I described here would work.

I have a bit of experience with it. Generally, PIC screws two things, which are far calls (I mean BX-based, of course) and read/write globals (that includes static variables within functions)

The reason is that near calls are relative, and (at least with -fPIC) constant data is placed inside .text, and is then accessed only with PC-relative opcodes, so all's dandy (I recommend using #pragma no_long_calls to make sure gcc doesn't generate far calls).

The problem of read-write globals can be solved in a semi-satisfactory way like so:

basically, create a function, which we'll call 'get_abs_addr' which will fix the absolute address, and instead of accessing globals directly, we'll first call get_abs_addr, and then use them.

here's some example code:

get_abs_addr.h:
void *get_abs_addr(void *addr)

get_abs_addr.s:

get_abs_addr:
SUB R1, PC, #(PC_points_here-bump_end)
add R0, R0, R1
PC_points_here:
LDR R1, =(__text_start)
LDR R2, =get_abs_addr // silly gcc won't compile LDR R1, =(get_abs_addr - __text_start)
SUB R1, R2, R1
SUB R0, R0, R1
BX LR

(I don't promise the code will work, since I didn't test it...)

In the code I make an assumption about how the image is loaded. Basically, I assume that during linking, the image is configured so that .text is the first section, located at 0x0, and that no section is given an absolute address except for .text. Then, basically, you can understand why the code works - what I do there is dynamically get the base address of the image (by getting PC while it is running my code, and subtracting the offset of my code from the base address), and add that base address to the pointer given to me.

you'd think that using this in code is awkward, and it is, but hopefully, you don't use too many globals, and even if you do, a simple way for a somewhat easier life is this:

#define DECLARE_GLOBAL(type,name) type __global__##name
#define USE_GLOBAL(name) #define name (*(typeof(__global__##name) *)get_abs_addr(__global__##name))

and then you just need to write two lines to declare the global, so that it'll be accessed in a safe way almost transparently (almost since it screws up the auto-completion of just about every editor).

Oh, yes, I just remembered that GCC doesn't know how to put constants inside the .text segment, so you'll have to use that ugly syntax for consts too. (initialization is done like so:
DECLARE_GLOBAL(char, somestr[]) = "yay?";
)

The part about the constants that's really annoying is when you use strings in your code, and suddenly, instead of printf("Hello World!\n"); you have printf((char *)get_abs_addr("Hello World!\n")); ... Gah...

Anyway, I think that sums up what I have to say...

Edit: Scratch that. I forgot to write about the alternative way of doing things. This way assumes that you want the linker as stupid as possible, and that the 'loading' of the binary will be simply to copy it to the RAM. The price you pay is both in little nags while developing, and, also, the overhead of calling get_abs_addr.

The overhead is negligible if you're clever, since there are generally two types of globals - the first is state-related variables, which are generally not accessed that frequently, so the overhead isn't really important, and data, which might be accessed intensively. In that case, the solution is simply to locally create a pointer to the global, and use it instead, and thus save all the calls to get_abs_addr (especially important if you have some inner-loop that's accessing the global).

Anyway, if you decide that those problems are too much for you, the other solution is to have a more powerful loader, and have it do the relocations upon load. This requires both some linker work that I don't know how to do, and a more complex loader. The general idea is to create an additional section which contains in it all the locations where absolute pointers exist in the binary, and that the loader, upon loading the binary, will fix them by adding the correct image base.

That's generally what's done in windows with PE files, but they're really complex beasts. I suggest you read Matt Pietrick's definitive PE article, which has an excellent explanation about how relocations work in windows, as well as many other features related to binary compatability that exist in PE files (remember, microsoft makes a living out of binary compatability). The article is at http://msdn.microsoft.com/msdnmag/issues/02/03/PE2/
_________________
"Facts Shmacts! They can prove even remotely true things!" -Homer Simpson

devkitARM release 18 contains elf2flt for generating BFLT binaries. You might like to have a play with that.

http://www.beyondlogic.org/uClinux/bflt.htm

I haven't played with this much - it was a requested feature for the DS Linux people.
_________________
devkitPro - professional toolchains at amateur prices
devkitPro IRC support
Personal Blog

I've managed to get a working system of "DLL" ripped from Moonshell, you should have to look to Moonshell's sources.

Edit: Oops, don't see it was an one-month-old thread

gbadev.org forum archive

DS development > Dynamically Linking Modules

#66202 - strager - Mon Jan 09, 2006 12:22 am

#66224 - sajiimori - Mon Jan 09, 2006 3:24 am

#66225 - sajiimori - Mon Jan 09, 2006 3:34 am

#66228 - sajiimori - Mon Jan 09, 2006 4:40 am

#66258 - juhees - Mon Jan 09, 2006 10:07 am

#66325 - sajiimori - Mon Jan 09, 2006 8:06 pm

#66326 - juhees - Mon Jan 09, 2006 8:17 pm

#66329 - sajiimori - Mon Jan 09, 2006 8:30 pm

#81373 - Fry_Day - Fri Apr 28, 2006 9:14 pm

#81381 - wintermute - Fri Apr 28, 2006 10:20 pm

#85385 - thoduv - Mon May 29, 2006 6:06 pm