gbadev.org forum archive

Hi all,
I've written a small NDS app to download files over wifi from a webserver and save them on a memory card.

My main motivation is to avoid having to swap the memory cards between the DS and the PC so often (I've already busted one M3PerfectSD through overuse).
Instead I'll just get the DS to grab it over wifi from the webserver running on my development machine.

It's up and running and seems to work great so far. I want to pack it up and make it available, but I've got a couple of questions first:

1) I'd like to get it to boot an nds file after downloading it. Can anyone point me in the direction of some good example code for launching NDSs?
(at the moment, I power down after the download, then power back up and boot it manually)

2) I need a name for it. I'm calling it webme for now, but that seems a bit crap. Any better suggestions?

Thanks!

You could call it "HTTP Download Play", especially if you manage to make it load .nds files without saving them to the memory card.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

This will be a great app!

I was just fearing the wear and tear as yesterday I plugged in my EZ4 to run my new code and it gave me a minisd card not detected error, took out and reinserted and it then worked but... :(

Are you going to have versions for the different cards? At least SC SD/CF, M3 SD/CF, and EZ4 (which will become more popular I'm betting when the lite version is out to buy).

What about Share DS ? (like Share kazza...)

I'M very impatient to see this project ! ! ! !

But is the speed download interresting ?
Have we to tape on the DS the link for the download, or is it usable with a config file in SD/CF?

Keep work, if you need beta testers I'm ready ;)
For .NDS launch, see the source code of Moonshell or DSOrganize. Both are certainly interresting...

Excuse my if my English is so bad... I'm Belgian ; )

Darkgilson
_________________
Soon Blubb in 3D !

Hasn't this been done already? Not meaning to offend! It is good for healthy competition.
_________________
Squak! etc.
My Blog

parrot_: Yes, there's already Bafio's Wifi Transfer app. But it is rather outdated because it uses UDP and is compiled against Wifilib 0.2, which does not yet have WEP. I patched it to work with the new version, but it is rather unstable, probably because of UDP. So, a project like this may be just what we need.

Eyeballkid: I've put a quick article in DSWiki that shows how to launch a NDS file (works on GBAMP at least). Keep it up!

Bye,
Tob

Actually, this is a good idea,
I have been thinking about converting my wifi transfer app to http myself, but I'm quite lazy lately. You can find the methods to boot an nds on GBAMP in moonshell and rain (I used the rain method becaused it seemd really simple to implement).
I would suggest to implement a very simple file list support as well...

I'm looking forward to it ;)

Bafio

This is a very interesting app.
I hope you finish it soon :).

Thanks for all the nice comments!

OK, it's now called DSlurper.
"D" for DS.
"Slurper" because that's what it does.

I've packed it up (both source and binaries), and you can download it from:
http://www.scumways.com/dslurper/dslurper-0.1.tar.gz.

Quick summary:

Think that covers it for now... feedback welcome!
Enjoy!

Yeah.. that would be great.. It's been talked about, but I don't know if anyone else has been working on something like this yet. Also, if you wanna use ndshb.com to pull files from, let me know and we can figure out how to make it work (how the files need to be stored on ndshb.com to work).

Also, to decrease download times, you might want to try implementing some kind of decompression.. an unzipper would be nice.. :)
_________________
NDS Homebrew Roms & Reviews

Lynx wrote:

Also, to decrease download times, you might want to try implementing some kind of decompression.. an unzipper would be nice.. :)

Yep, that'd be pretty cool. Definitely doable, and I'll add it to the TODO list.

At the moment the stuff I'm working on isn't big enough to be a problem, so I'm not likely to do it until my projects get big enough that the transfer speed starts to annoy me :-)

However, it occured to me this morning that the _correct_ solution is to just dump HTTP altogether and use a protocol like rsync (see http://samba.anu.edu.au/rsync/). I don't mind a slow initial transfer if subsequent transfers are ultra-quick.

(i have a supercardCF btw)

EyeballKid wrote:

Thanks for all the nice comments!

OK, it's now called DSlurper.
"D" for DS.
"Slurper" because that's what it does.

I've packed it up (both source and binaries), and you can download it from:
http://www.scumways.com/dslurper/dslurper-0.1.tar.gz.

Quick summary:

Think that covers it for now... feedback welcome!
Enjoy!

i opened up your source code and tried to build with the latest wifi lib(from cvs) and it gave me a missing function error

Code:

f:/dslurper/arm9/source/happyhttp.cpp: In function 'in_addr* happyhttp::atoaddr(const char*)':
f:/dslurper/arm9/source/happyhttp.cpp:208: error: 'inet_addr' was not declared in this scope
make[2]: *** [happyhttp.o] Error 1

this confusses me :D

//edit
to be really cheap i decided to comment those lines out,
now the app builds and i can run it on my ds...

when i try to download the file i put it says this

dslurper wrote:

init wifi...ok
fetch
http://[server_ip]/~[user]/[file]

ERROR:
recv(): Spftware caused connection abort

*** fin! ***

i replaced [..]'s with the right information and i check the url and it works on my pc
_________________

1x WII 2x remotes
2x NDS/L(FMv7-ORG:v4,FMv7-org:DSL)
1x GBAMP
2x 1gb (MicroDrive{typeII}&SanDisk{typeI})
1x SuperPass2
1x Supercard-CF
MoonShell skins

Could you make the config file a txt file.
That way it could be changed via the DS (via DSorganise for example).

-Or make an url input function ;)
I'm also wondering, does this only work locally?
When I enter a "real" url --> http://www.simonvandeberg.be/blabalabl
It gives me an invalid url error.

*sorry to get a bit of topic, but I'm wondering a bit, does the "new" Sata version of the FAT driver also work with CF devices (e.g. GBAMP and etc.)?

Scorpei wrote:

Could you make the config file a txt file.
That way it could be changed via the DS (via DSorganise for example).

-Or make an url input function ;)
I'm also wondering, does this only work locally?
When I enter a "real" url --> http://www.simonvandeberg.be/blabalabl
It gives me an invalid url error.

*sorry to get a bit of topic, but I'm wondering a bit, does the "new" Sata version of the FAT driver also work with CF devices (e.g. GBAMP and etc.)?

i made you a compiled one with dslurper.txt reader :),
{pm me your email addresse if you want it...
also if you look at his example it shows
http://IP/~user/
so find out your server's ip,
and then put in your username
and then the rest of the stuff...

this part worked for me
:P
(i also turned off the bottom backlight in my version :D)
_________________

1x WII 2x remotes
2x NDS/L(FMv7-ORG:v4,FMv7-org:DSL)
1x GBAMP
2x 1gb (MicroDrive{typeII}&SanDisk{typeI})
1x SuperPass2
1x Supercard-CF
MoonShell skins

I wish I was a coder like you :(.
:), time to learn some more (after I get done with my Japanese studies, and my test week starting next monday :S).

hey i wonder if we could write the app to sram,
and then launch it from sram...

(for supercard's and m3's)
it would be better because more support :P...
and it would be more of a testing app then a saving app...

i think its a good idea to try :P...

now if only i knew anything :P
_________________

1x WII 2x remotes
2x NDS/L(FMv7-ORG:v4,FMv7-org:DSL)
1x GBAMP
2x 1gb (MicroDrive{typeII}&SanDisk{typeI})
1x SuperPass2
1x Supercard-CF
MoonShell skins

swimgod wrote:

i opened up your source code and tried to build with the latest wifi lib(from cvs) and it gave me a missing function error

Code:

f:/dslurper/arm9/source/happyhttp.cpp: In function 'in_addr* happyhttp::atoaddr(const char*)':
f:/dslurper/arm9/source/happyhttp.cpp:208: error: 'inet_addr' was not declared in this scope
make[2]: *** [happyhttp.o] Error 1

this confusses me :D

Hmmm, sorry, but I don't really know why it's not finding inet_addr()...
It's part of libwifi (#include <netinet/in.h>). Are you sure it's finding the right dswifi include files? Maybe there's some stray ones left over from an earlier version?

inet_addr() handles numeric IP addresses, eg "192.168.1.1", so without it numeric addresses won't work.

Maybe try it with a named address "http://wibble.foo.com/thingy" and see if it gets further...

Either way, the inet_addr() currently in CVS is a bit broken at the moment anyway. It'd still compile, but would return the wrong address. Apparently a fix is going in soonish.

Other stuff:
I'll change the config file to .txt extension so other DS utils can edit it!

i decided to create the function :P,
tell me if this looks right...
because it compiles with this...

Code:

const char* inet_addr( char in );
char* out;

bool inet_addr( const char* in){
//
char* hostname = out;
while( *in && *in != ':' && *in !='/' )
   *hostname++ = *in++;
*hostname++ = '\0';

if( out[0] == '\0' ){
   return false; // hostname empty
}else{
   return out;
}
}

i don't know if it actually works right :P,
but like i said it does compile :)...
_________________

1x WII 2x remotes
2x NDS/L(FMv7-ORG:v4,FMv7-org:DSL)
1x GBAMP
2x 1gb (MicroDrive{typeII}&SanDisk{typeI})
1x SuperPass2
1x Supercard-CF
MoonShell skins

Something to be added (sorry haven't looked at the todo yet).
A funtion that if there is no txt/config file, the output is that there is no file.

Scorpei wrote:

Something to be added (sorry haven't looked at the todo yet).
A funtion that if there is no txt/config file, the output is that there is no file.

check your email :P...
it seems he forgot to put { and }'s in this part

Code:

void ParseConfigFile()
{
FAT_FILE* fp = FAT_fopen( CONFIG_FILE, "r" );
if( !fp )
{
   return;
}
   char line[256];
   while( FAT_fgets( line, sizeof(line), fp ) )
   {
      const char* p = line;
      while( *p && *p == ' ' )
         ++p;
      if( *p == '#' || *p == '\0' ) // skip comments and blank lines
         continue;
      strcpy( g_SourceURL, p );
   }
FAT_fclose(fp);
}

so i changed it to...

Code:

void ParseConfigFile()
{
FAT_FILE* fp = FAT_fopen( CONFIG_FILE, "r" );
if( !fp )
{
   printf("CONFIG FILE NOT FOUND!");
   return;
}else{

   char line[256];
   while( FAT_fgets( line, sizeof(line), fp ) )
   {
      const char* p = line;
      while( *p && *p == ' ' )
         ++p;
      if( *p == '#' || *p == '\0' ) // skip comments and blank lines
         continue;
      strcpy( g_SourceURL, p );
   }
}
FAT_fclose(fp);
}

:P
_________________

1x WII 2x remotes
2x NDS/L(FMv7-ORG:v4,FMv7-org:DSL)
1x GBAMP
2x 1gb (MicroDrive{typeII}&SanDisk{typeI})
1x SuperPass2
1x Supercard-CF
MoonShell skins

Very nice, now to make it understand the DNS.
Then one can lookup the URL in DSLinux.
vi the txt file, and startup the downloader :). Very nice.

Scorpei wrote:

Very nice, now to make it understand the DNS.
Then one can lookup the URL in DSLinux.
vi the txt file, and startup the downloader :). Very nice.

Erm, surely if you have DSLinux you could just use NFS/SSH/Netcat? But seeing as there's no Supercard SD support in DSLinux as yet, this app remains useful

EDIT: There is SD support in Linux now so retawq et all can be used

Last edited by sonny_jim on Tue Aug 08, 2006 2:20 pm; edited 1 time in total

Hi all, i'm new here, and was reading and loving to know about DSLurker's existance.
I don't know anything about coding, but...

And if it had a XML parser, and the GUI in DS listed all the files in the web folder available to download?

Lets say the DSLURKER.TXT config file were like this:

Code:

<?xml version='1.0' encoding='UTF-8'?>
<LIST ENTRIES='2'>
<ITEM
TITLE="My personal server"
URL="192.168.254.1/ndsfiles"
USER="user"
PASS="pass"/>
<ITEM
TITLE="A remote server"
URL="http://www.webhost.com/ndsfiles"
USER=""
PASS=""/>
</LIST>

All we had to do is put the downloadables in the server's apropriate folder and setup a right config file. Move DSLurker.nds and DSLurker.txt to SD/CF along a empty folder like "DOWNLOADED" where it would store the leeched files.

Personally i don't want it to autolaunch a executable after dowload, but it could be optional in the config, like <LIST ENTRIES='2' AUTOLAUNCH=0>.
Infact many more options could be in this XML fashion.

So, what you guys think?
[Images not permitted - Click here to view it]
A simplistic picture to give a idea.
http://img61.imageshack.us/img61/5841/dslurkerzk1.jpg

Hmm, tried it out on the GBAMP...it appeared to work, but....my game woudlen't run on the DS when downloaded though dsslurper. :(

It seems really cool though. It'd be cooler if it worked with my GBAMp though :-P

As for a couple other things...it'd be nice to be able to specify a new name for the file...it'd also be nice to be able to set multiple links to download in the config file and to be able to add files to be downloaded in the app itself.

Yeah, I wouln't mind naming all of the files in the .cfg, but editing it before evry time I send a file is kinda lame.
_________________
Hardware: GBAMP v2, Datel MML, NDS phat (Flashme v7)

The solution I'm leaning toward is:

1) You set up the URL to download in the config file (ideally editable on the DS using a GUI)
2) DSlurper downloads the file at the URL
3) If the URL was a DS-executable file, then it is run
4) if the URL is an HTML file then it is parsed for all the links. The user is presented with a them as a list of files and can pick one to download. Go to step 2.

This means that you could just set up directory browsing on your webserver (or manually create an index.html which links to NDS files you want to be able to access) and add/remove NDS files without changing the config on the DS.

I'm a bit busy at the moment, but hopefully I'll get onto this soon.
I'll also try switching to the new FAT library.

_My_ number-one missing feature is automatically rebooting with the downloaded NDS file. Looks like there is some new exec() code floating around out there, so I'll give that a whirl soon. Looks promising!

Cool, when that version is released, I will create a url on ndshb.com. Will it be able to go multiple html files deep? Meaning, a main html that points to html that points to html that lists the .nds files (for category lists, etc)? Also, I'm not sure what parameters you plan on allowing in the html file, but I would like to see file size on of them, so people know how big/long it might take to download at 1Mb (or slower).

Example I am thinking of would be like this:
index.html

Choose Category:
- Games
- Apps
- Demos
- Newly Added

Choose Size:
- Small (0-500K)
- Medium (500 - 1Meg)
- Large (1 - 2Meg)
- X-Large (2+ Meg)

Games.html

Arcomage - Bodom Child - "Magic The Gathering" type card game - 3Meg
A Touch of War - JimmyL - "Warcraft" style game - 940K

Also, I don't know if it matters, but xml might be easier?
_________________
NDS Homebrew Roms & Reviews

EyeballKid wrote:

Hi all,
I've written a small NDS app to download files over wifi from a webserver and save them on a memory card.

Hi - i patched dsluper and added some usefull functions.

1) based on libfat / requires DevKitPro 1.3.5

2) added iniparser library

3) Indirect download.

What is indirect download?

In dsluper.ini on your NDS you add a url - if inidrect download is enabled a 2nd Ini file will be fetched from that url instead of a single nds.

Within this ini file you can add other urls to download.

Example:

dslurper.ini -> http://foo.example.com/download.ini

NDS will fetch download.ini and parse it

download.ini contains:

http//foo.example.com/file/nds1.nds
http//foo.example.com/file/nds2.nds,
http//foo.example.com/file/README

now this files get's downloaded.

Here the Url to get dslurper 0.1a

http://m11s12.vlinux.de/nds/dslurper-0.1a.tar.gz

Have fun.

Tag for later testing.
_________________
Framebuffer is dead.
DynaStab.DrunkenCoders.com

Hi again!

here a PHP script to autogenerate "ini" Files to use with dslurper-0.1a

http://m11s12.vlinux.de/nds/dslurper.php.txt

In this script you enter directories and a filemask. It generates downloadable content for dslurper-0.1a.

I tried to be (somehow) secure but i suggest using this skript only in internal networks.

Sorry i didn't try this on Windows PHP installations.

Nice one Monsti!
I look forward to checking it out!

EyeballKid wrote:

Nice one Monsti!
I look forward to checking it out!

There is a small bug:

arm9_main.cpp - line 200

change this to:

Code:

for( k = 0; k <= b; k ++ )

I've tried the 0.1 and 0.1a.
Iv' tried patching the 0.1 release, hacking the 0.1a with the FAT lib from 0.1 a and then patching to supercard lite, no luck

Any ideas?

Thanks in advance
TOA

Hi,

i use devkitpro 19b with default libfat.

Can you try this?

sorry to dig out this old topic again eyeballkid, I just wanted to ask, if it would be possible to add DLDI to DSlurper and if you are gonna continue it or already abadoned it.

Thx for your work and answers in advance:

pas
_________________
Starcraft DS ?

pas wrote:

sorry to dig out this old topic again eyeballkid, I just wanted to ask, if it would be possible to add DLDI to DSlurper and if you are gonna continue it or already abadoned it.
pas

Funny you should ask - I changed it to use the new fat library (i.e. DLDI) quite recently.
I was also trying to add boot support (using Maxwell Goodman's bootlib) so it could automatically run the downloaded file... but just couldn't get that working... :-(

Anyway, I'll sort out a new release with the DLDI support soon!
Maybe someone else will have a go at adding reboot support ;-)

Use Chishm's loader, it's easy and works great.
_________________
I'm a PSP hacker now, but I still <3 DS.

gbadev.org forum archive

DS development > app to download files from webserver

#86632 - EyeballKid - Thu Jun 08, 2006 1:27 pm

#86636 - tepples - Thu Jun 08, 2006 1:54 pm

#86644 - Grey - Thu Jun 08, 2006 3:04 pm

#86647 - darkgilson - Thu Jun 08, 2006 3:26 pm

#86659 - parrot_ - Thu Jun 08, 2006 4:08 pm

#86670 - 0xtob - Thu Jun 08, 2006 4:40 pm

#86672 - bafio - Thu Jun 08, 2006 4:51 pm

#86707 - Scorpei - Thu Jun 08, 2006 7:08 pm

#86713 - EyeballKid - Thu Jun 08, 2006 7:29 pm

#86714 - Lynx - Thu Jun 08, 2006 7:34 pm

#86716 - EyeballKid - Thu Jun 08, 2006 7:43 pm

#86717 - swimgod - Thu Jun 08, 2006 7:48 pm

#86723 - Scorpei - Thu Jun 08, 2006 8:10 pm

#86731 - swimgod - Thu Jun 08, 2006 8:36 pm

#86733 - Scorpei - Thu Jun 08, 2006 8:39 pm

#86749 - swimgod - Thu Jun 08, 2006 10:06 pm

#86753 - EyeballKid - Thu Jun 08, 2006 10:26 pm

#86754 - swimgod - Thu Jun 08, 2006 10:47 pm

#86781 - Scorpei - Fri Jun 09, 2006 6:31 am

#86783 - swimgod - Fri Jun 09, 2006 7:03 am

#86815 - Scorpei - Fri Jun 09, 2006 3:47 pm

#87381 - sonny_jim - Tue Jun 13, 2006 5:56 pm

#97119 - MasterMan - Tue Aug 08, 2006 2:27 am

#97125 - chatterbug89 - Tue Aug 08, 2006 3:24 am

#97132 - brennan - Tue Aug 08, 2006 4:02 am

#97273 - EyeballKid - Tue Aug 08, 2006 11:04 pm

#97322 - Lynx - Wed Aug 09, 2006 5:25 am

#98318 - monsti - Mon Aug 14, 2006 5:39 pm

#98320 - DynamicStability - Mon Aug 14, 2006 5:43 pm

#98371 - monsti - Mon Aug 14, 2006 11:01 pm

#98438 - EyeballKid - Tue Aug 15, 2006 9:18 am

#98453 - monsti - Tue Aug 15, 2006 11:04 am

#98598 - toa - Wed Aug 16, 2006 9:07 am

#98633 - monsti - Wed Aug 16, 2006 5:24 pm

#157143 - pas - Mon May 19, 2008 7:41 pm

#157629 - EyeballKid - Tue May 27, 2008 12:17 pm

#157758 - HyperHacker - Wed May 28, 2008 11:57 pm