gbadev.org forum archive

This is a read-only mirror of the content originally found on forum.gbadev.org (now offline), salvaged from Wayback machine copies. A new forum can be found here.

DS Flash Equipment > How to modify firmware to do new things? (sample code?)

#41790 - DsPet - Sun May 01, 2005 11:04 pm

I see there are now *two* programs that modify the built-in firmware (FlashMe, and a demo video of Wireless Multiboot unencrypted games)
Cool !

---

My question: how is this done ?

I have found no information on the public BBSs or websites describing the details of how the firmware is organized. Here are some obvious questions:

What does the 256KB firmware file store look like ?
How do you patch the firmware file while keeping the original intact ?
How to rekey it for different device IDs?
Source code ? (eg: to FlashMe and related encoder program)
...

The work has been done and details figured out. Congratulations to all involved. Can you please share it?

-----
Yes I am aware this is a dangerous operation, and may "brick" your NDS.
I'd like to contribute and add more things at this level and share with the public community. Am I asking to much ?

Or another more provocative way of asking the question:
Are the techniques for opening up the closed Nintendo system going to remain closed source and hidden from the public ?

#41795 - tepples - Sun May 01, 2005 11:53 pm

I'm guessing that once the process becomes more reliable, it'll be explained better.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#41798 - [FireFly] - Mon May 02, 2005 12:12 am

tepples wrote:
I'm guessing that once the process becomes more reliable, it'll be explained better.

The firmware is a very critical part of the Nintendo DS. If even one bit is incorrect then you are the new proud owner of a high-tech brick. Firmware modifcations should be done by professionals to minimize that risk, hence why the information is kept private. That's a good thing, trust me.

#41824 - Alec - Mon May 02, 2005 6:05 am

You wouldn't want someone zeroing out a firmware and passing it off as one of these helpful, pre-hacked ones, now would you?
_________________
Eat Those Gopher Cakes 'Til You Explode!
[Images not permitted - Click here to view it]

#41861 - DsPet - Mon May 02, 2005 3:06 pm

> I'm guessing that once the process becomes more reliable, it'll be explained better.

I hope so.
I've never been a fan of intentionally keeping information away from people who want to use it - especially since the goal is to open up the system more.

FWIW: There is already enough information in the public domain to easily "brickify" your device if that is your malicious goal. The true goal here is to learn from others experiences and build on top of it.

----
Commentary:
Open source has many benefits to every one, which you don't need me to elaborate here.

If people have written their own programs, and aren't building it on top of GPL code, they have a right to keep it private if they so wish. However, all I can ask is that they consider releasing it under GPL so it doesn't have to be reinvented from scratch.

I would also feel a little more confortable when reflashing my device with an open source program - but that's not essential for most people.

Thanks for the consideration.

#41937 - Kyoufu Kawa - Tue May 03, 2005 2:54 pm

I'd like a peek at the firmware myself. Don't intend to flash it at all for now, but I just like senseless knowledge.

#41949 - Abcd1234 - Tue May 03, 2005 5:57 pm

[FireFly] wrote:

The firmware is a very critical part of the Nintendo DS. If even one bit is incorrect then you are the new proud owner of a high-tech brick. Firmware modifcations should be done by professionals to minimize that risk, hence why the information is kept private. That's a good thing, trust me.


Dude, no offense, but that's pretty damned arrogant. Yes, you're absolutely right, modifying the firmware on your DS is an incredibly dangerous thing to do. However, withholding information about the firmware just to protect a few idiots out there helps no one, while forcing people with the requisite expertise to work from scratch, rather than building off existing efforts. Besides, who's to decide who qualifies as a "professional"? Sounds like nothing more than elitism to me.

#41962 - TheMikaus - Tue May 03, 2005 8:46 pm

keeping the information private has two effects. One it pisses some people off and two prevents numbnut-jerks from creating a firmware that turns your DS into a brick.

If it's that important maybe you can talk to DarkFader or Loopy.

And as for professional, I would probably say that gets termed as someone who does this sort of thing (low-level hardware) as more than just a hobby. Unless they've been doing it as a hobby for a while. Maybe they count too.

::shrug::

Just be happy that they even made a firmware replacement that works.

Edit:
Also. You don't have to be an idiot to make a mistake.

#41963 - Abcd1234 - Tue May 03, 2005 8:54 pm

Quote:
One it pisses some people off and two prevents numbnut-jerks from creating a firmware that turns your DS into a brick.


By that logic, Microsoft should keep all their APIs under lock and key to prevent evil hackers from creating programs that can wipe hard drives and then tricking people into downloading them.

Sorry, but I look at homebrewers as a *community* (right or wrong), and in that community, you share information so that others can benefit. This brand of intellectual elitism does nothing but inflate the egos of a few while keeping the rest ignorant, and thus helps no one.

#41965 - Proteanruler - Tue May 03, 2005 9:02 pm

think that's a bit unfair - you wouldn't be happy if people passed around viruses in their code/demos and equally its a bit unfair to expect the guys behind the firmware to want to brickify people's DS's through bad firmware, stupidity etc. Patience is a virtue... isn't it?!

#41968 - TheMikaus - Tue May 03, 2005 9:17 pm

Difference being that you can reformat your own computer. Can't really reformat the DS. But point taken.

You don't act the least bit thankful in the first place. I'm sorry you didn't get exactly what you wanted from the firmware developers, but at least you could show some appreciation instead of trying to goad them or insult them because they aren't doing what you want them to.

And if they never release the information, then you can do the work and release the information and it'll be all good and you'll help the community in the way you see fit.

At least show some appreciation for their work instead of just bashing them because they aren't releasing the source for your enjoyment.

And as for the elitism. Maybe they honestly are keeping the information closed to try and protect the community. But you'll believe what you want and continue slappin' because they aren't releasing the information when you would right?

::shrug::
Maybe they'll release it eventually and you'll be happy.

Till then, enjoy what you can.

::blink:: that's a novel. Sorry :/.

#41972 - Abcd1234 - Tue May 03, 2005 9:41 pm

TheMikaus wrote:
You don't act the least bit thankful in the first place.


And in that you'd be totally wrong. Trust me, I'm *incredibly* grateful to these people who've done this work. After all, I myself have benefitted from it. Heck, the very day I flashed my DS I posted a huge thanks in these forums.

However, I also believe it's unproductive to hoard information for no good reason. Frankly, it's that kind of behaviour that turns me away from the homebrewing scene, as it becomes clear that people are more interested in being l33t hackers, rather than trying to grow and develop a cooperative community.

Quote:
I'm sorry you didn't get exactly what you wanted from the firmware developers, but at least you could show some appreciation instead of trying to goad them or insult them because they aren't doing what you want them to.


Interesting. So, if someone does a great thing, you can't criticize them? That's kinda silly, don't you think?

Think of it this way. Linus Torvalds has done a *ton* of great things for the open source community. However, he's also taken a ton of criticism for various decisions he's made (the BitKeeper fiasco being the most recent one). Does that mean people are ungrateful? No.

Quote:
And if they never release the information, then you can do the work and release the information and it'll be all good and you'll help the community in the way you see fit.


And I said just as much, and explained why I think that's silly: needless duplication of work.

Quote:
And as for the elitism. Maybe they honestly are keeping the information closed to try and protect the community.


Which is, by definition, elitism. They feel they are in a position to "protect" people, as if they need protecting in the first place.

Quote:
But you'll believe what you want and continue slappin' because they aren't releasing the information when you would right?


Nope. If my words are falling on deaf ears, I'll move on... I have DS hacking to do. :)

Quote:
Maybe they'll release it eventually and you'll be happy.

Till then, enjoy what you can.


Yup, and if that happens, awesome! I'll be the first to say thanks.

Ahh, a novel of my own. Good times.

#41974 - pipomolo42 - Tue May 03, 2005 10:01 pm

TheMikaus wrote:
prevents numbnut-jerks from creating a firmware that turns your DS into a brick.


Not true.

The first region of the flash requires you to shorten SL1 to be flashed. As long as :

- you keep a recovery routine in this part of the flash
- you don't run untrusted code with SL1 shorted

There is absolutely no risk bricking the DS.

So, please make a better excuse ;)


Regards,
Alex

#41975 - TheMikaus - Tue May 03, 2005 10:05 pm

Quote:

Interesting. So, if someone does a great thing, you can't criticize them? That's kinda silly, don't you think?

I don't think insulting is the same thing as criticism. And if it does fall under the same literal category it's not very contrsuctive and therefore moot.

So giving stuff to the community to make homebrewing code easier (which they've already done) isn't expanding the community?


Again for the elitism. I guess it depends on how you look at it. Caution says. Someone will abuse this, release is a no no. Elitism says you're too freaking stupid to do this correctly sorry. (This I suppose boils down to opinions)

Quote:
a novel of my own.

We should start our own book company ;)

and to pipomolo:
how do you know a piece of firmware will brickify your DS if no-one tries it first?

#41978 - Abcd1234 - Tue May 03, 2005 10:34 pm

TheMikaus wrote:
I don't think insulting is the same thing as criticism. And if it does fall under the same literal category it's not very contrsuctive and therefore moot.


True enough. I should mention that many of my comments come out of sheer frustration. In areas like this (homebrewing, hardware hacking, etc), it's been my experience that elitism is incredibly common. Yes, I realize that's not very fair and I shouldn't generalize, but given the prevalance of this behaviour, it's hard not to.

Quote:
So giving stuff to the community to make homebrewing code easier (which they've already done) isn't expanding the community?


Yeah yeah, I know. Like I say, it's primarily frustration, and also irritation with this idea that there are people out there who feel it's their job to "protect" me, as if I'm not perfectly capable of protecting myself. It's very condescending, to say the least...

Hey, I think I might be a Libertarian... crap... ;)

Quote:
Again for the elitism. I guess it depends on how you look at it. Caution says. Someone will abuse this, release is a no no. Elitism says you're too freaking stupid to do this correctly sorry. (This I suppose boils down to opinions)


OTOH, there's a balance to be reached. Releasing info about the firmware could be incredibly useful to many people. Who knows what kind of mods we could see? I think that outweighs the danger of a couple bad apples attempting to damage people's DSs.

#41980 - tepples - Tue May 03, 2005 10:38 pm

TheMikaus wrote:
Difference being that you can reformat your own computer. Can't really reformat the DS. But point taken.

If you have the recovery code present in the firmware, then you can reformat the DS as long as you don't overwrite the recovery code.

I'll suggest a solution that allows firmware experimentation whilst preventing brickification, an expansion on what pipomolo was talking about:
  1. Standardize on one recovery code.
  2. Keep the flashing program proprietary for now.
  3. Let the user pass any firmware to the flashing program.
  4. Have the flashing program verify that the SHA-1 of the recovery code in the firmware matches that of the standardized recovery code.
That way, any random user can flash any random firmware to a Nintendo DS system without risk of brickifying the system because the un-brickification code is guaranteed to be present.

And yes I do thank darkfader and the rest of the DS homebrew hardware and firmware hackers for what they've done.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#41992 - FourScience - Tue May 03, 2005 11:16 pm

Good idea, tepples. I'm not up to speed on the feasability, but I like that approach.

My 2 cents about not sharing the firmware/flashing code...

I don't think that keeping it closed-source is elitist at all. I don't expect everyone to make their code open, these guys have the inherent right to keep their code private regardless of their reasons.

There are good reasons to keep it closed, IMHO. It could be abused, badly, possibly to run pirated commercial ROMs. We don't want this work used in that way, and judging by the attitude I've seen by the homebrew community, this is of great concern.

To some degree they're protecting people from themselves... and that's a good thing if you ask me, but I wouldn't say this is the primary argument for keeping this code private. If tepples' suggestion works then this argument would be void of course.

I don't really know, but I wouldn't be surprised if another proven DS hacker asked to work on the source, he'd be given access. Going back to abusive uses, there's a security concern here. I advocate keeping code closed-source if there's a risk that warez people will rip it off and abuse it.

I love and advocate OSS/FS, but not all code should be open. Ultimately, it's up to the originating author whether the source is released.

#41994 - Abcd1234 - Tue May 03, 2005 11:22 pm

FourScience wrote:
I don't think that keeping it closed-source is elitist at all. I don't expect everyone to make their code open, these guys have the inherent right to keep their code private regardless of their reasons.


Oh, I couldn't care less about the code. It's this decision to keep *information* hidden that bothers me. You can answer DsPet's questions without giving your code away.

Quote:
Going back to abusive uses, there's a security concern here. I advocate keeping code closed-source if there's a risk that warez people will rip it off and abuse it.


Right... because the warez folks out there don't have the ability to hack the firmware themselves...

#41996 - FourScience - Tue May 03, 2005 11:29 pm

Quote:
Oh, I couldn't care less about the code. It's this decision to keep *information* hidden that bothers me. You can answer DsPet's questions without giving your code away.


Gotcha.

Quote:
Right... because the warez folks out there don't have the ability to hack the firmware themselves...


Yeah they can always do it themselves, but I wouldn't hand it to them on a silver platter. Also, if they based their work on homebrew code, that could potentially put the homebrew community at risk. If I were Nintendo I wouldn't persecute homebrew for this, but some companies do that sort of thing. According to some of the irresponsibly written, recently proposed legislation in the US, some of this homebrew development could potentially be outlawed. Btw, I don't know where legislation has gone, but it's been a real concern.

#41999 - DsPet - Wed May 04, 2005 12:07 am

re: elitism v. open source
Being open source (or at least open information) is generally a good thing IMHO
(also gives you legal protection to people doing risky things, not to mention protecting the community against one person deciding to take his toys and go home...)
This is a bigger issue than this one topic.

However, that said, you can't force people to share their toys ;->
=========

re: helping virus writers or other ill motivated people.

Much of this debate is moot !

We *do* have an open-source for a program to read the firmware.
We *do* have the full specs for the SPI firmware chips (read + write)
Very easy to write a program to wipe out the firmware (except for the first bytes, without the jumper)
Very easy to patch FlashMe to brick an unsuspecting user (for those foolish enough to use an unofficial version)

Keeping the details of the NDS specific firmware structure private will not stop someone who wanted to do this.
Flashing is easy, knowing what to flash is hard.

------

So to rephrase it:

I *do* know how brickify an NDS.

What I would like to know is how *not* to brickify an NDS.
And the advice of those who have done so. Please share.

The only logical "newbie" argument I understand is that if you describe how to do, more people will try it (on their own device) and possibly brick their own NDS. This can easily be addressed by a strong warning, and recommend people install FlashMe first - and use its emergency recovery mode [ie. safe unless you short the jumper]
The possible harm is only to those who are intentionally trying to do harm to themselves

A minor downside IMHO. The upside - many experienced programmers, sharing information, most/all working towards a common goal.

#42010 - FourScience - Wed May 04, 2005 4:16 am

Well said, DsPet. I see the fallacy in thinking that ill-motivated people would really be aided by this work.

#42014 - TheMikaus - Wed May 04, 2005 6:49 am

Tepples idea sounds pretty good.

More fun!

#42047 - DsPet - Thu May 05, 2005 2:08 am

Update:
I started looking it to this myself (re-inventing the wheel waiting for someone "in the know" to spill the beans ;-)
BTW: the getID process of FlashMe is not as important as I thought it was.

--
Now my surprise:
The FlashMe program (and related restore app) includes a mostly complete copy of the Nintendo firmware, as well as a patched version that changes the functionality.

Needless to say this is Nintendo copyrighted material. Regardless of form (plain, encrypted or compressed), such blatant copying can get you into big-time legal trouble even if everything else is "fair use" under the DMCA.

People may continue to ignore the request of "please open up your source code", however...
if there is no public information, a simple Cease-And-Desist order to the FlashMe website may kill off this tool (at least in its current form)

---
I also understand that people may want to keep some things to themselves - that is their option.
For everyone else, realize that in using a non-open source program, you may be violating some else's copyright without actually knowing it !

[[ I also realize most people will ignore this warning, because they don't care if they are using a program that is blatantly copying and redistributing parts of the Nintendo firmware. But for the few people who care, I thought you may want to know ]]
(end soap box)

I'm all for open source, DMCA fair use, copyright respecting hacks.

#42055 - kulgan_22 - Thu May 05, 2005 6:37 am

DsPet : Can you explain how you did to "see and analyze" the content of of flashme?

#42062 - olimar - Thu May 05, 2005 8:17 am



Last edited by olimar on Wed Aug 20, 2008 9:39 pm; edited 2 times in total

#42081 - DsPet - Thu May 05, 2005 3:21 pm

> DsPet : Can you explain how you did to "see and analyze" the content of of flashme?
You can dump/disassemble the programs rather easily (like reverse engineering any ds.gba file). The firmware copies are laid out rather obviously (let me know if you can't find where they sync up). The restore app has full copies of the two versions of Firmware. The customized Flashme app you download for your specific device have a shortened version of the needed one.

To get the Nintendo firmware binary, use the free program to grab it from your device (256KB in four 64KB regions). Open source (thanks loopy/olimar)
http://www.natrium42.com/blog/?p=20
Don't ask me how to interpret it (I only understand the rough structure -- why I am asking for more details from people "in the know" so I don't waste me time reinventing the wheel)

Since you are grabbing the firmware yourself, you are in safe legal territory. Posting a copy (/sharing it) is a different matter -- even if you only share it with fellow NDS owners.

=====
olimar wrote:
> After all your bitching, moans, complaints, open source flag-waving, name calling, etc.,
I'll plead guilty to the first four. I don't think I did any name calling [but I won't speak for others ;-]

FWIW: I made no "warez" aspersions, which IMHO gets at people's motives. I am pointing out that copying so many binary bits of Nintendo code is in dangerous territory (what I call a grey area of piracy, a lot of people do it -- no problem until the copyright holder complains)
A future flashing program may have the same legal complications.

----
> ... not a single one of you bothered to actually contact me.
Sorry for not knowing who to contact directly. There is no contact info on the FlashMe webpage or included in the program itself
I did post in this forum (which please correct me if I'm wrong is the right place for people to post these kind of things)

> the next update will probably leave the whole original firmware open, so you can start toying with it
Excellent. That's all I was asking for, and sharing of what you've learned of the structure of the firmware file.

[see the first post on this forum - that's what started it -- but got taken a little off track with the line "Firmware modifcations should be done by professionals"]

#42084 - caitsith2 - Thu May 05, 2005 4:28 pm

DsPet wrote:

> ... not a single one of you bothered to actually contact me.
Sorry for not knowing who to contact directly. There is no contact info on the FlashMe webpage or included in the program itself
I did post in this forum (which please correct me if I'm wrong is the right place for people to post these kind of things)


Well, there is #dsdev, efnet. Loopy hangs out there, along with most of the others in DS development.

#42086 - kulgan_22 - Thu May 05, 2005 5:49 pm

To disassemble ,you use IDA Pro, or another program?

#42092 - Abcd1234 - Thu May 05, 2005 8:30 pm

olimar wrote:
Thank you for your insightful comments and speculation. After all your bitching, moans, complaints, open source flag-waving, name calling, etc., not a single one of you bothered to actually contact me.


Just wanted to point out that my own comments were only in response to [FireFly]'s. Obviously I wasn't accusing you, in particular, of anything, since you hadn't yet weighed in on the subject.

#42193 - Proteanruler - Sat May 07, 2005 2:45 pm

is there an ETA on the new firmware patch?? Not pressuring - simply curious. Thanks

#42334 - olimar - Tue May 10, 2005 3:46 am



Last edited by olimar on Wed Aug 20, 2008 9:34 pm; edited 5 times in total

#42398 - krunkster - Tue May 10, 2005 10:51 pm

Why do you (or whoever) collect unique id and serial number?
I would prefer to do this as anonymously as possible.

#42401 - olimar - Tue May 10, 2005 11:22 pm



Last edited by olimar on Wed Aug 20, 2008 9:35 pm; edited 1 time in total

#42407 - Proteanruler - Tue May 10, 2005 11:58 pm

Go Olimar go ;)... everyone is seriously grateful for what you and your crew have done. Nobody has a right to be annoyed!! Jesus - all you are owed is thanks. Does sound good though :D!

#42429 - Veg - Wed May 11, 2005 6:52 am

I haven't flashed the first firmware yet - had trouble shorting the SL1, and then gave up.

Will this second update be possible from the original DS firmware, or will I have to put the first one on first?

#42503 - DsPet - Thu May 12, 2005 3:01 am

NOTE: the updated version is apparently now available (no annoying getID step, no announcement fanfare either ...)
http://ds.gcdev.com/dsfirmware

----
Obvious advice:
> I haven't flashed the first firmware yet - had trouble shorting the SL1, and then gave up.
If you have started the firmware update (> 0%) then don't give up until it is 100% - otherwise it most likely will not boot again.
NOTE: the new version reports percent complete not hex addresses.

> Will this second update be possible from the original DS firmware, or will I have to put the first one on first?
Works for me using the original FlashMe firmware and upgrades to the latest version.

=======
Technical stuff:
> Some firmware notes (only applies to the newest flashme, not available on the website at the time of this writing):
Thanks for the pointers. Jives well with my preliminary block analysis (didn't figure out the shift values, that's weird, but I won't ask how you found it ;-)

---
Any more hints on the compression algorithm details ?
I tried using the built-in SWI LZSS/LZ77 decryption but it didn't match the standard encoding (unless I'm missing something obvious)

If a few more hints are provided, hopefully the firmware will be easy for other people to decode and rewrite (especially the PictoChat program)

TIA

#42506 - Abcd1234 - Thu May 12, 2005 6:17 am

So, just to clarify, has anyone here upgraded to the latest firmware patch while having the old one installed at the time? Any issues? Or did you just uninstall the old patch and install the new one?

Incidentally, yes, I realize I'm being paranoid. :)

#42510 - caitsith2 - Thu May 12, 2005 7:41 am

DsPet: The firmware is not only compressed, but it is also ecrypted. It has to be decrypted first, before the decompression step will succeed.

#42514 - olimar - Thu May 12, 2005 8:56 am



Last edited by olimar on Wed Aug 20, 2008 9:35 pm; edited 1 time in total

#42516 - caitsith2 - Thu May 12, 2005 9:59 am

I stand corrected.

#42533 - Abcd1234 - Thu May 12, 2005 6:05 pm

olimar wrote:
Abcd1234 - the new flashme can be installed on top of a previous installation or on a fresh system, it shouldn't matter.


Sweet! Looks like I'll be reflashing my DS after work tonight. Awesome work, guys. Thanks again!

#42700 - lambi1982 - Sat May 14, 2005 2:59 am

i keep getting "gba file is corrupted" why?
_________________
Who, Me?

#42701 - olimar - Sat May 14, 2005 3:05 am



Last edited by olimar on Wed Aug 20, 2008 9:36 pm; edited 1 time in total

#42702 - lambi1982 - Sat May 14, 2005 3:17 am

dont have that cart buddy, never had a problem with the first firmware, I have ezf3 advanced loader.bin does nothing for this cart, when i put older flashme on cart works as expected, just this new download says corrupted
_________________
Who, Me?

#42704 - olimar - Sat May 14, 2005 3:24 am



Last edited by olimar on Wed Aug 20, 2008 9:36 pm; edited 1 time in total

#42707 - lambi1982 - Sat May 14, 2005 3:29 am

EZF ADVANCE III 256Mb, I downloaded it 10 times so far from http://ds.gcdev.com/dsfirmware every time I try and put it on the cart I get CRC32 error only with that flash file, I tried it with my older flash and all went well. just dont know what is up
_________________
Who, Me?

#42749 - lambi1982 - Sat May 14, 2005 4:57 pm

Is there any other place to download this copy _ new flashme ) so I can test another, I just keep getting GBA file is corrupted
_________________
Who, Me?

#42791 - Sektor - Sun May 15, 2005 1:30 am

Did you read this? "Note for EZ-Flash III users: Extract loader.bin of ezf3me.zip to the flash manager directory." I don't know if that applies to EZF Advance III, very similar names but no idea if software has the same problems.

Last edited by Sektor on Mon May 16, 2005 12:31 pm; edited 3 times in total

#42795 - lambi1982 - Sun May 15, 2005 2:15 am

did you read what I wrote, I do not have the ez flash 3 cart, i have ezf advance 3 cart, no loader.bin in the folder, does not use one...if adding loader.bin was all it was it would have worked by now
_________________
Who, Me?

#42907 - lambi1982 - Mon May 16, 2005 4:27 pm

http://imageserver0.textamerica.com/user.images.x/32/IMG_421032/_0516/TZ200516081801157.jpg

this is a screen shot just before I get crc32 error when transfering the flashme file.
_________________
Who, Me?

#43741 - pceslayer - Thu May 26, 2005 3:33 am

I think thats the card they meant... I could be wrong... try a different loader instead of Ecube