#153766 - yellowstar - Sat Apr 05, 2008 7:08 pm
I'm writing a tool to assemble WMB packet captures, and other protocols for DS Demo/NDS binary transfers.
See the posts near the end of this topic for the latest developments.
The tool has been released. Wmb Asm Google Code Project Page
The posts from here, the start, are the troubles, solutions, developments, and other things, from the very beginning of the Wmb Asm development. Note that most of the problems posted here with Wmb Asm, are pre-release issues, or have since been corrected since the last Google Code Wmb Asm release, 2.0b r2. Any issues mentioned after the r2 release are issues only with SVN. You may have these issues if you use Wmb Asm compiled from SVN. Not all of the issues, and the fixed SVN issues are mentioned here, so read this, for the links at the end, for the Wmb Asm SVN issues.
But, I'm having a lot of trouble piecing the beacons together.(Banner data)
Unlike what some sources say, the other seq fields, they are always zero. And that's how I'd normally piece together the beacons.
Does anybody have any ideas on how to piece together the beacons correctly? I tried finding beacon #9 by checking if the data is null, but it didn't even detect that packet.
Here's some log data and code to give an idea of what I'm talking about:
Code: |
FOUND BEACON DSSEQ 0 AD 0 RS 3 CON 8
FOUND BEACON DSSEQ 0 AD 0 RS 3 CON 8
FOUND BEACON DSSEQ 0 AD 0 RS 3 CON 8
FOUND BEACON DSSEQ 0 AD 0 RS 3 CON 8
FOUND BEACON DSSEQ 0 AD 79 RS 4 CON 8
FOUND BEACON DSSEQ 0 AD 79 RS 5 CON 8
FOUND BEACON DSSEQ 0 AD 79 RS 6 CON 8
FOUND BEACON DSSEQ 0 AD 79 RS 7 CON 8
FOUND BEACON DSSEQ 0 AD 79 RS 8 CON 8
FOUND BEACON DSSEQ 0 AD 79 RS 9 CON 8
FOUND BEACON DSSEQ 0 AD 79 RS 10 CON 8
FOUND BEACON DSSEQ 0 AD 79 RS 11 CON 8
FOUND BEACON DSSEQ 0 AD 79 RS 12 CON 8
FOUND BEACON DSSEQ 0 AD 79 RS 13 CON 8
FOUND BEACON DSSEQ 0 AD 79 RS 14 CON 8
FOUND BEACON DSSEQ 0 AD 79 RS 15 CON 8
FOUND BEACON DSSEQ 0 AD 79 RS 16 CON 8
FOUND BEACON DSSEQ 0 AD 79 RS 17 CON 8
|
Code: |
struct beacon {
unsigned char unk0[4];
unsigned char destmac[6];
unsigned char srcmac[6];
unsigned char bssmac[6];
unsigned short seq;// = RS
unsigned int ts1;
unsigned int ts2;
unsigned short interval;
unsigned short capability;
unsigned char tagparms[4];
unsigned char dsparms[3];
unsigned char tidm[7];
unsigned char vendor[2];
} __attribute__ ((__packed__));
struct ds_element {
unsigned char manufacturer[3];
unsigned char unk03;
unsigned char unk04[4];
unsigned char unk08[4];
unsigned char unk0c[4];
unsigned short streamcode;
unsigned char unk12[2];
unsigned char unk14[4];
unsigned char unk18[4];
unsigned char non_advert;
unsigned char unk1D;
unsigned char connected_clients;// = CON
unsigned char sequence_number;// = DSSEQ
unsigned short checksum;
unsigned char advert_sequence_number;// = AD
unsigned char advert_len;
unsigned short data_size;
unsigned char data[256];
} __attribute__ ((__packed__));
|
Last edited by yellowstar on Thu Nov 12, 2009 9:22 pm; edited 35 times in total
#153839 - yellowstar - Mon Apr 07, 2008 1:20 am
I am trying to assemble the beacons, without caring about the seq for now.(This capture I'm now using has packets with RS set to 1-11)
But, it's not working. The icon doesn't even show up at all...
Code: |
void CombineBeacons()
{
int magic = INTER_MAGIC;
int temp=0;
unsigned char buffer[188];
memset(buffer,0,188);
iee80211_framehead2 *framehead = (iee80211_framehead2*)buffer;
beacon *Beacon = (beacon*)buffer;
ds_element *ds = (ds_element*)&buffer[52];
ds_advert ad;
unsigned char *Ad = (unsigned char*)&ad;
tNDSHeader ndshdr;
tNDSBanner banner;
FILE *nds=NULL;
size_t length=98;
inter = fopen("itr.itr","rb");
fread(&temp,1,4,inter);
if(temp!=magic)return;
beacon_dig_seq=0;
while(temp==magic)
{
fread(&temp,1,4,inter);//The magic number is before every beacon
fread(buffer,1,188,inter);
if(beacon_dig_seq==8)length=72;
if(beacon_dig_seq==9)length=0;//So we don't copy anything
if(length>0)
memcpy(Ad,ds->data,length);
Ad+=length;
beacon_dig_seq++;
fread(&temp,1,4,inter);
}
fclose(inter);
memset(&ndshdr,0,sizeof(tNDSHeader));
memset(&banner,0,sizeof(tNDSBanner));
memcpy(banner.palette,ad.icon_pallete,32);
memcpy(banner.icon,ad.icon,512);
memset(banner.titles,0,6*128);
nds=fopen("output.nds","wb");
fwrite(&ndshdr,1,sizeof(tNDSHeader),nds);
fwrite(&banner,1,sizeof(tNDSBanner),nds);
fclose(nds);
fprintf(log,"BSQ %d\n",(int)beacon_dig_seq);//It only makes it to 1, it should be 10...
}
|
#154713 - yellowstar - Sun Apr 20, 2008 8:49 pm
EURKA! I was digging through gbatek yesterday, and I discovered it has stuff about WMB. It had about the same 0xDD tag as that other document I use, plus more info. Then, I opened up Wireshark to my captures I'm using.(Technically they're not mine, but anyway)
And I found that tag. It wasn't at the position my code, or masscat's WMB client presumed it to be, it was after a couple more tags. Then something clicked. I realized that to get the correct data, we'd need to read one byte after the TIM data. Then we'd repeatedly compare the number read to the Nintendo IE, 0xDD.(There's another tag that seems to be Nintendo's tag, but what is it?)
If the tag is the tag we want, we could compare the tag length to what it's supposed to be, at least normal, and then jump to the actual data if it's correct. Otherwise, we'd skip the tag by using the tag's length, then keep repeating until I find the right tag.
This tag contains the ds_element struct. So, now we'll have both the actual ad and the sequence number, and everything else we need.
I haven't tried anything yet, but this has to work.
#154933 - yellowstar - Wed Apr 23, 2008 12:49 am
I have gotten the icon to work. I get the big red N on the Namco Museum, and the DS Custom Robo cube on that demo, with the demos from here.
Problem is, only the icon seems to work. On namco I only get a bunch of characters used when the character wasn't recognized,(Like in notepad)
and Robo, with DS-X, only displayed filename.
Could somebody do me a favor? Could somebody capture a WMB transfer?(It needs to be English, I don't know if the Namco is English, and the other I'm sure that it is not English)
You should capture the whole transfer, and repeat 2 more times for a total of three times. I'm saying should because it would be great to have a capture that I'm sure that it's in English, and because, at the bare minim, you only need to capture the beacons/ad/banner for a while. It doesn't matter what the WMB transfer is, but I'd prefer a transfer of a official game doing multiplayer WMB, with a bootloader.(The little loader that loads/transfers the rest of the game)
The other ones that can be captured is demos, either from official games sending demos, or from Download Stations.(I could only assemble the Station's client if you would go that way)
#155118 - yellowstar - Fri Apr 25, 2008 4:23 am
I have finished the code to process the Authentication frames, to find the client's mac. But now I'm stuck on the rsa frame. I'm positive my code is correct, Wireshark agrees with what my code finds. For some reason, the size field is 0x73. As NdsTech Wiki and gbatek say, it's supposed to be:
Quote: |
There are several abortive sendings of empty RSA frames with a size field of 0x03, before the real frame is sent (always with a size field of 0x75).
|
Strangely, the captures I have do not have those 0x03 size packets. And the size is 0x73, not 0x75. Major problem - that screws the rsa signature. In decimal, the size is 115. The size of the wanted is 232. The difference and lost data is 117 bytes. What's going on???
Quote: |
Host-to-client packets (on the 0x00 flow)
0 1 2 3 4 5 6..e-3 e-2 e-1 e-0
06 01 02 00 Size Flags Payload 00 02 00
The size field is in terms of half-words (16 bits), and includes the flags byte along with the payload (so a size of 0x03 represents a flag byte, a command byte, and 4 bytes of payload).
|
Huh? In the little chart-thing it says it's one byte in size. But in the text it says it's a half-word? Huh? I tried shifting 115 by 2 to the left and right and didn't get correct values...
Hold on... I figured it out now. Shifting it to the left by one does the trick. However the result needs 2 added to it to match the size of the rsa frame struct.
I'll try this tomarrow... Time for bed.
#155188 - yellowstar - Sat Apr 26, 2008 4:44 am
Okay, I'm done with RSA. And I finished the header processing. Now I need to do the most important part: the Arm9 & 7 data.
Ah Ha! I checked that image included in the Namco archive, and guess what? This Namco demo really is Japanese! Horray!(Well... Not the fact I can't understand it, yay to the fact I know what language it is now. Note that on a hex editor, text that would appear as Japanese in-game, wouldn't look anything like Japanese in hex editors.)
Does anybody know of any libraries to read UFT-16 Little Endian strings?(For possibly writing my own .nds banner viewer since nobody responded to that topic.)
#155201 - tepples - Sat Apr 26, 2008 12:27 pm
Have you read through UTF-16? Windows uses it as one of its native character encodings. What about UTF-16 do you need to handle, and on what platform (PC or DS)?
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#155208 - yellowstar - Sat Apr 26, 2008 5:30 pm
The text that will be processed, would be the banner description in a .nds, on a PC.
The text that gets displayed in Wordpad and Notepad, is about the same as in Programmers Notepad.(Which is about the same as viewing the text in a hex editor)
EDIT:
Does anybody know of any documents for the DS Download Station protocol? I heard it's similar to WMB, but that's all I heard... I want to implement DS Download Station download capture assembly into this app, but I don't I have any documents to reference, and I don't have any Download Station captures either.(Captures please?)
#155210 - Filb - Sat Apr 26, 2008 6:03 pm
Are you talking about the American DS Download Station?
Why capture them? I think they're all being dumped.
I don't know much about them, just that the demos are being sent compressed after downloading some different download client over DS Download Play.
The Japanese Download Stations (and the Nintendo Channel (Wii)) send their content uncompressed and directly over DS Download Play. I think it'd be much nicer to have a tool for these. I could possibly provide captures for them (all Japanese).
#155213 - yellowstar - Sat Apr 26, 2008 6:26 pm
Yes, I meant American.
Download Station implemention is not going to delay the release of this tool. In fact, it's almost done - just need to write the code to grab the data from the capture, and writing the data to the .nds.
Once I'm done with this tool, I'm thinking about porting this tool to the DS. So you could transfer the captures to your homebrew card, assemble them, and run them from the card quickly.(Well, that depends on how long assembly takes, and the speed of the card.)
Maybe even a active capture system? A system that, on DS,(maybe PC, wouldn't work very well with most wireless cards though)
would capture wireless packets strait off the air, optionally save them in a cap file if wanted, and assemble them on-the-fly, processing data right off the air immidately, instead of from cap files.
(This depends on if I can get code working right to do captures - I tried capturing packets and saving them to fat before, but that failed. My hb card is broken, so that makes it really difficult. Perhaps I could start a public testing topic on these forums, so anybody and everybody could test?)
#155221 - Filb - Sat Apr 26, 2008 7:52 pm
Test what?
If I understood right your tool can currently capture the packets in pcap (or whatever it's called) format? That format Ethereal can read, right?
I could test that. Just add me to MSN or send a PM.
#155277 - yellowstar - Sun Apr 27, 2008 6:52 pm
Filb wrote: |
Test what?
If I understood right your tool can currently capture the packets in pcap (or whatever it's called) format? That format Ethereal can read, right?
|
It called the cap format. Ethereal is old and probably shouldn't be used. I use Wireshark.
For testing, I meant the DS port in the future. If it weren't for one bug, this tool would be done. The output .nds is way bigger than the Namco .nds that comes with the archive. I think the reason for this, is, the gap between the header and arm9 binary, between arm9 and 7, and between the 7 and banner. These gaps seem to be alot bigger than they should be. Debugging time later... Also there's this one index in a big array, which dictates whether or not a data packet was seen. And for some strange reason, at index 1983,(The indexes stand for the data packets sequence number)
is zero. I checked what it is right after it is set to 1, and it was fine. But at the point were I check if all the data packets were found, via that array, that index/seq is zero, whilst it should be 1. I hacked it so it is ALWAYS 1, no matter what. Then I ran into the above problem, with the big .nds.
By the way, I tried that NDSHeader tool again on the output .nds with the big ouput .nds, and this time I got a black icon...
#155288 - Filb - Sun Apr 27, 2008 8:30 pm
I can test your program whenever it's ready.
I don't know why your .nds file is so big, sorry. Does it still boot?
If you're working with my NAMCO MUSEUM capture, please note that the game is in there THREE times. (Three captures of the same game in one file.)
#155309 - yellowstar - Mon Apr 28, 2008 2:00 am
Filb wrote: |
I can test your program whenever it's ready.
I don't know why your .nds file is so big, sorry. Does it still boot?
If you're working with my NAMCO MUSEUM capture, please note that the game is in there THREE times. (Three captures of the same game in one file.) |
I have fixed that big .nds output problem. Now, the .nds is alot smaller than it should be. It's ~184KB, while it should be ~900KB. I discovered that I was doing the gap calculation code wrong. I thought the order in a .nds, was, Arm7 -> Arm9. But, it's really the other way around, Arm9 -> Arm7. And then it started crashing. Then I found that I was doing the subtraction wrong. Fixed that, and now I'm at the small .nds problem.
I don't think it would boot on hardware... All the emulators I tried,(I tried most of the emulators available)
just gave me double black screens, and reported that they're stuck at address zero. Dualis did the black screens, but when I tried the .nds that came with the archive, it worked. Well, it actually booted and ect. Debugging again...(If you didn't catch it the first time, my homebrew card is broken, and thus I can't test on hardware)
Multiple captures on the same WMB transfer in the same cap file can't affect this program. This program allocates memory for the data, will only copy data into the buffer if it didn't see that data packet before, and won't copy a second time if it sees it again.
I prefer to contact people via PM. The only time I consider other means of contacting, is when they seem to have not been here on gbadev in a while. I'm considering having you, Filb, test the program first, when it's done. So, I'd send you a PM with a link to the program, then if all of your captures work, I'll release the program.
Just a warning:
This program is command-line/DOS/ect. I can't write a GUI front-end for this. But, if people would help with the problems I have with wxDev-Cpp, maybe I could write such a front-end.(compiling errors.)
By the way, that Namco capture is my favorite. The others, not so much. The other captures have extra, non-wmb packets.(This is mainly when I analyze the captures. They don't cause problems during assembly, it's just me.)
#155383 - yellowstar - Mon Apr 28, 2008 11:41 pm
Take a look at this wacky output from fields in the header:
Code: |
ARM9SOURCE 16384 ENDIAN 16384
ARM7SOURCE 847360 ENDIAN 15600640
ARM9EXE 33556560 ENDIAN 1342701570
ARM7EXE 37224448 ENDIAN 37224448
ARM9DST 33554432 ENDIAN 2
ARM7DST 37224448 ENDIAN 14338
ARM9SIZE 830568 ENDIAN 1756105728
ARM7SIZE 168732 ENDIAN 479396352
ARM7S 1649 ARM7E 1984 DIFF 335
502 ARM7S 827798 ARM7E 995968 DIFF 168170
|
ARM7S and E is the total seqs in the transfer. S is the total seqs for Arm9, Diff is the seqs for Arm7. The 502 line is the size of binaries calculated from the seqs.
According to the above output, the execution addresses and dest is over an address of ~30 Mega Bytes! Very wrong! But I'm thinking that the values are stored in some way that a DS can read directly, while on PC it would need converted. I thought endians were the problem, but nope.
This values don't affect the program, at this time, but they need corrected. Why? Well, I opened up the Namco .nds in a hex editor, jumped to the Arm9 source, moved forward by 2008 bytes, which is the sum the total data packets with null data multiplied by 502, the size of the data packet. In this case, 4*502. Next I moved by the number of bytes till were the actual Arm9 binary should have started, by counting the number of bytes from the start of the first non-null data, to first non-null byte. Jumping by that number landed me at the wrong spot. So, I need to know how to fix the dest field to fix this.
#155387 - tepples - Tue Apr 29, 2008 12:33 am
yellowstar wrote: |
According to the above output, the execution addresses and dest is over an address of ~30 Mega Bytes! Very wrong! |
Really? The number 33,554,432 equals 0x02000000, which happens to be the address of the start of the DS's 4 MiB EWRAM. No es wacky.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#155390 - yellowstar - Tue Apr 29, 2008 1:15 am
So the bootloaders use that address to tell where to copy the binaries...
Keep on debugging...
#155477 - yellowstar - Tue Apr 29, 2008 11:43 pm
I got the output to around the correct size - but apparently it's about 3 Kbytes off/below than what it should be... I was using an 2D array, but when I switched to 1D, it worked.
And the emulator still crashed. I think I found the problem. The .nds in the download has the "DOWNLOAD PLAY" string after the header. Whenever this shows up, there's a duplicate right after that string, which is the one actually used. I need to know how to detect if that property exists in a WMB transfer - that string doesn't show in the transfer, I need to know however WMB tells when that string is present. That string screws all the addresses in the .nds, if originally the string was present, and is not anymore. "Landing at the wrong spot", as I said before, is the main problem caused by this.
#155496 - yellowstar - Wed Apr 30, 2008 4:20 am
Ugghhh...
I checked the addresses from an homebrew, and the destination and execution addrs are the same. And I tried changing the execute addrs to the to where the code seemingly starts, but nothing. What happens when hardware, or emu, encounters an opcode that is zero?(andeq r0,r0,r0)
Everything but iDeas has 1 FPS all the time, which means those emus crashed. But iDeas still has black screens. And when I try using the debugger, the code it's at doesn't change. Even in a working homebrew, it won't behave! I remember it worked before in a old version of iDeas. Strange... iDeas works with the included .nds for the Namco demo.(Dualis still works of course)
I tried using a calculation of mine to jump to a certain address, and I land at exact same spot as in my assembled .nds. Ahh ha! It's the sizes again! Because the sizes don't match, that's reading more data than it should normally, causing crashes and whatnot on emus & hardware. Question is, how do I determine the end sequence number for arm9, and the end for arm7? The way I'm doing it now, which is the way Juglak does it in his WMB_Host, doesn't do it exactly right, and in so produces this bug.
#155503 - HyperHacker - Wed Apr 30, 2008 7:22 am
Pretty sure opcode 0 is NOP, i.e. do nothing. The duplicated string might be a glitch in the capture.
_________________
I'm a PSP hacker now, but I still <3 DS.
#155518 - Filb - Wed Apr 30, 2008 4:00 pm
"DOWNLOAD PLAY" string?
Say, are you talking about this part?
This part is NOT from the .cap files.
It's ADDITIONAL information for wmb.exe (download station's name etc.).
I think it's even in the reserved area of the NDS header which is usually unused.
#155549 - yellowstar - Wed Apr 30, 2008 9:43 pm
Yes, that's what I'm talking about. WMB doesn't send that reserved section... That I know of... Juglak's WMB Host doesn't send that section anyway...
#155550 - Filb - Wed Apr 30, 2008 9:46 pm
It's a thing FireFly decided to put into the assembled files, I think.
At 0x390 you can see the hex for "ニンテンドーCh" (Nintendo Channel), which is the original sender of it. Normally this information is not stored in the binary. But WMB uses it to display this sender when resending it.
So yeah, you can completely ignore this area, if it causes errors.
#155556 - yellowstar - Wed Apr 30, 2008 10:33 pm
I checked the capture, and I saw what you described. Well, I guess I can just ignore that data. Would be nice to have the output EXACTLY the same as Firefly's though.
Maybe I could do that? Perhaps I could check the host's name, and check if it's the Wii Nintendo Channel, or if it's from... Hmm... It probably would be easier to just stick that data in all the time, if I ever decide to. But, I guess I'll just leave it out.
#155557 - Filb - Wed Apr 30, 2008 10:35 pm
Not sure, but you could probably just inject the data *after* assembling everything? That part of a nds binary should always be unused.
#155559 - yellowstar - Wed Apr 30, 2008 11:19 pm
No, that would have to be done during assembly, not after. And putting that string anywhere other than where it is normally wouldn't make the .nds %100 exactly the same as Firefly's.(I assume you meant that string and such)
#155586 - yellowstar - Thu May 01, 2008 4:25 am
I tried an method for handling the data, but it didn't work. I had it rigged so it will only process data packets in order, it would refuse to handle anything not immediately after the packet it last successfully processed.
Next I'm going to try another method. This time I'm going to have the sizes of each data packet stored alongside the saved data, which would be used during assembly. And also whether it's doing either arm9, 7, or if it's done, would be processed on-the-fly, based on the binaries sizes and other things.
#155886 - yellowstar - Mon May 05, 2008 4:57 am
I have tried to implement that DS DOWNLOAD PLAY string, but... The addresses are screwed, still working on that... And there's data in the reserved section in the header!(The duplicate one after the string, in the prebuilt nds)
That section isn't even sent in WMB, that I know of. It would help alot if I had another capture with a prebuilt .nds. There's a Japanese Meteos demo on Firefly's web site, so could you possibly capture that demo Filb?
#155888 - Filb - Mon May 05, 2008 9:13 am
Since I have to go out in a few minutes, I'll just give you another game I already captured and got assembled. It's SUDOKU2 Deluxe demo, also from the Wii Nintendo Channel (Japanese).
SUDOKU2 Deluxe Demo Version (2.7 MB)
Might capture a Japanese Meteos demo (using the commercial game) later, if still needed.
Good luck.
Edit: I also have some "worst case" captures of a real Japanese DS Download station: Download. Those captures have more than 1 game in one capture file. Those should be EIGODUKE, KURUKESHI!, GunBullet, BIOUM and maybe more.
#155932 - yellowstar - Mon May 05, 2008 10:26 pm
Thanks, I'll use that sudoku .nds & capture.
How did you assemble it? Is there some software out there I don't know about, or did you PM somebody here?
#156003 - Filb - Tue May 06, 2008 6:52 am
Well, SUDOKU2 was (like the Namco one) asssembled by FireFly back in 2007.
There is a tool (discontinued) from my good friend Frz that can assemble some captures, but it doesn't work with the Wii Nintendo Channel ones (like Namco and SUDOKU2) at all.
Some of the "successfully" assembled captures are not valid, too. :(
It does work with that "worst case" capture, but not perfectly.
#156111 - yellowstar - Tue May 06, 2008 11:51 pm
I checked the source for that tool, and, well, that didn't help me with my problem.
I have another idea to make this work. Right now the assembly is done all at once at the end. I'm thinking of trying to assemble on-the-fly. That is, data would be written to the .nds as soon the packets for them are found. Such as binary data, header, rsa...(RSA would have to be saved, to be written later after every thing else, because it's among the first thing sent, and the very last thing in the .nds.)
*gasp* ~45 MBytes for that DS Download Station capture? O_O
Guess I'll have to download that overnight, or have it download most of the day tomorrow, with Dial-Up. No need to do any more Download Station captures, this is fine for now.
#156168 - Filb - Wed May 07, 2008 11:30 am
Yeah, but please note that this is NOT from an American DS Download Station but a Japanese one, which doesn't use this strange custom loader but the normal "DS Download Play" feature instead.
#156225 - yellowstar - Wed May 07, 2008 10:11 pm
Well, I knew it was Japanese, just not that it was normal WMB. Nuts... I forgot to start the download overnight... I'll get it downloaded eventually...
#156319 - yellowstar - Thu May 08, 2008 9:52 pm
The Nintendo Channel has been released in America!(Would be nice to have English captures... No offense Flib. It would just be nice to see the fruits of my labor in a language I can understand.)
(By the way, the channel doesn't affect any Wii hacks, as Wiibrew said. Wiibrew is where I found out about the channel being release in the first place.)
#156322 - Filb - Thu May 08, 2008 9:57 pm
Good luck then...
The European channel will be out later this month, which I also have access to.
I don't have an American Wii though.
#156326 - yellowstar - Thu May 08, 2008 10:04 pm
European captures are fine, since they should be in English. Sadly, I can't try out the Nintendo Channel till my vacation, which is 3 months away. )-:
#156327 - Filb - Thu May 08, 2008 10:08 pm
I see.
By the way, if you're not going to work with the "worst case" Japanese DS Download Play captures I posted (and get them working), then your tool will pretty much be useless to me, sadly. :( I hope it'll be useful for other people, though.
#156330 - yellowstar - Thu May 08, 2008 10:17 pm
Don't worry - I'm working on it. It's downloading as we speak, half way done in fact. I'll need to write code so it would handle multiple games in the same capture, so that worst case capture is assembled correctly.
#156331 - Filb - Thu May 08, 2008 10:21 pm
Ah, cool.
Sorry, I still have a few questions about your tool. It would run on the DS, right? Will it capture the wireless data from other DS' traffic or will it be a WMB client communicating directly with the host?
If it's not a WMB client, would it tell you which games' beacons have been found and which games' data has been fully captured? (With an on-screen-message like "Game #1 (GAMETITLE_NTRJ) from host 12AB: _ of _ packets captured" etc.?)
#156358 - yellowstar - Thu May 08, 2008 11:28 pm
I was thinking about doing a DS port, it doesn't exist yet, only a PC command-line version. Remember, you offered it test it when it gets done.(Unless you meant something else...)
It's not a WMB client. Basically with the DS version would be running on one DS, while nearby there's a WMB transfer happening. The homebrew would capture & assemble the transfer on-the-fly. So the demo would be assembled after the transfer happens a few times.
Right now you use it like this:
wmb_asm.exe capture.cap <output.nds>
capture.cap is the capture is the capture file, and output is optional. Normally the output is the same as capture.cap but with an .nds extension. When the output option exists, that filename is used. As for that front-end... I really dislike wxDev-Cpp's RAD. I really like the rest of the IDE though. I had an awful time when trying to resize some windows and such when I was following a tutorial. *grumble*
Well, it reports errors... Like when it fails to find all the beacons, and data packets. It doesn't report what game name it found and such. I guess I should implement what you said.
Boy, this forum is really busy today.(I have it set so the forum sends me E-mails whenever somebody replies to the topics I post in)
#156361 - Filb - Thu May 08, 2008 11:31 pm
I see. Sounds great. Looking forward to testing it.
Personally, I don't mind needing a second DS for getting a nearby WMB transfer happening, so that's fine, too. ;)
#156379 - yellowstar - Fri May 09, 2008 3:26 am
I have implemented your suggestions.(About the game title)
The program now displays the game's title, and it now uses that for the filename when a output filename isn't specified. I also cleaned up the code.
#156482 - Filb - Sat May 10, 2008 3:30 pm
I have a capture from the American Nintendo Channel for you. It was captured by my friend.
→ Download (2.3 MB)
Good luck.
#156490 - yellowstar - Sat May 10, 2008 6:45 pm
Thanks! But I guess I'll need to find a assembled .nds of that demo. And that worst case download finished. But... Could you explain what all those captures are in the archive?
#156491 - Filb - Sat May 10, 2008 7:01 pm
I think I told you earlier what they are.
Those captures are from a real DS Download Service station from a real store in Japan.
- There are multiple demos in one capture file.
- There may be demos with data over multiple capture files.
- There are 5 (or more?) demos in those files. (Game titles: EIGODUKE, KURUKESHI!, GunBullet, BIOUM, NINTENDO.)
- Some of them might be incomplete. I don't know which.
I have assembled binaries of bioum and Kurukeshi!:
→ Download
I can't provide a perfectly assembled binary of the Brain Age 2 Demo from the American Nintendo Channel. But you said you wanted captures of it, so I got you one. :P There shouldn't be a technical difference to the Japanese Nintendo Channel captures.
#156494 - yellowstar - Sat May 10, 2008 7:34 pm
I found an assembled version of that Brain Age 2 demo.(From the same place I posted a link to in the "Official DS Demos" topic in the DS Flash Equipment section. And I also found an English Custom Robo .nds.)
Quote: |
- There may be demos with data over multiple capture files. |
Oh dear... Guess I'll have to add functionality to assemble from captures in one particular directory, along with assembling demos across multiple captures...(Meaning that might be a difficult task, coding the tool to assemble across several captures for a demo spread between them)
A while ago I tried checking the RSA signature with the tool I mentioned in the "Nintendo's signing bug" topic. But, the assembled .nds from my tool, the romSize field in the header is different than most. So different, it fails the RSA-signature check used to check if the binary has a signature. I'll need to redownload OpenSSL to mod that tool so it skips that check...(To see if it passes the RSA verification test, to see if things in the binary is different than they should be)(This also means that there's going to be an update to that WMB_Host mod of mine)
How does one get the character for foward slashes?(Directories on Unix. Piratically the only way to do WMB packet captures on Windows is with Firefly's capture tool & ralink cards, and some special USB adapter for it. No such trouble on Unix & co. That's one of the reasons why this tool will be cross platform. People will have to compile the non-windows binary however... Until people host a binary for people to download.)
Another thing: How does one handle directories in a cross-platform way?(Mainly scanning a directories contents for captures, and sub dirs for captures)
#156495 - Filb - Sat May 10, 2008 7:48 pm
Good find with the Brain Age 2 one. Seems like it's the same game binary, with just a different banner description.
It's up to you if you want to work with the worst case captures. ;)
I think you should probably ignore this part...
Quote: |
- There may be demos with data over multiple capture files. |
Sounds like it won't be of any use in your final app.
It seems like "capture-20060502-130439.cap" contains 100% of "Kurukeshi!".
And "capture-20060523-172331.cap" seems to contain 100% of "bioum".
These could be a good ones to work with. :)
I hope this helps.
#156497 - yellowstar - Sat May 10, 2008 8:21 pm
I should get that feature implemented. I can imagine pretty well that somebody would do exactly that, have each WMB transfer in a separate capture file.(I'm referring to anybody that will use this tool, for the capture stage. I'm not referring to you Filb)
#156499 - yellowstar - Sat May 10, 2008 8:40 pm
cprogramming.com wrote: |
To access a directory, and the files within it, is a very compiler/OS specific task. It's not practicable to show how to do this for all variations...
|
I guess I should investigate the Boost filesystem/directory library I heard about... But I'll need to write some kind NDS wrapper so it works on DS.
#156507 - tepples - Sat May 10, 2008 10:04 pm
yellowstar wrote: |
How does one get the character for foward slashes? |
'/'
Incidentally, MSVCRT's fopen() accepts paths containing '/' (a slash) as easily as it accepts paths containing '\\' (a backslash). I don't know whether this feature is a part of Win32 itself or a translation by MSVCRT, but it does work in programs compiled with MinGW. Command-line applications that use '/' instead of '-' to introduce switches might get confused, but that's no problem unless you're calling system(), ShellExecute(), or one of the exec or spawn functions.
Quote: |
Another thing: How does one handle directories in a cross-platform way?(Mainly scanning a directories contents for captures, and sub dirs for captures) |
Roughly, "scanning a directory" means producing a list of values of filename for which fopen(filename, "rb") will succeed. But unfortunately, ANSI C provides no way to do this.
- On *n?x, use dirent.
- On Windows, use Kevlin Henney's dirent, which implements the same interface.
- On devkitARM, use dir.h, which implements a slightly different interface that automatically calls stat() for each filename.
So to hit the important platforms, you can wrap dirent to look like dir.h, or vice versa.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#156509 - yellowstar - Sat May 10, 2008 10:24 pm
My bad, I meant back-slash. Thanks for mentioning the correct way to do back-slashes. GNU was reporting errors when I was using '\'. Now that I use '\\', it works correctly. I think I'll try what you said, wrapping dka dir to dirent.
#156517 - yellowstar - Sun May 11, 2008 1:40 am
I gotten my tool to handle the "DS DOWNLOAD PLAY" string correctly. Now the first part of the .nds, and the archive's nds, look the same for the first part, for the headers. Still no change for the behavior with the emulators.
I have found that my assembled .nds is 182 bytes smaller than the archive .nds. It is VERY important for my .nds & the archive .nds to be EXACTLY the same.
#156583 - yellowstar - Sun May 11, 2008 7:46 pm
I have found that new method I was going to try, wouldn't work.(/w same behavior as that other method)(I didn't actually try it, it's theory that's probably correct)
So, I was thinking, is there a way, with libpcap, to go back in a capture file? So my tool could go back to a previous packet, whenever it would run into the wrong packet.(That is, the wrong data packet, signified by if the internal seq isn't +1 higher.)
#156584 - zzo38computer - Sun May 11, 2008 8:04 pm
Filb wrote: |
I think I told you earlier what they are.
Those captures are from a real DS Download Service station from a real store in Japan.
- There are multiple demos in one capture file.
- There may be demos with data over multiple capture files.
- There are 5 (or more?) demos in those files. (Game titles: EIGODUKE, KURUKESHI!, GunBullet, BIOUM, NINTENDO.)
- Some of them might be incomplete. I don't know which.
I have assembled binaries of bioum and Kurukeshi!:
→ Download
|
The file doesn't exist. Can you put file on (and the other one as well), so it can be downloaded
_________________
Important: Please send messages about FWNITRO to the public forum, not privately to me.
#156588 - Filb - Sun May 11, 2008 8:41 pm
zzo38computer wrote: |
The file doesn't exist. Can you put file on (and the other one as well), so it can be downloaded |
I removed them as I thought they would no longer be needed. I uploaded them for you again.
#156650 - yellowstar - Mon May 12, 2008 3:43 am
I found have found that while libpcap doesn't directly support seeking through the capture file, it does have a function to get the FILE* of the capture. So I can use that to seek to where I want.(I'll have to do some more things in my code to get this to work right... Mainly calculating how to find the address to seek to)
Code: |
FILE *pcap_file(pcap_t *); //Use this to get the FILE* for the capture file
|
I think I might be on the edge of a breakthrough... This modified method of that bugged method I tried before will have to work... Hopefully anyway... But I can't try it tonight...
I have tried that rsa verification tool, modified to skip that step, and the assembled .nds failed really bad. Most of one of the hashes was almost all zero.(Perhaps it's because the size of this .nds is wrong?)
#156652 - zzo38computer - Mon May 12, 2008 3:58 am
Filb wrote: |
zzo38computer wrote: | The file doesn't exist. Can you put file on (and the other one as well), so it can be downloaded |
I removed them as I thought they would no longer be needed. I uploaded them for you again. |
OK, thanks now I downloaded that file(s) OK
_________________
Important: Please send messages about FWNITRO to the public forum, not privately to me.
#156708 - yellowstar - Tue May 13, 2008 1:45 am
Well, all I did was use that function to get the FILE*, called ftell/fgetpos with the returned FILE*, and it crashed for some reason...
However, I now have another idea: It's a cross between the original method, and the sequential method. There's two passes: On the first one, we go through the whole capture, and in the data stage, we only store the size of all the data packets. Next, on the second pass, we use the sizes to calculate where we copy the data in the data buffer. After that, we write the data to the .nds.(And there's no on-the-fly, nothing changed in that aspect - assembly is still done at the end.)
Once I get the basic assembly done, a few more things need done. One would be assembling multiple transfers in the same capture. Instead of doing it sequentially, I'll do it simultaneously. The latter would be easier to code then the former. The former would be kindof messy, while the latter would be more organized. The multiple captures feature would be easy too - with the previous code just mentioned, I'd just need to make it go through one particular directory, and the program takes care of the rest.(However I still need to work on the dka dir.h wrapping work)
#156775 - yellowstar - Tue May 13, 2008 10:29 pm
I tried that method... And... Piratically the same result as before: 182 bytes off. I need to find what's causing trouble: the Arm9/7 binary, or something else.(It's probably the 7 since it's after the 9 binary)
#156809 - yellowstar - Wed May 14, 2008 1:32 am
I have made some progress. I wrote an program to aid with debugging. It goes through both the assembled .nds and the .nds in the archive, and reports in a txt file which bytes are not equal.(It spits out a ~32 MB file...)
Using this, I got the filesize difference between the two down to 134 bytes.(It used to be ~180)
And I got the position where the arm9 execution would begin, to be the same as the archive .nds.
And now we have new no$gba behavior!(Other emus behave the same)
no$gba error wrote: |
undefined opcode - with no debug vector defined
|
There's one position in the arm9 binary, near the beginning, that my compare tool reports the byte value is different from the archive .nds.(In fact, most of the bytes after that is different)
I don't know what's going on... I found that spot in the capture, and it's the same as my .nds, not the archive... I don't know what's going on...
There's 6 bytes at the end of each data packet, which aren't part of the actual data. What are these bytes? The first two seem to always be 02 00.
#156816 - yellowstar - Wed May 14, 2008 3:21 am
Yikes! From position 18834 and foreword, every byte is different from the archive .nds. The last one is at the end of the RSA-signature. What's going on here?!
#156884 - yellowstar - Thu May 15, 2008 3:27 am
I have found the problem. The archive .nds, and the capture, are not the same. In other words, the demo has changed since the archive .nds was first assembled. The RSA-signatures of the archive .nds, and the output .nds, are different, which is even more proof that they are different. Major problem.
Filb, could you capture the Meteos demo?(Hopefully it's exactly the same as the Download Station demo...)
And now I have run into another problem: All of the captures, except Namco and Custom Robo captures,(I didn't try those other original captures, due to no pre-assembled .nds)
have truncated packets. That is, it didn't capture the whole packet for some reason. And, my program grinds to a halt when it runs into these packets; It won't finish assembly.(Including Brain Age 2. :-( )
I tried hacking it to ignore errors, and still continue, but it can't finish the Beacon stage - the stage in which the advertisement, banner and such are assembled.(Yes, the bad ones include that huge multi-game capture zip)
I'll throw together a command-line tool to check for this errors, to make things easier when doing captures in the future. Here it is. Instructions: Download & extract. Copy the capture to the program's directory, then rename it to capture.cap. And for the last step, run the program. If it says "Clear", the capture is good. Otherwise, when it mentions truncated in particular, it's a bad capture, and you need to do the capture all over again.
#156904 - Filb - Thu May 15, 2008 9:28 pm
The Meteos single-cart-multiplayer-demo is not the same as the Download Station one.
#156917 - yellowstar - Fri May 16, 2008 12:37 am
Nuts... Well, I'll update that error checking program so it finds the RSA frame, and compares it with a pre-assembled .nds. In this way, it's would very easy it find the right capture & .nds.(That is, the demo with identical captures & pre-assembled .nds)
I googled for that no$gba error, and it seems that error can also happen when a game is running...
Well, I'm curious what the hardware does... So, could somebody run the program,
and try the output on an actual DS? All you have to do is, download & extract, copy the capture to the directory, renamed to capture.cap, and double-click/execute asm.bat, and it should output an .nds, with filename the game title.(Tell me of any errors, if they happen)
The captures I want tested, are, Namco particular, and Custom Robo, since those to are the only good captures.(Valid captures without those truncated packets)
I almost got the program to compile for DS - minus the libpcap-related errors.(Because the files for that is missing)
Last edited by yellowstar on Sun May 18, 2008 8:22 pm; edited 1 time in total
#156952 - yellowstar - Fri May 16, 2008 8:07 pm
I have written a DS program for capturing packets, being saved into the cap format. Instructions: Download, then DLDI patch. Copy to card. Start the WMB host/Wii for the WMB transfer. Don't start the receiving DS yet. Run the hb program. It will go through all the channels, trying to find the one with WMB traffic. If it doesn't find it, it will say "Failed to find any WMB traffic."
Tell me if this happens. If it finds it, all the text will disappear, and a scroll bar, and some other text appear, just like in the wifi_lib_test packet capture feature.(This program is based on wifi_lib_test)
Now, start the receiving DS, to download the WMB game/program. You should see the packets being shown on the hb program. It's supposed to write the packets to fat when it displays each one. Once the transfer finishes, power off the hb DS, then plug the card into the PC. Now, upload the capture and send me the link.(I'd test this myself if my homebrew card wasn't broken...)
EDIT:
Do the WMB transfer 3 times. After the first one, just shutdown the receiving DS, and start the transfer again for each one. After all that, shutdown the hb DS and follow the rest of the instructions.
EDIT2:
The timestamps aren't
done as they should. Right now the seconds field is set to zero at first, then constantly incrementing each time a packet is written. Very wrong, when you read the Wiki linked to in that link I just posted. So, if anybody tests this program before I get a response, and get it working correctly, it will need tested all over again...
#156959 - Filb - Fri May 16, 2008 9:37 pm
Quote: |
Failed to find any WMB traffic!
Stop! |
#156993 - yellowstar - Sat May 17, 2008 5:09 pm
Updated. I think I found the problem - Wifi_update wasn't being called in the channel search function, in which the program is mostly idle. This is probably needed to capture packets and other things. I fixed that, and I update the timestamp code as I said in that "Getting the time" topic. I also increased the time the program stays on one channel for 2 seconds, to make sure the correct channel is found.
The wmb_asm program's output needs tested on an actual DS... Maybe I should just post a link to some .nds outputted from it, on my server...
#156995 - Filb - Sat May 17, 2008 5:30 pm
Sorry, it still fails to find any WMB traffic.
#157000 - yellowstar - Sat May 17, 2008 6:48 pm
I forgot to mention: The capture program now writes the packets it found, while in the channel scanning phase, to fat. Could you upload that capture, and post the link?
#157001 - Filb - Sat May 17, 2008 6:59 pm
CAPTURE.CAP? It's 0 byte.
#157004 - yellowstar - Sat May 17, 2008 7:12 pm
Strange... Sounds like fflush is acting up... I have updated the download again. I replaced fflush calls with file close & open commands.
#157007 - Filb - Sat May 17, 2008 7:24 pm
Now it says "Failed to open capture.cap in the root of the card".
#157008 - yellowstar - Sat May 17, 2008 7:29 pm
Whoops... I accidentally forgot to add the NULL to the check-if-the-file-was-opened-successfully check. So it says that when everything was fine, and continues when it's not fine. Fixed & Updated.
#157009 - Filb - Sat May 17, 2008 7:39 pm
Hmm, it includes private traffic from my home connection.
I can't upload this for security reasons.
Can you give me a build that only captures from channel 1 and 13?
These are the only channels my Wii sends demos on. My home network is on channel 9.
(But yes, the capture file included data this time. ;))
#157011 - yellowstar - Sat May 17, 2008 7:54 pm
I understand.(I was wondering if you had a router...)
There's only 3 channels that the official WMB system uses, at least on DS.(Hopefully Wii doesn't use any more...)
I have updated the channel so it only scans these three.
Did it make it past the channel scanning? Also, what kind of homebrew card do you have?(Wondering because of the fflush issue...)
#157013 - Filb - Sat May 17, 2008 8:03 pm
I use GBA Movie Player v2 (CF). Didn't make it past the channel scan.
I'll send you the link to the capture file in a private message. I'm not sure if there might be private data in it. (116 KBytes seems a bit much for just beacons...)
#157017 - yellowstar - Sat May 17, 2008 8:24 pm
Uh ho... Wireshark reported that the the network type is unsupported. The value it display looks like an endian problem. Changing the magic number field should fix that. Updated.
#157020 - yellowstar - Sat May 17, 2008 8:42 pm
Yet another problem. Wireshark thinks some packets are too big. I didn't get AVS implemented. Cap files are supposed to have that. Once I fix that, it should work. And Filb, you got PM.
#157026 - Filb - Sat May 17, 2008 8:58 pm
OK, I replied.
Is there private data in it? :/
#157027 - yellowstar - Sat May 17, 2008 10:30 pm
Updated. Got AVS implemented. Also there was file writing code that shouldn't have been, and should be removed before. That was removed.
@Filb:
Well, since your router's SSID shows up in the capture... As long as nothing is connected to your router and isn't using the internet/networking/Wifi, you should be safe. It's strange when that shows up on a channel that it's not even on, while with Firefly's capture tool doesn't find those packets... I'll try to figure out a way,(this will be optional)
to filter out Wifi APs and communications to & from them. I have one idea right now, but I can't do any filtering until I get the channel scanning code to find the correct channel, to work.(And the capture program needs testing before that can happen)
#157028 - yellowstar - Sat May 17, 2008 10:47 pm
My bad. I forgot about the AVS_header's channel field! Previously that field would stay at zero. Now, it has been updated to contain the current channel.
#157029 - Maxxie - Sat May 17, 2008 10:47 pm
yellowstar wrote: |
It's strange when that shows up on a channel that it's not even on, |
The channels overlap. When listening on one channel you will allways hear the neighbour channels if there is traffic.
#157030 - yellowstar - Sat May 17, 2008 11:06 pm
I knew they overlapped, but I never knew before when capturing, that packets from other channels can be detected... I guess the higher-level hardware & software(on both DS & PC)
have filters to stop packets from coming in from different channels, while on the capture level, those are bypassed.
#157031 - Filb - Sat May 17, 2008 11:10 pm
Sent you a link with the new file.
#157032 - yellowstar - Sat May 17, 2008 11:36 pm
Wireshark still is reporting that same packet size error...
Does anybody know how to calculate the RSSI on DS?(Seems to be receive signal basically)
EDIT:
Ah ha! Endians! When I check the lengths of some packets, there are in the range of Wireshark's values. But when I flip endians, it looks fine! The cap header is little-endian, and is automatically swapped as needed. But, the packets' header, it must always be big-endian,(Or the endian of the CPU)
because it is not swapped.(Bug in libpcap maybe?)
Manually swapping should fix it.
#157033 - yellowstar - Sun May 18, 2008 12:19 am
Updated with the fix I found the previous post, in the EDIT section.
Since nobody tested wmb_asm's output on hardware, here's a zip of the Namco and Custom Robo demos assembled by the program. All you have to do is, download & extract, copy to card, then try running on the actual DS. Tell me what happens.(The robo demo might behave different than Namco, because that demo was assembled a good while ago with an old version of the program. But please still test it.)
Last edited by yellowstar on Sun May 18, 2008 6:12 pm; edited 1 time in total
#157037 - Filb - Sun May 18, 2008 12:33 am
Wireshark says:
The capture file appears to be damaged or corrupt, (pcap: File has 2157445120-byte packet, bigger than maximum of 65535)
Your link points to a 404 error page.
#157052 - yellowstar - Sun May 18, 2008 6:15 pm
Fixed that 404. The end of the URL was supposed to be wmb_asm_output.zip, but it was actually wmb_asm_out.zip.
Guess I should dig around the libpcap soruce code...
#157053 - Filb - Sun May 18, 2008 6:23 pm
Both of them don't work.
#157054 - yellowstar - Sun May 18, 2008 6:28 pm
Did you get white or black screens?(This won't help me find the problem, but...)
#157055 - Filb - Sun May 18, 2008 6:29 pm
Namco has a black screen, Custom Robo a white screen.
#157056 - yellowstar - Sun May 18, 2008 7:48 pm
Filb wrote: |
Since I have to go out in a few minutes, I'll just give you another game I already captured and got assembled. It's SUDOKU2 Deluxe demo, also from the Wii Nintendo Channel (Japanese).
SUDOKU2 Deluxe Demo Version (2.7 MB)
|
I forgot about this capture until now... However this capture... I check the RSA signature, and the signature in the capture, and the one in the .nds in the download, are EXACTLY the same! The download boots fine. Now for the amazing, and ridiculous part: the .nds assembled by my program, it actually WORKS! Obviously the weird thing, is that this one works, while everything else, not.
Hmmm... Maybe it's my data method? Or maybe there's something this one has the others doesn't?(Like, maybe no-smaller-than-normal-data-packets) Well, I'll dig through the log and see what I can find...
Odd... My file compare tool is reporting similar file differences between my .nds & the download, and yet mine works...
#157060 - yellowstar - Sun May 18, 2008 8:41 pm
I have updated the archive with the assembled .nds demos. Robo & Namco have been re-assembled, and the strangely-working Sudoku is included. Please test Robo & Sudoku.(Sudoku should work since it booted fine on emu, but still...)
I think I might have found the problem. As I guessed earlier, every data packet is exactly the same size. Now I need to fix whatever is causing the bad .nds, for the other demos.
#157063 - Filb - Sun May 18, 2008 9:31 pm
I think it's safe to say that if those three demos doesn't work on emulator, then they won't work on hardware either. But maybe someone else wants to test them for you.
#157074 - yellowstar - Sun May 18, 2008 11:10 pm
It seems that Sudoku demo wasn't assembled correctly... I accidentally loaded the archive .nds instead of my .nds, and I thought my program worked. In reality, the sudoku demo crashes the emulators in the same way as the others.
I found something strange. The first byte that is different from the archive, I found I just need to remove one byte from that position,(value zero)
and the rest is identical for a while. Then the same thing happens, and that might happen every time. What's causing this?!
About the "do nothing" opcode... Does that mean the processor just skips over that opcode, and continues to the next one? Or does it just "goes to sleep", and not do anything because of the opcode?
EDIT:
Filb wrote: |
I think it's safe to say that if those three demos doesn't work on emulator, then they won't work on hardware either. But maybe someone else wants to test them for you. |
This is horrible having a broken, useless homebrew card, and needing a new card, and I'm poor. )-: (No job and such)
#157076 - yellowstar - Mon May 19, 2008 12:24 am
DSCaptureTest has been updated, with the time updates I mentioned in that time topic. Filb, could you send me the capture?(For investigating these endians) Your router's broadcasts are ridden through the whole capture, at least in the ones you gave me. Even though the current channel is no-where near the router's channel. It seems the router shows up on channel 1, even though the router is near the end of the channels.
#157088 - tepples - Mon May 19, 2008 3:15 am
yellowstar wrote: |
About the "do nothing" opcode... Does that mean the processor just skips over that opcode, and continues to the next one? Or does it just "goes to sleep", and not do anything because of the opcode? |
"NOP" style opcodes either get skipped or perform a tautology such as 'orr r0, r0, r0'. "HLT" style opcodes go to sleep until the next interrupt.
Quote: |
This is horrible having a broken, useless homebrew card, and needing a new card, and I'm poor. |
You could try developing for the PC instead of developing for the DS.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#157155 - yellowstar - Mon May 19, 2008 9:09 pm
tepples wrote: |
Quote: | This is horrible having a broken, useless homebrew card, and needing a new card, and I'm poor. |
You could try developing for the PC instead of developing for the DS. |
Actually, PC is the platform I developed on first, before I found out about DS homebrew.(Seems like it all started when I found out that I could make my own DS games with the Games 'n Music for booting the hb. And I got tired of that thing, and it got replaced. And now my DS-X is busted...)
Once I run out of stuff to do related to DS hb, the only thing left is to go back to working on that PC game I was making.(Actually, it's for PC, DS, and more, but it's only compiled for PC, since I didn't get some things working yet.)
yellowstar in WMB_Host topic wrote: |
I thinking about rigging my wmb_asm program so if it detects that if the host is the Wii Nintendo Channel, it would throw in some extra data. When sending the demo with this WMB_Host mod, the program will try to find that extra data. If it finds it, instead of having "DS Download Station Demo" for the game name and discription, it would have something like "Wii Nintendo Channel Demo".
|
I have made progress with the wmb_asm program. At the end of each data packet, there's what seems to be the FCS. Which, according to NDSTech Wiki, is similar to a CRC32 over the entire packet.(That is, everything before the FCS)
Strangely, the WMB_Host by Juglak doesn't deal with FCS, and it works... So I can't check that FCS for errors unfortunately...(Well, maybe I could, but you couldn't assemble binaries from that host)
Maybe the hardware handles this FCS, even in RAW mode? Filb, could you use the WMB_Host,(My mod sends RSA, so use my mod if your DS isn't flashed)
send an official demo,(not the ones assembled by my program)
and capture the transfer?(Three times. Would be great to have another demo that has an identical capture, which is what we'll have if you capture this 3 times)
Sorry about all this testing Filb...
Anyway, about the data packets: Now before the FCS, there's 2 bytes that are in every data packet, which aren't part of the actual data. The hex value is 02 00. Now before these two bytes, I thought that's where the end of the actual data is. Well, it seems that byte is also rigged to always be zero too. So there is actually 3 bytes before the FCS, with hex 00 02 00. The actual end of the data is before these three bytes. Once I fixed this in my code, the output was even closer to the archive .nds, for the bytes comparison. That log file by my file compare tool, it used be ~60 MB, but now it's only ~6 MB! I also found and fixed other bugs. Previously, when the program reached the end of the transfer, it would just continue. Now that I add code so it makes sure it found everything before continuing, it works better.
And the RSA-signature still isn't correct...(The binary still fails the RSA-verification. If you were to send the binary over WMB, all you'd would get is a faded Nintendo logo.)
And the emu still behaves the same.
Now the position where the bytes in the .nds & the archive .nds begin to not be the same, is 53624. That's still in the Arm9 binary. This continues to the very end of the .nds... It seems there's 2 extra bytes at the end of the RSA-signature in the assembled .nds. Shouldn't be too hard to fix... Probably another case of copying too many bytes. Or in other words, thinking the length is longer than it really is.
#157159 - Filb - Mon May 19, 2008 9:33 pm
Can you send me WMB_Host?
#157162 - yellowstar - Mon May 19, 2008 9:40 pm
Here it is.(libfat beta. It should work, but if not, tell me)
Interesting. Take a look at this values Wireshark reports on the third & forth captures:(The too-big-packets errors)
Code: |
No endians flipped 2149318912
Endians flipped 3892314112
|
#157163 - Filb - Mon May 19, 2008 10:05 pm
Quote: |
* Initializing wifi...done.
* MAC: 00:19:FD:...
* Testing wifi...seems ok!
* Finding test.nds...found it!
* Loading into RAM...done.
* Initializing data...This binar
y is RSA-signed.
done.
* ARM9/7 Size: 0/0
* Starting beacons...done.
* Using channel 6
* Host ready! |
My other DS doesn't find anything with DS Download Play, though.
#157166 - jello - Mon May 19, 2008 10:55 pm
yellowstar wrote: |
Here it is.(libfat beta. It should work, but if not, tell me)
Interesting. Take a look at this values Wireshark reports on the third & forth captures:(The too-big-packets errors)
Code: |
No endians flipped 2149318912
Endians flipped 3892314112
|
|
I just tested WMB_Host_RSA_mod_dldi.nds with the built in demo since I didn't have a test.nds; here are my results:
DS Lite running WMB_Host_RSA_mod_dldi (R4DS)
Quote: |
* Initializing wifi...done.
* MAC: 00:...
* Testing wifi...seems ok!
* Finding test.nds...not found.
* Using built in demo. This binary is not RSA-signed; It's homebrew.
done.
* ARM9/7 Size: 468536/30360
* Starting beacons...done.
* Using channel 1
* Host ready!
* DS ACKED DATA 0231...
--- Press X to Reset ---
|
The other DS that downloaded the homebrew demo could only do so once; reported a communication error if I tried a second time. It did not launch anything after completing the download (not sure if this is desired behavior or could be something to do with the not being RSA signed).
Hope this helps.
Update:
The two DS lites were both in my lap during the test so they weren't very far apart. On a second test at 5 feet apart there was a communication error, the download never really started.
For a third test I found out I can only see the host at ~5 feet. Transfer fails.
I have other wifi devices on channel 11, but I don't think that would be a problem since the host is on channel 1.
Last edited by jello on Mon May 19, 2008 11:05 pm; edited 1 time in total
#157167 - jello - Mon May 19, 2008 10:57 pm
yellowstar wrote: |
I have updated the archive with the assembled .nds demos. Robo & Namco have been re-assembled, and the strangely-working Sudoku is included. Please test Robo & Sudoku.(Sudoku should work since it booted fine on emu, but still...)
I think I might have found the problem. As I guessed earlier, every data packet is exactly the same size. Now I need to fix whatever is causing the bad .nds, for the other demos. |
I tried these three demos on a emulator and on some hardware, all three failed.
#157182 - yellowstar - Tue May 20, 2008 1:49 am
@Filb:
Oops... According to NDSTech Wiki, the DS firmware WMB client doesn't use channel 6. And that's the channel that the mod used for you. My code was accidently coded to skip channel 7, not 6. That has been fixed. Here's the new link. The other one is obsolete now. This zip contains both source & binaries.
@jello:
That faded logo/not booting behaviour is normal with homebrew, which the host sends when it doesn't fine test.nds. For this host, you're meant to download demos and such off the internet,(there's a demo web site linked to in the DS Flash Equipment forum section)
copy it to the root of your card, and rename it to test.nds.
@all:
This errors and such... There's not a whole lot I can do about them; If you were to use the original host by Juglak, you would get about the same amount/percentage of errors. I mentioned things in the main topic about trying to mod the Wifi code to improve this, but it's really hard to do, so it's not getting done anytime soon.(Well, whenever this wmb_asm stuff gets done anyway)
Really, the only thing you can do is, keep trying to send the binary until it works. With the test results from tepples, the success rate was %50. So it SHOULD work within ten attempts. This host has issues. It has problems whenever it is not on channel 11. When it is,(absolutely nothing must be using that channel beforehand)
it works very well, at least from my tests.
EDIT:
I got the banner to match the archive .nds better. Now the last different byte is above the banner.(Banner data is visible below it in hex editor)
There's lots of data around here. BUT, this is in what should be the gap between the banner and the Arm7 binary. That shouldn't be there, and yet, it is there, and the 7 ended long ago. Only the archive .nds has this, not mine .nds. Why?(Why does the archive .nds have this?)
How did you assemble Sudoku, Filb? That sachen program wouldn't work for me, as with everything else.(And how you assembled those other ones, since sachen doesn't work with hardly any of these captures)
#157190 - yellowstar - Tue May 20, 2008 3:52 am
DSCaptureTest has been updated, with the fixes I mentioned in the time topic. I found that libpcap & Wireshark automaticly flip the endian of each packet header, if needed. So I removed that code for flipping endians. Maybe I should have the program write a log of all these problematic-values...
#157197 - Filb - Tue May 20, 2008 8:39 am
yellowstar wrote: |
How did you assemble Sudoku, Filb? That sachen program wouldn't work for me, as with everything else.(And how you assembled those other ones, since sachen doesn't work with hardly any of these captures) |
This was also a demo assembled by FireFly. By the way, I wonder where he is. Haven't seen him here for a while...
Have you read this document? You most likely have, but I think all you need to know is here...?
Sachen doesn't work with the Wii download captures. It does work with the Japanese in-store download captures, though. At least with most of them. Not sure why. Haven't tried to understand the code. ;)
#157246 - yellowstar - Wed May 21, 2008 1:19 am
Yes, I read that document. It's very nearly identical to the NDSTech Wiki document I reference. That document has a little more information than the Wiki, but it still doesn't help my problem. All these WMB documents really explain is the WMB protocol, they say next to nothing on how to assemble.(That is, how to write the data collected from captures to an .nds)
O_O You mean Sudoku was assembled by Firefly, and this capture exactly the same as since then, despite being assembled a few a years ago?! O_O
Based from what I read in that topic that sachen was released in, it seems it was made for DS Download Station capture assembly. But... I can't find anything so I can make sure - I heard that protocol is very similar WMB, so... That tool doesn't even handle RSA, making them useless for WMB. Unless they're sent with the binary, instead of a dedicated packet.
I tried having my assembler compare the data it's writing, and the data in the archive. That was screwed up. It was reporting the data was wrong, even though both bytes were really identical.
And that capture from DSCaptureTest is still screwed up...
#157252 - yellowstar - Wed May 21, 2008 3:24 am
Ah Ha! I have found the problem!!! Take a look at this!
Code: |
SZ9 490 temp 0
SZ9 490 temp 512
SZ9 490 temp 1024
SZ9 490 temp 1536
SZ9 490 temp 2048
SZ9 490 temp 2560
SZ9 490 temp 3072
SZ9 490 temp 3584
SZ9 490 temp 4096
SZ9 490 temp 4608
SZ9 490 temp 5120
SZ7 490 temp 1818112
SZ7 490 temp 1818624
|
What does this mean? For some reason, my program is only writing, finding and such, ~5KB of the Arm9 binary, and for Arm7, only 512 bytes! The reset of the data is filled in by the gap code. Why? Well, time to debug once again.(By the way, 490 is the size of the actual data in each packet, for most packets)
#157254 - Filb - Wed May 21, 2008 6:03 am
yellowstar wrote: |
O_O You mean Sudoku was assembled by Firefly, and this capture exactly the same as since then, despite being assembled a few a years ago?! O_O |
Not sure what you mean. He assembled it about half a year ago. My friend and I were mailing him some captures back then. But then he stopped responding to e-mails regarding captures. I guess it was boring work. ;)
#157286 - yellowstar - Wed May 21, 2008 7:23 pm
I meant it's strange that the the .nds transfered, captured in the capture, and the .nds in the download, are identical, despite being assembled a long time ago, and the capture seemingly being captured recently. Wait a second... Is sudoku the very same capture you sent to Firefly for assembly?
Anyway, back to status report:
I fixed that number-of-bytes-written bug. I have a constraint in the binary writing code, which was supposed to stop the writing on the binary when it reached the end of the binary's data. But, the variable I was using for this, was the wrong one, once again.(I goofed on this one alot recently...)
Here I was using the sz variable, instead of the correct the_sz variable. sz is the size of the gap, for that gap-filling code. the_sz is the size of the current data size. Once I fixed that, it started writing the correct amount.
But, it still won't work right. At one position, there's a section that all zeros, 490 bytes long.(Again, that's the size of the actual data in data packets)
And I goto the section spot after this that's wrong, and I find the exactly the same problem! This is probably the main problem left before it works, most likely.(Hopefully)
I added some CRC32 code. A CRC32 is calculated from the actual data from the data in the cap file, not in the program's buffer. When it's writing, it calculates another checksum for each packet, and compares it with the stored checksum. If those checksums aren't identical, the data has been corrupted somehow, and the program tells me when this happens. The first two packets it reported, it's the exact same positions that my file compare tool reported to be wrong.
Also, I added code to check the FCS. Didn't help my problem though.
#157287 - Filb - Wed May 21, 2008 7:30 pm
yellowstar wrote: |
I meant it's strange that the the .nds transfered, captured in the capture, and the .nds in the download, are identical, despite being assembled a long time ago, and the capture seemingly being captured recently. Wait a second... Is sudoku the very same capture you sent to Firefly for assembly? |
Yeah, it's the same capture my friend and I sent to him. I just uploaded it for you, because you needed some example captures with binaries to work with.
#157303 - yellowstar - Wed May 21, 2008 10:05 pm
I'm getting closer! Arm9 is perfect, 100% identical to the archive .nds now! I did it be changing my data method a little. I removed the data pass thing. Now, when it finds a packet, it stores the size, and copies in the data at the same time.
The emulators don't crash no more! Instead, they show white screens. The Arm7 binary is screwed up. Need to debug that. I think the screwed Arm7 & the white screens are related. I think the official SDK trys to have the Arm7 & 9 sync. This never happens because of the messed up 7 binary, so the Arm9 never starts the game/demo.(With the logos and whatnot)
#157327 - yellowstar - Thu May 22, 2008 12:54 am
Eariler something strange happened and the emus started to do what they used to do before, but nothing seemed to have really changed really...(Something might have changed for the Arm7 that I don't know about)
But, now no$gba went back to the white screens. However, after a few seconds, *that* error happens once again:
"Undefined opcode - with no debug vector defined"
It's no wonder it throws this error, with the Arm7 being so messed up.
I found the packet that corresponds with the first problematic Arm7 data. And that packet looks NOTHING like the data in my .nds, or even the archive .nds! I need to find where my program is getting this strange data.(And where the assembled .nds gets it's Arm7 data)
#157332 - yellowstar - Thu May 22, 2008 3:03 am
Something is very wrong! At the end of the Arm7 binary, there's a ton of zero data, upto the end of any non-null data. The worst part is, that null data is NOT in the capture!
Another thing: In the archive .nds, it take 2 seconds before the graphics appear. However, on my .nds, it takes about 8 seconds before the error. Probably because of the huge Arm7 corruption.
Once I get the Arm7 binary 100% identical to the archive .nds, I'm all done. Arm7 is the only thing left. Header is done. Banner is done. Arm9 done. Only Arm7 remains.
#157363 - yellowstar - Thu May 22, 2008 7:40 pm
Getting really close! The Arm7 binary isn't as screwed up as I thought! Actually, the main problem, is that it is missing some data from the beginning, that's less than 490 bytes long. However, the downloads' .nds have this data. I'm thinking of trying to inject the data from the download into my nds.
#157379 - yellowstar - Fri May 23, 2008 3:19 am
I tried injecting the Arm7 binary from the downloads' Arm7 binary. And it worked. Namco even worked with the Sudoku Arm7 binary. However, the Custom Robo demo didn't work. I think it's because this was captured from Wii Nintendo Channel, while that Custom Robo download was from a DS Download Station. But, I can't keep using injected binaries. My assembler has to work with the capture alone, not with the hacky aid of a pre-assembled .nds.
About that data I injected into the Arm7 binary: That helped some with the identical-thing, but the rest of the binary is still screwed up bad. I'm thinking that the Arm7 binary transferred over WMB is either encrypted or compressed. It's probably encrypted if it's either of the above. How do I determine if some data encrypted? And if it's compressed?
And if the data really is encrypted/compressed,(I really do think it's one of those)
how do I decrypt/decompress the Arm7? And after that, how do I know if the data is encrypted/compressed or not? Juglak's WMB Host doesn't do this, and it transfers fine. So there must be some kind of command some where that enables the encryption.(At least the decryption code in the WMB client anyway)
Filb, could you use the DS WMB_Host mod, and Firefly's WMB Host for PC,(Anybody can do this)
to send one of those assembled .nds, of these Nintendo Channel demos, assembled by Firefly, and capture them? I'm curious if Firefly's WMB Host handles this encryption-thing.(If Firefly's host handles the crypto, I can also use both of the captures to figure out what command is used for enabling the encryption)
#157392 - Filb - Fri May 23, 2008 1:49 pm
Hmm, I can't send and capture at the same time with the same wireless card...
Also, it's the first time I hear about crypto stuff within the transfer (other than RSA)... are you sure about that?
#157399 - yellowstar - Fri May 23, 2008 4:53 pm
I figured that out, about doing WMB with the wireless card, and capturing the transfer, today.(Before I read your post)
I wasn't thinking about that when posted that previous post... The only way we could get this capture, is when I get the DSCaptureTest program working...
The thing with the crypto is only a guess. I haven't seen any documents, absolutely nothing on the net, mentioning that anything is done different for the Arm7 binary, that's not done for the Arm9 binary.(Such as crypto)
That capture of the DS WMB Host alone can help prove that *something* is done to the Arm7 with the official hosts. Because, if the assembled version of that capture, and the .nds used in the transfer, are EXACTLY the same, that alone should prove that *something* is done to the Arm7, that is not mandatory. Homebrew can get away with not using it, and yet, it seems all official hosts,(? Note sure. At least the Nintendo Channel seems to use this crypto)
use the crypto.
I'm going to try to contact davr about this Arm7 stuff. But, does anybody know how I can contact him? I tried to find a method of contacting him on his blog/web site, but I couldn't find anything. And on his old web site for DS stuff, there's a link to a topic here on gbadev. But, instead of the davr username, the username on that topic is josath. I guess I'll try contacting josath. At first, I'll just ask if he really is davr.
#157400 - Filb - Fri May 23, 2008 4:56 pm
I asked my friend and he says ARM7 is nothing to be handled in any different way. Apparently, the whole binary is being transfered in just that one data stream (except RSA and stuff from the beacons).
You do may have to edit the header (and fix its CRC) to get icons and descriptions right, though... but this is not important at all. It's just cosmetic.
#157402 - yellowstar - Fri May 23, 2008 5:05 pm
Well, I'm stumped. My Arm7 is VERY different from the download's Arm7, even though both were assembled from the exactly same capture. And almost all of the data packets pass the FCS check. Stranglely, it seems the official hosts don't send an FCS for the first packet of the Arm & 9 binaries. But, it can't be the FCS fault - the whole Arm7 binary is screwed, not just the first part.
Everything but the Arm7 is exactly the same as the download .nds. And as I said before, the header, banner, Arm9 are perfect, while the Arm7 is screwed up bad.
#157407 - yellowstar - Fri May 23, 2008 6:31 pm
Here's the source code to wmb_asm. This ONLY contains the source code, no binary.
#157409 - yellowstar - Fri May 23, 2008 8:29 pm
EURKA! I have found it! My program almost works now! Turns out I was storing the sequence numbers for the start and end of the Arm7 binary wrong. I was storing them as 1-based numbers, but they were supposed to be 0-based. Now that I fix that, it works with Namco now! However, with sudoku, it doesn't work. Seq numbers fault again. But, this shouldn't be much trouble. I'm going attempt to use the code in Juglak's WMB Host for the Arm7 binary beginning.
#157416 - yellowstar - Fri May 23, 2008 10:49 pm
Juglak's code fixed it! Only after I tweaked it slightly however. It worked with all of the captures!(Excluding the corrupted Brain Age 2 and the multi-game captures)
My code needs cleaned up before I release it though.
#157425 - yellowstar - Sat May 24, 2008 1:59 am
DSCaptureTest has been updated. Thanks to me and Filb chatting over MSN, we've made a lot of progress. The program finds the WMB channel now. The packet sizes bug has been fixed. Also AVS bugs have been fixed. Now, for some reason, the program keeps writing the exact same packet, excluding the AVS header. Now the program only writes captured packets to fat, after it finds the WMB channel. You can now shutdown the DS with the Select button.(Now no data corruption can now happen, when using this poweroff method)
#157432 - yellowstar - Sat May 24, 2008 4:55 am
I have gotten the base directory code implemented.(I added the includes/c files for unix directories, Win32, and I wrote the NDS dir.h wrapper. Testing the wrapper will have to wait for a later date.)
#157718 - yellowstar - Wed May 28, 2008 2:31 am
Wmb_asm has been released. Source and binary is included. Read the ReadMe file for usage. Programmer's Notepad project for NDS, and NDS Makefile not included, because that issue with libpcap. But, it's easy to add these by copying the template project files into the program's directory. Note that if you compiled the PC version, and are attempting to compile NDS, you must use make clean first. Also, there's a Technical document included about technical things about how my program works, and some things about WMB.
I have written code for assembling multiple captures. However, there's this strange bug.(This functionality is disabled in the above release)
I have Namco assembled first, which is done successfully. Now, Custom Robo is next. However, it fails to find the header. And strangely, when I use Namco for the second capture, the exact same error happens.
Filb gave me a DS Download Station capture over MSN. This captures seems to have the WMB transfers on the station client included. However, my program reports that it didn't find all of the WMB beacons...
#157720 - yellowstar - Wed May 28, 2008 3:00 am
I have fixed that strange bug with multiple-capture assembly. Here the 802_11 sequence number checking code was part of it. Once I reset the variable that code uses when it starts reading each capture, the program assembled all of the captures,(Excluding the DS DL Station client)
correctly. I had no$gba run them, and they all worked. No, I'm not releasing this new version just yet. The new version will most likely be released once I implement the directory scanning code.
#157725 - Filb - Wed May 28, 2008 8:01 am
American DS Download Stations don't send any WMB beacons for the games offered.
First, the client has to download a "DS Download Station client" over DS Download Play (this is the only WMB beacon you'll find). The client then connects to the "DS Download Station server" and gets a list of games, including their titles, and icons.
While waiting for the client to choose a game to download, ~900 packets are transfered every second. I don't know why. Maybe some kind of crazy keep alive stuff. (The client shows a connection quality icon...)
When the client chooses a game to download, it communicates with the server which then sends the game LZO compressed. After the transfer has finished, the binary will be uncompressed and executed if the RSA matches.
That's how it works, I think.
Screenshots:
Downloading the client software
Client software is connecting to the server
Got the list from the server
Downloading a demo
#157778 - yellowstar - Thu May 29, 2008 3:19 am
I got the DS Download Station client to assemble correctly now. That beacon problem was a bug in my app. While the rest of the bugs, well, Nintendo really rigged the transfer to get WMB assemblers to fail. Lets see... After beacons, RSA failed. Here the RSA frame wasn't the normal size. And that header... That was nasty. Here the size was 250 bytes long, while it's normally 352 bytes long. Nintendo chopped off 102 bytes off of the header. I had to grab the missing data from pre-assembled .nds Filb gave me. And yet, still not correct. Here the bannerOffset and romSize were wrong. The bannerOffset was zero. And romSize was screwed. These wrong values were crashing my program. I added code if these vars are wrong, they are set to the correct values. And Nintendo even rigged the data transfer. The very first data packet is a really small 102-byte data packet. It's not a real data packet - the other assembled ones don't have this data. And Nintendo even set the data sizes for most of the packets to 250. My code assumed it was always 490 bytes. Once I fixed all of these, it worked. But when I tried checking the RSA-signature, the program crashed...
Filb told me that the icons in my assembled demos are glitched. Parts of the icon are transparent. The CRC16 of the banner in my assembled Download Station client, and the pre-assembled one are different. So the icon in this one is probably glitched too. Filb gave me his icon extractor earlier today. With that, I can view the icon in an .nds on my computer, without a homebrew card. And, apparently this one is glitched too.
#157781 - HyperHacker - Thu May 29, 2008 4:51 am
It's likely not Nintendo doing that; it's data loss and stray packets during the capture. Packet sniffing isn't all that reliable.
There's a firmware dump in .nds format somewhere. Has anyone considered hacking that and/or the Download Station binary to save the downloaded program to a file (or even slot-2 RAM where it can be extracted later if you're quick on the power button) before/instead of running it?
_________________
I'm a PSP hacker now, but I still <3 DS.
#157784 - yellowstar - Thu May 29, 2008 4:55 am
I got the RSA problem fixed. The pre-assembled binary doesn't have the following number, while mine did. I rigged my program so it won't write the number when assembling the client. No clue why, but the RSA verification program started working once I fixed that.
In assembled demos, for some reason there's this number in the gap between the Arm7 and the banner:
08 00 00 00 00 00 01
#157785 - yellowstar - Thu May 29, 2008 5:25 am
@HyperHacker:
The header and the data packets have an FCS.(CRC32)
How can they possibly be corrupted, without the FCS being invalidated? Note that these captures were done with Firefly's capture tool.(The first data packet for Arm9 doesn't have an FCS, that's the one that is probably "fake".)
That idea of your's sounds interesting. But I can't try it. Hacking with assembly, RE, and whatnot is way out of my field.(Don't know hardly anything about it/no experience with it)
#157810 - yellowstar - Thu May 29, 2008 7:45 pm
I discovered something. In the second header in these assembled .nds binaries, in the reserved section, there's 16 bytes with value 0x2D.( - )
And, after those dashes, I can see the hostname! The name of the WMB host. After that there's 16 more dashes. With these Nintendo Channel binaries, I can see the hex for NintendoCh in Japanese. And with the DS Download station client, I can see Nintendo in that spot. Filb told me Firefly's WMB Host uses this hostname in the file for the hostname in WMB. If it's not present, it uses Firefly for the hostname.
#157811 - Filb - Thu May 29, 2008 7:56 pm
yellowstar wrote: |
(...) Firefly's WMB Host uses this hostname in the file for the hostname in WMB. If it's not present, it uses Firefly for the hostname. |
Filb wrote: |
At 0x390 you can see the hex for "ニンテンドーCh" (Nintendo Channel), which is the original sender of it. Normally this information is not stored in the binary. But WMB uses it to display this sender when resending it. |
Already posted this on page 2, too. :D
#157812 - yellowstar - Thu May 29, 2008 8:24 pm
I guess I didn't pay much attention to that, and forgot about it...
Wmb_asm v1.2 has been released.
The icon bug has been fixed. Other bugs with the banner has been fixed. The DS Download Station client can be assembled with this version. WMB transfers from Juglak's WMB Host should be assembled correctly now. Directory support hasn't been added, but the multi-capture assembly feature is in this version.
#157826 - yellowstar - Fri May 30, 2008 3:07 am
DSCaptureTest has been updated.
This should fix this bug with identical packets. Also the Select button should work properly now. It should now shutdown the DS when you press Select. This method of shutdown is recommended for this tool. No risk of capture file corruption. Also, I have found a major bug with the time in the packets. See that time topic for that.
EDIT:
Link fixed.
Last edited by yellowstar on Sat May 31, 2008 6:38 pm; edited 1 time in total
#157829 - yellowstar - Fri May 30, 2008 3:44 am
Wmb_asm v1.2b has been released.
Bugs with the DS DOWNLOAD PLAY string has been fixed. Also bugs with CRC checksums in the header has been fixed. Strangely, I think there might be a glitch in the icons again... This only shows up in one .nds, out of 8 assembled binaries. There's this short transparent/white horizontal line in the icon.
EDIT:
Filb gave me a installer for a program called NDS Header Shell Extension. It replaces the icon for .nds files displayed in Windows Explorer. The icon used is the one in banner. It also displays other useful information, such as the description in the banner. Unfortunately, I don't know what the URL is on the original server. If I find out what it is, I'll post the link. But there's other extensions if you google for this: NDS Header Shell Extension. This extension Filb gave me seems to be based on the NDS Header program. The one that used to not work. Both the NDS Header application & extension work with my assembled binaries.
#157854 - yellowstar - Fri May 30, 2008 7:38 pm
Wmb_asm 1.2c has been released. I have written my own code to handle cap files. It only supports link layer 163.(IEEE 802.11 with AVS, I think)
Both Firefly's capture tool, and my DSCaptureTest tool use this link layer. My code has the same interface as libpcap.(My code only has the cap file handling code)
So I can easily switch between my code, and libpcap, just by commenting and uncommenting two lines in defs.h. It seems the speed for libpcap and my code is the same.(It seems to take the same amount of time assemble one capture with both libpcap & my code.)
Here's why I wrote this cap handling code in the first place:
That English Brain Age 2 capture didn't work with my program because libpcap kept refusing to continue once it ran into a truncated packet. My code simply skips such packets. And, the demo is assembled successfully!
Filb reminded me about that multi-game assembly feature...(Assembling multiple WMB transfers in the same capture)
The main code for this shouldn't be too hard to implement. But, the main thing is: I need a way to tell when the program runs into the next transfer for another game. If that wouldn't be in place, well, crashes with the assembled binary are sure to happen.(Because the program gets the data for a transfer, from the wrong transfer.)
So, how do I do this? I was thinking about using beacons. Note that if the beacon isn't from host currently being used, it would be ignored. I'm thinking of using the random stream code, and the banner CRC. Even with these precautions, it could still fail to recognize when an different transfer of a demo has begun. Stream code could, even though it's random, land on the same stream code as the stream code for the current transfer's beacons. And that CRC can still be the same. How can both of these be true? Homebrew. Say the homebrew wouldn't have an icon.(hb with with an banner isn't a problem.)
The icon in WMB would always be the same, if the host is coded correctly. So the banner CRC would always be the same. If a different transfer would show up, identical banner CRC, and identical stream code, BOOM. Any ideas?
EDIT:
Here's proof that this can happen.(Getting the data from the wrong transfer)
I tried having my tool assemble all of these multi-game captures. It can only assemble the very first transfer in the capture at the moment. Out of 4 captures, only one worked. All of them flunked the RSA-check. The first caused my program to crash.(Don't know why, if that continues after I get multi-game support added, I'll be debugging again.)
The second crashed the emulator. The third worked, but still failed RSA. The forth crashed the emulator.
#157872 - Filb - Sat May 31, 2008 11:47 am
I worked on analyzing packets of the American DS Download Station Vol. 7.
When the client-software finished downloading (over regular DS Download Play; I explained that part earlier), it will be requesting a list of games.
The client requests the menu with a "Requesting Packet", which includes the ID of a binary stored on the server:
→ [Images not permitted - Click here to view it]
For menu requests, 0x5A~0x61 holds the ID (eg. "menu_ger", can also be "menu_eng", "menu_jpn", etc.). (Screenshot doesn't show a menu-request, but it'd look the same.)
The server will then be sending "Menu Packets":
→ [Images not permitted - Click here to view it]
These packets include a small LZO compressed binary which includes all the menu items with their titles, icons, descriptions, and the internal binary-names of the offered downloads (eg. "A8NEdemo" etc.).
I uploaded a menu binary here. An uncompressed one can be downloaded from here.
The menu file is pretty simple. Ignore the first 4 bytes. One menu item has a length of 0x72C bytes. First, there are 0x200 bytes of icon-data followed by a 0x20 bytes palette. The next 0x80 bytes contain the title. Another 0x80 bytes hold the subtitle. The following 0x402 bytes can be ignored. The last 8 bytes hold the internal binary-name (ID) (eg. "A8NEdemo"). The menu item ends with two binary zero bytes.
After the menu has been received, there will be lots of keep-alive packets, which can all be ignored.
When the client chooses a demo, it'll send another "Requesting Packet", which looks exactly like this:
→ [Images not permitted - Click here to view it]
It works just like a menu-request.
After countless weird handshakes and stuff, the server will begin sending the demo (LZO compressed). It'll use "Download Binary Packets":
→ [Images not permitted - Click here to view it]
Since all of this data is sent to a broadcast address, there's a Client ID at offset 0x0E. I think the client will get this ID when first getting a list from the server within the countless weird packets I haven't been able to figure out. By the way, up to 15 clients can be connected to the download station at the same time and this is what the server looks like when there are two clients connected (#1 is downloading something (6% done), #2 is idle).
However, I'm not sure about the Client ID at all, though.
Requesting Packets don't have any Client ID in them, so the server might check the Source MAC address instead. I'm not sure.
When building a capture-assembler, it would most likely be necessary to somehow check which MAC address has which Client ID. I don't know how to do this yet.
Another way would be just ignoring the menu stuff and not modify the banner and header of the assembled binary (some of them might have a "?" icon then, though).
Of course it would also be possible to just let the user of the assembler decide which menu item to use instead of doing it automatically...
Edit: Here's a capture file of the whole process:
→ capture_3x-Brain-Age-2.rar
You will find this inside:
Downloading the client-software → Getting a list of demos from the server → Downloading "Brain Age 2 Demo". And all this three times in case some packets were not captured.
Would be cool if someone could find out which menu item belongs to which download.
#157886 - yellowstar - Sat May 31, 2008 6:21 pm
I don't know hardly anything about cracking protocols, such as the DS Download Station protocol. But I think I found the point where the client & host first start the station communications, but that's just about all I could find. Looks like you covered a lot of ground. Unfortunately, I can't try to use that information to implement that protocol at the moment. I'm trying to implement that multi-game feature. And the assembly process is more abstract now. Making it easy to add the the assembly part to DSCaptureTest when it's ready for that. Anyway, at the moment my code is screwed with that bug I mentioned before. Now it's worse. Now the program can't even assemble one single normal capture. Don't worry - all I should have to do is implement that guard I mentioned earlier, and these bugs should be fixed, and the multi-game feature should work.
#157897 - Filb - Sun Jun 01, 2008 1:31 pm
I think I found out how the clients get their ID.
In Association Response packets, there is an interesting number at offset 0x5C:
Client #1: 01
Client #2: 02
Client #3: 03
Client #4: 04
They also match the numbers displayed on the server's top screen.
Obviously, the client seems to store that number, since it's some kind of a Client ID (or Session ID).
Data packets (DL Binary Packets) are sent to a broadcast MAC address. The Client ID is needed for the clients to check which packets must be ignored and which not.
I already found the offset of a Client ID, but it's a bit different:
Client #1: 0x02 (00010b)
Client #2: 0x04 (00100b)
Client #3: 0x08 (01000b)
Client #4: 0x10 (10000b)
Does anyone have an idea why it could be like this? Up to 15 clients can work with the Download Station at the same time...
Edit: Since it's possible that more than 1 client is downloading from the same data-stream, this makes perfectly sense. I think that's all we need for assembling them from captures. ;)
Edit 2: There's no need of handling RSA separately. It's part of the regular compressed data-stream.
#157918 - yellowstar - Sun Jun 01, 2008 11:20 pm
I found the problem with that time bug in DSCaptureTest. Go see the getting the time topic for that.
Also, DSCaptureTest is working better now. Now it actually captures packets, but mainly only beacons for some reason.
#157926 - yellowstar - Mon Jun 02, 2008 1:37 am
I'm having problems getting wmb_asm to compile for the DS.
First, there's the CRC32 checksum problem:(Code taken from sachen program)
Code: |
c:/YellowstarConsoleStuff/wmb_asm/wmb_asm/source/utils.cpp:398: warning: operation on 'oldcrc32' may be undefined
c:/YellowstarConsoleStuff/wmb_asm/wmb_asm/source/utils.cpp:398: warning: operation on 'oldcrc32' may be undefined
|
Code: |
define UPDC32(ch, crc) (crc = (crc >> 8) ^ crc_32_tab[(crc ^ (ch)) & 0xff])
u32 crc32buf(char *buf, size_t len);
static u32 crc_32_tab[] = { /* CRC polynomial 0xedb88320 */
0x00000000, 0x77073096, 0xee0e612c, 0x990951ba, 0x076dc419, 0x706af48f, 0xe963a535, 0x9e6495a3, 0x0edb8832, 0x79dcb8a4, 0xe0d5e91e, 0x97d2d988, 0x09b64c2b, 0x7eb17cbd, 0xe7b82d07, 0x90bf1d91, 0x1db71064, 0x6ab020f2, 0xf3b97148, 0x84be41de, 0x1adad47d, 0x6ddde4eb, 0xf4d4b551, 0x83d385c7, 0x136c9856, 0x646ba8c0, 0xfd62f97a, 0x8a65c9ec, 0x14015c4f, 0x63066cd9, 0xfa0f3d63, 0x8d080df5, 0x3b6e20c8, 0x4c69105e, 0xd56041e4, 0xa2677172, 0x3c03e4d1, 0x4b04d447, 0xd20d85fd, 0xa50ab56b, 0x35b5a8fa, 0x42b2986c, 0xdbbbc9d6, 0xacbcf940, 0x32d86ce3, 0x45df5c75, 0xdcd60dcf, 0xabd13d59, 0x26d930ac, 0x51de003a, 0xc8d75180, 0xbfd06116, 0x21b4f4b5, 0x56b3c423, 0xcfba9599, 0xb8bda50f, 0x2802b89e, 0x5f058808, 0xc60cd9b2, 0xb10be924, 0x2f6f7c87, 0x58684c11, 0xc1611dab, 0xb6662d3d, 0x76dc4190, 0x01db7106, 0x98d220bc, 0xefd5102a, 0x71b18589, 0x06b6b51f, 0x9fbfe4a5, 0xe8b8d433, 0x7807c9a2, 0x0f00f934, 0x9609a88e, 0xe10e9818, 0x7f6a0dbb, 0x086d3d2d, 0x91646c97, 0xe6635c01, 0x6b6b51f4, 0x1c6c6162, 0x856530d8, 0xf262004e, 0x6c0695ed, 0x1b01a57b, 0x8208f4c1, 0xf50fc457, 0x65b0d9c6, 0x12b7e950, 0x8bbeb8ea, 0xfcb9887c, 0x62dd1ddf, 0x15da2d49, 0x8cd37cf3, 0xfbd44c65, 0x4db26158, 0x3ab551ce, 0xa3bc0074, 0xd4bb30e2, 0x4adfa541, 0x3dd895d7, 0xa4d1c46d, 0xd3d6f4fb, 0x4369e96a, 0x346ed9fc, 0xad678846, 0xda60b8d0, 0x44042d73, 0x33031de5, 0xaa0a4c5f, 0xdd0d7cc9, 0x5005713c, 0x270241aa, 0xbe0b1010, 0xc90c2086, 0x5768b525, 0x206f85b3, 0xb966d409, 0xce61e49f, 0x5edef90e, 0x29d9c998, 0xb0d09822, 0xc7d7a8b4, 0x59b33d17, 0x2eb40d81, 0xb7bd5c3b, 0xc0ba6cad, 0xedb88320, 0x9abfb3b6, 0x03b6e20c, 0x74b1d29a, 0xead54739, 0x9dd277af, 0x04db2615, 0x73dc1683, 0xe3630b12, 0x94643b84, 0x0d6d6a3e, 0x7a6a5aa8, 0xe40ecf0b, 0x9309ff9d, 0x0a00ae27, 0x7d079eb1, 0xf00f9344, 0x8708a3d2, 0x1e01f268, 0x6906c2fe, 0xf762575d, 0x806567cb, 0x196c3671, 0x6e6b06e7, 0xfed41b76, 0x89d32be0, 0x10da7a5a, 0x67dd4acc, 0xf9b9df6f, 0x8ebeeff9, 0x17b7be43, 0x60b08ed5, 0xd6d6a3e8, 0xa1d1937e, 0x38d8c2c4, 0x4fdff252, 0xd1bb67f1, 0xa6bc5767, 0x3fb506dd, 0x48b2364b, 0xd80d2bda, 0xaf0a1b4c, 0x36034af6, 0x41047a60, 0xdf60efc3, 0xa867df55, 0x316e8eef, 0x4669be79, 0xcb61b38c, 0xbc66831a, 0x256fd2a0, 0x5268e236, 0xcc0c7795, 0xbb0b4703, 0x220216b9, 0x5505262f, 0xc5ba3bbe, 0xb2bd0b28, 0x2bb45a92, 0x5cb36a04, 0xc2d7ffa7, 0xb5d0cf31, 0x2cd99e8b, 0x5bdeae1d, 0x9b64c2b0, 0xec63f226, 0x756aa39c, 0x026d930a, 0x9c0906a9, 0xeb0e363f, 0x72076785, 0x05005713, 0x95bf4a82, 0xe2b87a14, 0x7bb12bae, 0x0cb61b38, 0x92d28e9b, 0xe5d5be0d, 0x7cdcefb7, 0x0bdbdf21, 0x86d3d2d4, 0xf1d4e242, 0x68ddb3f8, 0x1fda836e, 0x81be16cd, 0xf6b9265b, 0x6fb077e1, 0x18b74777, 0x88085ae6, 0xff0f6a70, 0x66063bca, 0x11010b5c, 0x8f659eff, 0xf862ae69, 0x616bffd3, 0x166ccf45, 0xa00ae278, 0xd70dd2ee, 0x4e048354, 0x3903b3c2, 0xa7672661, 0xd06016f7, 0x4969474d, 0x3e6e77db, 0xaed16a4a, 0xd9d65adc, 0x40df0b66, 0x37d83bf0, 0xa9bcae53, 0xdebb9ec5, 0x47b2cf7f, 0x30b5ffe9, 0xbdbdf21c, 0xcabac28a, 0x53b39330, 0x24b4a3a6, 0xbad03605, 0xcdd70693, 0x54de5729, 0x23d967bf, 0xb3667a2e, 0xc4614ab8, 0x5d681b02, 0x2a6f2b94, 0xb40bbe37, 0xc30c8ea1, 0x5a05df1b, 0x2d02ef8d
};
u32 updateCRC32(unsigned char ch, u32 crc)
{
return UPDC32(ch,crc);
}
u32 crc32buf(char *buf, size_t len)
{
register u32 oldcrc32;
oldcrc32 = 0xFFFFFFFF;
for ( ; len; --len, ++buf)
{
oldcrc32 = UPDC32(*buf, oldcrc32);
}
return oldcrc32 ^ 0xffffffffUL;
}
unsigned int CalcCRC32(unsigned char *data, unsigned int length)
{
return crc32buf((char*)data, length);
}
|
Then there's more errors:
Code: |
include/pcap.h:34: error: expected specifier-qualifier-list before 'bool'
|
Code: |
struct pcap_t
{
FILE *file;
bool swap;
struct pcap_pkthdr header;
unsigned char *pktdata;
};
//I tried changing it to the below, but still nothing
typedef struct spcap_t
{
FILE *file;
bool swap;
pcap_pkthdr header;
unsigned char *pktdata;
} __attribute__ ((__packed__)) pcap_t;
|
#157929 - yellowstar - Mon Jun 02, 2008 2:53 am
DSCaptureTest has been updated. Download is now an archive, containing both the binary and source.
Could somebody test this? I'll need the capture.cap file. Filb won't be able to test for a while.
#157979 - yellowstar - Mon Jun 02, 2008 7:22 pm
I think the solution to the time bug in DSCaptureTest has been found.(See that getting the time topic)
But, I'll need to update to devkitARM 23, and latest libnds, over slow internet...
I think I found the way to do this block on wmb_asm. First, the banner would be checked. If the previous game's banner CRC is identical to the current beacon's banner CRC, the block is triggered, and assembly of this game is aborted. When the header is found, it is compared with the previous game's header. If they're identical, abort. Lastly, once the RSA-frame is found, it's compared with the previous game's RSA-frame. If they're identical, abort.
#157983 - yellowstar - Mon Jun 02, 2008 8:23 pm
It seems the advert in the beacons don't have an banner CRC... I can either assemble the advert and then compare it, or don't check the advert at all...
EDIT:
I think I'll go with skipping the beacon check.
#157993 - yellowstar - Mon Jun 02, 2008 10:23 pm
I have gotten wmb_asm to compile for DS. Once I renamed all of the source code files that were c code, to Cpp, most of the errors disappeared. Now there's only one thing remaining: those CRC32 warnings. How do I calculate a CRC32, on DS, without undefined behavior, as my warnings say?
Code: |
c:/YellowstarConsoleStuff/wmb_asm/wmb_asm/source/utils.cpp:398: warning: operation on 'oldcrc32' may be undefined
c:/YellowstarConsoleStuff/wmb_asm/wmb_asm/source/utils.cpp:398: warning: operation on 'oldcrc32' may be undefined
|
Code: |
define UPDC32(ch, crc) (crc = (crc >> 8) ^ crc_32_tab[(crc ^ (ch)) & 0xff])
u32 crc32buf(char *buf, size_t len);
static u32 crc_32_tab[] = { /* CRC polynomial 0xedb88320 */
0x00000000, 0x77073096, 0xee0e612c, 0x990951ba, 0x076dc419, 0x706af48f, 0xe963a535, 0x9e6495a3, 0x0edb8832, 0x79dcb8a4, 0xe0d5e91e, 0x97d2d988, 0x09b64c2b, 0x7eb17cbd, 0xe7b82d07, 0x90bf1d91, 0x1db71064, 0x6ab020f2, 0xf3b97148, 0x84be41de, 0x1adad47d, 0x6ddde4eb, 0xf4d4b551, 0x83d385c7, 0x136c9856, 0x646ba8c0, 0xfd62f97a, 0x8a65c9ec, 0x14015c4f, 0x63066cd9, 0xfa0f3d63, 0x8d080df5, 0x3b6e20c8, 0x4c69105e, 0xd56041e4, 0xa2677172, 0x3c03e4d1, 0x4b04d447, 0xd20d85fd, 0xa50ab56b, 0x35b5a8fa, 0x42b2986c, 0xdbbbc9d6, 0xacbcf940, 0x32d86ce3, 0x45df5c75, 0xdcd60dcf, 0xabd13d59, 0x26d930ac, 0x51de003a, 0xc8d75180, 0xbfd06116, 0x21b4f4b5, 0x56b3c423, 0xcfba9599, 0xb8bda50f, 0x2802b89e, 0x5f058808, 0xc60cd9b2, 0xb10be924, 0x2f6f7c87, 0x58684c11, 0xc1611dab, 0xb6662d3d, 0x76dc4190, 0x01db7106, 0x98d220bc, 0xefd5102a, 0x71b18589, 0x06b6b51f, 0x9fbfe4a5, 0xe8b8d433, 0x7807c9a2, 0x0f00f934, 0x9609a88e, 0xe10e9818, 0x7f6a0dbb, 0x086d3d2d, 0x91646c97, 0xe6635c01, 0x6b6b51f4, 0x1c6c6162, 0x856530d8, 0xf262004e, 0x6c0695ed, 0x1b01a57b, 0x8208f4c1, 0xf50fc457, 0x65b0d9c6, 0x12b7e950, 0x8bbeb8ea, 0xfcb9887c, 0x62dd1ddf, 0x15da2d49, 0x8cd37cf3, 0xfbd44c65, 0x4db26158, 0x3ab551ce, 0xa3bc0074, 0xd4bb30e2, 0x4adfa541, 0x3dd895d7, 0xa4d1c46d, 0xd3d6f4fb, 0x4369e96a, 0x346ed9fc, 0xad678846, 0xda60b8d0, 0x44042d73, 0x33031de5, 0xaa0a4c5f, 0xdd0d7cc9, 0x5005713c, 0x270241aa, 0xbe0b1010, 0xc90c2086, 0x5768b525, 0x206f85b3, 0xb966d409, 0xce61e49f, 0x5edef90e, 0x29d9c998, 0xb0d09822, 0xc7d7a8b4, 0x59b33d17, 0x2eb40d81, 0xb7bd5c3b, 0xc0ba6cad, 0xedb88320, 0x9abfb3b6, 0x03b6e20c, 0x74b1d29a, 0xead54739, 0x9dd277af, 0x04db2615, 0x73dc1683, 0xe3630b12, 0x94643b84, 0x0d6d6a3e, 0x7a6a5aa8, 0xe40ecf0b, 0x9309ff9d, 0x0a00ae27, 0x7d079eb1, 0xf00f9344, 0x8708a3d2, 0x1e01f268, 0x6906c2fe, 0xf762575d, 0x806567cb, 0x196c3671, 0x6e6b06e7, 0xfed41b76, 0x89d32be0, 0x10da7a5a, 0x67dd4acc, 0xf9b9df6f, 0x8ebeeff9, 0x17b7be43, 0x60b08ed5, 0xd6d6a3e8, 0xa1d1937e, 0x38d8c2c4, 0x4fdff252, 0xd1bb67f1, 0xa6bc5767, 0x3fb506dd, 0x48b2364b, 0xd80d2bda, 0xaf0a1b4c, 0x36034af6, 0x41047a60, 0xdf60efc3, 0xa867df55, 0x316e8eef, 0x4669be79, 0xcb61b38c, 0xbc66831a, 0x256fd2a0, 0x5268e236, 0xcc0c7795, 0xbb0b4703, 0x220216b9, 0x5505262f, 0xc5ba3bbe, 0xb2bd0b28, 0x2bb45a92, 0x5cb36a04, 0xc2d7ffa7, 0xb5d0cf31, 0x2cd99e8b, 0x5bdeae1d, 0x9b64c2b0, 0xec63f226, 0x756aa39c, 0x026d930a, 0x9c0906a9, 0xeb0e363f, 0x72076785, 0x05005713, 0x95bf4a82, 0xe2b87a14, 0x7bb12bae, 0x0cb61b38, 0x92d28e9b, 0xe5d5be0d, 0x7cdcefb7, 0x0bdbdf21, 0x86d3d2d4, 0xf1d4e242, 0x68ddb3f8, 0x1fda836e, 0x81be16cd, 0xf6b9265b, 0x6fb077e1, 0x18b74777, 0x88085ae6, 0xff0f6a70, 0x66063bca, 0x11010b5c, 0x8f659eff, 0xf862ae69, 0x616bffd3, 0x166ccf45, 0xa00ae278, 0xd70dd2ee, 0x4e048354, 0x3903b3c2, 0xa7672661, 0xd06016f7, 0x4969474d, 0x3e6e77db, 0xaed16a4a, 0xd9d65adc, 0x40df0b66, 0x37d83bf0, 0xa9bcae53, 0xdebb9ec5, 0x47b2cf7f, 0x30b5ffe9, 0xbdbdf21c, 0xcabac28a, 0x53b39330, 0x24b4a3a6, 0xbad03605, 0xcdd70693, 0x54de5729, 0x23d967bf, 0xb3667a2e, 0xc4614ab8, 0x5d681b02, 0x2a6f2b94, 0xb40bbe37, 0xc30c8ea1, 0x5a05df1b, 0x2d02ef8d
};
u32 updateCRC32(unsigned char ch, u32 crc)
{
return UPDC32(ch,crc);
}
u32 crc32buf(char *buf, size_t len)
{
register u32 oldcrc32;
oldcrc32 = 0xFFFFFFFF;
for ( ; len; --len, ++buf)
{
oldcrc32 = UPDC32(*buf, oldcrc32);
}
return oldcrc32 ^ 0xffffffffUL;
}
unsigned int CalcCRC32(unsigned char *data, unsigned int length)
{
return crc32buf((char*)data, length);
}
|
#157999 - Cydrak - Tue Jun 03, 2008 12:46 am
When you said:
Code: |
#define UPDC32(ch, crc) \
(crc = (crc >> 8) ^ crc_32_tab[(crc ^ (ch)) & 0xff])
...
oldcrc32 = UPDC32(*buf, oldcrc32);
|
You get this:
Code: |
oldcrc32 = (oldcrc32 = (oldcrc32 >> 8) ^ crc_32_tab[(oldcrc32 ^ (*buf)) & 0xff]);
|
The cryptic warning, I think, means you've changed a variable "twice on the same line." According to http://gcc.gnu.org/ml/gcc/2004-10/msg00026.html , the assignments can happen in any order (in other words, it's the order that's not defined).
(I'm really hand-waving here. To be drippingly technical the term is sequence point. They've nothing to do with program lines, except that ";" tends to herald a new one. But as with most things in C, several can (and do) coexist on the same line.)
PS: Please try and google this yourself (or if you did, say so!): warning "operation on" "may be undefined"
This gives the linked post as the top result. As a side note, the macro does hide the problem here. Which is good reason to be careful with them; seems to me updateCRC32 could easily be "static inline" and reused instead.
#158005 - yellowstar - Tue Jun 03, 2008 4:06 am
Thanks. Removing that assignment in crc32buf fixed it.
I tried googling after I posted eariler, but I guess I googled for the wrong thing I guess...
Is there anyway to tell when the project is being compiled for Win32, Unix, or DS? WIN32 and NDS aren't defined unless windows.h/nds.h are included...(I have to manually comment/uncomment the include command for windows.h right now...)
Wmb_asm now automatically checks the RSA-signature, if ndsrsa.exe is in the working directory. This is done by executing that app I just mentioned, so it checks the signature. Only on Windows however. If somebody posts how to execute programs on Unix, I can add support for the automatic rsa-check on Unix.(In other words, Lunix, or Unix-based operating systems)
Strangely, Sudoku, and probably all the other demos, are failing the RSA-check again...
#158010 - yellowstar - Tue Jun 03, 2008 5:26 am
The invalid-RSA signature bug has been fixed. Here the second header's CRC wasn't reset to the original CRC, so the RSA was invalidated.(It was the checksum of the modified first header)
But now there's problems with the description in the banner... To much null data between the game name and the new line character.
#158024 - yellowstar - Tue Jun 03, 2008 7:06 pm
That block is somewhat working now. It doesn't work on all of these captures though. But that block isn't done yet.
It works on all of the normal captures, excluding the station capture and multi-game captures. However, the station and Meteos RSA-signatures are invalid. Once again, this is caused by that header CRC. This time, I'm not sure how to fix it... I tried calculating the second header's CRC again, but that didn't work...
#158029 - yellowstar - Tue Jun 03, 2008 7:35 pm
Never mind, I got it working now. All I had to do to fix it was to increment the the size used for calculating the header by one, and that fixed it.
And that transparent-pixel bug seems to have disappeared.
Yikes... I just had my program assemble all of the normal captures at once, and it kind of worked... The program reports that they were assembled correctly anyway... Get this: Almost every .nds has the icon/banner of other .nds binaries. Two .nds binaries have the same icon/banner. While other .nds files, different. Tash has Lup Salad's banner. Card Hero has Tash's banner. Lup Salad has it's own banner. Namco has Picross' banner. Picross has Card Hero's banner. Custom Robo has Namco's banner. Sudoku has Custom Robo's banner. And Sudoku's real banner is nowhere to be found.
I checked the program's output, and it seems after it assembles everything, it assembles everything all over again in the exactly the same order.
#158041 - yellowstar - Tue Jun 03, 2008 10:03 pm
The block now uses the beacons, the RSA-frame, and the header for checking if it ran into the demo it just assembled.
Quote: |
I checked the program's output, and it seems after it assembles everything, it assembles everything all over again in the exactly the same order.
|
Fixed. That was caused by a mistake of mine in my batch file.
RSA-signature checking is now optional. Now, normally it won't check RSA. However, if the option -rsa is the first parameter, it will check RSA. Right now, the output .nds binaries are supposed to go in the same directory as the capture. But for some reason, sometimes they land in the same directory as the program...
The DS Download Station client assembled correctly now, without assembling the client again and failing. The first Multi-game capture still crashes. The other ones... The first demo in the multi-game capture of Meteos/Polarium, only Meteos is assembled. The first demo in the second Multi-game capture, wasn't assembled correctly-bad RSA-signature and crashes on emulator. The first demo in the third is assembled fine, the rest in the capture not at all. The fourth, bad signature and crashes.
Now, when assembling multiple captures at once, it's behaving similar to the multi-game capture assembly. Only the first capture is assembled, the rest not. I think I know what's causing this behaviour - the code for the block which checks the beacons.
#158046 - yellowstar - Tue Jun 03, 2008 10:33 pm
Now that multiple capture assembly is working again. Banners are still wrong.
Multi-game capture 1 still crashes. #2, bad sig and crashes. Then for the next demo in the capture, it fails to find all the data packets. #3, only the first demo is assembled, the rest not. #4, the first demo crashed on emu, the second, GunBullet succeeded, then the next one caused the program to crash. Multi-game assembly seems to have the same banner bug as multi-capture assembly.
#158051 - yellowstar - Tue Jun 03, 2008 11:11 pm
Filb wrote: |
I think I told you earlier what they are.
Those captures are from a real DS Download Service station from a real store in Japan.
|
(Multi-game captures)
? You mean in Japan, there's stores that have a Wii running the Nintendo Channel?
Some of these demos seem to be incomplete.
Wikipedia wrote: |
Like the Wii Shop channel, the Nintendo Channel gets updated every Monday with different DS Demos and new videos every week.
|
Interesting, didn't know that before. So demos for every single DS game aren't available yet.
I have fixed that bug with the output from wmb_asm landing in the wrong directory.
#158054 - yellowstar - Tue Jun 03, 2008 11:48 pm
Quote: |
Some of these demos seem to be incomplete.
|
Never mind, my tool was reporting that because of a bug.
The Meteos/Polarium capture assembles correctly now.
#1 still crashes. The first two demos in capture #2 assemble fine, but the rest aren't assembled at all. In #3, only the first demo is assembled. #4, only the first two demos assembles, then the next one causes the program to crash.
All of these still have that banner bug though.
#158067 - yellowstar - Wed Jun 04, 2008 3:35 am
I need to know exactly how many demos are in each multi-game capture. I checked one capture, and it had the same amount as my program assembled, but I need to make sure.
I found the cause of the crashing:
The first capture's demo has data packets with size 250, like the DS Download Station client. And the memory it requires... See, each packet has a 512 byte block for it. You can see how much unused memory is wasted. Because of this extra memory, it takes a lot of memory to assemble correctly. I try to allocate enough, but it seems there's a maximum to memory dynamically allocated. So I couldn't allocate enough. The only way I could fix this is to use a modified version of that old broken data method, but last time, with the original, it didn't work...
I figured out the problem with those banners. The first demo assembled has the proper banner. But, the next demo has the banner of the previous/first demo. The demo after that has the banner of the previous demo. And the one after that has the banner of the previous demo, and that keeps on repeating.
Order assembled:------------------------Picross Namco Robo Sudoku2 Salad Tash Card
The banners put in the binaries:------Picross Picross Namco Robo Sudoku2 Salad Tash Card
#158068 - yellowstar - Wed Jun 04, 2008 4:00 am
Wmb_asm DS beta has been released.
This is just to test if the DS version works. Test please? If this works, in the next wmb_asm release, the DS binary and source/project files will be included with the next release. Instructions:
DLDI patch, then copy to card. Copy your capture to the root of your card, then rename it to capture.cap. Run the program. It should assemble fine. Otherwise, send me the log file in the program's directory, and the output .nds, if it exists. Next, restart your DS, then run the assembled .nds. If it works, send me the .nds so I can check the RSA-signature.
EDIT:
Updated. Fixed minor bug with the program displaying wmb_asm.exe instead of wmb_asm.nds at the top of the sub screen.
#158087 - yellowstar - Wed Jun 04, 2008 5:11 pm
DSCaptureTest has been updated.
This should fix the bugs with the timestamps for the packets.
#158150 - yellowstar - Thu Jun 05, 2008 9:06 pm
That bug with the description in the banner has been fixed.
Directory support has been added. The directory scanning code scans the whole directory and it's sub directories, in search of captures. And you can do something like this too, instead of just directories:
Code: |
wmb_asm.exe capture.cap captures
|
This would assemble capture.cap, then assemble all the captures in the captures directory. Now the options don't have to be before everything else - they can be mixed with the list of files/directories. Previously, the assembled binaries would be written to the captures' directory. Now, with options, that can be changed. By default, assembled binaries will land in the directories' of the captures. The same will happen with the -nds_capdir option. With the -nds_curdir option, the assembled binaries will be written to the program's working/current directory. And with the -nds_dirDir option, the assembled binaries will be written to Dir. So -nds_dirbinaries would write the assembled binaries to the binaries directory. If the directory doesn't exist, it's supposed to be created. But, on Windows, with that win32dirent.cpp code, mkdir isn't implemented... So the directory creation is temporarily disabled, and will only work if the directory all ready exists.
However, there's some strange bugs with this. The multi-game captures are not assembled at all, the download play client isn't assembled, and one European capture is skipped too. The program says it's reading those captures however. Everything else is assembled fine, minus the banner bug.
#158157 - yellowstar - Fri Jun 06, 2008 12:24 am
DSCaptureTest can only capture beacons for some reason. And stranglely, Filb reported that wifi_lib_test behaves the same way, capturing only beacons... DSCaptureTest is based on wifi_lib_test. So it's not my program's fault, it's wifi_lib_test's fault. Wonder how that dswifi rewrite is coming along?
#158162 - yellowstar - Fri Jun 06, 2008 2:10 am
I had my program report when it leaves the Beacon stage. On the first capture, it does so, and then assembles. With the next capture, it does not report that, it goes and does the assembly.
I tried calling the function to reset assembly vars, which is called at the end of the assembly process, after reading each capture. The banners were correct. And the program tried to assemble the multi-game captures. Something must be wrong with the code which calls ResetAsm...
#158165 - yellowstar - Fri Jun 06, 2008 2:40 am
Ah Ha!
It was reset-related. Here, right before the next capture was to be read, the program just finished the beacon stage, and was in the Auth stage. This wouldn't happen if the beacon block code would work. If it would work, ResetAsm would be called, and this banner bug would be fixed. So, the next capture would have the banner/advert of the previous transfer. This repeats for all of the captures.
#158170 - yellowstar - Fri Jun 06, 2008 4:04 am
Well, I have gotten it to work kind of better... The first and second banners are fine. The third has the second's banner. The fourth has almost the whole banner of the previous demo, but a small part is it's own banner. The next one has about half of the previous demo's banner, and the other half it's own. The next one has most of it's own banner, but a few parts are from the previous demo. The last has the whole, complete banner of the previous demo.
#158179 - Filb - Fri Jun 06, 2008 10:18 am
Not sure how exactly you do all this. Maybe you should try to do this without "states"?
First, read the whole capture file (all of it). Then collect data by going through every packet you can find. Add their data to different arrays (one for beacons, one for data, etc.).
Since both beacons and the data contain a GameID (starting with 0), you could have an array key of something like HostMAC and GameID concatenated. "dataArr[hostMAC+gameID][count++] = curData;" or so... you get the idea.
After you've done the data collection, you just go through all those arrays and merge the data.
#158182 - yellowstar - Fri Jun 06, 2008 9:46 pm
I can't read the whole capture, and then process it - that wouldn't work with the way my code is. The scheme with my code is, all that the code feeding the assembler has to do is, call InitAsm for initializing, then HandlePacket for each packet it finds/reads. This was designed this way so it could be integrated with the future capture apps.(Programs to capture & assemble at the same time, on PC, and hopefully DS)
I got my old data method to work. It works with all of the normal captures. That EU Brain Age capture used to crash the program, but I fixed that.(Banner description bug)
Before, the program used to think it reached the start of Arm7/end of binaries too soon. This was because of one variable and it's calculation. The code used to not check if a packet was missed, and the offset would be wrong. Now, when a packet was missed, instead of just incrementing by zero for the size, the size for most of the packets calculated before is used.
But my code still has issues with multi-game captures. The bug with only assembling one demo out of the whole capture is still there.(Not related to this different data method, that bug was there before)
Crashing is gone, invalid RSA-signature is in. I'm still working on it.
About that banner bug:
The reset code is done correctly now. But now I need to figure out how to make sure the beacons it's getting are from the same demo...(GameID can't be used, since it's zero with most of these captures)
#158185 - yellowstar - Fri Jun 06, 2008 10:40 pm
It would help a lot if I could have an assembled binary of one of the demos in these multi-game captures... My bioum and the one in that download are way different... Only the first demo in each capture can be used though.
#158217 - yellowstar - Sat Jun 07, 2008 5:09 pm
The output filenames are done different now. Instead of only using the gameTitle in the header, the gameTitle, gameCode, and makercode are used.(A _ character is inserted between these values in the filename)
Most demos for games made by Nintendo have the gameTitle set to NINTENDO all the time. Because of the old way of doing filenames, when there were more than one demo with the NINTENDO title, the previous one would be overwritten with the old. That doesn't happen anymore.
Fixed bugs in my pcap reading code which causes the program to freeze.(Has to do with truncated packets)
#158230 - yellowstar - Sat Jun 07, 2008 9:36 pm
It seems these multi-game captures aren't what I thought they were. They aren't from a Wii, they are from a store in Japan.
First, I must explain what this gameID is. This ID is the ID of the game. It is right after the nonadvert flag, before the connected_clients field. Multiple games can be broadcasted & transferred at the same time. You don't see this in America, or anywhere really anymore, but in the past, in Japan, DS Download Stations didn't exist. Instead, you'd go into a store, start your DS for WMB, and the demos would show up right away, no special client or anything. These multi-game captures are captures of these WMB hosts in those stores. This was done with gameIDs. This shows up in data packets, right after the command byte. But for the RSA-frame, we're not sure where it is...(This gameID is always zero in captures from a Wii since only one demo can be served at a time.)
EDIT:
My bad, I forgot something. It wasn't me who discovered this, it was Filb.
#158240 - HyperHacker - Sun Jun 08, 2008 2:33 am
I wonder why Nintendo switched to that silly second-stage loader. Is it faster or more stable? Or did they just want to make the games more difficult to capture? Maybe they just wanted to be able to broadcast 16+ different demos without tying up the channel for someone who wanted to play multiplayer. It'd be interesting to see what the difference is (and pretty much essential if you want to keep capturing demos).
_________________
I'm a PSP hacker now, but I still <3 DS.
#158281 - yellowstar - Sun Jun 08, 2008 7:56 pm
Maybe it's more user-friendly? Also, that old method seems to be something thrown together rather quickly. The Download Stations of today are more planned out and such.
I forgot to post about the rest of the gameID: The gameID is stored in the data packets, right after the command byte. The RSA, and Authentication frames, I think I found where the gameID is in those frames. Not 100% sure, still needs confirmed. I think it's stored in the duration field in the ieee 802.11 frame header. The second byte is the gameID, 1-based, unlike zero-based IDs in the beacons and data packets. However, some math needs done on it first. The first byte determines what kind of math, if any.
#158309 - yellowstar - Mon Jun 09, 2008 12:53 am
Ugh...
Now wmb_asm is really screwed up... It can't even assemble one normal capture correctly... Unfortunately, I didn't back up the version with this new data method... It didn't take long to implement the last time, so hopefully it will be like-wise this time. I think this might have started when I started implementing the gameID, I'm not sure.
#158313 - yellowstar - Mon Jun 09, 2008 2:53 am
I have added a new feature: There is a new option called -run. With this, the output is opened in Windows Explorer as if you double-clicked on it. With my setup, that means every time I run the assembler, No$gba starts running the assembled binary.(When I want it to that is)
In the future, I might add a option so the assembled binary is automatically copied to the homebrew card.
I backed up that bugged version, and I'm now using a version based on my last back up, before the bugged back up. I'm going to write this data method all over again. Same one as before, shouldn't take long hopefully. Any bugs that were fixed since that backup will be fixed.
#158315 - yellowstar - Mon Jun 09, 2008 3:14 am
Done! That data method implemention is done! Next step: Attack the bugs!
#158318 - yellowstar - Mon Jun 09, 2008 4:02 am
All of the bugs previously fixed between the time when I got that data method working, and the bugged version, has been fixed again. Also, the features that were added, have also all been added. And this time I did a back up. Next target: Get these multi-game captures working.
How am I supposed to use a banner in a DS homebrew, with an arm9-only program? I tried writing my own tool to do it, but all I get is lines in the icon. Everything else is fine.
#158345 - yellowstar - Mon Jun 09, 2008 3:16 pm
That mkdir problem has been fixed. All I had to do was include io.h, and it worked. New options has been added: -stop, -nostop, -copy_dirDir
-stop has the usual "Press any key to continue" thing when done, without any errors and such.(This is the default)
-nostop doesn't display the above text.
-copy_dirDir is similar to -nds_dirDir, except the output is copied to Dir. Which can be used to copy the output to the homebrew card.
Previously, I tried writing 32-bit and 16-bit versions of memcpy.(Copying 2 and 4 bytes at a time instead of just one byte at a time)
But with my code all I get in the target is zeroes...
Code: |
void memcpy32(void* in, void* out, size_t length)
{
int len = (int)length % 4;
int Len=(int)length;
if(len!=0)Len-=len;
unsigned int *input, *output;
input=(unsigned int*)in;
output=(unsigned int*)out;
for(int i=0; i<(int)Len; i+=4)
{
*output=*input;
input++;
output++;
}
if(len!=0)
memcpy(output,input,(size_t)len);
//fprintf(Log,"MEM32 %d %d\n",(int)length,(int)len);
}
void memcpy16(void* in, void* out, size_t length)
{
int len = (int)length % 2;
int Len=(int)length;
if(len!=0)Len-=len;
unsigned short *input, *output;
input=(unsigned short*)in;
output=(unsigned short*)out;
for(int i=0; i<Len; i+=2)
{
*output=*input;
input++;
output++;
}
if(len!=0)
memcpy(input,output,len);
}
|
The data copying takes a lot of time. Assembly is several seconds faster if the call to memcpy is disabled. But I still need a way to copy in data, but I want it to be even faster.
#158372 - yellowstar - Mon Jun 09, 2008 8:37 pm
I just had my assembler assemble all of these mulit-game captures, and those other captures.(gameID not implemented yet)
The first capture and third capture, it assembled tetris out of them fine, but it ignored the rest of the demos in those captures. The second, it didn't find anything at all. In the fourth, EIGODUKE failed the RSA-check and crashed the emulator. But, it actually assembled the next demo, and it assembled successfully. And the Meteos/Polarium capture, both demos were assembled correctly.
#158375 - yellowstar - Mon Jun 09, 2008 10:58 pm
GameID has been implemented. There's a minor banner bug with this. Should be easily fixed - it's caused by getting the wrong advert for the transfer. Easily fixed, I think. Most of the binaries passed the RSA-check, not all. One demo crashed the assembler, the only failures were all three of EIGODUKE demos with that gameTitle.
That gameID in the data packets... Well, it doesn't seem to be correct. In a bunch of packets,(header/data)
this value was 04, while it was supposed to be 00. I got around that by using the ID in the duration_id.
Okay, here's the arthrogram for getting the gameID out of the duration_id, in the ieee 802.11 header:
WARNING: This is subject to change, if I find any different behavior for this.
The second byte in the duration, Byte2, is the ID. However, some math needs done on it to get the correct ID. Byte1 determines the math, if any. If Byte1 is 0x00, subtract Byte2 by one. If Byte1 divided by 8, if the result is 25, do nothing. Otherwise, add Byte2 by one. The gameID is Byte2 after these operations.
Code: |
unsigned char GetGameID(unsigned char *data)
{
iee80211_framehead2 *framehead = (iee80211_framehead2*)data;
unsigned Byte1, Byte2, gameID;
unsigned char *bytes;
Byte1=0;Byte2=0;gameID=0;
bytes = (unsigned char*)&framehead->duration_id;
Byte1 = bytes[0];
Byte2 = bytes[1];
if(!nds_data.multipleIDs)return 0;//Always return gameID zero if there isn't more than one game broadcasted.
if(Byte1==0x00)
{
Byte2--;
}
else
{
if(Byte1 % 8 != 25)//Don't do anything if the remainder from this devision operation is 25
{
Byte2++;
}
}
//Byte2--;//I thought this value was 1-based, but apparently it's not...
gameID = Byte2;
return gameID;
}
|
#158390 - HyperHacker - Tue Jun 10, 2008 9:41 am
yellowstar wrote: |
Previously, I tried writing 32-bit and 16-bit versions of memcpy.(Copying 2 and 4 bytes at a time instead of just one byte at a time)
But with my code all I get in the target is zeroes...
Code: |
void memcpy32(void* in, void* out, size_t length)
{
int len = (int)length % 4; //*********BUG HERE*********
int Len=(int)length;
if(len!=0)Len-=len;
unsigned int *input, *output;
input=(unsigned int*)in;
output=(unsigned int*)out;
for(int i=0; i<(int)Len; i+=4)
{
*output=*input;
input++;
output++;
}
if(len!=0)
memcpy(output,input,(size_t)len);
//fprintf(Log,"MEM32 %d %d\n",(int)length,(int)len);
}
void memcpy16(void* in, void* out, size_t length)
{
int len = (int)length % 2; //*********BUG HERE*********
int Len=(int)length;
if(len!=0)Len-=len;
unsigned short *input, *output;
input=(unsigned short*)in;
output=(unsigned short*)out;
for(int i=0; i<Len; i+=2)
{
*output=*input;
input++;
output++;
}
if(len!=0)
memcpy(input,output,len);
}
|
|
You're doing length % 4 when you should be doing length / 4 (or 2 in the 16-bit version).
[edit] Can't do bold within the code... <_<
_________________
I'm a PSP hacker now, but I still <3 DS.
#158399 - Cearn - Tue Jun 10, 2008 11:59 am
HyperHacker wrote: |
yellowstar wrote: | Previously, I tried writing 32-bit and 16-bit versions of memcpy.(Copying 2 and 4 bytes at a time instead of just one byte at a time)
But with my code all I get in the target is zeroes...
Code: |
void memcpy32(void* in, void* out, size_t length)
{
int len = (int)length % 4; //*********BUG HERE*********
int Len=(int)length;
if(len!=0)Len-=len;
unsigned int *input, *output;
input=(unsigned int*)in;
output=(unsigned int*)out;
for(int i=0; i<(int)Len; i+=4)
{
*output=*input;
input++;
output++;
}
if(len!=0)
memcpy(output,input,(size_t)len);
//fprintf(Log,"MEM32 %d %d\n",(int)length,(int)len);
}
void memcpy16(void* in, void* out, size_t length)
{
int len = (int)length % 2; //*********BUG HERE*********
int Len=(int)length;
if(len!=0)Len-=len;
unsigned short *input, *output;
input=(unsigned short*)in;
output=(unsigned short*)out;
for(int i=0; i<Len; i+=2)
{
*output=*input;
input++;
output++;
}
if(len!=0)
memcpy(input,output,len);
}
|
| You're doing length % 4 when you should be doing length / 4 (or 2 in the 16-bit version).
[edit] Can't do bold within the code... <_< |
That's not a bug. The code uses two variables for the number of chunks and the remainder: Len and len. Len (uppercase) is used for the main stint and len (lowercase) for the remainder. It's a poor choice of names, yes, but not an actual bug. The separation of variables - as well as all the int and size_t casts - are unnecessary anyway, since you can just use for(i=0; i<length/4; i++) in the loop, get the remainder afterwards and then fill in the tail end of the array.
What are (potential) bugs are the pointer casts. If the source and destinations aren't properly aligned, the copies will fail. If you want functions that work properly regardless of alignment, size and memory section (you know, like VRAM), go here.
As an aside, memcpy already uses a 4x unrolled loop of word copies if the conditions are right. In most cases it will already be faster than the code you have there. And then there is always DMA and CpuSet.
#158417 - yellowstar - Wed Jun 11, 2008 1:07 am
There's some assembly code in tonclib which has assembly code for copying like how I want. Hopefully it will work on PC too... If not, I'll try what you posted Cearn.(The link)
And because this is cross-platform,(PC, DS...)
DMA, BIOS functions, and such, can not be used.(They could be used instead of normal copy functions in the DS port, but I'd rather not)
I'm thinking about putting wmb_asm on Google Code.(For the free SVN server mainly)
However, I need to know something about these licenses:
My tool uses code copied from other homebrewers code. In the source, it is noted when it is somebody else's code, for these blocks of code that were copied. This code would be in the public domain, correct? So it would be okay to release source code with GNU GPL v3, with code copied from other peoples code, being note when code isn't mine, and not everything was coded by me?(Most of the code is mine, but important code, such as structs and defines for WMB data, are not mine.)
I forgot about adding the authors of the code which were copied in the ReadMe... That will be there in the next release, but here's the list now:
Credits:
Juglak, for his data structures and defines for WMB.
Masscat, for his WMB client for reference and code.
Frz, for his CRC calculation code, and other things.
#158419 - yellowstar - Wed Jun 11, 2008 2:31 am
I tried to compile that assembly with Dev-Cpp and GNU, but I couldn't get the .s files to compile at all... I tried the code in that link, and all I get is zeroes again.(PC)
I added more options: -showtime and -notime. -showtime is the default. With the -showtime option, the total time it took to assemble everything is displayed at the end. With -notime, that's not displayed. According to this, it takes ~8 seconds to assemble all 7 captures. Assembly time to assemble every single capture is ~38 seconds.
That gameID in the duration thing... It seems the duration doesn't contain the gameID. At the moment the only way to get the gameID is to get it out of the data packets. But, how do we guarantee the RSA-frame is for this gameID? We can't, till we find out how the WMB client tells the host the gameID it wants. This seems to be the cause of one demo failing the RSA-check, and still working on emulator.(Getting the wrong RSA-frame and signature)
Other than that demo, there's two other demos not working. Everything else is assembled fine. The first screwed demo turns out to be the very first demo assembled out of the multi-game captures. And the other two are the EIGODUKE demos. Which are demos of a Japanese game in which you seem to learn Japanese.
My program writes a log file when run for assembly. I'd like to disable this for releases, but I don't know how... One really bad way would be commenting out all the log file stuff, but that would take too much time. Is there an easy way, which I could just define/undefine some define, like DEBUG?(All the log file stuff use fprintfdebug defines, which are really fprintf calls)
(Assembly speed might increase too, with log file disabled)
EDIT:
I forgot to mention:
The banners still aren't correct with multi-game assembly. EIGODUKE, the broken one, has a complete banner, Meteos has almost the whole banner with one piece missing. Polarium has the other piece. All the other demos don't have a banner at all.
#158422 - yellowstar - Wed Jun 11, 2008 4:08 am
I found the problem with that first demo: For some reason, the arm7 start seq is being set 3 seqs sooner than it should be. Resulting in crashing and failed RSA-check.
#158453 - yellowstar - Wed Jun 11, 2008 8:12 pm
It seems the first broken demo is actually starting with the correct arm7s seq...
#158455 - yellowstar - Wed Jun 11, 2008 8:36 pm
There's a ton of zeroes at the Arm7 binary... And for some reason, my program isn't finding the end of the Arm7 binary/data transfer, and yet, it's continuing into the assembly stage...
#158464 - yellowstar - Wed Jun 11, 2008 9:39 pm
Out of all 6 invalid-RSA sig demos, 4 have the exact same arm7e bug. The other two don't have this glitch. Strangely, the arm7e pos in the saved_data buffer is zero, while the seq is set to a non-zero value. Strange...
#158478 - yellowstar - Thu Jun 12, 2008 2:54 am
I just had my tool assemble all of the original, normal captures, and... The very first demo assembled, it failed... Crashed on emu, and failed the RSA-check. All of the other demos are fine though.
#158480 - yellowstar - Thu Jun 12, 2008 3:05 am
Never mind, that has been fixed.
#158483 - yellowstar - Thu Jun 12, 2008 4:22 am
The banner bug with multi-capture assembly has been fixed! The banner glitch previously went back to the each-demo-has-the-previous-demo's-banner-bug. Once I put the code back for that beacon block code, it worked!
But that EUBA capture is acting up. The description has most of the DS Download Station's description. Except for the first part. The first part is correct. That's Dr.kawashi. The rest has the DS Download Station's description. The icon is correct though. When I change some code so the the advert is copied in, even when it found that beacon before, the banner is correct. But then another demo's icon has a section that's wrong. The banner isn't the only problem though, it fails the RSA-check and crashes the emulator now.
#158518 - yellowstar - Thu Jun 12, 2008 7:45 pm
Things started working differently now that I change some things in my code. The EUBA demo has the correct banner now, but still crashes on emu. That other demo's banner is still fine. There's 6 demos(in mulit-game captures, this doesn't include the EUBA demo)
that fail the RSA-check, and yet, all but two don't crash the emulator. I think this is because of that problem with getting the wrong RSA-frame and signature. The banners in these demos in these multi-game captures are working better. The MP demos' banners are correct, pass RSA, and run fine. The first EIGODUKE demo assembled has the correct banner, but bad RSA and crashes emu. The other duke demo runs fine, fail RSA, some parts of the icon are fine, while the rest are blank. NEKOSOGI has correct banner, runs fine, and fails RSA. The middle of the banner for KURUKESHI! is blank, fails RSA, and runs fine.
#158519 - yellowstar - Thu Jun 12, 2008 7:54 pm
yellowstar wrote: |
I think this is because of that problem with getting the wrong RSA-frame and signature. |
Confirmed, this is the problem. I injected the RSA-signature from a assembled binary from one of these multi-capture demos into my assembled demo, and it passed the RSA-check. The only way to fix this is to find how the official WMB client tells the host the gameID it wants.
#158575 - yellowstar - Fri Jun 13, 2008 6:52 pm
I think I fixed that multi-game banner bug - not sure. Now it's only assembling the MP demos, attempts to assemble that problematic demo that's assembled first, Tetris, while everything else, it reports that it missed packets.
#158576 - tepples - Fri Jun 13, 2008 7:28 pm
Why do they still have demos for Tetris when it was discontinued after two years?
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#158578 - yellowstar - Fri Jun 13, 2008 7:57 pm
Well, these multi-game captures were captured in April 2006. I checked this assembled demo of it, and the copyright year is that same year. But I wonder if the Tetris demo is available on Nintendo Channel, even it was discontinued? Hmm... I justed googled, and the Tetris demo is available on the Nintendo Channel.
I have gotten Tetris and the EUBA capture to assemble correctly. I tried using code similar to the data packet handling code in old backup, which used to assemble these captures correctly. And it worked. Now every demo has the correct banner, and every demo passes the RSA-check. Now I need to get these other demos to assemble again.(The ones that my tool reports that packets were missed)
I have one method for this - didn't try it yet though. There would be this "unused" packet file. When a packet isn't used, it is written into this file. When the end of the capture is reached, and the assembly wasn't finished, the program would go through the whole file, and calling HandlePacket for each packet in the file. In this way, packets that were ignored, but were for this transfer, not the other transfer, can be used.
#158581 - Filb - Fri Jun 13, 2008 10:05 pm
Odd. They still sell it in Germany. Amazon.de also has it in stock.
Sorry, can't help on this project for a while. I'm having exams very soon.
#158593 - yellowstar - Sat Jun 14, 2008 2:09 am
It would probably make more sense to have the Tetris demo available only in the countries that still have the Tetris DS game, but they released the demo everywhere...
I tried that method I mentioned, and it works well. But, when there's still more demos after using that trick, it isn't assembled. Which seems to be happening to the KURUKESHI! demo, as I think it's the only demo not assembled.(At least the whole assembly process isn't finished)
I tried to fix this by recursion, but, the program got stuck calling the function it's executing indefinitely. I tried to fix that, but, I have yet to find a solution that works properly. Very close to finishing this. But after this, well, there's DS Download Stations to tackle still... So, what are your opinions on this, everyone? Should station support be in the next release, or after? If the former, it will be longer still before the next release. If the latter, the next release would most likely be once this multi-game functionality is done.
@Filb:
I see. There's things to discuss when you get the time.(Obtaining captures and other things)
#158596 - yellowstar - Sat Jun 14, 2008 3:20 am
Or not... I added code so my program reports how many demos it finds broadcasted in each capture, the ID of each game right as it finds it, and whether or not it assembled all of the demos in the capture. And... With the first capture, which has Tetris, it assembles the first demo fine, then the other 4 demos aren't assembled at all. Likewise for the next capture, but it's not Tetris. And there's that "failed to find the header" message... For the next capture, it assembles 3 out of 6 demos. And that header error showed up again. The MP capture was assembled fine. For the next one, only 1 out of 4 demos were assembled.
#158633 - HyperHacker - Sun Jun 15, 2008 6:25 am
yellowstar wrote: |
So, what are your opinions on this, everyone? Should station support be in the next release, or after? |
I think you should focus on releasing a working version before adding more major features.
_________________
I'm a PSP hacker now, but I still <3 DS.
#158656 - yellowstar - Mon Jun 16, 2008 12:50 am
Okay then, station support will wait for the release after this next one.
That one demo wasn't the only one not being assembled. Several others weren't being assembled. Now only one isn't being assembled. I know this because I had my program dump the advert from the beacons into .nds files. This way I can easily see the icons and such for the demos, even for the ones not assembled. And this demo was the only one not assembled. The others were already assembled. Banners are perfect. RSA-signatures are perfect. Previously two demos had bad signatures due to that wrong RSA-frame bug. All I had to do to fix that was, jump back to the Auth/Assoc/RSA stage, and that's all it took. The problem was it was previously getting the signature for a demo it already assembled. For some strange reason my program crashes after the system("PAUSE") code... And when debugging, it always hangs on the function call to my implemention of pcap_close. With normal execution, that doesn't happen.
A while ago I noticed that the filesize of the wmb_asm PC binary grew to ~1.33 MB. It used to be ~300 KB. The DS version right now is ~300 KB. I think this happened because of the switch to 100% C++, to fix DS compiling problems. And I'm not switching to 100% C just for a small binary. Is there any way to make this PC filesize smaller, with GNU/wxDev-Cpp?
A while ago I came up with more GUI features. The front-end would have an install feature. It would install the Windows Shell functions that I manually did to my PC. With this, you could right-click the captures(s), click Assemble, and the capture(s) would be assembled on-the-spot. And you could right-click on directories, and assemble all the captures in the directory, and it's sub-directories right away. Unless there some library out there I don't know about, this feature would be restricted to Windows only. And another feature: There would be this capture/assemble utility running in the background, with an icon in the system tray, or as on Vista, "Notification area". This program would be based on wmb_asm. It would capture packets all the time, unless you explicitly disable it. And it would assemble the demo from the packets right as they are captured. The system tray icon would change depending on if it found WMB traffic and is assembling, or if it didn't find anything. You can check it's status by hovering the mouse over the system tray icon, or by clicking on it.
Note that these features and the front-end will not be in the next release, that will wait for later.
#158702 - yellowstar - Mon Jun 16, 2008 5:24 pm
I have fixed that crash bug. It used to crash after that "Press any key to continue" was displayed when assembling all of the multi-game captures. It was caused by a buffer overflow in the beacon handling code.
Now it doesn't report any errors, other than the "failed to assemble all..." errors.
#158704 - yellowstar - Mon Jun 16, 2008 6:07 pm
These headers need explaining. As I said before, there two headers. The second is one that is supposed to be sent in WMB, not the first. Some demos don't set the romSize and bannerOffset fields properly in the header. This header is put in the second header in the binary. When this happens, an assembler needs to take these fields, fix them, and put them in the first header. When the header is modified, a WMB Host that uses the first header, not the second, the demo will not boot because of the failed RSA-verification/check. When sending, the second header needs to be used, but for getting the romSize/bannerOffset, that data must be grabbed from the first header. WMB Host mod doesn't have any problems with this, and wmb_asm doesn't either. However, I know of one host that probably has this problem.
#158720 - yellowstar - Mon Jun 16, 2008 8:43 pm
Wmb Asm is just about ready for the next release. I think the problem with these multi-game captures is, they are incomplete. I checked one capture, and compared the output from my tool, with Filb's Python assembler, and it was the same for that capture. There was a bug with the description in the Polarium demo. Which was in the capture too. I have fixed that. Wmb Asm now reports when a multi-game capture is incomplete, when at least missing one demo entirely.
#158735 - yellowstar - Mon Jun 16, 2008 10:53 pm
wiki.akkit.org - DSDemoCapture wrote: |
Capturing DS demos
At present, the best way to capture demos for DS is to use a 802.11 capable capture program on a computer, and download the demo repeatedly on the DS - capturing raw data is the technique that has been used to capture most/all of the demos listed on the Download page. Capturing can be done with a compatible capture program (anything that can capture raw 802.11 data), and the data must then be given to an individual in the community who can process it into a usable file. There are some tools in the works to make this process much easier.
(todo: explain this better)
|
Are these tools referring to an assembler, tools to easily capture WMB transfers, or both?(Which seems to have been dropped by the people who started it; That page was there for a long time)(I guess it would probably be both)
Making the process easier is what I'm aiming to do, also. But before I can write the front-end and the capture tool, with assembly, I need to somehow get the assembly code into a DLL. But, I like how it is... If it would be DLL, changing the wmb_asm main console code and such wouldn't be easily done.(That code and the assembly are in the same project right now, but with DLL, otherwise)
There's workarounds however, to keep console and assembly code easily accessible and compiled in the same project, and still have DLL.
#158736 - yellowstar - Mon Jun 16, 2008 11:31 pm
How would I compile a DLL across multiple platforms?(PC, not consoles)
EDIT:
Never mind, I have found how to write Linux .so binaries.
#158787 - yellowstar - Tue Jun 17, 2008 11:17 pm
Wmb Asm is now modular. Meaning that the assembly code, and the pcap reading code at the moment, is in a DLL/shared library. Not sure why, but once again I had to use decorated function names.(Maybe it's because wxDev-Cpp/Dev-Cpp doesn't create a def file? How would I force it to do that?) I used this to find what those names were. Here's some examples of such names: _Z12HandlePacketP12spcap_pkthdrPhiPPcP7spcap_tbS2_bS2_b _Z7InitAsmPFvvE
What would a Makefile for compiling shared libraries on unix look like? And the host of the DLL?
Here's my Windows Makefiles. How would I edit these for Unix?
DLL
Code: |
# Project: wmb_asm
# Compiler: Default GCC compiler
# Compiler Type: MingW 3
# Makefile created by wxDev-C++ 6.10.2 on 17/06/08 17:18
CPP = g++.exe
CC = gcc.exe
WINDRES = windres.exe
OBJ = source/dllmain.o source/ndsdirent.o source/pcap.o source/utils.o source/wmb.o wmb_asm_private.res
LINKOBJ = source/dllmain.o source/ndsdirent.o source/pcap.o source/utils.o source/wmb.o wmb_asm_private.res
LIBS = -L"C:/Dev-Cpp/Lib" --no-export-all-symbols --add-stdcall-alias
INCS = -I"include"
CXXINCS = -I"C:/Dev-Cpp/lib/gcc/mingw32/3.4.2/include" -I"C:/Dev-Cpp/include/c++/3.4.2/backward" -I"C:/Dev-Cpp/include/c++/3.4.2/mingw32" -I"C:/Dev-Cpp/include/c++/3.4.2" -I"C:/Dev-Cpp/include" -I"C:/Dev-Cpp/" -I"C:/Dev-Cpp/include/common/wx/msw" -I"C:/Dev-Cpp/include/common/wx/generic" -I"C:/Dev-Cpp/include/common/wx/fl" -I"C:/Dev-Cpp/include/common/wx/gizmos" -I"C:/Dev-Cpp/include/common/wx/html" -I"C:/Dev-Cpp/include/common/wx/mmedia" -I"C:/Dev-Cpp/include/common/wx/net" -I"C:/Dev-Cpp/include/common/wx/ogl" -I"C:/Dev-Cpp/include/common/wx/plot" -I"C:/Dev-Cpp/include/common/wx/protocol" -I"C:/Dev-Cpp/include/common/wx/stc" -I"C:/Dev-Cpp/include/common/wx/svg" -I"C:/Dev-Cpp/include/common/wx/xml" -I"C:/Dev-Cpp/include/common/wx/xrc" -I"C:/Dev-Cpp/include/common/wx" -I"C:/Dev-Cpp/include/common"
RCINCS = --include-dir "C:/Dev-Cpp/include/common"
BIN = wmb_asm.dll
DEFINES =
CXXFLAGS = $(CXXINCS) $(DEFINES) -DBUILDING_DLL=1 -fexpensive-optimizations -O3
CFLAGS = $(INCS) $(DEFINES) -DBUILDING_DLL=1 -fexpensive-optimizations -O3
GPROF = gprof.exe
RM = rm -f
LINK = g++.exe
.PHONY: all all-before all-after clean clean-custom
all: all-before $(BIN) all-after
clean: clean-custom
$(RM) $(OBJ) $(BIN)
$(BIN): $(OBJ)
$(LINK) -shared $(STATICLIB) $(LINKOBJ) $(LIBS) -Wl,--out-implib,libwmb_asm.a -o wmb_asm.dll
source/dllmain.o: $(GLOBALDEPS) source/dllmain.cpp include/dll.h include/wmb_asm.h include/dll.h include/pcap.h include/dirscan.h include/win32dirent.h include/ndsdirent.h include/dll_load.h
$(CPP) -c source/dllmain.cpp -o source/dllmain.o $(CXXFLAGS)
source/ndsdirent.o: $(GLOBALDEPS) source/ndsdirent.cpp include/wmb_asm.h include/dll.h include/pcap.h include/dirscan.h include/win32dirent.h include/ndsdirent.h include/dll_load.h include/ndsdirent.h
$(CPP) -c source/ndsdirent.cpp -o source/ndsdirent.o $(CXXFLAGS)
source/pcap.o: $(GLOBALDEPS) source/pcap.cpp include/wmb_asm.h include/dll.h include/pcap.h include/dirscan.h include/win32dirent.h include/ndsdirent.h include/dll_load.h
$(CPP) -c source/pcap.cpp -o source/pcap.o $(CXXFLAGS)
source/utils.o: $(GLOBALDEPS) source/utils.cpp include/wmb_asm.h include/dll.h include/pcap.h include/dirscan.h include/win32dirent.h include/ndsdirent.h include/dll_load.h
$(CPP) -c source/utils.cpp -o source/utils.o $(CXXFLAGS)
source/wmb.o: $(GLOBALDEPS) source/wmb.cpp include/wmb_asm.h include/dll.h include/pcap.h include/dirscan.h include/win32dirent.h include/ndsdirent.h include/dll_load.h
$(CPP) -c source/wmb.cpp -o source/wmb.o $(CXXFLAGS)
wmb_asm_private.res: wmb_asm_private.rc
$(WINDRES) --input-format=rc -o wmb_asm_private.res $(RCINCS) WMB_AS~1.RC -O coff
|
Host
Code: |
# Project: wmb_asm
# Compiler: Default GCC compiler
# Compiler Type: MingW 3
# Makefile created by wxDev-C++ 6.10.2 on 17/06/08 17:59
CPP = g++.exe
CC = gcc.exe
WINDRES = windres.exe
OBJ = source/main.o source/console.o source/dirscan.o source/ndsdirent.o source/win32dirent.o wmb_asm_private.res
LINKOBJ = source/main.o source/console.o source/dirscan.o source/ndsdirent.o source/win32dirent.o wmb_asm_private.res
LIBS = -L"C:/Dev-Cpp/Lib" -g3
INCS = -I"include"
CXXINCS = -I"C:/Dev-Cpp/lib/gcc/mingw32/3.4.2/include" -I"C:/Dev-Cpp/include/c++/3.4.2/backward" -I"C:/Dev-Cpp/include/c++/3.4.2/mingw32" -I"C:/Dev-Cpp/include/c++/3.4.2" -I"C:/Dev-Cpp/include" -I"C:/Dev-Cpp/" -I"C:/Dev-Cpp/include/common/wx/msw" -I"C:/Dev-Cpp/include/common/wx/generic" -I"C:/Dev-Cpp/include/common/wx/fl" -I"C:/Dev-Cpp/include/common/wx/gizmos" -I"C:/Dev-Cpp/include/common/wx/html" -I"C:/Dev-Cpp/include/common/wx/mmedia" -I"C:/Dev-Cpp/include/common/wx/net" -I"C:/Dev-Cpp/include/common/wx/ogl" -I"C:/Dev-Cpp/include/common/wx/plot" -I"C:/Dev-Cpp/include/common/wx/protocol" -I"C:/Dev-Cpp/include/common/wx/stc" -I"C:/Dev-Cpp/include/common/wx/svg" -I"C:/Dev-Cpp/include/common/wx/xml" -I"C:/Dev-Cpp/include/common/wx/xrc" -I"C:/Dev-Cpp/include/common/wx" -I"C:/Dev-Cpp/include/common"
RCINCS = --include-dir "C:/Dev-Cpp/include/common"
BIN = wmb_asm.exe
DEFINES =
CXXFLAGS = $(CXXINCS) $(DEFINES) -O3 -g3
CFLAGS = $(INCS) $(DEFINES) -O3 -g3
GPROF = gprof.exe
RM = rm -f
LINK = g++.exe
.PHONY: all all-before all-after clean clean-custom
all: all-before $(BIN) all-after
clean: clean-custom
$(RM) $(OBJ) $(BIN)
$(BIN): $(OBJ)
$(LINK) $(LINKOBJ) -o "wmb_asm.exe" $(LIBS)
source/main.o: $(GLOBALDEPS) source/main.cpp dll/include/wmb_asm.h dll/include/dll.h dll/include/pcap.h dll/include/dirscan.h dll/include/win32dirent.h dll/include/ndsdirent.h dll/include/dll_load.h
$(CPP) -c source/main.cpp -o source/main.o $(CXXFLAGS)
source/console.o: $(GLOBALDEPS) source/console.cpp include/defs.h include/pcap.h include/dirscan.h include/win32dirent.h include/ndsdirent.h
$(CPP) -c source/console.cpp -o source/console.o $(CXXFLAGS)
source/dirscan.o: $(GLOBALDEPS) source/dirscan.cpp include/defs.h include/pcap.h include/dirscan.h include/win32dirent.h include/ndsdirent.h
$(CPP) -c source/dirscan.cpp -o source/dirscan.o $(CXXFLAGS)
source/ndsdirent.o: $(GLOBALDEPS) source/ndsdirent.cpp include/defs.h include/pcap.h include/dirscan.h include/win32dirent.h include/ndsdirent.h include/ndsdirent.h
$(CPP) -c source/ndsdirent.cpp -o source/ndsdirent.o $(CXXFLAGS)
source/win32dirent.o: $(GLOBALDEPS) source/win32dirent.cpp include/win32dirent.h
$(CPP) -c source/win32dirent.cpp -o source/win32dirent.o $(CXXFLAGS)
wmb_asm_private.res: wmb_asm_private.rc
$(WINDRES) --input-format=rc -o wmb_asm_private.res $(RCINCS) WMB_AS~1.RC -O coff
|
#158809 - masscat - Wed Jun 18, 2008 1:06 pm
yellowstar wrote: |
There's some assembly code in tonclib which has assembly code for copying like how I want. Hopefully it will work on PC too... If not, I'll try what you posted Cearn.(The link)
And because this is cross-platform,(PC, DS...)
DMA, BIOS functions, and such, can not be used.(They could be used instead of normal copy functions in the DS port, but I'd rather not)
I'm thinking about putting wmb_asm on Google Code.(For the free SVN server mainly)
However, I need to know something about these licenses:
My tool uses code copied from other homebrewers code. In the source, it is noted when it is somebody else's code, for these blocks of code that were copied. This code would be in the public domain, correct? So it would be okay to release source code with GNU GPL v3, with code copied from other peoples code, being note when code isn't mine, and not everything was coded by me?(Most of the code is mine, but important code, such as structs and defines for WMB data, are not mine.)
I forgot about adding the authors of the code which were copied in the ReadMe... That will be there in the next release, but here's the list now:
Credits:
Juglak, for his data structures and defines for WMB.
Masscat, for his WMB client for reference and code.
Frz, for his CRC calculation code, and other things. |
I am happy for you to use my code. I am also happy for you to release your wmb_asm (include my code snippets) under GPL v3.
I cannot answer for Juglak or Frz.
Ben (aka masscat)
#158824 - yellowstar - Wed Jun 18, 2008 8:19 pm
Thanks, a response to that PM was the main thing that was delaying the next version from being released. Now that's gone. When the next version is released, the Google Code project page should be available also. And remember, this Google Code means SVN server. So once it's available, anybody that wants to use the very latest, and possibly buggy, version/revision, can do so.
While I was waiting for a response to that PM, I decided to work on these extra features. First was the DLL. After that I was working on the front-end. Speaking of which... I'm using wxDev-Cpp and wxWidgets. But... wxWidgets is large, at least in my opinion, mainly considering the slow upload with Dial-Up. Because of wx, the front-end is ~5.05 MB. Wmb Asm command-line is ~1.28. Unless something is done about this huge filesize, the entire Wmb Asm package, which will contain Wmb Asm command-line, module, Wmb Asm front-end, and the capture tool /w assembler, that would add upto ~11.33 MB. And that would need updated over Dial-Up. Does anybody have any idea how to shrink this? Please? The assembler module could be updated by itself, but then the package wouldn't contain the latest assembler... There's the option of using the wxWidgets DLL. That would probably chop the package size downto ~6.61 MB. If the DLL would be in a separate download than the package, than the package would probably be ~4.24 MB. This size issue has to be addressed. The maxium data I'm allowed to use on my file server is 10 MB. Obviously a big problem.
These extra apps won't be in the next release, but I'll probably put the source code for the front-end on the SVN server.
#158833 - yellowstar - Thu Jun 19, 2008 1:57 am
With GNU GLP v3, is it okay for other people, who downloaded the program, to redistribute it? The GPL FAQ seems to say yes, but another site, well...(It might have meant something else though, for that other site)
I'd like somebody to confirm this.
#158835 - DensitY - Thu Jun 19, 2008 2:22 am
its fine so long source code access is provided as well (either a link to the original source code if they've made no changes or a link to their changes).
the main thing GPL 3 really offers that is different from GPL 2 is todo with patents.
#158837 - yellowstar - Thu Jun 19, 2008 2:36 am
Okay. Now I just need to finish polishing the code, adding documentation/comments for most of the code, and such.
#158867 - yellowstar - Thu Jun 19, 2008 8:52 pm
How should I word the sections of the GPL which will be included in the software source code, which I need to customize? "Copyright 2008 yellowstar, excluding code written by other developers?"(Would the GPL cover the entire program, despite pieces being written by other devs? I'm thinking it probably would. Not covering everything would mean only covering my code)
Or should it just be worded like this: "copyright 2008 yellowstar"?
And what about DSCaptureTest? It's based on code written by sgstair, so how should I word that?(DSCaptureTest will be uploaded unto SVN, and when I get around to it, I'm going to attempt to do a mod to the Arm7 code to change the value of a Wifi register, in an attempt to get the capture software to capture more than just WMB beacons.)
Code documentation/comments is done for wmb_asm,(command-line)
and the front-end. Now I need to document DSCaptureTest, and the assembler module.(Which does the actual assembly)
#158871 - yellowstar - Thu Jun 19, 2008 11:22 pm
I've ran into a problem with DSPacketCapture.(Previously named DSCaptureTest)
There's a license on the code it's based on, the MIT license. I'm going to put my code under GPL v3, but the code DSPacketCapture is based on, wifi_lib_test, is MIT. I guess I can leave the license alone for DSPacketCapture. But, in the future, DSPacketCapture will have integration with the assembler. And the assembler will be under GPL v3. Are GPL v3 and MIT compatible?(So could I use code under MIT and GPL in the same program?)
#158877 - yellowstar - Fri Jun 20, 2008 3:03 am
The code documentation/commenting is done, for DSPacketCapture. Now all that remains is the assembler. People, if those questions about these licenses aren't answered... The next release will be delayed until those are answered, if I finish documentation before then.(Sorry for the rush, but...)
#158902 - tepples - Fri Jun 20, 2008 12:58 pm
Code under most permissive licenses, such as the "MIT license" used for X11, Expat, and Lick's RAM API, is compatible with the GNU GPL. When in doubt, see the GNU license list.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#158911 - yellowstar - Fri Jun 20, 2008 7:07 pm
I guess wmb_asm module documentation is done...(I don't really like commenting and such...)
Most of the code is the code which handles the packets for each stage. They should be self-explanatory, but if anybody has any questions about the code, ask.(After the next release)
Okay, now only that wording question remains.
GPL web site wrote: |
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
|
So should I put something like this in my source code? Even though code was copied from other programs, written by other developers?(With the copied code originally being in the public domain?)
Code: |
Wmb Asm Module, assembles packet captures of DS WMB transfers into an .nds binary
Copyright (C) 2008 yellowstar
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
|
I see on the web site, that I'll need to display the above in the front-end About message box. And I'll need to display a brief version every time wmb_asm(command-line)
is run...
#159063 - yellowstar - Wed Jun 25, 2008 1:30 am
Ahem... People, Wmb Asm will not be released until these licensing questions are answered...
Another thing:
How should the DSPacketCapture license be worded? "Copyright yellowstar, with base code originally coded by SgStair"?
Please answer these licensing questions. No answers to these questions, no Wmb Asm. Answer them, and you'll get Wmb Asm people.
#159068 - yellowstar - Wed Jun 25, 2008 3:03 am
I found bugs in the directory scanning code. Before when testing Wmb Asm DS beta, the directory scanning code was acting up, so that need temporarily disabled on DS. On my main dev computer, the scanning works fine, but this computer I'm using on this vacation, well, it's acting up. I have scanning code that should work, on my main dev PC, but I can't get that code for ~2 weeks. So even if those licensing questions are answered before then/before it's fixed, Wmb Asm won't be released till these bugs are released. The filenames it's displaying, it has a <null> tacked onto the end. And it's trying to open directory . , which it is supposed to be skipping.
And the only capture I have on this computer for vacation is the MP capture. The only transfer method I had was my file server, so I could only transfer one capture, due to slow Dial-Up and not much time.(Here on vacation I can use DSL)
Anyway, Filb, could you please re-upload these captures, wiidemocaptures, Namco, Sudoku, multi-game captures, BBA, and EUBA captures, please?(Send a PM with the URLs)
(I tried the URLs, but they were removed from the server)
(At the bare minimum, upload wiidemos and BBA)
#159073 - HyperHacker - Wed Jun 25, 2008 3:27 am
Can you not put your name in the copyright notice, and mark or list the pieces that were written by other people? (Isn't software licensing fun? -_-)
_________________
I'm a PSP hacker now, but I still <3 DS.
#159084 - yellowstar - Wed Jun 25, 2008 5:47 am
Sounds good HyperHacker. But before I release it, I need to fix these bugs.
Those bugs I mentioned in the previous post, that was minor. The Win32dirent code wasn't being compiled at all due to a mistake on my part. Now there's a much more serious bug: it's crashing now. On some malloc code.
Code: |
//DLLIMPORT is a define generated by wxDev-Cpp, not my idea to name it that
DLLIMPORT void ConvertEndian(void* input, void* output, int input_length)
{
if(machine_endian==ENDIAN_BIG)return;
unsigned char *in, *out;
in=(unsigned char*)malloc((size_t)input_length);//It never makes it past this point
out=(unsigned char*)malloc((size_t)input_length);
if(in==NULL || out==NULL)
{
printf("FATAL ERROR: FAILED TO ALLOCATE MEMORY FOR CONVERTENDIAN.\n");
system("PAUSE");
#ifndef NDS
exit(1);
#endif
}
...
}
|
It's an access violation. Now why is that happening...
EDIT:
Hmm... When I define DEBUG again this stops happening... When that define is defined, an log file written, otherwise no log file is used. What's going on here... Maybe I didn't enclose one block of code which writes to the log, thus causing a crash?(Yes, I got around to adding the functionality to enable/disable the log)
#159088 - yellowstar - Wed Jun 25, 2008 6:21 am
There was pieces of code without the DEBUG ifdef blocks around them, but that still didn't fix it. What sort-of fixed it was checking if DEBUG was already defined, and defining it if it's not already defined. So it's kind of working now. With DEBUG defined, it works. But without it defined, it crashes again.
#159114 - yellowstar - Wed Jun 25, 2008 6:35 pm
Previously IDA Pro had the compiler field set wrong. It was set to Visual C++, instead of GNU. After fixing those things, I get this from the debugger:
Code: |
Debugged application message: HEAP[wmb_asm.exe]: .
Debugged application message: Heap block at 003D2988 modified at 003D2991 past requested size of 1
.
wmb_asm.dll: Software breakpoint exception (0x7C901230)
|
#159212 - yellowstar - Fri Jun 27, 2008 4:34 am
It seems the packet it's crashing on, it's the second Arm9 data packet. There was was some code that didn't have the ifdef DEBUG code around them, but that still didn't help... I redirected the debug output to stdout, temporarily, commented out the ifdef for the debugging in the data packets, and discovered the above. This crashing with the log disabled, this must be why the DS beta was crashing.
#159959 - yellowstar - Tue Jul 08, 2008 9:19 pm
I have fixed all these bugs. This crash, it was caused by only setting a variable which contains the amount of memory to allocate for the saved_data buffer, when using the debug log. When it wasn't debugging, crash, due to allocating zero memory. Once I moved that code to the proper place, it stopped crashing. I found more bugs, but I have fixed them.
Once again licensing is stopping the release again.
dovoto wrote: |
I certainly would not put any effort into making your library compatible with a fundamentally broken license such as GPL.
|
I guess I need to find another license...
I'm going to investigate the MIT license. I can't use Google Code, and probably not any other open-source project hosting web site either, unless the project is under a license, public domain is not an option.(Remember, I want that SVN server feature)
(About that post, I'm thinking about the broken part, not the rest of that post, for this project at least.)
#159972 - yellowstar - Tue Jul 08, 2008 10:25 pm
This MIT license looks good. And there's no huge document to read, only a short page, unlike GPL. :-)
#160069 - yellowstar - Thu Jul 10, 2008 2:24 am
Okay, licensing is almost done. Here's the text at the top of every source code file, and in the Copying_license.txt file:
Code: |
/*
Wmb Asm and all software in the Wmb Asm package are licensed under the MIT license:
Copyright (c) 2008 yellowstar
Permission is hereby granted, free of charge, to any person obtaining a copy of this
software and associated documentation files (the ?Software?), to deal in the Software
without restriction, including without limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons
to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies
or substantial portions of the Software.
THE SOFTWARE IS PROVIDED ?AS IS?, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE
FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
*/
|
Credits are given in the ReadMe, and the code written/based on code written by other programmers, those are marked in the source code in comments.
Now, the last thing left to tackle, the DSPacketCapture license wording.
Here's the current wording. It already has a license, and thus a copyright holder, so how should I change this?
Code: |
// DSWifi Test Project - copyright Stephen Stair 2005
// Based on template project in ndslib by wntrmute
/******************************************************************************
DSWifi Lib and test materials are licenced under the MIT open source licence:
Copyright (c) 2005 Stephen Stair
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
of the Software, and to permit persons to whom the Software is furnished to do
so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
******************************************************************************/
//This program, written by yellowstar, is based on sgstair's wifi_lib_test program.
|
Should I do something similar to Wmb Asm? Change the copyright holder to yellowstar, and move the comment at the bottom to the top? And what should I do with the text at the top? Also, credit is given to sgstair in the ReadMe.
#160082 - HyperHacker - Thu Jul 10, 2008 7:18 am
It looks like you're on the right track, just put your name as the copyright holder and list what parts were written by other people. I'm no lawyer though.
_________________
I'm a PSP hacker now, but I still <3 DS.
#160116 - yellowstar - Thu Jul 10, 2008 4:38 pm
@HyperHacker:
Sounds good.
If weren't for even more bugs, well, Wmb Asm would have probably been released sometime today. But sadly, there's more bugs to deal with. Filb told me there's this one demo that's crashing. He said on hardware, and on no$gba, after the title screen, it crashes. I compared his binary which he attempted to assemble with his assembler and the old version of Wmb Asm, and my binary, and they are exactly the same. And they do the exact same thing, no crashes. But that might be because I'm not getting very far in the demo level... Unfortunately he didn't tell me what the errors were...
And there's another bug I found today. The directory scanning code isn't working correctly. When it tries to scan sub-directories, it fails. It says it "failed to open directory ." I know what directory . is, but it's not supposed to open that directory... I have code that's supposed to stop that, but...
#160118 - yellowstar - Thu Jul 10, 2008 5:13 pm
The licensing is done. Now I need to fix these bugs...
#160175 - yellowstar - Fri Jul 11, 2008 12:43 am
That directories bug has been fixed. That glitch with that one demo has been resolved. That demo is in English, but Filb's DS was set for another language. Once he set it to English, it worked.
I'm working on setting up the Wmb Asm Google Code project page. I forgot TortoiseSVN wasn't installed on this computer, so I have to download it again. Which will take an hour. )-: If all goes as planned, the Wmb Asm Google Code project page should be available and linked to, from here, in ~1 hour. And of course when that happens, the next version of Wmb Asm will be released also.
#160183 - yellowstar - Fri Jul 11, 2008 2:43 am
This is taking longer than I thought... I got SVN installed, but now I'm trying to get source code into SVN... It will probably be done either tonight or tomorrow.
#160186 - yellowstar - Fri Jul 11, 2008 3:07 am
Okay, SVN is just about done. I just need to add documentation for the new command-line options in the ReadMe, and possiblely other things, then commit that onto SVN, then upload the compiled binaries in the Download section. With Google Code, there is a Web SVN feature built-in.
#160216 - yellowstar - Sat Jul 12, 2008 12:19 am
No more release time estimations - it will be released when it is done.(I'm done with SVN troubles, now it's other things I need to finish.)
#160255 - yellowstar - Sat Jul 12, 2008 10:36 pm
Wmb Asm 2.0b has been released. Google Code project page.
There's flaws in the ReadMe files for the module download... Unfortunately, the only way to fix that is to upload another separate download, and I'm not doing that.(That will be fixed in the next release, however)
I thinking it would a very good idea to do something similar to Devkitpro:
Have setup instructions on a Wiki page, then have the ReadMes link to that. Much easier to maintain.
#160307 - yellowstar - Sun Jul 13, 2008 9:13 pm
Wmb Asm 2.0b R2 was released yesterday. Changes for this release was fixed Read Me files, and added the files that were missing.(The following is not in the r2 release)
I have found the fix to this decorated names problem in these modules. I needed to use this in the headers:(One has to be careful to not have standard includes between these)
Code: |
#ifdef __cplusplus
extern "C" {
#endif
//Put your prototypes and other things here
#ifdef __cplusplus
}
#endif
|
Now the function name used in the source code is used for the function name in the module. But now my module loading code isn't working...
#160317 - yellowstar - Sun Jul 13, 2008 11:47 pm
Apparently I made a mistake for the code for just one line for loading one function... I forgot the , for parameter separation. Apparently C/C++ combines two strings into one when there's something like this: "str1" "str2". And in between the two strings, it injects an underscore. ( _ )
With the way my function loading function is designed, no compiler errors were thrown, and my error buffer was empty, due to this bug.
#160395 - yellowstar - Mon Jul 14, 2008 11:52 pm
I'm working on the Wmb Asm SDK. This SDK is for developing client applications of the Wmb Asm Module, such as the command-line program, and also Packet Module Plugins. Those of you that read the SVN commit logs, and possiblely use the code in SVN, probably all ready know about this. These plugins enable the Wmb Asm Module to use many different protocols, when those plugins are written. The code which handles the WMB packets will be moved into a plugin. The Download Station code will be in a plugin. The Wmb Asm Module will use the plugins for handling packets, no built-in packet handling code will be built-in.(Protocol packets)
The files needing compiled will change dramatically. Right now, in Wmb Asm module, there is 9 source code files. When the SDK, and WMB plugin is done, the module will only have 2 source code files. 5 files will be moved into the SDK. Another file will be moved into the WMB plugin. The module compile time will be much faster. Compiling the SDK will take around the same as the module, but fortunately, after the first SDK version is done, the code probably won't be changed a whole lot.(That is, probably not enough for a whole re-compile, due to changing one important header.)
Other than one accidental Google Code command-line download link-click by me, only one person downloaded Wmb Asm. I'm thinking that person was running Linux, since he/she only downloaded the command-line. The command-line and module are useless without the other. Wmb Asm is only compiled for Windows, since I don't have Linux. But since it seems to be easy to do packet capture on Linux, a build of Wmb Asm is needed for Linux. Does the GNU compiler come with the "normal" distros of Linux? I've found that it is possible to compile for Linux on Windows, but it might be a while 'till I take the time to install the gnu cross-binutils for this...
#160402 - tepples - Tue Jul 15, 2008 3:17 am
yellowstar wrote: |
Wmb Asm is only compiled for Windows, since I don't have Linux. |
Do you have broadband and a CD burner? If so, you can has Puppy Linux.
Quote: |
But since it seems to be easy to do packet capture on Linux, a build of Wmb Asm is needed for Linux. Does the GNU compiler come with the "normal" distros of Linux? |
Yes. A GCC-based toolchain is part of the popular *Linux distributions' repositories.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#160432 - yellowstar - Tue Jul 15, 2008 7:20 pm
tepples wrote: |
yellowstar wrote: | Wmb Asm is only compiled for Windows, since I don't have Linux. |
Do you have broadband and a CD burner? If so, you can has Puppy Linux.
|
When I go on vacation I can use DSL. But every time I went there I kept forgetting about downloading Linux. )-: And besides, the desktop computer I use there, the CD burner might be broken. The CD burner software isn't detecting the the burner anyway.
So I could just add instructions in the Read Me for compiling Linux, when running Linux, I guess. But first the Linux Makefile needs to be created. Could somebody post examples, or links to examples of Linux Makefiles for compiling a command-line program, a shared library/module, and a statically linked library?
#160442 - yellowstar - Tue Jul 15, 2008 10:35 pm
I'm having problems with this SDK... It seems that I can't use the same variable in a statically linked library, and in a program. For example:
SDK:
Command-line:
Or:
Code: |
//In SDK:
extern int somevar;
//In Command-Line:
int somevar;
|
In my code there's a bunch of function pointers for Wmb Asm Module functions. I guess I need to figure out a work around...
#160446 - yellowstar - Wed Jul 16, 2008 12:50 am
The Wmb Asm Google Code Project Page has been updated.
Also, the tool used for checking/verifying the RSA-signature, it's URL is missing from the Read Me. The link is available on the Setup Wiki page. The URL will be available in the next release, in the Read Me. The fixed Read Me is available in SVN right now.
The source code in SVN most likely won't be updated until the source code is stable again.(This is SDK-related issues.)
#160449 - tepples - Wed Jul 16, 2008 2:01 am
yellowstar wrote: |
I'm having problems with this SDK... It seems that I can't use the same variable in a statically linked library, and in a program. |
What you are experiencing is called "namespace pollution".
Quote: |
In my code there's a bunch of function pointers for Wmb Asm Module functions. I guess I need to figure out a work around... |
In C, you can work around namespace pollution by prefixing names with the name of your library, or by putting global variables into a big global struct (which is good pre-OO design anyway, as it's one step away from the singleton). In C++, Java, and Python, the language provides a mechanism to declare namespaces (or "packages" in Java or "modules" in Python).
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#160450 - yellowstar - Wed Jul 16, 2008 2:18 am
These problematic variables are in one header. Both the library, and the program include this header. And thus there's variables with the exact same name in both the library and program.
What I'd like to do, is, I'd like to have a variable declaration in either the library or program, and use extern to access it in the other.
EDIT:
Quote: |
putting global variables into a big global struct...
|
That still wouldn't help, since the contained variables need to be accessed by both the library and program.
#160453 - tepples - Wed Jul 16, 2008 3:54 am
yellowstar wrote: |
Quote: |
putting global variables into a big global struct...
|
That still wouldn't help, since the contained variables need to be accessed by both the library and program. |
Then make the struct global, and use extern in the header so that the program can access it.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#160469 - yellowstar - Wed Jul 16, 2008 5:05 pm
Wouldn't the same thing that happened with the variables, happen again with that method?(Linker errors)
I'm using another method. With this, I get no compile time errors. I have a global struct,(just the declaraction)
which contains these function pointers. The Asm DLL loading code in the SDK takes this struct as an parameter, and puts the addresses of the module's functions into that struct. Now after loading the DLL, there's another function call. The code for this is in the SDK header - I'd like to place the code elsewhere, but the code has to be in the client's source code. Any ideas? This function takes the this struct, and copies it into the global function pointers. These function pointers are declared in the SDK header, but these global pointers can only be used by the client applications.
This method works great. Next, I need to get code add for debugging-related things. After that's done, the final steps will be to get the SDK integrated with the Wmb Asm Module, writing the plugin part of the SDK, and moving the WMB Packet Module to a plugin.
How does one write a Windows DLL in C?(For the plugin template in C)
#160475 - yellowstar - Wed Jul 16, 2008 6:18 pm
Could somebody please help with these Wmb Asm SDK DS compiling errors? The next version of Wmb Asm, and the SDK, will not be released until these errors are fixed, even if everything else is ready.(That is, if Wmb Asm and the SDK are ready for release, but these errors still exist, the next version will not be released until these are fixed.)
Code: |
In file included from c:/YellowstarConsoleStuff/wmb_asm/SVN/trunk/wmb_asm/SDK/source/..\include\wmb_asm_sdk.h:112,
from c:/YellowstarConsoleStuff/wmb_asm/SVN/trunk/wmb_asm/SDK/source/dirscan.c:24:
c:/YellowstarConsoleStuff/wmb_asm/SVN/trunk/wmb_asm/SDK/source/..\include\/../include/pcap.h:55: error: expected specifier-qualifier-list before 'bool'
|
Code: |
/*
Wmb Asm, the SDK, and all software in the Wmb Asm package are licensed under the MIT license:
Copyright (c) 2008 yellowstar
Permission is hereby granted, free of charge, to any person obtaining a copy of this
software and associated documentation files (the ?Software?), to deal in the Software
without restriction, including without limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons
to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies
or substantial portions of the Software.
THE SOFTWARE IS PROVIDED ?AS IS?, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE
FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
*/
#ifndef lib_pcap_h
#ifndef _H_CUSTOM_PCAP
#define _H_CUSTOM_PCAP
#define PCAP_VERSION_MAJOR 2
#define PCAP_VERSION_MINOR 4
#define PCAP_ERRBUF_SIZE 256
typedef unsigned char u_char;
typedef unsigned int u_int;
typedef struct spcap_file_header {
unsigned int magic;
unsigned short majorVersion;
unsigned short minorVersion;
unsigned int timeZoneOffset;
unsigned int timeStampAccuracy;
unsigned int snapshotLength;
unsigned int linkLayerType;
} __attribute__ ((__packed__)) pcap_file_header;
typedef struct spcap_pkthdr {
long tv_sec;
long tv_usec; /* time stamp */
unsigned int caplen; /* length of portion present */
unsigned int len; /* length this packet (off wire) */
} __attribute__ ((__packed__)) pcap_pkthdr;
typedef struct spcap_t
{
FILE *file;
bool swap;//<---Error here
pcap_pkthdr header;
unsigned char *pktdata;
char *error_buffer;
} __attribute__ ((__packed__)) pcap_t;
#ifdef __cplusplus
extern "C" {
#endif
pcap_t *pcap_open_offline(const char *filename, char *errbuf);
int pcap_next_ex(pcap_t *file, pcap_pkthdr **hdr, const u_char **pktdata);
void pcap_close(pcap_t *file);
char *pcap_geterr(pcap_t *file);
int GetPacketNumber();
int GetSnapshotLength();
#ifdef __cplusplus
}
#endif
#endif
#endif
|
I tried making that boolean the only variable in the struct, removing the packed attribute, and removing the typedef and the code at the end of the struct, but all of those failed.
#160516 - yellowstar - Thu Jul 17, 2008 3:48 am
Well, I got the SDK code for the most part done. The SDK has been integrated with the Command-Line, the Wmb Asm Module, and the built-in Wmb Packet Module has been moved to a plugin.
However, I'm having problems with this. When I run the command-line as normal, it crashes. And when I try to debug with wxDev-Cpp/gdb, the directory scanning code doesn't even find the WMB plugin. It appears to be not scanning the directory for the WMB plugin for some reason.
#160636 - yellowstar - Sat Jul 19, 2008 12:06 am
Directory scanning code is even worse now... The plugin is still detected when running normally, but when debugging, it doesn't even find any files in any directories at all, even when scanning just the WMB directory.
I temporarily edited the code so directory scanning would be temporarily disabled, and it would directly load the plugin. It's crashing. Access Violation. For some reason, the sdk_nds_data global variable in the SDK is reset to zero after it is initially set, so the program crashes. With this WMB plugin, there's just one SDK function used that uses the sdk_nds_data variable, and it crashes when that is called. When I disable that function, Wmb Asm doesn't crash, it assembles fine, and everything works fine. But, these bugs need fixed, in particular the directory scanning code.
#160637 - yellowstar - Sat Jul 19, 2008 12:19 am
Here's my directory scanning code.(dirscan)
I guess nobody knows of any other directory scanning code I could use?
dirscan.h
Code: |
/*
Wmb Asm, the SDK, and all software in the Wmb Asm package are licensed under the MIT license:
Copyright (c) 2008 yellowstar
Permission is hereby granted, free of charge, to any person obtaining a copy of this
software and associated documentation files (the ?Software?), to deal in the Software
without restriction, including without limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons
to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies
or substantial portions of the Software.
THE SOFTWARE IS PROVIDED ?AS IS?, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE
FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
*/
#ifndef _H_DIRSCAN
#define _H_DIRSCAN
#ifdef __cplusplus
extern "C" {
#endif
typedef struct TFILE_LIST
{
char *filename;
TFILE_LIST *next;
} __attribute__ ((__packed__)) FILE_LIST;
FILE_LIST *ScanDirectory(FILE_LIST *filelist, char *dirname, char *ext);
void FreeDirectory(FILE_LIST *filelist);
#ifdef __cplusplus
}
#endif
#endif
|
dirscan.c
Code: |
/*
Wmb Asm, the SDK, and all software in the Wmb Asm package are licensed under the MIT license:
Copyright (c) 2008 yellowstar
Permission is hereby granted, free of charge, to any person obtaining a copy of this
software and associated documentation files (the ?Software?), to deal in the Software
without restriction, including without limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons
to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies
or substantial portions of the Software.
THE SOFTWARE IS PROVIDED ?AS IS?, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE
FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
*/
#define BUILDING_SDK
#include "..\include\wmb_asm_sdk.h"
inline void AllocDir(FILE_LIST *files)
{
files->filename = (char*)malloc(256);
strcpy(files->filename,"");
files->next = (FILE_LIST*)malloc(sizeof(FILE_LIST));
memset(files->next,0,sizeof(FILE_LIST));
}
DLLIMPORT FILE_LIST *ScanDirectory(FILE_LIST *filelist, char *dirname, char *ext = NULL)
{
FILE_LIST *files=NULL;
FILE_LIST *cur_files=NULL;
DIR *dir = NULL;
dirent *ent = NULL;
char *DirName = (char*)malloc(256);
strcpy(DirName,dirname);
if(filelist!=NULL)
{
files = filelist;
while(files->next!=NULL)
{
files = files->next;
}
if(files->filename!=NULL && files->next!=NULL)files = files->next;
}
cur_files = files;
FILE *f = fopen(DirName,"rb");
if(f!=NULL)
{
printf("FOUND BA %s\n",DirName);
if(ext!=NULL)
{
if(strcmp((const char*)&ent->d_name[strlen(ent->d_name)-4],(const char*)ext)!=0)
{
return files;
}
}
if(cur_files==NULL)
{
cur_files = (FILE_LIST*)malloc(sizeof(FILE_LIST));
memset(cur_files,0,sizeof(FILE_LIST));
}
if(cur_files->filename==NULL)
AllocDir(cur_files);
fclose(f);
strcpy(cur_files->filename,DirName);
printf("FOUND %s\n",cur_files->filename);
return files;
}
dir=opendir(DirName);
if(dir==NULL)
{
return NULL;
}
else
{
printf("Scanning directory %s\n",DirName);
}
char c = DirName[strlen(DirName)];
if(c!='/' && c!='\\')
{
sprintf(DirName,"%s/",DirName);
}
char *str;
str = (char*)malloc(256);
strcpy(str,"");
int skip=0;
if(files==NULL)
{
if(files==NULL)
{
files = (FILE_LIST*)malloc(sizeof(FILE_LIST));
memset(files,0,sizeof(FILE_LIST));
}
AllocDir(files);
cur_files = files;
}
ent=(dirent*)1;
while(ent!=NULL)
{
ent = readdir(dir);
if(ent==NULL)break;
if(skip<2)
{
skip++;
continue;
}
else
{
sprintf(str,"%s%s",DirName,ent->d_name);
f = fopen(str,"rb");
if(f==NULL)
{
char *Str = (char*)malloc(256);
strcpy(Str,str);
if(ScanDirectory(cur_files,Str,ext)==NULL)
{
printf("Failed to open file or directory %s\n",Str);
}
else
{
while(cur_files->next!=NULL)
{
cur_files = cur_files->next;
}
if(cur_files->filename!=NULL && cur_files->next!=NULL)cur_files = cur_files->next;
}
free(Str);
}
else
{
fclose(f);
printf("FOUND A %s\n",ent->d_name);
if(ext!=NULL)
{
if(strcmp((const char*)&ent->d_name[strlen(ent->d_name)-4],(const char*)ext)!=0)
{
continue;
}
}
if(cur_files->filename==NULL)
AllocDir(cur_files);
strcpy(cur_files->filename,str);
printf("FOUND B %s dir %s\n",ent->d_name, cur_files->filename);
cur_files = cur_files->next;
}
}
}
closedir(dir);
free(str);
free(DirName);
return files;
}
DLLIMPORT void FreeDirectory(FILE_LIST *filelist)
{
FILE_LIST *files = filelist;
FILE_LIST *next_file = files;
files=next_file;
while(files!=NULL)
{
next_file = files->next;
free(files->filename);
free(files);
files = next_file;
}
}
|
#160716 - yellowstar - Mon Jul 21, 2008 1:30 am
It seems the SDK compiled in C isn't working correctly with everything else compiled in C++... When I compiled the SDK in C++, it didn't crash. This directory scanning code I found,(I asked for directory scanning code on gamedev forums)
well, it's not working correctly... ATM, the code still uses Win32 API code. Before this can be ported to dirent, this bug needs fixed. Does anybody have any idea how to get the C SDK to work correctly with the C++ plugins, applications, and the Wmb Asm Module?
#160726 - yellowstar - Mon Jul 21, 2008 4:14 am
Never mind, it's still crashing. Win32 directory scanning code is working... Somewhat better now... Now it's doing the exact same thing as my code: Fine when running normally, but it finds nothing when debugging.
#160730 - DensitY - Mon Jul 21, 2008 5:25 am
not sure if this will help
Win32 has some Api functions for getting directory and file listings
have a read up on FindFirstFile,FindNextFile,FindClose Windows API functions (MSDN, the functions themselfs are located in kernel32.dll).
you should be able to write an implementation that is sigificantly smaller and faster.
#160760 - yellowstar - Mon Jul 21, 2008 3:59 pm
Unforuntately that doesn't help much. )-: This code I discovered,(somebody sent me a link to it)
it's already written with the Win32 API. It needs to be ported to dirent so it is portable to Unix/Linux, DS, and other platforms also. But there's not much point in doing that when it doesn't work correctly as it is now.
#160774 - melw - Mon Jul 21, 2008 9:57 pm
Here's a basic (although not too clean, feel free to improve!) code example for reading directories and files on DS, Win32 and *nix target platforms. IIRC some parts are loaned from libnds/libfat examples. Been using this for a simple file browser while testing applications on different target platforms, worked well enough for that.
Code: |
// hacky multi-platform file browser experiment 2007
// fast made, but hey it works!
#ifdef TARGET_NDS
#include <fat.h>
#include <sys/dir.h>
#else
#include <sys/types.h>
#include <dirent.h>
#endif
#include <sys/stat.h>
#define MAX_FILES 255
char *currentDir;
char *fileNameList[MAX_FILES];
int fileType[MAX_FILES];
int numFiles;
#define TYPE_DIR 0
#define TYPE_FILE 1
void changeDirectory(int entry);
void getFileList(char *dirName);
#ifndef TARGET_NDS
DIR *dip;
struct dirent *dit;
#else
DIR_ITER* dir;
#endif
void initFile()
{
int j;
for(j=0;j<MAX_FILES;j++)
{
fileNameList[j] = (char*)malloc(255); memset(fileNameList[j], 0, 255);
fileType[j] = -1;
}
currentDir = (char*)malloc(255);
#ifndef TARGET_NDS
getFileList(".");
#else
if(PersonalData->name[0]) // don't init FAT in emulators
fatInitDefault();
getFileList("/"); // TODO root for now
#endif
}
void getFileList(char *dirName)
{
struct stat st;
int i=0;
#ifndef TARGET_NDS
if ((dip = opendir(dirName)) == NULL)
return;
while ((dit = readdir(dip)) != NULL)
{
stat(dit->d_name, &st);
char *loadFileName = dit->d_name;
#else
dir = diropen(".");
char loadFileName[256];
while ( dirnext(dir, loadFileName, &st) == 0 )
{
#endif
if(st.st_mode & S_IFDIR)
fileType[i] = TYPE_DIR;
else
fileType[i] = TYPE_FILE;
memset(fileNameList[i], 0, 255);
strncpy(fileNameList[i], loadFileName, strlen(loadFileName));
i++;
}
}
numFiles = i;
#ifndef TARGET_NDS
closedir(dip);
#endif
}
void changeDirectory(int entry)
{
strncpy(currentDir, fileNameList[entry], 255);
chdir(currentDir);
#ifdef TARGET_NDS
dir = diropen(".");
#endif
getFileList(".");
}
|
#160780 - yellowstar - Tue Jul 22, 2008 1:24 am
I found the cause of this directory bug: the . directory. When I try to scan the working directory, ( . ) running as normal works, but debugging fails. Now, when I use an absolute directory,(the actual working directory instead of .)
running as normal works, and debugging works. How do I get the absolute directory name, for the working/current directory?(With something unix/cross-platform portable.)
(This other method, it works with my original code.)
#160801 - yellowstar - Tue Jul 22, 2008 6:33 pm
I tried adding code so after the initial . scan, recursive scans of that directory would have the ./ at the beginning removed. But that still didn't fix this debugging bug...
#160812 - yellowstar - Tue Jul 22, 2008 10:59 pm
I tried getcwd, but my implemention for this still won't work with debugging... It's scanning only one directory in the directory it started in, recursively, but for the other directories in the first directory, well, they never get scanned when debugging.
#160816 - yellowstar - Tue Jul 22, 2008 11:27 pm
This directory scanning bug has been solved. Apparently, there's a bug in wxDev-Cpp/Dev-Cpp... My IDE was setting the working directory to the project's/DLL's working directory, not the command-line. Once I tried debugging the command-line, it worked. Now, time to debug this crash bug...
#160856 - yellowstar - Wed Jul 23, 2008 6:41 pm
The latest Wmb Asm source code has been committed to SVN. While this was mainly for backup reasons, anybody that wants to, feel free to check out the latest revision for attempting to help debug these bugs. For the rest of you, whom just want source code that works, check out only revison 39. Once these major bugs are fixed, go ahead and commit the latest revision.
#161037 - yellowstar - Sat Jul 26, 2008 3:43 pm
Yesterday I captured some Nintendo Channel protocol transfers. The protocol I'm referring is the protocol used for downloading the DS Demos from the server.
The list of DS Demos... When the DS Download service is started, with my capture setup, no Wii packets were captured. I guess it's downloaded earlier. When the Nintendo Channel boots up, it downloads data from a server. It connects to a server with hostname entu.wapp.wii.com. It then downloads this:
/0/US/en/wc24data.LZ.
I successfully downloaded http://entu.wapp.wii.com/0/US/en/wc24data.LZ, on PC, with Firefox, even without the user agent being set for Wii.(The directories in the URL depend on the the region you're in, most likely)
I think this file is compressed, hence the filename. At the start of the file, as seen in a hex editor, I see that there's the following: At the beginning there's the text WC24. 4 bytes after the WC24 text,(excluding any terminating null character)
there's a byte with value 1. After that, there's 4 bytes with all zeroes. After that, there's a byte with value 1. And following that is 34 null bytes. Then the real data begins, it appears. I think the demos list is inside this file somewhere. But we need to figure out how to decompress it, first.
As for the actual data transfer. It's encrypted. TLS 1.0 "Simple" is used. The server hostname is a248.e.akamai.net. Here's the client Cipher Specs in the ClientHello packet, from Wireshark:
Cipher Spec: TLS_RSA_WITH_AES_256_CBC_SHA (0x000035)
Cipher Spec: TLS_RSA_WITH_AES_128_CBC_SHA (0x00002f)
Cipher Spec: TLS_RSA_WITH_3DES_EDE_CBC_SHA (0x00000a)
Cipher Spec: TLS_RSA_WITH_DES_CBC_SHA (0x000009)
Cipher Spec: TLS_RSA_WITH_RC4_128_SHA (0x000005)
Cipher Spec: TLS_RSA_WITH_RC4_128_MD5 (0x000004)
Like Wikipedia says, the strongest cipher is used.(AES is the very same encryption used on Wii discs. That's inpossible to crack, unless some one would Reverse Enginner the Nintendo Channel, or dump the Wii's RAM after running the Nintendo Channel in DS Download Service after sending a demo.)
Which is the first one. RSA, with AES 256-bit CBC SHA. The transfer is not compressed. The field for this in the ServerHello packet is set to null. The RSA public key is sent in this packet. I see Item: 1 item (id-at-organizationalUnitName=GTE CyberTrust Solutions, Inc.) in one of the fields in the signedCerificate section. In the Certificate Polices Syntax section, I see this: http://www.public-trust.com/CPS/OmniRoot.html
In the DistrubutionPoints section, I also see this: uniformResourceIdentifier: http://www.public-trust.com/cgi-bin/CRL/2018/cdp.crl
With this Nintendo Channel protocol, and TLS, it seems that some messages are put in the same packet.(The previous packets did this, too)
In the ClentKeyExchange packet, in the second message, it's a Change Cipher Spec, meaning everything after this message will be encrypted. And the last message in this packet is a Encrypted Handshake Message. I don't know for sure if this is the preMasterKey, but it's possible.
After this, a packet with another Change Cipher Spec message is sent, and another encrypted message is sent.
After that, the actual demo transfer begins. In encrypted messages of course. I think the RSA signature is sent first in the demo transfer. For the first byte in the encrypted data, it's value is AC. I've seen that in the start of piratically every signature I've viewed in hex. But in this case, it's probably just a coincidence, since this data is encrypted.
I'll post links to captures of this later. I'll also add this information I found into the Nintendo Channel wiki page on the Wmb Asm Wiki. Unfortunately, only Project Members of Wmb Asm can edit the Wiki. I'm thinking about moving the DS Download Station Wiki page to NDS Tech Wiki, once the gbadev posts are wikified. This way anyone can edit the information. Also, I might move this Nintendo Channel information over to Wiibrew.
Last edited by yellowstar on Sat Jul 26, 2008 8:33 pm; edited 1 time in total
#161040 - yellowstar - Sat Jul 26, 2008 7:30 pm
I forgot to mention: Apparently when watching Nintendo Channel videos,(the intro video at least)
the first time a video is downloaded, it's streamed from the server, then, with the intro video anyway, it gets saved to the internal memory, and doesn't get streamed again. I have found that the videos are encrypted/downloaded the same way as the demos. Goto the Wmb Asm captures wiki page for the Nintendo Channel captures download. The startup capture, of when the Nintendo Channel starts up, is included in the Other capture. A video capture is in that download also. The other capture downloads are of DS Demos.
#161083 - yellowstar - Sun Jul 27, 2008 11:59 pm
Apparently what thought was the RSA public key, it's not RSA. It's this 256-bit CBC AES key I was trying to find. The next step is to find how the random key, and the preMasterKey/AES are combined into the master key. After that's done, I'll need to find how the random key is generated.
#161110 - pepsiman - Mon Jul 28, 2008 1:54 pm
yellowstar wrote: |
That's inpossible to crack, unless some one would Reverse Enginner the Nintendo Channel, or dump the Wii's RAM after running the Nintendo Channel in DS Download Service after sending a demo.) |
http://pastie.org/private/cup4mwvmdyyekvdaaxhsg
#161115 - yellowstar - Mon Jul 28, 2008 4:07 pm
That text you quoted, it's outdated. My understanding of TLS/SSL was lacking. The AES key it self is sent during the handshake process. However, there's a random key, and that's somehow combined with the AES key.
Well, at least I did something. I didn't even try to crack how the random key and AES key are combined yet. But people already cracked it. :-)
Have you cracked how the random key is generated?
If people would send me the cracked protocol information, I will add the information into the Wiki page. Along with credits of course.
EDIT:
That pastie wasn't posted by pepsiman, it was by bushing.(Debug output) AFAIK, nobody cracked the protocol yet. And, it seems that .lz file is encrypted.(I didn't discover that encryption on the .lz, marcan did)
#161636 - yellowstar - Fri Aug 08, 2008 5:49 pm
Wmb Asm Roadmap wrote: |
Roadmap
Wmb Asm Roadmap - list of features planned for future versions
This page is the Wmb Asm Roadmap - a list of features planned for future versions of Wmb Asm. Note that the future version numbers may change before the releases.
v2.2
* Packet module plugins - enables programmers to develop plugins. Plugins include WMB, DS Download Station, Nintendo Channel, ect. This feature is already implemented and available in SVN.
* File module plugins. Allows programmers to add more supported capture formats to Wmb Asm when there plugin is downloaded, and also file converters. Such as the Nintendo Channel .bin converter built into the Wmb Asm Command-Line in SVN.(File module plugin code has not been implemented yet, however.)
* More modes. Packet module plugins can be used to make the device the software is running on, become a host, a client, or an assembler, which it can already assemble. For example, the PC/DS could become a WMB host, and broadcast/transfer demos to DS systems. Also, the DS, could become a WMB client, like the DS Download Play function on every stock DS' firmware/startup menu. Additionally, the client could save the downloaded .nds to the flash card, optionally.(If I find a working solution for this - a good way to send/receive raw packets when being a host/client on DS)
* Support for DS Download Station protocol. At least the assembler anyway. Note that the Nintendo Channel plugin will remain in SVN, until the Nintendo Channel protocol is cracked. Unless the Nintendo Channel protocol is cracked by the time Wmb Asm v2.2 is ready, the Nintendo Channel plugin will not be included in the downloads for v2.2.
* Linux/unix based platforms builds possibly.
|
Development on the DS Download Station plugin has begun recently. However, there seems to be something broken in either the Wmb Asm Module, or the WMB module. The Station plugin isn't getting the packets it should be. No Association Response packets, no WMB beacons... Only data and ack packets.
#161645 - yellowstar - Fri Aug 08, 2008 11:58 pm
It seems the problem, is that the WMB module is interfering. The packets that the DLStation module needs are being sent to the WMB module. The DLStation assoc response packets are sent right after some WMB beacons, and at that time Wmb Asm is using the WMB module, not the DLStation module.
#161692 - yellowstar - Mon Aug 11, 2008 12:49 am
There's new(this happened a while ago)
DS Download Stations in Japan. It's called Nintendo Spot. It has a new/changed protocol. I've tried to crack the protocol, but all I've found is what seems to be menu packets, I haven't found where the actual demo download happens. I have tried to assemble the client, with no success. Wmb Asm mysteriously stops at one packet and exits with the "Press any key to continue" message. Now that I test it again, with my hack to force logging without the bugged global logging system, with WMB plugin, I get no debug output anymore... Now not even the Namco capture works anymore. I really should have done that SVN commit before these major changes... :-(
O_o No packets at all are making it to the plugins... Major bug.
A WMB client and host for DS, based on Wmb Asm code, is in the works. This is being done with a liblobby mod. It's very simple. Liblobby can already send raw packets via Send802_11. Receiving is done by hooking Handle802_11 on the Arm9. The PoC for this kind of works. Sending WMB beacons works, and receiving packets works. However, the hardware seems to be doing strange things. It's automatically handling and responding to packets the client sends, even though the software does nothing about responding to the client because of those packets. These are authentication, association req/response, and ping packets. This is difficult for me to debug since this seems to be the hardware's doing. It would be great it somebody could do the following: Do a capture on a computer, capturing WMB traffic, while running the PoC. First, a mode in which the PoC runs without handling these packets the client sends. The capture software would be stopped, and the capture saved. Then repeat, except this time the PoC would be run in a mode in which the software handles these packets. The goal behind this is to find out if the hardware automatically responds to these packets if the software doesn't do anything packet sending-wise, and if the hardware doesn't automatically respond if the software does handle these packets. If the hardware keeps interfering in both cases, it will most likely be impossible to make a host/client capable of the gameID feature, or even a homebrew DS DL/N Spot station.
#161693 - yellowstar - Mon Aug 11, 2008 2:23 am
I got WMB assembly working better now. The previous problem was that I forgot I commented out the line which calls the function to load the plugins. Now Wmb Asm is crashing. Due to attempting to allocate and write memory in the assembly function with -1024 length. And in the header almost everything is a bunch of zeroes...
#161702 - yellowstar - Mon Aug 11, 2008 6:53 pm
Wmb Asm doesn't crash anymore. At least with the "normal" WMB captures. Apparently, the Wmb Asm wasn't seeing the same memory/data for the plugins nds_data structs. This led it to think that the plugins wanted the binaries assembled and the .nds written, at the very beginning, when in fact, from the module's perspective, the data was not cleared/memset'd. I placed volatile in front of the Nds_data struct global variables, and now it doesn't crash, at least with the Namco capture. However, the icon is incomplete. Only part of it is in the .nds. And there's a bug with one return value, which causes the time it takes to assemble, to not be displayed.
#161705 - yellowstar - Mon Aug 11, 2008 8:59 pm
I got the assembly of the Nintendo Spot captures working better. It doesn't crash, or do anything abnormal, anymore. It seems all of the N Spot captures WMB transfers for the client, all of them are incomplete. Some captures have ~100-300 missed packets, the largest one, has 3 missed packets.
#161713 - yellowstar - Tue Aug 12, 2008 4:01 am
Wmb Asm can now assemble multiple protocols at once. This is necessary because the WMB plugin was interfering with DS Download Station plugin, back when only one protocol could be assembled at once. The DS Download Station client assembly is not working correctly anymore. It assembles it once. Then twice. And next it reports that it failed to find the header. It's not supposed to assemble the client multiple times, obviously. The demos are fine. This assembly time bug is breaking the directory assembly/multiple capture assembly feature. One function is returning an error, while it shouldn't be doing that. I had a printf right before the return 1 at the end of the function, and the text got printed. When one capture fails like this, Wmb Asm stops, and will not assemble any more captures.
#161725 - yellowstar - Tue Aug 12, 2008 2:22 pm
I got this Nintendo Spot client assembled. The banner is broken due to Wmb Asm bugs, however. I merged all 4 captures together into one capture. With the mergecap tool that comes with Wireshark. It's a command-line tool inside the same directory as Wireshark. I'm planning on adding a utility/integration with Wmb Asm, for a tool used to merge captures. When I ran the client in a emulator, a screen which had a number on it, appeared soon after I started the client. It looked like a wifi error code: 51099. US Nintendo Wifi Error Code Lookup wrote: |
The only access point within range of your Nintendo DS is not configured or is not compatible with the Nintendo Wi-Fi Connection (such as those found in many Hotspots). |
You'd think it would display the error for no access point in range, or something similar. But perhaps this error is shown when the host doesn't respond to the client in a specific timeframe? Of course in this case, there is no host, so no host would ever respond. But this client may not know there isn't any host.
EDIT:
Here's a translation of the Japanese the client displays:(I didn't translate it, someone else did)
Quote: |
Access Point could not be found. Approach to the Access Point and please try again.
|
I'm thinking that the Nintendo Spot client gets the demos from a server on Internet. There's already free wifi/access points in every game store in Japan, so it's very likely. It may not be the server Nintendo Channel uses - there's no .bin string in the client binary, and a server URL in the binary doesn't look anything like the Nintendo Channel server URL.
#161784 - yellowstar - Wed Aug 13, 2008 11:20 pm
A priorities system for plugins has been implemented. The plugins with the highest priority have there Handle802_11 functions called first. The lower priorities have there Handle802_11 functions called next, and so on. This was needed because the DS Download Station plugin will needed to check some WMB packets. If this priority system was not in place, the DLStation plugin would never get the needed WMB packets - only the WMB plugin would get these packets.
#161807 - yellowstar - Thu Aug 14, 2008 8:12 pm
The DLStation plugin can now detect packet requests. One interesting thing I discovered, was that the client sends out a bunch of request packets with byte(s) of the whole character-set in the payload, that is, numbers, upper/lower case letters, and more. These seem to be sent before the menu request packets. Not sure, but these may also be sent before the download requests.
#161867 - yellowstar - Fri Aug 15, 2008 10:27 pm
It seems DLStation packets are almost the same as WMB data packets. The "magic numbers", 06 02 01 00 are identical to WMB data packets. Unlike what Filb found eariler, the next two bytes are not the packet type. The byte after the magic numbers, is the size of the actual data, divided by two. The next byte is the packet type. When the type is 0x1f for data packets, the size field needs 32 added to the result of the previous operation done on it, to get the correct size. Following this, is two bytes for the data packet's sequence number. After this, is the data. After the data, there's two bytes for the clientID. The rest is unknown, except for the last 6 bytes. The last 4 bytes of each packet is the CRC32 checksum,(FCS)
if the 2 bytes preceding it has the values 02 00, as with WMB. Really, the main difference with DLStation packets, and WMB, is the removed flags byte. The rest is an added clientID after the data, and unknown data after that. It appears that the DLStation protocol still uses the WMB ping/pong packets. The DLStation multicast addresses were changed since WMB. Flow 00 is host-to-client packets. 03 seems to be the same as WMB. Flow 10 seems to be client-to-host packets. It seems flows 10 and 00 uses were swapped for DLStation.
I noticed an odd data sequence number in a menu packet, at the end of the menu transfer. The seq was ffff, size was 2 bytes, and the data was 77 00. The data appears to be random. This packet appears to be a end-of-menu packet, which seems to tell the client(s) that the menu has been successfully transferred, and there are no more menu packets.
#161878 - yellowstar - Sat Aug 16, 2008 6:43 pm
A couple days ago I investigated these "non-advert", sequence number 9, WMB beacons. The following is what I found.
When no data is contained in this beacon, the data_size is 48, otherwise
in one capture, it's 3 bytes long. Not sure if this is fixed, or has an
meaning, other than the fact that it actually contains data. For the first two bytes in the payload, it's 02 00. This may
be the number of connected clients' data in this payload, minus one, or a "magic number",
I don't know for sure.(See note at the end of this post)
Following this, this seems to be entries for each connected client DS. At the start of each entry,
there seems to be a byte, which seems to be the length of the user's UTF-16 user name. That is, the total characters in the
user name, multiplied by two. Next is a byte, which seems to be the user's favorite color, which is the same as in the user settings.
Following this, is the user name, in UTF-16 format. I'm not sure how multiple clients' data are stored, since I only have one capture with one client. I might obtain a capture with at least two clients, if someone tests the upcoming WMB client, whom has three DSes, one for the host, and two for being clients, and would send the capture to me for non-advert beacon analyzing. The client/host will be integrated with Wmb Asm. However, the DS version, which is what the client/host is being written for primarily, the SDK won't compile correctly, and the client can not exist, at least a compiled binary, until those errors are fixed.
#161887 - yellowstar - Sat Aug 16, 2008 11:23 pm
The DLStation plugin can now handle menu packets, and dump the menu data to a file. As Filb said before, the the menu data is compressed. I'm attempting to use this to decompress LZO data. And I'm using the decompression function from lzo1x.h - it's the only one that can decompress this without any LZO errors. But I'm having a lot of trouble getting a perfect decompressed menu. I'm chopping 17 bytes off the input length since my code seems to be not getting the exactly right menu size. Not chopping this data results in LZO decompression errors. I tried not decompressing the first 4 bytes at all. I tried chopping 16 bytes bytes off, instead of 17. All of these failed to produce a perfect menu. Jumping to offset 0x72C, the size of one menu item, lands 16 bytes after the first menu item. Jumping to offest 0x220, relative to both the start of the file, and offset 0x04, doesn't land at the start of the title of the item.
#161890 - yellowstar - Sun Aug 17, 2008 1:50 am
The Wmb Asm advert/banner code is really broken... I dumped the advert in the function which assembles and writes the .nds, and in the function which handles WMB data packets, after the code which triggers the assembly of the .nds.(Assembly happens only after the function returns)
The two dumps are exactly the same. I dumped the advert as soon as the beacon stage is completed, and the two dumps are different. I did another dump after some advert code in the beacon function, and the two dumps were still different, and the dump from the the wmb plugin changed since moving the dump code. This dump from this point in wmb code, and the dump at the end, they are different for some reason... I thought the cause of the broken banner bug was caused by the Wmb Asm Module not "seeing" the same data in the advert struct as the wmb plugin, but it's apparently something else...
#161898 - yellowstar - Sun Aug 17, 2008 7:37 pm
I found the cause of this bug with the DLStation menu. The dumped menu does not start with the first menu packet, seq 0, it starts with the data from seq 1, despite detecting the packet. The first menu packet is logged in the log file. Memory is being overwritten, or something like that, I guess...
#161900 - yellowstar - Sun Aug 17, 2008 10:39 pm
I got the raw uncompressed menu dumping working. This menu data collection code is basically the same as the WMB code used for about the same purpose. However, the WMB code is designed to ignore the first data packet, which is the .nds header. There was more bugs, but I have fixed those. The menu uses the same header as demos on the host's ds card. I successfully decompressed the menu with chishm's decompressor. I have not yet written code for this header - I'm working on it.
EDIT:
This LZO header code is done, and the DLStation plugin can now decompress the menu. The code is available in SVN, and credits to chishm were added to the ReadMes. No actual C/C++ structs from chishm were used, but the information on the header format was.
#161913 - yellowstar - Mon Aug 18, 2008 4:39 pm
In Japan, in every game store, there's access points available. These use WEP, and most likely need paid for to get the key. This are only intended for NDS - they are rigged to only allow NDS use. These Nintendo Spot clients, I'm thinking they use these access points. Passive cracking failed, due to a huge number of lacking WEP IVs. The number of needed IVs is way to high to crack it in a feasible about of time. Active cracking can't be done. Filb suggested that the clients might use a hard-coded WEP key, that all Japanese game store access points use. To get this key, I'd need to reverse engineer the client. However, IDA Pro is way to expensive. I'll need to write my own software from scratch. Hopefully some emulator's disassembly code could be used, but if the DS emulators' dasm cores are anything like the GC/Wii emulators dasm cores, I'll need to write Arm7/9 dasm code from scratch. It will be quite a while before I start on this.
#161927 - yellowstar - Tue Aug 19, 2008 1:11 am
It seems those DLStation menu request packets I found earlier, the ones when printed appeared as letters in the charset, those aren't DLStation packets. Those are WMB ack packets. Once I filtered those packets out, only the "normal" kind of request packets were detected, menu and download requests.
#161936 - tepples - Tue Aug 19, 2008 7:36 pm
yellowstar wrote: |
To get this key, I'd need to reverse engineer the client. However, IDA Pro is way to expensive. I'll need to write my own software from scratch. Hopefully some emulator's disassembly code could be used |
Would arm-eabi-objdump from devkitARM help?
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#161940 - yellowstar - Tue Aug 19, 2008 7:58 pm
tepples wrote: |
yellowstar wrote: | To get this key, I'd need to reverse engineer the client. However, IDA Pro is way to expensive. I'll need to write my own software from scratch. Hopefully some emulator's disassembly code could be used |
Would arm-eabi-objdump from devkitARM help? |
Code: |
arm-eabi-objdump -d -S DLSRV_NTRJ_01__41.nds >dump.txt
arm-eabi-objdump: DLSRV_NTRJ_01__41.nds: File format not recognized
|
How would I convert the Arm7/9 binaries to .elf?(Actually, the writing-my-own-disassembler idea originated with Nintendo Channel dasm - obj-dump would always crash, output ... with the converted .elf)
EDIT:
dsd looks like what I need to use to dasm this .nds.
EDIT2: How do I save command-line programs' output to a text file on Windows? The >dump.txt thing doesn't seem to work...(When running the software from cmd.exe)
EDIT3:
Using dsd.exe > dump.txt doesn't work either, and it doesn't help when specifying the full path.
EDIT4:
Apparently nothing is written to the text file when no parameters are passed to the program... I got it to work now. Would be nice if devkitPPC would have a program similar to dsd.
#161944 - yellowstar - Tue Aug 19, 2008 10:29 pm
The DLStation plugin now collects the data in an buffer, and attempts to decompress the data. But for some reason the plugin crashes... I dumped the compressed data, and the data is fine... Chishm's decompressor crashes too.
#161947 - yellowstar - Tue Aug 19, 2008 11:37 pm
Apparently more than the actual data is in this dump/buffer... For some reason, from the first byte of data, from that point on, everything after that till the end of the packet is copied into the buffer. The length seems to be correct...
#161949 - yellowstar - Wed Aug 20, 2008 12:40 am
I forced the length to be subtracted by 50 to removed the extra data. The extra data disappeared from the dump/buffer, but decompression still fails... I really need a compressed Big Brain Academy DLStation data file, for comparing it with mine... :\
#161950 - yellowstar - Wed Aug 20, 2008 4:31 am
I have fixed this broken DLStation data packet code. One issue was that the size of the compressed data my code found, without using the header for getting the size, didn't match the one from the header. Another was that packets were being missed. Decompression works correctly now. I had chishm's decompressor decompressor the data into an .nds, and it worked fine. However, I still need to write the code to do the assembly/writing of the .nds for this plugin. But for this plugin, I just need to write code to move/copy data around, and set arm binaries' start/end variables.
#161951 - yellowstar - Wed Aug 20, 2008 5:06 am
Wmb Asm now attempts to assemble and write the .nds for DLStation captures. However, this is broken. The .nds crashes on no$gba. Strangely, the banner is near perfect, minus a mistake on my part.(This mainly happened because American DLStations use UFT-8 in the menu, while an .nds needs UFT-16.) Fortunately, I have an assembled binary of this demo decompressed by chishm's decompressor, so I can find the problem faster, unlike the decompression bug.
#161961 - yellowstar - Wed Aug 20, 2008 7:41 pm
The DLStation plugin can now assemble DS Download Station captures flawlessly, at least with the capture I assembled. The banner bugs have been fixed, and the assembled binary works with no$gba. It seems the decompressed data sent over DLStation protocol, it's an .nds. All that needs to be done is collect the data packets, decompress them when all the packets are found, then write the decompressed data to an .nds file directly.
I tried decompressing the Nintendo Spot menu packets and dumping the decompressed data, but when doing that, the plugin crashes... I tried having the WMB plugin directly copy the advert data into the advert struct, instead of waiting to copy the data from an array into the advert struct. That didn't help with this banner bug either.
#161970 - yellowstar - Thu Aug 21, 2008 8:18 pm
I have finished the Wmb Asm wiki page for the DS Download Station protocol information.
#162002 - yellowstar - Sat Aug 23, 2008 12:17 am
The Wmb Asm SDK has been almost successfully compiled for DS. I asked on gamedev about these errors, and the solution to these errors were found quickly. However, there's one warning thrown when compiling: fat.h: No such file or directory. This header isn't really used in the SDK, and shouldn't cause trouble with the rest of Wmb Asm,(which I have not yet compiled, some changes need done to do that)
but this should be resolved anyway.
EDIT:
This warning has been solved/fixed. It no longer occurs. The Wmb Asm SDK can now be compiled flawlessly. It was fixed by editing the SDK Makefile to something similar to this.
#162071 - yellowstar - Sun Aug 24, 2008 9:28 pm
I have fixed the bug which caused the assembly time display code, and the directory scanning code, to break. The bug, was that there was a glitched if condition: if(!ReadCaptureLoop(...)==0) It supposed to be if(!ReadCaptureLoop(...)) This caused the command-line program to think that ReadCaptureLoop failed every time.
Yesterday I got Wmb Asm DS compiling correctly without errors.
#162075 - yellowstar - Sun Aug 24, 2008 11:55 pm
The WMB banner code is working better now. Now, some of the demos' banners have portions of other demos' banner's in there demos. I assembled 7 of the original demos. 4 had perfect banners, the other 3 didn't. E.x: Sudoku had parts of the banner from Lup Salad. Now the Nintendo Spot client won't assemble at all - it never makes it past the beacon stage. I tried adding the code to ignore the fake gameID again, but still no success... I got the banners working better, by removing code in the beacon handling code that forced the assembler continue to the next stage, when the first WMB beacon is found. If iirc, I put that code there to see if I could get the Nintendo Spot client assembled. It worked, but with a broken banner, and apparently I forgot about disabling it. And since then banner bugs were introduced in the background, and I never found out about these additional bugs till now... :\
EDIT:
I got the Nintendo Spot client's banner working correctly now.
EDIT2:
As mentioned in SVN commit logs a while ago, this banner bug has been fixed. However, the Nintendo Spot banner bug still exists. Go here for the list of of bugs in Wmb Asm SVN.
Last edited by yellowstar on Mon Sep 15, 2008 3:21 am; edited 1 time in total
#162446 - yellowstar - Wed Sep 03, 2008 3:06 am
How do I use a non-standard Makefile name/filename? I tried make -f Makefile.ds, make -fMakefile.ds, make --file=Makefile.ds, but all of those methods results in make complaining about Makefile missing - it should be using Makefile.ds, not Makefile.
#162520 - Sektor - Thu Sep 04, 2008 9:45 am
I think that is a bug with C:\devkitPro\msys\bin\make.exe
I tried compiling gnumake and that found test.ds but it didn't like the unix style paths, so it couldn't find ds_rules.
_________________
GTAMP.com/DS
#162561 - yellowstar - Thu Sep 04, 2008 11:42 pm
Apparently I have to use filename Makefile - Makefile_ds, and Makefileds failed...
#162868 - yellowstar - Mon Sep 15, 2008 3:14 am
I installed msys from MinGW, (I recently switched and updated from the MinGW included with Code::Blocks)
and when I attempted to use Makefiles with filenames Makefile and Makefile.ds, both compiled Wmb Asm correctly. I guess I'll need to add in the DS compiling instructions/notes that the msys included with devkitpro can't be used, or Makefile.ds must be renamed to Makefile...
#163806 - yellowstar - Sun Oct 12, 2008 7:00 pm
I added this to the wiki a while ago, but anyway: DS Station information Help from the community is needed in order to dump the DS Station software...
#164678 - cboomf_ - Thu Nov 13, 2008 2:30 pm
Is there any way i can submit caps ...
_________________
M3 Simply - 1GB SanDisk
Flashme'd DS
#164691 - yellowstar - Fri Nov 14, 2008 1:13 am
If you want a capture hosted on the Wmb Asm Google Code Project, upload your capture unto a file server temporarily, then send me the URL so I can upload the capture unto Google Code.
#164702 - cboomf_ - Fri Nov 14, 2008 6:30 pm
Gd 2 know ...
I have a few caps (most broken), if i get gd ones i will post, cheers tho
_________________
M3 Simply - 1GB SanDisk
Flashme'd DS
#171306 - yellowstar - Thu Nov 12, 2009 9:18 pm
ninchdl-listext has been released. This tool can directly access the Nintendo Channel servers, and dump the dl list to a text file with info and URLs. ninchdl-listext replaces wmb_asm, for assembling recent Nintendo Channel demos. Thus wmb_asm is now only useful for old NinCh captures, and assembling WMB binaries from DS games and download services' WMB binaries. http://code.google.com/p/wmb-asm/