gbadev.org forum archive

Hi,
I'm doing a remake of Lode Runner: The Legend Returns for the DS, which works pretty well so far, and am now trying to build in the original music.
These are .mid files, and I don't know of a .mid player lib for the DS.
But as I use PALib for the project (it is likely to be my last project with PALib, however), there is a .mod player I could use which comes with PALib. I tried endless ways of converting the files to .mod, but it simply doesn't work right.
Then I found libmikmod, tried it with the files converted to .s3m or .it, but it had too much bugs.
My newest try is to convert the music to mp3 and play it using the Helix mp3 decoder.

Therefore I took a look at a ringbuffer example from this forum and tried to exchange the .raw music playback with mp3 playback, but I didn't succeed (I don't have much experience with audio programming ...).
This was the original code to fill the sound buffer in the example:

Code:

void SoundMixCallback(void *stream,u32 numsamples)
{
int i;

for(i = 0; i < numsamples; i++)
{
   if(soundoffset >= sizeof(snd_pirate)) soundoffset = 0;

   if(soundsystem->format == 8)
   {
      ((s8*)stream)[i] = snd_pirate[soundoffset++]; // normal sound 8bit ---> buffer 8bit
   }
   else
   {
      ((s16*)stream)[i] = (snd_pirate[soundoffset++] << 8); // sound 8bit ---> buffer 16bit
   }
}
}

I exchanged snd_pirate with snd_mp3, which holds the content of a mp3 file, and added the helix mp3dec files to the project. The initialization of the sound system is done like this:

Code:

SoundSystemInit(22050,(1 << 24) / 22050, 0, 16); // u32 rate,u32 buffersize,u8 channel,u8 format
SoundStartMixer();

if ((hMP3Decoder = MP3InitDecoder()) == 0)
{
   iprintf("Init mp3dec failed.");
   while(1)
      swiWaitForVBlank();
}
readPtr = snd_mp3;
bytesLeft = 957126;

And the new SoundMixCallback for testing purposes looks like this:

Code:

void SoundMixCallback(void *stream, u32 numsamples)
{
int i, err, offset;
short outBuf[MAX_NCHAN * MAX_NGRAN * MAX_NSAMP];

offset = MP3FindSyncWord(readPtr, bytesLeft);
if (offset < 0)
{
   outOfData = 1;
   iprintf("\x1b[13;0HFINISH");
   return;
}
readPtr += offset;
bytesLeft -= offset;

// decode one MP3 frame - if offset < 0 then bytesLeft was less than a full frame
err = MP3Decode(hMP3Decoder, &readPtr, &bytesLeft, outBuf, 0);
if (err)
{
   outOfData = err;
   iprintf("\x1b[13;0HERROR");
   return;
}
}

This doesn't change the buffer but simply decodes the mp3 file completely, and it works - after a while, "FINISH" is displayed.

Now to the strange part:
Filling the stream pointer causes the mp3 decoding to fail. It seems like the mp3 decoder uses the stream memory for something ... ?

For test purposes, I added the following to the function:

Code:

for (i = 0; i < numsamples; i++)
((s16*)stream)[i] = 2<<14;

With this change, it freezes instantly in no$gba.
On a real DS, it additionally displays that the error -6 occurred, which means ERR_MP3_INVALID_FRAMEHEADER.
If I comment out the mp3 decoding after this change, some short noise can be heard, which should be correct.

So for a strange reason, I can't fill the sound buffer while decoding the mp3 file.

Can anybody point me to what I'm doing wrong, or has anybody already written some code to use the Helix mp3 player and can post it here?
Thanks in advance!
_________________
<dsFont> <Game Collection>

Last edited by ThomasS on Thu Jun 28, 2007 6:10 pm; edited 2 times in total

ThomasS wrote:

I'm [cloning a PC game] for the DS, which works pretty well so far, and am now trying to build in the original music.
These are .mid files, and I don't know of a .mid player lib for the DS.

Then how does MoonShell play .mid? If the MoonShell MIDI player is too heavyweight for your purposes, can you record each instrument, and then interpret the standard MIDI file per the spec?
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

Quote:

Then how does MoonShell play .mid?

Moonshell plays the music fine, but I looked at the midrcp plugin source and am unsure if I could extract it and have no idea if it would be fast enough.

Quote:

If the MoonShell MIDI player is too heavyweight for your purposes, can you record each instrument, and then interpret the standard MIDI file per the spec?

If I had more experience with audio programming, I probably could do that.

But converting to mp3 and playing these files seems to be a good solution to me, if I only could get the playback to work ...

EDIT:
Works almost.

The first error was that the example I was using reserved too few memory and thus was overwriting the data after it which caused the mp3 decoding to fail.
The second problem was that no$gba doesn't emulate this program correctly - it freezes randomly.

Now it plays on a real DS, but it has some deep noise in it.
Perhaps one of you who has more experience with audio knows a possible reason?

This is the code I'm using to fill the ring buffer (tell me if the approach is generally wrong, I haven't done this before):

Code:

// Overview:
// If data is needed, the function first looks if the buffer "musicBuf" contains some buffered data and if yes, it fills it into the stream.
// If there is still more data needed, it decodes one mp3 frame and fills it into the stream
// If the mp3 frame contained more data than actually needed, the rest is copied to "musicBuf".

short musicBuf[MAX_NCHAN * MAX_NGRAN * MAX_NSAMP];
int nMusicBuf;

#define min(a, b) (((a) < (b)) ? (a) : (b))

void SoundMixCallback(void *stream, u32 numsamples)
{
int err, outSample, restSample, minSamples, offset;
short outBuf[MAX_NCHAN * MAX_NGRAN * MAX_NSAMP];
outSample = 0;

// Fill buffered data into stream
minSamples = min(nMusicBuf, numsamples);
if (minSamples > 0)
{
   // Copy data from musicBuf to stream
   numsamples -= minSamples;
   nMusicBuf -= minSamples;
   memcpy(((s16*)stream) + outSample, musicBuf, minSamples * sizeof(s16));
   outSample += minSamples;
}

// Still more data needed?
if (numsamples > 0)
{
nextFrame:;
   // More data is needed. Decode a mp3 frame to outBuf
   // Find start of next MP3 frame - assume EOF if no sync found
   offset = MP3FindSyncWord(readPtr, bytesLeft);
   if (offset < 0)
   {
      outOfData = 1;
      return;
   }
   readPtr += offset;
   bytesLeft -= offset;

   // Decode one MP3 frame
      err = MP3Decode(hMP3Decoder, &readPtr, &bytesLeft, outBuf, 0);
   if (err)
   {
      outOfData = err;
      return;
   }

   // Copy data to stream
   MP3GetLastFrameInfo(hMP3Decoder, &mp3FrameInfo);
   minSamples = min(mp3FrameInfo.outputSamps, numsamples);
   restSample = mp3FrameInfo.outputSamps - numsamples;
   numsamples -= minSamples;

   memcpy(((s16*)stream) + outSample, outBuf, minSamples * sizeof(s16));
   outSample += minSamples;

   // Is still more data needed? Then decode the next frame
   if (numsamples > 0)
      goto nextFrame;

   // Copy the rest of the decoded data to musicBuf
   // TODO: how about decoding directly to this location, copying only the played part to the stream and
   // setting a nMusicBufStart offset?
   if (restSample > 0)
   {
      memcpy(musicBuf, &outBuf[minSamples], restSample * sizeof(s16));
      nMusicBuf = restSample;
   }
}
}

_________________
<dsFont> <Game Collection>

Hi,

I'm really interested by integrating a mp3 player on my homebrew.
So I'm really interested by your work.

Did you success to compile it and to make it run finally ?

Regards

It compiles, but there is still some noise in the output. The biggest part of the noise seemed to be a timing problem, but some noise is still there and I have no idea why. Perhaps more experimenting with this will show the reason ...
_________________
<dsFont> <Game Collection>

Ok, now it works.
The only problem left is that it gets "error -6" at the end of the file instead of "out of data", but the whole song is played, so it's a minor problem here.

For everyone who wants to use the mp3 player, here is the download of my sample project - sorry for the upload site :-(

Note that the helix files are not included because I don't know if their license would allow it. Look at the ReadMe on what files you need (tip: it's probably faster to download all files individual through the "Browse CVS" feature than to follow the "quick"start here)
_________________
<dsFont> <Game Collection>

Mirror http://lickr.org/files/ThomasS_MP3PlaybackDemo_DS.7z

I believe you may add the Helix source code to your project, if your project is also licensed with one of these licenses.
_________________
http://licklick.wordpress.com

Nice work ThomasS !

I'll have a look at it asap.

What would it cost to make this run on arm7 ?

Thanks for mirroring the file, Lick.

The performance on arm7 and arm9 is described by Noda in this thread.

And here's the arm7 example.

It tried to license it under the zlib license so I can include the helix files. Please tell me if I did anything wrong - I haven't done this before.
_________________
<dsFont> <Game Collection>

please someone mirror this, i can't download from megaupload. Thanks

Arm9 playback example did not work for me with devkit r20. On real hardware i heard nothing. What is wrong with source code ?

Mirror of 2nd file http://lickr.org/files/ThomasS_MP3PlaybackDemo_Arm7_DS.7z

Good job!
_________________
http://licklick.wordpress.com

As I can't test it for now, could you please tell me if this is working fine on arm7 ? It would be really good.

Lick:
Thanks for mirroring again!

MaXXik:
Strange. Did you download and place the helix files correctly?
What does the program display: "playing", "out of data" or "decoding error"?
Does the arm7 example work for you?
Or perhaps the volume was set too low, as the music file is very quiet at the beginning.
_________________
<dsFont> <Game Collection>

ThomasS wrote:

MaXXik:
Strange. Did you download and place the helix files correctly?
What does the program display: "playing", "out of data" or "decoding error"?
Does the arm7 example work for you?
Or perhaps the volume was set too low, as the music file is very quiet at the beginning.

Yes, i download and place helix playback source code in arm9/helix folder, add -D__GNUC__ -DARM to arm9 makefile and comment two lines of code in file subband.c (//PolyphaseStereo(pcmBuf, sbi->vbuf + sbi->vindex + VBUF_LENGTH * (b & 0x01), polyCoef); //PolyphaseMono(pcmBuf, sbi->vbuf + sbi->vindex + VBUF_LENGTH * (b & 0x01), polyCoef); ). Then i compile mp3 playback using devkitARM r20 and start nds executable on NDS. It displays "status: playing", sound volume on console set to maximum, but i do not hear any sound.
Thats my source code including compiling nds executable.
http://www.megaupload.com/?d=V2Z2QFVM

You spelled "asmpoly_gcc.s" as "asmpoly_gcc.S". If you correct this, you can add the functions PolyphaseStereo / Mono which you commented out in subband.c again. Then it should work.

And yes, ARM should be defined somewhere. Sorry about that, I placed it directly in an helix header file and forgot to mention it ...
_________________
<dsFont> <Game Collection>

Wow, thank you ThomasS! When I uncomment PolyphaseStereo / Mono source code lines and build nds executable it work nice on hardware. And as for me i did not notice any sound glitches. Thanks again !

Hmm. I'm using your arm7 example, the sound stops about a minute into the song, and there is some background noise.

This happened initially, and continued after I edited the arm7 template to match my existing one. There have been no changes made to any of the helix files.

Idea on where to start? The sound is 64kbps mono CBR, so I can't imagine it's a speed issue.
_________________
FireSlash.net

Quote:

the sound stops about a minute into the song,

This is most likely because the byte count of the music file is entered directly in in soundcommon.h and you didn't adjust it:

Code:

#define mp3_bytes 529105

Quote:

and there is some background noise

Well, with the released version, I never had any background noise.
But the files with which I tested it had a very low bit rate (I think it was VBR from 8 to 40 kbps or so), so it could be a speed problem, or the buffer size is too small for longer decoding times ...
_________________
<dsFont> <Game Collection>

Hmm. Is there a way to remove that? I'm dealing with multiple files and defines don't jive too well with that :)

Also finding some files don't play. It seems to be at random, and sometime the same file converted to a smaller size no longer works. However, the state still shows as 1 (playing). Hmm.

EDIT:
It seems that files around 1.3mb or so (Yes, they're loading into memory correctly) have this problem, as well as some more exotic ABR/VBR configurations. The latter are probably just limiations in the code, the former might be memory limitations.
_________________
FireSlash.net

Quote:

Is there a way to remove that?

Of course.
In the example, the file is included using ".incbin" - I don't know how you can get the size of the file using this method.
But the default template makefiles from libnds offer the possibility to include data using "bin2o", this way you'll get the size automatically. Just create a folder called "data" in the arm9 directory, place your music files there (you don't need the *.s files anymore) and rename them to *.bin. When building the project, there should be some .h files generated in the "build" directory, containing variables to access the file data and the size of the files - #include these files and you can use the information.

Quote:

Also finding some files don't play.

The ~1.3MB files may be big, but if they load into memory correctly, they should play ... the decoder goes through them frame per frame and doesn't have to allocate much memory on the arm7 side. I'll test it tomorrow ...
_________________
<dsFont> <Game Collection>

ThomasS wrote:

Quote:

Is there a way to remove that?

Of course.

I'm loading via FAT, I have the file sizes, I just don't know where helix want's them :)

I could remove the define and add it to the struct, but I don't know what type it's looking for. I'll sort it out sooner or later, but if you already know it'll save me the trouble of digging through the source. :)
_________________
FireSlash.net

Quote:

I could remove the define and add it to the struct, but I don't know what type it's looking for.

What do you mean with type? It's simply the byte count, this could be stored, for example, in an u32.

Quote:

I just don't know where helix want's them :)

The size of the file is stored in the variable "bytesLeft" to know where the file ends.

So you could add something like u32 mp3Bytes; to the transfer struct and change
bytesLeft = mp3_bytes;
in sound7.c to
bytesLeft = soundsystem->mp3Bytes;
Then set the field in the arm9 part before playing the file and it should work.

EDIT:
I did some tests, like announced:
A 2MB 192kbps mp3 plays, but has some background noise with the default settings.
Changing the buffer size from 2 * MAX_NCHAN * MAX_NGRAN * MAX_NSAMP to 3 * MAX_NCHAN * MAX_NGRAN * MAX_NSAMP lets the noise disappear (caution: this value is in the code twice!).
Making the multiplier (2, 3) bigger than 3 results in too less memory being left for the mp3 decoder - the initialization will fail.
_________________
<dsFont> <Game Collection>

Any way to read file directly from FAT (without loading the full file into memory) ? I had "Port Helix decoder" on my todo list for ToD2 (Yes, I'll use MP3 as music), and loading a song to memory would be a waste.
ARM7 decoding really solves my speed concerns.
_________________
Private property.
Violators will be shot, survivors will be shot again.

You could take a look at the official example for the decoder in
[Helix Community] / datatype / mp3 / codec / fixpt / testwrap / main.c
It decodes a mp3 file, but doesn't hold the whole file in memory.
Instead, it has a small buffer which is always refilled if more data is needed.

This is likely to get difficult when decoding the file on the arm7, however, because libfat doesn't work on the arm7.

A possible solution:

simonjhall wrote:

I did it in the end by doing 9<->7 IPC, by wrapping fopen/fread/etc and passing the command through to the ARM9's IPC handler. It does slow down the main processor a bit (something I found out when streaming mp3) but it does mean you don't lose huge amounts of ARM7 storage by including all the file stuff in there.

_________________
<dsFont> <Game Collection>

gbadev.org forum archive

DS development > mp3 playback examples (was: Problems using the helix mp3dec)

#131502 - ThomasS - Sat Jun 16, 2007 11:51 am

#131523 - tepples - Sat Jun 16, 2007 9:12 pm

#131563 - ThomasS - Sun Jun 17, 2007 12:50 pm

#131727 - omalone - Tue Jun 19, 2007 2:50 pm

#131746 - ThomasS - Tue Jun 19, 2007 6:05 pm

#132184 - ThomasS - Sun Jun 24, 2007 11:06 am

#132199 - Lick - Sun Jun 24, 2007 4:15 pm

#132281 - omalone - Mon Jun 25, 2007 2:29 pm

#132430 - ThomasS - Tue Jun 26, 2007 4:56 pm

#132440 - DVSoftware - Tue Jun 26, 2007 6:57 pm

#132446 - MaXXik - Tue Jun 26, 2007 8:54 pm

#132447 - Lick - Tue Jun 26, 2007 9:07 pm

#132491 - omalone - Wed Jun 27, 2007 8:27 am

#132517 - ThomasS - Wed Jun 27, 2007 12:54 pm

#132545 - MaXXik - Wed Jun 27, 2007 7:14 pm

#132555 - ThomasS - Wed Jun 27, 2007 8:10 pm

#132561 - MaXXik - Wed Jun 27, 2007 9:03 pm

#132678 - FireSlash - Thu Jun 28, 2007 5:23 pm

#132680 - ThomasS - Thu Jun 28, 2007 6:07 pm

#132682 - FireSlash - Thu Jun 28, 2007 6:27 pm

#132696 - ThomasS - Thu Jun 28, 2007 7:26 pm

#132756 - FireSlash - Fri Jun 29, 2007 4:14 am

#132797 - ThomasS - Fri Jun 29, 2007 1:57 pm

#133587 - LiraNuna - Sat Jul 07, 2007 8:08 am

#133596 - ThomasS - Sat Jul 07, 2007 10:50 am