gbadev.org forum archive

This is a read-only mirror of the content originally found on forum.gbadev.org (now offline), salvaged from Wayback machine copies. A new forum can be found here.

DS development > Can the arm7 play MIDI files?

#136413 - Lazy1 - Wed Aug 01, 2007 2:21 pm

I know moonshell can run timidity but that is on the arm9, is there enough speed and memory on the arm7 to do this?
I have recently found out that the guy who made the music for wolf3d has released several midis which sound MUCH better than the old adlib versions.

#136423 - Sunray - Wed Aug 01, 2007 4:13 pm

Well, most commercial GBA games use midi-like music. Running on an ARM7 at haft the speed.

#136432 - Lazy1 - Wed Aug 01, 2007 7:32 pm

I'm not sure that is the same thing though.

#136447 - tepples - Wed Aug 01, 2007 9:55 pm

Yes, it would be straightforward to write a MIDI interpreter and run it on the ARM7 if you have a suitably licensed sample bank.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#136453 - Lazy1 - Wed Aug 01, 2007 10:59 pm

tepples wrote:
Yes, it would be straightforward to write a MIDI interpreter and run it on the ARM7 if you have a suitably licensed sample bank.


I don't like the sound of that, or understand it.
Is more text going to keep me from using midi music?

#136454 - tepples - Wed Aug 01, 2007 11:14 pm

A sample bank is a set of wave files that represent the sounds that the DS is supposed to play when processing a NoteOn event in MIDI. For instance, you might have samples of notes being played on a piano, a bass guitar, a violin, etc. Most people do not both own and know how to play these instruments, so they rely on third-party sample banks.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#136458 - Lazy1 - Thu Aug 02, 2007 12:40 am

I see...
The trouble is that I have no idea about audio programming so writing a midi interpreter is way out of my skill level.
Does one already exist that will run on the arm7, and is there enough memory for all of the needed samples along with a running game?

#136466 - tepples - Thu Aug 02, 2007 2:12 am

Lazy1 wrote:
I see...
The trouble is that I have no idea about audio programming

It's not hard to learn, just as you might have learned graphics programming. There are 16 voices; each has a set of audio registers. You write the starting address of wave data, the length of the data, the pitch, and the volume, and the DS handles the rest. It's a lot like OAM, just out of the speakers.

Quote:
is there enough memory for all of the needed samples along with a running game?

If you load only those instruments and sound effects that your current scene uses, there should be no problem. Case in point: commercial DS games work.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#136469 - Lazy1 - Thu Aug 02, 2007 3:20 am

Ok, I see now...
But if I need 16 channels for MIDI, what will I do since the game needs two channels for sound effects.

Unless not all channels are used?

#136470 - tepples - Thu Aug 02, 2007 4:02 am

Lazy1 wrote:
But if I need 16 channels for MIDI

A lot of tunes use only 8 channels. One trick is to define an instrument that contains a chord, so that you can use 1 channel instead of 3. A lot of music from the 4-channel MOD era does this.

Quote:
what will I do since the game needs two channels for sound effects.

Kill the two quietest notes.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#136473 - zefrench - Thu Aug 02, 2007 4:32 am

So far the only suitable librairies I can find are:

WildMidi http://sourceforge.net/projects/wildmidi/

and libTiMidity http://sourceforge.net/projects/libtimidity/


Neither have been ported to Nintendo DS

Low quality alternative is MidiKey2Freq

MoonShell has a MIDI plugin, not usre how usable that would be (midrcp plugin source ).

Nice post of an ARM7 based mp3 player at
http://forum.gbadev.org/viewtopic.php?t=13448&postdays=0&postorder=asc&highlight=midi&start=15

Another alternative is using the hardware base sound compression using sound compressed using ima-adpcm (usallually does nto soudn so well on a desktop, but typically sound just fine on the DS tiny speaker)

http://nocash.emubase.de/gbatek.htm#dssound

#136503 - kusma - Thu Aug 02, 2007 12:19 pm

tepples wrote:
Quote:
what will I do since the game needs two channels for sound effects.

Kill the two quietest notes.

Uh, I'd probably try a bit better heuristic than that, you might end up killing notes that are early in the attack-part of it's envelope.
http://www.kebby.org/articles/fr08snd4.html has a slightly better approach in section 4.4.

#136562 - tepples - Thu Aug 02, 2007 9:22 pm

kusma wrote:
tepples wrote:
Quote:
what will I do since the game needs two channels for sound effects.

Kill the two quietest notes.

Uh, I'd probably try a bit better heuristic than that, you might end up killing notes that are early in the attack-part of it's envelope.

I hadn't thought about that. Most of the attacks in my music engines are part of the wave, not part of some external envelope.

Or you could use a music format like S3M or XM, which gives the music a fixed number of channels (e.g. 8) and has the composer perform voice allocation.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#136590 - Lazy1 - Fri Aug 03, 2007 12:52 am

tepples wrote:

Or you could use a music format like S3M or XM, which gives the music a fixed number of channels (e.g. 8) and has the composer perform voice allocation.


I am 100% sure that is not an option, any chance on timidity though?

#136591 - zefrench - Fri Aug 03, 2007 1:43 am

If you post the location of the midi files in questions, I can play them on a good MIDI player and then incode them in acpm and let you listen to them if you are interested,

#136594 - Lazy1 - Fri Aug 03, 2007 2:08 am

RAW Sounds are too large, especially when there are 27 tracks.

[Edit]
Fair enough, ADPCM seems to compress fairly well at 32768Hz.

I have the tools to convert imf to raw, what would I need to get a DS compatible ADPCM file and what is the best way to stream it from disk?

[Edit again]
Ok, two conversion utilities failed...

1) sox produces broken output on the ds
2) audacity forces the output to 44100Hz

I need a utility that can convert to adpcm, works on linux and can do batch jobs.

#136609 - thegamefreak0134 - Fri Aug 03, 2007 5:28 am

On your MIDI channel issue, I can't honestly think of any music that I have written that uses more than 16 notes at once. I also can't see the point in having something that does so, unless you need an echo effect, which you can fake once again by using an echoing sample to save channels.

Don't confuse the 16 channels of MIDI with the 16 channels on the DS. The channels on a standard midi controller can usually play more than one note at once, and in fact you are more limited in the number of instruments you can have going at once than you are in the number of notes. The DS is a different beast, in a way. Each channel is only capable (without mixing software, easy to write, but unnecessary in most cases) of playing one sample (note) at once. As such, when a MIDI interpreter comes across a chord in a MIDI channel, it should split it into 3 separate channels and use the first ones available. This means that a composer of music in this format would only have to be careful about not using more than 16 (or less, if you need SFX) notes at once, not necessarily more than 16 tracks.

Personally, my music often ends up having more than 16 "tracks" with shared channels, mainly because I can't handle percussion on only one staff when I compose in MIDI. Since the most standard midi file will split the notes up by track, you would have instances where you have more tracks than the DS has channels, even if not all of them are being used at the same time.

---

Lazy 1, you posted right after I hit reply. Odd... I just read your most recent post.

32768Hz seems like an awfully high sample rate for a DS game. Remember that your engine will likely send most of it's output to the DS's speakers. Although considerably better than other handheld's speakers, they still aren't that great, and a lower sample rate would hardly be noticed, I think, especially for certain instruments. Of course, if you must, you must.

I'm afraid I cannot help you terribly on the tools issue though, I haven't delved terribly into DS sound personally, I just understand it. I've only used the RAW format myself.

In any case, good luck. Oh, and members feel free to correct me, it's late and I'm sure I have made more than one error in my sleepiness. ^_^

-gamefreak
_________________
What if the hokey-pokey really is what it's all about?

[url=http:/www.darknovagames.com/index.php?action=recruit&clanid=1]Support Zeta on DarkNova![/url]

#136645 - Lazy1 - Fri Aug 03, 2007 3:07 pm

Ok, 22050Hz gives nice audio quality with a small file size. (just under 700KB for 57s)
I finally figured out how to get audacity to export at the right sample rate and using the ima2raw converter posted on here it plays perfectly on the DS.

All I need now is to implement streaming the music from disk and finding an application to convert the audio to adpcm from the command line.

[EDIT]
FFMPEG Seems to provide DS compatible IMA-ADPCM audio.

#136821 - Lazy1 - Sun Aug 05, 2007 5:59 pm

Sorry to bump but I cannot get streaming from disk to work, are there any existing examples of streaming raw adpcm audio?

#137188 - Lazy1 - Wed Aug 08, 2007 6:45 pm

Maybe it would help if I provided code?

ipcsound.h
Code:

#ifndef _IPCSOUND_H_
#define _IPCSOUND_H_

#define MusicChunkSize ( 65535 )

typedef struct {
   volatile int Arm9StartMusic;
   volatile int Arm9StopMusic;
   volatile int Arm9Busy;

   volatile int Arm7RequestBufferFill;
   volatile int Arm7Busy;

   volatile void* Buffer;
   volatile int Length;
   volatile int Rate;

   volatile int Segment;
} MusicCommand;

#define MusicControl ( *( ( volatile MusicCommand* ) IPC + sizeof( TransferRegion ) ) )

#endif


arm9/source/main.c
Code:

#include <nds.h>
#include <fat.h>
#include <stdio.h>
#include "../../ipcsound.h"

volatile char AudioBuffer[ MusicChunkSize * 2 ];
FILE* MusicFile = NULL;

void InitMusicSystem( void ) {
   MusicControl.Arm9StartMusic = 0;
   MusicControl.Arm9StopMusic = 0;
   MusicControl.Arm9Busy = 0;
   MusicControl.Arm7RequestBufferFill = 0;
   MusicControl.Arm7Busy = 0;
   MusicControl.Buffer = ( void* ) 0;
   MusicControl.Length = 0;
   MusicControl.Rate = 0;
   MusicControl.Segment = 0;
}

void UpdateMusicSystem( void ) {
   char* ReadPosition = NULL;
   int BytesRead = 0;

   if ( MusicControl.Arm7RequestBufferFill == 1 ) {
      MusicControl.Arm7RequestBufferFill = 0;
      MusicControl.Arm7Busy = 0;

      switch ( MusicControl.Segment ) {
         case 0:
            ReadPosition = ( char* ) &AudioBuffer[ 0 ];
            break;
         case 1:
            ReadPosition = ( char* ) &AudioBuffer[ MusicChunkSize ];
            break;
         default:
            break;
      };

      if ( feof( MusicFile ) )
         fseek( MusicFile, 0, SEEK_SET );

      if ( ( BytesRead = fread( ReadPosition, 1, MusicChunkSize, MusicFile ) ) < MusicChunkSize ) {
         ReadPosition+= BytesRead;

         fseek( MusicFile, 0, SEEK_SET );
         fread( ReadPosition, 1, MusicChunkSize - BytesRead, MusicFile );
      }

      printf( "ARM7: More data needed! Reading to segment %d\n", MusicControl.Segment );
   }
}

int main( void ) {
   powerON( POWER_ALL_2D | POWER_SWAP_LCDS );

   irqInit( );
   irqEnable( IRQ_VBLANK );

   InitMusicSystem( );
   consoleDemoInit( );

   fatInitDefault( );

   if ( ( MusicFile = fopen( "roster.adpcm.bin", "r" ) ) != NULL ) {
      fread( AudioBuffer, 1, MusicChunkSize * 2, MusicFile );
      printf( "It begins...\n" );

      MusicControl.Buffer = ( void* ) AudioBuffer;
      MusicControl.Length = MusicChunkSize * 2;
      MusicControl.Arm9StartMusic = 1;
   }

   while ( 1 ) {
      swiWaitForVBlank( );
      UpdateMusicSystem( );
   }

   return 0;
}


arm7/source/template.c
Code:

#include <nds.h>
#include <stdlib.h>
#include "../../ipcsound.h"

//---------------------------------------------------------------------------------
void startSound(int sampleRate, const void* data, u32 bytes, u8 channel, u8 vol,  u8 pan, u8 format) {
//---------------------------------------------------------------------------------
   SCHANNEL_TIMER(channel)  = SOUND_FREQ(sampleRate);
   SCHANNEL_SOURCE(channel) = (u32)data;
   SCHANNEL_LENGTH(channel) = bytes >> 2 ;
   SCHANNEL_CR(channel)     = SCHANNEL_ENABLE | SOUND_ONE_SHOT | SOUND_VOL(vol) | SOUND_PAN(pan) | (format==1?SOUND_8BIT:SOUND_16BIT);
}


//---------------------------------------------------------------------------------
s32 getFreeSoundChannel() {
//---------------------------------------------------------------------------------
   int i;
   for (i=1; i<16; i++) {
      if ( (SCHANNEL_CR(i) & SCHANNEL_ENABLE) == 0 ) return i;
   }
   return -1;
}

int vcount;
touchPosition first,tempPos;

//---------------------------------------------------------------------------------
void VcountHandler() {
//---------------------------------------------------------------------------------
   static int lastbut = -1;
   
   uint16 but=0, x=0, y=0, xpx=0, ypx=0, z1=0, z2=0;

   but = REG_KEYXY;

   if (!( (but ^ lastbut) & (1<<6))) {
 
      tempPos = touchReadXY();

      if ( tempPos.x == 0 || tempPos.y == 0 ) {
         but |= (1 <<6);
         lastbut = but;
      } else {
         x = tempPos.x;
         y = tempPos.y;
         xpx = tempPos.px;
         ypx = tempPos.py;
         z1 = tempPos.z1;
         z2 = tempPos.z2;
      }
      
   } else {
      lastbut = but;
      but |= (1 <<6);
   }

   if ( vcount == 80 ) {
      first = tempPos;
   } else {
      if (   abs( xpx - first.px) > 10 || abs( ypx - first.py) > 10 ||
            (but & ( 1<<6)) ) {

         but |= (1 <<6);
         lastbut = but;

      } else {    
         IPC->mailBusy = 1;
         IPC->touchX         = x;
         IPC->touchY         = y;
         IPC->touchXpx      = xpx;
         IPC->touchYpx      = ypx;
         IPC->touchZ1      = z1;
         IPC->touchZ2      = z2;
         IPC->mailBusy = 0;
      }
   }
   IPC->buttons      = but;
   vcount ^= (80 ^ 130);
   SetYtrigger(vcount);

}

//---------------------------------------------------------------------------------
void VblankHandler(void) {
//---------------------------------------------------------------------------------
   u32 i;


   //sound code  :)
   TransferSound *snd = IPC->soundData;
   IPC->soundData = 0;

   if (0 != snd) {

      for (i=0; i<snd->count; i++) {
         s32 chan = getFreeSoundChannel();

         if (chan >= 0) {
            startSound(snd->data[i].rate, snd->data[i].data, snd->data[i].len, chan, snd->data[i].vol, snd->data[i].pan, snd->data[i].format);
         }
      }
   }
}

void IRQTimer1( void ) {
   static int Segment = 0;

   switch ( Segment ) {
      case 0:
         // Fill the 1st segment
         MusicControl.Segment = 0;
         MusicControl.Arm7RequestBufferFill = 1;
         MusicControl.Arm7Busy = 1;
         break;
      case 1:
         // Fill the 2nd segment
         MusicControl.Segment = 1;
         MusicControl.Arm7RequestBufferFill = 1;
         MusicControl.Arm7Busy = 1;
         break;
      default:
         break;
   }

   SCHANNEL_LENGTH( 0 ) = MusicChunkSize >> 2;

   Segment = ! Segment;
}

//---------------------------------------------------------------------------------
int main(int argc, char ** argv) {
//---------------------------------------------------------------------------------
   MusicControl.Arm9StartMusic = 0;
   MusicControl.Arm9StopMusic = 0;
   MusicControl.Arm9Busy = 0;
   MusicControl.Arm7RequestBufferFill = 0;
   MusicControl.Arm7Busy = 0;
   MusicControl.Buffer = ( void* ) 0;
   MusicControl.Length = 0;
   MusicControl.Rate = 0;
   MusicControl.Segment = 0;

   // Reset the clock if needed
   rtcReset();

   //enable sound
   powerON(POWER_SOUND);
   SOUND_CR = SOUND_ENABLE | SOUND_VOL(0x7F);
   IPC->soundData = 0;

   irqInit();
   irqSet(IRQ_VBLANK, VblankHandler);
   SetYtrigger(80);
   vcount = 80;
   irqSet(IRQ_VCOUNT, VcountHandler);
   irqSet( IRQ_TIMER1, IRQTimer1 );
   irqEnable(IRQ_VBLANK | IRQ_VCOUNT | IRQ_TIMER1);

   // Keep the ARM7 idle
   while ( 1 ) {
      swiWaitForVBlank( );

      if ( MusicControl.Arm9StartMusic == 1 ) {
         SCHANNEL_TIMER( 0 ) = SOUND_FREQ( 22050 );
         SCHANNEL_SOURCE( 0 ) = ( u32 ) MusicControl.Buffer;
         SCHANNEL_LENGTH( 0 ) = ( u32 ) MusicControl.Length >> 2;
         SCHANNEL_REPEAT_POINT( 0 ) = ( u32 ) 0;
         SCHANNEL_CR( 0 ) = SCHANNEL_ENABLE | SOUND_VOL( 0x7F ) | SOUND_PAN( 63 ) | SOUND_REPEAT | SOUND_FORMAT_ADPCM;
   
         TIMER0_DATA = TIMER_FREQ( 22050 ) * 2;
         TIMER0_CR = TIMER_DIV_1 | TIMER_ENABLE;
   
         TIMER1_DATA = 0x10000 - 65535;
         TIMER1_CR = TIMER_CASCADE | TIMER_IRQ_REQ | TIMER_ENABLE;

         MusicControl.Arm9StartMusic = 0;
      }
   }
}


I obviously have no idea what I'm doing :)

#137423 - Diddl - Sat Aug 11, 2007 6:39 pm

I know you want to implement music with a sampled variant. The better way would be to play this midi data in the WL6 files. So no other files are required for playing wolf3d and no audio rights are broken.


I have found a IMF player source here: click

IMF is the music data format from wolfenstein WL3 and WL6 files. The format is very midi like but is special for this OPL chip within Adlib sound cards (fm synthese).


Additionally I found a Source for an OPL emulator: click

Both source codes would it do. We could play music directly like original Wolfenstein 3d on an old PC.

Maybe a source of an NES emulator would also help, cause a NES also had a fm synthese audio chip with OPL2 support.

#137431 - Lazy1 - Sat Aug 11, 2007 8:20 pm

The ARM7 does not have enough power to emulate the OPL2 chip, at least not without lots of optimizations that go way beyond my skill level.

There will be no licensing issues since the IMF utilities will be released as a separate package and require the user to do all of the converting.

RAW ADPCM audio works fine if it's loaded all into memory, however some songs are too large and must be streamed.
Unfortunately I still cannot get this to work.

#137437 - tepples - Sat Aug 11, 2007 9:04 pm

Lazy1 wrote:
The ARM7 does not have enough power to emulate the OPL2 chip, at least not without lots of optimizations that go way beyond my skill level.

Then go for high level emulation. Recognize which instruments they're trying to make, and match them up to sampled instruments taken from some synthesizer's sound bank.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#137447 - Lazy1 - Sat Aug 11, 2007 9:45 pm

That may be possible, but that may be for a later date.
In the meantime adpcm audio should be fine for music, but I have no idea why streaming is not working properly.

#137499 - Diddl - Sun Aug 12, 2007 6:00 pm

Streaming means, reading from SD card? I tought libfat also works for ARM7? In the libfat source are #ifdef ARM7 statements?

#137501 - Lazy1 - Sun Aug 12, 2007 6:49 pm

I need libfat on the arm9 only, the problem is not access to the filesystem but when to load audio into the buffer.

#137516 - Diddl - Sun Aug 12, 2007 9:33 pm

ARM9 could ARM7 give a path to a sound and ARM7 plays sound directly from this file?

#137518 - Lazy1 - Sun Aug 12, 2007 9:46 pm

Both CPUs cannot use libfat at the same time (correction?) and Wolfenstein needs the ARM9 to load resources from disk.
Even if I could get libfat on the ARM7 it would make no difference since the problem is knowing when to fill the buffer with audio.

I can think of two things that could be wrong:
1) Timing is waaay off/incorrect
2) The method I use is completely wrong

#137522 - DekuTree64 - Sun Aug 12, 2007 10:44 pm

Lazy1 wrote:
I can think of two things that could be wrong:
1) Timing is waaay off/incorrect
2) The method I use is completely wrong

Or 3) all of the above :)

After a quick look at the code, I can see a couple things wrong with the timing. First, the sound channel eats data 4 bytes at a time, so a chunk size of 65535 won't work. Second, this bit of code:
Code:
         TIMER0_DATA = TIMER_FREQ( 22050 ) * 2;
         TIMER0_CR = TIMER_DIV_1 | TIMER_ENABLE;
   
         TIMER1_DATA = 0x10000 - 65535;
         TIMER1_CR = TIMER_CASCADE | TIMER_IRQ_REQ | TIMER_ENABLE;

I think should be using SOUND_FREQ( 22050 ) * 2, to match timer0 exactly to the sound channel. Also, timer1's data should use the chunk size constant, because that 0x10000 - 65535 confused me at first :)

But even with that, it will not work. You can't stream ADPCM continuously, because when the channel loops back to the start of the buffer, it restores the ADPCM state from that time. So you basically have 2 options:
1) Split the source data into frames, and restart the sound channel for each frame.
2) Decode the chunk in software.

For the frame based method, the best I can think of is to write a little PC tool to run on the ADPCM file. Basically software-decode the whole thing, and poke in a word with the decoder state at the start of each frame in the compressed data. GBATek has the format of the decoder state word.
Then each time you refill the buffer and restart the sound channel, it will start up with whatever the decoder state should be at that point in the stream.
_________________
___________
The best optimization is to do nothing at all.
Therefore a fully optimized program doesn't exist.
-Deku

#137523 - Lazy1 - Sun Aug 12, 2007 11:23 pm

Thanks for all the information, I will try and write a software decoder and see how that goes :)

#137603 - zefrench - Mon Aug 13, 2007 10:31 pm

Maybe look at the source of Scummvm for DS? AgentQ implemented acpm music playback from files located on a dldi based fat system, might be worth looking at since I beleive source code is what you are after.

ScummVM DS port 0.10.0a is the latest

#137731 - Lazy1 - Wed Aug 15, 2007 3:11 am

Couldn't I stream the audio data in, but instead of setting the channel to repeat I track the song length and restart the process manually when it ends?

#138433 - zefrench - Fri Aug 24, 2007 3:32 am

Well I finally looked it up and here are the files I think you want to look at:


scummvm-0.10.0a\backends\platform\ds\arm9\source\cdaudio.cpp

and

scummvm-0.10.0a\backends\platform\ds\arm7\source\main.cpp


Search for "adpcm" in each file and you should find all the relevant functions.


Source is located at http://prdownloads.sourceforge.net/scummvm/scummvm-0.10.0a.tar.bz2?download

I hope this helps

#138437 - Lazy1 - Fri Aug 24, 2007 3:52 am

Thanks.
I was actually writing a proper adpcm decoder which supported ms ima-adpcm wav files oddly enough.
Though due to a recent mysterious source loss I have to re-add features missing since the last backup I did.