gbadev.org forum archive

This is a read-only mirror of the content originally found on forum.gbadev.org (now offline), salvaged from Wayback machine copies. A new forum can be found here.

DS development > I need advice to choose sound format for my game

#109633 - Lupi - Sun Nov 19, 2006 10:50 pm

My situation:

I'm doing a musical game, is a Elite Beat Agents/Osu! Tatakae! Ouendan clone. This means I want to load wav/raw/mp3 in my proyect, and the most important thing, I want people to upload their own songs in the specified format via the SD/CF card (this means reading data with the FAT libraries) and if posible via chunks of song.

The game must also be somehow sincronized with the secuence executing in the ARM9 (the circles you have to tap in the right moment), so the hardware must always be at 100% speed (this excludes the mp3 option I believe).

Considering that I have no idea of how to decode a compressed audio or a different format to be able to be played in the Nintendo DS, what is my best option for format.

Thanks in advance.

PD: I have tried http://www.wotsit.org/search.asp?s=music and tried to read RIFF WAVE format, but I don't get a word of what it says.

#109648 - tepples - Mon Nov 20, 2006 12:41 am

Lupi wrote:
I'm doing a musical game, is a Elite Beat Agents/Osu! Tatakae! Ouendan clone.

Stop. DDR is patented, and Konami has successfully sued someone over it. Is EBA patented?

Quote:
This means I want to load wav/raw/mp3 in my proyect, and the most important thing, I want people to upload their own songs in the specified format via the SD/CF card (this means reading data with the FAT libraries)

It also means using CF cards. The library that has full support for SD cards does not support reading a list of files on the card.

Quote:
The game must also be somehow sincronized with the secuence executing in the ARM9 (the circles you have to tap in the right moment), so the hardware must always be at 100% speed (this excludes the mp3 option I believe).

If you decode up to a second ahead (at 32000 Hz that's 125 KiB) into a circular buffer, you can always pretend the hardware is 100% speed.

Quote:
Considering that I have no idea of how to decode a compressed audio or a different format to be able to be played in the Nintendo DS, what is my best option for format.

If you want the absolute fastest compressed audio stream format to decode, consider porting my 8ad player.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#109655 - Lupi - Mon Nov 20, 2006 1:05 am

Quote:
Stop. DDR is patented, and Konami has successfully sued someone over it. Is EBA patented?


I'm not going to sell it or anything, it would be a free musical game with posibilities to add your own songs ;)

Quote:
It also means using CF cards. The library that has full support for SD cards does not support reading a list of files on the card.


I'm using rein gba_nds_fat libraries, and works well in my Supercard SD :S
I think it also has a few functions to get a filelist ready, if not I can find a different method

Quote:
If you decode up to a second ahead (at 32000 Hz that's 125 KiB) into a circular buffer, you can always pretend the hardware is 100% speed.


If you want the absolute fastest compressed audio stream format to decode, consider porting my 8ad player.


Well, I have seen your code, and considering I don't have much knowledge about GBA, I think I couldn't port it :P, I'll try to load directly from RAW for the moment and see how it progresses.

#109676 - Lick - Mon Nov 20, 2006 4:54 am

I've had it with those "your game plays like mine so I'll sue you" kind of bullshit. I rather see you develop a great game and receive a C&D letter from those people, at least you know that you're good enough to get sued. Then you can even write a blog about it and get famous! I mean, really famous!
_________________
http://licklick.wordpress.com

#109687 - Lupi - Mon Nov 20, 2006 9:37 am

I don't want to be famous, I want to make a good game, that's it...
:P

#109695 - silent_code - Mon Nov 20, 2006 1:01 pm

Lupi wrote:
I don't want to be famous, I want to make a good game, that's it...
:P


you're soooooo right! good games, that's what we need! :D

#109753 - Lupi - Tue Nov 21, 2006 12:13 am

Well, I have tried several settings to try to make it sound continuosly:
The raw I'm trying to play is a 22050Hz 16 bit PCM raw song
Code:

u32 PlaySong(FAT_FILE * SONG, u32 fileorigin){
   u16 * BUFFER = new u16 [22050];
   FAT_fseek (SONG,0,fileorigin);
   FAT_fread ((void *)BUFFER, sizeof(u16), 22050, SONG);
   TransferSoundData * sound = new TransferSoundData();
   sound->data = BUFFER;
   sound->len = 22050;
   sound->rate = 22050;
   sound->vol = 127;
   sound->pan = 64;
   sound->format = 0;
   playSound(sound);
   
   delete [] BUFFER;
   delete[]sound;
   return (FAT_ftell(SONG));
}


And here is how I use it in the Arm9 code:

Code:

BEFORE THE MAIN GAME LOOP...

FAT_FILE * SONG;
SONG = FAT_fopen ("cancion.raw", "r");
u32 position = 0;
u32 contador = 0;




INSIDE THE MAIN GAME LOOP...

contador++;
if (contador%60 == 0)
   position = PlaySong(SONG,position);



What I get with this is a strange sound, lets say this is the song:

|-----------------------------------------------------------|

Then it does play by chunks of song, one chunk is played, one isn't...

Play........|Not sounding..|Play........|Not sounding..
|-----------------------------------------------------------|

I actually can sing the song and follow it when is sounding and when its not...
what's wrong with this code?

Thanks in advance

#109756 - Lick - Tue Nov 21, 2006 12:32 am

I don't think you need to or should allocate memory every second.

It might be your delete[]s being called too soon. Use one TransferSoundData struct and allocate one array of 22050 shorts. Do this beforehand, NOT while you're mainlooping.
Each frame you would simply update the contents in the array. The TransferSoundData stays EXACTLY the same.
_________________
http://licklick.wordpress.com

#109770 - josath - Tue Nov 21, 2006 3:16 am

Lick is right about the memory allocation issues, but there's a more fundamental problem with your code: It will only work if your sound data will load instantly. Unfortunately, it takes some time to load (even if the time is very small).

Here is what is happening:

Code:
|--play second #1---||---load second #2---||----play second #2---||----load second #3----|

#109783 - tepples - Tue Nov 21, 2006 5:46 am

You need to keep two buffers: one that loads and one that plays.

I would also recommend against 22050 Hz for some technical reasons related to how the DS DAC (digital to analog converter) works. Better rates are 16384 Hz and 32768 Hz.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#109802 - Kir - Tue Nov 21, 2006 9:33 am

Would it be better to use ADPCM compressed music instead of raw ? Because if I remember correctly , DS already supports this compression format via hardware (at least Neimod's DSTek says so - http://neimod.com/dstek/ ). And Ouendan/EBA songs are encoded in ADPCM.

#109825 - tepples - Tue Nov 21, 2006 2:54 pm

The problem with decoding ADPCM in hardware is that the codec is stateful, and it's hard to start one packet precisely when the previous packet ends.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#109981 - Lupi - Thu Nov 23, 2006 10:51 pm

josath wrote:
Lick is right about the memory allocation issues, but there's a more fundamental problem with your code: It will only work if your sound data will load instantly. Unfortunately, it takes some time to load (even if the time is very small).

Here is what is happening:

Code:
|--play second #1---||---load second #2---||----play second #2---||----load second #3----|


Actually, that's not happening, what is happening is this:
Code:
|--play second #1---||---load second #2??---||----play second #[b]3[/b]---||----load second #4??----||--play second #[b]5[/b]---||---load second #6??---||----play second #7---||----load second #8??----|


And the game doesn't stop, since I can see my sprites doing what they have to do and no stops, I think it may be some problem with the channel I'm trying to load and play the sound being busy, but I have no idea of how to fix this.

#109984 - ProblemBaby - Fri Nov 24, 2006 12:23 am

Just a question about game play, have you thought anything about tempo/beat and how to generate an actual "beat-track" (dont know what to call it) for a song? Letting the user have to find out the exact BPM by himself sounds like pain, and consider changed is in thempo makes it even harder.

Not to spoil your idea, but wouldnt it better to make a game based on modules if you are going to let people load their own songs?

Maybe you have a good idea how to solve this that I couldnt think about;)

#109985 - tepples - Fri Nov 24, 2006 12:27 am

ProblemBaby wrote:
Just a question about game play, have you thought anything about tempo/beat and how to generate an actual "beat-track" (dont know what to call it) for a song? Letting the user have to find out the exact BPM by himself sounds like pain, and consider changed is in thempo makes it even harder.

Yet StepMania users on bemanistyle.com manage to figure it out.

Quote:
Not to spoil your idea, but wouldnt it better to make a game based on modules if you are going to let people load their own songs?

And watch people ask how to import their CD rips. I tried supporting MOD, XM, S3M, and IT background music in a PC game that I made, yet I still got repeated requests for some sort of straight recording.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#109987 - ProblemBaby - Fri Nov 24, 2006 12:43 am

Quote:

Yet StepMania users on bemanistyle.com manage to figure it out.

You mean find out BPM from a stream? well I guess that scanning through for peaks is thw way, but still sounds hard to make it accurate and not letting the system be tricked by something.

Code:

And watch people ask how to import their CD rips. I tried supporting MOD, XM, S3M, and IT background music in a PC game that I made, yet I still got repeated requests for some sort of straight recording.

well you know how people are "wouldnt it be better if the in-game graphics looked like the prerendered movies." ^^

Its boring to know that you will never know if the song will work good, and probably most songs will be somehow inaccurate. an alternative that would be really fun though, would be to be able to load nsfs, that would be so cool!

Last edited by ProblemBaby on Fri Nov 24, 2006 12:52 am; edited 1 time in total

#109988 - tepples - Fri Nov 24, 2006 12:52 am

ProblemBaby wrote:
Quote:
Yet StepMania users on bemanistyle.com manage to figure it out.

You mean find out BPM from a stream? well I guess that scanning through for peaks is thw way, but still sounds hard to make it accurate and not letting the system be tricked by something.

No, StepMania uses BPM change commands in the .sm file. A lot of people on Bemanistyle use times2bpm, a tool I wrote to convert a list of times (one every 8 beats) into a list of BPM changes.

Quote:
Quote:
And watch people ask how to import their CD rips. I tried supporting MOD, XM, S3M, and IT background music in a PC game that I made, yet I still got repeated requests for some sort of straight recording.

well you know how people are "wouldnt it be better if the in-game graphics looked like the prerendered movies." ^^

Except Rare actually took people up on that offer in Donkey Kong Country by using prerendered movies to create sprite cels.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#109989 - ProblemBaby - Fri Nov 24, 2006 1:09 am

Quote:

No, StepMania uses BPM change commands in the .sm file. A lot of people on Bemanistyle use times2bpm, a tool I wrote to convert a list of times (one every 8 beats) into a list of BPM changes.

My point is, that it is hard to find the times (or BPM) for a song in the first place, well if you havent record it yourself.
A normal user dont want to fiddle with finding out tempos or time they just want it to work.
Maybe we talk about different things?

#109998 - tepples - Fri Nov 24, 2006 3:56 am

ProblemBaby wrote:
A normal user dont want to fiddle with finding out tempos or time they just want it to work.

And in the typical use of a configurable rhythm game, the pirate providing the song has already done the work so that all the normal users can benefit.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#110147 - bsder - Sun Nov 26, 2006 12:08 am

Back to the original question, if I were selecting a music format, I'd use Ogg Vorbis. I believe that Ogg Vorbis has the capacity for fairly exact timestamping. Also, Ogg Vorbis takes less MIPS to decode (at the cost of some RAM and a larger initial startup-a good tradeoff on the DS).

With Tremor and DSWifi, I can stream a 64K average Ogg Stream and play it back at 22050 and still have some idle time left over.

I actually ported the Tremor library to the DS. If you have the ability to take a Mercurial repository, I could pack it up for you.

This doesn't solve the circular buffering problem, though. You have to start a free running interrupt timer on the ARM7 which is synchronized to the sound timer (unless someone has found a sound interrupt). The interrupt timer is used to signal the ARM9 when the samples are played and the ARM7 needs more data. I use the SYNC system to communicate back sound status.

Here is what I used on the ARM7. I set up two circular buffers in uncached memory space at 0x2600000 (left channel) and 0x2680000 (right channel).

WARNING: This code causes the libnds IPC system to break (it doesn't sample the touch screen, buttons, etc.). You have been warned!

Code:

#include <nds.h>

#include <dswifi7.h>

#define PURE_TIMER_CASCADE (1<<2)

#define PURE_TIMER_DIV_1 0
#define PURE_TIMER_DIV_64 1
#define PURE_TIMER_DIV_256 2
#define PURE_TIMER_DIV_1024 3

// Note: this is close to 0x2000000 which libnds uses (<.1% error)
#define BUS_CLK_HZ 33513982

// The sound timer ticks at 1/2 the bus clock rate.
// The sound timer counts *UP* from the SCHANNEL_TIMER value.
// When the value reaches 0, the sound system outputs the sample and reloads.
// Therefore, the value for SCHANNEL_TIMER should be 0 - ticks.
#define SOUND_HZ_TO_BUS_TICKS(f) BUS_CLK_HZ/((f))
#define SOUND_HZ_TO_SOUND_TICKS(f) SOUND_HZ_TO_BUS_TICKS((f))/2
#define SOUND_HZ_TO_TIMER(f) 0-SOUND_HZ_TO_BUS_TICKS((f))
#define SOUND_HZ_TO_SCHANNEL_TIMER(f) 0-SOUND_HZ_TO_SOUND_TICKS((f))

#define SOUND_SAMPLE_RATE 22050

const u16 soundSampleRate = SOUND_SAMPLE_RATE;
const u16 soundTimer = SOUND_HZ_TO_TIMER(SOUND_SAMPLE_RATE);
const u16 soundChannelTimer = SOUND_HZ_TO_SCHANNEL_TIMER(SOUND_SAMPLE_RATE);

#define SAMPLES_PER_MEMORY_FRAME 32768
#define FRAME_TIMER -(SAMPLES_PER_MEMORY_FRAME)

const u16 samplesPerMemoryFrame = SAMPLES_PER_MEMORY_FRAME;
const u16 frameTimer = FRAME_TIMER;

// const u16 samplesPerMemoryFrameScaler = 1024;
// const u16 soundTimerBusTicksPerMemoryFrame =
//                             soundTimerBusTicksPerSample *
//                             (samplesPerMemoryFrame / samplesPerMemoryFrameScaler) ;
// const u16 timer2BusTicksPerMemoryFrameValue = (u16)(-soundTimerBusTicksPerMemoryFrame);

volatile u32 soundMemoryFrameCount = 0;

void initSoundAndTimers() {
  // Initialize the sound system
  SOUND_CR = SOUND_ENABLE | SOUND_VOL(0x7f);

  u8* leftStart  = (u8*) 0x02600000;
  u8* rightStart = (u8*) 0x02680000;

  u8* leftEnd  = (u8*) 0x02680000;
  u8* rightEnd = (u8*) 0x02700000;
   

  SCHANNEL_CR(0) = 0;
  SCHANNEL_TIMER(0) = soundChannelTimer;
  SCHANNEL_SOURCE(0) = (u32)leftStart;
  SCHANNEL_LENGTH(0) = (u32)(leftEnd - leftStart) >> 2;
  SCHANNEL_REPEAT_POINT(0) = 0;

  SCHANNEL_CR(1) = 0;
  SCHANNEL_TIMER(1) = soundChannelTimer;
  SCHANNEL_SOURCE(1) = (u32)rightStart;
  SCHANNEL_LENGTH(1) = (u32)(rightEnd - rightStart) >> 2;
  SCHANNEL_REPEAT_POINT(1) = 0;

  // Initialize timers but do not start

  // Set up timer 1 to count up samples
  TIMER1_CR = 0;
  // FIXME: Why the +1?  Supposedly, the sound timer runs at 1/2
  // the bus frequency (which drives the normal timers).  Empirically,
  // this causes the sound to slip relative to the timer.  The error
  // is cancelled out by the +1?  Where does it come from?  I don't
  // know.  Possibly there is a 1 bus cycle delay as the timer reloads a
  // data value.
  TIMER1_DATA = (soundTimer+1);

  // Set up timer 2 to count up "memory frames"
  TIMER2_CR = 0;
  TIMER2_DATA = frameTimer;
  TIMER2_CR = TIMER_IRQ_REQ | PURE_TIMER_CASCADE;

  // Set up timer 3 to count overflows from timer 2
  TIMER3_CR = 0;
  TIMER3_DATA = 0;
  TIMER3_CR = PURE_TIMER_CASCADE;

  soundMemoryFrameCount = 0;
}


void irqAcknowledge(u32 irqBits) {
  REG_IF = irqBits;
}

#define START_SOUND 1
#define END_SOUND 2

const int startSound = START_SOUND;
const int endSound = END_SOUND;

void ihIPC(void) {
  u32 syncCommand = IPC_GetSync();

  switch(syncCommand) {
  case START_SOUND:
    // Start other timers--not time critical as it is a cascade
    TIMER3_CR |= TIMER_ENABLE;
    TIMER2_CR |= TIMER_ENABLE;

    // Might want to actually turn off interrupts so that this
    // all occurs as close to simultaneously as possible

    TIMER1_CR = TIMER_ENABLE;

    SCHANNEL_CR(0) = SCHANNEL_ENABLE | SOUND_REPEAT |
      SOUND_VOL(0x7f) | SOUND_PAN(0) | SOUND_16BIT;
    SCHANNEL_CR(1) = SCHANNEL_ENABLE | SOUND_REPEAT |
      SOUND_VOL(0x7f) | SOUND_PAN(0x7f) | SOUND_16BIT;
 
    break;
  case END_SOUND:
    TIMER1_CR = 0;
    SCHANNEL_CR(0) = 0;
    SCHANNEL_CR(1) = 0;

    TIMER3_CR &= ~TIMER_ENABLE;
    TIMER2_CR &= ~TIMER_ENABLE;

    // Reset system for next start command
    initSoundAndTimers();

    break;
  }


  // Flag the fact that we have handled the interrupt
  irqAcknowledge(IRQ_IPC_SYNC); 
}

void ihTimer2(void) {
  // Use counter rather than burn an entire timer
  IPC_SendSync(soundMemoryFrameCount & 0x7);

  ++soundMemoryFrameCount;
  irqAcknowledge(IRQ_TIMER2);
}

void ihVBlank(void) {
  Wifi_Update(); // update wireless in vblank

  irqAcknowledge(IRQ_VBLANK);
}

// ****************WIFI STUFF****************
// callback to allow wifi library to notify arm9
void arm7_synctoarm9() {
  // send fifo message
  REG_IPC_FIFO_TX = 0x87654321;
}

// interrupt handler to allow incoming notifications from arm9
void arm7_fifo() { // check incoming fifo messages
  u32 msg = REG_IPC_FIFO_RX;
  if(msg==0x87654321) {
    Wifi_Sync();
  }
}

void fifoInit() {
  // enable & prepare fifo asap
  REG_IPC_FIFO_CR = IPC_FIFO_ENABLE | IPC_FIFO_SEND_CLEAR;
}

void wifiInit() {
  // sync with arm9 and init wifi
  u32 fifo_temp;   

  while(1) { // wait for magic number
    while(REG_IPC_FIFO_CR&IPC_FIFO_RECV_EMPTY) {
      swiWaitForVBlank();
    }

    fifo_temp=REG_IPC_FIFO_RX;
    if(fifo_temp==0x12345678) break;
  }

  while(REG_IPC_FIFO_CR&IPC_FIFO_RECV_EMPTY) {
    swiWaitForVBlank();
  }

  fifo_temp=REG_IPC_FIFO_RX; // give next value to wifi_init
  Wifi_Init(fifo_temp);
 
  irqSet(IRQ_FIFO_NOT_EMPTY, arm7_fifo); // set up fifo irq
  irqEnable(IRQ_FIFO_NOT_EMPTY);
  REG_IPC_FIFO_CR = IPC_FIFO_ENABLE | IPC_FIFO_RECV_IRQ;
 
  Wifi_SetSyncHandler(arm7_synctoarm9); // allow wifi lib to notify arm9
  // arm7 wifi init complete
}


int main(void) {
  fifoInit();

  // FIXME: Reset clock if needed (WHY?)
  rtcReset();

  powerON(POWER_SOUND);

  irqInit();
  irqSet(IRQ_VBLANK, ihVBlank);
  irqEnable(IRQ_VBLANK);
  irqSet(IRQ_IPC_SYNC, ihIPC);
  irqEnable(IRQ_IPC_SYNC);
  irqSet(IRQ_TIMER2, ihTimer2);
  irqEnable(IRQ_TIMER2);

  // set up wifi interrupt
  irqSet(IRQ_WIFI, Wifi_Interrupt);
  irqEnable(IRQ_WIFI);

  initSoundAndTimers();

  // wifiInit needs VBlank and wifi running
  wifiInit();

  // Enable incoming SYNC interrupt
  REG_IPC_SYNC = IPC_SYNC_IRQ_ENABLE;

  int vBlankCount = 0;
  while(1) {
    ++vBlankCount;

    swiWaitForVBlank();
  }

  return 0;
}

#110157 - Lupi - Sun Nov 26, 2006 2:25 am

Ok, I managed to understand the PCM WAV format (it was hard, my first time decoding a wav)

I also managed to play the wav PCM song in my Nintendo DS second by second using the wav header data.
But it still lacks "perfection". Every time a second of sound ends, the DS stop sound and inmediately start the new second of sound (the game continue as normal, so it's not about loading, it's about SYNC)

I'm thinking about implementing the sound loading directly on the ARM7. Does the gba_nds_fat.h lib works on arm7?
To do this I have to make some kind of comunication between ARM7 and ARM9 with the IRQ_IPC_SYNC, right?
But how will the process be? I can't figure it out

Thanks a lot for all your help :)

#110168 - K-Duke - Sun Nov 26, 2006 5:37 am

Hi Lupi,

although wav-files are easy to handle because of their lack of compression they are a bad choice exactly because of that missing compression.
First of the files are huge and I mean !!HUGE!!
Second: Resulting from the big filesize the DS has to handle with a lot of data to send at least some little sound to the boxes. So the handheld has to work with the sound a lot rendering the DS to work almost at full capacity for a complete song (well actually depending on the quality).

Like bsder I would recommend Ogg Vorbis as it is pretty good to handle, has nice compression and is free to use (no licensing or such).

Correct me if I'm seeing something wrong at this :-D

cheers
K-Duke

#110175 - Lupi - Sun Nov 26, 2006 12:37 pm

Hi K-Duke, I have been searching some info about ogg. I have found that the most used is the vorbis
I went to ogg page to find some info about the structure to read it, but I don't understand a thing of it >_>;;

I also found a libvorbis or something like that, if that is to decode and encode raw audio from ogg and ogg from raw, could it be possible to somehow import that libraries into my proyect?

Though I don't think that would solve the syncronization problem, but I should start trying the ogg audio.

Thanks a lot to all and excuse my english please ^_^