gbadev.org forum archive

This is a read-only mirror of the content originally found on forum.gbadev.org (now offline), salvaged from Wayback machine copies. A new forum can be found here.

Audio > Software mixer for MIDIs

#14611 - ramirez - Fri Jan 09, 2004 4:19 pm

Ok, here's the deal.
I will have to write a software mixer for MIDI playback, not exactly now, but I found it a good idea if I did some researching early so I could do it when the time comes.
A lot of this games' music has been composed in MIDI format, and I really don't feel comfortable about converting them to MODs or other formats, I am willing to work to get this done (and I have time :]).
However, I have few questions; I am fairly new to sound programming, and by no means I am not a composer, so I do not have that knowledge about i.e. MIDI format either (I've read a bit about the format though).

Well, here's the questions, I'd like to thank everyone for any possible help. :)

1) Since GBA uses raw PCX for music playback (I will use one Direct Sound channel for music, and this will not use tone generators at all, other DS channel will be for sound effects), how am I actually going to get MIDIs play the right sounds? Do I have to have somekind of sound bank or what (my friend was working on GBA soundbank for MIDI playback, hence I am asking this)?

2) Since the channels what the MIDIs uses are resampled into one channel, what is accetpable amount of channels in the MIDI, so the quality would still be somewhat decent for the GBA (the composer said he'd like at least 8, and I said that maybe 6-8 would be possible)?

3) Can someone shortly describe the steps how to get this done (I will research every topic more myself then, I just need basically a framework so I can get started; as I stated before I am rather new when it comes to sound programming :]).

4) What is the best way to attach the MIDIs to the binary? I will have a struct (this is C/C++ mostly, ASM used a bit) for chunks, so should I have an array of these in a header for the MIDIs?

5) Are there any premade software mixers, free to download and look at code, which could also be a nice guide to this (well commented is always better :])?

Well, these are the questions that firstly pop up to my head, any help is highly appreciated. Thank you everyone. :)

#14613 - poslundc - Fri Jan 09, 2004 4:52 pm

ramirez wrote:
1) Since GBA uses raw PCX for music playback (I will use one Direct Sound channel for music, and this will not use tone generators at all, other DS channel will be for sound effects),


When you write a mixer you will quickly see that this is not an effective use of the two sound channels. You are better off using only one channel and mixing everything into that unless you want to support stereo sound, in which case you should mix the left side into one channel and the right side into the other channel.

Quote:
how am I actually going to get MIDIs play the right sounds? Do I have to have somekind of sound bank or what (my friend was working on GBA soundbank for MIDI playback, hence I am asking this)?


Yes. You will need to have samples stored on the GBA for every instrument that you want to use. You will then have to digitally resample the frequency to match the different notes that you want to play.

Quote:
2) Since the channels what the MIDIs uses are resampled into one channel, what is accetpable amount of channels in the MIDI, so the quality would still be somewhat decent for the GBA (the composer said he'd like at least 8, and I said that maybe 6-8 would be possible)?


I think 8 is fine for a video game, but some people get more ambitious and support 16, or even more. It starts to take a lot of processor time to support more channels, though.

I don't think any SNES game ever used more than 8 channels (although I could be wrong; I'm sure someone will correct me if I am). Also, traditionally sound effects are handled by having them bump out the instruments in the upper channels. This way you just tell your composer to put the heavy instrumentation in the lower channels and you're fine.

Quote:
3) Can someone shortly describe the steps how to get this done (I will research every topic more myself then, I just need basically a framework so I can get started; as I stated before I am rather new when it comes to sound programming :]).


Step 0: come up with some basic data structures for storing samples and the necessary pulse-code modulation data. Now would be a good time to make sure you are doing everything compliant with the format you are using (MIDI, in your case). You will need to write an application that converts samples into the format you use on the GBA.

Step 1: write a program that can play a sampled sound to the FIFO.

Step 2: write a program that resamples to let you adjust the frequency of the sound up or down. This is when you should make sure you can take a "note" in your preferred format (MIDI in your case) and have the instrument play that note correctly on the GBA.

Step 3: write a mixer that combines different samples into a single channel. This is potentially the most challenging step and where most different sound systems diverge in terms of capabilities and processor consumption.

Step 4: write a music player that tells the samples when to play.

These are the steps I followed when writing my MOD player, and I think they're the most natural progressions for writing a music player.

Quote:
4) What is the best way to attach the MIDIs to the binary? I will have a struct (this is C/C++ mostly, ASM used a bit) for chunks, so should I have an array of these in a header for the MIDIs?


Any data you attach should be in a C file or an ASM file; not in a header file. Header files should only contain statements that say that the data exists somewhere, not the actual data itself.

You can create an array of all of your MIDI files if you want; I just give each one a separate variable name (in my case, they are all prefixed with the tag "mod" and postfixed with "Info", eg. "struct ModInfo modMySongInfo") and then I can refer to it in my code as necessary.

Quote:
5) Are there any premade software mixers, free to download and look at code, which could also be a nice guide to this (well commented is always better :])?


There probably are, but I will have to let someone else answer this one. You might take a look around yourself (perhaps check the devrs site and the tutorials/demos on gbadev) and see what you can find. Because writing a music-player is a multi-tiered project, though (as I described above) and there are a lot of design decisions to be made along the way, it can be very difficult to read/adapt someone else's code. You may as well give it a shot, though.

Dan.

#14619 - jd - Fri Jan 09, 2004 10:15 pm

ramirez wrote:

2) Since the channels what the MIDIs uses are resampled into one channel, what is accetpable amount of channels in the MIDI, so the quality would still be somewhat decent for the GBA (the composer said he'd like at least 8, and I said that maybe 6-8 would be possible)?


8 channels would probably give the best performance/quality compromise, but more than that is certainly possible. (I'd guess maybe as many as 128 channels would be theoretically possible with an efficient mixer, but that would be complete overkill and would leave no CPU time for anything else.)

ramirez wrote:

4) What is the best way to attach the MIDIs to the binary? I will have a struct (this is C/C++ mostly, ASM used a bit) for chunks, so should I have an array of these in a header for the MIDIs?


Probably the easiest way would be to make a converter that writes the data out to a .s file with a corresponding .h file that you include in your project. (.s files are much faster to assemble than .c files.)

ramirez wrote:

5) Are there any premade software mixers, free to download and look at code, which could also be a nice guide to this (well commented is always better :])?


If you'll forgive the blatant plug, you could use my mixer:

http://www.apex-designs.net/tools_aas.html

The source code isn't included but you can access the mixer directly - although I'd recommend using the SFX playing commands instead as they're a lot easier to use. Doing things this way also automatically sets up the GBA's sound hardware for you.

#14632 - ramirez - Sat Jan 10, 2004 1:17 am

Thanks for help. A appreciate this. :)
I still want to write my own mixer, for two reasons: I am really interested about sound programming, so I am looking for experience and I am hoping that we can get this project commercially published when it's done.

However, about this soundbank which will hold the samples of instruments, I am not quite sure how this must be done as I don't know how raw PCX data actually works, can anyone help a bit here?

It seems like making a MOD player is somewhat easier than MIDI (everyone seems to have made one :x). Is there a good tool that can convert MIDIs into MODs without (big) loss of quality? And when using MODs, do I need to have a soundbank holding the samples of different instruments?

I'd like to hear where you guys mostly learnt this stuff, do you have any tutorials about this stuff? I have so much questions that it'd take a lot time to ask them. x.x

Thanks for help again. :)

#14633 - tepples - Sat Jan 10, 2004 1:19 am

poslundc wrote:
I don't think any SNES game ever used more than 8 channels (although I could be wrong; I'm sure someone will correct me if I am).

The Super NES's hardware mixer had 8 hardware channels. The Super NES's sample compression and slow CPU speed ruled out software mixing. See if you can't make kick+hat or chord samples so that you can get more instruments playing on the same number of voices.

Quote:
Step 1: write a program that can play a sampled sound to the FIFO.

Step 2: write a program that resamples[...]

Step 3: write a mixer[...]

Step 4: write a music player that tells the samples when to play.

You can parallelize the work load a bit by having one programmer do step 4 on the tone generators until another programmer has finished doing steps 1-3 on the PCM channels. This may also give your composers four more channels to work with to create richly textured music.

Quote:
Any data you attach should be in a C file or an ASM file

I'd recommend either in a .s (assembly) file or in an appended datafile.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#14634 - tepples - Sat Jan 10, 2004 1:29 am

Quote:
I am not quite sure how this must be done as I don't know how raw PCX data actually works, can anyone help a bit here?

For one thing, "PCX" is an image format. You want "PCM" which is stored in WAVE files. Read up on the basics of digital signal processing. Then read a quick tutorial on how to write a digital audio mixer.

Quote:
It seems like making a MOD player is somewhat easier than MIDI (everyone seems to have made one :x).

That's because MIDI players need to carry a megabyte or more of sounds to fill out the General MIDI sound set.

Quote:
Is there a good tool that can convert MIDIs into MODs without (big) loss of quality?

Some trackers (programs that produce .mod, .s3m, .xm, .it) can import .mid files.

Quote:
And when using MODs, do I need to have a soundbank holding the samples of different instruments?

Each .mod file carries its own samples.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#14637 - poslundc - Sat Jan 10, 2004 1:46 am

tepples wrote:
Quote:
It seems like making a MOD player is somewhat easier than MIDI (everyone seems to have made one :x).

That's because MIDI players need to carry a megabyte or more of sounds to fill out the General MIDI sound set.


Also, MOD is the oldest (and therefore plainest) standard for music files around; it comes from good ol' Amiga days. This makes it the easiest format to implement, and requires the least computational power to play (insofar as I'm aware, anyway).

Quote:
Quote:
Is there a good tool that can convert MIDIs into MODs without (big) loss of quality?

Some trackers (programs that produce .mod, .s3m, .xm, .it) can import .mid files.


Several tools import; I have yet to find one that can adequately export MOD. The two formats are extremely different. I've come up with a way of doing it using some Macintosh software (my composer is using MIDI, so I pretty much have to), but it's non-trivial and less than perfect.

Quote:
Quote:
And when using MODs, do I need to have a soundbank holding the samples of different instruments?

Each .mod file carries its own samples.


... part of what MIDI a poor choice over most other music formats for a system like the GBA.

Dan.

#14638 - tepples - Sat Jan 10, 2004 2:09 am

poslundc wrote:
MOD is the oldest (and therefore plainest) standard for music files around; it comes from good ol' Amiga days. This makes it the easiest format to implement, and requires the least computational power to play (insofar as I'm aware, anyway).

S3M doesn't really take any more power than MOD, except for the tendency of S3M files to have been written later and thus on newer hardware and thus with more channels than the typical 4-channel Amiga MOD. S3M's big theoretical advantage over MOD is a separate column in each channel for expression and panning as well as a bit of simple compression of the pattern data.

Quote:
tepples wrote:

Each .mod file carries its own samples.


... part of what MIDI a poor choice over most other music formats for a system like the GBA.

Others would claim that the fact that several .mid files can share one sample set makes .mid a better choice than the tracker formats because the tracker formats would have to repeat the samples for each song.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#14644 - poslundc - Sat Jan 10, 2004 3:16 am

tepples wrote:
S3M doesn't really take any more power than MOD, except for the tendency of S3M files to have been written later and thus on newer hardware and thus with more channels than the typical 4-channel Amiga MOD. S3M's big theoretical advantage over MOD is a separate column in each channel for expression and panning as well as a bit of simple compression of the pattern data.


I'm not sure what "expression" is, but there is an effect for panning in MOD. Besides which, many mixers on the GBA are only half-panning anyway. (Well, mine is. So there.)

Quote:
Others would claim that the fact that several .mid files can share one sample set makes .mid a better choice than the tracker formats because the tracker formats would have to repeat the samples for each song.


Touch?. (Although my MOD converter tool I've written reuses samples from other MODs wherever possible.)

Dan.

#14645 - tepples - Sat Jan 10, 2004 3:32 am

poslundc wrote:
tepples wrote:
S3M's big theoretical advantage over MOD is a separate column in each channel for expression and panning [...]

I'm not sure what "expression" is

"Expression" is the MIDI term for changing the volume of a channel.

Quote:
but there is an effect for panning in MOD.

Having a dedicated column for volume and pan effects lets such an effect happen on the same row as another effect.

Quote:
Besides which, many mixers on the GBA are only half-panning anyway. (Well, mine is. So there.)

By "half panning" do you mean hard left/hard right?
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#14646 - ramirez - Sat Jan 10, 2004 3:37 am

Ok, it seems that making this soundbank for MIDIs myself, would be pretty hard, and in the other hand the composer doesn't probably know how to do it either. I'm probably going for MOD afterall, just need to try out some ways to get good quality even by converting (the composer do not do tracking so).

Well, that's about it.
I have few more questions, as you could expect. :)

1) Since the channels has to be resampled into one channel, how am I supposed to do this?

2) Can't I generate the raw 8-bit signed samples beforehand when I load the music/sound effect, and then just feed the direct sound channel with the generated samples? Or does this take too much memory?
EDIT: But of course freeing it when music changes.

3) When you do realtime mixing, and you have a DMA set to automatically feed the FIFO when it needs new data, how am I supposed to do this if I do the mixing in realtime?

4) Is there an open source MOD mixer? I'd really like to see source code, because that's how I learn the best.

#14650 - poslundc - Sat Jan 10, 2004 4:57 am

tepples wrote:
By "half panning" do you mean hard left/hard right?


Yes, although I'm not certain where I picked up that slang from.

Dan.

#14651 - poslundc - Sat Jan 10, 2004 5:09 am

ramirez wrote:
1) Since the channels has to be resampled into one channel, how am I supposed to do this?


Not resampled; mixed. You mix two waveforms by adding them together.

You have to watch for overflow, however: if you add two values and it comes up to greater than +/-128 then it no longer fits into 8 bits. Two ways of dealing with this are clipping and scaling: clipping is where you check to see if your value is out-of-bounds and if so just replace it with either + or - 128. Scaling is where you divide your samples (scale them down) by a constant before adding in order to make sure they can't overflow.

Clipping typically takes more processing than scaling, but prevents volume loss that comes with scaling.

Quote:
2) Can't I generate the raw 8-bit signed samples beforehand when I load the music/sound effect, and then just feed the direct sound channel with the generated samples? Or does this take too much memory?
EDIT: But of course freeing it when music changes.


Not entirely sure what you're asking. Buffering is a good idea. But you can only buffer so far ahead, or you will run into problems if the music changes before your buffer is emptied. Most mixers that use buffers tend to buffer between 200-400 bytes.

Quote:
3) When you do realtime mixing, and you have a DMA set to automatically feed the FIFO when it needs new data, how am I supposed to do this if I do the mixing in realtime?


If you're mixing in realtime, then you may as well ditch the DMA and write to the FIFO yourself. I can tell you from experience, though, that realtime mixing seems like a much better idea than it works out in practice for most mixers. There may be some exceptions for very advanced/creative mixers (nod towards AAS), but generally a simple double-buffered system running alongside DMA works best.

Quote:
4) Is there an open source MOD mixer? I'd really like to see source code, because that's how I learn the best.


I might consider letting you have a peek at mine (e-mail me if interested), but it's written entirely in assembly so you may find it difficult to decipher.

Dan.

#14653 - tepples - Sat Jan 10, 2004 6:05 am

poslundc wrote:
Buffering is a good idea. But you can only buffer so far ahead, or you will run into problems if the music changes before your buffer is emptied.

"If the music changes"? Are you talking about branching soundtracks as used in Windows DirectX music?

Quote:
Most mixers that use buffers tend to buffer between 200-400 bytes.

True, most mixers do mix about 10-30ms of music at a time (some are synchronized to 60hz, that is, 16.7ms), but there do exist exceptions. I've seen demos that buffer several seconds of music before running a particularly complicated scene. Of course, the viewer doesn't notice this while viewing a relatively subdued lead-in scene, designed as a sort of calm before the storm.

Quote:
If you're mixing in realtime, then you may as well ditch the DMA and write to the FIFO yourself.

I think "realtime mixing" was supposed to mean as opposed to precomputed mixing.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#14667 - ramirez - Sat Jan 10, 2004 3:30 pm

Quote:
If you're mixing in realtime, then you may as well ditch the DMA and write to the FIFO yourself.

I think "realtime mixing" was supposed to mean as opposed to precomputed mixing.[/quote]
Exactly what I meant.
Well, the reason why my question(s) doesn't make sense is probably because I've misunderstood something.
What is the most common way to do this? Precomputed mixing or "realtime" mixing?
poslundc, I'd love to see your mixer, I've done assembly before so I don't find it problem, I'll e-mail it about you so you get my e-mail.
Thanks a lot for help guys. :)

Oh btw, this "double-buffered" system. How does this work? Do I mix other buffer while playing another? And if so, how does this work in action?

#14670 - tepples - Sat Jan 10, 2004 6:00 pm

The "right way" is to mix 10ms to 30ms of audio at once. Many GBA programs mix at 18157 Hz, that is, 304 samples per vblank. They mix 16.7ms chunks of sound into one of two buffers while playing the other with DMA to the FIFO.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#14687 - ramirez - Sun Jan 11, 2004 2:18 pm

tepples wrote:
The "right way" is to mix 10ms to 30ms of audio at once. Many GBA programs mix at 18157 Hz, that is, 304 samples per vblank. They mix 16.7ms chunks of sound into one of two buffers while playing the other with DMA to the FIFO.

This is what confuses me about the double buffering.
I mean, if you mix to the other buffer while playing the other, when do you play the mixed buffer? And the "other" buffer has to has mixed sampled too. x_x

Also, I was thinking, my friend said that MIDI mixer would take too much CPU time for GBA, is this true? Let's say if I want the mixer to take 10% of the CPU at most.

#14689 - poslundc - Sun Jan 11, 2004 2:53 pm

ramirez wrote:
This is what confuses me about the double buffering.
I mean, if you mix to the other buffer while playing the other, when do you play the mixed buffer? And the "other" buffer has to has mixed sampled too. x_x


You have two buffers: A and B.

While A is playing, you are mixing content into B.

When A is done playing, it starts playing B.

While B is playing, you are mixing content into A.

etc.

That's double buffering.

Quote:
Also, I was thinking, my friend said that MIDI mixer would take too much CPU time for GBA, is this true? Let's say if I want the mixer to take 10% of the CPU at most.


The mixer should be pretty much the same regardless of the music format. It's the music player that will be significantly different. Usually the music player doesn't consume the greatest number of resources, though. I'm not aware how brutal the MIDI format is.

You will have a tough time getting down to less than 10% with conventional algorithms. My MOD player is 10-13%, and that's limited to 8 channels, fixed panning, scaling instead of clipping, and pretty much the entire thing in hand-optimized assembly. The only "indulgence" it takes in processor consumption is that the music player runs at 128 Hz (many run only at 60 Hz).

Dan.

#14698 - tepples - Sun Jan 11, 2004 6:47 pm

poslundc wrote:
The mixer should be pretty much the same regardless of the music format.

Not if the format includes support for time-varying resonant filters, as IT does.

Quote:
It's the music player that will be significantly different. Usually the music player doesn't consume the greatest number of resources, though. I'm not aware how brutal the MIDI format is.

MIDI is straightforward to interpret, if a bit bigger than S3M.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#14699 - poslundc - Sun Jan 11, 2004 7:37 pm

tepples wrote:
Not if the format includes support for time-varying resonant filters, as IT does.


That may very well be. Of course, you can always decide what features you want to support in your players.

Quote:
MIDI is straightforward to interpret, if a bit bigger than S3M.


Hey tepples, you say an entire MIDI soundbank is about 1 MB? Do you know where I can download one? Is there a "standard" one for software playback? I've searched google with no luck. I'm starting to wonder if maybe I should write a MIDI instead of a MOD player, since that's what my composer is using. 1 MB is an order of magnitude larger than the space consumed by most MOD samples, I think, but it's still not a wholly unreasonable amount.

When you say MIDI is straightforward to interpret, how comparable is it to MOD in terms of complexity?

For example, in MOD my mixer only needs to worry about either looping a sample or cutting it off when it's done. But I know some other formats support things like ping-pong loops and whatnot, which I would rather do without.

Thanks,

Dan.

#14706 - ramirez - Sun Jan 11, 2004 9:40 pm

Well, I would of course want to write a MIDI mixer.
Basically what I need is a soundbank maybe with 16-32 instruments? (I thought that 1-2MB would be acceptable size for it).
Then a mixer and the player would take 10-15% (or 20% at most) of the CPU.
But I am not 100% aware how am I going to implement this sound bank.
Is writing a MIDI mixer a lot harder than MOD mixer?

EDIT. Also, you said that your player runs at 128Hz, what does this mean? Does it tell how fast you play the samples or?

#14711 - DekuTree64 - Sun Jan 11, 2004 10:19 pm

Actually if you only use a few of the MIDI instruments, it shouldn't take any more space for the sample data than MOD, I would think. I was under the impression the 1MB would be for all 120 or however many instruments MIDI has.

As for mixers, 10% is a fairly low usage for 8 channels. Mine takes more like 13-15%, but it supports full panning and clipping instead of scaling, so that's to be expected. 25% for 16 channels, but that's ok for me since my game is an RPG, and has CPU cycles to burn. But if you don't mind mono or hard stereo, 10-15% isn't too much to ask for 8 channels. PM me or something for my ICQ# if you ever get into mixer writing, I've done a lot of studying on them so I'd be happy to give some optimization tips once you're ready for them. Actualy I may write an article on the subject sometime, if enough people would like to read it.

I don't think there would be any problem with using a regular tracker-style mixer for MIDI, as long as you can deal with some limits.
I was trying to decide between doing a MOD or IT player, and then it occurred to me that I may as well do IT, and just cut the features that aren't really necessary, and would make it slower (i.e. ping-pong loops and resonance filters). If you look at it that way, trying to cnovert a MIDI to a MOD is forcing a lot worse problems on you than playing it in its original form from a not-quite-complete player. Besides, I've heard more stories about people wanting to play MIDIs on GBA than I can remember, so even if it's a partial player, I'm sure a lot of people will appreciate it.
_________________
___________
The best optimization is to do nothing at all.
Therefore a fully optimized program doesn't exist.
-Deku

#14713 - ramirez - Sun Jan 11, 2004 10:37 pm

I for one am sure interested about a GBA mixer article.
Some guy was writing a 3 part article about GBA direct sound system, and the parts 2 & 3 were supposed to tell about the mixer stuff, but unfortunately the part 1 was the only released part.
I've read some of the other topics around here, Audio forum, and to me it seems like many people are/was in the same situation as I am now; So I am pretty sure that it would be highly appreciated. In matter of fact I was suprised that nobody has done that here; seeing how many of you know about this stuff. I believe that gbadev.org is the place where the most beginners looks for articles and documentation about developing for GBA.
And thanks for your help, I'll sure remember your help offer. :)
It'd be great to have a MIDI mixer (and oh yeah, this is an RPG as well, so I guess that I have some cpu cycles to burn too then, hehe).

EDIT.
Poslundc, my friend showed me this: http://www.midi.org/about-midi/dls/dlsoview.shtml
Maybe it'd be possible to have a DLS bank for GBA?

#14751 - gb_feedback - Mon Jan 12, 2004 10:11 am

I did a MIDI player some time back. I think there's still a demo of it on my web site. There's also a pogoshell plugin. The samples data ended up at about 64k bytes. It achieved this by mapping the instruments onto a smaller representative set of instruments. I didn't think it was terribly complicated; in fact when I went on to look at mods was when it seemed to get difficult (I gave up in favour of other activities).
I must say I've always wondered why people considered it was better to include the samples in with every tune (mods), as opposed to a one-off inclusion in the player. As an outside observer it seems to me that the advantage of mods is the potential of a different 'sound' (with more complex synthesised instruments). A normal MIDI is probably going to use conventional instruments.
_________________
http://www.bookreader.co.uk/

#14759 - ramirez - Mon Jan 12, 2004 5:00 pm

Yeah, well my friend who is kind of talented and experienced MIDI composer (also knows a lot about the format itself) adviced me to use a DSL bank (I believe that Final Fantasies for PSX used DLS banks).
He also pointed me a program, Awave Studio 9.0 (costs 150$) that can convert soundfonts into a DLS files.
What I am hoping is to make a MIDI mixer and player ("General MIDIs"), playing at 128 Hz and mixing all the 16 channels (and only a mono). It may sound a bit much though (does it? :x), but at the moment I really can't figure if I need that much CPU for something else so it's ok I guess.
I only need to have like 3 controllers as well I think (loop, pitch and volume).
My friend also offered to do the DLS bank for me when the time comes, as long as I buy the software for him.

What I would like to know, do you think that this could work, and how fast? I could even go for Stereo if I can afford it, I really want to give the composers as much as freedom as they can get.

#14763 - gb_feedback - Mon Jan 12, 2004 5:24 pm

I'm remembering one or two more things now. I wrote a pc front-end app which reprocessed the midi to be all on one track and to have all note changes occur on 1/60 second boundaries. This simplified the processing in the GBA no end, as you just had to do one midi file processing routine per frame to pass on which notes had changed to the mixer. Didn't sound any different from the original midi file either.
The app also stripped out any messages I wasn't going to handle, and checked that converting to one track wasn't causing 2 identical notes on the same instrument at the same time to play.
I only used 8 channels BTW. That was all I dared as the mixer was written in c and highly unoptimised.
_________________
http://www.bookreader.co.uk/

#14765 - poslundc - Mon Jan 12, 2004 5:32 pm

ramirez wrote:
What I would like to know, do you think that this could work, and how fast? I could even go for Stereo if I can afford it, I really want to give the composers as much as freedom as they can get.


How fast you can make it go is entirely up to you. If you are awesome at assembly and clever coding, you might get it as low as 15% or so. I think unoptimized mixers usually hover around 20-30%.

Fixed-panning stereo (half the channels in the left side, half in the right side) is not very expensive computationally if you make some basic optimizations, so you might want to consider it. There are tricks you can do if you decide to go full-panning as well.

Dan.

#14771 - ramirez - Mon Jan 12, 2004 6:34 pm

gb_feedback wrote:
I'm remembering one or two more things now. I wrote a pc front-end app which reprocessed the midi to be all on one track and to have all note changes occur on 1/60 second boundaries. This simplified the processing in the GBA no end, as you just had to do one midi file processing routine per frame to pass on which notes had changed to the mixer. Didn't sound any different from the original midi file either.
The app also stripped out any messages I wasn't going to handle, and checked that converting to one track wasn't causing 2 identical notes on the same instrument at the same time to play.
I only used 8 channels BTW. That was all I dared as the mixer was written in c and highly unoptimised.

Well, sounds like a great method, I should look more into it.

I'll probably write the final mixer in assembly, even though I am not that great at it, but I can manage it. And Deku promised to give me some optimization tips. :)

Also, to make the player faster, I thought about always feeding the samples at 22050HZ frequency (if the DLS bank samples are the same, I don't need to do any sample scaling, does this fastens the processing much?)

While I am at it, I have a question to ask (Still haven't read much about MIDI format, but this is more like a general question about mixing):

How can I make the individual channel volume to work in the final mixer/player, MIDI has a volume control for every channel, and I'd like to know what is the algorithm/way to make the channel play on its own volume (I will also have a master volume).

#14780 - tepples - Mon Jan 12, 2004 7:51 pm

You'll need to play sounds faster or slower if you want to be able to play more than one pitch with a given instrument. Usually, on the GBA, this will involve nearest-neighbor resampling.

To scale the volume of a sampled sound, multiply each sample by the volume before adding it to the mix bus.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#14788 - ramirez - Mon Jan 12, 2004 8:42 pm

tepples wrote:
You'll need to play sounds faster or slower if you want to be able to play more than one pitch with a given instrument. Usually, on the GBA, this will involve nearest-neighbor resampling.

To scale the volume of a sampled sound, multiply each sample by the volume before adding it to the mix bus.

So it really is that easy?
If I am right, if the channel volume is 'default' I would like to multiply it with 1, right?

#14793 - tepples - Mon Jan 12, 2004 9:41 pm

You'll use bit of fixed-point arithmetic to implement the actual volume scaling, but you are on the right track.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#14794 - poslundc - Mon Jan 12, 2004 9:43 pm

ramirez wrote:
So it really is that easy?


Famous last words... but yes, the principle behind it is simple enough.

Quote:
If I am right, if the channel volume is 'default' I would like to multiply it with 1, right?


You've got the right idea, except remember that you need to use fixed-point math instead of floating-point, so 1 will actually probably be something more like 64.

Edit: Curse you, tepples! <shakes fist>

Dan.

#14800 - ramirez - Mon Jan 12, 2004 10:11 pm

Yeah, fixed point. :)

Currently only thing that concerns me is the pitch, I mean, the sound of the instrument is changed by alterning the pitch right?
How do I implement this? :o
Do I need to scale the samples or?

#14811 - edwdig - Mon Jan 12, 2004 10:57 pm

ramirez wrote:
Yeah, fixed point. :)

Currently only thing that concerns me is the pitch, I mean, the sound of the instrument is changed by alterning the pitch right?
How do I implement this? :o
Do I need to scale the samples or?


If you've got a sample at 22050 Hz, and you want to play it one octave higher, then you play it back at 44100 Hz. One octave lower would be 11025 Hz. 12 notes to an octave, evenly spaced apart. Figure the notes out based on that.

Use fixed point to track what point in the sample you're at, and just chop it down to integer to actually reference the data.

#14818 - DekuTree64 - Mon Jan 12, 2004 11:05 pm

Pitch altering is done by using a fixed-point position as well. To go up one octave, multiply the pitch by 2, to go down one, divide by 2. Therefore, to play a sound one octave below normal, you'd need to advance .5 samples at a time, so you need fixed-point. The exact note frequencies are best precalculated because they involve the exponential function, which uses floating point numbers, which are very slow on GBA.
But really, it is an easy concept, just load sample, multiply by vol, add to mix, add increment to pos, and repeat. Of course, you have to check if the sample is ending this run through the loop, and if it is, then either disable it or subtract the loop length from the pos, which is a good place for optimization, since you can check if a sample is going to end this frame and if not, use a loop that doesn't do any checking.
_________________
___________
The best optimization is to do nothing at all.
Therefore a fully optimized program doesn't exist.
-Deku

#14852 - gb_feedback - Tue Jan 13, 2004 10:43 am

You may need to think about the following:

To play a realistic sounding note you need to shape the envelope of the sample. Mainly that means when the note starts it gets louder and quieter at different rates depending on the instrument. Also when the midi file says 'release the note' you don't just turn it off. You leave it playing for a while, while you reduce its volume. And until that note has reached 0 in volume, you can't re-use that channel (or voice).

My approach to channels was to look for an unused channel when I had a new note to start. If there wasn't one, then I killed the oldest note playing, and used its channel.

For the envelope I cheated and had a single volume multiplier for each channel. This was decayed by a value specified by the instrument, and at a different rate when the note was released, and gave a realistic sounding note. When the volume reached zero, the channel was marked as free again. Handling the decay was a separate and not very time-consuming operation after each mixing session, because I chose the numbers to only change the envelope size once per frame. (There is also an attack phase along similar lines.)

Another interesting feature was that each sample was stored with its own frequency parameter, which was the base of the sample increment calculation. This meant that the samples didn't have to be at the same frequency, which proved quite useful in keeping their size down.

One of my sound channels looked like this:

Code:

typedef struct
{
   u8         control;      // 0 = off
                        // 1 = attack
                        // 2 = decay
                        // 3 = sustain
                        // 4 = release
                        // 0x80 = portamento flag
   u8         rVol;         // use for now for mono vol               
   u8         lVol;      
   u8         ampl;      // envelope amplitude      
   u8   attackValue;   // volume increased by this every frame
   u8   decayValue;      // volume reduced by this every frame
   u8   sustainValue;   // decay doesn't go below this value
   u8   releaseValue;   // volume reduced by this every frame after 'note off'
   const s8*         wp;      
   u32         bytesInSample;   // 16.16
   u32         bytesInLoop;   // 16.16
   u32         currentSampleByte;
   u32         sampsIncrDelta;      // 16.16 - portamento delta / frame
   u32         sampsIncrTarget;   // 16.16 - portamento target
   u32         sampsIncrValue;      // 16.16 - used for making freq
   u8         channel;      // midi channel
   u8         key;         // midi key
   u8         seq;         // order of notes started
} SndChannel;



_________________
http://www.bookreader.co.uk/

#15062 - Maddox - Fri Jan 16, 2004 7:37 pm

The ordering of member variables in that struct made me wanna barf! I mean... it's just AWFUL!

Did somebody say MADDOX?
_________________
You probably suck. I hope you're is not a game programmer.

#15067 - animension - Fri Jan 16, 2004 8:30 pm

gb_feedback, there's nothing wrong with your struct member order. You have 8 u8's which translates to 2 32-bit aligned memory locs, a 32-bit pointer to an array of u8's, 6 u32's which are 32-bit anyway, trailed by 3 u8's which fit in one 32-bit location, for a total of 10 32-bit memory locations.


The only inefficiency in this struct is that you could fit one more 8-bit member at the tail end and still use only 10 32-bit memory locations, but the ordering / alignment of your members is fine, which is what the community dickhead who posted before me decided to criticize.
_________________
"Beer is proof that God loves us and wants us to be happy."
-- Benjamin Franklin

#15069 - gb_feedback - Fri Jan 16, 2004 8:46 pm

animension: Thanks for your comments. As the previous poster didn't explain his outburst, I still don't know what he was getting at.

The struct in itself was unimportant of course, it just represents the data structure at a point halfway through the midi decoder processor. One side is a midi format decoder which drives an array of these structures; the other side is a mixer which uses them to create the output sound wave. In my experience data structure is the most defining thing about a software algorithm, so this is what I presented to give the OP an idea how I achieved the midi player functionality.

I would just say that if a poster can't be constructive, the least they could do is to shut up.
_________________
http://www.bookreader.co.uk/

#21895 - Midori - Tue Jun 08, 2004 10:17 pm

Quote:
Is there a good tool that can convert MIDIs into MODs


ModPlug Tracker converts from MIDI to IT with no loss in quality. It supports MOD although I've never tried it. It should work fine if you pick some nice samples.