gbadev.org forum archive

This is a read-only mirror of the content originally found on forum.gbadev.org (now offline), salvaged from Wayback machine copies. A new forum can be found here.

DS development > [xm7lib] Release this lib? ... if it's worth it...

#163928 - sverx - Wed Oct 15, 2008 4:01 pm

Hi!
I know, I already opened this topic to ask about 'how to release a lib'... and I still have to understand clearly how to do that.

I've been thinking... is it worth it? I mean, is the code (and the idea!) good enough to become a library that somebody would find useful?

So I prepared a little demo of the yet unreleased library. It's a beta, you won't find so much, forgive me... but gives you the idea, at least.

It's a library that will play XMs (FastTrackerII eXtended Modules) using only ARM7 resources. So it's like you 'fire and forget', and you've got the ARM9 100% free from this task.

I've uploaded the demo, which is just a little program that uses the library and contains four XMs that raina, (c)runX, rhinostrich and setrodox decided to sacrificate [;)] for that demo. (Thank you so much!)

The XM effects support isn't complete at the moment, most of them are actually still ignored (there are almost none of these in the 4 XMs in the demo) but:
- volume column commands and
- instrument volume envelopes are supported (with sustain, loops and fade)
- samples of 8 & 16 bits are ok
- sample loops (normal & ping-pong) supported too.
(Thank you raina for your lot of beta testing!)

Well, I need the community feedback. You know, I'm a newbie...

Here are the links:[...] removed, were broken. Check http://forum.gbadev.org/viewtopic.php?t=16198 instead.

p.s. : the scopes are cheating ;) Don't pay too much attention to what you SEE :)

edit: raina took a video of the demo, it's here on youtube.
re-edit: removed broken links


Last edited by sverx on Mon Nov 10, 2008 12:40 pm; edited 1 time in total

#164019 - 0xtob - Sat Oct 18, 2008 10:11 pm

Hey! The demo is looking good! I'm just wondering: Since libntxm does the same thing (XM on ARM7) why did you write another lib for this?

About releasing: Putting the source on the web will harm no one. Either people will use it or they won't but you have nothing to loose. Also I get some competition :)
_________________
http://blog.dev-scene.com/0xtob | http://nitrotracker.tobw.net | http://dsmi.tobw.net

#164020 - tepples - Sat Oct 18, 2008 10:46 pm

0xtob wrote:
I'm just wondering: Since libntxm does the same thing (XM on ARM7) why did you write another lib for this?

Because two different XM players can have different levels of coverage of XM format features, or they might use different time-space tradeoffs, etc. For example, does libntxm support the effects column yet?
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#164023 - 0xtob - Sat Oct 18, 2008 10:55 pm

I'm sure he had a good reason to start a new library, I just want to know from him what it was.

libntxm does support some effects already, this is wip.

What do you mean by time-space tradeoff?
_________________
http://blog.dev-scene.com/0xtob | http://nitrotracker.tobw.net | http://dsmi.tobw.net

#164029 - furrykef - Sat Oct 18, 2008 11:43 pm

0xtob wrote:
What do you mean by time-space tradeoff?


Optimizing for size vs. optimizing for speed is a very common tradeoff that one has to make here and there in pretty much any project. Though we don't always make such optimizations consciously, and many of us (like me) initially optimize for "whatever is the simplest to write". ;)

#164030 - tepples - Sat Oct 18, 2008 11:50 pm

Q: What's a time-space tradeoff?

First look up -Os and other optimization options in the GCC manual. Then try compiling the same program with -Os, -O2, and -O3.

Some programs use more memory so that they can run faster. For example, in a music player, a program may have a period table for all octaves (for speed), or it may have a period table only for the 12 tones of the lowest octave with divisions by 2 to reach other octaves (for size). Or a program may use linear interpolation on a lookup table for a trigonometric function (for speed) or CORDIC iterations (for size).
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#164069 - sverx - Mon Oct 20, 2008 8:53 am

0xtob wrote:
Hey! The demo is looking good!


Thank you! :)

0xtob wrote:
I'm just wondering: Since libntxm does the same thing (XM on ARM7) why did you write another lib for this?


Ahem... because I didn't know about libntxm... :| Well, at least it confirms me that the idea of playing XM on ARM7 is good! :)

0xtob wrote:
About releasing: Putting the source on the web will harm no one. Either people will use it or they won't but you have nothing to loose.


No plan to release the source at the moment. At least until I want to keep on working on it by myself :)

0xtob wrote:
Also I get some competition :)


lol :) I didn't mean to compete... anyway I'll check a demo of libntxm, if you can give me a link to it :)

#164070 - sverx - Mon Oct 20, 2008 9:00 am

0xtob wrote:
I'm sure he had a good reason to start a new library, I just want to know from him what it was.


:rolleyes: :rolleyes: :rolleyes:

0xtob wrote:
libntxm does support some effects already, this is wip.


xm7lib supports:

Code:
- 0xy Arpeggio
- 8xx Set note panning
- 9xx Sample offset
- Axy Slides note volume up or down
- Cxx Set note volume
- E8x Set note panning
- E9x Retrig note
- EAx Slide note fine volume up
- EBx Slide note fine volume down
- ECx Note cut
- EDx Note delay
- EEx Pattern delay
- Fxx Set song speed
- Gxx Set global volume
- Hxy Slides global volume up or down
- Lxx Set envelope position
- Pxy Panning slide
- Rxy Retrig note with volume slide

and partially
- 5xy (Portamento to note with) volume slide
- 6xy (Vibrato with) volume slide


I'll go back to work on effects support after the release of this version of the library (I still need to learn how to do that...)

#164072 - 0xtob - Mon Oct 20, 2008 9:10 am

Cool! Of course XM playback on the ARM7 is a good idea. I mean, the DS sound system is actually a hardware XM player :) Good luck with coding!

I've developed libntxm for NitroTracker, so check it out to see what libntxm sounds like. Judging from your feature list I'd say we're pretty much on par feature-wise. I've got some more effects, and you have better volume envelopes.

edit: Oh, you got more effects, so you win :)

So, nice to have some competition :) Let's stay in touch!
_________________
http://blog.dev-scene.com/0xtob | http://nitrotracker.tobw.net | http://dsmi.tobw.net

#164073 - sverx - Mon Oct 20, 2008 9:22 am

0xtob wrote:
[...] the DS sound system is actually a hardware XM player :)


ahahah! You're right! :D
(edit: unfortunally it misses ping-pong loop hardware support...)

0xtob wrote:
I've developed libntxm for NitroTracker, so check it out to see what libntxm sounds like.


I've read about NitroTracker some time ago (I remember I read that it wasn't supporting any effect...) bit I already was thinking it's a cool project! I'll try it with the same XMs and I'll tell you my opinion (well, I'm not a musician...)
Anyway the idea behind xm7lib was (is) to write a library for playing XMs in games, not to write players/trackers for the DS... (and, again, I didn't know somebody was already writing an XM replay library working the same way...)

0xtob wrote:
Let's stay in touch!


Sure! :D
Bye!

#164082 - tepples - Mon Oct 20, 2008 12:54 pm

sverx wrote:
(edit: unfortunally it misses ping-pong loop hardware support...)

And compressed samples. DS uses IMA ADPCM; XM uses a less sophisticated method.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#164083 - sverx - Mon Oct 20, 2008 1:15 pm

tepples wrote:
DS uses IMA ADPCM; XM uses a less sophisticated method.


AFAIK XM have no compression at all on the samples. Anyway I'd be happy to learn I'm wrong :)

#164084 - 0xtob - Mon Oct 20, 2008 1:19 pm

XM only has this stupid delta coding, no idea what that's supposed to bring. Anyway, I was thinking of on-the-fly compressing samples with ADPCM when loading an XM, so less RAM is used.
_________________
http://blog.dev-scene.com/0xtob | http://nitrotracker.tobw.net | http://dsmi.tobw.net

#164087 - sverx - Mon Oct 20, 2008 3:50 pm

0xtob wrote:
XM only has this stupid delta coding, no idea what that's supposed to bring.


And I wouldn't call it 'a compressed sample' for sure ;)

0xtob wrote:
Anyway, I was thinking of on-the-fly compressing samples with ADPCM when loading an XM, so less RAM is used.


Sounds like it's a good idea, if it's possible to enable/disable it...
(and it's a better idea than the one I had... :rolleyes: it envolved trashing the XM format :D ... )

#164133 - zzo38computer - Tue Oct 21, 2008 5:46 pm

0xtob wrote:
Cool! Of course XM playback on the ARM7 is a good idea. I mean, the DS sound system is actually a hardware XM player :) Good luck with coding!
My idea is yo make a new XM format call NXM (Nintendo XM) which has a few differences from XM that are more closer to Nintendo DS, such as making the compression the same as Nintendo DS compression, removing smoe things not support by Nintendo DS, and adding a few other things that Nintendo DS has but XM doesn't have. And then write a program to convert XM file to NXM file.
_________________
Important: Please send messages about FWNITRO to the public forum, not privately to me.

#164138 - tepples - Tue Oct 21, 2008 7:39 pm

A feature request for NXM: Sample sharing, where multiple songs can share samples that have the same length and MD5. For instance, if I use a particular kick drum sample in half the songs in a collection, I don't want to duplicate the sample several times in the archive.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#164147 - eKid - Wed Oct 22, 2008 1:09 am

ADPCM compression on everything would probably mess up effect 9xx (sample offset) which is used fairly a lot...

BTW, my player lib is almost complete (in fact, the music/sfx part IS complete). And I suspect it will dominate over all existing mod players for gba/nds. :) It has complete mod/s3m/xm/IT support, sample sharing (preprocessed), optional adpcm compression, and other goodies.

#164154 - sverx - Wed Oct 22, 2008 9:30 am

zzo38computer wrote:
My idea is yo make a new XM format call NXM (Nintendo XM) which has a few differences from XM that are more closer to Nintendo DS, such as making the compression the same as Nintendo DS compression, removing smoe things not support by Nintendo DS, and adding a few other things that Nintendo DS has but XM doesn't have. And then write a program to convert XM file to NXM file.


mmm... I disagree. NDS doesn't support ping-pong loops but it's not a good idea to remove them because it's possibile to transform them in regular loops in the loading routines. Anyway I think it would be interesting to have some ADPCM support, as long as it is optional, and per sample... so it means that NitroTracker -for instance- could add an XM 'subtype' based on FT2's file format and call it NXM (as in Nitro XM, if you want). Of course it won't play on other already existing players/trackers...

tepples wrote:
A feature request for NXM: Sample sharing, where multiple songs can share samples that have the same length and MD5. For instance, if I use a particular kick drum sample in half the songs in a collection, I don't want to duplicate the sample several times in the archive.


In mods/XMs I've seen sometimes that a single file contains more than one song so that they all share the same samples. Using pattern jump effect it's possible to keep each tune 'separated' from the others...

eKid wrote:
ADPCM compression on everything would probably mess up effect 9xx (sample offset) which is used fairly a lot...


mmm... got to run some tests about that...

eKid wrote:
BTW, my player lib is almost complete (in fact, the music/sfx part IS complete). And I suspect it will dominate over all existing mod players for gba/nds. :) It has complete mod/s3m/xm/IT support, sample sharing (preprocessed), optional adpcm compression, and other goodies.


Can't wait to see it :) A demo? :)

#164161 - tepples - Wed Oct 22, 2008 1:28 pm

eKid wrote:
BTW, my player lib is almost complete (in fact, the music/sfx part IS complete). And I suspect it will dominate over all existing mod players for gba/nds. :)

You said "gba". Do you intend to follow through?

sverx wrote:
In mods/XMs I've seen sometimes that a single file contains more than one song so that they all share the same samples. Using pattern jump effect it's possible to keep each tune 'separated' from the others

Once you keep all your music and sound effects in one file, you will end up running into the format's maximum number of distinct patterns, the format's maximum number of order table entries, or the platform's memory limit.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#164163 - eKid - Wed Oct 22, 2008 1:38 pm

sverx wrote:
eKid wrote:
BTW, my player lib is almost complete (in fact, the music/sfx part IS complete). And I suspect it will dominate over all existing mod players for gba/nds. :) It has complete mod/s3m/xm/IT support, sample sharing (preprocessed), optional adpcm compression, and other goodies.

Can't wait to see it :) A demo? :)


Working on/testing the last few things now, a demo will come very soon.. :)

tepples wrote:
You said "gba". Do you intend to follow through?

Yep, the lib builds for both gba and nds.

#164164 - kusma - Wed Oct 22, 2008 3:18 pm

eKid wrote:
tepples wrote:
You said "gba". Do you intend to follow through?

Yep, the lib builds for both gba and nds.

Are you challenging me to a duel?! ;)

#164183 - sverx - Thu Oct 23, 2008 9:28 am

tepples wrote:
Once you keep all your music and sound effects in one file, you will end up running into the format's maximum number of distinct patterns, the format's maximum number of order table entries, or the platform's memory limit.


That's not false, but this solution gives the game the ability of having a sort of 'continuous play' instead of stopping and loading a new tune. Of course in each situation there's a choice to be made... (btw there are 256 different patterns and 256 table entries in XM, not really few...)

eKid wrote:
Yep, the lib builds for both gba and nds.


So it mixes in software instead of using the 16 hardware channels (of the NDS...) ?

#164188 - eKid - Thu Oct 23, 2008 9:50 am

sverx wrote:

eKid wrote:
Yep, the lib builds for both gba and nds.


So it mixes in software instead of using the 16 hardware channels (of the NDS...) ?


It has different interfaces for gba/ds audio, so it uses the hardware channels on ds. (it also has a special mode that mixes hardware+software together :P)

#164194 - kusma - Thu Oct 23, 2008 10:52 am

eKid wrote:
It has different interfaces for gba/ds audio, so it uses the hardware channels on ds.

How do you support ping-pong looping on the DS hardware?
eKid wrote:
(it also has a special mode that mixes hardware+software together :P)

Outch :P. How do you keep the two mixers in sync?

#164195 - eKid - Thu Oct 23, 2008 11:03 am

kusma wrote:
eKid wrote:
It has different interfaces for gba/ds audio, so it uses the hardware channels on ds.

How do you support ping-pong looping on the DS hardware?

The modules get converted into a custom format so those get unrolled.
kusma wrote:

eKid wrote:
(it also has a special mode that mixes hardware+software together :P)

Outch :P. How do you keep the two mixers in sync?

Magically :P... I delay the hardware channels to match the latency.

#164202 - sverx - Thu Oct 23, 2008 12:06 pm

eKid wrote:
kusma wrote:
How do you support ping-pong looping on the DS hardware?
[...] those get unrolled.


I guess we're all doing that ;) Or is that somebody using a different approach? 8| Talk now! :)

#164204 - kusma - Thu Oct 23, 2008 12:34 pm

I'm not using the DS hardware, so I don't have the problem ;)
Unrolling the loops does work, but doing so does give some odd interactions with some effects, like sample offset.

#164207 - sverx - Thu Oct 23, 2008 1:12 pm

kusma wrote:
I'm not using the DS hardware, so I don't have the problem ;)


Of course you don't ;) BTW, have you got a demo of your work to see? (To hear!)

kusma wrote:
Unrolling the loops does work, but doing so does give some odd interactions with some effects, like sample offset.


for example? Can you explain what could happen with sample offset effect when on a unrolled ping-pong loop?

#164210 - eKid - Thu Oct 23, 2008 1:23 pm

If you use sample offset to jump past the end of a sample, the offset gets zeroed (unless using IT format). If you unroll the loop then it makes the sample longer, which might stop the zeroing. You can't start from inside the sample loop with the DS sound hardware though anyway... :/

#164213 - sverx - Thu Oct 23, 2008 2:33 pm

eKid wrote:
You can't start from inside the sample loop with the DS sound hardware though anyway... :/


That's true, and it would be true even if we had support for ping-pong loop so there's no way out... or you have to copy the unrolled loop portion twice (ram waste...)

#164229 - tepples - Thu Oct 23, 2008 10:30 pm

sverx wrote:
That's not false, but this solution gives the game the ability of having a sort of 'continuous play' instead of stopping and loading a new tune. Of course in each situation there's a choice to be made... (btw there are 256 different patterns and 256 table entries in XM, not really few...)

Imagine patterns of sixteenth notes at 150 bpm, averaging 64 rows per pattern. This makes each order table entry 6.4 seconds long. I can think of a lot of games that have more than a 27-minute soundtrack.

This whole concept of multiple songs in one .xm file reminds me of how early Super NES games' sound drivers worked: there were multiple SPC700 overlays, each of which contained a more or less self-contained set of samples and songs, and the main CPU would have to send each overlay to the SPC700 CPU through a (comparatively slow) link. For example, Super Mario World and SMB3 in Super Mario All-Stars appear to have used one overlay for world maps and one for gameplay.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#164244 - sverx - Fri Oct 24, 2008 11:25 am

tepples wrote:
Imagine patterns of sixteenth notes at 150 bpm, averaging 64 rows per pattern. This makes each order table entry 6.4 seconds long. I can think of a lot of games that have more than a 27-minute soundtrack.


Never calculated it... wow... 27 minutes it's a LOT!
And you didn't consider you're talking about 27 minutes of different patterns, when a lot of tunes have loops made of patterns...
And if it's still not enough you could use 256 rows patterns instead of 64 rows ones, so you'll have 1h 48min of music stuffed in one XM.
It could be enough for most of the applications, at least in my opinion... :)

#164246 - eKid - Fri Oct 24, 2008 11:34 am

What about instrument limit? :)

Also, what sucks about XM is that instruments can't share samples, so to play a sample with different envelopes (or panning setting..) then you have to duplicate the sample... :/ I think pre-processing is really the way to go with XMs.

#164249 - sverx - Fri Oct 24, 2008 11:51 am

eKid wrote:
What about instrument limit? :)
Also, what sucks about XM is that instruments can't share samples, so to play a sample with different envelopes (or panning setting..) then you have to duplicate the sample... :/ I think pre-processing is really the way to go with XMs.


Are we talking about playing XMs or are we talking about writing a different format starting from XM and adding the features we want/need?

Because I think you're all right about the fact that XM has its limits... but that's not the point... XM it's a widespread format and I think it's good to have a library thay plays them the best it's possible while not slowing down the ARM9 cpu, it can be a really good system for having music in homebrew videogames. Don't you think? :)

#164251 - tepples - Fri Oct 24, 2008 12:22 pm

Even Tetris Attack or Tetrisphere has a soundtrack longer than 27 minutes.

sverx wrote:
And you didn't consider you're talking about 27 minutes of different patterns, when a lot of tunes have loops made of patterns...

When a tune loops its patterns, it still uses entries in the order table.

Quote:
And if it's still not enough you could use 256 rows patterns instead of 64 rows ones, so you'll have 1h 48min of music stuffed in one XM.

And you still have all their samples loaded into RAM, including samples that only one song ever uses, like a TB303-style acid line or like the "bells frogs bing cherries"/"estuans interius" vocals from "One Winged Angel" in Final Fantasy VII.

Yes, XM has its limits, but it's a start. I'm just trying to make sure that when we leave room for extensions to the XM format (like IMA ADPCM compressed samples), we leave room at the same time for other extensions (like instruments sharing samples within or across files). Not all extensions have to be implemented in the first release, of course.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#164253 - sverx - Fri Oct 24, 2008 1:10 pm

tepples wrote:
When a tune loops its patterns, it still uses entries in the order table.


I meant using effect Bxx, which does a jump in the order table.

tepples wrote:
I'm just trying to make sure that when we leave room for extensions to the XM format (like IMA ADPCM compressed samples), we leave room at the same time for other extensions (like instruments sharing samples within or across files).


Thanks, I'm sure these ideas will be considered when(if) somebody would want to create an "extended XM" format. Personally, I'm not planning to do that, at least now.

#166408 - hacker013 - Sat Feb 07, 2009 2:39 pm

any updates yet?
_________________
Website / Blog

Let the nds be with you.

#166529 - sverx - Mon Feb 09, 2009 12:57 pm

hacker013 wrote:
any updates yet?


about?
_________________
libXM7|NDS programming tutorial (Italiano)|Waimanu DS / GBA|A DS Homebrewer's Diary

#166548 - hacker013 - Mon Feb 09, 2009 5:27 pm

this library, has it any updates?
_________________
Website / Blog

Let the nds be with you.

#166554 - sverx - Mon Feb 09, 2009 6:00 pm

hacker013 wrote:
this library, has it any updates?


Yes, actually it's ver. 0.81 . Details here (and in other topics in that same forum...)
_________________
libXM7|NDS programming tutorial (Italiano)|Waimanu DS / GBA|A DS Homebrewer's Diary