gbadev.org forum archive

This is a read-only mirror of the content originally found on forum.gbadev.org (now offline), salvaged from Wayback machine copies. A new forum can be found here.

DS development > Sound recognition with mic

#115518 - Sweater Fish Deluxe - Tue Jan 16, 2007 10:28 pm

I would like to build some (relatively) very simple sound recognition features into the game I'm working on. I'm not talking about actual voice or speech recognition, but just a few basic sounds like blowing on the mic, humming, yelping, etc. Things that it seems would be easily differentiated.

However, I don't see any functions in NDSlib or PAlib that could help me implement something sound recognition. Just recording and playback. Does that mean, I'd have to use ARM7 ASM or something? If so, that's out of my league, I'm afraid.

When recording from the mic, does the recorded sound get stored as an array of some sort? One that I can actually access flexibly? If so, perhaps I could use the data in the array to recognize the sound that was recorded just the trial and experimentation.

But I don't want the player to have to press a specific button or something before making the noise (that would sort of defeat the purpose), so will it be possible to record from the mic continuously (and inspect the recorded data continuously) so that whenever the player makes a recognizable noise it could be detected?

The more I think about this, the more I'm led to the conclusion that it's unfortunately beyond my current programming abilities, but I thought I'd ask in case anyone has any bright ideas or obvious solutions that could help me.


...word is bondage...

#115519 - tepples - Tue Jan 16, 2007 10:43 pm

To turn a recorded voice signal into recognized commands, you'll need to extract patterns from this signal. Recognition is one of the tasks of digital signal processing.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#115534 - simonjhall - Tue Jan 16, 2007 11:40 pm

You could run the mic permanently (assuming there's no technical reason why you can't) and use the ARM7 to do the sound recognition. If you're looking to do plain tone recognition then that sounds do-able - you should first do some kind of FFT code and then visualise the data to see what kind of sound the player is making.

Don't assume at that everyone hums or yelps the same way though.
(I do computer vision 9-5 - I imagine sound is just as hard!)
_________________
Big thanks to everyone who donated for Quake2

#115548 - nolen - Wed Jan 17, 2007 2:10 am

i'm doing something similar to what you want to do but i'm just doing it by the volume of the input the mic is "hearing".

Say the mic has a volume range of 0-255 and blowing is about as loud as 130 or so. To have something react to blowing, set a variable to the current mic volume and when that variable >= 130, have your code for the reaction run.

So just find what volume humming and yelling would be and do something similar.

#115555 - tepples - Wed Jan 17, 2007 3:21 am

Volume isn't a reliable way to distinguish blowing, humming, and yelling. Pitch detection is what you need.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#115598 - Sweater Fish Deluxe - Wed Jan 17, 2007 6:59 pm

Though if blowing into the mic is the only thing you want to detect, volume probably would work since it's likely to be louder than any other ambient noise or even talking from the player. I think I need at least three different sound detections, though, so volume probably wouldn't work for me.

I haven't actually experimented with the mic functions yet (I'm too busy with other stuff to do any actual coding, but I like to daydream about the projects I'll be working on, you know), but assuming that I can get things like continous recording and the ablity to inspect the mic data working well, do you guys know of any PC programs that will analyze recorded data and find patterns across a number of sample recordings? I assume that such programs exist as freeware since it's exactly the sort of thing I see computer science majors having to write as term projects all the time.


...word is bondage...