gbadev.org forum archive

This is a read-only mirror of the content originally found on forum.gbadev.org (now offline), salvaged from Wayback machine copies. A new forum can be found here.

DS development > Accurate touchscreen readings...

#53350 - audiotheory - Wed Sep 07, 2005 7:49 pm

Hello all,
this is my first real post here. I've been lurking for a while and last night I finally decided to install the nds dev tools on my system. I was quite active in the GBC dev scene for a while back in the day and also have done a few GBA projects - it is now time to jump into the wonderful world of the DS!!
I am beginning work on a little notepad style app - the big difference to other such apps that are floating around in the DS homebrew realm?
Handwriting recognition!
I have devised a simple HR engine that I think will work pretty nicely and should be rather easy to implement - which leads me to the title of this thread.
All of the examples I have compiled that give the X, Y coordinates of the touchscreen suffer from the same problems:

Top-left of the screen jumps around the -2, -1 mark
Bottom-right of the screen jumps around the 256, 190 mark
Holding the stylus in one point gives jittery results

I need to be able to read stable coordinates from the touch screen so that:

Top-left of the screen always reads 0, 0
Bottom-right of the screen always reads 255, 191
Holding the stylus in one point gives the same coordinates without jittering

I have read various posts on this topic over the months but I dont feel like there has been a definative answer yet - also searching is becoming harder and harder with every subsequent thread!

Any help on this will be greatly appreciated - even if its just a "use the f**kin search button - heres a previous thread that answers your questions" style response :D

cheers


:theory

#53354 - headspin - Wed Sep 07, 2005 8:05 pm

Have you tried callibrating your touchscreen in your DS preferences?

Anyways, I thought handwriting recognition was a good idea, and found a single character recognition example in java a while ago. Perhaps it can give you some ideas on improving your app. Check it out, it's called JMerlin

Good luck with the app.. you will probably get a better reply from someone regarding your touchscreen issues..
_________________
Warhawk DS | Manic Miner: The Lost Levels | The Detective Game

#53356 - Sausage Boy - Wed Sep 07, 2005 8:19 pm

The offset stuff is just wrong calibration. It's almost impossible
to calibrate good enough to make the top right corner 0x0 and so on.

You might solve your jittering by reading the touch lots of times per
frame and calculate the middle value. I've heard people mention that.
_________________
"no offense, but this is the gayest game ever"

#53368 - audiotheory - Wed Sep 07, 2005 10:29 pm

headspin wrote:
Have you tried callibrating your touchscreen in your DS preferences?

Anyways, I thought handwriting recognition was a good idea, and found a single character recognition example in java a while ago. Perhaps it can give you some ideas on improving your app. Check it out, it's called JMerlin

Good luck with the app.. you will probably get a better reply from someone regarding your touchscreen issues..


Thank you very much for that link, I downloaded the source - it makes for interesting reading.
The method I've come up with is alot simpler though - Im going to use a basic 3x3 position matrix to define strokes. Using this idea I could store a simple stroke (an 'R' for example) in as little as 2 bytes!!
On further googling I've discovered that the 3x3 matrix idea I came up with last night has actually been used in a free stroke-recognition library - its always reasuring to find out an idea you've had has been put into practice by someone else! :)

As for my original question regarding touchscreen coordinates, again thanks for the advice.

@Sausageboy: taking the mean value of multiple samples definately sounds like a good idea to minimise jitter. As for getting the same coordinate values on every DS - there has to be a way of reading the calibration data and taking that into account right?

Can anyone point me in the right direction on this subject?

Cool 3D stuff and wifi capabilities are definately things that I am excited about - but for me personally, I would really like to get the basics accurate first!
ie. standard code that gives reliable readings of the touchscreen on every DS.

cheers again


:theory

#53376 - deltro - Thu Sep 08, 2005 1:33 am

That'd be an interesting function to add to the library, and save processor time. Take 3-5 reads on the touch screen, and average the coord's.

#53389 - natrium42 - Thu Sep 08, 2005 2:54 am

Try taking touchscreen readings more often than a VBLANK interrupt (i.e. use a timer interrupt) and then average values. Also, discard clearly bad readings when averaging.
_________________
www.natrium42.com

#53393 - cybereality - Thu Sep 08, 2005 3:22 am

You may also want to look at finding the mode value instead of the median. For instance, you said you want the null touch to be a solid 0,0 on the touch screen input. If you just averaged the values (with the -1s and -2s) you would get an average of -1,-1 which is still not what you want. If you, however, took a sample of 10 readings and in 6 of them you got 0, you can say that the function will return zero and discard jitter data. Similarly, you can just set an x or y value less then zero to equal 0 (same with max values). This will give more accurate results then just averaging because then you must deal with converting floats to ints and the jitter that will create.

// cybereality

#53540 - deltro - Fri Sep 09, 2005 1:48 am

Why was that 3D pool demo so jittery?

#53567 - audiotheory - Fri Sep 09, 2005 6:41 am

deltro wrote:
Why was that 3D pool demo so jittery?


Pretty much all the demos/examples I've seen that require moving the stylus suffer from this problem.
There doesn't seem to be a standard solution to this yet - which is why I was asking.

I'm working on music during the day so my time to work on DS stuff is limited - ie late at night when the wifes in bed and I cant work on music! But I will be sure to share anything I figure out.


:theory

#53586 - tepples - Fri Sep 09, 2005 3:48 pm

Do any of the official Nintendo DS games require touching at a precision within plus or minus two pixels?
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#53589 - dis169 - Fri Sep 09, 2005 4:41 pm

I could be wrong, its been a while since i looked at the code and even then it was briefly, but doesn't the touch screen generate heat values. IIRC they're wrapped via some macros or function calls to give the x/y values but you might have more luck if you grab the original data and wrap it with knowledge of what your trying to achieve.

Likely the conversion was made to fit the majority of apps where small precision isn't needed and its probably culled in an effort to remove small errors. Unfortunately you may find that you have to introduce the chance of jitter when the touch stick is relatively static if you want finer procision.

Course as i said, its been a while and i could be wrong.

#53591 - Sausage Boy - Fri Sep 09, 2005 4:45 pm

Libnds uses the calibration points automagically. The problem is, it's hard to calibrate
good enough to have exact pixel positions, most users don't want to spend hours
with a microscope to calibrate.
_________________
"no offense, but this is the gayest game ever"

#53624 - audiotheory - Fri Sep 09, 2005 9:50 pm

tepples wrote:
Do any of the official Nintendo DS games require touching at a precision within plus or minus two pixels?


A good point and I guess there aren't many applications where that kind of precision would be required - however I find it hard to believe that Nintendos official librarys simpley say "if you need to read precise values - tough"!
However this is splitting hairs slightly. I guess the bigger problem to be resolved is the jitter and this is something that official Nintendo games most definately do not suffer from.


:theory

#53625 - krunkster - Fri Sep 09, 2005 10:08 pm

audiotheory wrote:
tepples wrote:
Do any of the official Nintendo DS games require touching at a precision within plus or minus two pixels?

A good point and I guess there aren't many applications where that kind of precision would be required - :theory


I don't think you need this sort of precision either.
Just record all valid plot points while the pen is down and then when you lift run your recognition routines. Hopefully the character recognition routine you use is good enough to compensate for the jitter.

#53845 - headspin - Mon Sep 12, 2005 5:24 pm

I know of two methods to remove the jitter, but I'm unsure how accurate they are between different DS'.

Method 1: Compare absolute value of last reading with new reading
Code:
#define MOVE_MAX 16

int new_x, new_y;
new_x = IPC->touchXpx;
new_y = IPC->touchYpx;

if(abs(new_x-old_x)<MOVE_MAX && abs(new_y-old_y)<MOVE_MAX)


Method 2: Check Z1 & Z2 is not zero
Code:
int z1, z2;
z1 = IPC->touchZ1;
z2 = IPC->touchZ2;

if(z1!=0 && z2!=0)

_________________
Warhawk DS | Manic Miner: The Lost Levels | The Detective Game

#53863 - Mollusk - Mon Sep 12, 2005 8:30 pm

I had a pretty simple method to remove jittering, although it's not perfect... I used it on a bitmap viewer, to scroll the bitmap around (it was a pain to have the bitmap move by itself although the stylus was held still).

My method : calculate the distance between the preceding point and the new one... Simply (StylusX - oldStylusX)^2 + (StylusY - oldStylusY)^2... You don't need to take the square root. I chose a value of 9 (so that would a a minimum distance of 3 pixels), but you can chose, say 6 or 7, and not move unless you have that minimum distance...

It's works fairly well

#54103 - Mollusk - Thu Sep 15, 2005 5:46 pm

I changed my arm7 stylus code to limit the range to 0-255 for x and 0-191 for y...

Then I added a sort of anti-jitter code, takes 2 lines... just what I said : check the distance between the last point and the new point, update if big enough, and it works perfectly, no more jittering ! :)

#54111 - natrium42 - Thu Sep 15, 2005 6:16 pm

How often do you take readings?
_________________
www.natrium42.com

#54112 - Mollusk - Thu Sep 15, 2005 6:17 pm

in the arm7 vbl -> 60 times per second, I guess

#54121 - Defect - Thu Sep 15, 2005 6:34 pm

Can this same approach be taken with the stylus support in the heritic and hexen ports?

#54125 - Mollusk - Thu Sep 15, 2005 6:40 pm

it could be taken anywhere, I guess...

#54127 - Defect - Thu Sep 15, 2005 6:43 pm

The jittery stylus movement and freezing up, are the only things holding me back from playing more of those ports.

#54136 - natrium42 - Thu Sep 15, 2005 8:35 pm

Mollusk wrote:
in the arm7 vbl -> 60 times per second, I guess

You should take it more often than that. Especially, when you are averaging :)
_________________
www.natrium42.com

#54157 - cybereality - Thu Sep 15, 2005 11:48 pm

I am working on a method to get solid input from the touch screen. One thing I have noticed in my tests is that the cursor jitters if you dont press hard enough. If you jab hard on the screen it will produce accurate results. If you just slightly graze the screen (ie barely touch it) i wont give back anything useful. My method is based on interpolating the most likely touch point based on surrounding data. If you limit the range of movement from one point to another you also limit the speed you can move the stylus. So it may elliminate jitter, but you also cannot move the stylus quick across the screen. I'll be finishing up the test in the next day or so, I will post the code. Right now I am having trouble getting the timer irqs working so I can poll the screen more than the VBlank. Once I fix that, it should be no problem.
_________________
// cybereality

#54171 - deltro - Fri Sep 16, 2005 1:48 am

Defect wrote:
The jittery stylus movement and freezing up, are the only things holding me back from playing more of those ports.
The only thing holding me back from playing more of those ports is my intense lack of flash cart ._.

#54183 - Mollusk - Fri Sep 16, 2005 7:16 am

Quote:
If you limit the range of movement from one point to another you also limit the speed you can move the stylus.


It's actually the opposite I did : I give a minimum range to have ! So it doesn't change anything for when you move quickly around the screen

#54191 - headspin - Fri Sep 16, 2005 10:48 am

cybereality wrote:
...I am working on a method to get solid input from the touch screen. One thing I have noticed in my tests is that the cursor jitters if you dont press hard enough...


Don't forget that there are at least two different types of touch screens which seem to respond in slightly different ways.
_________________
Warhawk DS | Manic Miner: The Lost Levels | The Detective Game

#54207 - Yoko - Fri Sep 16, 2005 4:59 pm

While handwriting recognition may seem like a "natural" choice, it my opinion it is the wrong choice for the touch screen text entry.

Think about how the letters were evolved; definitely not for speed or convenience. Many letters require multi strokes and multi turns and stops, whereas a single tap or stroke on the touch screen should suffice to convey the intention.

There are other input systems, besides HR or Graffiti, that comes much closer to ideal, and much, must faster. Once that I am using on my table pc which I think is quite appropriate for DS is MessagEase (www.exideas.com). It uses a SINGLE tap or a SINGLE stroke to enter almost any letter, number or character, making it super fast.

I believe in taking advantage of the strength of the machines we have, not trying to fit what we have to the older inefficient ways.

Old habits are hard to break. It took a few decades to convince Europe to discard the old Roman numerals!
[Images not permitted - Click here to view it]

#54210 - Mighty Max - Fri Sep 16, 2005 5:35 pm

Yoko wrote:
Old habits are hard to break.


Do the first step and replace your clock with a metric compliant one.

With the next Beep it's 9 deziday and 5 centidays :p ... oh wait ... a day? Screw it, we just make the year have 100 days to match the pattern.

*g*

Old habits have a reason, they are evolved. New and designed means always lack somewhere. Let the new method grow a few years, and check if it really has a better advantages to disadvantages ratio. Doubt it tbh.
_________________
GBAMP Multiboot

#54241 - Yoko - Sat Sep 17, 2005 1:27 am

Surely you are jesting!

Reductio Ad Nauseum has never been a wise man's "proof". Your distortion of the metric system is neither funny, nor self serving!

We are still stuck at the egotistical whims of Julius and Augustus to have two meaningless months, shifting the seventh month to the ninth position, etc., and you are talking about the old rusted habits as evolution?

Try multiplying using roman numerals!

Your "logic" echoes the simpleton cries for reaming with the "tried and true number system" which resulted in the delay of Western Civilization's renaissance and enlightenment for about eighty years.

Wise man invents; ordinary man follows.

:)

:)

#54381 - deltro - Sun Sep 18, 2005 9:38 pm

Mighty Max wrote:
Yoko wrote:
Old habits are hard to break.


Do the first step and replace your clock with a metric compliant one.

With the next Beep it's 9 deziday and 5 centidays :p ... oh wait ... a day? Screw it, we just make the year have 100 days to match the pattern.

Metric > *

#54619 - beejayzed - Tue Sep 20, 2005 10:42 pm

Em, just wanted to say that pictochat doesn't seem to filter out the jittering. I get horizontal lines when I'm trying to tap dots quite often.
I dunno, maybe it's just my DS...

The SM64DS drawing app doesn't do this, although the drawing does seem to have some lag to it. The slight lag might be intentional though.

#54876 - Zapf_Bandit - Fri Sep 23, 2005 7:20 am

I was having issues when requiring a tap of the stylus to be in a correct position. I kept finding that the first cycle or two after pressing (and when releasing) gave wierd positions. It also drifted towards the correct for about 8 frames.

In order to fix this I let the position converge before using it.

Code:

uint16 /*bool*/ wasDown = false;
uint16 x = 0;
uint16 y = 0;
   
while(1)
{
     
   uint16 /*bool*/ doIt = false;
         
   if (!(IPC->buttons & 0x40)) // If the touchpad is down
   {
      
      if (wasDown)
      {
         uint16 nx = IPC->touchXpx;
         uint16 ny = IPC->touchYpx;
         doIt = (((nx-x)*(nx-x)) + ((ny-y)*(ny-y)) <= 2); //Could use 1 here if you wanted...
         x = nx;
         y = ny;
      }
      else
      {
         x = IPC->touchXpx;
         y = IPC->touchYpx;
      }
         
      if (doIt)
      {
         //OK, now pos has converged and is (x,y)
         //Do something now......
      }
   }
   else
   {
      wasDown = false;
   }

   swiWaitForVBlank();
}


This will avoid both press and release problems. Keep in mind it won't track fast movement well and is mainly for clicking. I hope this helps someone. It took me a while to think of and works well for me.

Zapf Bandit

#54878 - Mollusk - Fri Sep 23, 2005 7:23 am

What I did to try to avoid wrong values when you do a new press was just retard the newpress by 1 frame... If you do a new press, it doesn't do anything (returns not held), and if the stylus is still held, it then returns a new press with the values... Seems to have worked pretty well for me