gbadev.org forum archive

This is a read-only mirror of the content originally found on forum.gbadev.org (now offline), salvaged from Wayback machine copies. A new forum can be found here.

Coding > Fixed point multiplication accuracy [sorted]

#130526 - keldon - Mon Jun 04, 2007 5:25 pm

I'm just working on some code in fixed point and encountered terrible blockiness, so I rearranged the multiplies and the shifting of the code like the mode 7 example. But why is it more accurate??? (I later found out from writing this post)

Note: PA,PB,PC and PD are 16.16 and the rest are integers at the moment (xRef and yRef will be 16.16 later).

Code:
// example 1
         long a,b;
         a = (long)PA * (long)x;
         a -= (long)PA * (long)xRef;
         b = (long)PB * (long)y;
         b -= (long)PB * (long)yRef;
         return (int)((a + b)>>16) + xRef;


Code:
// example 2
         return (fmul(PA , (x - xRef)) + fmul(PB , (y - yRef)) + xRef)>>16;


Despite every other variable being an integer, every multiplication still may produce detail past the point; each multiplication truncates this detail. What a massive difference in quality!!!

Well since I figured it out it's not really a question but I still decided to post it as it may help someone in some random way, despite it being said in the Mode 7 tutorial.

#130588 - Miked0801 - Tue Jun 05, 2007 4:45 pm

Can you post the original order? That will help me (us) show you where the over/underflow was occuring.

#130599 - tepples - Tue Jun 05, 2007 7:41 pm

Fixed-point multiplications consist of a multiplication followed by a division by a denominator. If you can rearrange things so that you can multiply before dividing without overflowing 32 bits, you preserve more precision into the final coordinates. A lot of the fine-tuning of the effects engine in TOD went toward this.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#130608 - keldon - Tue Jun 05, 2007 8:18 pm

tepples wrote:
Fixed-point multiplications consist of a multiplication followed by a division by a denominator. If you can rearrange things so that you can multiply before dividing without overflowing 32 bits, you preserve more precision into the final coordinates. A lot of the fine-tuning of the effects engine in TOD went toward this.

Funnily enough that's kind of what I was working on (well the hardware implementation side of it). It's basically a Java emulation of [some of] the GBA hardware but you could easily recode it into C to work with the PC TOD to make for an easy port of the GBA version's features as it emulates the hardware really good. In fact I've created a Mode 7 demo for it already! EDIT: ooh completely forgot about the winTOD that has it

Miked0801 wrote:
PostPosted: Tue Jun 05, 2007 3:45 pm Post subject:
Can you post the original order? That will help me (us) show you where the over/underflow was occuring.

Thanks, but the problem is pretty much solved and understood. Plus now that BGnX and BGnY are 16:16 (in the GBA Hardware emulator type thing I'm working on) the old code doesn't quite give the same looking error as the naive method will now display the pixels fine but move funny when moving around the reference points.

It was something like this:
Code:
return (PA * (x - xRef)) + (PB * (y - yRef))>>16 + (xRef>>16)