gbadev.org forum archive

This is a read-only mirror of the content originally found on forum.gbadev.org (now offline), salvaged from Wayback machine copies. A new forum can be found here.

Coding > s16 don't go high enough

#3059 - pollier - Mon Feb 17, 2003 11:16 am

My LUT for rotating my background, when rotated at 0 or 90 degrees, has to use the value 0x7fff instead of 0x8000 because of the nature of 8-bit-fractional signed numbers; this gives a seemingly insignificant but misaligning artifact when rotated at these angles (one row is repeated). Is there any other way of specifying the DA/B/C/D values so that they have the full positive 0x80 out of 0x80 range for these angles?

Thanks.
_________________
(Works for me!)

#3062 - ampz - Mon Feb 17, 2003 11:47 am

Read the GBATEK doc and you'll see that all angles of rotation are possible.
I belive some values like theese will result in 90degrees of rotation counterclockwise:
PA=0
PB=1
PC=-1
PD=0

(untested)

#3090 - pollier - Tue Feb 18, 2003 2:32 am

I looked through there, stuff I already knew, but my problem is that I can't fully specify the number 1; I only get within 0.99996... which is really not a very big difference, but it still bugs me because one of the lines gets repeated and offsets the sprite positions for the rest of the screen.

PA=0
PB=1 <-- how do I do this? closest I get is 0x7fff before I reach negative numbers
PC=-1 <-- this is 0x8000
PD=0

I know, I shouldn't be worrying about something that insignificant for now but it keeps sticking in my mind :(
_________________
(Works for me!)

#3091 - col - Tue Feb 18, 2003 3:26 am

pollier wrote:

PA=0
PB=1 <-- how do I do this? closest I get is 0x7fff before I reach negative numbers
PC=-1 <-- this is 0x8000
PD=0


afair, the hardware uses fixed point with 7bits for the whole part and 8bits for the fractional part!

so 1 is 0x0100
and -1 is 0xff00

cheers

col

#3096 - pollier - Tue Feb 18, 2003 5:51 am

ah, that's what I was doing wrong. I just shifted my LUTs right one bit, and now it works great--Thanks!
_________________
(Works for me!)