#152454 - dheine - Sun Mar 16, 2008 9:18 am
Hello,
I'm about to update Mr Jandujars DSGo with current GnuGO sources using his playing surface. Well the requirements for the heap and stacks have not grown less:
/usr/local/devkitPro/devkitARM/lib/gcc/arm-eabi/4.1.1/../../../../arm-eabi/bin/ld: address 0x24e26b0 of build.elf section .bss is not within region ewram
collect2: ld returned 1 exit status
Just 4 fun I extended ewram in ds_arm9.ld to 8MB and voila the build loads in desmume emulator and blanks the screen ! However Jandujars dsGo 1.2 works there.
So I'm not sure if all my debugging work in phase 1 was already in vain. Are there too many build objects ? Lies the problem in the huge GO tables with static and/or dynamic pattern data ? (3MB)
I tried to find similiar problems here and there is one:
http://forum.gbadev.org/viewtopic.php?t=13055&highlight=ewram
However the message for ewram meant full when it was full. I got a wrong address not within region.
Still some hope for a new GnuGO 4 NDS ?
_________________
with kind regards, euer Dieter, still or again 8th kyu GO ranking
Only ... "Beginners play atari"
( http://senseis.xmp.net/?path=GoProverbs&page=BeginnersPlayAtari )
#152492 - Dwedit - Sun Mar 16, 2008 5:49 pm
Yeah... the address you gave is about 5MB into main memory, and main memory is 4MB large.
_________________
"We are merely sprites that dance at the beck and call of our button pressing overlord."
#153022 - dheine - Mon Mar 24, 2008 9:08 am
Hello,
GnuGO uses several MBs of pattern matching game tables in opposite to chess that has few 100 Ks of opening tables. 4MB would not be enough. So the 32MB of the cartdrige RAM are required for GnuGO pattern matching of the thinking engine.
Maybe the only topic that helps me would be Lick's extended RAM API. First I was surprised while reading here about so many capacity problems whithout mentioning cartridge extensions in most cases. I felt that most people doesnot like it for performance reason because they develop fast games. However GnuGO and Chess are "thinking machines". Performance is only for secondary prio.
My final question therefor should be if Lick's extra library is the only way to access memory above 32M anyway ? I understood that the linker even with ds_cart.specs cannot link my M3 RAM to the main RAM and therefor no compiler directive wouldnot give direct access to it (right ?).
P.S: I wonder how the DSLinux people managed the 32M extension that is shown by the free command. GnuGO is a typical Linux project. Maybe it is easier to use DSLinux to realize GnuGO.
_________________
with kind regards, euer Dieter, still or again 8th kyu GO ranking
Only ... "Beginners play atari"
( http://senseis.xmp.net/?path=GoProverbs&page=BeginnersPlayAtari )
#153027 - antiaverage - Mon Mar 24, 2008 11:45 am
I would be all over this if you completed it. I've been hoping for a decent Go player for a while, and GNUGo would be excellent. You should definitely be looking into using the external RAM cartridge. Lick's extended RAM API is definitely the way to go. I would prefer if you could do this in an nds and not just get it running under DSLinux.
#153029 - TwentySeven - Mon Mar 24, 2008 12:10 pm
(research: dldi and libfat)
Well if you're just pattern matching, I assume each pattern is actually fairly small.
Why not just page in a bit of the table at a time?
What it sounds suspiciously like is you're still thinking in GBA land where you need to fit the tables into "rom" which is accessed from the upper regions of address space.
The DS doesn't work that way, anything you want to read in has to come in over a "IO" layer like libfat.
#153035 - M3d10n - Mon Mar 24, 2008 1:18 pm
You should definitely consider breaking that table into multiple files and dynamically load/unload them as you need them.
As you just said, it's a thinking game, the extra time spent in traversing the tables sounds much better than having to require an extra RAM cart for a table game.
Even if you use the RAM expansion, you'll still need to keep your table in a file and load it manually, since your ARM9 binary cannot exceed 4MBs.
#153037 - dheine - Mon Mar 24, 2008 1:51 pm
Uuuh, nice 3 answers while I had some lunch and already searched for lick's RAM_API. The lunch was fine but the search for ram_api 1.3 looks a bit in vain ;-(
Though I agree for some kind with the latter 2 suggestions I would be still interessted not to waste internal M3 RAM.
However both of you should consider that your hint silently require to rewrite GnuGO coding. Yes there is some kind of pattern db in GnuGO. But I'm not so sure as you are because I'm no so sure about memory mgmt inside a complex program. I prefered your GBAlandish model because every future GnuGO update would be easyier to adapt: namely as long as GnuGO doesnot offer interfaces for file/memory mgmt itself as part of its PC version.
Since many patterns tables change during every move (e.g. opening, attack, defense ...) now I'd like to know how performance of libfat compares to the one with RAM ext. ?
_________________
with kind regards, euer Dieter, still or again 8th kyu GO ranking
Only ... "Beginners play atari"
( http://senseis.xmp.net/?path=GoProverbs&page=BeginnersPlayAtari )
#153044 - dheine - Mon Mar 24, 2008 3:30 pm
Hello ,
Now I got some infos about the trade-off:
GnuGO builds without EWRAM restriction an 4,5 MB ELF-Binary (that wont fit). I guess during a game 1/2M should be additional available. The patterns have 2.8M from 25 files with more than 25 tables .
These patterns increase steadily during GnuGOs development since GO is _the_ pattern matching game since 4000 years.
On the other hand we have 108 .c-files with more than 300 functions. All calling different patterns at different situations.
My GBAlandish approach was to load all patterns at the beginning while filling their struct pointers as it would be like having enough space. The program starts almost without any modification at this point.
A Game has typically between 150-170 moves (only of the machine) each matching 100s of patterns.
I'm not sure but exchanging those 25 patterns 1000 times in memory via libfat requires one or more complete new handler for disloading and reloading the patterns . There is not only one central matching function nor one only pattern structure.
Sorry, I will have to ask the GnuGO dev-scene about a GnuGO-MMU that make sense in this case.
_________________
with kind regards, euer Dieter, still or again 8th kyu GO ranking
Only ... "Beginners play atari"
( http://senseis.xmp.net/?path=GoProverbs&page=BeginnersPlayAtari )
#153050 - antiaverage - Mon Mar 24, 2008 4:29 pm
While that seems a bit of work, I just want you to know there are people, myself included, who really appreciate what you're doing. Please continue to pursue this, as an up-to-date playable Go game on the DS would be excellent. Really just playing against other humans would be enough, but if you end up porting GNUGo, you're my hero. Thank you.
#153063 - dheine - Mon Mar 24, 2008 6:29 pm
Too much honour, although I really missed the game and searched many places not only sourceforge. It sounds a bit silly, but man against machine seems to me the first alternative. Man2Man should be best from Jandujars DSGo, 1.2 where it is already usable. Of course only on the same device player beside player and very limited.
At the end of eastern our discussion here in comparison with all possible new outviews revealed following ranking to me:
1. DSLinux: comes with the best development environment for 32 linear memory to compile the original tarball from GnuGO. No problem within hours. Together with cgoban everything goes.
2. GnuGO with the RAM_API: Unfortunately the ram_api from lickcr.org is missing (lickcr.org is missing) . It would be great help to make ram_api available. It means less work without original code modification. I cannot see why it is slower than 3):
3. GnuGO with pattern overlay memory region as it was recommended here. The charme would be a gnugo with only 4MB extending the radius to more slot-1 devices. This means really work because I have to find out when to drop the big fuseki patterns and how the remaining puzzle of patterns works for attack, defense, eyeshapes, josekis ... endgame (many more)
_________________
with kind regards, euer Dieter, still or again 8th kyu GO ranking
Only ... "Beginners play atari"
( http://senseis.xmp.net/?path=GoProverbs&page=BeginnersPlayAtari )
#153075 - sonny_jim - Mon Mar 24, 2008 7:48 pm
| dheine wrote: |
1. DSLinux: comes with the best development environment for 32 linear memory to compile the original tarball from GnuGO. No problem within hours. Together with cgoban everything goes.
|
Pretty much the only benefit of DSLinux is that it's unbelievably easy to port existing Linux applications to it. Here is a quick and dirty port that seems to work, needs some kind of extra RAM and built with the following compile time options:
| Code: |
--enable-cache-size=1 --enable-semeai-node-limit=5 --enable-owl-node-limit=5 --disable-dfa --disable-chinese-rules --disable-experimental-break-in --disable-cosmic-gnugo --disable-large-scale
|
If it works as expected I'll get it integrated into the main builds
#153080 - simonjhall - Mon Mar 24, 2008 9:00 pm
Well that sounds easy! Pretty much nullifies what I'm about to say ;-)
Yeah I would have gone for a disk-based approach (by modifying the original code to only load bits of the tables on demand) which then wouldn't require extra RAM. It'll probably pay off in the end. Getting non-trivial programs to use slot-2 RAM is such a *chore* that you'll probably need to have access to all the different RAM cards to confirm that it works the way you expect. Each flash card has its own quirks...esp the M3!
_________________
Big thanks to everyone who donated for Quake2
#153094 - tepples - Mon Mar 24, 2008 10:06 pm
| dheine wrote: |
| Uuuh, nice 3 answers while I had some lunch and already searched for lick's RAM_API. The lunch was fine but the search for ram_api 1.3 looks a bit in vain ;-( |
RAM API 1.3 is contained in its entirety inside memtestARM.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#153365 - dheine - Sat Mar 29, 2008 5:56 am
Hello ,
back again after a short week but hard days work at my company
And gbadev.org there is blocked by our firewall of course.
I was taken by surprise when I read that gnugo 3.x already exists. I followd the link to rapidshare and 2 surprises followed up: Neither does it contain sources nor does it start on my dslinux built from 11/07. It complains about bad magic from its BINFMT_FLAT : 0 where 4 is needed.
Will I have to compile using the complete linux toolchain or is there a way to overcome only the bad file magic ?
_________________
with kind regards, euer Dieter, still or again 8th kyu GO ranking
Only ... "Beginners play atari"
( http://senseis.xmp.net/?path=GoProverbs&page=BeginnersPlayAtari )
#153374 - sonny_jim - Sat Mar 29, 2008 1:10 pm
Erm, it should just work? Have you tried using a more recent build from Kineox? I take it you did unpack it properly? How are you executing it?
To be honest, it was just a quick build that I knocked together and I was quite surprised that it worked. If you want a diff against the sources let me know and I'll provide them.
#153790 - dheine - Sun Apr 06, 2008 9:41 am
sorry for beeing absent a week but last weekend I was a bit frustrated while making my first DSLinux build. I struggled more with microwin/pixil sources until I realized my first goal to compile only the text interface of gnugo 3.7.11
I compared with the compileflags given here but wanted to try to include DFA pattern matching. Means that I tried to use all original flags.
Making mkpat as the requited pattern compiler (subdir patterns) fails because the linker complains about multiple definitions of an int array (meaningless-white-moves) that is included several times by liberty.h in several c-files . Means to me that my x86 gcc overcomes such things by less restrictions during compilation because there it works. For me the arm compiler is right because in include files there should only be declations and no definitions that allocate an array more than one time.
Any ideas or compiler/linker options ?
_________________
with kind regards, euer Dieter, still or again 8th kyu GO ranking
Only ... "Beginners play atari"
( http://senseis.xmp.net/?path=GoProverbs&page=BeginnersPlayAtari )
#153793 - dheine - Sun Apr 06, 2008 10:11 am
Think I solved linker difference between x86 and arm: The int[BOARDSIZE] definitions are not static althought they are global. For beeing really portable this looks to me like a small mistake in the original gnugo source.
Hoever I understand now the -no-dfa flag given here in the forum when it disables the mkpat call during crosscompiling. Now I will use these flags but I wonder what gnugo means without some of its intelligent pattern matching routines. Since mkpat during compilation only turns pattern-DBs into c-files I see no reason why mkpat compilation shouldnot be replaced by its x86 part .
_________________
with kind regards, euer Dieter, still or again 8th kyu GO ranking
Only ... "Beginners play atari"
( http://senseis.xmp.net/?path=GoProverbs&page=BeginnersPlayAtari )
#153794 - dheine - Sun Apr 06, 2008 1:09 pm
After using configure with all given options the compiler/linker phase still tries to mkpat for generating c-files from the pattern database.
_________________
with kind regards, euer Dieter, still or again 8th kyu GO ranking
Only ... "Beginners play atari"
( http://senseis.xmp.net/?path=GoProverbs&page=BeginnersPlayAtari )
#153798 - dheine - Sun Apr 06, 2008 3:23 pm
After exchanging all standalone pattern generators against x86 versions and adjusting Makefile by commenting their creation out you may try the result from:
http://rapidshare.com/files/105322354/gnugo.html
It is also not tested well. But I was not sure to be the first who ports gnugo this way. Hence it starts playing in my dslinux from 7/07 means that came from DSLinux as binary with M3 Support. The configure flags have been used but I'll try to port it as close to the original as possible.
_________________
with kind regards, euer Dieter, still or again 8th kyu GO ranking
Only ... "Beginners play atari"
( http://senseis.xmp.net/?path=GoProverbs&page=BeginnersPlayAtari )
#153814 - dheine - Sun Apr 06, 2008 9:54 pm
Finally I don't want to conceal that gnugo is actually still too slow to enjoy playing it. Even on 13x13 and level 5 every move takes between 2-5 minutes. This means for a single game more than 2 hours only for the NDS without the time the user needs for every move. I tried to change cache size but I have no idea how to speed up efficiently. I really would like to know how "Many faces of GO" (Igo) has been brought down to NDS. Maybe the scret lies in using both processors arm7 and 9 concurrently. Interesting enough is memory consumption. It's about 13 MB in dslinux. Since the biggest pattern samples belong to the most important ones there should be no real chance to get by on less memory e.g. without extra RAM pack. Only libfat may not be a solution.
_________________
with kind regards, euer Dieter, still or again 8th kyu GO ranking
Only ... "Beginners play atari"
( http://senseis.xmp.net/?path=GoProverbs&page=BeginnersPlayAtari )
#153825 - TwentySeven - Sun Apr 06, 2008 11:30 pm
Isnt the extra ram in things like the M3 incredibly slow?
#153912 - dheine - Tue Apr 08, 2008 8:10 am
| TwentySeven wrote: |
| Isnt the extra ram in things like the M3 incredibly slow? |
Any Idea about other extra RAM ? I believe many people waited for a fast porting not for a fast game.Also I think that move duration should be compared to games like DSChess. Yes, DSChess doesnot need extra RAM and is slow when it is good.
_________________
with kind regards, euer Dieter, still or again 8th kyu GO ranking
Only ... "Beginners play atari"
( http://senseis.xmp.net/?path=GoProverbs&page=BeginnersPlayAtari )
#153914 - simonjhall - Tue Apr 08, 2008 8:58 am
All external RAM is slow, but the internal stuff is also pretty slow too. I think that if it's taking minutes per move when run from external memory then it's also going to take minutes per move when run from internal memory! Do you have data caching of external memory turned on btw? That may help a little.
_________________
Big thanks to everyone who donated for Quake2
#153919 - dheine - Tue Apr 08, 2008 12:01 pm
yes while my precursor here took 1M for gnugo 3.6 I extended to 3MB but crashed with 8MB. Performance depends first on the commandline option for gnugo --level 1-10 . I use level 5 on13x13 and still get reasonable moves in 2.5 minutes. There should be more control about the steps the engine climbs until a move is found. These steps need to be made visible which seems to be quite possible with an open source development. More profiling should be made on x86-gnugo but its a problem that no arm emulator emulates RAM card extensions. Maybe I can pump up my desmume emulator with PC-RAM only for glancing at the effects while manipulating arm-gnugo.
I enabled the new dfa pattern matching without knowing any performance effects. The limits for owl and semeai nodes seem to be rather critical and I know pretty less about it. My precursor may have more information about the critical options. However fast enough is saving and reloading of existing games so that a game maybe resumed when I enter tram or bus again on my way back home.
However my next project is of course CGoBan for grafical frontend especially for 2-humans mode and of course remote connect to another fast GO playing machine.
_________________
with kind regards, euer Dieter, still or again 8th kyu GO ranking
Only ... "Beginners play atari"
( http://senseis.xmp.net/?path=GoProverbs&page=BeginnersPlayAtari )