#106662 - sgeos - Sun Oct 22, 2006 9:22 am
I want an engine written in C or C++. I'd like to hook a scripting system to that engine. NPC behavior, messages, scripted events, etc will be driven by the scripting system- ie, multiple scripted functions will be called every frame.
What is a good scripting language to use? I'm looking for something that is easy to learn, powerful, and fast enough. Lua is interperted, which kind of scares me, although I have not actually tested it. Is there something that might be better?
-Brendan
#106668 - Sausage Boy - Sun Oct 22, 2006 12:13 pm
Isn't there some kind of compiled lua code? I don't have much experience with scripting languages, but Lua seems pretty nice. I hear Python is good too.
_________________
"no offense, but this is the gayest game ever"
#106673 - sgeos - Sun Oct 22, 2006 1:03 pm
| Sausage Boy wrote: |
| Isn't there some kind of compiled lua code? |
I briefly looked but didn't find anything right away. It seems like there should be.
| Sausage Boy wrote: |
| I don't have much experience with scripting languages, |
Niether do I- why I'm asking. I'll probably take chosen scripting engine out of the box, modify it to use integral arithmetic and then add game engine communication to the scripting engine. (Critters need to move. Print wants to communicate with the engine buffer. Etc, etc.) Unless, of course, this is blatantly stupid for some reason I've overlooked. A proprietary scripting language from scratch seems ... needlessly time consuming and non-portable.
| Sausage Boy wrote: |
| but Lua seems pretty nice. |
Lua looks nice and a lot of people seem to use it. I looked at Lua because I recalled the DSLua project. Unfortunately, being able to touch sprites from the scripting engine is a big no-no. I'm still not sure if Lua is the right choice for a scripting language either.
| Sausage Boy wrote: |
| I hear Python is good too. |
Ruby is also commonly used- or at least seems to be. Python turns up the most google hits. With nothing else to go on, I'm going to guess that there are more python programmers than lua or ruby programmers. (I'll also note that python and ruby are "real" words, which is skewing the results. Googled "XYZ scripting language".)
All of them have Java counterparts, which is a good thing- I don't need to worry about that if I ever do a Java port. I might actually end up porting *from* Java.
I'll script in chosen language until I find someone to take over the scripting. Because I don't know who that person will be, I want something easy to learn. All else being equal, I'll take the most popular or most portable language.
-Brendan
#106685 - tepples - Sun Oct 22, 2006 6:48 pm
| sgeos wrote: |
| I looked at Lua because I recalled the DSLua project. Unfortunately, being able to touch sprites from the scripting engine is a big no-no. |
Then don't expose that interface to Lua. The developer who integrates Lua can show or hide any interface to Lua. DSLua just happens to expose a lot of interfaces because like QBasic, it's intended for the complete implementation of a program from scratch.
| Quote: |
| I'm still not sure if Lua is the right choice for a scripting language either. |
If its interpreter is lighter weight than a lot of scripting languages that have been adapted to "scripting" web applications, then it may be a more "right" choice than those.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#106692 - Yare - Sun Oct 22, 2006 7:38 pm
Lua is about as fast as you can get for a scripting language.
What you want to do is write all of your time-critical stuff in your C++ engine, and then expose engine calls to your Lua scripts.
That's how I designed the engine for my team's current project (PC), and it has worked wonderfully. Every line of our game logic is scripted, and we can use the same engine and toolset to make a completely different game in a very short amount of time.
Let me know if you have any questions.
#106740 - sgeos - Mon Oct 23, 2006 4:00 am
| Yare wrote: |
| Let me know if you have any questions. |
Why did you choose Lua? Are there any pitfalls to be wary of?
| Yare wrote: |
| What you want to do is write all of your time-critical stuff in your C++ engine, and then expose engine calls to your Lua scripts. |
How hard would it be to port your game to a Java based cell phone? (Ie, rewriting the engine. Could the scripts likely be used "as is"?)
| Yare wrote: |
| That's how I designed the engine for my team's current project (PC) |
Does your project have a home page?
-Brendan
#106744 - Yare - Mon Oct 23, 2006 4:33 am
| sgeos wrote: |
Why did you choose Lua? Are there any pitfalls to be wary of?
|
A professor of mine (Steve Rabin, Nintendo of America) has said in several of his books and his lectures that there are 5 levels of data-drivenness, ranging from entirely hard-coded, to entirely scripted. My previous project was around level 2 -we had database files that we loaded for items, but that was it. It was a tactical RPG so we could have made it lot more of it data-driven.
I spent most of the summer thinking about how to divorce the game engine from the game logic while adding a higher level of scriptability. I read up on different scripting languages and Lua stood out because it is extremely lightweight, but meant to be interfaced into C/C++. I decided in order to leverage the full power of an embedded scripting system, my next project would be somewhere between level 4 and 5. That is, only the engine is in C++ and all game logic is in Lua.
There are always pitfalls with any decision you make in game programming. With a scripting language, the biggest drawback is that you generally don't have a debugger that is as good as your C++ debugger. Speed isn't an issue because all critical processes (rendering, pathfinding, line-of-sight calculations) are handled by calling C++ functions.
There are also many ways to interface Lua with your game engine. Some people like to use metatables and userdata to basically use classes in Lua, but I did something very different (keeps the scripts cleaner and makes it easier for non-programmers to write). The implementation is fairly detailed, but I can elaborate (privately) if you wish.
The best part about this engine is that it can be extended to a 3D engine with the addition of a scene graph -not much else has to change.
| sgeos wrote: |
How hard would it be to port your game to a Java based cell phone? (Ie, rewriting the engine. Could the scripts likely be used "as is"?)
|
The only modification the scripts would need is if you wanted to change the position of some graphics because of a difference in screen dimensions. Not even that, if you script it right.
| sgeos wrote: |
Does your project have a home page?
|
My project doesn't have a home page yet, but my college does:
DigiPen Institute of Technology
#106770 - sgeos - Mon Oct 23, 2006 8:31 am
| Yare wrote: |
| I decided in order to leverage the full power of an embedded scripting system, my next project would be somewhere between level 4 and 5. That is, only the engine is in C++ and all game logic is in Lua. |
My goal is to put all data in spreadsheets. Links to scripts will be stored in the spreadsheets as script names. Data tables will be autogenerated from the spreadsheets as part of the build.
One interface script a file seems like the way to go.
| Yare wrote: |
| There are always pitfalls with any decision you make in game programming. With a scripting language, the biggest drawback is that you generally don't have a debugger that is as good as your C++ debugger. |
In a large production your script writers and engine coders will be different people. Your script writers probably won't know how to utilize a debugger, so I'm not sure that this is a big deal. Then again, if one of the scripts has a strange bug that the script writer can't figure out and the engine coder has to track it down... =)
| Yare wrote: |
| Speed isn't an issue because all critical processes (rendering, pathfinding, line-of-sight calculations) are handled by calling C++ functions. |
I'll trust you on this one. Lexical parsing on an embedded system seems slow, but computers are fast these days...
| Yare wrote: |
| There are also many ways to interface Lua with your game engine. Some people like to use metatables and userdata to basically use classes in Lua, |
Scripts need to be associated with objects- monsters, maps, etc. Those associations want to live in documents like the monster and map tables.
What is the cleanest way to for the script and object (map/monster) to communicate? A C/C++ side setActiveObject() call paired with script side getActiveObject() and syncActiveObject() calls is the first thing that springs to mind. The scripts are going to be doing things like changing monster locations, adjusting gravity, spawning items and saving the game.
| Yare wrote: |
| but I did something very different (keeps the scripts cleaner and makes it easier for non-programmers to write). The implementation is fairly detailed, but I can elaborate (privately) if you wish. |
Sounds interesting. I'll send you a private message.
| Yare wrote: |
| sgeos wrote: | | How hard would it be to port your game to a Java based cell phone? (Ie, rewriting the engine. Could the scripts likely be used "as is"?) |
The only modification the scripts would need is if you wanted to change the position of some graphics because of a difference in screen dimensions. Not even that, if you script it right. |
Why do your scripts know what the display size is? (Because a non-programmer wrote the script... nevermind. =) Display strikes me as engine level. IMHO, scripted move camera calls and such should be fetching the screen resolution from the engine.
-Brendan
#106819 - sajiimori - Mon Oct 23, 2006 7:27 pm
You can compile Lua to bytecodes during the build process. It's still interpreted, but you can exclude the lexical analyzer, parser, and code emitter from the runtime environment.
I set up an object-ish interface to C++ classes with the help of the luabind library.
| Code: |
| myActor:setPosition(newPosition) |
The biggest hack I did to Lua was changing it to use fixed point, which was potentially a very error-prone process, but turned out to be painless (if a little tedious) because I luckily got started on the right foot.
Another worthwhile hack is making it a compile-time error to implicitly write to a global from inside a function. Variables are global by default in Lua, and there's a good reason for that, but it leads to some very devious bugs when you forget to type "local" and a pair of functions start overwriting each other's data.
#106847 - sgeos - Tue Oct 24, 2006 3:37 am
| sajiimori wrote: |
| You can compile Lua to bytecodes during the build process. It's still interpreted, but you can exclude the lexical analyzer, parser, and code emitter from the runtime environment. |
And it's interperting bytes, not strings. I'll look into this.
| sajiimori wrote: |
| The biggest hack I did to Lua was changing it to use fixed point, which was potentially a very error-prone process, but turned out to be painless (if a little tedious) because I luckily got started on the right foot. |
I had intended to convert it to int, but fixed sounds better... maybe. It depends on what the scripts want to do. When dealing with fixed, would it be worth having the bytecode compiler convert float to fixed? This way the script writers can express themselves in float, albeit with huge roundoff error.
| sajiimori wrote: |
| Another worthwhile hack is making it a compile-time error to implicitly write to a global from inside a function. Variables are global by default in Lua, and there's a good reason for that, but it leads to some very devious bugs when you forget to type "local" and a pair of functions start overwriting each other's data. |
If I give every critter a different lua state, will this happen? Is giving every critter a different lua state a broken idea? The lua state struct doesn't look very heavy, but I didn't look at it very carefully.
I'll definitely want the printing system to use message IDs instead of message strings.
-Brendan
#106914 - sajiimori - Tue Oct 24, 2006 7:12 pm
The compiler and interpreter are part of one codebase, so changing to fixed-point means it's going to write fixed-point values into the bytecodes. The only part that really has to deal with floating point is lua_str2number, which is used during lexical analysis (and therefore not needed at runtime).
Creating multiple Lua states with lua_newstate is unusual and a giant waste of memory. For instance, "shared" Lua code would have multiple copies in memory, and all your engine hooks would have to be re-exported into each state, which is not cheap.
Creating multiple Lua states with lua_newthread (from C) or coroutine.create (from Lua) is normal, and results in a shared global environment.
Note that there are two equivalent terms for separate Lua states with a shared global environment. The Lua VM internally calls them "threads," whereas the Lua language calls them "coroutines."
The "global by default" issue is not related to threading. The only time it's not an issue is, I guess, if you never call any functions. Then your entire program exists in one scope, so there's no difference between local and global variables, and forgetting to declare a variable "local" can't do any harm. Wouldn't that be nice?? ;)
#106919 - tepples - Tue Oct 24, 2006 7:52 pm
| sajiimori wrote: |
Creating multiple Lua states with lua_newstate is unusual and a giant waste of memory. For instance, "shared" Lua code would have multiple copies in memory, and all your engine hooks would have to be re-exported into each state, which is not cheap.
Creating multiple Lua states with lua_newthread (from C) or coroutine.create (from Lua) is normal, and results in a shared global environment. |
Is this analogous to the difference between processes and threads under multitasking PC operating systems?
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#106925 - sajiimori - Tue Oct 24, 2006 8:36 pm
The difference seems similar, yes.
#106928 - Yare - Tue Oct 24, 2006 10:27 pm
| sgeos wrote: |
If I give every critter a different lua state, will this happen? Is giving every critter a different lua state a broken idea? |
It's certainly messy. Each Lua state is essentially its own program with its own global memory space.
#106935 - sajiimori - Tue Oct 24, 2006 11:19 pm
As I explained, separate Lua states do not necessarily have separate global namespaces.
Besides, "messy" is a strange way to describe a technique that reduces the scope of a namespace. Isolated components seem to be the exact opposite of a mess.
But again, I don't recommend instantiating entire separate VMs. Even if you really wanted separate global namespaces, there are better ways.
In other news, coroutines rock my socks.
#106948 - Yare - Wed Oct 25, 2006 3:11 am
| sajiimori wrote: |
As I explained, separate Lua states do not necessarily have separate global namespaces.
Besides, "messy" is a strange way to describe a technique that reduces the scope of a namespace. Isolated components seem to be the exact opposite of a mess.
But again, I don't recommend instantiating entire separate VMs. Even if you really wanted separate global namespaces, there are better ways.
In other news, coroutines rock my socks. |
Then my understanding of the lua_State system must be flawed. But then, I've only ever needed to use one Lua state for an entire game engine.
I was saying it would be messy since you'd need to expose your engine functions for each lua_State you wanted to use them in, but you've suggested that lua_States work differently than the way I originally thought, so I presume I'm incorrect in that matter as well.
Again, I avoided all of this with the way I integrated lua with my engine so... =)
#106969 - sgeos - Wed Oct 25, 2006 9:48 am
| sajiimori wrote: |
| Creating multiple Lua states with lua_newthread (from C) or coroutine.create (from Lua) is normal, and results in a shared global environment. |
Can I unload modules without recreating the Lua state?
| sajiimori wrote: |
| The "global by default" issue is not related to threading. |
I'm actually looking for a module management strategy. My motion sensitive door has nothing to do with my police officer. I don't need to load the police officer module if he's not on the map, and I'd like to be able to unload him.
| sajiimori wrote: |
| The only time it's not an issue is, I guess, if you never call any functions. Then your entire program exists in one scope, so there's no difference between local and global variables, and forgetting to declare a variable "local" can't do any harm. Wouldn't that be nice?? ;) |
I can't tell if you being sarcastic. If you are serious, my thoughts are- functions were invented for a reason. Not being able to declare/use functions would really frustrate me as a programmer.
On the otherhand, if you are joking, this approach might be worth some serious thought. Most scripts are just function batch jobs with a little logic. The entire script body could exist in one scope, and call service functions that do magical things. Timeline actionscript in flash is like this. It doesn't take a person with mad skills to write these kinds of scripts.
| sajiimori wrote: |
| For instance, "shared" Lua code would have multiple copies in memory, and all your engine hooks would have to be re-exported into each state, which is not cheap. |
Engine hooks could be implemented as byte codes for efficiency. I don't have a solution for "shared" Lua code. In theory you could do without, but that is a really bad approach. I really don't know how much memory bloat multiple copies of shared code cause.
| sajiimori wrote: |
| But again, I don't recommend instantiating entire separate VMs. Even if you really wanted separate global namespaces, there are better ways. |
What I'm really looking for is a way to separate and unload modules. Separate VMs seemed like the obvious solution (after verifying that the lua state struct didn't look too heavy). FWIW, each VM would be given only a small amount of memory to work with.
Given that all of the scripts should be fairly simple, globals didn't really concern me- especially if each script lives in a different VM. Tripping an error is still probably a good idea- one VM or many.
How expensive is it to instantiate a new lua VM?
There are two kinds of scripts- scripts that finishing in one frame, and scripts that span multiple frames. A single frame script for a treasure hunting weapon would look something like this:
| Code: |
damage(target, this.power);
if (isDestroyed(target))
{
gold = getGold(target);
gold += rand(0, gold);
setGold(gold);
drop = getDropRate(target);
setDropRate(2 * drop);
// repeated clone on death is bad
sterilize(target);
} |
This gets treated as a function and executed. Fine, no problems here.
How do I deal with something like this? The script, although fairly simple, takes multiple frames to execute:
| Code: |
portrait = getPortrait(target);
shopId = getShopId(target);
message(portrait, FPV_HAPPY, MID_SHOP_WELCOME); // blocking
enterShop(portrait, FPV_HAPPY, shopId); // blocking
if (0 < getBuyCount())
{
message(portrait, FPV_HAPPY, MID_SHOP_THANKYOUMUCH); // blocking
}
else if (0 < getSellCount())
{
message(portrait, FPV_HAPPY, MID_SHOP_THANKYOU); // blocking
}
else
{
oldBgm = getBgm();
setBgm(BGM_WILDMAN);
shake(SHAKE_NORMAL, SECONDS(0.25)); // blocking
message(portrait, FPV_ANGRY, MID_SHOP_WILLYOUBUY); // blocking
if (yesNo()) // blocking
{
message(portrait, FPV_SAD, MID_SHOP_BUYSOMETHING); // blocking
}
else
{
fadeOut(FADE_NORMAL); // blocking
setBgm(BGM_NONE);
soundEffectStall(SE_SMACK); // blocking
soundEffectStall(SE_SLAP); // blocking
soundEffectStall(SE_POW); // blocking
soundEffectStall(SE_POW); // blocking
soundEffectStall(SE_SLAP); // blocking
message(portrait, FPV_ANGRY, MID_SHOP_NEVERCOMEBACK); // blocking
fadeIn(FADE_CIRCLE); // blocking
}
setBgm(oldBgm);
} |
Coroutines, perhaps?
-Brendan
#107025 - sajiimori - Wed Oct 25, 2006 7:58 pm
Yare,
Lua 5.0 has a built-in language feature for creating multiple Lua states with a shared VM. They're called coroutines, they're available regardless of how you integrate Lua, and they magically make a lot of code design issues disappear. Avoiding them would be a real shame! :)
sgeos,
There's no real concept of unloading a module in Lua. In fact, the term "loading a module" is somewhat misleading as well: all you're doing is running it. For instance, if you have a Lua chunk (that is, a compiled Lua file) in memory, you can have the VM execute it (which is a blocking call), then you're free to delete the chunk after the call returns; the VM is done with it.
Data are garbage collected when nothing refers to them. That includes the bytecodes for a function body. If only one variable refers to a function, then assigning some other value to that variable will allow the function bytecodes to be garbage collected.
If you store everything related to a module in a single table, and store a reference to that table in a single variable, then setting that variable to nil will allow the "module" to be garbage collected. However, if any other variables refer to some part of the table, then that part (and anything it refers to, recursively) can't be collected.
I thought the ;) symbol would make my joking clear in regards to writing programs as one long function, but if you want to, go ahead.
There are many reasons to not instantiate entire separate VMs, but if you want to, go ahead. If you want to know what the cost is, try logging allocations. Re-export all your engine hooks to every VM and keep track of how much memory is used.
Coroutines gracefully solve the multi-tick script problem.
#107064 - Yare - Thu Oct 26, 2006 4:00 am
| sajiimori wrote: |
Yare,
Lua 5.0 has a built-in language feature for creating multiple Lua states with a shared VM. They're called coroutines, they're available regardless of how you integrate Lua, and they magically make a lot of code design issues disappear. Avoiding them would be a real shame! :)
|
I'm familiar with coroutines and the environment concept in Lua. However, I was able to data-drive an entire engine without the need to create multiple lua_States. Coroutines are nice for things like having your AI update itself, but IME most real games need complex load-balancing architectures that don't translate directly to coroutines. That is, the object shouldn't decide when it is done executing, a task manager should decide when it is done executing.
#107114 - sajiimori - Thu Oct 26, 2006 7:46 pm
Ah, but preemptive threading opens up a whole can of worms, doesn't it? Coroutines can be used to your heart's content, with nary a mutex in sight.
One of the nicest things about coroutines is that the client code doesn't know they're calling a coroutine. Normal control flow is retained, and you can transparently substitute a state machine for a coroutine and vice versa.
Preemptive threading is good for system-level tasks (although not so much on the DS), but it's hardly ever appropriate for game logic. It's more important to provide a clean, simple, safe architecture than to optimize game logic.
#107228 - tepples - Fri Oct 27, 2006 9:37 pm
| sajiimori wrote: |
| Preemptive threading is good for system-level tasks (although not so much on the DS), but it's hardly ever appropriate for game logic. |
Then what should be done if path finding or sprite cel or 3D texture/model loading/decompression takes longer than a frame?
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#107247 - sajiimori - Fri Oct 27, 2006 11:48 pm
I break up pathfinding using coroutines.
For large cart reads, parallelization can be good because the DS supports asynchronous reads.
Decompression could be done as a thread, but there's little point because the ARM9 will be completely busy while the decompression thread is active, so there's not much room to do anything in the background. You may as well use a state machine or coroutine and avoid all the extra complexity.
#107250 - tepples - Sat Oct 28, 2006 12:38 am
| sajiimori wrote: |
| For large cart reads, parallelization can be good because the DS supports asynchronous reads. |
Using which homebrew FAT library? Or only using the NitroSDK?
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
#107251 - sajiimori - Sat Oct 28, 2006 12:42 am
I haven't used any homebrew DS software.