SquirrelGRF: Squirrel scripting for GRF-like files

Forum for technical discussions regarding development. If you have a general suggestion, problem or comment, please use one of the other forums.

Moderator: OpenTTD Developers

Moriarty
Tycoon
Tycoon
Posts: 1395
Joined: 12 Jun 2004 00:37
Location: United Kingdom of Great Britain and Northern Ireland
Contact:

Re: SquirrelGRF: Squirrel scripting for GRF-like files

Post by Moriarty »

Just a reminder on the matter of TTDPatch compatibility - The folks over in the "ottd graphics" section of this board are working away (slowly but surely) on creating nice shiny 32bit graphics with super-zoom and details and who knows what else. Those graphics won't be compatible with TTDPatch as I understand it, so this represents a perfect time to the recreate the GRF language from scratch into something mere-mortals can dabble with.
DaleStan
TTDPatch Developer
TTDPatch Developer
Posts: 10285
Joined: 18 Feb 2004 03:06
Contact:

Re: SquirrelGRF: Squirrel scripting for GRF-like files

Post by DaleStan »

cirdan wrote:...
Please post example code for some other actions and features.
Notably, actions 2 and 6, station property 09, and industry property 0A.

Also, are you volunteering to keep this SquirrelGRF reader in sync with the NFO reader? Or is that going to get foisted off on the OpenTTD devs as soon as they accept your patch?


frosch wrote:I like the analogy of a microcontroller when talking about newgrfs: E.g. though microcontrollers can be programmed in C or in assembler or whatever, no language can help you to setup a timer or a PWM generator correctly.

In summary it is quite wrong to think of newgrfs as of some interpreted language. It is more like some language that initializes peripheral elements of some microcontroller.
Somewhat more verbosely:

A GRF is a program in about the same sense that libstdc is a "program". It is a library containing around fifty functions[0] and a huge collection of static data.

But wait. It gets worse. A GRF is like libstdc in about the same sense that libstdc is like its sources.
There's a fancy thing ... we'll call it a "preprocessor" ... that is Turing complete, save that memory is non-infinite, and that decides which functions and data tables get exported.
This preprocessor has read/write access to 512 bytes of memory that the functions can read (but not write), for whatever purpose they please, and it has write-only access to the data tables and the functions' code.

But wait. It gets worse. All functions have access to 1 KB of read-write memory, and 64 bytes of write-only memory, but all of that memory is zeroed after the function returns. All other memory is, at best read-only. Some memory is reference-only -- the function can return a pointer to the memory, but cannot dereference the pointer.

But wait. It gets worse. Each function can return at least two different types, possibly three[1], and not all functions have the same set of return types. The "correct" return type depends on context, but a function is free to return an "incorrect" type, if it feels like doing so.

There. I think I covered everything.

[0] Give or take an order of magnitude, usually.
[1] It depends on whether you count 8-bit integer and 15-bit integer as separate types. Yes, fifteen. I did not mean to type sixteen.
To get a good answer, ask a Smart Question. Similarly, if you want a bug fixed, write a Useful Bug Report. No TTDPatch crashlog? Then follow directions.
Projects: NFORenum (download) | PlaneSet (Website) | grfcodec (download) | grfdebug.log parser
User avatar
Korenn
Tycoon
Tycoon
Posts: 1735
Joined: 26 Mar 2004 01:27
Location: Netherlands
Contact:

Re: SquirrelGRF: Squirrel scripting for GRF-like files

Post by Korenn »

Hey, I'm one of those who's actually *for* having a way to create GRF files (or similar) through a higher abstraction level. I don't care by what means it's done, so if Squirrel works, then I'm all for it.

But DaleStan, despite his unfriendly tone, has a point. Before actually jumping into implementation, you should try and define how to implement each Action or Varaction from NFO, or how to do things with similar functionality. Changing properties is the simplest thing, but how will you handle callbacks? and livery overrides? or base sprites, for that matter?
User avatar
cirdan
Director
Director
Posts: 539
Joined: 07 Apr 2007 18:08

Re: SquirrelGRF: Squirrel scripting for GRF-like files

Post by cirdan »

First of all, thanks to all that have contributed to the thread so far. To tell you the truth, I expected it to degenerate real fast and I'm surprised it hasn't yet.
frosch wrote:To give you some numbers I did for george to optimise his grfs:
http://www.math.tu-clausthal.de/~mtce/o ... 081222.txt
http://www.math.tu-clausthal.de/~mtce/o ... tailed.txt
Note that those are only industries, there are a lot more houses. And some vehicles callbacks are called even more often.
Unfortunately, I'm afraid those numbers are currently of little use to me. It's not how often callbacks are used that matters, but how much slower callbacks in Squirrel would be compared to NFO. A factor of 2 or 3 would not be that bad (just wait a year or two and let Moore's Law take care of it), while anything greater would. Also, keep in mind that performance-critical GRFs will still be able to be coded in NFO.
frosch wrote:Also note that there are fundamental differences between newgrf callbacks and an imperative language.
- There are no global storage variables.
- I.e. houses have only one 8 bit storage, which can only be written in a few strictly defined situations.
- There are no API functions at all to get e.g. some random value. You can always only access a few variables, which might be restricted to reading/writing in various cases.
And to be honest, your example shows only how to realise some simple action0. And you can already find two languages which can deal with that in the TTDP graphics forum. Defining a callback is something completely different, especially as writing down which variables the callback shall use, what to compute with them, and what to return, is hardly the difficult part of newgrf.
I like the analogy of a microcontroller when talking about newgrfs: E.g. though microcontrollers can be programmed in C or in assembler or whatever, no language can help you to setup a timer or a PWM generator correctly. (Edit: Or as that might sound to easy. Imagine to setup a USB port on a descriptor level)
Thanks for your insight. It is evident that your knowledge on this subject is far deeper than mine (I've never tried to setup a USB port, whether on a descriptor level or otherwise; that's what I have an OS kernel for. :-) ). And I'm not trying to "code NewGRF in an imperative language"--perhaps I haven't made this point clear enough--but to have a scripting language usable for more or less the same things, even if its inner workings are totally different. What gave me the impression that this could be possible was, ironically, this claim that NFO is Turing-complete. I thought that anything doable in NFO should be doable in another Turing-complete language; but I could be wrong.
frosch wrote:In summary it is quite wrong to think of newgrfs as of some interpreted language. It is more like some language that initializes peripheral elements of some microcontroller.
I'm not thinking of NewGRF as an interpreted language, which it is not. I'm thinking of an interpreted language that could be used to achieve similar goals.
DaleStan wrote:Please post example code for some other actions and features.
Notably, actions 2 and 6, station property 09, and industry property 0A.
What, exactly, would be the problem with those actions/properties?

Also, in case I haven't made this sufficiently clear: I'm not saying that the new system should be able to do everything NFO does. It may have a reduced feature set, so GRF coders can choose to use it if they only need basic features, or can write NFO if they need advanced features.
DaleStan wrote:Also, are you volunteering to keep this SquirrelGRF reader in sync with the NFO reader? Or is that going to get foisted off on the OpenTTD devs as soon as they accept your patch?
The previous paragraph applies. And, personally, I don't think that adding new features to this interpreter would be any harder than adding them to the NewGRF interpreter.
DaleStan wrote:Somewhat more verbosely:
...
I understand that NewGRFs are extremely complex. But that's fine, since I'm not trying to mimic their behaviour, but to do a similar thing in a different way.

Brief summary: We've already seen the arguments "you won't be able to do as much as with NFO", "you won't be able to do things as they are done with NFO", and "you won't be able to do things as well as with NFO", which are valid concerns. However, even if those predictions are right (which is something, in my opinion, hard to perfectly foresee now, but anyway), they don't invalidate the usefulness of a new coding system. Not every possible GRF requires the full power of NFO; if the less complex GRFs can be coded in a scripting language, that's already a benefit.

As I already said, the question is: Does making GRF coding easier, for at least some (not necessarily all) GRFs, warrant the addition of an interpreter in OpenTTD?
frosch
OpenTTD Developer
OpenTTD Developer
Posts: 988
Joined: 20 Dec 2006 13:31
Location: Aschaffenburg

Re: SquirrelGRF: Squirrel scripting for GRF-like files

Post by frosch »

cirdan wrote:Thanks for your insight. It is evident that your knowledge on this subject is far deeper than mine (I've never tried to setup a USB port, whether on a descriptor level or otherwise; that's what I have an OS kernel for. :-) ).
Err, well, I am not able to do that either. But my point was that no language can help you to understand using animation-frames-as-general-storage, production-multipliers, sprite-layouts or refit-cycles.
cirdan wrote:Not every possible GRF requires the full power of NFO; if the less complex GRFs can be coded in a scripting language, that's already a benefit.
Isn't that what grfmaker is for? But anyway, there are not many interested in easy grfs. Rather they become more complicated all the time :p

And also wrt. "easy grfs". Maybe you should change that into "grfs with common behaviour". In that case you could help andythenorth/zephyris/DJNekkid&Co to create tutorials with standard code for e.g. standard vehicles.

Edit: corrected quoting and grammatic
⢇⡸⢸⠢⡇⡇⢎⡁⢎⡱⢸⡱⢸⣭⠀⢸⢜⢸⢸⣀⢸⣀⢸⣭⢸⡱⠀⢰⠭⡆⣫⠰⣉⢸⢸⠀⢰⠭⡆⡯⡆⢹⠁⠀⢐⠰⡁
Eddi
Tycoon
Tycoon
Posts: 8257
Joined: 17 Jan 2007 00:14

Re: SquirrelGRF: Squirrel scripting for GRF-like files

Post by Eddi »

cirdan wrote:What gave me the impression that this could be possible was, ironically, this claim that NFO is Turing-complete. I thought that anything doable in NFO should be doable in another Turing-complete language; but I could be wrong.
just because it can be done does not mean it will be anywhere near efficient.

just take calculating vs. counting with fingers.

it is proven that a counting automaton with two stacks [aka. 2 hands with infinite amount of fingers] is turing complete.

(a counting automaton is a special push down automaton which has only one stack symbol, and can do the following operations on the stack:
increase the stack value by 1 (push)
decrease the stack value by 1 (pop)
check if the stack is 0 (empty)
)
DaleStan
TTDPatch Developer
TTDPatch Developer
Posts: 10285
Joined: 18 Feb 2004 03:06
Contact:

Re: SquirrelGRF: Squirrel scripting for GRF-like files

Post by DaleStan »

cirdan wrote:What gave me the impression that this could be possible was, ironically, this claim that NFO is Turing-complete.
I misspoke. One pass -- the preprocessing pass (which is also the self-modifying pass) -- would be Turing-complete if there were enough memory. Callbacks and sprite lookups are decidedly not Turing complete.
cirdan wrote:
DaleStan wrote:Please post example code for some other actions and features.
Notably, actions 2 and 6, station property 09, and industry property 0A.
What, exactly, would be the problem with those actions/properties?
Write the example code and find out for yourself. Or write the example code, discover it's blindingly easy, and ask me what I'm smoking.
cirdan wrote:[the new system] may have a reduced feature set, so GRF coders can choose to use it if they only need basic features, or can write NFO if they need advanced features.
Name one GRF that uses only the features that the current version of SquirrelGRF supports. Got one? Good. I got one too. Now name a second. Go on. Try.
cirdan wrote:I don't think that adding new features to this interpreter would be any harder than adding them to the NewGRF interpreter.
Except that now the work has to be done twice. In my book, that counts as "more than twice as hard".
cirdan wrote:Brief summary: We've already seen the arguments "you won't be able to do as much as with NFO" [and others], which are valid concerns. However, even if those predictions are right (which is something, in my opinion, hard to perfectly foresee) they don't invalidate the usefulness of a new coding system.
What. What!? It's hard to "perfectly forsee" that your SquirrelGRF completely fails to have any method of bloody SPECIFYING SPRITES? And this doesn't make SquirrelGRF something rather closely resembling "completely useless"?
cirdan wrote:As I already said, the question is: Does making GRF coding easier, for at least some (not necessarily all) GRFs, warrant the addition of an interpreter in OpenTTD?
Not while "at least some GRFs" means "exactly one GRF".

Stop wasting your time defending your "I kan do actinz z3r0 and ate!!!11!!one1!!eleventyone!!!1" toy language and try turning it into a real language that supports more than two of the twenty actions. (2 of 23 if you also count RealSprites, RecolorSprites, and BinaryIncludes, but they're not actions, strictly speaking.)
To get a good answer, ask a Smart Question. Similarly, if you want a bug fixed, write a Useful Bug Report. No TTDPatch crashlog? Then follow directions.
Projects: NFORenum (download) | PlaneSet (Website) | grfcodec (download) | grfdebug.log parser
User avatar
AndersI
Tycoon
Tycoon
Posts: 1732
Joined: 19 Apr 2004 20:09
Location: Sweden
Contact:

Re: SquirrelGRF: Squirrel scripting for GRF-like files

Post by AndersI »

cirdan wrote:Not every possible GRF requires the full power of NFO; if the less complex GRFs can be coded in a scripting language, that's already a benefit.
If you're on Windows, this 'simpler' system already exists - it's called GRFMaker. It's a GUI with 'canned dialogs' for a subset of the possible NFO codes. As it is also possible to embed plain NFO bytes, you can do almost anything with it, but when you come to that point it is probably better to code NFO directly. But you don't have to ditch your GRFMaker project when you come to your first small experiments with plain NFO.

It definitely is a good help for fixing the offsets of the sprites, and there are sets out there which are made with GRFMaker, so it's not a 'worthless toy'. I have only used it for trains, so I have no idea how suited it is for the other kinds (planes, ships, houses, terrain etc.).

What's missing to make it 'easy' are lots of examples. That goes (in a slightly lesser degree) for plain NFO too.
User avatar
Ralph
Engineer
Engineer
Posts: 87
Joined: 21 Jun 2004 15:25

Re: SquirrelGRF: Squirrel scripting for GRF-like files

Post by Ralph »

Kudos to cirdan for trying this, and while it is only a demo to show how a solution might work, I still feel it makes more sense than NFO does.

This may be because one of my programming experiences was QuakeC, and have since also enjoyed making stuff for NWN using NWScript. So to be frank, when presented with the idea of having to clunk hexdecimal together to add content to the game, I was a bit taken aback.
User avatar
cirdan
Director
Director
Posts: 539
Joined: 07 Apr 2007 18:08

Re: SquirrelGRF: Squirrel scripting for GRF-like files

Post by cirdan »

DaleStan wrote:...
I withdraw my words about having so far a civilised discussion. Perhaps it was precisely my saying so that led you to think that you were not being hostile enough?
DaleStan wrote:Stop wasting your time defending your "I kan do actinz z3r0 and ate!!!11!!one1!!eleventyone!!!1" toy language and try turning it into a real language that supports more than two of the twenty actions.
Polite and friendly as usual.
1. I, and only I, decide what I waste my time on, whether you like it or not.
2. Please respect me as I respect you, or anybody else for that matter.

AndersI wrote:If you're on Windows, this 'simpler' system already exists - it's called GRFMaker. It's a GUI with 'canned dialogs' for a subset of the possible NFO codes. As it is also possible to embed plain NFO bytes, you can do almost anything with it, but when you come to that point it is probably better to code NFO directly. But you don't have to ditch your GRFMaker project when you come to your first small experiments with plain NFO.
Unfortunately (or, rather, fortunately), I'm not on Windows. Anyway, I'm not doing all this thinking of myself; strange as it may seem on this forum, I don't use any GRFs (1) in my games. I started this for the benefit of those wishing to code GRFs, a set that does not include myself. This is the reason why I really have no use for GRFMaker, and why I don't have a strong interest in seeing this patch through either: it won't directly affect me in any way (yes, that too is uncommon). I'm mainly doing this for fun, just because I like coding and I thought this was a nice project to tackle, and because I'm tired of hearing people say that it cannot be done.

(1) Pedantic note that some may need: Other than the system GRFs. "A word to the wise shall be sufficient."
AndersI wrote:It definitely is a good help for fixing the offsets of the sprites, and there are sets out there which are made with GRFMaker, so it's not a 'worthless toy'.
I'm sure it isn't, if it scratches your itch.
AndersI wrote:What's missing to make it 'easy' are lots of examples. That goes (in a slightly lesser degree) for plain NFO too.
I don't think that is the only reason. Sure, there's a lack of examples, but writing hex codes by hand is not a pleasant task. It is like writing a computer program in machine code, by reading the processor specs; no amount of examples is going to make me feel confortable about that.
Ralph wrote:This may be because one of my programming experiences was QuakeC, and have since also enjoyed making stuff for NWN using NWScript. So to be frank, when presented with the idea of having to clunk hexdecimal together to add content to the game, I was a bit taken aback.
I don't know these programs, but I do know similar cases, and I agree that using an extension language in "machine code" is not the best option in 2009. It may have been in the 80s, when machine time was more important than human time, and people were expected to make things easier for the computer, and I can understand why it was chosen for TTDPatch at the moment it was, and now OpenTTD has to live with that, but this is not a reason not to make people's lives easier from now on. I think a scripting language is the way to go in the long run.

----------

Anyway, I'm attaching a new version of the patch, which is growing at a steady rate. I've made the following changes in the API:

· Tar support is here! You can throw a .tar file into your data/ directory and it will be scanned for .nut files. All right, I admit this isn't my work, it is just because of OpenTTD's .tar handling, so thank the devs for this. But it happens to save me the tedious work of adding explicit .tar support, since it is needed to properly pack the .nut file along with the sprites it needs, making it easier to distribute files. Since this came as an "accidental feature", I'm not certain which tarfiles do work; it seems that a file foo.tar in which all files are under directory foo/ works well.

· RailVehicleChangeSpeed has been replaced by SetRailVehicleStats, with the syntax

Code: Select all

SetRailVehicleStats (0, { speed=100, power=200, weight=300, sprite=0xFD } );
I hope this change in the API doesn't cause major inconveniences. :-) Only speed, power, weight and sprite can be changed, for now. And don't try to turn an engine into a wagon, or vice versa; it still doesn't work.

· New functions DefineRailSpriteSet(), DefineRailSpriteGroup() and MapRailSpriteGroup() have been introduced, with the syntax

Code: Select all

DefineRailSpriteSet ( [ "file1", "file2", ... ] );
DefineRailSpriteGroup ( [0], [0], setid);
MapRailSpriteGroup (engineid, setid);
DefineRailSpriteSet() defines a new sprite set (in NFO speech: action 1), with as many views as entries in the array. DefineRailSpriteGroup() defines a new graphics set, with the named setid (action 2). Finally, MapRailSpriteGroup() assigns a graphics set id to an engine (action 3).

Caveats:

· Currently the code tries to share as many paths as possible with the existing NewGRF code, and this imposes some limitations (that will be lifted in the future). Most notably, a DefineRailSpriteGroup() must immediately follow a DefineRailSpriteSet(), and must only refer to the sprite sets defined there (at the moment only one), and must do so by their relative position (necessarily 0). In practical terms, this means that sprite groups defined must have only one loaded stage and one loading stage, and both must share the same sprite set. Improving this, while planned, requires more changes than I'm willing to risk at the moment.

· Also, the sprites named in DefineRailSpriteSet() are not actually loaded at all. I'm working on this, but OpenTTD does not currently have an easy way (or I haven't found it) to have a standalone file as a main sprite, since it expects some kind of header. For now, the default fallback sprite is substituted, until I can fix this.

The attached sample.tar file, as its name suggests, is a small file that exercises the above functions. It replaces the Kirby Paul Tank (again) with a new locomotive whose graphics I borrowed from the OpenGFX project (thanks!) and shuffles its stats. Of course, the actual graphics won't show because of what I've just said, but you can have a nice red question mark wandering around.

A word of caution: There are almost no sanity checks in the functions exported to Squirrel, so use them with care.

DaleStan wrote:Write the example code and find out for yourself. Or write the example code, discover it's blindingly easy, and ask me what I'm smoking.
I'll oblige. What are you smoking?
Attachments
squirrel-grf-r15113.patch
Patch against r15113
(17.64 KiB) Downloaded 72 times
sample.tar
Sample file
(20 KiB) Downloaded 60 times
User avatar
Korenn
Tycoon
Tycoon
Posts: 1735
Joined: 26 Mar 2004 01:27
Location: Netherlands
Contact:

Re: SquirrelGRF: Squirrel scripting for GRF-like files

Post by Korenn »

I suggest you completely ignore DaleStan's remarks until he stops posting destructive comments, and starts posting constructive comments instead. It has happened before :D

In the mean time, great work! Getting those sprites loaded will be a big hurdle.
frosch
OpenTTD Developer
OpenTTD Developer
Posts: 988
Joined: 20 Dec 2006 13:31
Location: Aschaffenburg

Re: SquirrelGRF: Squirrel scripting for GRF-like files

Post by frosch »

cirdan wrote:New functions DefineRailSpriteSet(), DefineRailSpriteGroup() and MapRailSpriteGroup() have been introduced, with the syntax
Now you start to confuse me. Are you now heading for a 1:1 translation of grf actions to functions? I.e. the squirrel code is only executed on graphics initialisation and not while the game is running?

In that case: Why do you want to patch OTTD for that? You could just as well print out the resulting nfo, and pipe it through nforenum and grfcodec. Btw. adding inline-nfo is quite simple that way.

E.g. see attached Gnu R script (supporting VarAction2 chains, Action7 and D; the rest via inline). I used that about a year ago to start writing a industry newgrf, which I never finished.
Also attached is a little snapshot of the newgrf code, though it is not finished at all, and likely I stopped while editing, so it might not even run.

Why I choosed Gnu R? I cannot remember the reasoning anymore :? But as I bothered adding binary operators, there must have been some reason :)
Attachments
nfobase.R.txt
some nfo library for Gnu R
(4.9 KiB) Downloaded 57 times
example.R.txt
some example using the library
(22.04 KiB) Downloaded 77 times
⢇⡸⢸⠢⡇⡇⢎⡁⢎⡱⢸⡱⢸⣭⠀⢸⢜⢸⢸⣀⢸⣀⢸⣭⢸⡱⠀⢰⠭⡆⣫⠰⣉⢸⢸⠀⢰⠭⡆⡯⡆⢹⠁⠀⢐⠰⡁
User avatar
ostlandr
Chairman
Chairman
Posts: 882
Joined: 12 May 2007 01:09
Location: Northeastern USA

Re: SquirrelGRF: Squirrel scripting for GRF-like files

Post by ostlandr »

Okay. (stops, takes deep breaths)

1. Awesome work by Cirdan on this.

2. The whole point of the thread was that Cirdan actually coded something rather than just saying "Why doesn't somebody do (X)?"

3. The whole point of programming languages in the first place was to save programmers from having to type endless strings of zeroes and ones. What the language is isn't as important as what the language does- help the programmer get their mind around what the code is doing. Yes, you can get by with minimalist languages- an excellent example is here: http://en.wikipedia.org/wiki/Brainf***

4. Not being a programmer, I find it hard (read: impossible) to get my limited brain around the code. With much time and effort spent, I could probably figure out how to code NFO.

5. If somebody like Cirdan wants to write an NFO compiler for lolcode, I would probably be able to do some things I have wanted to do with GRFs. (Note: this is just a fictional example, please do not expect correct variable names, etc. Of course one would need permission from the author(s) to use the sprites, etc.)

Code: Select all

hai
i can haz nfolib kthx
i can haz grfid 123456  kthx
i iz in ur grf changin thingz
vehid 01 ur te iz 200
vehid 01 ur power iz 200
vehid 01 ur intro_yr iz 2004
vehid 01 ur speed iz 100
vehid 01 ur veh_name iz "Porta Jinty"
savin new grfid 123457
kthxbye
6. Assuming the above is possible, would it not then be theoretically possible (not necessarily practical, or desirable, but theoretically possible) for some brilliant and bored coder to create a GUI to do the same things? (I.E. grfmaker on steroids)
Let's start with a truly ludicrous example. Someone on the forum gets the idea of making a GRF that converts all the original vehicle sprites to athletic shoes. If this theoretical GUI existed, they could draw or otherwise create sprites of athletic shoes, use the GUI to get them properly aligned into bounding boxes, and test them until they looked right (should they slide along the road/rail, or hover above it?) Oh, Converse Chuck Taylor all-stars should be refittable freight wagons in company colour? No problem, the GUI can do that, too. Want to tweak the intro date of the Nike Air? No problem- drop downs, menu boxes, radio buttons, etc.

7. Coming up with ways to make all this stuff easier and more accessible doesn't in any way diminish all the hard work of the sprite coders and GRF programmers.

8. Cirdan is talking about using the capabilities of OTTD as it currently exists to do this, not demanding that the devs do even one keystroke of extra work. If the devs change something and it kills squirrelgrf, than it's up to Cirdan, or whoever chooses to pick up the banner, to change squirrelgrf to match the new code, or else it will die.
Who is John Galt?
DaleStan
TTDPatch Developer
TTDPatch Developer
Posts: 10285
Joined: 18 Feb 2004 03:06
Contact:

Re: SquirrelGRF: Squirrel scripting for GRF-like files

Post by DaleStan »

cirdan wrote:
DaleStan wrote:Write the example code and find out for yourself. Or write the example code, discover it's blindingly easy, and ask me what I'm smoking.
I'll oblige. What are you smoking?
OK. Let's see if I can figure it out:
The Action2 portion of the spec, according to the wiki dump I downloaded a few days ago, is 181 KB. By my count, you have demonstrated support for approximately 9.46 KB of that (~5%). Once you s/Rail/RoadVeh/g, s/Rail/Aircraft/g, and s/Rail/Ship/g, those numbers will run up to 13.5 KB and 7.5%.

And you haven't at all touched stations or industries, nor action 6.

I think you need a bit more evidence before claiming that my drug of choice is notably more powerful than water.
ostlandr wrote:8. Cirdan is talking about using the capabilities of OTTD as it currently exists to do this, not demanding that the devs do even one keystroke of extra work. If the devs change something and it kills squirrelgrf, than it's up to Cirdan, or whoever chooses to pick up the banner, to change squirrelgrf to match the new code, or else it will die.
And as long as this holds, SquirrelGRF will remain essentially useless, because anyone who wants to use a SquirrelGRF will have to compile their own version of OpenTTD first.

If it gets committed, then the devs will be responsible for the code, and responsible for not-breaking it themselves.
To get a good answer, ask a Smart Question. Similarly, if you want a bug fixed, write a Useful Bug Report. No TTDPatch crashlog? Then follow directions.
Projects: NFORenum (download) | PlaneSet (Website) | grfcodec (download) | grfdebug.log parser
maquinista
Tycoon
Tycoon
Posts: 1828
Joined: 10 Jul 2006 00:43
Location: Spain

Re: SquirrelGRF: Squirrel scripting for GRF-like files

Post by maquinista »

I like this idea. It's easier than NFO coding.
Sorry if my english is too poor, I want learn it, but it isn't too easy.[/list][/size]
User avatar
cirdan
Director
Director
Posts: 539
Joined: 07 Apr 2007 18:08

Re: SquirrelGRF: Squirrel scripting for GRF-like files

Post by cirdan »

frosch wrote:Now you start to confuse me. Are you now heading for a 1:1 translation of grf actions to functions? I.e. the squirrel code is only executed on graphics initialisation and not while the game is running?
No, I'm not heading for a bijective mapping of grf actions to functions, but I thought that coding the easier actions would be a good start. And yes, the squirrel code, at the moment, is only executed on graphics initialisation; in the future, I expect that callbacks can be implemented in Squirrel, but that's something to address when, and if, we reach it.

The main problem is that OpenTTD's internal structures are, in some ways, tightly coupled with the NFO format, which is something hard to overcome at first; as I said, I'm trying to reuse as many code paths within OpenTTD as possible, to reduce patch management. Take, for instance, struct ResultSpriteGroup in src/newgrf_spritegroup.h; it effectively imposes that all sprites used in the group are sequential, since only the pair (first,number) is stored. Overcoming this would require more modifications than I currently want to do.

I'm trying to keep a balance between adding features to the patch (to show its possibilities) and keeping it simple and maintainable: Every time a dev makes a commit in trunk, there's the possibility that it breaks my patch, so I'm keeping it as compact as possible to reduce the probability of a conflict.
frosch wrote:In that case: Why do you want to patch OTTD for that? You could just as well print out the resulting nfo, and pipe it through nforenum and grfcodec. Btw. adding inline-nfo is quite simple that way.
Well, the new version of the patch, which I'm attaching (see below), already allows you to do certain things that are impossible in NFO, such as having a spritegroup with different-sized spritesets (not a great deal, but anyway).

The point is that I don't have a master plan for this patch; I have some guidelines of what it should and shouldn't be, but not a picture of the final result. In fact, if you like the patch, please feel free to throw in your suggestions about how the language should be. API changes are easier when there's not much API to change. And your suggestions will likely be better than mine. :-)
ostlandr wrote:3. The whole point of programming languages in the first place was to save programmers from having to type endless strings of zeroes and ones. What the language is isn't as important as what the language does- help the programmer get their mind around what the code is doing. Yes, you can get by with minimalist languages- an excellent example is here: http://en.wikipedia.org/wiki/Brainf***
There are people out there that find this difficult to understand; to them, there is apparently no difference between writing C and programming a Turing machine.
ostlandr wrote:4. Not being a programmer, I find it hard (read: impossible) to get my limited brain around the code. With much time and effort spent, I could probably figure out how to code NFO.
Well, I myself am a programmer, and still find it hard and error-prone.
DaleStan wrote:I think you need a bit more evidence before claiming that my drug of choice is notably more powerful than water.
I claimed nothing. You requested that I asked that question if I found the example code was blindingly easy, and I did. Next time don't ask for things you don't want to happen. And need I remind you that you only asked for example code?

----------

Today's version of the patch features:

· Sprite loading has arrived. :-) The patch can now succesfully load sprites specified via DefineRailSpriteSet(). This means that you can already define rail vehicles in SquirrelGRF. I'm attaching a new sample.tar that replaces my last one, which won't work because the sprites weren't in the correct format (indexed PNG).

Keep in mind that the initial "hackish" warning still applies; currently, the sprites are loaded bypassing the deferred-loading mechanism of spritecache.cpp, although I hope that this won't be noticeable during gameplay.

· The DefineRailSpriteSet() function has had its syntax enhanced. You can now do

Code: Select all

DefineRailSpriteSet([ ["foo/sprite1.png",-3,-4], ["foo/sprite2.png",-6,-10], ... ])
to specify sprite offsets for the loaded files, in case they aren't set in a text chunk (see PNGCodec) or to override them if they are.

· The interplay between DefineRailSpriteSet(), DefineRailSpriteGroup() and MapRailSpriteGroup() has been overhauled, and now works in a more sensible way. The syntax is

Code: Select all

local s1 = DefineRailSpriteSet ([ ... ]);
local s2 = DefineRailSpriteSet ([ ... ]);
local s3 = DefineRailSpriteSet ([ ... ]);

local sg = DefineRailSpriteGroup ([s1,s2], [s3]);

MapRailSpriteGroup (0, sg);
This would load three sprite sets (s1, s2 and s3), then pack them into a spritegroup (sg), with the first two of them used for loaded stages and the last one during loading, and finally assign the spritegroup to engine 0. This means you can use whatever spritesets you want in a spritegroup, not only the last one (as long as they have been defined), and that load stages are supported. Additionally, you no longer explicitly assign group-ids to the spritegroups; instead, you store the result of DefineRailSpriteGroup() and use that.

As usual, I'm attaching a new sample file to show the new features. Your Kirby Paul Tanks will now look like a certain OpenGFX engine while in transit, and a different one at stations, just for fun.
Attachments
squirrel-grf-r15130.patch
(24.19 KiB) Downloaded 76 times
sample.tar
(30 KiB) Downloaded 35 times
DaleStan
TTDPatch Developer
TTDPatch Developer
Posts: 10285
Joined: 18 Feb 2004 03:06
Contact:

Re: SquirrelGRF: Squirrel scripting for GRF-like files

Post by DaleStan »

cirdan wrote:And need I remind you that you only asked for example code?
When I ask for example code for $THING I mean "Convince me that your language supports all of $THING, or can easily be made to do so." I'm convinced that SquirrelGRF can easily be made to support 7.5% (counted by size of pages in the wiki dump) of the action 2 spec. That's approximately as useful as presenting Hello World as an example what your C compiler can do.
To get a good answer, ask a Smart Question. Similarly, if you want a bug fixed, write a Useful Bug Report. No TTDPatch crashlog? Then follow directions.
Projects: NFORenum (download) | PlaneSet (Website) | grfcodec (download) | grfdebug.log parser
AndyLandy
Engineer
Engineer
Posts: 30
Joined: 10 Sep 2008 20:09

Re: SquirrelGRF: Squirrel scripting for GRF-like files

Post by AndyLandy »

The biggest issue here is the frequently-quoted observation from Patchman -- Why not NFO? NFO might be foul to look at, and that's why I (and others) are looking at a higher-level abstraction of it.

That being said, there are some big wins to be had from a complete replacement. NFO grew out of TTDPatch, which grew out of manipulating the original TTD game. OpenTTD is something totally different, it can be changed internally to make new features much easier to implement. You could almost certainly do away with a lot of the callbacks if you changed internal structures in OpenTTD. Here are a few examples:

Livery changes for refit/time progression. If the code to do this was in OpenTTD, you could simply provide an API to specify "here are default graphics, here are graphics for this other cargo, here are graphics to switch to after date N"

The same is true of sound effects. In a vehicle definition, you just say "Here are the sounds to use for driving/stopped/broken down"

You could also do away with the base-value/multiplier way of specifying running costs. Simply specify an initial running cost (Assume normal difficulty with no inflation) and OpenTTD can work out what to multiply these by for hard mode/N-years of inflation. You could also get away from the "plane speeds are an 8-bit number, multiplied by 8" thing, and just specify a top speed in MPH and be done with.

I've no idea if you could eliminate all the callbacks, or if you'd even want to. DaleStan seems to be pretty knowledgeable about the more in-depth features of what NFO can do, I'd be interested to hear his take on this.

So, yes, replacing NFO isn't an unreasonable proposition (If people didn't invent new languages, we'd still be coding in COBOL and FORTRAN), but it has to be worthwhile.

Also, don't underestimate the work involved in such a project. It's very much a pipe-dream at the moment, I don't have time to pull apart the internals of OpenTTD and work out what makes it tick, much as I'd like to.
User avatar
Ralph
Engineer
Engineer
Posts: 87
Joined: 21 Jun 2004 15:25

Re: SquirrelGRF: Squirrel scripting for GRF-like files

Post by Ralph »

On the API: The NoAI API is nice and object orientated, and while this could be implemented in Squirrel code using proceedual functions (another layer yeah!), having the API understand objects for locomotives/industries/ect would make sense.
User avatar
CommanderZ
Tycoon
Tycoon
Posts: 1872
Joined: 07 Apr 2008 18:29
Location: Czech Republic
Contact:

Re: SquirrelGRF: Squirrel scripting for GRF-like files

Post by CommanderZ »

That's approximately as useful as presenting Hello World as an example what your C compiler can do.
I think that was exactly what he wanted to do :)

I agree with the people who are saying, that current state of GRF coding is way too hardcore. The argument, that we wouldn't have most of today's applications without higher level languages surely makes sense. The clearly IS lack of GRF coders, especially when it comes to more complicated applications like station/industry coding...there are so many graphics "waiting to be coded". RED*STARs gorgeous stations aren't still coded, are they? Shame. At the same time, there are many C++ capable programmers willing to help, but scared by the unfriendliness of GRF coding.

These indices clearly point, that potent replacement for GRF would be benefitial, but I don't think patching OTTD is the way to go. The disadvantages are clear:
1) Code duplication in OTTD in case current standart was kept alive (which is likely)
2) Incompatibility with TTDP -> artists might be unwilling to release graphics only for OTTD (I can't imagine michael blunk releasing OTTD-only graphics) -> limited support
3) Significantly increased CPU load (with callbacks) can be expected

External something->NFO converter is IMO the way to go, it has in fact only advantages:
1) Separated development - the project is not dependent on will of OTTD devs to include it into trunk
2) Both OTTD and TTD compatible
3) If written competently, the new code could be combined with pure NFO - just as C can be seamlessly combined with ASM, so it could be used even before all/most features were implemented
Post Reply

Return to “OpenTTD Development”

Who is online

Users browsing this forum: Google [Bot] and 14 guests