GRL (GRF Language)

[Alberth]

Hello all,

Inventing a new language for GRF seems a popular hobby here, so here is my version
Everything is early in development and largely untested, including the examples.

The program is a console utility written in C++ at a Linux system. It takes a .grl file, and outputs a .nfo file.
Documentation and source code is attached here.

[Eddi]

hm, i had a look at your definition

looks like you have a slightly different design goal than me, you were more or less replacing each nfo action with a textual description

i was aiming at a higher level of abstraction, e.g. the "feature" did not appear anywhere in my specification, it gets implicitly defined by which original object you were replacing, graphics will automatically get "linked" to the object they are specified in

also, in my plan, callbacks will look like functions in most regular programming languages, a programmer in a higher level language doesn't have to know what "registers" are

this approach generally needs a higher amount of analysis and transformations in the compiler

one last comment: using "-" in identifiers is a really bad idea

[DaleStan]

GRL looks like a 1:1 transform between actions and GRL primitives; that is (excluding grl-bytes) that every NFO sprite has exactly one corresponding primitive, and every primitive has exactly one corresponding sprite.

For this type of replacement, the standard stumbling block is action 6, specifically the <offset> entity. In NFO you just use the offset of the byte(s) that need(s) to be changed. In any other language, you can't do that, as byte offsets aren't readily accessible.

I'll second the "no hyphens in identifiers" comment.

[Alberth]

hm, i had a look at your definition

looks like you have a slightly different design goal than me, you were more or less replacing each nfo action with a textual description

I consider this a first step to get acquainted with the problems and challenges in NFO. GRL is a step forward imho compared to NFO (less hexadecimal coding to do) while still being able to express everything.

i was aiming at a higher level of abstraction, e.g. the "feature" did not appear anywhere in my specification, it gets implicitly defined by which original object you were replacing, graphics will automatically get "linked" to the object they are specified in

So what do you do when I make something completely new then? Somewhere I will have to tell whether it is a train or a ship.

I completely agree with you that key=value is not what you want (that's why it is a first step). The problem I see happening at some point in time though is that the higher level language is not completely supporting what a user wants, either because he wants something that the higher level language was not designed for, or because GRF and NFO were extended in some direction that the higher level language has not been developed for (eg the new airport branch in ottd). In that case, I don't want to be forced back to NFO-like coding.
In other words, my key=value stuff is intended as fallback to the higher level language.

At the same time, I get to understand the NFO language, and I develop code that understands how to make a NFO sprite from a set of values. That should make development of higher level primitives easier.
Last but not least, in this way I get a framework where I can put new primitives in and always be able to create complete NFO files without having to code some parts in NFO.

In other words, I don't exclude such higher level primitives as the next step, I merely added an additional step, namely getting some feeling for where the tricky parts are going to be, and at the same time developing useful (both for me and for the community (I hope)).

also, in my plan, callbacks will look like functions in most regular programming languages, a programmer in a higher level language doesn't have to know what "registers" are

I am also quite annoyed at the reversed action2+3 nodes. Function-like syntax sounds like a good idea.
Do you have a nice syntax in mind?

this approach generally needs a higher amount of analysis and transformations in the compiler

yep, and that's a good thing. The computer should go figure out what to generate rather than me.

one last comment: using "-" in identifiers is a really bad idea

I had some doubts about this one. I still think it looks very nice, but you are probably right, I should eliminate it.

[Alberth]

GRL looks like a 1:1 transform between actions and GRL primitives; that is (excluding grl-bytes) that every NFO sprite has exactly one corresponding primitive, and every primitive has exactly one corresponding sprite.

True in spirit.The main idea of this version is to eliminate the calculations from a set of field-values to a sequence of hexadecimal coded bytes.
Not true in the details. The 'sprite-sets' primitive includes not only the action 1 sprite but also the sprites themselves, and it generates thus a sequence of NFO sprites.

For this type of replacement, the standard stumbling block is action 6, specifically the <offset> entity. In NFO you just use the offset of the byte(s) that need(s) to be changed. In any other language, you can't do that, as byte offsets aren't readily accessible.

Also in NFO you first have to count how many bytes offset there, before you can fill in the field. The offset does not magically comes falling out of the sky.
Therefore, a working but not very nice solution would be the require a numerical offset from the user.
I think a better solution would be to make the value for 'offset' field equal to the name of the key of the field in the next sprite whose value you want to change.
For example 'offset = feature' means that I want the offset of the 'feature' field in the next node.

I'll second the "no hyphens in identifiers" comment.

Noted.

[DaleStan]

For this type of replacement, the standard stumbling block is action 6, specifically the <offset> entity. In NFO you just use the offset of the byte(s) that need(s) to be changed. In any other language, you can't do that, as byte offsets aren't readily accessible.

Also in NFO you first have to count how many bytes offset there, before you can fill in the field. The offset does not magically comes falling out of the sky.

True. But I can count bytes in NFO. I can't count the bytes in GRL, because the bytes aren't there.

Another similar issue comes with wanting to write word-sized values into an extended-byte field. Will the translator be intelligent enough to detect this and use the 3-byte extended byte encoding, even if the supplied value fits into a single byte?

[Bilbo]

For offsets, you could use labels and the numerical offset would be calculated during translation to NFO. And for extended bytes, the translator can have tables telling which properties of which actions are bytes, words, extended bytes, dwords or whatever, so it can correctly put the correct type ....

I personally like this proposal more than NDL - it have nicer syntax (no python syntax with enforced indenting which may not always be the best) and "better" compiler (c++ is available on more systems than python)

[DaleStan]

And for extended bytes, the translator can have tables telling which properties of which actions are bytes, words, extended bytes, dwords or whatever

You missed the question:

Given the key-value pair "extended_byte_property = 0", how should the value be encoded?
If preceded by "action6 { param = 0 count = 2 offset = extended_byte_property }", then how should the value be encoded?
Will this work properly?

For more fun and games, offset is itself extended, and I have (albeit only once) used one action 6 to modify the following action 6.

[Eddi]

no python syntax with enforced indenting which may not always be the best

syntax is the easiest thing to replace

if people are really against indenting syntax, one can switch to a bracket based block organisation ('begin'...'end' like in pascal, or '{'...'}' like in C)

personally, i found that the only disadvantage of python style indenting is that it totally screws up if you mix tabs and spaces

[eis_os]

I think you need or should have a two pass system.

A high level language shouldn't depend that much on the low level part.

This means, the Action 6 definition should work transparent, this means the compiler needs to know the size it writes, the offsets and give them names. If the user needs to know for the action6 how the "thing" is internally stored there is something wrong. How the "thing" is stored should be up to the compiler. If something need to write to "thing" via action6 then the compiler may opt to always use the long form. Again thats a compiler internal and shouldn't be in the language definition...

[Bilbo]

Given the key-value pair "extended_byte_property = 0", how should the value be encoded?
If preceded by "action6 { param = 0 count = 2 offset = extended_byte_property }", then how should the value be encoded?
Will this work properly?

For more fun and games, offset is itself extended, and I have (albeit only once) used one action 6 to modify the following action 6.

Oh, I see, so the extended byte won't be just constant parameter but in fact a pre-initialized variable. And then it would be importat whether it occupies 1 or 3 bytes. This could be solved by explicit typecasting, like
"extended_byte_property = (extended_wide)0" telling the compiler to use 3 bytes even for small values.

[Alberth]

Another similar issue comes with wanting to write word-sized values into an extended-byte field. Will the translator be intelligent enough to detect this and use the 3-byte extended byte encoding, even if the supplied value fits into a single byte?

I can see that this is what you'd like to have with an extended byte and some self-modifying code.
With all the actions that I currently support, I encode extended bytes with maximum efficiency (since their values are all fixed). With adding support for action6, this concept becomes broken.

The minimal requirement would be that the compiler complains about existence of such a problem, so the user is at least aware of it.
The next level is to provide a work-around for the user, so that at least he can continue working until the compiler is fixed. For that situation, a type-cast along the lines of what Bilbo suggested would be nice (tnx for this! it is a simple generic solution for a potentionally nasty problem).
The best solution would be what eis_os suggest, namely letting the compiler figure it out. In terms of the type-cast that means that the compiler will insert the type-case rather than the user.

[Alberth]

no python syntax with enforced indenting which may not always be the best

syntax is the easiest thing to replace

True. However, take into account that people judge stuff w.r.t. what it currently provides, not what it can be (just like you said that my GRL is only a seperation into a set of key/value pairs).
(ie both you and Bilbo are right, the starting points were different though).

personally, i found that the only disadvantage of python style indenting is that it totally screws up if you mix tabs and spaces

I stopped using tabs about a year ago, and that works great, until you start doing programming in OTTD where they only use tabs
I also have occasional problems with line-continuations (the \<NL> token) in Python. It makes sense in Python as full language, but it may be something to watch out for in your application.

[Alberth]

This means, the Action 6 definition should work transparent, this means the compiler needs to know the size it writes, the offsets and give them names. If the user needs to know for the action6 how the "thing" is internally stored there is something wrong. How the "thing" is stored should be up to the compiler. If something need to write to "thing" via action6 then the compiler may opt to always use the long form. Again thats a compiler internal and shouldn't be in the language definition...

The problem imho is not the size of the replacement, it's the existence of the Action6. The level of abstraction between Action6 and the other Actions does not match. It's a bit like having a toolbox filled with hammers, screwdrivers, pincers, pliers, and tucked away at the bottom a nuke, for those special cases that all else fails.

Existence of Action6 is a sympton that a problem needed to be solved which was so big that the best workable solution for NFO was to allow self-modifying code.

The right solution for GRL imho would be to consider this action itself a compiler internal rather than exposing it to a user. That however leads to the question which higher level problem exists for which Action6 is the best possible answer. I would need to being able to express that problem in my GRL language, then have the compiler generate an Action6 as needed.

[eis_os]

As example you have to abstract GRF Resource Management.. Again, you can still give the user this kind of sledgehammer if your compiler has some kind of internal design, that each stuff you write has some kind of addressable nature and state, so you can access that from your action 6.

This means, you need to know before you write the action6 how the other actions look.

However to be able to make your compiler internal action6 know where it should store something like GRM offsets, it needs to know how an action internally will be written and the offsets anyway.


Or well you do the complete reverse:
An action can create an action6 in front itself to change itself, or if you like perl, do both
Josef would surely love this...

The joys of compiler construction

[AndersI]

The right solution for GRL imho would be to consider this action itself a compiler internal rather than exposing it to a user. That however leads to the question which higher level problem exists for which Action6 is the best possible answer. I would need to being able to express that problem in my GRL language, then have the compiler generate an Action6 as needed.

This sounds like the right approach to me. Something like

Code: Select all

        id = Byte(Parameter(2)),

or

Code: Select all

        value = 0020+Word(Parameter(2)),

Now the Action6 is implicit instead of explicit. There are of course many other possible ways to use Action6, but who says that your GRL must be able to express existing GRF/NFO files? Isn't it enough if GRL can be used to create new GRF:s?

[Bilbo]

In terms of the type-cast that means that the compiler will insert the type-case rather than the user.

Well, compiler should know what is expected (whether extended byte, word, byte, etc...) and use it, UNLESS user specifies otherwise. Therefore if user says dword, put there a dword even if spec says byte :) (extended byte short (1 or 3 byte according to value) and 3-byte would be then basically the "same" type) Perhaps just print some warning if user chooses something incompatible (either specs may perhaps change from what the compiler thinks is correct, or user is trying to do some advanced trick :) ...

[Bilbo]

Now the Action6 is implicit instead of explicit. There are of course many other possible ways to use Action6, but who says that your GRL must be able to express existing GRF/NFO files? Isn't it enough if GRL can be used to create new GRF:s?

If there would be some feature not possible to be done using GRL and you need that feature, you have to stick with old-style NFO for your entire project. This would be quite a bit of disadvantage ....

[AndersI]

If there would be some feature not possible to be done using GRL and you need that feature, you have to stick with old-style NFO for your entire project. This would be quite a bit of disadvantage ....

As the grl-bytes is included in the definition, you should be able to express anything a plain NFO can (by using plain old NFO commands).

[DaleStan]

If there would be some feature not possible to be done using GRL and you need that feature, you have to stick with old-style NFO for your entire project. This would be quite a bit of disadvantage ....

As the grl-bytes is included in the definition, you should be able to express anything a plain NFO can (by using plain old NFO commands).

Well, yes and no. Encoding action 6 with grl-bytes (currently the only way to do it, but I expect this will change) also requires that the following sprite be a grl-bytes. If this is an action 2, then all sprites it references, and all sprites that reference it, must also be grl-bytes, unless grl-bytes parser has quite a bit of intelligence, and grl-bytes is allowed to do things that I wouldn't expect in grl-bytes.

This sounds like the right approach to me. Something like

Code: Select all

        id = Byte(Parameter(2)),

or

Code: Select all

        value = 0020+Word(Parameter(2)),

I see two (or maybe 3) problems with this:
1) How do you conditionally skip the action 6?
1b) (Corollary) How do you specify the value that is used when the 6 isn't?
2) What if you want to modify something that is implied by the grl, not explicit?

These questions come from a convoluted double-tac-nuke I pulled some years ago: I wrote an action 6 that made three modifications, except that the third modification used parameter FF. (Making it the terminator, not a modification, and making the following 3 bytes "extraneous".) I then used another 6 to change the FF to the desired parameter number, and a 9 to skip the first 6 if I didn't want the second 6 to make its third modification at all.

Something like

Code: Select all

* 0   09 00 01 \7> 02 01
* 0   06 00 01 07 FF
* 0   06 01 01 08 01 01 0C FF 01 05 FF

(The offsets in the second 6 may not have any realistic use. I know the one I wrote did what I intended for it to do.)