Transport Tycoon Forums

The place to talk about Transport Tycoon
It is currently Wed Mar 20, 2019 6:55 pm

All times are UTC




Post new topic  Reply to topic  [ 38 posts ]  Go to page 1 2 Next
Author Message
 Post subject: GRL (GRF Language)
PostPosted: Sun Dec 30, 2007 12:16 pm 
Offline
OpenTTD Developer
OpenTTD Developer

Joined: Sun Sep 09, 2007 5:03 am
Posts: 4694
Location: home
Hello all,

Inventing a new language for GRF seems a popular hobby here, so here is my version :)
Everything is early in development and largely untested, including the examples.

The program is a console utility written in C++ at a Linux system. It takes a .grl file, and outputs a .nfo file.
Documentation and source code is attached here.


Attachments:
File comment: documentation
grldoc.pdf [134.87 KiB]
Downloaded 200 times
File comment: conversion prog source code
grl-0.0.1.tar.gz [202.44 KiB]
Downloaded 85 times
Top
   
 Post subject: Re: GRL (GRF Language)
PostPosted: Sun Dec 30, 2007 1:59 pm 
Offline
Tycoon
Tycoon

Joined: Wed Jan 17, 2007 12:14 am
Posts: 7300
hm, i had a look at your definition

looks like you have a slightly different design goal than me, you were more or less replacing each nfo action with a textual description

i was aiming at a higher level of abstraction, e.g. the "feature" did not appear anywhere in my specification, it gets implicitly defined by which original object you were replacing, graphics will automatically get "linked" to the object they are specified in

also, in my plan, callbacks will look like functions in most regular programming languages, a programmer in a higher level language doesn't have to know what "registers" are

this approach generally needs a higher amount of analysis and transformations in the compiler

one last comment: using "-" in identifiers is a really bad idea

_________________
You might not exactly be interested in Ferion, but if you are, have fun :)


Top
   
 Post subject: Re: GRL (GRF Language)
PostPosted: Sun Dec 30, 2007 3:47 pm 
Offline
TTDPatch Developer
TTDPatch Developer

Joined: Wed Feb 18, 2004 3:06 am
Posts: 10285
GRL looks like a 1:1 transform between actions and GRL primitives; that is (excluding grl-bytes) that every NFO sprite has exactly one corresponding primitive, and every primitive has exactly one corresponding sprite.

For this type of replacement, the standard stumbling block is action 6, specifically the <offset> entity. In NFO you just use the offset of the byte(s) that need(s) to be changed. In any other language, you can't do that, as byte offsets aren't readily accessible.

I'll second the "no hyphens in identifiers" comment.

_________________
To get a good answer, ask a Smart Question. Similarly, if you want a bug fixed, write a Useful Bug Report. No TTDPatch crashlog? Then follow directions.
Projects: NFORenum (download) | PlaneSet (Website) | grfcodec (download) | grfdebug.log parser


Top
   
 Post subject: Re: GRL (GRF Language)
PostPosted: Sun Dec 30, 2007 4:02 pm 
Offline
OpenTTD Developer
OpenTTD Developer

Joined: Sun Sep 09, 2007 5:03 am
Posts: 4694
Location: home
Eddi wrote:
hm, i had a look at your definition

looks like you have a slightly different design goal than me, you were more or less replacing each nfo action with a textual description

I consider this a first step to get acquainted with the problems and challenges in NFO. GRL is a step forward imho compared to NFO (less hexadecimal coding to do) while still being able to express everything.

Eddi wrote:
i was aiming at a higher level of abstraction, e.g. the "feature" did not appear anywhere in my specification, it gets implicitly defined by which original object you were replacing, graphics will automatically get "linked" to the object they are specified in

So what do you do when I make something completely new then? Somewhere I will have to tell whether it is a train or a ship.

I completely agree with you that key=value is not what you want (that's why it is a first step). The problem I see happening at some point in time though is that the higher level language is not completely supporting what a user wants, either because he wants something that the higher level language was not designed for, or because GRF and NFO were extended in some direction that the higher level language has not been developed for (eg the new airport branch in ottd). In that case, I don't want to be forced back to NFO-like coding.
In other words, my key=value stuff is intended as fallback to the higher level language.

At the same time, I get to understand the NFO language, and I develop code that understands how to make a NFO sprite from a set of values. That should make development of higher level primitives easier.
Last but not least, in this way I get a framework where I can put new primitives in and always be able to create complete NFO files without having to code some parts in NFO.

In other words, I don't exclude such higher level primitives as the next step, I merely added an additional step, namely getting some feeling for where the tricky parts are going to be, and at the same time developing useful (both for me and for the community (I hope)).

Eddi wrote:
also, in my plan, callbacks will look like functions in most regular programming languages, a programmer in a higher level language doesn't have to know what "registers" are

I am also quite annoyed at the reversed action2+3 nodes. Function-like syntax sounds like a good idea.
Do you have a nice syntax in mind?

Eddi wrote:
this approach generally needs a higher amount of analysis and transformations in the compiler

yep, and that's a good thing. The computer should go figure out what to generate rather than me.

Eddi wrote:
one last comment: using "-" in identifiers is a really bad idea

I had some doubts about this one. I still think it looks very nice, but you are probably right, I should eliminate it.


Top
   
 Post subject: Re: GRL (GRF Language)
PostPosted: Sun Dec 30, 2007 4:22 pm 
Offline
OpenTTD Developer
OpenTTD Developer

Joined: Sun Sep 09, 2007 5:03 am
Posts: 4694
Location: home
DaleStan wrote:
GRL looks like a 1:1 transform between actions and GRL primitives; that is (excluding grl-bytes) that every NFO sprite has exactly one corresponding primitive, and every primitive has exactly one corresponding sprite.

True in spirit.The main idea of this version is to eliminate the calculations from a set of field-values to a sequence of hexadecimal coded bytes.
Not true in the details. The 'sprite-sets' primitive includes not only the action 1 sprite but also the sprites themselves, and it generates thus a sequence of NFO sprites.


DaleStan wrote:
For this type of replacement, the standard stumbling block is action 6, specifically the <offset> entity. In NFO you just use the offset of the byte(s) that need(s) to be changed. In any other language, you can't do that, as byte offsets aren't readily accessible.

Also in NFO you first have to count how many bytes offset there, before you can fill in the field. The offset does not magically comes falling out of the sky.
Therefore, a working but not very nice solution would be the require a numerical offset from the user.
I think a better solution would be to make the value for 'offset' field equal to the name of the key of the field in the next sprite whose value you want to change.
For example 'offset = feature' means that I want the offset of the 'feature' field in the next node.

DaleStan wrote:
I'll second the "no hyphens in identifiers" comment.

Noted.


Top
   
 Post subject: Re: GRL (GRF Language)
PostPosted: Sun Dec 30, 2007 4:55 pm 
Offline
TTDPatch Developer
TTDPatch Developer

Joined: Wed Feb 18, 2004 3:06 am
Posts: 10285
Alberth wrote:
DaleStan wrote:
For this type of replacement, the standard stumbling block is action 6, specifically the <offset> entity. In NFO you just use the offset of the byte(s) that need(s) to be changed. In any other language, you can't do that, as byte offsets aren't readily accessible.

Also in NFO you first have to count how many bytes offset there, before you can fill in the field. The offset does not magically comes falling out of the sky.
True. But I can count bytes in NFO. I can't count the bytes in GRL, because the bytes aren't there.

Another similar issue comes with wanting to write word-sized values into an extended-byte field. Will the translator be intelligent enough to detect this and use the 3-byte extended byte encoding, even if the supplied value fits into a single byte?

_________________
To get a good answer, ask a Smart Question. Similarly, if you want a bug fixed, write a Useful Bug Report. No TTDPatch crashlog? Then follow directions.
Projects: NFORenum (download) | PlaneSet (Website) | grfcodec (download) | grfdebug.log parser


Top
   
 Post subject: Re: GRL (GRF Language)
PostPosted: Sun Dec 30, 2007 5:21 pm 
Offline
Tycoon
Tycoon
User avatar

Joined: Wed Jun 06, 2007 9:07 pm
Posts: 1710
Location: Czech Republic
For offsets, you could use labels and the numerical offset would be calculated during translation to NFO. And for extended bytes, the translator can have tables telling which properties of which actions are bytes, words, extended bytes, dwords or whatever, so it can correctly put the correct type ....

I personally like this proposal more than NDL - it have nicer syntax (no python syntax with enforced indenting which may not always be the best) and "better" compiler (c++ is available on more systems than python)

_________________
If you need something, do it yourself or it will be never done.

My patches: Extra large maps (1048576 high, 1048576 wide) (FS#1059), Vehicle + Town + Industry console commands (FS#1060), few minor patches (FS#2820, FS#1521, FS#2837, FS#2843), AI debugging facility

Other: Very large ships NewGRF, Bilbo's multiplayer patch pack v5 (for OpenTTD 0.7.3)


Top
   
 Post subject: Re: GRL (GRF Language)
PostPosted: Sun Dec 30, 2007 5:42 pm 
Offline
TTDPatch Developer
TTDPatch Developer

Joined: Wed Feb 18, 2004 3:06 am
Posts: 10285
Bilbo wrote:
And for extended bytes, the translator can have tables telling which properties of which actions are bytes, words, extended bytes, dwords or whatever
You missed the question:

Given the key-value pair "extended_byte_property = 0", how should the value be encoded?
If preceded by "action6 { param = 0 count = 2 offset = extended_byte_property }", then how should the value be encoded?
Will this work properly?

For more fun and games, offset is itself extended, and I have (albeit only once) used one action 6 to modify the following action 6.

_________________
To get a good answer, ask a Smart Question. Similarly, if you want a bug fixed, write a Useful Bug Report. No TTDPatch crashlog? Then follow directions.
Projects: NFORenum (download) | PlaneSet (Website) | grfcodec (download) | grfdebug.log parser


Top
   
 Post subject: Re: GRL (GRF Language)
PostPosted: Sun Dec 30, 2007 5:45 pm 
Offline
Tycoon
Tycoon

Joined: Wed Jan 17, 2007 12:14 am
Posts: 7300
Bilbo wrote:
no python syntax with enforced indenting which may not always be the best
syntax is the easiest thing to replace

if people are really against indenting syntax, one can switch to a bracket based block organisation ('begin'...'end' like in pascal, or '{'...'}' like in C)

personally, i found that the only disadvantage of python style indenting is that it totally screws up if you mix tabs and spaces

_________________
You might not exactly be interested in Ferion, but if you are, have fun :)


Top
   
 Post subject: Re: GRL (GRF Language)
PostPosted: Sun Dec 30, 2007 5:49 pm 
Offline
TTDPatch Developer
TTDPatch Developer
User avatar

Joined: Fri Mar 07, 2003 1:10 pm
Posts: 3602
Location: Germany
I think you need or should have a two pass system.

A high level language shouldn't depend that much on the low level part.

This means, the Action 6 definition should work transparent, this means the compiler needs to know the size it writes, the offsets and give them names. If the user needs to know for the action6 how the "thing" is internally stored there is something wrong. How the "thing" is stored should be up to the compiler. If something need to write to "thing" via action6 then the compiler may opt to always use the long form. Again thats a compiler internal and shouldn't be in the language definition...


Top
   
 Post subject: Re: GRL (GRF Language)
PostPosted: Sun Dec 30, 2007 6:21 pm 
Offline
Tycoon
Tycoon
User avatar

Joined: Wed Jun 06, 2007 9:07 pm
Posts: 1710
Location: Czech Republic
DaleStan wrote:
Given the key-value pair "extended_byte_property = 0", how should the value be encoded?
If preceded by "action6 { param = 0 count = 2 offset = extended_byte_property }", then how should the value be encoded?
Will this work properly?

For more fun and games, offset is itself extended, and I have (albeit only once) used one action 6 to modify the following action 6.


Oh, I see, so the extended byte won't be just constant parameter but in fact a pre-initialized variable. And then it would be importat whether it occupies 1 or 3 bytes. This could be solved by explicit typecasting, like
"extended_byte_property = (extended_wide)0" telling the compiler to use 3 bytes even for small values.

_________________
If you need something, do it yourself or it will be never done.

My patches: Extra large maps (1048576 high, 1048576 wide) (FS#1059), Vehicle + Town + Industry console commands (FS#1060), few minor patches (FS#2820, FS#1521, FS#2837, FS#2843), AI debugging facility

Other: Very large ships NewGRF, Bilbo's multiplayer patch pack v5 (for OpenTTD 0.7.3)


Top
   
 Post subject: Re: GRL (GRF Language)
PostPosted: Mon Dec 31, 2007 7:26 am 
Offline
OpenTTD Developer
OpenTTD Developer

Joined: Sun Sep 09, 2007 5:03 am
Posts: 4694
Location: home
DaleStan wrote:
Another similar issue comes with wanting to write word-sized values into an extended-byte field. Will the translator be intelligent enough to detect this and use the 3-byte extended byte encoding, even if the supplied value fits into a single byte?

I can see that this is what you'd like to have with an extended byte and some self-modifying code.
With all the actions that I currently support, I encode extended bytes with maximum efficiency (since their values are all fixed). With adding support for action6, this concept becomes broken.

The minimal requirement would be that the compiler complains about existence of such a problem, so the user is at least aware of it.
The next level is to provide a work-around for the user, so that at least he can continue working until the compiler is fixed. For that situation, a type-cast along the lines of what Bilbo suggested would be nice (tnx for this! it is a simple generic solution for a potentionally nasty problem).
The best solution would be what eis_os suggest, namely letting the compiler figure it out. In terms of the type-cast that means that the compiler will insert the type-case rather than the user.


Top
   
 Post subject: Re: GRL (GRF Language)
PostPosted: Mon Dec 31, 2007 8:00 am 
Offline
OpenTTD Developer
OpenTTD Developer

Joined: Sun Sep 09, 2007 5:03 am
Posts: 4694
Location: home
Eddi wrote:
Bilbo wrote:
no python syntax with enforced indenting which may not always be the best
syntax is the easiest thing to replace

True. However, take into account that people judge stuff w.r.t. what it currently provides, not what it can be (just like you said that my GRL is only a seperation into a set of key/value pairs).
(ie both you and Bilbo are right, the starting points were different though).

Eddi wrote:
personally, i found that the only disadvantage of python style indenting is that it totally screws up if you mix tabs and spaces

I stopped using tabs about a year ago, and that works great, until you start doing programming in OTTD where they only use tabs :)
I also have occasional problems with line-continuations (the \<NL> token) in Python. It makes sense in Python as full language, but it may be something to watch out for in your application.


Top
   
 Post subject: Re: GRL (GRF Language)
PostPosted: Mon Dec 31, 2007 8:37 am 
Offline
OpenTTD Developer
OpenTTD Developer

Joined: Sun Sep 09, 2007 5:03 am
Posts: 4694
Location: home
eis_os wrote:
This means, the Action 6 definition should work transparent, this means the compiler needs to know the size it writes, the offsets and give them names. If the user needs to know for the action6 how the "thing" is internally stored there is something wrong. How the "thing" is stored should be up to the compiler. If something need to write to "thing" via action6 then the compiler may opt to always use the long form. Again thats a compiler internal and shouldn't be in the language definition...

The problem imho is not the size of the replacement, it's the existence of the Action6. The level of abstraction between Action6 and the other Actions does not match. It's a bit like having a toolbox filled with hammers, screwdrivers, pincers, pliers, and tucked away at the bottom a nuke, for those special cases that all else fails.

Existence of Action6 is a sympton that a problem needed to be solved which was so big that the best workable solution for NFO was to allow self-modifying code.

The right solution for GRL imho would be to consider this action itself a compiler internal rather than exposing it to a user. That however leads to the question which higher level problem exists for which Action6 is the best possible answer. I would need to being able to express that problem in my GRL language, then have the compiler generate an Action6 as needed.


Top
   
 Post subject: Re: GRL (GRF Language)
PostPosted: Mon Dec 31, 2007 10:38 am 
Offline
TTDPatch Developer
TTDPatch Developer
User avatar

Joined: Fri Mar 07, 2003 1:10 pm
Posts: 3602
Location: Germany
As example you have to abstract GRF Resource Management.. Again, you can still give the user this kind of sledgehammer if your compiler has some kind of internal design, that each stuff you write has some kind of addressable nature and state, so you can access that from your action 6.

This means, you need to know before you write the action6 how the other actions look.

However to be able to make your compiler internal action6 know where it should store something like GRM offsets, it needs to know how an action internally will be written and the offsets anyway.


Or well you do the complete reverse:
An action can create an action6 in front itself to change itself, or if you like perl, do both :D
Josef would surely love this...

The joys of compiler construction :mrgreen:


Top
   
 Post subject: Re: GRL (GRF Language)
PostPosted: Mon Dec 31, 2007 10:58 am 
Offline
Tycoon
Tycoon
User avatar

Joined: Mon Apr 19, 2004 8:09 pm
Posts: 1732
Location: Sweden
Alberth wrote:
The right solution for GRL imho would be to consider this action itself a compiler internal rather than exposing it to a user. That however leads to the question which higher level problem exists for which Action6 is the best possible answer. I would need to being able to express that problem in my GRL language, then have the compiler generate an Action6 as needed.

This sounds like the right approach to me. Something like
Code:
        id = Byte(Parameter(2)),

or
Code:
        value = 0020+Word(Parameter(2)),

Now the Action6 is implicit instead of explicit. There are of course many other possible ways to use Action6, but who says that your GRL must be able to express existing GRF/NFO files? Isn't it enough if GRL can be used to create new GRF:s?

_________________
Image
Swedish trains * Swedish Town Names * BlockCAD * The GRFMaker Community Page


Top
   
 Post subject: Re: GRL (GRF Language)
PostPosted: Mon Dec 31, 2007 1:23 pm 
Offline
Tycoon
Tycoon
User avatar

Joined: Wed Jun 06, 2007 9:07 pm
Posts: 1710
Location: Czech Republic
Alberth wrote:
In terms of the type-cast that means that the compiler will insert the type-case rather than the user.


Well, compiler should know what is expected (whether extended byte, word, byte, etc...) and use it, UNLESS user specifies otherwise. Therefore if user says dword, put there a dword even if spec says byte :) (extended byte short (1 or 3 byte according to value) and 3-byte would be then basically the "same" type) Perhaps just print some warning if user chooses something incompatible (either specs may perhaps change from what the compiler thinks is correct, or user is trying to do some advanced trick :) ...

_________________
If you need something, do it yourself or it will be never done.

My patches: Extra large maps (1048576 high, 1048576 wide) (FS#1059), Vehicle + Town + Industry console commands (FS#1060), few minor patches (FS#2820, FS#1521, FS#2837, FS#2843), AI debugging facility

Other: Very large ships NewGRF, Bilbo's multiplayer patch pack v5 (for OpenTTD 0.7.3)


Top
   
 Post subject: Re: GRL (GRF Language)
PostPosted: Mon Dec 31, 2007 1:25 pm 
Offline
Tycoon
Tycoon
User avatar

Joined: Wed Jun 06, 2007 9:07 pm
Posts: 1710
Location: Czech Republic
AndersI wrote:
Now the Action6 is implicit instead of explicit. There are of course many other possible ways to use Action6, but who says that your GRL must be able to express existing GRF/NFO files? Isn't it enough if GRL can be used to create new GRF:s?


If there would be some feature not possible to be done using GRL and you need that feature, you have to stick with old-style NFO for your entire project. This would be quite a bit of disadvantage ....

_________________
If you need something, do it yourself or it will be never done.

My patches: Extra large maps (1048576 high, 1048576 wide) (FS#1059), Vehicle + Town + Industry console commands (FS#1060), few minor patches (FS#2820, FS#1521, FS#2837, FS#2843), AI debugging facility

Other: Very large ships NewGRF, Bilbo's multiplayer patch pack v5 (for OpenTTD 0.7.3)


Top
   
 Post subject: Re: GRL (GRF Language)
PostPosted: Mon Dec 31, 2007 5:19 pm 
Offline
Tycoon
Tycoon
User avatar

Joined: Mon Apr 19, 2004 8:09 pm
Posts: 1732
Location: Sweden
Bilbo wrote:
If there would be some feature not possible to be done using GRL and you need that feature, you have to stick with old-style NFO for your entire project. This would be quite a bit of disadvantage ....

As the grl-bytes is included in the definition, you should be able to express anything a plain NFO can (by using plain old NFO commands).

_________________
Image
Swedish trains * Swedish Town Names * BlockCAD * The GRFMaker Community Page


Top
   
 Post subject: Re: GRL (GRF Language)
PostPosted: Mon Dec 31, 2007 7:07 pm 
Offline
TTDPatch Developer
TTDPatch Developer

Joined: Wed Feb 18, 2004 3:06 am
Posts: 10285
AndersI wrote:
Bilbo wrote:
If there would be some feature not possible to be done using GRL and you need that feature, you have to stick with old-style NFO for your entire project. This would be quite a bit of disadvantage ....

As the grl-bytes is included in the definition, you should be able to express anything a plain NFO can (by using plain old NFO commands).
Well, yes and no. Encoding action 6 with grl-bytes (currently the only way to do it, but I expect this will change) also requires that the following sprite be a grl-bytes. If this is an action 2, then all sprites it references, and all sprites that reference it, must also be grl-bytes, unless grl-bytes parser has quite a bit of intelligence, and grl-bytes is allowed to do things that I wouldn't expect in grl-bytes.

AndersI wrote:
This sounds like the right approach to me. Something like
Code:
        id = Byte(Parameter(2)),

or
Code:
        value = 0020+Word(Parameter(2)),

I see two (or maybe 3) problems with this:
1) How do you conditionally skip the action 6?
1b) (Corollary) How do you specify the value that is used when the 6 isn't?
2) What if you want to modify something that is implied by the grl, not explicit?

These questions come from a convoluted double-tac-nuke I pulled some years ago: I wrote an action 6 that made three modifications, except that the third modification used parameter FF. (Making it the terminator, not a modification, and making the following 3 bytes "extraneous".) I then used another 6 to change the FF to the desired parameter number, and a 9 to skip the first 6 if I didn't want the second 6 to make its third modification at all.

Something like
Code:
* 0   09 00 01 \7> 02 01
* 0   06 00 01 07 FF
* 0   06 01 01 08 01 01 0C FF 01 05 FF
(The offsets in the second 6 may not have any realistic use. I know the one I wrote did what I intended for it to do.)

_________________
To get a good answer, ask a Smart Question. Similarly, if you want a bug fixed, write a Useful Bug Report. No TTDPatch crashlog? Then follow directions.
Projects: NFORenum (download) | PlaneSet (Website) | grfcodec (download) | grfdebug.log parser


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic  [ 38 posts ]  Go to page 1 2 Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 6 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000-2019 phpBB Limited

Copyright © Owen Rudge/The Transport Tycoon Forums 2001-2019.
Hosted by Zernebok Hosting.