NML - a Newgrf Meta Language

Discussions about the technical aspects of graphics development, including NewGRF tools and utilities.

Moderator: Graphics Moderators

Post Reply
User avatar
planetmaker
OpenTTD Developer
OpenTTD Developer
Posts: 9432
Joined: 07 Nov 2007 22:44
Location: Sol d

NML - a Newgrf Meta Language

Post by planetmaker »

What is it about?
NML is a high-level NewGRF language compiler which compiles nml files into newgrf files (and nfo files, if asked to do so)

It was mentioned in a few threads already, but let's present to the newgrf author community officially a new newgrf language, NML. Yexo started this project it a few months back and other big contributors to this newgrf creation tool are Hirundo and Alberth.

Meanwhile NML has matured enough that it can officially be made public, though till 0.1 will be released some changes to the syntax might still happen.

How does it work?

Writing a NewGRF in NML is similar to writing in any other programming language except that there are no real sub-routines; you can use the usual definition and condition statements. See below for a comparison between NFO and NML with a small rail types newgrf as example. One of the convenient changes is also NML's possibility to read other image formats, most notably also png (those images still need to supply the proper 8bpp palette and are supported by the python image library). Personally I find png quite convenient as the same format can also easily be viewed in browsers (no nasty conversions anymore needed).

The grf can be created by a simple command line call:

Code: Select all

nmlc --nfo swedishrails.nfo --grf swedishrails.grf swedishrails.nml
The creation of nfo code can, of course, be skipped as the grf is written directly.

What can I do with it so far?
  • Supported Features:
  • 0x00 Trains
  • 0x01 Road vehicles
  • 0x02 Ships
  • 0x03 Planes
  • 0x08 General variables
  • 0x09 / 0x0A Industries and industry tiles
  • 0x0B Cargos
  • 0x0D / 0x11 Airports and airport tiles
  • 0x0F NewObjects
  • 0x10 Railtypes
  • Missing Features:
  • 0x04 Stations
  • 0x05 Canals
  • 0x06 Bridges
  • 0x07 Houses
  • 0x0C Sound effects
  • 0x0E Signals
What do I need? Where do I get it?
You need python 2.5 ... < 3.0, and a range of modules, most notably the python image library and the python lz77 library and NML itself.

Get the latest version of [https://github.com/OpenTTD/nml/releases]NML here[/url] and have a look at the documentation which unfortunately is still missing a lot. Any help there is also appreciated.

Many (most?) of the important newgrf features are already implemented in NML, albeit there are still bits and pieces which are missing. But those are getting fewer by the day. Should one particular thing be missing for your very needs, it usually should not be a big problem to go ask and that will become a top priority in the implementation list. Feedback, comments, feature requests and bug reports warmly welcome! Also have a look at our bug tracker.


I'd like to finish with a brief comparison of tradition NFO programming of NewGRFs as compared to writing the same thing using NML. During the last days I started writing the SwedishRails set, using NML in order to get things going, also as an example how things work in NML. I'll show first the whole NFO and further down the NML way of doing the snippets presented here. I leave out parts which will not add new knowledge but just repeat previously shown things. If you want two full NML-style newgrfs, look at OpenGFX+ Airports, OpenGFX+ Trains, OpenGFX+ Road Vehicles and SwedishRails source codes. Now some snippets from the Swedish rails in both languages:

The NFO way
We first define this to be a newgrf at all:

Code: Select all

// Automatically generated by GRFCODEC. Do not modify!
// (Info version 7)
// Escapes: 2+ = 71 = D= = DR 2- = 70 = D+ = DF 2< = 7= = D- = DC 2> = 7! = Du* = DM 2u< = 7< = D* = DnF 2u> = 7> = Du<< = DnC 
2/ = 7G = D<< = DO 2% = 7g = D& 2u/ = 7gG = D| 2u% = 7GG = Du/ 2* = 7gg = D/ 2& = 7c = Du% 2| = 7C = D% 2^ 2sto = 2s 2rst = 2r 2+
 2ror = 2rot
// Format: spritenum pcxfile xpos ypos compression ysize xsize xrel yrel

0 * 4 \d416 
1 * 92 08 07 "SER0" "ÞSwedish Rails  nightly-r52M" 00 "ÞSwedish rails are a replacement for the default rails" 00 
Then we go about just re-defining the normal rails (e-rail is compatible) as a rail type and we assign sprite sets for track overlays, track underlays, depots and tunnels. We only do this, if we're running on the OpenTTD platform:

Code: Select all

412 * 9 07 9D 04 \7! \dx00000001 04 

413 * 18 00 10 \b2 01 FF \wx000A 
08 "RAIL" 
0E \b1 "ELRL" 

414 * 22 03 10 01 0A \b5 
01 \wx00FB 
02 \wx00FC 
07 \wx00FD 
03 \wx00F8 
08 \wx00FA 
\wx00FE 
We use var40 in order to decide to use snowy track sprites for tunnels or normal track sprites:

Code: Select all

383 * 6 01 10 \b1 FF \wx0004 

384 src/gfx/tunnel_track_normal.png 75 0 01 31 64 -31 0 
385 src/gfx/tunnel_track_normal.png 0 0 01 31 64 -31 0 
386 src/gfx/tunnel_track_normal.png 75 50 01 31 64 -31 0 
387 src/gfx/tunnel_track_normal.png 0 50 01 31 64 -31 0 

388 * 7 02 10 FA \b1 \b0 
\w0 

389 * 6 01 10 \b1 FF \wx0004 

390 src/gfx/tunnel_track_snow.png 75 0 01 31 64 -31 0 
391 src/gfx/tunnel_track_snow.png 0 0 01 31 64 -31 0 
392 src/gfx/tunnel_track_snow.png 75 50 01 31 64 -31 0 
393 src/gfx/tunnel_track_snow.png 0 50 01 31 64 -31 0 

394 * 7 02 10 F9 \b1 \b0 
\w0 

395 * 23 02 10 F8 89 
40 00 \dx000000FF \b1 
\wx00F9 \dx00000004 \dx00000004 
\wx00FA 
Re-define temperate tunnel portal sprites for TTDPatch or when asked so by grf parameter:

Code: Select all

2 * 9 0D 40 \D- 9D FF \dx00000000 
3 * 5 0D 40 \D/ 40 40 
4 * 9 0D 40 \D- FF 40 \dx00000001 

5 * 5 0D 0A \D| 40 00 
6 * 9 0D 41 \D- 83 FF \dx00000000 
7 * 5 0D 41 \D/ 41 41 
8 * 9 0D 41 \D- FF 41 \dx00000001 
9 * 9 0D 42 \D- 0A FF \dx00000001 
10 * 5 0D 42 \D/ 42 42 
11 * 9 0D 42 \D- FF 42 \dx00000001 
12 * 5 0D 40 \D& 41 42 
13 * 9 07 40 04 \7= \dx00000000 11 

48 * 5 0A \b1 \b8 \wx093D 
49 src/gfx/temperate_rails_TTD.png 4 6 01 33 33 -31 -2 
50 src/gfx/temperate_rails_TTD.png 52 6 01 38 40 -7 -38 
51 src/gfx/temperate_rails_TTD.png 95 6 01 19 27 -31 2 
52 src/gfx/temperate_rails_TTD.png 129 6 01 23 42 -9 -30 
53 src/gfx/temperate_rails_TTD.png 177 6 01 18 25 8 3 
54 src/gfx/temperate_rails_TTD.png 209 6 01 23 43 -31 -30 
55 src/gfx/temperate_rails_TTD.png 258 6 01 33 33 0 -2 
56 src/gfx/temperate_rails_TTD.png 295 6 01 38 40 -31 -38 

109 * 2 10 11 
and now the whole thing in
the NML way:

Code: Select all

grf {
 grfid : "SER0";
 name : string(STR_GRF_NAME);
 desc : string(STR_GRF_DESCRIPTION);
}
and in the language file which follows the convention used for OpenTTD translation files:

Code: Select all

lang: 7F
STR_GRF_NAME                                                    :Swedish Rails {VERSION}
STR_GRF_DESCRIPTION                                             :Swedish rails are a replacement for the default rails
Defining the rail type for OpenTTD only:

Code: Select all

if (ttd_platform == PLATFORM_OPENTTD) {

 item(FEAT_RAILTYPES, rail, 0x0A) {
  property {
   label: "RAIL";
   compatible_railtype_list: ["ELRL"];
   // We don't want to change rail properties. Just provide new graphics
  }
  graphics {
   TRACKOVERLAY: ground_switch_overlay;
   UNDERLAY: ground_switch_underlay;
   LEVEL_CROSSINGS: level_crossing_group;
   TUNNELS: tunnel_switch;
   DEPOTS: depot_normal_rail_group;
   track_overlay_group; // irrelevant, we just need one set here
  }
 }
} // of OpenTTD only
Defining snowy and normal tunnel tracks (we can re-use the alignment as the snowy and normal sprites are in identical files):

Code: Select all

template tmpl_tunnel_tracks() {
 [ 75, 0, 64,31, -31, 0]
 [ 0, 0, 64,31, -31, 0]
 [ 75, 50, 64,31, -31, 0]
 [ 0, 50, 64,31, -31, 0]
}

 spriteset(tunnel_overlay, "src/gfx/tunnel_track_normal.png") {
  tmpl_tunnel_tracks()
 }
 spritegroup tunnel_group {
  default: tunnel_overlay;
 }
 spriteset(tunnel_overlay_snow, "src/gfx/tunnel_track_snow.png") {
  tmpl_tunnel_tracks()
 }
 spritegroup tunnel_snow_group {
  default: tunnel_overlay_snow;
 }

switch(FEAT_RAILTYPES, SELF, tunnel_switch, terrain_type) {
 TILETYPE_SNOW: tunnel_snow_group;
 tunnel_group;
}
Replacing snowy tunnel portal sprites for TTDPatch or when requested so by the first newgrf parameter. Again we re-use the alignment as we find the tunnel portal sprites for all climates neatly aligned, this time in the same file, just at different vertical offsets:

Code: Select all

template tmpl_tunnel(y) {
 [ 4, 6, 33,33, -31, -2]
 [ 52, 6, 40,38, -7,-38]

 [ 95, 6, 27,19, -31, 2]
 [129, 6, 42,23, -9,-30]

 [177, 6, 25,18, 8, 3]
 [209, 6, 43,23, -31,-30]

 [258, 6, 33,33, -0, -2]
 [295, 6, 40,38, -31,-38]
}

param[10] = (ttd_platform == PLATFORM_TTDPATCH) | (param[0]); // Use TTD ground sprites

if (((climate == CLIMATE_ARCTIC) | (climate == CLIMATE_TEMPERATE)) & (param[10] == 1)) {

 // Standard temperate rail sprites
 replace (1037, "src/gfx/snow_rails_TTD.png") {
  tmpl_ttd_ground(358)
 }

 // Replacement of standard rail tunnels in temperate
 replace (2397, "src/gfx/snow_rails_TTD.png") {
  tmpl_tunnel(6)
 }
} // snow for arctic + temperate TTD mode

EDIT 4 Nov 2010: Update of documentation links and some syntax changes
EDIT 23 Dec 2020: Update of NML link for latest version
Last edited by planetmaker on 16 Dec 2011 15:37, edited 5 times in total.
Yexo
Tycoon
Tycoon
Posts: 3663
Joined: 20 Dec 2007 12:49

Re: NML - a Newgrf Meta Language

Post by Yexo »

Thanks planetmaker for the nice introduction (and reserving some space here for possible updates)
User avatar
cmoiromain
Chief Executive
Chief Executive
Posts: 655
Joined: 15 Jan 2007 21:45
Location: FRANCE
Contact:

Re: NML - a Newgrf Meta Language

Post by cmoiromain »

Nice! That might get me into trying to create newgrfs again. Good job!
I am little, ugly, and nasty. How do you do?
User avatar
Lord Aro
Tycoon
Tycoon
Posts: 2369
Joined: 25 Jun 2009 16:42
Location: Location, Location
Contact:

Re: NML - a Newgrf Meta Language

Post by Lord Aro »

woah, i must of missed this...
is it supposed to eventually succeed NFO?
AroAI - A really feeble attempt at an AI

It is practically impossible to teach good programming to students that have had a prior exposure to BASIC: as potential programmers they are mentally mutilated beyond hope of regeneration. --Edsger Dijkstra
audigex
Tycoon
Tycoon
Posts: 1996
Joined: 09 Dec 2007 21:28
Contact:

Re: NML - a Newgrf Meta Language

Post by audigex »

Are there any specific advantages/limitations over NFO? A more complete list of syntax etc might be handy, too :)

This looks excellent though, NFO writing has always seemed to me to be about the biggest obstacle to getting new sprites in game.
Jon
User avatar
Hyronymus
Tycoon
Tycoon
Posts: 13233
Joined: 03 Dec 2002 10:36
Location: The Netherlands
Contact:

Re: NML - a Newgrf Meta Language

Post by Hyronymus »

So I can decompile existing GRF's and have a look at NML, even when that GRF was compiled 'the old way'?
Hirundo
Transport Coordinator
Transport Coordinator
Posts: 298
Joined: 27 Jan 2008 13:02

Re: NML - a Newgrf Meta Language

Post by Hirundo »

audigex wrote:Are there any specific advantages/limitations over NFO? A more complete list of syntax etc might be handy, too :)
The advantage can be compared to the advantage of C over asm. You can do the same things with it, but C resp. NML requires far less mental effort to read and write.
So, NML allows writing grfs more quickly

Documentation can be found in docs/reference.html:
http://hg.openttdcoop.org/nml/raw-file/ ... rence.html
Hyronymus wrote:So I can decompile existing GRF's and have a look at NML, even when that GRF was compiled 'the old way'?
no(t yet)
Create your own NewGRF? Check out this tutorial!
User avatar
Hyronymus
Tycoon
Tycoon
Posts: 13233
Joined: 03 Dec 2002 10:36
Location: The Netherlands
Contact:

Re: NML - a Newgrf Meta Language

Post by Hyronymus »

OK, I'll wait.

Upon looking through the specs I see this code:

Code: Select all

item (FEAT_ROADVEHS, hereford_tram) {
	property {
		name:               string(STR_NAME_HEREFORD_TRAM);
		climates_available: CLIMATE_ALL;
		introduction_date:  date(1940,1,1);
		speed:              60 km/h;
	}
}
Does this mean that you can have vehicles appear halfway a month? Or does 1940,1,1 mean something else then Januari 1st, 1940?
User avatar
planetmaker
OpenTTD Developer
OpenTTD Developer
Posts: 9432
Joined: 07 Nov 2007 22:44
Location: Sol d

Re: NML - a Newgrf Meta Language

Post by planetmaker »

Hyronymus wrote:Does this mean that you can have vehicles appear halfway a month? Or does 1940,1,1 mean something else then Januari 1st, 1940?
Yes, it means exactly that. But it's not a particularily new feature: http://wiki.ttdpatch.net/tiki-index.php ... on_date_2A_ - just better exposed :-)
User avatar
Hyronymus
Tycoon
Tycoon
Posts: 13233
Joined: 03 Dec 2002 10:36
Location: The Netherlands
Contact:

Re: NML - a Newgrf Meta Language

Post by Hyronymus »

planetmaker wrote:
Hyronymus wrote:Does this mean that you can have vehicles appear halfway a month? Or does 1940,1,1 mean something else then Januari 1st, 1940?
Yes, it means exactly that. But it's not a particularily new feature: http://wiki.ttdpatch.net/tiki-index.php ... on_date_2A_
Oh, I haven't noticed that feature :P. And come to think of it, TTD added a randomiser to intro dates, does OpenTTD still have that 'quirk'?

Further reading: the cargo table seems more condensed but that might be me. Are cargotypes from PBI and ECS (noticed a reference to ECS tourists) also suupported i.e.?
User avatar
planetmaker
OpenTTD Developer
OpenTTD Developer
Posts: 9432
Joined: 07 Nov 2007 22:44
Location: Sol d

Re: NML - a Newgrf Meta Language

Post by planetmaker »

Hyronymus wrote:TTD added a randomiser to intro dates, does OpenTTD still have that 'quirk'?

Further reading: the cargo table seems more condensed but that might be me. Are cargotypes from PBI and ECS (noticed a reference to ECS tourists) also suupported i.e.?
IIRC, introduction dates of the first two years after starting are not randomized. All others are.

Cargotypes should all be supported.
User avatar
Hyronymus
Tycoon
Tycoon
Posts: 13233
Joined: 03 Dec 2002 10:36
Location: The Netherlands
Contact:

Re: NML - a Newgrf Meta Language

Post by Hyronymus »

planetmaker wrote:
Hyronymus wrote:TTD added a randomiser to intro dates, does OpenTTD still have that 'quirk'?

Further reading: the cargo table seems more condensed but that might be me. Are cargotypes from PBI and ECS (noticed a reference to ECS tourists) also suupported i.e.?
IIRC, introduction dates of the first two years after starting are not randomized. All others are.

Cargotypes should all be supported.
That would be nice for the cargotypes.

I wonder when introduction date randomisation becomes a game option in the mean time.
User avatar
planetmaker
OpenTTD Developer
OpenTTD Developer
Posts: 9432
Joined: 07 Nov 2007 22:44
Location: Sol d

Re: NML - a Newgrf Meta Language

Post by planetmaker »

Hyronymus wrote:That would be nice for the cargotypes.

I wonder when introduction date randomisation becomes a game option in the mean time.
I guess introduction date randomization doesn't really warrant a new setting. What would be the game play advantage? Only in the first two years it's important that things are introduced as otherwise vehicles are not available from start as intended by the map creator. Later it doesn't matter and adds a bit of randomness to the game. Also one player gets the chance to test it exclusively, so availability varies between players anyway.

Concerning cargo types: there's different meanings of this: http://wiki.ttdpatch.net/tiki-index.php?page=CargoTypes
Cargo labels can anyway be freely assigned and chosen. No limitation there.

Looking at the default cargo types (e.g. trains, prop. 15) the vehicle gets, column 'B' in the first table in the link, it's IMHO mostly useless, as industry sets can change that and cargos might not be available and such. So only the values FF (first refittable and to some degree 00 (=pax) and 01 (=mail) as they're hardly changed, make sense. But still, if you disagree with me, you can chose any value for that which you desire.

Of course the cargo translation table can be freely chosen, as such the values in column 'A' of the link, which determine the refittable cargo types, are all available in NML. You can in NML specify them directly as cargo label, the NML documentation might still be wrong about that.
User avatar
Hyronymus
Tycoon
Tycoon
Posts: 13233
Joined: 03 Dec 2002 10:36
Location: The Netherlands
Contact:

Re: NML - a Newgrf Meta Language

Post by Hyronymus »

There comes confusion again: cargo table, cargo labels and cargo types :P.
User avatar
planetmaker
OpenTTD Developer
OpenTTD Developer
Posts: 9432
Joined: 07 Nov 2007 22:44
Location: Sol d

Re: NML - a Newgrf Meta Language

Post by planetmaker »

Well,... in principle, when using NML, you should only need to care about
- cargo labels which reference a specific cargo like IORE, PASS, WOOD,...
- cargo classes like CC_PASSENGERS, CC_MAIL, CC_BULK...
while you normally don't need the cargo type.

EDIT: edited links to definitions
User avatar
Hyronymus
Tycoon
Tycoon
Posts: 13233
Joined: 03 Dec 2002 10:36
Location: The Netherlands
Contact:

Re: NML - a Newgrf Meta Language

Post by Hyronymus »

Thanks planetmaker.

I'm a bit surprised by the loud silence, I genuinly thought this would cause a revolution...
Kogut
Tycoon
Tycoon
Posts: 2493
Joined: 26 Aug 2009 06:33
Location: Poland

Re: NML - a Newgrf Meta Language

Post by Kogut »

planetmaker wrote: If you want two full NML-style newgrfs, look at OpenGFX+ and SwedishRails source codes.
Where I can find NML code of OpenGFX+?
Correct me If I am wrong - PM me if my English is bad
AIAI - AI for OpenTTD
User avatar
Lord Aro
Tycoon
Tycoon
Posts: 2369
Joined: 25 Jun 2009 16:42
Location: Location, Location
Contact:

Re: NML - a Newgrf Meta Language

Post by Lord Aro »

If you hunted around for about 20 seconds, you would find this:
http://dev.openttdcoop.org/projects/ogf ... ow/sprites
AroAI - A really feeble attempt at an AI

It is practically impossible to teach good programming to students that have had a prior exposure to BASIC: as potential programmers they are mentally mutilated beyond hope of regeneration. --Edsger Dijkstra
Kogut
Tycoon
Tycoon
Posts: 2493
Joined: 26 Aug 2009 06:33
Location: Poland

Re: NML - a Newgrf Meta Language

Post by Kogut »

Thanks
Correct me If I am wrong - PM me if my English is bad
AIAI - AI for OpenTTD
User avatar
keoz
Transport Coordinator
Transport Coordinator
Posts: 321
Joined: 16 Jul 2009 10:04

Re: NML - a Newgrf Meta Language

Post by keoz »

This project looks extremely interesting.

I'd be particularly interested in the possibility of decoding existing grf into nml code. You stated that this feature is not possible (for now), but is it already under development ?
Patch - Let's timetable depot waiting time with the Wait in depot patch.
GameScript - Searching a new way to make your cities growing ? Try the Renewed City Growth GameScript.
My screenshots thread.
Post Reply

Return to “NewGRF Technical Discussions”

Who is online

Users browsing this forum: No registered users and 3 guests