NML - a Newgrf Meta Language

[planetmaker]

That was actually a bug in NML. I've fixed in it r1880. They should both behave like the 'name" property already did.

Great. I guess it will be released within the next few days?

There's daily a new version compiled, around 18h CET: http://bundles.openttdcoop.org/nml/nightlies/
Of course there's no new version, if nothing was changed or in the rare case that a bug prevents compilation of the new version (like today unfortunately).

[FooBar]

Great. I guess it will be released within the next few days?

Tomorrow. Or right now if you want to build it yourself.

[Bad_Brett]

Version r1881 is now available and it seems to work A LOT better. It's much faster and several bugs have been removed. Great work everyone!

Just three more questions...

1. I see that many of the files are named *.pnml. Do these work like void functions or something like that? It seems wise to learn how to divide the code into seperate files when working with huga projects.

2. Many news contain info about the closest town, such as "New Sawmill under construction near *********". Is there a function I can call to get the name of this town?

3. My cargo still won't show up at the industry production and I can't seem to figure out what I'm doing wrong. The cargo shows up in the payment graph, but not on the industry production:

cotton.png (71.56 KiB) Viewed 4025 times

Code: Select all

cargotable {                       
    WDPR, SCRP, CMNT, COTT
}

item (FEAT_CARGOS   , item_cotton, 4) {
	property {
      number:                               4;
	   type_name: 							  string(STR_NAME_COTTON);
	   cargo_label:						  COTT;
    }
}

item (FEAT_INDUSTRIES   , item_cottonplantation) {
	property {
      substitute:                           INDUSTRYTYPE_FRUIT_PLANTATION;
	   override:                             INDUSTRYTYPE_FRUIT_PLANTATION;
	   life_type: 							  IND_LIFE_TYPE_ORGANIC;
	   name:								  string(STR_NAME_COTTON_PLANTATION);	
	   prod_cargo_types:					  [cargotype("COTT")]; 	
	   prod_multiplier:					  [1]; 		
    }
}

[planetmaker]

1. I see that many of the files are named *.pnml. Do these work like void functions or something like that? It seems wise to learn how to divide the code into seperate files when working with huga projects.

These files usually only make sense in the context of the makefile framework, they're NML files which need pre-processing by CPP (thus the "p" for pre-processing"). You cannot split your NML file when only using NML. You'll need some kind of pre-processor for that means.

2. Many news contain info about the closest town, such as "New Sawmill under construction near *********". Is there a function I can call to get the name of this town?

Those news are generated by OpenTTD using the industry's name. You do not write those news nor do you have access to the town name. Why would you need the town name?

3. My cargo still won't show up at the industry production and I can't seem to figure out what I'm doing wrong. The cargo shows up in the payment graph, but not on the industry production:

cotton.png

I see a green line of the same shade in the cargo graph (lower part).

[Bad_Brett]

These files usually only make sense in the context of the makefile framework, they're NML files which need pre-processing by CPP (thus the "p" for pre-processing"). You cannot split your NML file when only using NML. You'll need some kind of pre-processor for that means.

Ok! I guess an option mighh be to create several .grf's instead and put it all together when everything is finished.

Those news are generated by OpenTTD using the industry's name. You do not write those news nor do you have access to the town name. Why would you need the town name?

My mistake. I was confused when the gray sign on my fish resource only said "Fishes", but then I realized that if I click on the actual industry, the town name will be included.

I see a green line of the same shade in the cargo graph (lower part).

Yes, it shows up in the cargo graph. It replaces fruit just as I had planned. The problem is that my plantation won't produce any cotton. The box is just blank and after a month the plantation shuts down. I guess I've forgotten some important parameter, but I can't seem to figure out which one.

[Eddi]

1. I see that many of the files are named *.pnml. Do these work like void functions or something like that? It seems wise to learn how to divide the code into seperate files when working with huga projects.

they basically work like C header files (.h). you make a "main.pnml" file with lots of "#include <blah.pnml>", and then run a command like

Code: Select all

cpp main.pnml > main.nml

and the resulting code can be processed by nml.

[FooBar]

Did anything fancy happen between r1834 and r1881? I just upgraded, and now I'm getting a:

Code: Select all

nmlc: "src/steam/Arend_item.pnml", line 39: Unknown property name: length

That's on a piece of code that I haven't touched in quite a while. If I downgrade back to r1834 everything works again.
Now trying yesterday's version to narrow it down a bit...
EDIT: narrowed it down to that r1872 is working and r1876 isn't (same error).
EDIT2: I don't see any relevant changes in the revisions between that. Weird :S
EDIT3: by the way, I'm using the prebuilt rpms from the bundles server
EDIT4: when I compile NML myself (also r1881), I get a different error with slightly more information:

Code: Select all

nmlc: An internal error has occurred:
nmlc-version: r1881 (ff3ced18bee8)
Error:      (TypeError) "'tuple' object does not support item assignment".
Command:    ['/usr/bin/nmlc', '-c', '-p', 'DOS', '--grf', 'dutchtrains.grf', 'dutchtrains.nml']
Location:   File "/usr/lib/python2.7/site-packages/nml-r1881-py2.7.egg/nml/output_grf.py", line 420, in wsprite

EDIT5: It gets weirder: a self-built r1876 is working.
EDIT6: And a self-built r1880 also works, so I guess I've reduced the error to the same as Ammler reported in #3797. It doesn't explain why the rpm isn't working, but that may very well be on my part. As it did work on the compile farm previously. I added my findings to said issue.

[FooBar]

As of NML r1879, writing (big) real sprites should be much faster. The speedup is roughly x2 for opengfx, it might be significantly more or less depending on the grf involved.

I've run some tests myself on the dutchtrains r315, with the following results. In all cases I used a preprocessed nml file, as not to include the slowdown of the makefile[0].

nml r1874: 1m24s
nml r1874 -u: 1m16s
nml r1880: 1m15s
nml r1880 -u: 1m10s

While that may not look like much, it's still about 15% off the original time! The filesize increase by -u isn't significant: from 1 MB to 1.5 MB.

These are all times on a virtual system, limited to one core and 1 GB of memory. And not that the dutchtrains don't include any big sprites, only normal 8bpp ones, in a lot of separate files.


[0] What did surprize me is that the makefile, assuming a make remake, adds about a minute to the above times.

[andythenorth]

[0] What did surprize me is that the makefile, assuming a make remake, adds about a minute to the above times.

You may find you get the fastest compile time using nml->nfo->grfcodec. For BANDIT this yields ~7s instead of ~13s with pure nmlc -u route.

If you can skip the makefile, you may get faster times, but for a project using CPP this won't work

This is how I build BANDIT for development only. I rely on the makefile for making release bundles, and on the compile farm.

Code: Select all

#!/bin/bash
  ./src/build_bandit.py 
  && nmlc bandit.nml --nfo=sprites/bandit.nfo 
  && grfcodec -e bandit.grf 
  && mv bandit.grf /Users/andy/Documents/OpenTTD/data

[Eddi]

You may find you get the fastest compile time using nml->nfo->grfcodec. For BANDIT this yields ~7s instead of ~13s with pure nmlc -u route.

for CETS that took like 2 minutes off the compile time (40%).

[planetmaker]

That's rather not a problem of make (itself) but of the dependency check I added. If you kill the dependency check (which is a separate NML run in this context) you'll probably increase run speed by... 40%.

[FooBar]

If you can skip the makefile, you may get faster times, but for a project using CPP this won't work

On the contrary, that works great for CPP dependent projects as well!

I made this from your idea:

Code: Select all

#!/bin/bash
echo Preprocessing NML
cpp -C -E < src/dutchtrains.pnml > dutchtrains.nml
echo Writing version info
sed -i s/REPO_REVISION/$(hg parents --template="{rev}")/ dutchtrains.nml
echo Writing custom_tags
echo VERSION :r$(hg parents --template="{rev}")>custom_tags.txt
echo TITLE :Dutch Trains r$(hg parents --template="{rev}")>>custom_tags.txt
echo FILENAME :dutchtrains.grf>>custom_tags.txt
echo Running NML
nmlc -p DOS dutchtrains.nml
echo Done!

That may not be the cleanest script, but then I have no idea how to write this sort of thing But it works, it writes the repo revision to the grf block and creates the custom tags. It doesn't add an "M" when building from modified source, but I guess that doesn't matter in this context.

The speedup, like Eddi/planetmaker indicated, is indeed around 40% when compared to a full makefile remake. I will investigate what the route via grfcodec can offer additionally to this.

EDIT: I cannot use the grfcodec route for this project. We're using Windows and DOS graphic files mixed together. That's only something NML can handle. However, it cut the build down to 23 seconds (from 1m30), so it's worth changing to the DOS palette completely.

And any speedup in NML itself must be found in the processing of graphic files it appears.

[Eddi]

also see CETS r548 (custom dep check) and CETS r549 (use grfcodec for image compression)

[andythenorth]

EDIT: I cannot use the grfcodec route for this project. We're using Windows and DOS graphic files mixed together.

You can batch palette-convert them, it's a one-off operation and no problem if the direction is Windows -> DOS.

I would use a photoshop action, or maybe write a PIL script for it. Other tools will be able to do it too...

[Bad_Brett]

Is it possibly to create an industry that requires two raw materials? For example, a steel mill requires both iron ore and coal to be able to produce steel.

[Eddi]

yes, you have to use the production callback for that.

[planetmaker]

Is it possibly to create an industry that requires two raw materials? For example, a steel mill requires both iron ore and coal to be able to produce steel.

An industry can have three input and two output cargos. It doesn't matter in the least whether it's flubber, star dust, mutated bunnies, coal or passengers. See the FIRS source code for many examples.

EDIT: Use of a callback is not required in order to have it accept and require two cargos. That is only needed if you want a custom cargo consumption behaviour.

[Bad_Brett]

An industry can have three input and two output cargos. It doesn't matter in the least whether it's flubber, star dust, mutated bunnies, coal or passengers. See the FIRS source code for many examples.

EDIT: Use of a callback is not required in order to have it accept and require two cargos. That is only needed if you want a custom cargo consumption behaviour.

There's a difference between accepting two materials and requiring two materials. For example, my Textile Mill accepts both Cotton and Wool. Either one of these materials can be turned into to cloth, so the ratio doesn't really matter.

But...

A Steel Mill requires both iron ore and coal to be able to produce steel. This means that if only iron ore is supplied, no steel should be produced.

Or let's say I include dyes in the game. It makes sense that the Textile Mill accepts dyes, but dyes alone can not be turned into to cloth.

I thought I could set the input_multiplier parameter to 0, but it seems that these are set to 0 by default. Another idea might be to set individual tiles to accept a cargo, but not the industry itself, so that the tiles of a steel mill accepts coal, but only iron ore is converted.

[FooBar]

If you want to require a cargo, you'll be needing callback magic to do that.

[Bad_Brett]

If you want to require a cargo, you'll be needing callback magic to do that.

Ok, thanks! It's great to hear that there seems to be so many possibilities.