Planned additions and changes: NewGRF v8

[planetmaker]

Over the course of years and months a number of limitations and inconsistencies with the current NewGRF version 7 crept up. Frosch industriously has been gathering the ideas and implemented solutions for many of these things which expand existing ideas in a more future-proof way, some which prepare the ground for potentially upcoming features and addtions and others flatten-out some inconsistencies which crept in in the course of time.

There's a summary of changes planned for NewGRF v8.

We'd like to ask you for feedback and critical review, maybe we miss something important. Please also put specific attention on the changes to how heightlevels will / should be treated. Also do we miss something in that list which needs a change and a grf version bump?

The time frame for comments is about roughly a week from now on

[Eddi]

one request that popped up recently:
in the refit cost callback, treat the result as signed value, to allow negative cost (e.g. HEQS trams refitting from 15 wagons to 4 wagons should give some money back, to simulate selling of wagons)

[DaleStan]

There's an untruth in that page. "Just like GRFs have to use GRF version 7 to correctly support new cargos" is false. GRF support for newcargos was added before GRF version 7, and nothing in version 7 is required for newcargos[0]. Version 7 was (at least according to the wiki) defined with the release of 2.5 beta 1, not with the release of newcargos.

Variable 20/A0:
Nearby tile info variables of all features (e.g. industry tile variable 60)
Callback 28 variable 8A
With grf version < 8 and more height levels, I suggest clamping, not masking.
Generally, this is 00-FE: as before; FF: pixel heights > FE
For snow line in particular, I'd recommend this variant
00-FD: as before
FE: pixel (not height-level) heights > FD
FF: no snow line

I believe this will cause the least breakage possible:
GRFs that mask out the high bytes will continue to work correctly as long as the snow line is below the FF barrier, and will do unspecified things when the snowline hits FE, but will not fall back to their no-snow-line behaviour:
GRFs that do not mask out the high bytes, despite instructions to the contrary, have the same behaviour as those that do, and do not gain any undue advantage for their misbehaviour.

Alternatively, scale these to some meaningful range, so that they can be compared against each other, but not necessarily to constants (other than FF). MB and George will know if this will break them.

Vehicle variable 44 "aircraft altitude"
This is height above ground, not absolute height. I'd encourage clamping in version 8 too. I see no reason why a GRF would need to know the difference between "255 pixels above the ground" and "more than 255 pixels above the ground". Especially since sprites themselves cannot exceed 255 pixels high.

Callback 144 /var10:
Documented as "xTHHRRtt", but no documentation is given for T.




A part of me is tempted to put fill reserved bytes with implementation-defined randomness. Obviously this would have to be multi-player safe, but it would discourage misuse of reserved bits/bytes as known-zero. I don't really have any great expectation of seeing this actually happen.

[0] To my recollection. Feel free to cite particular features otherwise. Note that newcargos was added in a56, and 2.5b1 followed a73.

[Kogut]

"Revise height variables" so morehightlevels is closer to trunk? Thanks!

[Eddi]

With grf version < 8 and more height levels, I suggest clamping, not masking.

i tend to agree here, see the previous discussion

For snow line in particular, I'd recommend this variant
00-FD: as before
FE: pixel (not height-level) heights > FD
FF: no snow line

i don't think this is necessary, since current maximum heightlevel 15 is 15*8 = 120 = 0x78, so there's plenty of "unused" space between 0x78 and 0xFF that the corner case you are (supposedly) trying to avoid here never occurs for old grfs. and new grfs should just use version 8.

Alternatively, scale these to some meaningful range, so that they can be compared against each other, but not necessarily to constants (other than FF). MB and George will know if this will break them.

i don't think that quite works, e.g. FIRS does a "magic +8" in checks for current snowline height, which would break by some other "magic scaling"

[DaleStan]

A part of me is tempted to put fill reserved bytes with implementation-defined randomness. Obviously this would have to be multi-player safe, but it would discourage misuse of reserved bits/bytes as known-zero. I don't really have any great expectation of seeing this actually happen.

Upon further thought, this could also encourage the abuse of these bits as spare randoms, depending on how the implementation was defined. So care would have to be taken.

For snow line in particular, I'd recommend this variant <snip>

i don't think this is necessary, since current maximum heightlevel 15 is 15*8 = 120 = 0x78, so there's plenty of "unused" space between 0x78 and 0xFF that the corner case you are (supposedly) trying to avoid here never occurs for old grfs.

It never occurs today, but that's not what we're trying to guard against. What happens when a version 6 or 7 grf is loaded in a game with at least 32 height levels?

[ChillCore]

It never occurs today, but that's not what we're trying to guard against. What happens when a version 6 or 7 grf is loaded in a game with at least 32 height levels?

Can you please explain what a version 6 or 7 grf is exactly as I am not at all familiar with grf callbacks.
Also if you point me to one I'd be glad to test and report as I am going to have a closer look at the mhl code again for the next couple of days (trying to get it to function correctly again after recent trunk commits).

[Rubidium]

It never occurs today, but that's not what we're trying to guard against. What happens when a version 6 or 7 grf is loaded in a game with at least 32 height levels?

I'd say that clamping is the least bad solution for those NewGRFs. For GRFv8 we intend to use the number of height levels instead of pixels when returning heights (including snow lines). So we don't use more bits for the return value. The variable will just return the height level for version 8 instead of height level in pixels for versions before 8.

So for a GRFv7 you'd get 0, 8, 16, 24, ..., for GRFv8 you'd get 0, 1, 2, 3, ... There is, however, a problem with clamping height levels (in pixels) as they would not allow distinguishing between 32 or 64. But what if the snow level is 64? Then it'd return FE for the height level of the snow, and for anything from 32-64 the NewGRF would think it is snowy. We could make the height levels somewhat related to the snow level, so you return 0 for height levels 0-48, use 08..FD for 49-79, and FE for 80 and higher.
The major caveat with that is things that want to, e.g., build at height level 1. That wouldn't work anymore.

So all in all I'd say: for GRF versions before 8 use heightlevel in pixels clamped to FE. For GRF version 8 use the actual height level as number.

[Eddi]

Can you please explain what a version 6 or 7 grf is exactly as I am not at all familiar with grf callbacks.

the version of the GRF specs that a grf is using is defined in the Action 8 of the GRF. The GRF (specs) version needs to be increased whenever the semantics of an existing language feature changes in a not-backwards-compatible way (adding new features like railtypes or airports does not need that, as no existing GRF will change its behaviour). e.g between version 6 and version 7, the lang-IDs were changed to allow support for more languages than the ones originally supported by TTD.

So if a GRF declares itself as version 6, then openttd translates the lang-IDs into the old format (losing information when one of the additional languages is used, falling back to english).

The same thing will happen for more height levels, GRFs that don't use the new version 8 format will have trouble/glitches when more than 16 heightlevels are used. it may be fine up to 32, but definitely not beyond that... the question here is how to minimize these glitches.

[supermop]

I would request more than 6 unique sprites per bridge, and the ability to define custom sprite layouts for bridges (that is, the order in which those 6+ sprites can be displayed).

[PikkaBird]

I would request more than 6 unique sprites per bridge, and the ability to define custom sprite layouts for bridges (that is, the order in which those 6+ sprites can be displayed).

That kind of feature request is beyond the scope of this discussion...

[frosch]

Thanks everyone for the review and feedback.

I incorporated these changes to the wiki / the patch queue.
* Callback 10 "visual effect" is kept and not integrated into CB 36 since it is very likely to gain specific var10/18 meanings and other return values at some point.
* Var 17/97 (< version 8 ), var 20/A0 (< version 8 ), vehicle var 44 (any version), CB 28 var 8A (< version 8 ) and the nearby tile informations (< version 8 ) are now clamped to byte values. So heightlevels >= 32 will return 0xFF resp. 0xFE for snowline-related variables. (0xFF means 'no snow')
* The changes to CB 144 are now part of the queue, since OpenTTD recently gained support for it.

[Michi_cc]

one request that popped up recently:
in the refit cost callback, treat the result as signed value, to allow negative cost (e.g. HEQS trams refitting from 15 wagons to 4 wagons should give some money back, to simulate selling of wagons)

Done in r23124.

-- Michael Lutz

[ChillCore]

* Var 17/97 (< version 8 ), var 20/A0 (< version 8 ), vehicle var 44 (any version), CB 28 var 8A (< version 8 ) and the nearby tile informations (< version 8 ) are now clamped to byte values. So heightlevels >= 32 will return 0xFF resp. 0xFE for snowline-related variables. (0xFF means 'no snow')

Just a heads up ...
When ic111 started with the mhl patch we had some glitches that started to appear before reaching level 32, IIRC.

See:
- The last picture in this post http://www.tt-forums.net/viewtopic.php? ... 4&start=71 and some explanation about the glitches (after testing) here http://www.tt-forums.net/viewtopic.php? ... 4&start=85 . Ofcourse I forgot to write down the exact heightlevel at the time but I guess I could still count them (if only I had not zoomed out so much).
- Another picture here http://www.tt-forums.net/viewtopic.php? ... 4&start=88
- And ic111 his post after testing some more http://www.tt-forums.net/viewtopic.php? ... 4&start=90


Maybe it would be wiser to set the clamping lower to prevent any possible problems?
Personally I'd clamp to level 16 (15 without counting water) for version 7 and older.


ps:
Sorry for not giving more feedback but you guys are speaking a language I do not understand yet so feel free to ignore me on this matter.

[Eddi]

See:
[...]

i don't think any of these are related to what is being discussed here.

[Eddi]

ah, something i remembered just now: unify the format of "speed units" throughout vehicle types. e.g. a word-sized value XXXY, where Y is treated as the fractional value. all values in km/h (exactly, not -ish). resulting range is then 0.0625km/h to 4095.9375km/h, which *should* be enough for any sane vehicle. (i hereby declare "logic train" as "not sane" ) this allows for exchanging the internal handling in the future.

[Terkhen]

What about unifying weight for different vehicles too?. I was also wondering if making all "date" properties extended would be desirable.

[planetmaker]

Then we also need unified power.

[Supercheese]

And while we're at it, why not unified field theory?

[Snail]

Hi guys,

One of the features that I find most useful in the NewGRF structure is the "user-defined bit mask".
As of now, if I understand correctly, it has the size of one byte per vehicle.

Would it be possible to allocate more bits to it, so to make it a word instead of a byte? I'm planning to use it for the trainsets I'm developing, and it would be very helpful to be able to play with 16 bits instead of 8...

Thanks in advance!