YAGL - Yet Another GRF Language (maybe)

Discussions about the technical aspects of graphics development, including NewGRF tools and utilities.

Moderator: Graphics Moderators

User avatar
UnicycleBloke
Engineer
Engineer
Posts: 71
Joined: 30 Aug 2011 14:39
Location: Cambridge, England

Re: YAGL - Yet Another GRF Language (maybe)

Post by UnicycleBloke »

I don't have any feedback on the project, but I didn't want this thread to pass by with only cricket noises.
Thanks. I really appreciate that.

In fact, I have recently been speaking to wallyweb, who gave it a quick spin, about how to get a little feedback. To this end, I have switched to working in Windows 10 and created a self-contained binary which hopefully people can run without any trouble. I mostly develop embedded firmware, so I'm a bit new to this whole release-online-for-multiple-platforms thing. For some reason I have not yet determined, the Release build (smallish) is unhappy, but the Debug build (largish) is fine: so I have released the larger image... https://github.com/UnicycleBloke/yagl/releases/tag/v0.3. This was built with Visual Studio 2019 for x64 if it matters.

I'm calling this release an Alpha because there are sure to be a ton of little things wrong with it. I'm pretty sure the coverage of the NewGRF specs is complete or very nearly so, including some stuff which I had to find in the OpenTTD source, but I did spot a lacuna just yesterday...

During development, I have focused on reasonably sizeable GRFs such as FIRS in order to get better coverage, but suggest playing with something small at first. In truth, it seems more likely to be useful for the smaller projects which people might be creating in NFO at the moment, but I'm only guessing. I feel confident that it will fall over pretty soon on some silly oversight. Obviously there is not much in the way of documentation: I'm hoping that the YAGL output is at least half-way comprehensible.

Anyway, please give it a whirl and let me know when it goes bang.


Al
User avatar
jfs
Tycoon
Tycoon
Posts: 1750
Joined: 08 Jan 2003 23:09
Location: Denmark

Re: YAGL - Yet Another GRF Language (maybe)

Post by jfs »

If a Release build is broken but a Debug build works, my first suspicion is that you depend on some undefined behaviour, which causes code to compile to one thing in a debug build, and something else when you turn on optimizations.
User avatar
UnicycleBloke
Engineer
Engineer
Posts: 71
Joined: 30 Aug 2011 14:39
Location: Cambridge, England

Re: YAGL - Yet Another GRF Language (maybe)

Post by UnicycleBloke »

undefined behaviour
Maybe. The particular error code I get seems related to heap corruption. This is not generally a feature of my code, but no one is perfect. I have used shared_ptr and STL containers throughout, and perform no direct resource management. I'll see if I can track it down this evening.
User avatar
UnicycleBloke
Engineer
Engineer
Posts: 71
Joined: 30 Aug 2011 14:39
Location: Cambridge, England

Re: YAGL - Yet Another GRF Language (maybe)

Post by UnicycleBloke »

jfs wrote:undefined behaviour
Right first time: twice incrementing a counter between sequence points in the chunked tile decoder. At least it wasn't too hard to find.

This release build executable should work:
yagl.zip
Zipped Windows 10 64-bit EXE
(396.22 KiB) Downloaded 228 times
Please let me know if not.

The repository is at https://github.com/UnicycleBloke/yagl.
User avatar
Gadg8eer
Traffic Manager
Traffic Manager
Posts: 190
Joined: 14 Dec 2019 14:22

Re: YAGL - Yet Another GRF Language (maybe)

Post by Gadg8eer »

Wait, so is this a method of decompiling NFO, a new programming language for OpenTTD mods, both, or nether? It looks impressive.
I have Asperger's, please be easy on me about stuff. My apologies if I've been a problem for you in the past.
User avatar
UnicycleBloke
Engineer
Engineer
Posts: 71
Joined: 30 Aug 2011 14:39
Location: Cambridge, England

Re: YAGL - Yet Another GRF Language (maybe)

Post by UnicycleBloke »

Gadg8eer wrote: 20 Feb 2020 03:49 Wait, so is this a method of decompiling NFO, a new programming language for OpenTTD mods, both, or nether? It looks impressive.
Thanks very much. :)

yagl is a tool for both compiling and decompiling GRFs from and to a new script language, YAGL. This is broadly comparable in function to grfcodec, which compiles and decompiles GRFs from and to NFO. Unlike NML, YAGL is a low level script: there is a direct one-to-one correspondence between blocks in YAGL and the equivalent records in NFO. The difference is that YAGL is intended to be more human readable than NFO, and things like the sizes of structures, lengths of arrays, lengths of strings, and so on are worked out automatically.

My original goal was just to decompile GRFs created with NML to see what was inside them in terms of the low level NewGRF specs, but I got carried away. I think the most likely use case would be to decompile an existing GRF, make a few mods (e.g add or fix some translations, or whatever), and then re-compile. If you have the NML source for the GRF, using nmlc would probably be the better choice. There is no reason why you could not create a whole new GRF written in YAGL, if you were so minded. This would be equivalent to creating a GRF in NFO. The major barrier to this is that I have not yet written any significant documentation. I'm thinking about that... Also yagl does not have any of the linting capabilities that grfcodec does. I'm not sure to what extent they are needed. Adding specific checks for things wouldn't be too hard.

Since yagl has the ability to read and write a GRF into memory, it would in principle be quite simple to read and write NFO too (at lot easier than YAGL was, at any rate). I don't know if that would be useful at all.

Please have a go with it, and let me know how it works out. There are sure to be some bugs. If you simply decompile/recompile your favourite GRF, and then it no longer works in openttd, I'd very much like to know about that. As far as I can tell, all the discrepancies between the before and after GRFs are innocuous.
User avatar
kamnet
Moderator
Moderator
Posts: 8548
Joined: 28 Sep 2009 17:15
Location: Eastern KY
Contact:

Re: YAGL - Yet Another GRF Language (maybe)

Post by kamnet »

I would love to see some documentation and tutorials developed. I simply have not been able to wrap my head around NML, I seem to have a little easier time with NFO, but that's hard for a lot of people to work on. Perhaps this can be a nice in-between and help me step up into NML, or maybe just be able to start cranking out some projects faster.
User avatar
UnicycleBloke
Engineer
Engineer
Posts: 71
Joined: 30 Aug 2011 14:39
Location: Cambridge, England

Re: YAGL - Yet Another GRF Language (maybe)

Post by UnicycleBloke »

kamnet wrote: 21 Feb 2020 00:09 I would love to see some documentation and tutorials developed.
I have been a bit distracted but have started a little documentation. It's mostly a pretty thin thing at the moment, barely a beginning with some structure but lots of empty pages, but I would be very grateful for feedback. These pages and a few others are worth a look:
https://github.com/UnicycleBloke/yagl/b ... s/intro.md
https://github.com/UnicycleBloke/yagl/b ... _syntax.md
https://github.com/UnicycleBloke/yagl/b ... ction01.md

The key thing to note is that YAGL blocks directly map onto NFO records. If you understand NFO, my hope is that using YAGL will be a breeze. For this reason, I am hoping that the documentation need not be too much more than a gloss over the existing NewGRF specs on which I based my software. I at least need to explain the structure of each block. And I use a whole bunch of enumerations with keywords like "Aircraft" where NFO would have the number 03 for that feature. I need to explain those, too.

But even before all that is complete there is nothing preventing you from using grfcodec to compile one of your existing NFO projects, and then decompiling with yagl just to have a look. It will be a lot easier to know what to add if people actually have a go and then ask questions, or complain, or whatever. :)

I have attached the latest Windows version of the executable, which includes a number of minor changes and fixes related to 32bpp decoding:
yagl.zip
(379.33 KiB) Downloaded 109 times
User avatar
UnicycleBloke
Engineer
Engineer
Posts: 71
Joined: 30 Aug 2011 14:39
Location: Cambridge, England

Re: YAGL - Yet Another GRF Language (maybe)

Post by UnicycleBloke »

Work continues - slowly - on the documentation. With Wallyweb's help, I found a little bug which prevented non-vehicle GRFs from working (a slight misreading of the spec on my part). This was simple to fix and is included in the attached Windows build.

Just for fun, I tested this build by making a small modification to Dutch Trainset 2: https://www.tt-wiki.net/wiki/Dutch_Trainset_2. This has been my default test bed all along.

First I decoded the GRF into YAGL and a sprite sheet:

Code: Select all

yagl.exe -d dutchtrains.grf 
Then I edited the name of the "NS 5500 (Steam)" train a little in the file sprites/dutchtrains.yagl:

Code: Select all

// Record #988
strings<Trains, default, 0x013A> // <feature, language, first_id> Action04, Default
{
    /* 0x013A */ "{black}NS 5500 (Steam {green}locomotive {red}(added in YAGL){black})"; 
}
// Record #989
strings<Trains, de_DE, 0x013A> // <feature, language, first_id> Action04, German
{
    /* 0x013A */ "NS 5500 (Dampf)";
}
Then I re-encoded the GRF:

Code: Select all

yagl.exe -e dutchtrains.grf 
Using the revised GRF in game, I got this:
trains.png
trains.png (20.57 KiB) Viewed 5545 times
Not that I'm recommending changing the works of others. Just trying to show how it could work with something you've already written.

As ever, please give it a try and let me have any feedback. Same for the fledgling documentation.
Attachments
yagl.zip
(375.16 KiB) Downloaded 105 times
User avatar
kamnet
Moderator
Moderator
Posts: 8548
Joined: 28 Sep 2009 17:15
Location: Eastern KY
Contact:

Re: YAGL - Yet Another GRF Language (maybe)

Post by kamnet »

I got a bunch of errors trying to run the program.

The code execution cannot be processed because MSVCP140_CODECVT_IDS.dll was not found.
The code execution cannot proceed because MSVCP140.dll was not found.
The code execution cannot proceed because VCRUNTIME140_1.dll was not found.
the code execution cannot proceed because VCRUNTIME140.dll was not found.
User avatar
UnicycleBloke
Engineer
Engineer
Posts: 71
Joined: 30 Aug 2011 14:39
Location: Cambridge, England

Re: YAGL - Yet Another GRF Language (maybe)

Post by UnicycleBloke »

kamnet wrote: 05 Apr 2020 19:54 I got a bunch of errors trying to run the program.
Bugger! These are Visual Studio re-distributable files. I was pretty sure that using a static build would eliminate this sort of thing.

This page appears to contain what you need: https://support.microsoft.com/en-gb/hel ... -downloads. The software was built with Visual Studio 2019.

Please let me know if installing the relevant package fixes the issue, and I'll update the documentation.


Al
User avatar
UnicycleBloke
Engineer
Engineer
Posts: 71
Joined: 30 Aug 2011 14:39
Location: Cambridge, England

Re: YAGL - Yet Another GRF Language (maybe)

Post by UnicycleBloke »

Oops. Further investigation showed that I had not set up the static build options as I thought. The attached image only depends on kernel32.dll, which is part of Windows.
yagl.zip
(575.69 KiB) Downloaded 142 times
Sorry for the confusion.
User avatar
UnicycleBloke
Engineer
Engineer
Posts: 71
Joined: 30 Aug 2011 14:39
Location: Cambridge, England

Re: YAGL - Yet Another GRF Language (maybe)

Post by UnicycleBloke »

andythenorth wrote: 05 Apr 2020 19:33 It's not something I can use, I've too much work tied up with nml, but it's nice to see a new toolchain develop.
Sure. But I hope you'll give it a spin. My original goal was to look inside the GRFs generated with NML. I thought it would be interesting to see how the NML constructs are mapped onto the underlying NewGRF specs used for NFO. I wondered if it might help with debugging or something.
User avatar
kamnet
Moderator
Moderator
Posts: 8548
Joined: 28 Sep 2009 17:15
Location: Eastern KY
Contact:

Re: YAGL - Yet Another GRF Language (maybe)

Post by kamnet »

UnicycleBloke wrote: 06 Apr 2020 13:47 Sure. But I hope you'll give it a spin. My original goal was to look inside the GRFs generated with NML. I thought it would be interesting to see how the NML constructs are mapped onto the underlying NewGRF specs used for NFO. I wondered if it might help with debugging or something.
I'm using it to help demystify uncommented NFO source code, especially when the only thing that exists is the compiled GRF. There's a lot of NewGRFs whose authors, experience and documentation have disappeared, leaving behind only the finished product.

Right now I'm working on Zepherys' Ekranoplan file. I hacked on it a few years ago and was able to introduce some new variables and make some minor adjustments, but much of the code was just unreadable for me. Having it decoded into much more human-readable YAGL, I've been able to make some significant improvements. Hopefully within the next few days I will be able to provide a better set with some commented code.
User avatar
UnicycleBloke
Engineer
Engineer
Posts: 71
Joined: 30 Aug 2011 14:39
Location: Cambridge, England

Re: YAGL - Yet Another GRF Language (maybe)

Post by UnicycleBloke »

kamnet wrote: 06 Apr 2020 15:38 Having it decoded into much more human-readable YAGL, I've been able to make some significant improvements.
You just made my day! Let me know if there is anything I can help with or add to the software to make life easier.
User avatar
kamnet
Moderator
Moderator
Posts: 8548
Joined: 28 Sep 2009 17:15
Location: Eastern KY
Contact:

Re: YAGL - Yet Another GRF Language (maybe)

Post by kamnet »

Right now I'm getting stuck trying to increase the cost of purchasing a vehicle. I discovered the base cost property, and at first increasing it worked, and then it apparently stopped working, so I'm not sure what the issue is.
User avatar
wallyweb
Tycoon
Tycoon
Posts: 6102
Joined: 27 Nov 2004 15:05
Location: Canada

Re: YAGL - Yet Another GRF Language (maybe)

Post by wallyweb »

kamnet wrote: 06 Apr 2020 20:19 Right now I'm getting stuck trying to increase the cost of purchasing a vehicle. I discovered the base cost property, and at first increasing it worked, and then it apparently stopped working, so I'm not sure what the issue is.
You might need to consult MacBeth's three witches who noted toil and trouble.
The cost of each vehicle type can be adjusted in two places:
Action 0 for global settings
Action 0 for the vehicle type

Code: Select all

08 	Global Settings - Cost base multipliers (08)
00 	Trains - Cost factor (17)
01 	Road Vehicles - Cost factor (11)
02 	Ships - Cost factor (0A)
03 	Aircraft - Cost factor (0B)
and then we encounter the esoteric and all pervasive rate of inflation.
User avatar
kamnet
Moderator
Moderator
Posts: 8548
Joined: 28 Sep 2009 17:15
Location: Eastern KY
Contact:

Re: YAGL - Yet Another GRF Language (maybe)

Post by kamnet »

Here's what I have so far:

Code: Select all

properties<GlobalSettings, 0x0000> // Action00
{
    // instance_id: 0x0000
    {
        cost_base_multipliers: 0xFF;
    }
}

Code: Select all

properties<Ships, 0x0000> // Action00
{
    // instance_id: 0x0000
    {
        introduction_date: 0x0000;
        reliability_decay_speed: 32;
        vehicle_life_years: 30;
        model_life_years: 40;
        climate_availability: Temperate | Arctic | Tropical | Toyland;
        sprite_id: 0xFF;
        is_refittable: true;
        cost_factor: 0xFF;
        speed_2_kmh: 0xFF;
        cargo_type: 0xFF;
        cargo_capacity: 0x0F28;
        sound_effect_type: 0x07;
        refit_cargo_types: 0x00000000;
        callback_flags_mask: 0x00;
        refit_cost: 0x32;
        ocean_speed_fraction: 0x00;
        canal_speed_fraction: 0x5B;
        miscellaneous_flags: 0x02;
        refittable_cargo_classes: 0x01D1;
        non_refittable_cargo_classes: 0x0000;
    }
}
This should result in a ridiculously high $6.3 billion for cost. But no matter what I set base cost to, it doesn't affect cost factor. When I adjust cost factor between 00 and FF, it ranges appropriately from $0 to $97k.
Post Reply

Return to “NewGRF Technical Discussions”

Who is online

Users browsing this forum: No registered users and 4 guests