GRFCodec feature discussion thread

[frosch]

@Csaboka:
If you want it as easy as C: May I suggest you to write a C library instead of a compiler/assembler. I.e. the commands of your "human-readable NFO"-language are C-functions that print NFO to stdout (for example).
Compiling the "h.-r. NFO" then means running the C-compiler and running the C-compiled program, that finally prints the nfo to somewhere.
This way one gets the full strength of C into a preprocesser-level and writing macros will become as simple as C-coding. (well, then it IS C-coding)

(I once used this technique to code a strong macro-assembler for my microcontrollers.)

[Csaboka]

The idea sounds nice and easy (easier that what I plan to do, anyway), but I don't know how exactly this would work for NFO. You can't map an NFO action to a function with fixed parameters, there can be repeated parts in most of them. Varargs would be major pain and we'd lose the type checking C can do.

Maybe it could work with C++ templates, but that would be hell of an ugly syntax...

I'm not dismissing your idea; I just wonder how exactly it could work for NFO.

[DaleStan]

Now that just might be possible. VarAction 2, for example, could be:

Code: Select all

typedef int Variable; // or maybe an enumerator. In any case, appropriate symbolic constants would be defined.
typedef int Feature; // As above
typedef int Operation; // As above

template<typename width>class Adjust {

   // shift-and-add-div/mod. For just shift-and, specify only the first two.
   Adjust(int shift = 0, width mask = -1, bool isDiv=true, width add=0, width divisor =1);

   // Advanced varaction 2
   // probably some variations on this to so not all args have to be specified
   Adjust(int shift, width mask, bool isDiv, width add, width divisor, Operation, Variable, const Adjust<width>&);
};

template<typename width> Range {
   Range(); // to make vector happy
   Range(width min, width max, <type> target);
};

template<typename width> void VarAction2[_other] (Feature, Variable, const Adjust<width>&, const vector<Range<width> >&, <type> default);

As you can see, I haven't figured out what type a cargoID should be, but everything else seems relatively straightforward. To generate an advanced varaction2, just nest the Adjust constructors appropriately. When other things need to be repeated (action 0 props, action 4 strings, &c.), use a vector, or some other container type. The templating is mostly so the output functions can be specialized to output things in the correct width.

You could also provide subclasses of Adjust, eg one for shift-and-add-div, and the other for shift-and-add-mod, but they should all always be intelligent enough to produce the minimum-length encoding.

I'm not certain this will work, but it might.

[Csaboka]

This seems to be an abuse of the template feature. But that doesn't mean anything, things that seem abuses to me seem to be the clever usage of the declarative part of C++ to others.

The only problem I'm seeing right now that you can't construct the vector<Range<width> > parameter in the parameter list, since vector doesn't have any constructor that allows specifying the elements. You only have constructors for empty vector and copying via iterators.

The other downside is that, with the current compilers, having an error in template code makes the compiler spit out long and useless error messages that are intimidating even for those familiar with C++.

[DaleStan]

It is hard to specify an arbitrarily-long list of anything in a type-safe argument list. But not impossible.
You could make a function that takes up to ten $TYPEs, and merges them into one vector<$TYPE>, and another function that takes up to four vector<$TYPE>s and merges them. The limits here are arbitrary, but, I feel, reasonable, at least for action 2s. They may be a little restrictive for action 0.

Probably better, use a class like

Code: Select all

template<typename _Ty> class JoinRanges : public std::vector<_Ty> {
public:
   JoinRanges ();
   JoinRanges (const _Ty& item) {
      push_back(item);
   }
   JoinRanges& operator ()(const _Ty& item) {
      push_back(item);
      return *this;
   }
};

which would allow code like "JoinRanges (first) (second) (third) (fourth) ...". (Have I been looking at boost's abuses of C++ too much?)

Again better, at least for JoinRanges, would be to pass the Range constructor's arguments in for first, &c., instead of requiring that the Range be pre-constructed. Or you could allow both.

As for the type for CargoIDs, they could be magic numbers returned by the *Action2* functions. SpriteIDs, for standard 2s, would be more difficult, though. Maybe construct them with magic number returned by Action1, and a user-supplied index to the desired sprite set?

And I'm lazy; I'll abuse templates, or the preprocessor, or operator overloading, or anything else, if it makes for less typing somewhere else. Or if I just think it'll make the problem more interesting. I'm not particularly tied to any of these suggestions; they just happened to seem good at the time.

[AndersI]

Is mangling NFO into C++ really making it easier to read? Or more accessible to newbies?

[DaleStan]

That all depends on how badly we mangle it.

[michael blunck]

NFO into C++ really making it easier to read? Or more accessible to newbies?

Not really.

But apparently it seems to amuse Dale and Csaba to no end.

regards
Michael

[Csaboka]

DaleStan: If we're abusing things for real, we shouldn't forget the unholy comma operator. I mean, it doesn't really have a function if you don't overload it, but when you do overload it, you can go really creative. We need something like:

Code: Select all

vector<Range> operator, (const Range &left, const Range &right) {
   vector<Range> result;
   result.push_back(left);
   result.push_back(right);
   return result;
}

void operator, (vector<Range> &left, const Range &right) {
   left.push_back(right);
}

And then we can have parameters like

Code: Select all

( Range(...), Range(...), my_range )

and have it magically converted to a vector

EDIT: oh wait, return values are always const, so that wouldn't work. So much about my C++ abuse skills

michael blunck: don't worry, this last post of mine isn't serious. The idea seemed interesting for first, but the more I think about it, the more strange it seems to make GRF coders install a C++ compiler to make NFO coding easier.

[RK]

EDIT: oh wait, return values are always const, so that wouldn't work. So much about my C++ abuse skills

You can allocate memory for it and hustle with pointers.

So any ideas then?

[DaleStan]

If we're abusing things for real, we shouldn't forget the unholy comma operator.

You, sir, are EVIL.
I'd forgotten that wonderful expedient to unreadable code could be overloaded.

Who needs a semicolon when you've got the comma operator?

[DaleStan]

To discuss something else briefly, this bug has just, I hope, been fixed. Here's a executable, if you need it, or if you want any of the other changes since ... 0.9.10, it appears.

[michael blunck]

[...] if you want any of the other changes

Apropos "automatically crop extraneous transparent blue" - What´s the status according to the discussion on the first page of this thread? Especially with the idea of "marking" special full-blue sprites which may not be cropped because of sprite sorter issues?

regards
Michael

[frosch]

Ok, I have set up a little example. Hopefully it is not too easy. To keep it easy I ignored most type-compatibility and the resulting NFO is rather incomplete and possibly wrong. But I tried to include some common things, like CargoTransTable, Action0, VarAction2, Callbacks, Action7, Action10. I hope you will get an idea of how it may work. I think it can be very strong and remain readable.

The compiling would consist of
1) running a C++ compiler.
2) running the compiled programm
3) running nforenum to fix sprite numbers and sprite no. 0
4) running grfcodec
Optionally 2.5/3.5) postprocess Action10 to remove some labels and use relative jumps instead.

[DaleStan]

[...] if you want any of the other changes

Apropos "automatically crop extraneous transparent blue" - What´s the status according to the discussion on the first page of this thread?

As of right this instant? Nothing.

As of half an hour from now? No cropping will be performed if bit 6 (40h) of the compression byte is set, as suggested by Patchman, and the wiki will be updated appropriately.
I hope. I seem to be doing spectacularly well at performing stupid tricks right now.

Ok, I have set up a little example. Hopefully it is not too easy.

I'd really be more interested in seeing the library interface (that is, the contents of nfo.h) than a sample usage of the library.

And by the way, ixnay on the "A0_T_WEIGHT_HIGH"way. The separation of train weight into two separate properties is not something that I think should be revealed to the user. I don't think it should be that hard to make the action0 generator intelligent enough to generate both of those properties.

[michael blunck]

[grfcodec: cropping extraneous transparent blue]

No cropping will be performed if bit 6 (40h) of the compression byte is set [...]

Very nice.

Now, could we set up a list of those full blue sprites in ttdpbase[w].grf which don´t may get cropped because of sprite sorter issues?

AFAIR, there are two in the "elrails" block. Any more in other feature blocks? There are quite some more in the "trkfound" block and one in the "tramtrk" block. What are they for? Could a developer comment on it?

When done with localizing each of them I propose to take a "cropping" run on ttdpbase[w].grf.

regards
Michael

[DaleStan]

The only two *solid* blue sprites are in the elrails block; they were what I used to confirm that GRFCodec was behaving. An appreciable quantity of the trkfound block needs to be marked too, and possibly all of it. The tramtrk sprites don't *seem* to be causing any problem, though. Steven may know more there.

[frosch]

I'd really be more interested in seeing the library interface

Here it is. First I am sorry about some issues with the above example: I had forgotten how awful C's varargs are compared to other languages I use more regulary. Therefore in the presented interface no varargs are present.
Second the action0 was specified transposed.

The interface is a bit ugly when it comes to naming things, and some names are a little long, but it is only meant as a general idea.

And by the way, ixnay on the "A0_T_WEIGHT_HIGH"way.

Well you can always add another level on top of this interface. But on this level I do not agree with you: Currently every property with size 1 to 4 can be specified by a simple Action0 class. You only have to define a single constant.
Adding some intelligence w.r.t. coupled properties will make things a lot more complicated. If you start hiding the second "weight"-property, someone will request to specify properties 1D, 28, 29 together. Here the meaning of prop. 1D depends on the values of prop. 28 and 29.
I do not know NFO properly to give a real example, but: If the meaning of a property changes with values of other properties, and this property can be changed through a callback, this intelligence will break.
What I want to tell with that: It is impossible to hide dependencies, and everyone who wants to use this interface, still needs to know NFO (except the hex-numbers, as said by Csaboka on his "roadmap").

And last:
This has gone further than I wanted it to go. I do not know NFO enough, nor I am experienced enough in NFO-coding to design such a interface thoroughly. My only cause of entering this discussion was to show the strength of the preprocesser when using a C-interface instead of a stand-alone compiler/assembler. E.g. the action0 construction out of a csv-file (I think most newgrf-creators will first collect technical data in some kind of spreadsheet-application.) including unit-conversions; or computation of realsprite-bounds (in the example: sprites with distance 16).
I do not know, how to code stations. But it seems, that it does not matter which station-newgrf to decode, it will always end up with action0's up to 120 kB. I have no clue how the experienced coders create such actions (and I have not had the heart to explore it up to now), but I hope they could be created a lot easier with a strong precompiler. If not, one should consider to copy the paragraph about becoming insane of the patch-readme to the newstations wiki.

[Csaboka]

Actually, this library seems to be a good idea. I'd need to write something like this for the code generator of my program anyway, so it may as well be a separate library with a more user-friendly interface.

For advanced users, this would allow parametrized code generation. For each animated building in TTRS3, I had to add an action2 for each animation frame, the only difference being the building sprite selected. A for loop would have been a lot easier. I imagine that similar common tasks come up in every major GRF; the ability to automatize those would be cool.

People who don't want to learn (a subset of) C++ for the sake of NFO coding would probably still need my planned program. Or would a small tutorial with the needed basics of C++ suffice? I'm not sure about that.

[DaleStan]

The interface is a bit ugly when it comes to naming things, and some names are a little long, but it is only meant as a general idea.

Namespaces may be able to reduce that problem somewhat. Usually, people only use one or two features, so you could declare the property names in namespaces eg "train", "rv", "ship", "industrytile", &c. and then a simple "using namespace building;" would bring all the building::INTRO_DATE type constants into ::, while not making anything else inaccessible.

What's there looks pretty decent. But a lot of complicated stuff is missing. The advanced <varadjust> element, especially, is rather complex, and some of the action 0 properties aren't much better. Now that I think about it some more, I'd make a few changes to the Adjust class above. I'd rename it to VarAdjust, and make it take also the Variable, so the VarAction2 does not. I'd then permit the entire "Variable, int shift = 0, ..." argument list be replaced with a single int, which would expand to reference to var 1A.
I might also add a family of VarAdjust::<OpName>(const VarAdjust&); and VarAdjust::<OpName>(Variable, int shift = 0, ...); functions, instead of or in addition to the constructor nesting above.

People who don't want to learn (a subset of) C++ for the sake of NFO coding would probably still need my planned program. Or would a small tutorial with the needed basics of C++ suffice? I'm not sure about that.

That would depend on how much C++ knowledge is required to write a decent NFO-producing program, and I don't think we can know that until the library is written.