Potential NFO format update

[DaleStan]

Would it be possible to allow those string codes (e.g. \94) found in ttdpttxt.txt to be placed in plain text strings in action-4 and then have them converted to hex by GRFCodec. That would make it simpler to define those F8xx texts in a .grf.

Yes. (That is, it will be possible.)

I've put together a few other ideas over here.

Take a look, and tell me what you think.

[michael blunck]

Mmh, the idea comes to mind to work towards a more general method of text replacement, not just sticking to that (just as bad) cryptic backslash-number-letter feature?

regards
Michael

[DaleStan]

I'm not quite following.
Do you want to type "*0 07 00 00 bit_set 02 05"? "If bit 2 of parameter 0 is set, skip the next five sprites"? Something else?

Granted, \7g, \7G, \7gg, \7gG, and \7GG are not particularly good, but I couldn't come up with anything better, and I do think they're better than typing 06..0A.

[Csaboka]

I agree with Michael. Decrypting something like \2+ isn't much better than decrypting a hex byte.

If we want to make NFOs more readable, I think we should make them more like a high level language instead, making it more verbose, but more human-readable at the same time.

Something like this comes to mind:
instead of:

Code: Select all

02 00 00 85 0A 00 FF FF 03
00 FF 00 00 10 00
01 00 11 00 15 00
05 FF 40 00 50 00
03 FF

we have

Code: Select all

action2 word trains id 0 var(0a) shift 0 mask FFFF
   case 00..10: FF00
   case 11..15: 0001
   case 40..50: FF05
   case default: FF03

(I've just made up some example syntax, there's probably a better one. I've just made up the action as well, it doesn't do anything meaningful.)

Personally, I always need to use the wiki when I want to code something, because I can't remember the format of the actions. (Using GRFMaker solves this problem, but brings up others, so I can't always use it.) If NFO used keywords instead of arbitrary bytes, it would be a lot easier to remember and read, not to mention you could omit some parts (for example, omitting shift and mask would simply mean shift=0 and mask=FF).

This, of course, may be too much work for too little benefit, but I think it would make more sense than making cryptic shortcuts for cryptic originals.

[DaleStan]

I agree with Michael. Decrypting something like \2+ isn't much better than decrypting a hex byte.

Personally, I always need to use the wiki when I want to code something, because I can't remember the format of the actions.

I must code entirely too much NFO then. I need to check the wiki when writing action 0, 6, and 12s, when using most of the listed constructs, when checking bits in action 7 variable 85, and when looking for an action2 variable that contains the info I want.
I'd probably have to check for 5s, As, and Fs, but those are rare enough that it's not annoying.

This, of course, may be too much work for too little benefit, but I think it would make more sense than making cryptic shortcuts for cryptic originals.

ObNit: Unless you're talking about the quoted escapes, they're cryptic long-cuts. The originals are all two characters long, and case insensitive, while the escape sequences run from three to five characters, and are case sensitive.

Personally, I would expect even the ugliest of those (with the possible exception of \U####; a hex string really isn't better than a single character) to be better than its byte(s), or at least no worse (\[bwd].*)

If we want to make NFOs more readable, I think we should make them more like a high level language instead, making it more verbose, but more human-readable at the same time.

That's tantamount to writing a compiler, something I'd prefer to avoid, at least not without a more complete format.
At one point I had generated a very C-like action 2 format:

Code: Select all

CapacityCallback(){
 switch_byte(var47&FF){
  case 00:fail;//Passengers
  case 01:return d132; //tourists
  default:return (d112*d16)/(var47[1]&FF);
 }
}

CallbackCheck(){
 switch_byte(var0C){
  case 19:CapacityCallback();
  case 15:LiveryCallback();
  default:GetGraphics();
 }
}

And so on, but I haven't done anything similar for Action1, Standard2, or Action3, and writing a parser for that does not strike me as particularly easy.
(Parsing backslash-escapes, OTOH, is quite easy.)

In a different vein, it shouldn't be too hard (just tedious) to create a header file with a bunch of defines:
#define action0 00
...
#define random_self 80
#define byte_self 81
#define byte_related 82
#define random_related 83
...
#define action2_add 00
...
and then #include it in an NFO and run the C preprocessor before pointing NFORenum or grfcodec at it.
This doesn't solve the format problem, though; you still have to get all the right keywords in all the right orders.
I'd also like to avoid adding a C preprocessor to grfcodec, or settling on a format that requires a Makefile to encode.

[Korenn]

what Csaboka proposed is not really a compiler but more a translator, it has a 1-1 relation to the hex codes, so all it would require is a pre-processor before grfcodec. (granted, it's a big "all")

[edit] doh I should read the whole post before replying. though having to install a c compiler to use grfcodec doesn't sound very accessable.

[DaleStan]

what Csaboka proposed is not really a compiler but more a translator, it has a 1-1 relation to the hex codes,

Almost. Not quite. And less so if the "shift 0" and "mask FF"/"mask FFFF"/"mask FFFFFFFF" are made implicit.

Code: Select all

02 00 00 85 0A 00 FF FF 03
00 FF 00 00 10 00 
//...

becomes

Code: Select all

action2 word trains id 0 var(0a) shift 0 mask FFFF
// 02    85    00    00    0A      00      FF FF       03
   case 00 .. 10:   FF05
//     00 00 10 00  05 FF

though having to install a c compiler to use grfcodec doesn't sound very accessable.

No no no no.
Either grfcodec would understand those constructs, or a separate compiler/assembler would be distributed with grfcodec that can assemble the C-like code to NFO and/or GRF.

[Wile E. Coyote]

I like Csaboka's idea. Altough is DaleStan's NFO renum very useful to warn you if you make some mistakes in code, that could be more understandable code than hex only.
(I know I'm noob in coding. That's probably reason why i like idea to write English words instead of hex code.)

[SpComb]

XML?

But I think XML is somewhat ugly... a more textual format with keywords and data would look nicer.

[eis_os]

A reason I didn't continue it because most dev
a) use grfmaker
b) dev think it was to much to write.

It was able to interprete some actions already...

[Csaboka]

I was just saying what I would like; if no one feels like coding it, it's fine with me. Actually, I'm learning about compilers in this term, so I might be able to code something useful, but don't hold your breath.

As for using XML: I don't think that's an useful idea. XML is too verbose to read or write with an ordinary text editor, plus, it forces you to use hierarchy even in places where it's not really needed. Just compare with my original example with an XML version:

Code: Select all

action2 word trains id 0 var(0a) shift 0 mask FFFF 
   case 00..10: FF00 
   case 11..15: 0001 
   case 40..50: FF05 
   case default: FF03

Code: Select all

<action2 varsize="word" feature="trains" cargoid="0" variable="0a" shift="0" mask="FFFF">
   <range low="00" high="10" value="FF00" />
   <range low="11" high="15" value="0001" />
   <range low="40" high="50" value="FF05" />
   <defcase value="FF03" />
</action2>

If you had to type lots of those, which one would you prefer? And, if you just want to skim through a file looking for a particular action, in which version would you find it easier: the one which is readable while still being terse, or the one that has a lot of "noise" (property names repeated over and over again)?

[SpComb]

As well, there isn't really any need to make it create .grf files directly from the format. I would imagine that that isn't exactly easy. If it's easier, a translator between NFO and this language would probably be simpler to make (could be wrong), and work just as well. Dunno about comments...

[DaleStan]

As well, there isn't really any need to make it create .grf files directly from the format. I would imagine that that isn't exactly easy. If it's easier, a translator between NFO and this language would probably be simpler to make (could be wrong), and work just as well.

Almost as well. As I've said before, I'd prefer that creating a GRF not require a Makefile.
grfcodec and NFORenum now (as of r148) use basically the same reading engine, so it's hardly more difficult to change one than it is to change both.

Dunno about comments...

My view is that NFO is to GRF the way source is to executable, and even more so for any text-based format. Anything that doesn't belong in an executable (such as comments) doesn't belong in a grf either. If I write a converter, it will quite happily ignore comments.

I was just saying what I would like;

I've recently gotten some ideas that make decent sense regarding such a text-driven format.

I may implement something like that later, but I'm currently looking for comments on the list of backslashed escape sequences I posted.

I will implement most of the listed escape sequences. Most. But not all. The current questions are:
1) Is there anything you want implemented? (Whether listed or not; some of the listed items are not on my personal todo; a request will ensure that it does get implemented.)
2) Is there anything listed you think particularly useless?
3) Is there anything listed that you'd prefer to see implemented with a different escaped string?

[eis_os]

Well make the preprocessor extensible...

[DaleStan]

I'm not sure I understand you. You want to be able to add additional escape sequences without recompiling?

For simple ones (\2*, for example), this shouldn't be too hard.

[eis_os]

well, in source, so you don't need to change 10 files to add a new one

[DaleStan]

After a lot of playing with grfcodec, a lot of playing with nforenum, and a little just doing nothing at all (OK, so maybe my proportions are off there.), I have a copy of grfcodec that can read all and write many of the above-mentioned escape sequences.

If you want to compile for yourself, or investigate the source, check out revision 310 from [url]svn://svn.ttdpatch.net/misc/grfcodec[/url].

[DaleStan]

As some of you may have noticed, I've somewhat vanished from these fora.

Whether that's a good thing or a bad thing is open for debate, but please do so somewhere else.

I do have regular access to my computer (where I do my grfcodec and nforenum development), and regular read-only access to these fora, so feel free to continue discussing any proposed updates to the language; I'll do my best to either reply or implement promptly, but no guarantees.

I do not have regular SVN access, so commits will tend to be in bursts as seen earlier today.

Currently, the best specification I have states that the first non-whitespace characters in "English" sprites be the characters "action" (case-sensitive(?)) followed by the action number, in hex. What follows depends on the action being defined, and may usually be in any order:

1. Actions 0..4 require a feature (eg "trains", "rvs", "ships", ...)
2. Action 2 requires "word" or "dword", and may optionally specify "byte"
3. Action 3 requires a list of vIDs, as "vid(x,y,z,...)", or "vid()" for generics
4. Action 2 may specify either a CargoID, as "id(##)", or a name, as "id(foo)". ## must be exactly two hex characters in the 00..7F range, else it is a name. The CargoID associated with a name is undefined, but will be in the 80..FF range.
5. After all necessary specifiers, Actions 2 and 3 take the following format:

   Code: Select all

   switch(expr){ // "(expr)" is only valid for Action 2, and may be any valid C expression, using the operators +, -, /, %, *, &, |, ^, and the functions min, max, umin, and umax.
   // The four functions take two or more arguments. Where relevant, the operators are the unsigned versions.
   // In action 3, (expr) is implicitly the translated cargo type.
   // RPN expression instead? -- Far easier for me to implement.
   case foo[..bar]: cid(cid); // "[..bar]" is only valid in Action 2.
   case baz[..quz]: return expr; // the stated callback return 
   // any number of case expressions is permitted.
   // foo, &c. may be expressed in decimal or hex (0x prefix).
   default: id(vid); // exactly one of these is required.
   default: return expr; // exactly one of these is required. This format is only valid in Action 2.
   }

   Alternatively, the above may be simply be replaced with either "id(vid);" or "return expr;

In most cases, exactly one of each of the above is required.

Action 1 specifies "sets(x)" and "sprites(y)", where x and y are any appropriately ranged decimal or hexidecimal constant.

Whitespace is treated generally as in C/C++.

Action 5 follows the same general pattern as 1.

Similar specifications are required for the remaining actions.
Feel free to suggest modifications and/or new specs for missing actions.