Page 7 of 10

Re: GRFCodec feature discussion thread

Posted: 02 Nov 2007 15:08
by DaleStan
I thought about $ too, but I thought they were harder than necessary to produce on UK keyboards. It appears, however, that I was mistaken. Noted and changed.

Re: GRFCodec feature discussion thread

Posted: 29 Mar 2008 10:08
by George
Sorry, I'm a bit confused.
This page http://www.ttdpatch.net/grfcodec/ specifies "Get the most recent GRFCodec toolset, version 0.9.10", but files in the archive are dated
15/08/2006. Is that the latest version of the codec? If not, where can I download the latest version?

it does not understand \2sto and \2rst, while NFO renum generates them

Re: GRFCodec feature discussion thread

Posted: 29 Mar 2008 20:24
by DaleStan
That would be because NFORenum matches what's stored in Subversion, not what was compiled a few months before \2sto and \2rst were added. Search and replace is your friend, until I figure out a decent method of keeping the parsers from desyncing.

Does any one else machine-parse NFO? If so, my current thought is to add another comment between the "Info version" and "Format:" lines. The easiest way for me to do that would probably be to print a decimal count followed by printf(" %02x:%s",byte,string); for each escape (probably without the backslashes). To use standard RFC nomenclature, writers MUST generate a correct count, but readers MAY ignore it.

This would, of course, only be used for escapes that the hard-coded parser does not already know.

EDIT: No, before Format, not after.

Re: GRFCodec feature discussion thread

Posted: 29 Mar 2008 21:01
by George
DaleStan wrote:That would be because NFORenum matches what's stored in Subversion, not what was compiled a few months before \2sto and \2rst were added. Search and replace is your friend,
:lol: about 10 times a hour or even more often :lol: :lol: :lol: Fortunately I use NFORenum together with my own parser, so it is only 2 strings of code :roll:
DaleStan wrote:until I figure out a decent method of keeping the parsers from desyncing.
Does any one else machine-parse NFO? If so, my current thought is to add another comment after the "// Format:" line. The easiest way for me to do that would probably be to print a decimal count followed by printf(" %02x:%s",byte,string); for each escape (probably without the backslashes). To use standard RFC nomenclature, writers MUST generate a correct count, but readers MAY ignore it.
This would, of course, only be used for escapes that the hard-coded parser does not already know.
Can't you simply realize GRFCodec 0.9.11?

Re: GRFCodec feature discussion thread

Posted: 29 Mar 2008 21:06
by DaleStan
George wrote:Can't you simply realize GRFCodec 0.9.11?
No, because it would break again. And again. And again. I want to fix this once and for all, rather than having to shovel new versions of both out every time a new escape sequence appears.

You can also turn off NFORenum's generation of escape sequences (which still won't make U+E0XX come out quoted.)

Re: GRFCodec feature discussion thread

Posted: 30 Mar 2008 05:34
by George
DaleStan wrote:
George wrote:Can't you simply realize GRFCodec 0.9.11?
No, because it would break again. And again. And again. I want to fix this once and for all, rather than having to shovel new versions of both out every time a new escape sequence appears.
You can also turn off NFORenum's generation of escape sequences (which still won't make U+E0XX come out quoted.)
May be you could merge GRFcodec and NFOrenum into one package then?

Re: GRFCodec feature discussion thread

Posted: 04 Apr 2008 08:36
by Wile E. Coyote
I couldn't find that by search, so sorry if I'm asking something which is allready asked.

Is it possible to add binary format to GRF Codec? There are escape sequences for <dec>, <year>, <hex> and <date>. <bin> could be useful, for example, for properties which are set by bitmask, to avoid calculating, it'll be easier just to type binary number than to convert to hex and if some mistake is made, to convert to binary format again.

Re: GRFCodec feature discussion thread

Posted: 07 Apr 2008 05:32
by DaleStan
After more thought, although I can fix the operand and condition escapes, I wouldn't be able to fix any other escapes that way. This is an improvement, but not perfect.

I can put binary in, but I'm not sure how useful it is. You don't have to do much calculation to convert bitmasks to hex, and \wb1000100000000 is fraught with chances to type too many zeros.
For completeness, maybe I should add octal too (\<bwd>o<oct>) but I feel like that's not often used.

In any case, here's a GRFCodec that will read all current escape sequences. It'll also generate an extra three lines of header. An RFC-style specification for the NFO header is available on request. NFORenum will scramble the lines slightly, but will not break things; the former is a bug I intend to fix. I'm entirely thrilled with that line order for the header, so I've not committed the code; there's a patch instead.

Re: GRFCodec feature discussion thread

Posted: 08 Apr 2008 06:37
by athanasios
Thanks Dale.

Re: GRFCodec feature discussion thread

Posted: 08 Apr 2008 08:20
by Wile E. Coyote
Wow, what a speed! Dale is God!!! :bow: Thanx! :D
And yes, I think too there will be probably no use of octal format.

I'm probably boring, but I have one more idea (not sure is it allready on TODO list?), altough I don't know is it possible (it's probably too much complicated if it's possible): to make GRF Codec "smarter", i.e. to, for example, decode strings only where they are expected to appear (in Action 4, 8, etc. and not in Action 2 chains, now when they are decoded, lots of strange characters appear instead of normal hex numbers).

Re: GRFCodec feature discussion thread

Posted: 08 Apr 2008 14:12
by DaleStan
Wile E. Coyote wrote:Wow, what a speed! Dale is God!!! :bow: Thanx! :D
I think there's a misunderstanding. Binary didn't go in. This is up to date as of just before your request. Sorry to be misleading.
Wile E. Coyote wrote:I'm probably boring, but I have one more idea (not sure is it allready on TODO list?), altough I don't know is it possible (it's probably too much complicated if it's possible): to make GRF Codec "smarter", i.e. to, for example, decode strings only where they are expected to appear (in Action 4, 8, etc. and not in Action 2 chains, now when they are decoded, lots of strange characters appear instead of normal hex numbers).
It is not on the todo. That's what NFORenum is for. On the other hand NFORenum and GRFCodec are, more and more, starting to share code, so maybe it's time for the two to be merged. But I don't want to think about how hard this job would be.

Re: GRFCodec feature discussion thread

Posted: 28 Apr 2009 04:51
by DaleStan
This is similar to the above preview executable, but will correctly read the output from NFORenum r2100. See my latest post in the NFORenum thread for notes on that.

Also, I am still open to suggestions for the missing escapes. Because I've changed how they're parsed, longer escapes can be read with a sane quantity of work.

RPN in escapes is in the TODO. \<bwd>b (binary encoding) is unlikely unless patches for both NFORenum and GRFCodec are provided.

Re: GRFCodec feature discussion thread

Posted: 13 May 2009 20:20
by Rubidium
Attached is a build log with warnings and a diff to fix those warnings. The log is of a compile of GRFCodec r2103 by GCC 4.3 on an x86_64 machine on Linux (Debian/SID).

unused extra bytes in grf with nfo renum working fine

Posted: 21 May 2009 12:48
by planetmaker
Hi,
when opening the depot and thus the purchase screen for trains OpenTTD 16254 issues the following line in the console:

Code: Select all

dbg: [sprite] Ignoring 12 unused extra bytes from the sprite from 2ccdj at position 168866

Code: Select all

renum -w 141 2ccdj.nfo | tee -a renum.log
echo Running GRFCodec. . .
grfcodec -c -e -p 2 2ccdj.grf
The resulting newgrf doesn't issue an error when compiled without parameter -c

I'm using grfcodec 2103E:

Code: Select all

~/ottd/grfdev/grfcodec> make clean && svn up && make grfcodec
rm -rf *.o *.os *.obj *.OBJ *.exe *.EXE *.map *.MAP *.bin grfmrg.ah
At revision 2112.
[CPP] grfcodec.o
grfcodec.cc:102: warning: deprecated conversion from string constant to ‘char*’
grfcodec.cc: In function ‘int main(int, char**)’:
grfcodec.cc:792: warning: deprecated conversion from string constant to ‘char*’
grfcodec.cc:813: warning: deprecated conversion from string constant to ‘char*’
grfcodec.cc:834: warning: deprecated conversion from string constant to ‘char*’
[CC] grfcomm.o
[CPP] pcxfile.o
pcxfile.cc: In member function ‘int pcxfile::subofsx(int, int)’:
pcxfile.cc:446: warning: suggest explicit braces to avoid ambiguous ‘else’
pcxfile.cc: In member function ‘int pcxfile::subofsy(int, int)’:
pcxfile.cc:458: warning: suggest explicit braces to avoid ambiguous ‘else’
[CPP] sprites.o
[CPP] pcxsprit.o
[CPP] info.o
info.cc: In member function ‘virtual void infowriter::newband(pcxfile*)’:
info.cc:278: warning: suggest parentheses around && within ||
[CC] error.o
[CC] getopt.o
[CC] path.o
[CPP] readinfo.o
[CPP] file.o
[LD] grfcodec
~/ottd/grfdev/grfcodec> grfcodec
-bash: /usr/local/bin/grfcodec: No such file or directory
~/ottd/grfdev/grfcodec> ./grfcodec
GRFCodec version 0.9.10 r2103E - Copyright (C) 2000-2005 by Josef Drexler
.
.
.
~/ottd/grfdev/grfcodec> svn diff
Index: Makefile.setup
===================================================================
--- Makefile.setup      (revision 2112)
+++ Makefile.setup      (working copy)
@@ -18,7 +18,7 @@
 # If the boost folder auto-detection fails, set your boost include folder here.
 # If boost can be found in your standard include search paths, setting this to
 # any non-empty value will cause make to assume that boost is present.
-#BOOST_INCLUDE = 
+BOOST_INCLUDE = /usr/include/boost 
 # If make is convinced that you do have boost installed when you do not,
 # uncomment the following line:
 #NO_BOOST = 1
@@ -126,7 +126,7 @@
 grfmerge.exe:  $(GRFMERGESRC:%.c=%.obj)
 
 clean:
-       rm -rf *.o *.os *.obj *.OBJ *.exe *.EXE *.map *.MAP *.bin grfmrg.ah
+       rm -rf *.d *.o *.os *.obj *.OBJ *.exe *.EXE *.map *.MAP *.bin grfmrg.ah
 
 remake: clean all

Re: GRFCodec feature discussion thread

Posted: 22 May 2009 01:59
by DaleStan
The NFO alone is not sufficient for me to watch GRFCodec break; I'll need the PCX for sure. Possibly one or both GRFs as well.

Also, it would be helpful if you could figure out the sprite number. I think I could convince GRFCodec to provide UI, but I'd might end up flailing wildly with a hex editor.

Re: GRFCodec feature discussion thread

Posted: 22 May 2009 09:36
by planetmaker
Oh, sorry, yes, the pcx are needed, of course, too. Attached is a version of the grf's archive.

With that version and OpenTTD 16254 I get

Code: Select all

dbg: [sprite] Ignoring 12 unused extra bytes from the sprite from 2ccdj at position 265148
UnfortunatelyI've no idea as to how to get an idea which sprite actually is responsible for that message and my search for such (grfcodec, grf2html) was so far not fruitful either.

EDIT: you can also obtain the grf via mercurial:
hg clone http://mz.openttdcoop.org/hg/2cctrainset
you'll need something like revision <89 or modify the apprpriate build script to include the -c again for grfcodec.

Re: GRFCodec feature discussion thread

Posted: 22 May 2009 16:13
by frosch
grf2html provides that info

Re: GRFCodec feature discussion thread

Posted: 28 May 2009 23:48
by DaleStan
frosch wrote:grf2html provides that info
Thank you. I hadn't noticed that feature before, but it made my life far far easier.
planetmaker wrote:

Code: Select all

dbg: [sprite] Ignoring 12 unused extra bytes from the sprite from 2ccdj at position 265148
This should be fixed too (r2116).

EDIT: Also, the gcc 4.3 warnings have been silenced. Thanks, Rubidium.

Re: GRFCodec feature discussion thread

Posted: 25 Jul 2009 01:42
by edorfaus
Not sure this is the right place, but best I found anyway...

I wanted to use GRFCodec etc. on an amd64 box, but grfdiff wouldn't compile (nasm complained about elf not supporting 64-bit).

Looking into it, I found out that it must be told to use elf64 explicitly - and managed to make the NASMFORMAT var in the Makefile reflect this when the machine type is x86_64 (and only then, I believe).

This change lets it compile successfully, so I figured I'd tell you about it. (Wasn't sure where to do that though... This forum has far too many subforums and topics (not to mention posts) to go through :P )

Here's the svn diff:

Code: Select all

Index: Makefile
===================================================================
--- Makefile    (revision 2150)
+++ Makefile    (working copy)
@@ -6,7 +6,7 @@
 MAKEFILE = Makefile

 # OS dependent variables
-NASMFORMAT = $(shell [ \( $(ISCYGWIN) -eq 1 \) ] && echo coff || echo elf )
+NASMFORMAT = $(shell [ \( $(ISCYGWIN) -eq 1 \) ] && echo coff || [[ "`uname -m`" == "x86_64" ]] && echo elf64 || echo elf )
 REL_EXE = $(shell [ \( $(ISCYGWIN) -eq 1 \) ] && echo .exe )

 # Borland compiler settings
EDIT:
By the way - I noticed in the grfcodec output, and during searching have seen it mentioned, that pure white pixels are warned against... but I haven't been able to find any clear explanation for why they are considered to be a bad thing. Do you have any pointers or hints on that?

Re: GRFCodec feature discussion thread

Posted: 25 Jul 2009 06:40
by DaleStan
edorfaus wrote:Not sure this is the right place, but best I found anyway...
I am not aware of a better place.

grfdiff and grfmerge are ancient and minimally supported, but it is nice to have them compile. However, it seems like everyone is complaining that nasm-doesn't-like-my-this, nasm-doesn't-like-my-that, nasm-doesn't-like-my-other-thing. I'm rather tempted to write a Perl script to turn it grfmrgc.bin into C and solve the problem once and for all.
edorfaus wrote:By the way - I noticed in the grfcodec output, and during searching have seen it mentioned, that pure white pixels are warned against... but I haven't been able to find any clear explanation for why they are considered to be a bad thing.
  1. If grfcodec is encoding pure white pixels then, 9 times out of 10, you typoed the real-sprite. I could make it more intelligent and only trigger on white pixels on the edge of the sprite, and/or only on full lines of white pixels, but ...
  2. Pure white pixels do not fade out when TTD fades to black, leaving mostly-black with a few bright white dots
There is a visually identical #FCFCFC in the palette that I recommend you use instead.