32bit Graphics Extra Zoom Patch

[Jupix]

Any news on the nightly build?

[GeekToo]

I did have a discussion about it on the #openttdcoop.devzone irc channel recently.

According the Rubidium, the process is pretty simple:
1 have a repo that can be checked out with the patch applied
2 have a place where the binaries can be stored
3 notify one of the openttd developers, to verify a safe compile can be made on the openttd compile farm (i.e. chance the patch compiles is pretty high, no security risks like coding a trojan into the patch present etc. , again according to Rubidium the chance that this patch would pass this test is pretty high, being around here for longer time, and downloaded and used by larger groups of users) and then request a run on the openttd compile farm.

2 can be arranged on the openttdcoop server, can be set up pretty easy when I request Ammler or Planetmaker.
3 would require 1, but then I foresee no problem there

That leaves step 1. I created a repo that was completely functional, but started it using an svn checkout. Although that works, it has the nasty side-effect that later hg updates, will always have to be merged for every file that got changed in trunk. Apart from that, it will show incorrect author names for files, which is not a funtional problem, but also not as clean as we would want it. Unfortunately, hg offers no means to repair that (at least not with the user rights I have on the openttdcoop server. So I decided to change the course a bit: I created a new repo, with the complete patch history in the patch queue. This has the advantage that every patch user/developer always can have the latest patch version and apply it to trunk with the correct revision, regardless of the cvs being used.

But for the compile farm, we also need a complete repo, and there we also had a discussion, a.o. with Ammler how to best arrange that: it seems kind of overkill for every patch on the openttdcoop server to have a complete repo, a combined repo that alternately applies one of the patches, and requests a compile run does seem more efficient, considering the fact that most patches only change a small amount of the trunk code base. But that discussion is not completely ended, it also would require some scripting. Another complicating factor is that I've been kind of limited in my spare time lately.

So, concluding, there is progress, but not as fast as we all would like.

[Jupix]

Interesting stuff, thanks for the update. Sorry I can't be of any help with all that. Though I'll go ahead and say a nightly -EZ build will be a great step towards EZ support in trunk.

[Lord Aro]

i know i'm not supposed to do this, but is there a new patch coming along soon?
(sorry)

[GeekToo]

Sure, twice on one night: improved recolour mechanism (reported by Maquinista, on airport sprites), and an update to latest trunk version. Patch file is ez.diff.

http://dev.openttdcoop.org/projects/32b ... es/ez.diff

[Wasila]

Glad to see progress in the patch! How hard would it be to make the patch support proper, full-on base sets?

[GeekToo]

A lot of work, and absolutely not worthwile imho. I mean, what would it add, that we cannot do now?

[Wasila]

The ability to have a real base set?

[GeekToo]

Yeah, but what would it add? I don't see any advantages for that. Don't get me wrong, I think having a full, high level quality 32bpp replacement set is a major goal, but that is something different than a base graphics set.

The end user wont notice the difference, maybe startup time / loading would be a little faster, and we would have to change quite a lot of tooling, grf loading ingame etc, to have the same end result.

[Wasila]

If it's not a base set, surely that would mean that a) it would not be available on BaNaNaS & 2) you would not be able to choose it and something else on the installer?

Also would you like to contribute to the discussion in the Base set thread?

[Ben_Robbins_]

Not my own words if this looks a little out of place: There is more chance of it getting into trunk if there is a 'real base set'..i.e. it works the same.

If that is an advantage or not depends if getting it to trunk is your current intention; if not, then how it currently is works well.

[Jupix]

It's always been our aim to distribute 32 bit gfx through BaNaNaS, and tars can't go on there. Nor would I ever want them there because of the way they currently work; you can't toggle them on/off in the so-called addons window.

Though robust, IMO the tar/png-sprite loading mechanism is hacky in nature. The way we "overwrite" the sprites loaded from GRFs is bad philosophy when we have an established file format for delivering sprites.

Correct me if I'm wrong but sprite-filled tars can never be standalone. There needs to be a base set and it's stupid if the player needs to 1) download OpenGFX to play in 32 bits, or 2) download a tar so that a GRF works.

I also struggle to see the devs looking at the whole of the 32bit enterprise in a positive light because of the nature of the sprite delivery method, which is that it's purpose built for the uses of 32 bit gfx, and sort of circumvents the real graphics delivery system built into the game.

What it would add is simplicity in the eyes of the end user, and standardisation/uniformity in the eyes of devs. Want new stuff? Get GRFs off BaNaNaS. Or, want to draw new stuff? Google GRFs. In contrast to how things work currently, I think it's an improvement.

[Lord Aro]

Sure, twice on one night: improved recolour mechanism (reported by Maquinista, on airport sprites), and an update to latest trunk version. Patch file is ez.diff.

http://dev.openttdcoop.org/projects/32b ... es/ez.diff

oh so you can extract the file from there
it would be nice if there was a download link, instead of just showing it (if you get what i mean)

[Thief^]

There is a download link, and it works fine. If it doesn't for you, then it's your browser's fault. You can still work around it by right-clicking on the link and choosing "save as" or similar.

[Lord Aro]

woops i was looking at this link:
http://mz.openttdcoop.org/hg/32bpp-ez-patches
someone should update the link on the wiki (unless its already been done) and maybe publish some binaries

[GeekToo]

Jupix, thanks for your comments, it did clarify quite a bit for me, especially the following: it is not about having a base set that the discussion is about, but the way 32bpp sprites get loaded. I completely agree with your statement that 32bpp graphics handling is pretty immature (in trunk, and thus also in ezl, because apart from the naming convention for extra zoom level sprites, I have not changed anything in the way the sprites get loaded from tars).

Openttd was developed as an 8bpp graphics game, and it still shows. Lots of examples for this: company colour masks use an 8bpp palette, 32bpp animation is not possible (there is an 32bit blitter, but it is based upon an 8bpp palette shuffle, and palettes are a concept limited to 8bpp, 32bpp palettes would be like millions of colours large). Also the way pngs get loaded, always having to replace an 8bpp sprite does show it is not a native 32bpp game, not to mention the way the sprites need to be numbered following an 8bpp (new)grf.

But, all these arguments also apply to newgrfs, not only to basesets. Only difference is that you activate newgrfs in the newgrf settings window, and base sets in the main game window.

So, if we distill a change in requirement from the fact that an 32bpp sprite always needs to replace an 8bpp pcx with an 32bpp png it would be something like:
*the game shall be able to load (new)grf with 32bpp pngs.

that already requires major changes:
-define a (new)grf file format that supports loading of 32bpp pngs
-implement the loading and parsing of this format in the game
-implement the new format in tools that encode the newgrfs, like grfcodec and nml

Especially the last point can be very hard, since it requires external parties to cooperate, and would require lots of communications with those parties to start with.

I'm not saying it is not a good idea, but it may be a project on its own (after all the problems are not limited to extra zoom, but also present in trunk).

Since we're brainstorming on extensions for the patch, here's a couple of other directions we may aim for:
-sprite animation. Problem here may be technically oriented: png loading and displaying is a well-established multiplatform algorithm. Available by means of the libpng library. But for animated pngs, competing standards do exist: mpng and apng, and I don't think a library does exist that works for all ottd-supported platforms (and if anyone knows about such a library, a pointer would be very much appreciated).
-extra zoom level gui. I've seen several requests on the forums by people with hi-res screens to have larger gui buttons
-implement improvements to correct graphics issues that become more obvious in extra zoom levels, like height of rails, embankment sprites drawing order, v-vally water sprites and several others.

All of these ideas are pretty large projects, and probably other people have more ideas. So it would be a good idea to have some kind of agreement where we should be going.

[Jupix]

Jupix, thanks for your comments, it did clarify quite a bit for me, especially the following: it is not about having a base set that the discussion is about, but the way 32bpp sprites get loaded.

Correct, in the case of this thread and the discussion sparked here, it is about sprite loading. The discussion is of course a spinoff from the various organisatory discussions where it was highlighted that an important milestone for the project would be having a 32bpp base set that behaves the same as 8bpp base sets. As that is not currently possible from a technical standpoint we need multitask a little to solve the underlying technical hurdles.

I completely agree with your statement that 32bpp graphics handling is pretty immature (in trunk, and thus also in ezl, because apart from the naming convention for extra zoom level sprites, I have not changed anything in the way the sprites get loaded from tars).

Openttd was developed as an 8bpp graphics game, and it still shows. Lots of examples for this: company colour masks use an 8bpp palette, 32bpp animation is not possible (there is an 32bit blitter, but it is based upon an 8bpp palette shuffle, and palettes are a concept limited to 8bpp, 32bpp palettes would be like millions of colours large). Also the way pngs get loaded, always having to replace an 8bpp sprite does show it is not a native 32bpp game, not to mention the way the sprites need to be numbered following an 8bpp (new)grf.

Thanks for detailing the current technological state of the game. There is clearly some need for improvements when it comes to handling gfx. Me and you should perhaps start working on a roadmap/todo style document for what needs to be changed/implemented at a code level to bring the game client properly to the 32 bit gfx era. If you would be interested in helping me write a spec, I could start soon(tm).

But, all these arguments also apply to newgrfs, not only to basesets. Only difference is that you activate newgrfs in the newgrf settings window, and base sets in the main game window.

Of course - if I've understood this correctly, base sets are packaged using the exact same underlying methods as NewGRFs. The difference is the extra functionality provided by NewGRFs actions, which is of course outside the scope of a hypothetical spriteloader / .grf parser revamp.

So, if we distill a change in requirement from the fact that an 32bpp sprite always needs to replace an 8bpp pcx with an 32bpp png it would be something like:
*the game shall be able to load (new)grf with 32bpp pngs.

that already requires major changes:
-define a (new)grf file format that supports loading of 32bpp pngs
-implement the loading and parsing of this format in the game
-implement the new format in tools that encode the newgrfs, like grfcodec and nml

Correct, and understood. Very logical and I was expecting those issues.

Especially the last point can be very hard, since it requires external parties to cooperate, and would require lots of communications with those parties to start with.

You're talking sense but I wouldn't get too pessimistic as the tools are, after all, developed for the most part by people in this very community. What would have to be done first is of course a spec for the supposed new "32bit NewGRF" format. In fact, that would be critical as the encoding tools for one are a necessary step before content can be created. So in this case, as is the case with the whole 32bit gfx project, it's the game that adapts to a changed ecosystem; not the other way around. Some content creation tools need to be developed before there's any content to adapt the game to.

I'm not saying it is not a good idea, but it may be a project on its own (after all the problems are not limited to extra zoom, but also present in trunk).

Definite agreement from me.

-extra zoom level gui. I've seen several requests on the forums by people with hi-res screens to have larger gui buttons

This point can perhaps be expanded to an actual vector-based GUI, but overall as you said these "improvements" go way beyond just improving the EZ patch and as such should constitute their own standalone projects. Of those, I think the modernised grf format would be priority #1 and the others of lesser priority.

[Lord Aro]

hmm
i have a couple of problems compiling the latest patch (r19894)

1. to patch the source you have to do

   Code: Select all

   patch -p1 -i <pathname>

   - not really a problem i know but slightly annoying none the less
2. when patching you get:

   Code: Select all

   patching file src/blitter/32bpp_optimized.cpp
   Reversed (or previously applied) patch detected!  Assume -R? [n] 
   Apply anyway? [n] 
   Skipping patch.
   11 out of 11 hunks ignored -- saving rejects to file src/blitter/32bpp_optimized.cpp.rej
   

   and

   Code: Select all

   patching file src/spriteloader/png.cpp
   Hunk #1 FAILED at 1.
   1 out of 6 hunks FAILED -- saving rejects to file src/spriteloader/png.cpp.rej
   

3. when configuring i get half way through:

   Code: Select all

   checking revision... no detection
   WARNING: there is no means to determine the version.
   WARNING: please use a subversion, mercurial, or git checkout of OpenTTD.
   WARNING: you can only join game servers that have been compiled without
   WARNING:   version detection.
   WARNING: there is a great chance you desync.
   WARNING: USE WITH CAUTION!

4. finally - the fail during make that stops the .exe from forming:

   Code: Select all

   [SRC] Linking openttd
   spritecache.o: In function `ReadSprite(SpriteCache*, unsigned int, SpriteType)':
   spritecache.cpp:(.text+0x6a2): undefined reference to `Blitter_32bppOptimized::RescaleSpriteHalfSize(SpriteLoader::Sprite const*, SpriteLoader::Sprite*, bool)'
   spritecache.cpp:(.text+0x732): undefined reference to `Blitter_32bppOptimized::RescaleSpriteHalfSize(SpriteLoader::Sprite const*, SpriteLoader::Sprite*, bool)'
   spritecache.cpp:(.text+0x8fa): undefined reference to `Blitter_32bppOptimized::RescaleSpriteDoubleSize(SpriteLoader::Sprite const*, SpriteLoader::Sprite*)'
   spritecache.cpp:(.text+0x928): undefined reference to `Blitter_32bppOptimized::FillRGBFromPalette(SpriteLoader::Sprite*)'
   collect2: ld returned 1 exit status
   make[1]: *** [openttd] Error 1
   make[1]: Leaving directory `/home/the_overlord/ottd/objs/release'
   make: *** [all] Error 2
   

i think some of them aren't actually errors but i hope this helps somewhere!

[Bus]

Hm I was about to try this patch today but I'm getting the same errors as Lord Aro. I tried to apply r38 of ez on r19894.

[GeekToo]

I tried to reproduce it, but could not at first. Then I tried an svn checkout, and then I have the same issue. I use mercurial normally as vcs.

With svn, it has a problem with the $id lines in the diff. You can fix it easily yourself by removing the $id chunks in the ez.diff. They are present in 32bpp-optimized.cpp and png.cpp sections of the diff.
Then the patch applies fine for me. I will fix it soon on the repo.

The configure problem that Aro had is not reproducable by me. Judging by the message, you copied the source dir without the .svn dir, so configure cannot detect the version, a real svn co, or copying including .svn dir may fix it.

The last error Aro had, is a result of the first.

EDIT: fixed on the repo now