[Patch] Downsampling done properly for 32bpp blitters

Forum for technical discussions regarding development. If you have a general suggestion, problem or comment, please use one of the other forums.

Moderator: OpenTTD Developers

Post Reply
LeXa2
Engineer
Engineer
Posts: 87
Joined: 15 Jul 2012 03:54
Location: Moscow, Russian Federation

[Patch] Downsampling done properly for 32bpp blitters

Post by LeXa2 »

Last version: g55e1a3d
Last update: 2012/08/14
Target OTTD versions: separate patches for 1.2.x branch and for post r24111 trunk

Compiled binaries for Windows with this patch in coupled with some other fancy things could be found in the 32bpp-anim-aa blitter's thread.

Hello, folks, here is another report on my quest for the pixel-perfect OpenTTD zoom out :-).

As I had been mentioning in the 32bpp Sprite-Based Anti-Aliasing Blitter thread there's another approach to the problem of increasing perceived quality of the zoomed out rendering produced by the engine. In 32bpp-anim-aa blitter most of the work for smoothing unremapped pixels (unmasked 32bpp and some of masked 32bpp and 8bpp pixels) had been done in Blitter::Encode() - which is called once per sprite load - and rest pixels (remapped, including palette-animated ones) were processed at Draw() and PaletteAnimate() time.

It is possible to hook into a more upstream point - the moment when the engine creates smaller versions of the sprites for zoom levels it hadn't been supplied - and copy a part of the code from Encode() into there. I mean - it is possible to replace the currently used nearest neighbour like downsampling technique with something more advanced (and producing more visually pleasing results). That's exactly what this patch is about.

You could find patch in the attachments area with some illustrating screenshots (and more screenshots would follow in the second post of this thread).

Gory details could be found in the second message of this thread.

P.S. This patch is a "sister-project" to the 32bpp-anim-aa blitter. Best visual results could be achieved by using both of them coupled with native 32bpp baseset. For now I'd recommend to stick with hand-compiling 32bpp "megapack" into the 32bpp GRF using the instructions posted on these forums or - if messing with cygwin and/or linux is too much for you - simply using OpenGFX coupled with 32bpp NewGRF's from here (except for zBase). zBase in its current form is a bit flawed and using it would result in visible glitches (details could be found below). These glitches are not the alignment-related problems and they could not be fixed by messing with sprites alignment. Hopefully this flaw would be addressed in future or I would find a way to safely incorporate some "zBase auto-fix" into patch that won't hurt resulting quality for other unflawed basesets & NewGRFs.
Attachments
2012-08-14-openttd-downsampling-done-properly-g55e1a3d.tar.gz
(20.07 KiB) Downloaded 211 times
2012-08-14_OpenTTD-Blended_downsampling-32xZO-new_algo.png
(2.12 MiB) Downloaded 2 times
2012-08-14_OpenTTD-Blended_downsampling-32xZO-orig_algo.png
(2.12 MiB) Downloaded 2 times
Last edited by LeXa2 on 16 Aug 2012 22:41, edited 4 times in total.
LeXa2
Engineer
Engineer
Posts: 87
Joined: 15 Jul 2012 03:54
Location: Moscow, Russian Federation

Re: [Patch] Downsampling done properly for 32bpp blitters

Post by LeXa2 »

Well, now to the gory details.

Resizing down is always being done by 4x so the easiest thing to employ here is to use linear interpolated downsampling. For 4x downsizing case (each 4 pix quad translated into 1 target pixel) it boils down to a dumb simple mean value (in case we don't have an alpha channel to worry about) or to a - still simple - weighted average with alpha channel values acting as weights. Trouble is that this is world of CG and sprites are raster bitmaps so we have to deal with the aliasing and all other kinds of rounding errors & problems to get this right. Size of the tile sprite at 32x zoom out level is 8x4 so a cost of a rounding error here is extremely high. Existing sprites had been created keeping in mind the specifics of the original downsampling algo. Shapes of the land tiles had been formed in a way so downsized versions would opaquely cover the entire area they're expected to cover when laid out in grid by the engine. This property of opaquely covering the area is one of the extremely important to held as failing to do so would end up with random garbage/glitches appearing on the screen due to the way screen updating works.

Due to reason detailed above it is impossible to simply linear downsize sprites and hope that resulting picture would be correct - it won't. Special techniques must be employed to warrant that resized picture would opaquely cover exactly the same area that would had been covered if the sprite had been resized down using the original algorithm. This problem is not a new one - I had covered it in 32bpp-blitter-aa thread and proposed a possible solution that results in not-so-visually-bad results for tile sprites in case they only have fully transparent<=>fully opaque pixels transition on shape edges. This solution was given a name "FORCE_XXX_HACK" (initially it was "force opaque hack", but as later hack had been extended to either force pixel into opaque of into fully transparent - the name had been changed) and it had been proven to work pretty well for existing 8bpp basesets (original Dos, original Windows and OpenGFX) and older and not officially "existing" 32bpp megapack.

Enough intro talks, let me cover what's all the buzz about.

"Resizing done properly" patch:
  • would improve visual appearance of the zoomed-out viewports for any baseset and/or NewGRFs in use;
  • would work with any 32bpp blitter which is descendant of the Blitter_32bppBase - all existing 32bpp blitters are (including my 32bpp-anim-aa);
  • wouldn't improve visual appearance of areas covered by remapped and/or palette-animated pixels (water + 32bpp-anim blitter = looks bad, just the same as without the patch);
  • would improve visual appearance of the water (and rest palette-animated) areas in case non-palette-animated blitter is being used (like 32bpp-simple or 32bpp-optimized);
  • would work best for 32bpp basesets and GRFs which have as low amount of masked 32bpp sprites as possible;
  • would uncover flaws/bugs in existing basesets and GRFs and could made these glitches more noticeable than they were with original downsampling (it's easier to spot one wrongly coloured pixel on the smooth clean surface rather than on "the pixelated piece of garbage"-like surface that you'd get with nearest neighbour like resizing);
  • would hurt the performance a tiny bit but not nearly as much as 32bpp-anim-aa blitter would;
  • could serve as a complement to the 32bpp-anim-aa blitter allowing to use low level AA (2x) to get output quality level that's comparable to (or is even the same as with) higher levels of AA (4x-16x).
Thus the best effect you'd get if couple this blitter with the 32bpp baseset and "my 32bpp set of choice" is the self-cooked version of the famous 32bpp "megapack".
zBase might seem to be a good choice here but unfortunately that's not the case due to the fundamental flaw it has (as of r123) for all tile sprites with "shape edges" having some pixels semi-transparent instead of one-step jumps between alpha == 0 and 255. I've been reporting about this problem here but it isn't known so far if would it be ever addressed or not. It's not that hard to fix though and I had even manually done it (as a matter of experiment) for some part of the zBase r123 but the "Right Way (TM)" to address this issue is not a decode-edit-reencode sequence but rather a change in the render/build sequence of the zBase build system itself. On screenshots posted later "Semi-fixed zBase" reffers to the my modification of the zBase r123 (decode -> manually processe some sprites in GIMP -> re-encode back). "Unmodified zBase r123" stays for "zBase as it was downloaded from openttdcoop repo".

Due to zBase being flawed with producing "accumulating garbage at tile seams on redraw" glitches even with original blitters & downsizing it was desirable to try to "address" this issue by making viewport updates deterministic. So I've created yet another patch that forces the engine to fill each viewport area it's going to update with the neutral grey colour and only then proceed with sprites output. It helps a bit ("seams between tiles" glitch became always visible but it mostly fades into surrounding pixels for majority of temperate terrain tiles) but is not perfect and drains performance a bit more (negligible amount really in case you're on PC which is non older than 4 years from now). You could find this patch in the "downsampling done properly" tarball in the attachments section of the first message in the thread. I highly recommend using it for general gameplay.

It seems that now it just the right time for some fancy pics.
Here is 1x1 comparison of the old algo vs. new one (scaled up 2x so differences could be noticed more easily):

Image

What to look at:
  • Original algo is crap. Aliasing artefacts on water and fields are eyes-hurting. Ignoring that and comparing visual appearance between original and "semi-fixed" zBase some differences could be noticed proving that flaws in this baseset affect the original blitters & downsampling too but it's just pretty hard to notice glitches on a "grainy s*** picture" like that.
  • New algo with FORCE_XX_HACK active produces smooth and clean picture when coupled with "semi-fixed" zBase. With stock unmodified zBase there are some "visible seams between tiles" glitches. The result isn't perfect though as some sprites end up being not anti-aliased like field fences or power station smoke clouds.
  • Turning FORCE_XX_HACK off makes all sprites anti-aliased as expected (compare field fences and power station smoke clouds on the pictures above) but it "brights up" the entire viewport noticeably, especially for 32x zoom out. What actually happens is that 8x4 tile sprites end up being semi-transparent for the most part and neutral gray (#7f7f7f) viewport-filling colour starting to "shine through". It could be easily spotted looking at the "green glass" tiles looking like "4x2 green rects surrounded by washed-out green". IMO it looks really bad for 32x and somewhat bad for 16x level.
  • For 8x zoom out level it looks like linear downsampling without FORCE_XX_HACK produces a bit better result comparing to the new algo with FORCE_XX_HACK on. It's hard to spot the difference but it is right there: look at the edge of the factory wall to the right, it is "sharp" with FORCE_XX_HACK and "transparently blended" without it.
  • zBase-related "seams between tiles" could be seen for 8x zoom out also (for "unmodified zBase") but for this zoom level they are not so severe (because the cost of 1 pixel error for 32x16 sprite is minor comparing to same 1 pixel vs. 8x4 sprite).
In attachments you could find two more fullsize (~1.5-2MB each) comparison screenshots for old algo vs. new one for 8x zoom out mode.
You could find original fullsize screenshots here (as long as my server is up and functional): https://lexa2.ru/lx2/OpenTTD/2012-08-14/
Attachments
2012-08-14_OpenTTD-Blended_downsampling_compare.png
(359.64 KiB) Downloaded 2 times
2012-08-14_OpenTTD-Blended_downsampling-orig_algo.png
(1.86 MiB) Downloaded 2 times
2012-08-14_OpenTTD-Blended_downsampling-new_algo.png
(1.6 MiB) Downloaded 2 times
Last edited by LeXa2 on 14 Aug 2012 20:21, edited 7 times in total.
User avatar
Vaulter
Traffic Manager
Traffic Manager
Posts: 185
Joined: 21 Dec 2004 05:35
Skype: andrey-zaharov
Location: St. Petersburg, Russia
Contact:

Re: [Patch] Downsampling done properly for 32bpp blitters

Post by Vaulter »

:!:
User avatar
Digitalfox
Chief Executive
Chief Executive
Posts: 708
Joined: 28 Oct 2004 04:42
Location: Catch the Fox if you can... Almost 20 years and counting!

Re: [Patch] Downsampling done properly for 32bpp blitters

Post by Digitalfox »

I .... don't know what to say ... So much information to process in my head right now .... But wow :bow:
User avatar
Zephyris
Tycoon
Tycoon
Posts: 2890
Joined: 16 May 2007 16:59

Re: [Patch] Downsampling done properly for 32bpp blitters

Post by Zephyris »

This does look very good.

I guess one of the questions devs will have is about performance, how much slower is this?
LeXa2
Engineer
Engineer
Posts: 87
Joined: 15 Jul 2012 03:54
Location: Moscow, Russian Federation

Re: [Patch] Downsampling done properly for 32bpp blitters

Post by LeXa2 »

Zephyris wrote:This does look very good.

I guess one of the questions devs will have is about performance, how much slower is this?
Negligible amount. Well... almost :-).

I haven't got exact benchmark numbers on hand that been done for this one ATM, but for pretty similar proc PalletteAnimate() from 32bpp-anim-aa blitter worst case throughput for 4 anim slots case is ~31ms per 1920x1200 screen target when running single threaded on AMD FX 8120 CPU. For new downsampling speed should be approx the same order. The algo used is as simple as it could be and despite being obviously more costly compared to original one. Memory accesses to separate CPU cache lines with misses and CPU execution pipe flush on the incorrectly predicted branching are the things that bottleneck performance here. Expected speed dropdown depends on the source sprite and could be as much as 20x-10x times for the worst cases. That's a lot but who really cares if this is only done once per sprite load (assuming that you've got sufficiently big enough sprite cache size so sprites don't get kicked off from it)?

Let's assume the unrealistic worst case of all sprites in the zBase r193 baseset being loaded at once (namely 4792) and each sprite being 256x128 in size and engine should resize it down 5 times (it should really be 3 times as usually sprites are supplied for three higher zoom levels leaving up to engine to generate smaller ones). It gives us 4792 * ((256*128) + (128*64) + (64*32) + (32*16) + (16*8)) ~= 209Mpix to process. With processing speed of 1920*1200*4/31ms ~= 297Mpix/s processing time would be less than a second. Judge for yourself :-).
User avatar
Zephyris
Tycoon
Tycoon
Posts: 2890
Joined: 16 May 2007 16:59

Re: [Patch] Downsampling done properly for 32bpp blitters

Post by Zephyris »

It does sound good, though big patches are hard to get approved :s Good luck :)
LeXa2
Engineer
Engineer
Posts: 87
Joined: 15 Jul 2012 03:54
Location: Moscow, Russian Federation

Re: [Patch] Downsampling done properly for 32bpp blitters

Post by LeXa2 »

Zephyris wrote:... big patches are hard to get approved :s Good luck :)
Thanks :-).
This patch is mostly a pretty obvious one - it's short enough to read as a one piece and is not tough to understand.
But it's not something that could and should be accepted into trunk in its current form IMO because it could cause problems with GFRs that use non-standard recolour ranges. It is not hard to change so any recolour range would be handled correctly but that would mean that for majority of the existing GRFs - any with 8bpp sprites or 32bpp masked ones - this patch won't improve anything on the visuals side thus it'd end up being useless. And IMO there's no point in including patch in trunk that would be useless for majority of end-users as of today.

With the emerging popularity of the zBase (and if it would be fixed wrt to opaquely covering the entire tile area so "seams between tiles" problem wouldn't get worse due to linear interpolated downsampling) something like this patch might be viable to include into trunk but it's a matter of a not so distant future (several months from now). As for the patch this post is about - it is still pretty damn good wrt to visuals with 32bpp sprites so my position on it is: "it's good to have a patch like this available on hand so users - who want it - could benefit from using it". :-)
peter1138
OpenTTD Developer
OpenTTD Developer
Posts: 1732
Joined: 30 Mar 2005 09:43

Re: [Patch] Downsampling done properly for 32bpp blitters

Post by peter1138 »

Personally I prefer the current down-sampling; it's less blurry.
He's like, some kind of OpenTTD developer.
LeXa2
Engineer
Engineer
Posts: 87
Joined: 15 Jul 2012 03:54
Location: Moscow, Russian Federation

Re: [Patch] Downsampling done properly for 32bpp blitters

Post by LeXa2 »

petern wrote:Personally I prefer the current down-sampling; it's less blurry.
Out of curiosity: basing on pics upwards or on real gaming experience with some 32bpp tile sprites other than zBase?
Post Reply

Return to “OpenTTD Development”

Who is online

Users browsing this forum: No registered users and 38 guests