32bit base set development thread

[Jupix]

b) there's no need for a 32bpp base set, they may easily exist as add-on as they do now, replacing possibly every single sprite of a base set.

I absolutely loathe that idea and think that nothing implies that there is "no need" for a standalone 32 bit base set. In fact the need has concrete foundations detailed in this very thread. Not to mention the fact that having the 32 bit base set replacement represent a "second-class citizen" in comparison to its 8 bit counterparts frankly devalues our artists' hard work.

The rest of the points on that list I pretty much agree with.

A completely different issue is in this matter the zoom-levels. But it could follow a similar pattern

The way I thought about zoom levels, and recall going through it with Ben a while back, is that the GRF format would support adding alternative sprites for game items, both in the bit depth sense and the zoom sense. So like you said, the changes would be rather similar.

My experience is that "the 32bpp project" wants to push a lot of changes that would make OpenTTD not like Transport Tycoon, which basically goes directly against what OpenTTD stands for; it being a clone with improvements. Good examples of this are the odd scaled vehicles and angular corners, both which "had to be changed" for the "cause" of 32 bits graphics.

That made, at least for me, quite a strong case that you weren't interested in 32 bits graphics, but in pushing your own agenda w.r.t. changing the gameplay instead of just replacing the graphics.

I don't see how getting rid of chopped vehicles and angular corners is a gameplay change. It's a visual improvement, which the project is all about. You don't want it? Fine. But a lot of people disagree with you.

Even if you consider them gameplay changes, and refuse to accept anything else; there are already plenty of gameplay changes in OpenTTD, most of them toggleable. What's wrong in implementing more?

Anyway, we are not even discussing those features at this time. They are separate issues that we're not trying to push with these changes.

In 2007 we added support for 32 bits graphics in OpenTTD and seemingly no-one could be bothered.

Quite apart from the fact that "no one" is a massive overstatement, there are multiple reasons why the standard zoom 32 bit movement didn't generate a lot of buzz:

• Standard zoom is incredibly underwhelming once you've had a taste of extra zoom.
• As a result of the above, most of the content available in 2007 were rubbish resized EZ sprites that looked blurry and glitchy in standard zoom.
• Outside the above, the content that was available in 2007 was lacking and poorly standardised in comparison to what we have in 2010.

By improving the standards that graphics development adheres to, we are in the process of making even standard zoom 32 bit gfx more valuable. This includes sharpness, LOD and such. So to say no one cares about 32 bit standard zoom is just plain wrong. What's right is that we care more about extra zoom and that's why we want it in trunk.

Also looking at the patch itself made me almost cry; it's a coding style mess and I spotted cases where trunk fixes seem to have been reverted. That leads me to conclude that no one has bothered with a serious review of the(ir) code and as such not considered ready for even considering inclusion into trunk.

This is a valuable piece of text. Something that has to be posted in the patch thread so that it's more likely to be addressed.

Of course, even more valuable would be a point-by-point list of what's actually wrong with the code, so we can, you know, fix it.

Things like getting bananas support for the 32bpp graphics is something completely different, however given that the 32 bpp nightly packs are currently around 30 MiB and cover roughly 500 sprites, and a quick count of resizable sprites, i.e. landscape/vehicle/building sprites, exceeds 5000 that would mean a pack of at least 300 MiB. [ ... ] This would mean uploading 300 MiB to bananas via a web form, which probably might not be such a good idea. [ ... ]
I know for a fact that the current upload backend for bananas will barf at anything where the compressed/uncompressed size is more than some 60 MiB, so that would need to be reworked.

Is there a reason not to use the obvious solution, which is to bypass the webapp? Use something like scp to insert the files and a remotely callable PHP script to update the database.

It's going to be a while before we'll have a 300 MiB graphics set available for download. Until then the package sizes are going to be somewhat manageable.

The mirrors will definitely help with solving the "download" problem, however with 300 MiB per release doing frequent releases would fill our server's harddrives quite quickly.

You don't need to host old versions, just the latest one.

But then, that only makes sense when there is something to actually put there that actually works.

Define "works".

If it's tars you want to work, then there's lots of stuff available right now that works. And you don't need the EZ patch for it to work. You only get more zoom levels if you have it.

If it's 32 bit GRFs you want to work, then yeah, we need to sort out the format before we start making changes to BaNaNaS to accomodate it.

The base set contains, like said before, some recolour sprites which might not be that important... however, it also contains a lot of characters, i.e. the font. Most of these characters can be fetched from a norml font but a number of them cannot. Examples of this are the train, truck, bus, ship and aircraft icons in the station name. Ofcourse you can add a load of special casing here, but it would be (arguably) better if you were to supply them in the same way as is currently done; that would mean less duplication of code and therefor less chances of anything going wrong.

I don't see a reason why we can't duplicate the functionality of the 8 bit base sets when it comes to that.

[ ... ] no need to create a separate NewGRF format for 32bpp graphics, [ ... ]

We're not trying to create a separate format, we're trying to push improvements to the old one. As for how that's technically done, you seem to have a lot of ideas that I don't have the expertise to evaluate.

We *could* even consider bananas splitting the uploaded 32bpp package into an 8bpp package and 32bpp package where the 32bpp package depends on the 8bpp package and based on your (current) settings it'll download only the (base) 8bpp or both versions.

If you're talking about NewGRFs, then that sounds like a nice, useful feature.
If, OTOH, you're talking about a base set... well, I've covered that.

Deciding the behaviour of using 32bpp graphics vs 8bpp graphics on the blitter in use might not be such a good idea though. [ ... ]

IMHO it's a "must have" type of feature. Whatever problems there are preventing its implementation need to be fixed, not just going "meh, too bothersome".

[Ben_Robbins_]

Rubidium: 'No-one could be bothered'? I'm unclear on how this would ever be possible to disprove. The 32-bpp work is made along side the 32-bpp FZ work. Therefore, to be 'bothered' it seems would require dropping the calling for FZ and just make 32-bpp work. Not being bothered, and not compromising (if that's even what this is about), are completely different! The visual step from 8bpp to 32bpp is tiny compared to the step to 32bpp FZ. Especially considering the very nice use of the 8bpp pallet by the OpenGFX artists.

Let's also not intermingle visually apparent coding alterations, and game-play alterations. I certainly am an adamant supporter of keeping the TT-game play feel. In fact, I'm also pro the look of TT. It's like saying, correcting spelling mistakes, makes a new book.

As for the FZ coding not meeting OTTD requirements. Thank you for taking the time to look at this. However, this is news to me; and I don't think it should be. Alterations are not going to made with no calling for them to be made, and your views on the code are/were important.

[Rubidium]

Deciding the behaviour of using 32bpp graphics vs 8bpp graphics on the blitter in use might not be such a good idea though. [ ... ]

IMHO it's a "must have" type of feature. Whatever problems there are preventing its implementation need to be fixed, not just going "meh, too bothersome".

Sorry, but there is hardly anything *we* can do about an operating system's incapability of drawing 8 bits graphics. We have to provide 32 bits graphics to the OS, so we have to use something that blits 32 bits graphics. That kinda implies you need a 32bpp graphics blitting blitter, right? You want 32bpp to be toggleable, right? If you're forced to the 32bpp blitter by your operating system, deciding to go for 32bpp graphics because of the blitter would mean it is not toggleable on that operating system. That would violate your wish of having it toggleable.

[Jupix]

I think you're confusing having graphics releases toggleable with having the whole of the 32 bit "system" toggleable.

[Kogut]

<offtopic>
What does blitter mean?
According to wikipedia it is part of hardware produced in years 1980-1990.
I am a bit confused.
</offtopic>

[Zephyris]

<offtopic>
What does blitter mean?
According to wikipedia it is part of hardware produced in years 1980-1990.
I am a bit confused.
</offtopic>

It is the code in OpenTTD which does the same thing as that hardware does, ie. it is the code involved in taking graphics data and actually assembling it into something to display on the screen.

[planetmaker]

I'll try to summarize what was said on IRC:

We *could* even consider bananas splitting the uploaded 32bpp package into an 8bpp package and 32bpp package where the 32bpp package depends on the 8bpp package and based on your (current) settings it'll download only the (base) 8bpp or both versions.

Given today's IRC discussion this seems like the preferred solution than over changing the NewGRF specs; it implies that basically no change on the OpenTTD side is needed as the 32bpp sprites can be shipped along with any newgrf or baseset inside the same tar. The "only" real problem here is to teach OpenTTD that those two files (8bpp only tar and 8bpp+32bpp tar) are the same thing and when to use which.

On the BaNaNas side it might need work that it accepts also tars (or zip, gzip,...) which contain folders with appropriately named 32bpp pngs.

Deciding the behaviour of using 32bpp graphics vs 8bpp graphics on the blitter in use might not be such a good idea though.

Switching blitters is IMHO one thing, switching the use of 8bpp vs. 32bpp sprites might be another. I understand you that switching the blitter might be considerably more difficult than just allowing a switch which tells OpenTTD to use the 32bpp sprites (if present) or to not use them even if a blitter is active which would support them. The latter might make more sense or at least pose much less of an implementation problem than my initially proposed "blitter switch".

b) there's no need for a 32bpp base set, they may easily exist as add-on as they do now, replacing possibly every single sprite of a base set.

I absolutely loathe that idea and think that nothing implies that there is "no need" for a standalone 32 bit base set. In fact the need has concrete foundations detailed in this very thread. Not to mention the fact that having the 32 bit base set replacement represent a "second-class citizen" in comparison to its 8 bit counterparts frankly devalues our artists' hard work.

Much hate I sense in you Here you got my words and intention completely wrong. I do not nor intend to de-value any work, least the 32bpp sprite work on OpenTTD; it's great work being done. But please understand that not everything within the OpenTTD framework should be re-modeled only for the sake of "32bpp only" (why would that be desirable at all?):
That'd be disadvantagous for several reasons, the least not being "currently everything (newgrf, base set, AI) works everywhere" and "much change without gain" as 32bpp already works as it is. Also: a base set is per definition minimalistic, the minimum to make OpenTTD work irrespective of hardware and other pre-conditions. Everything else is 'extra'. If it helps to give peace to the mind, one can of course call a combined 8bpp + 32bpp base set '32bpp base set'; but that's semantics.

Further, it'd help IMHO a lot, if the 32bpp discussion could be separated from the extra-zoom discussion. Mixing both doesn't help either thing, it's two completely different features. Even if you desire both for the same reasons; and I don't think that 32bpp without extra-zoom is worthless.

[Jupix]

Given today's IRC discussion this seems like the preferred solution than over changing the NewGRF specs; it implies that basically no change on the OpenTTD side is needed as the 32bpp sprites can be shipped along with any newgrf or baseset inside the same tar. The "only" real problem here is to teach OpenTTD that those two files (8bpp only tar and 8bpp+32bpp tar) are the same thing and when to use which.

On the BaNaNas side it might need work that it accepts also tars (or zip, gzip,...) which contain folders with appropriately named 32bpp pngs.

As I said on IRC, the "changing the NewGRF specs" part is just a solution to a problem, that can be replaced with another solution, so long as the problem is solved either as well or better.

Switching blitters is IMHO one thing, switching the use of 8bpp vs. 32bpp sprites might be another. I understand you that switching the blitter might be considerably more difficult than just allowing a switch which tells OpenTTD to use the 32bpp sprites (if present) or to not use them even if a blitter is active which would support them. The latter might make more sense or at least pose much less of an implementation problem than my initially proposed "blitter switch".

The problem here is that the player needs to manually edit a .cfg to get 32 bit gfx working. You're again free to come up with whatever solution you like. If it was up to me, I'd somehow make it work outta box.

Much hate I sense in you

Don't, I may use stern words but the intention is one of determination, not hate or hostility.

Here you got my words and intention completely wrong. I do not nor intend to de-value any work, least the 32bpp sprite work on OpenTTD; it's great work being done. But please understand that not everything within the OpenTTD framework should be re-modeled only for the sake of "32bpp only"

Depends on your definition of "32bpp only". If you mean adapting the framework so that 32 bit only -type GRFs can become the norm, then I disagree. If you mean adapting the game so that 8 bit GRFs can't be used, then you've misunderstood what I'm hoping for here.

That'd be disadvantagous for several reasons, the least not being "currently everything (newgrf, base set, AI) works everywhere" and "much change without gain" as 32bpp already works as it is. Also: a base set is per definition minimalistic, the minimum to make OpenTTD work irrespective of hardware and other pre-conditions. Everything else is 'extra'. If it helps to give peace to the mind, one can of course call a combined 8bpp + 32bpp base set '32bpp base set'; but that's semantics.

Our views on that just differ, I guess.

When you have to ship the 8 bit base set to make a 32 "base set" (replacement) work, what you're doing is shipping useless data, IMHO.

I also think you're (not just you personally) blowing the "minimalist" hardware/bit depth compatibility issue way out of proportion. If the client can display 32 bits, then there's no problem. If not, the client won't be downloading 32 bit gfx anyway, so there's no problem. And if the client only has the 32 bit base set, but disables the 32 bit blitter, then... well, like I said on IRC, the outcome depends on how stupid you make the blitter enabling/disabling logic.

Honestly, I can't see the problem - any arguments I've seen against this have been comparatively minor issues that are easily solved with some pseudo-code logic style thinking inside my head. If you as a knowledgeable programmer know it can't be done, then fine & I'll accept it, but so far what I've seen pretty much translates to the people responding not being willing to think it over properly (thinking about solving the problem, not just countering my arguments), or be the one to write the actual code.

Further, it'd help IMHO a lot, if the 32bpp discussion could be separated from the extra-zoom discussion. Mixing both doesn't help either thing, it's two completely different features. Even if you desire both for the same reasons; and I don't think that 32bpp without extra-zoom is worthless.

If you haven't already, I suggest you read the project spec draft's section 5.1 which discusses this issue.

[maquinista]

When you have to ship the 8 bit base set to make a 32 "base set" (replacement) work, what you're doing is shipping useless data, IMHO.

You can fill the GRF file with 1×1 blue sprites.

[GeekToo]

Come on Rubidium, I don't know who or which part of the 32bpp community managed to get your p*** lukewarm (nl:lauw), but I think you're exaggerating tremendously.

Then there are the many changes in how to handle the grahpics files over time; especially the v9 then v10 and then back to v9 thing comes to mind, although there are undoubtedly more cases.

FYI, the way the patch loads the tars has always been the same, and always compatible with trunk tar loading, with extra rules for the zoomed in sprites. There has been exactly one version (v10), where the trains were shown as long vehicles, and that was marked as experimental, and I reverted it because of lack of interest, and because of glitches and code design issues. So you may want to support your statements "many cases" and "undoubtedly more cases" with some facts, instead of making suggestive remarks.

The last time that I know of that tar-loading was really broken, was with the implementation of 32bpp in trunk, that made tars created for the "official" 32bpp branch unusable.

Also looking at the patch itself made me almost cry; it's a coding style mess and I spotted cases where trunk fixes seem to have been reverted. That leads me to conclude that no one has bothered with a serious review of the(ir) code and as such not considered ready for even considering inclusion into trunk.

I'm disappointed that you didn't use any real arguments why trunk inclusion can't be considered, and exaggerating again:
I did take a look at what makes you (almost) cry, and what made you call it a coding style mess. All I could find was some whitespace issues, which of course need need to be fixed, but it makes me sad because I have a high opinion of you as OTTD-dev, so I know you know they can be fixed with one or two hours work (in fact, I already started fixing it).
For the non-programmers among us, we're talking about things like:

Code: Select all

if (gagaga) {
if (gagaga){       

The second line is incorrect style (whitespace at the end, and not before the bracket), but wouldnt make me cry (btw you know that you do have a lot of codestandard offending lines in trunk? You always seem to use C-style comments after #endif, which may be for legitimate reasons, but against your standards).
The other thing you mention, reverting trunk fixes, is more serious, but I don't think many of those exist.

If I were an OTTD-dev, I wouldn't include the patch either, but for the following reasons:
-only support for 32bpp-optimized blitter.
-no solution for 32bpp animations
-several bugs, like minimap clicks being incorrect
-too complex/ too large

Sorry if I sounded a bit too harsh, I still do regard you as a gifted programmer, that has done a lot for the OTTD community, but I think this was not one of your best forum posts.

[Rubidium]

Then there are the many changes in how to handle the grahpics files over time; especially the v9 then v10 and then back to v9 thing comes to mind, although there are undoubtedly more cases.

FYI, the way the patch loads the tars has always been the same, and always compatible with trunk tar loading, with extra rules for the zoomed in sprites. There has been exactly one version (v10), where the trains were shown as long vehicles, and that was marked as experimental, and I reverted it because of lack of interest, and because of glitches and code design issues. So you may want to support your statements "many cases" and "undoubtedly more cases" with some facts, instead of making suggestive remarks.

The versions (as per hg repository) v1->v2->...->v9-v10(b-g)->v9->v11->v9->v12->...
New company colour handling (in my opinion this is handling of grahpics files) in v11, reverted to the v9 version, new version in v12, "improvement" in v13-ish (not sure whether that changes the behaviour again).
So as far as I can see the way the colours are remapped has changed a number of times; don't know how much they change the actual behaviour, but I've seen many occasions where a fairly simple change in the specifications "warranted" ditching all already created graphics.

In my opinion switching back and forth in version numbers isn't such a great way to go, especially if you do trunk syncs in between because you're either going to remove manually remove the vN changes from v9 or you're going to resync a few hundred commits accidentally forgetting some. More on this later.

Also looking at the patch itself made me almost cry; it's a coding style mess and I spotted cases where trunk fixes seem to have been reverted. That leads me to conclude that no one has bothered with a serious review of the(ir) code and as such not considered ready for even considering inclusion into trunk.

I'm disappointed that you didn't use any real arguments why trunk inclusion can't be considered, and exaggerating again:
I did take a look at what makes you (almost) cry, and what made you call it a coding style mess. All I could find was some whitespace issues, which of course need need to be fixed, but it makes me sad because I have a high opinion of you as OTTD-dev, so I know you know they can be fixed with one or two hours work (in fact, I already started fixing it).

The problem with finding many coding style issues and reverted commits is that it gives me the feeling that you haven't critically looked at the code. And yes, a missing space between if and the ( isn't such a big deal... however when newlines are added/removed at totally unrelated places (e.g. window.cpp) it makes me wonder what caused those changes. Which in turn makes me very suspicious of the other code changes in that file. Also, what's the point in removing perfectly fine comments (e.g. spritecache.h)

Furthermore I have a habit to start at the bottom of a diff because that is generally the least reviewed part of the diff. In this case in the first (or rather the last) 10 chunks I saw 2 genuine changes, 2 changes I have my doubts about / which I seriously wonder, 5 trunk reverts and 1 totally pointless whitespace change. The next 10 chunks another 3 totally pointless whitespace changes, 1 changed magic number (if you change it you must have a clue what it means, so what does it mean?), 1 chunk that violates the "declare upon first usage" style. The rest is all "magic" viewport stuff which I don't pretend to understand.
How would you react if the first two screens full of diff are full of trunk reverts and pointless whitespace changes? I would feel quite damned sad about the quality of the patch and maybe crying is a bad/too strong word for it.

You know that you do have a lot of codestandard offending lines in trunk? You always seem to use C-style comments after #endif, which may be for legitimate reasons, but against your standards.

Yes, we are aware that some of the old code isn't fully complying with the coding style, but whenever it is rewritten it will be. As for the C-style comments after #endif: that not being in the coding style document is due to someone (accidentally?) removing it in 2007 and no one noticing it since.

The other thing you mention, reverting trunk fixes, is more serious, but I don't think many of those exist.

r11597: change of allocation handling
r19558: fix FS#3730
r19563: fix FS#3733
r19587: desync testing related

And that's just from what I recognized fairly quickly without doing extensive diggings of the source code for each of the changed lines. Possibly these reverts aren't that bad as the ones that are/were in the copy-paste patch, but still the patch has broken MP/desync testing.


Final note: the reasons for me not working on the whole 32bpp thing is because I don't care for it/don't like it, which is my personal reason for not committing it/working on it. This is ofcourse, again, a very subjective reason. That doesn't mean other OpenTTD developers may not work on it.

[GeekToo]

I was tempted to start a discussion about which remarks I disagree with, but decided that I will probably do both of us a favour by just starting to implement the ones I agree with, and ignore the rest (like the spritecache.h remark):
http://dev.openttdcoop.org/projects/32b ... s/activity

Apart from that, of course you're free to work on what you want, I never pushed people to use extra zoom, or pushed to get the patch into trunk.