Re: 2.6 nightly/alpha discussion

[DaleStan]

... - For properly written vehicle grfs I am quite confident to raise the sprite limit to 64k sprites (including pseudo sprites, ...

I'm curious what you mean by "properly written".

Currently, CanRail has about 10,000 pseudo sprites. But, what pseudo sprites are counted ? Action-0, -1, -2. -3 naturally. What about Action-6, -7, -9, -D ? Then, what about, if I define all 116 vehicle descriptions in one action-4 or in 116 action-4s.

The reason for the current 15-bit limit is action 2 ID translation. After TTDPatch processes action 2s, each 2-byte ID in action 2 is either a callback result (with the high bit set) or the GRF-local sprite number of the referenced action 2.
For vehicle standard action 2s, it's a 14-bit index into spritespace or a callback result.

I think that everything would work just fine if all action 2s appeared before sprite 32768. This is unhelpful, as actions 1 and 2 constitute the vast majority of most sets.

... for Action0 for stations layout ... we would need a good way to address them ...

This works fine at the moment, they get extremely large, sometimes exceeding 64kB,

Seriously? I've been idly kicking around the idea of making GRFCodec support pseudo sprites up to 4 GB for years; do I need to pull that out of "idle thoughts" and move it into "things to do somewhat soon-like"?

[eis_os]

Properly written means, no action6 usage to alter the sprite action2.

Some general concepts:

Max Sprite Counts by Actions:
The action system more or less can only handle 65535 sprites, all action2 code work with word sizes ids.
Some actions callback result, bit usage for recoloring only allow 14bit, results in 16383 spriteids.


Different SpriteID Systems (all 16bits):
Each GRF have a local array of spritepointers (so for the GRF specific local spriteid system, every real and pseudo sprite counts)
TTD can only handle 16383 spriteids while drawing, means 14bits while caching works internal with full 16bit.

Feature based spriteids now uses the fact that while you can only have spriteids with 14bit at drawing stage the underlying spritecache and most ttdpatch action code will work with 16bit. So when storing a sprite into TTD spriteid system, we return a faked spriteid, that is used by action2 to return the spriteid for drawing. The difference now, we pass the current feature (House, Station, Rail) around. Simply before any code tries to read the spritedata from cache or any sprite dimension the code translates the feature based faked 14bit spriteid to a 16bit spriteid.(1)

Now you think why don't I get 16k sprites and only ~11k sprites?
Accessing grm and original sprites successfully

The code ensures that for all features the grm and the ttd original sprites are always at the same spriteid (low range), regardless of feature selection. As example drawing a house: it's not necessary to know the difference between a action1 sprite, or some ttd original or grm based sprite. The low spriteids are the same TTD or ActionA changed sprites and the upper range will be the feature based spriteid.

References:
1) SpriteID layout in TTDPatch

-edit-
And the new system:
Sprite System 2010

[OzTrans]

...

[DaleStan]

That action 2 doesn't parse. Did you skip the type byte?

Anyway, I'll bang up a spec and post it in the GRFCodec thread in a few days, I hope.

[OzTrans]

...

[eis_os]

Are these Action6 parameter depending or depend on other grfs? The former way work, the later won't in my draft, but it's good that you told me that, so I guess I will keep the action2 code and use some other tricks.

Michael, what usage of ActionA sprites with stations do you use? I know you use recolouring sprites and if I remember well ground sprites?

[OzTrans]

...

[eis_os]

I did want to reduce the overall complexity of action2 processing to do a one shot translating while loading the grf. hmm, I guess that should work regardless as long as the action2 ids aren't changed. I will try to use a safe method and simply change the way the old code works, so instead of faked sprite numbers we use action1 local sprite numbers internally.


About ActionA usage, generally I wish to change it a bit:
- special syntax to allow grf local sprites for certain features. This way recolouring sprites and ground sprites won't count in the general sprite "other" category
- find a backward compatible way to define them
- as soon as you tell TTDPatch it should use grf local sprites your ActionA will be in the grf local sprite range...

[michael blunck]

Michael, what usage of ActionA sprites with stations do you use? I know you use recolouring sprites and if I remember well ground sprites?

Yes, I´m using action0A together with GRM.

With regards to the sprite block size problem, I think NewStations was the first newgrf using sprite block sizes to their limit (hence why it was set to word size still by Josef ). Meanwhile, by a procedure somewhat like "convolution", sprite block sizes in the next release of NewStations have been significantly reduced, e.g. from 48,069 to only 6,489 bytes. Highest number is still 57,789, but that´s an old AI station handling sprite, something that I´ll have to change yet, especially because of OTTD´s very different handling of AI stuff.

So, no use for >64k at the moment from my side, but who knows about the future?

regards
Michae

[eis_os]

The question is, would you and others use GRF local Sprites for stations?

The largest usage is with station tiles. Why ? We cannot access action-1 based sprites from several action-1s for one single station tile (except for ground sprites and foundations). Using action-1s only, means loading the same sprites multiple times, because we have very slight variations when it comes to waiting cargo, or climate and terrain dependencies.

So you want a way to address several Action1 in an Action2?
This is theoretical possible after the rewrite, but:
a) How would you like to access them (Backward compatible Action2?)
b) Would it be good to be able to access a different set? (If yes, how should the syntax look?)
c) Do we really need the concept of sets when defining non-vehicle features?

I mean I think that several action1 station sprites and a houses may share sprites more often, so they need to be duplicated.

[OzTrans]

...

[eis_os]

So you want a way to address several Action1 in an Action2?

We can already do that .. but we don't know which sprite is currently being drawn.

Ehm, you can use the recent action1 with an action2, not any previous ones. TTDPatch doesn't know the sprite drawing either, that is the problem. Currently TTDPatch changes the spritelayout pointer and the base sprite. The real drawing is more or less done by original TTDPatch enhanced with buildonslopes, erails. Quite a mess actually.

Is it the shadow of a station platform roof, then it should come out of the action-1 for common sprites; i.e. they exist only once, regardless of terrain, snow, era. There would be only 1 set in this action-1.

Is it the platform halves, then it should come out of action-1 for platform sprites; i.e. here we have normal, snowy, generation-1 and generation-2 platforms, may be even desert, rainforest and more. This action-1 would have 4+ sets. We would determine the set to use in var act2's.


Is it the waiting passengers, then it should come out of a third action-1; i.e. an action-1 with 4 sets for waiting cargo levels of none, a few, many, a lot.

I am not quite sure if that is possible.

We would need a hint of the type of sprite being drawn; i.e. common, platform or cargo state. This, in my view, would have to be stored with the sprite data in property 09 for stations as an ID or type. You cannot just provide us with an iteration to the sprite being drawn, because the iteration (or sequence number) would differ between tile layouts for the same sprite and even differ as the sprite sorter decides the sequence of sprites to be drawn; that sequence is unknown to us.

a) How would you like to access them (Backward compatible Action2?)

Totally hypothetical …
. Store a byte size ID with every sprite we define in sprite data (beware the ground sprite may need one too) in property 09 for stations [or store the information in a new property, that is mapped to property 09].
. Provide that ID in byte1 of variable 10 when drawing the sprite (byte0 would be 0, or 1 for ground sprite, or 2 for foundations as is already the case).

That sounds doable, not sure if it's possible without rewriting the station drawing code...

Do we really need the concept of sets when defining non-vehicle features?

For grouping of same number of sprites for the same thing; i.e. a platform half as normal, snowy, 1st gen and 2nd gen keeps things tidy and together in one action-1.

For houses, I have (for a single house ID) 80 sets with one sprite each in one action-1. I believe this would result in 80 action-1s; because we can also have 2-, 3- and 4-sprite sets that do behave differently. Therefore, having just one set per action-1 would not be such a good idea. Industries and objects may be the same as they share the same syntax.

Thanks for your input

[michael blunck]

Is it the shadow of a station platform roof, then it should come out of the action-1 for common sprites; i.e. they exist only once, regardless of terrain, snow, era. There would be only 1 set in this action-1.

Is it the platform halves, then it should come out of action-1 for platform sprites; i.e. here we have normal, snowy, generation-1 and generation-2 platforms, may be even desert, rainforest and more. This action-1 would have 4+ sets. We would determine the set to use in var act2's.

Is it the waiting passengers, then it should come out of a third action-1; i.e. an action-1 with 4 sets for waiting cargo levels of none, a few, many, a lot.

I am not quite sure if that is possible.

Just put all the real sprites (needing to interact) in one large action1 block (possibly expanded, see Dalestan´s proposal) and use CB14.

regards
Michael

[DaleStan]

And again. I am not, repeat NOT, proposing changing anything that is in any way associated with the word "action". I'm changing things that exist two levels below that.

[OzTrans]

...

[eis_os]

GRF Local sprite range: You will get them via grm, a grf local spritenumber will be returned (Not necessary the one you think, neither would several grm requests for special ActionA return a contiguous sprite range, so the first may return sprite 120..130 the next 400..410)

A Action2 is bound to an Action1:
http://svn.ttdpatch.net/trac/browser/tr ... =2305#L799

While you can have severalaction2s pointing to different action1 in one Action3, you can't bind a single action2 to several action1s.

The difference for each action2 the last action1 would start again with zero, so we can support more then 11k sprites per GRF! On the sprite number site, the sprite number callback would return more times the same spritenumber. On the other hand, an Action3 could be bound to the first action1 and would make all local sprites relative to this spritenumber. => 11k max sprites per Action321 chain.

I guess the best way would be to some commented nfo dumps from your station grfs via email so I can see what you did and what you want to do

[OzTrans]

...

[eis_os]

GRF Local sprite range: You will get them via grm, a grf local spritenumber will be returned ...

Good …

Dealing with action-As : normally, I request a single 'find' (and if successful, a 'reserve') for all sprites stored in all action-As. If I can, I'll use only one action-A in a GRF (bridges being the exception). With bridges, the contiguous sprite range reserved is carved up, so that I can use action-7s to store same size blocks depending on player choices somewhere within the reserved space. There are 4 blocks, one each for normal rail, monorail and maglev, as well as one for road bridges. Sprite blocks (which are normal and narrow gauge rail, as well as 'brick' or 'bitumen' bridge decks) will then be stored in one of the blocks depending on parameters current at game start. This approach works very well and should serve me just as well with grf local sprite blocks.

That should work, as said, you should get actually a sprite number local from your grf or if old code is used an ActionA in TTD sprite number space.

The following is a sample with 2 action-1s for one vehicle ...

Code: Select all

Code Sample

That is a good sample how I deal with multiple action-1s for the same feature and I use this approach with all features.

Ok, that helps me a bit.

… the best way would be to some commented nfo dumps from your station grfs via email so I can see what you did and what you want to do ...

I only have a source (.SRC) and an undocumented output (.NFO). I'll see what I can do, give me a few days.

SRC? Anyway, I think I know roughly what you do, I will come back to your offer later if something in my code doesn't work

To sum it up. I understand what you are proposing as follows and it should not have a material impact on my already coded GRFs :
= I decide to use GRF local sprites.

Yes, the format how to support them with stations is still in flux as a new station drawing code may be necessary. For stations this could mean:
You can use Action1, Action A and TTD sprites, or you can use Action 1, X (GRF Local Sprites) and TTD sprites.

= I can use any number of Action-As; together with GRM to get an offset, which I add to the action-A sprite via an action-6.
= To address individual sprites in an action-A, I use the relative position (and adding the via GRM received offset to it).
= I use an action-A sprite in the same way, as I would use and address a TTD-sprite from trg1r.grf.

I will try to keep Action A much or less the same. Otherwise all GRFs would break

= Action-1s are used unchanged via sets, with the offset into each set provided by the patch and I need not know what the sprite number is (e.g. 0 to 7 for vehicles, or sprite number from the tile layout for stations). Actual sprite numbers are all translated and mapped the the GRF local sprite block and I need not be aware of what they are.

TTDPatch will resolve action2ids locally to Action1, so every Action1 virtually uses sprite number 1 instead of a feature based sprites number. This should be transparent to user. Only problem if you try to load a new big grf without my patches enabled: To much sprites will be used then.

Just one question, how is the GRF Author Helper feature affected by this ? Can we still use the sprite number in .NFO to access individual sprites stored in an action-1 block.

It should work better after it will be fixed, as Sprites won't be cached or inserted anymore. The TTDPatch windows version may introduce a reload GRF button as GRFs could be reloaded much easier. (This would need tweaking code so all currently relevant static information will be reloaded too)

I think we can all be very happy chaps, if we get 11k of sprites per action-123 chain in a GRF. CanStn won't use more than 5,000 across all action-As and -1s. CanRail, will be very happy with about 16k of them. If this is possible, the entire TTDPatch community will owe you a big thank you …

Finally, if we get 4GB pseudo sprite sizes, proposed by DaleStan, the sky is the limit ...

My current status:
+ New experimental feature switch, still not useable
+ Sprite caching code killed, TTD now uses always the loaded sprite.
+ Checked the grf spec what TTD really means with some bit flags
+ Started to write a new grf loader
+Marked some code for rewrite

Plans:

As soon as the no caching code works, I can test the base. If this works, move this code to be the default.
Then I can change Action2 to use grf local sprites. (maybe only certain features as start)
Remove the features from the feature based sprite system. Remove the code completely as soon as there aren't any feature based sprite ranges any more.