[PATCH] cut trees randomly in area instead of radial

[Ammler]

Patch is here: http://www.tt-forums.net/viewtopic.php?p=745901#p745901
Modified Lumber Mill grf: http://www.tt-forums.net/viewtopic.php?p=746876#p746876

-------------------------------------------------------------------
Hello

Just released final version of a little NewGRF. It enabled the Lumber Mill of tropic also in temperate and arctic. But I do not like the "type" it uses to cut the trees. So I ask here for a patch, which changes that.

Something like picking a tree tile randomly from the whole area...

Greets and Thanks
Ammler

[B-b0y]

But isn't radial the way it's done in real life? Seems more logical to me.

[Ishka]

In the forest next to me trees are cut only from areas where enough trees are still around after cutting, so the wood can regenerate faster. Picking trees randomly would be a more realistic simulation for this forest.

[SirXavius]

Disagree.

Cutting radially away from the lumberyard is more beneficial to the company in so many ways, even if it's not the most environmentally effective. Plus, we rely on the lumberyard to cut out the nearby trees so we can set up a station without the "tree murder" counting against us by the towns (that's the important reason!)...

[PhilSophus]

I will only go fast over the realism part of the argument: Both what Ishka and SirXavius say happens. In the tropics rain-forest is completely cut, the ground then erodes and there will never be forest again, on the other hand, sustainable forestry in developed countries does only cut as much trees as to leave the forest intact, not only because of ecology, but because the company can not just switch to another area, but has to make their living from its piece of forest.

For the more important part, impact on the game: I think it would just look a lot better, if the wood just became sparser instead of vanishing from lumber mill outwards, so the primeval forests (as they look like now) become cultivated ones in populated areas. So, I think it is worthwhile to provide another tile search scheme and to allow to only cut only part of the trees on a tile.

That said, I've been thinking about how to do it, read a bit in the NewGRF Specification and the OpenTTD source code, and I think I'll give it a try. Hacking it together would actually be quite easy, but as I'm a perfectionist , I want to do it right, and IMHO this means clean NewGRF integration in this case. Before I start my work, I would like to discuss my idea how to do it here.

What is already there: For a NewGRF industry to cut trees a special flag has to be set (feature 1A, bit 1). Callback 3B can be used for the NewGRF to control, whether trees should be cut, when OpenTTD would like to (return 0: don't cut, everything else: cut).

My suggestion: Enhance the meaning of the callback 3B result to control how the right tile should be found and how the trees are cut:

• bit 0: Master switch 0: Don't cut trees (and don't search of course), 1: Cut trees
• bit 1-2: How many trees to cut minus 1 (i.e. 3 will cut all trees)
• bit 3-4: How many trees must be at a tile at least to do cutting minus 1
• bit 5-6: Search strategy (00: Default circular tile search, 01: Randomly-centered circular search (see below for explanation, 10: Unused, 11: Default)
• bit 7-14: Parameter(s) for search strategy:
  • For strategy 00: Length of circular search (00000000 and 11111111 mean default of 40)
  • For strategy 01:
    • bit 7-10: Radius for random selection of a search center divided by 4
    • bit 11-14: Length of circular search divided by 8
• bit 15: Always set for callback result

All bits 1-15 clear or all bits set will interpreted as default to give a limited backwards compatibility with previous callback behavior (I would expect old NewGRFs to return either 1 or all bits set to enable cutting, I know the assumption is a bit waggly).

Now, at last, let me explain my proposed Randomly-centered circular search strategy. Let R be the radius (bits[7..10]*4) above, L the circular search length (bits[11-14]*8) and (X0, Y0) the base coordinates of the lumber mill. N is the number of trees to cut (bits[1-2]+1):

1. A random tile (X,Y) is selected from the area [X0-R,X0+R] x [Y0-R, Y0+R]
2. From this tile (X, Y) a circular search with maximum length for an adequate tile is done (the criteria: trees of growth state >= 3, i.e. all trees fully grown, at least the given amount)
3. If number of trees on this tile <= N: clear tile, otherwise reduce number of trees on the tile by N

I think, if that is really a good(-looking) approach is hard to say, without just trying, But when the code is there, it is not so hard to try (i.e. implement) different strategies.

So, now I appreciate your comments and suggestions

Edit: Reworked Bit allocation and Randomly-centered circular search strategy a bit

[michael blunck]

See http://www.tt-forums.net/viewtopic.php?p=745618#p745618

regards
Michael

[XeryusTC]

Both approaches are realisticly correct actually. Cutting everything around from one certain point is the most common one AFAIK. This method has the benefit that you can use heavy machinery to get to the hard wood trees that you want, the downside of it is that you have to cut the entire forest away for a few trees in a 1hm² area. This is what happens in the rainforests and A LOT of forests here in europe and the US (mostly in forests that are planted for their wood though).
The other method is scouting for the hard wood trees you want to cut and only cut those (with the exception of other trees that are standing too close around it to become hazardous). The trees are usually removed from the forest using animals (mostly mules) and put on trucks outside of the forest or in an open area which is connected to a road. This has the benefit that you don't cut down all the young trees and don't get erosion in your forest, it also tends to keep the forest alive since you only cut a few trees of the hundreds if not thousands of trees in the forest.

[Alberth]

Nice proposal!

Your strategy will clear out the trees close to the lumber mill and gradually leave more trees as you move away from the mill (trees further away have less chance to be within your search+cutting area).

Production will drop considerably once the center R tiles have been cleared, I think.



A possible extension:To support ecologically sound lumber mills, you may want to add a minimum number of trees that must be present before allowing cutting.

[PhilSophus]

Nice proposal!

Thank you

Your strategy will clear out the trees close to the lumber mill and gradually leave more trees as you move away from the mill (trees further away have less chance to be within your search+cutting area).

Yes, that's what I hope.

Production will drop considerably once the center R tiles have been cleared, I think.

Maybe, I'm not so sure. On the other hand, if you don't clear the tile, but only remove one or two trees, they will last longer. I would think that for a certain radius you would get a reasonable equilibrium of growing trees and cutting trees. However, this is hard to tell in advance without trying.

A possible extension:To support ecologically sound lumber mills, you may want to add a minimum number of trees that must be present before allowing cutting.

I've thought about harder per-tile conditions, too (which is what you propose). On the other hand, circular search will take longer to find an appropriate tile then. If the performance impact was significant would be something, one had to determine by profiling then.


Edit: I've taken another look into the source code an there is something I find very illogical: No matter whether the special effect "cut trees" is set normal production is done first (i.e. production callback executed, produced cargo added to the waiting cargo), then the cutting of trees is done if appropriately (special flag set, callback allows to). If cutting trees succeeds another hardcoded 45 items are added to i->produced_cargo_waiting[0], i.e. the first cargo slot.

IMHO, it's inappropriate that custom production is always done no matter whether cutting trees succeeded and that the effect is hardcoded. In any case the production callback should be delayed until cutting has been tried. The easiest solution I can imagine is then to call the production callback only when cutting did succeed.
An even better solution would be to call the production callback always and make the result of cutting available to the production callback (I don't know if and how this is possible) and let it decide, what to do. However, I'm kind of lazy and I think the easy solution is good enough.

What do you think?


Edit2: Reworked suggestion a bit.

[Alberth]

A possible extension:To support ecologically sound lumber mills, you may want to add a minimum number of trees that must be present before allowing cutting.

I've thought about harder per-tile conditions, too (which is what you propose). On the other hand, circular search will take longer to find an appropriate tile then. If the performance impact was significant would be something, one had to determine by profiling then.

I thought about this last night, and concluded your reasoning (dutch) "snijdt geen hout" (literally "does not cut wood", meaning "is not a good argument"). Basically, adding such conditions is equivalent to reducing the amount of forest around the mill, ie a player founding a mill in the middle of a desert (or, knowing Joe the Plumber, filling the entire desert with them).
Such an action should not bring down the application. As a result, reducing the chance of finding a 'good' site should also be allowed without bad performance effects.

I also thought about reducing computational effort. A weak point in the circular search is the fact that you start anew each time, especially for strategy 00. Maybe you can start a search from the last position you stopped the previous time instead (and 'wrap around' if necessary).

Another (much more complex) option may be to cache search results. Imagine laying out a grid of points every n tiles, where you only store points within reach of a search start-position. At each point you keep track of the minimal radius needed to find a cutting position (the global grid has the advantage that 2 lumbermills close to each other can benefit both from the same information).
After deciding your start search position, you get the minimal radius from the nearest point, and use that to decide on the minimal radius (taking the difference between both positions into account). After the search you update the point information (again taking the position-difference into account).
The tricky part of this scheme is how to reset the cached information. Ideally, a tree would warn the points ("he, I am fully grown now, please cut me"). Another approach is to do a wrap-around when a search from the minimal radius to the end doesn't give any result, and do the search from 0 to the minimal radius. If that gives a cutting position, the cached information was outdated.
The latter approach however does not prevent heavy search costs when there are no trees nearby (thus defeating the caching).




Anyway, glad to see that you added the condition in the definition (bits 3 and 4). For completeness, you could add to it that you currently do not use these bits.



Then about the numbers:

R is between 0 and 4*15=60.

After re-reading, I am a bit confused about the circular search length L. What is 'length' here? Another radius? a number of tiles? something else?

If length is another radius, I am very scared of the total size. L is then between 0 and 15*8=120.
So total max area is 2*(60+120)=2*180=360 diameter. Strategy 00 is even more scary with 2*254=508 in diameter (I don't understand your 255=>40 mapping, but you probably have a good reason for it).
None the less, 2 of these babies at max range (1 with strategy 00) would kill all trees in my entire world (I normally play at 256x512).

If length is number of tiles searched, it is much better, max radius (Area=pi*r*r <=> r=sqrt(Area/pi)) would then be sqrt(120/3.14)=6.2 or sqrt(254/3.14)=8.9 for strategy 00 (default: sqrt(40/3.14)=3.6).
In this case, I think R is a bit unbalanced w.r.t. L.

Last but not least, maybe you want to introduce an offset somewhere, eg R=1 or L=1 does not seem very useful.

Edit: I've taken another look into the source code an there is something I find very illogical: No matter whether the special effect "cut trees" is set normal production is done first (i.e. production callback executed, produced cargo added to the waiting cargo), then the cutting of trees is done if appropriately (special flag set, callback allows to). If cutting trees succeeds another hardcoded 45 items are added to i->produced_cargo_waiting[0], i.e. the first cargo slot.

Ok, so I simply get more lumber if trees can be cut.

IMHO, it's inappropriate that custom production is always done no matter whether cutting trees succeeded and that the effect is hardcoded. In any case the production callback should be delayed until cutting has been tried.

The nice thing about the current implementation is that production doesn't drop to 0 due to lack of trees.

The easiest solution I can imagine is then to call the production callback only when cutting did succeed.
An even better solution would be to call the production callback always and make the result of cutting available to the production callback (I don't know if and how this is possible) and let it decide, what to do. However, I'm kind of lazy and I think the easy solution is good enough.

I agree with you that the industry should know about the cutting result.

I don't know whether it is allowed to simply not call an industry when cutting fails. At least it means you don't give it an option of deciding what to do in that situation (for example, you couldn't code a GRF file that duplicates current behaviour).


While thinking this over, I concluded that we may need another parameter, namely how often should we try cutting?

Your proposal rests on the idea that trees will grow fast enough to get some non-zero cutting ratio. For the game this is most likely some constant. Trees grow at some constant rate, and querying industry for production has a constant rate, thus our cutting speed is also constant.
As a result, for a sustainable production (in these environment-friendly times), there is some fixed minimal area that we need. Larger areas have no use (other than leaving more trees alive), and smaller area will ultimately kill the industry. On the other hand, if we can specify the rate of trying to cut, it is much easier to get different useful sizes.

[PhilSophus]

Okay, here is the first release of the advanced-tree-cutting patch. All bits are implemented as specified. However, this is work in progress and thus may still change a lot.

For industries that have the tree cutting bit set, production only happens (and production callback is only called) when cutting was successful. This is in accordance with the default lumber mill, as its production rate is 0 and it only produces the hardcoded 45 items when a tree is cut.

The hardcoded 45 items are produced if (and only if) no production callback is defined. Otherwise, only the production callback is called.
In both cases the defined production rates (which might of course be zero) are applied when trees were cut successfully.

[caching]

Caching would be real overkill (or as we say in German "shooting sparrows with cannons") for a minor feature like tree cutting.

Anyway, glad to see that you added the condition in the definition (bits 3 and 4). For completeness, you could add to it that you currently do not use these bits.

I intended to use them, and in fact I do. As I said, the result should rather be tested when implemented, as that is easier than estimating in advance in this case.

Then about the numbers:
[...]

It might be a bit confusing, because in the first version I wrote, I hadn't properly looked at CircularSearch. All areas that appear here are actually squares and not circles. L is the side length of the square to be searched (i.e. maximal +/- L/2 around the center).
R is half of a square side length (to be exact the square from which the center is selected has side length 2*R+1). As for allowing up to 254 for default strategy. I need 8 bits for the parameters of the other strategy anyway, so I don't scale nor limit the parameter in any way. I don't expect that such large values make sense, though. The maximum coverage area for the randomized scheme is a square of 181x181 tiles, so is in a similar dimension. Maybe cutting those in half and thus achieving a higher resolution may be appropriate. Still, while this total area is huge, it might be reasonable to set something like select center in a 121x121 area, but only search a 16x16 area around this spot.

Edit: I've taken another look into the source code an there is something I find very illogical: No matter whether the special effect "cut trees" is set normal production is done first (i.e. production callback executed, produced cargo added to the waiting cargo), then the cutting of trees is done if appropriately (special flag set, callback allows to). If cutting trees succeeds another hardcoded 45 items are added to i->produced_cargo_waiting[0], i.e. the first cargo slot.

Ok, so I simply get more lumber if trees can be cut.

No, production rate of lumber mill is zero, so it only produces, if it can cut.

IMHO, it's inappropriate that custom production is always done no matter whether cutting trees succeeded and that the effect is hardcoded. In any case the production callback should be delayed until cutting has been tried.

The nice thing about the current implementation is that production doesn't drop to 0 due to lack of trees.

See above.

The easiest solution I can imagine is then to call the production callback only when cutting did succeed.
An even better solution would be to call the production callback always and make the result of cutting available to the production callback (I don't know if and how this is possible) and let it decide, what to do. However, I'm kind of lazy and I think the easy solution is good enough.

I agree with you that the industry should know about the cutting result.

I don't know whether it is allowed to simply not call an industry when cutting fails. At least it means you don't give it an option of deciding what to do in that situation (for example, you couldn't code a GRF file that duplicates current behaviour).

While it would be nice to have, I dropped it as "too much work for to little gain".

While thinking this over, I concluded that we may need another parameter, namely how often should we try cutting?

The callback is (and always was) called each 256 ticks and can opt out at each round (so, it's actually at most every 256 ticks, but may be less often). Tree cutting industries without the callback cut each 512 ticks (also as it was before). I don't intend to change those values.

Your proposal rests on the idea that trees will grow fast enough to get some non-zero cutting ratio. For the game this is most likely some constant. Trees grow at some constant rate, and querying industry for production has a constant rate, thus our cutting speed is also constant.
As a result, for a sustainable production (in these environment-friendly times), there is some fixed minimal area that we need. Larger areas have no use (other than leaving more trees alive), and smaller area will ultimately kill the industry. On the other hand, if we can specify the rate of trying to cut, it is much easier to get different useful sizes.

Actually, my tests with a hacked version of Ammlers Lumber Mill GRF revealed that more trees vanish than expected as trees also die by themselves. Actually, being able to test, what makes sense, is also a reason to allow rather a too large area than a too small one at this point in time.


Edit: As the GetTreeCount() fix in FS#2423 went straight into trunk, a first little update, that just removes the trunkified part from the advanced-tree-cutting patch.

Edit: New patch version at http://www.tt-forums.net/viewtopic.php?p=749318#p749318

[Ammler]

just if others like to test the patch: here the modified grf

lumbermill_advanced.grf

modified by PhilSophus

(533 Bytes) Downloaded 282 times

I played already a little bit around but didn't find best settings, maybe it is also possible to give the industry production like the Forest and just cut trees for eye candy but then not that much?

Greets
Ammler

[PhilSophus]

just if others like to test the patch: here the modified grf

lumbermill_advanced.grf

Just a little notice on the usage: For playing around you have to set the fourth parameters to the value that is supposed to be returned by the cutting callback. The default if the parameter is not set is 0x1407, which results in the classical behavior.

I played already a little bit around but didn't find best settings, maybe it is also possible to give the industry production like the Forest and just cut trees for eye candy but then not that much?

Hmm, if the callback returns zero (or more precisely has bit 0 clear), no trees will be cut in this turn (256 ticks), so the last part is easily possible. As for the first part: So, you mean it should be possible to produce even if no trees are cut? Then we really need a possibility to tell the production callback whether (or even better how many) trees were cut. It seems that the lowest byte of var 0x18 (extra callback info 2) could be used for this as only the values 0 and 1 seem to be defined so far and it seems to be the natural place to pass such information.

I also had some problems with balancing, but then noticed that there might be one problematic issue when testing: I planted as much trees manually as possible (i.e. 4 trees on each tile). However, I have the impression that then all trees are synchronized (i.e. in the same growth stage) and thus also die by themselves at nearly the same time. I didn't have time again to test what happens with naturally grown trees, though.

As the growth stage is not affected when you cut only part of the trees, I think it also gives better results not to set a minimum number of trees that must be present, i.e. to allow to cut the tile to zero, when cutting single trees, because the trees will vanish when their time has come anyway.



Edit: Okay, I think I'll do it as follows:
First I reduce the reason value (i.e. why the production callback was called, so far 0=cargo arrived, 1=periodic processing) which occupies the complete lowest byte of var18h so far to fewer bits (maybe 4 bits to leave some room for more reasons and 4 bits for reason-specific data).

1. In the periodic processing (i.e. every 256 ticks) the production callback is always called with reason 1 and production values set as properties are also applied in this stage unconditionally as it was before my patch.
2. Then the special effects callback is called and cutting is tried, as the current version of my patch does it.
3. If (and only if) cutting trees succeeds, the production callback is called a second time with reason set to 2 and bits 6 and 7 of var18h are set to the number of trees cut minus one (i.e. 0 means 1 tree cut). If the production callback is not used the hardcoded 45 items will be produced as before.

Any comments?

[Ammler]

I did something wrong the first time and it might have used the default, now it woks well. I guess, the settings should be up to the grf author in the final version.

Greets
Ammler

[PhilSophus]

I did something wrong the first time and it might have used the default, now it woks well. I guess, the settings should be up to the grf author in the final version.

Yes, the direct pass-through of the parameter to the call-back was just meant as a testing version.

As for the stuff I wrote in my last edit. I think it is still useful anyway. And the most work is already done by thinking it through and understanding the related locations in the NewGRF spec and the OpenTTD source code. So, at this point, implementing it is quite a no-brainer.


Thinking much about trees in OpenTTD recently, I got the idea of a "tree line" or "tree limit" which is supposed to work similarly to the snow line in arctic. Above the tree line no trees will grow, in the height level below at most one tree per tile will grow, in the next at most two, and so on. I think that would allow for quite nice looking mountains. So, a patch doing this is also somewhere on my todo list.

[PhilSophus]

As promised a new version reworking the calling of the production callback.

For a description, I just quote myself as this is exactly what I implemented:

First I reduce the reason value (i.e. why the production callback was called, so far 0=cargo arrived, 1=periodic processing) which occupies the complete lowest byte of var18h so far to fewer bits (maybe 4 bits to leave some room for more reasons and 4 bits for reason-specific data).

1. In the periodic processing (i.e. every 256 ticks) the production callback is always called with reason 1 and production values set as properties are also applied in this stage unconditionally as it was before my patch.
2. Then the special effects callback is called and cutting is tried, as the current version of my patch does it.
3. If (and only if) cutting trees succeeds, the production callback is called a second time with reason set to 2 and bits 6 and 7 of var18h are set to the number of trees cut minus one (i.e. 0 means 1 tree cut). If the production callback is not used the hardcoded 45 items will be produced as before.

Still one question is left. Should the production callback with reason 2 be called when the periodic processing callback is enabled (prop.21, bit2) and the industry has tree cutting enabled (prop.1A, bit1). This is how it is implemented now. Or should a new bit be added to prop.22 explicitly enabling the extra call of the production callback.

So, now we need a NewGRF using all these features and after thorough testing with that, I'd submit the patch to flyspray.

[DaleStan]

So, now we need a NewGRF using all these features and after thorough testing with that, I'd submit the patch to flyspray.

You forgot the proof that this does not mess up any GRFs written to the old spec.

Without that proof, everything else is a moot point.

[PhilSophus]

So, now we need a NewGRF using all these features and after thorough testing with that, I'd submit the patch to flyspray.

You forgot the proof that this does not mess up any GRFs written to the old spec.

Without that proof, everything else is a moot point.

Software can never be proofed to be bug-free. It can only be proofed to have bugs

Anyway, if you happen to know a NewGRF that provides a NewIndustry with the tree cutting ability, I'm happy to test it. All other GRFs are unaffected.

I already stated that the interpretation of the callback result of the special effects callback isn't foolproof. Do you have a suggestion how to improve this? Is there a way how a NewGRF could state that it is fully aware of the new behavior?

As for the production callback. How should NewGRFs behave when they get a production callback with unknown reason value? If they are to ignore it everything will be fine. Otherwise, I already suggested that a new bit could be added to prop.22 to explicitly enable this callback.

[frosch]

I already stated that the interpretation of the callback result of the special effects callback isn't foolproof. Do you have a suggestion how to improve this? Is there a way how a NewGRF could state that it is fully aware of the new behavior?

Search for "grf version 8" and wait until it is implemented.

[PhilSophus]

I already stated that the interpretation of the callback result of the special effects callback isn't foolproof. Do you have a suggestion how to improve this? Is there a way how a NewGRF could state that it is fully aware of the new behavior?

Search for "grf version 8" and wait until it is implemented.

Hmm, I understand how a new GRF version would help, as the one can use the old behavior for version <8 and the new one for >=8. However, either advanced tree cutting would have to be part of version 8, or the spec. should at least be changed as follows:

Callback 3B (Special effects for industries) for effect 1:
Currently: zero: don't try cutting trees, non-zero: try cutting trees
Proposal: 0: don't try cutting trees, 1: try cutting trees in the traditional scheme, >1: reserved for advanced tree cutting
Grf-Version: Needs bump to version 8

Industry production callback:
Currently: Behavior for reason > 1 not specified
Proposal: NewGRFs should gracefully ignore production callback with unknown reason value
Grf-Version: Not sure if that isn't implied anyway

BTW: Your suggestions for version 8 didn't trigger much response Is there actually current effort towards it?