Patch: The Rainfall River Generator

Forum for technical discussions regarding development. If you have a general suggestion, problem or comment, please use one of the other forums.

Moderator: OpenTTD Developers

Wahazar
Tycoon
Tycoon
Posts: 1451
Joined: 18 Jan 2014 18:10

Re: Patch: The Rainfall River Generator

Post by Wahazar »

I got following error:

Code: Select all

rivers_rainfall.h:1239:169: error: ‘TileWithValue’ was not declared in this scope
  void PrepareRiversAndLakes(std::vector<TileWithHeightAndFlow> &water_tiles, int *water_flow, byte *water_info, DefineLakesIterator *define_lakes_iterator, std::vector<TileWithValue> &extra_river_tiles);
(1.5.2 was patched, one landscape.util.h patch was rejected)
ic111
Director
Director
Posts: 608
Joined: 17 Jul 2007 17:56

Re: Patch: The Rainfall River Generator

Post by ic111 »

TileWithValue is added in patch 040, which on my system applies even without any warning (about being shifted etc.).

Thus, could you provide the rej-file (i.e. landscape_util.h.rej) or at least the information which patch was partly rejected?

That you get a compile error after a reject is not really surprising...
Wahazar
Tycoon
Tycoon
Posts: 1451
Joined: 18 Jan 2014 18:10

Re: Patch: The Rainfall River Generator

Post by Wahazar »

Sorry, I forgot to attach rej file, it was late night...

Code: Select all

--- src/landscape_util.h
+++ src/landscape_util.h
@@ -141,6 +141,17 @@
     }
 };
 
+struct HeightInterval {
+	bool initialized;
+	int min_height;
+	bool min_raised;
+	int max_height;
+	bool max_raised;
+
+	HeightInterval() { this->initialized = false; }
+};
+
+
 Direction GetOppositeDirection(Direction direction_index);
 
 inline Direction GetPrevDirection(Direction direction_index) { return direction_index == DIR_N ? DIR_NW : (Direction)(direction_index - 1); }
Formerly known as: McZapkie
Projects: Reproducible Map Generation patch, NewGRFs: Manpower industries, PolTrams, Polroad, 600mm narrow gauge, wired, ECS industry extension, V4 CEE train set, HotHut.
Another favorite games: freeciv longturn, OHOL/2HOL.
ic111
Director
Director
Posts: 608
Joined: 17 Jul 2007 17:56

Re: Patch: The Rainfall River Generator

Post by ic111 »

Hm, this reject will be gone in this release, since I (independent of your post) realized that that struct has become dead code in the meantime.

But this is no explanation why the compiler complained about TileWithValue. That one is added before, maybe there was another reject before, whose rej file was overwritten by the latter one?

I know that I manually edited some of the patch file, because of a change I could do by find&replace throughout the queue.
Attachments
rivers_v58.zip
(265.34 KiB) Downloaded 124 times
ic111
Director
Director
Posts: 608
Joined: 17 Jul 2007 17:56

Re: Patch: The Rainfall River Generator

Post by ic111 »

Finished.

As a last algorithmic change, I added more randomness to the way wider valleys are generated.

Some background about how this is done:
- Wider valleys are set up by (many times for different start tiles) lowering tiles on a straight line, that points more or less away from the river.
- The farer away from the river, the higher the probability is that I don´t lower tiles to the river height, but to a somewhat higher heightlevel.
- The length of that line, and how fast it ascends are chosen in a random manner.

So far, randomness was only introduced on a per-tile basis. I.e. on a large scale, all valleys of about the same river size looked similar. What I now introduce is that larger sections of the valleys are calculated narrow or wide. Compare the two screenshots below for what this means. The valley slopes both in the heightmap look quite steep, but one of them is generated in a narrow manner, while the other one is generated in a wide manner with quite a lot of space for towns, railways, etc.

This is done by setting up a grid of multipliers. Each multiplier refers to a 16x16 section on map. Iteratively, I adapt sections of that grid of multipliers by adding or subtracting the same offset. The longer I do this, the more random it looks like.

The newly introduced randomness parameter basically defines how long I do this. If you set it to zero, each section of the grid will gain the same value (default : 500 = divide the calculated (max) width value by two).

Example: River has width 3, the wide valleys multiplier in the settings is set to 6.
Then, if the grid of multipliers contains the value 100, tiles at most (100 / 1000) * 6 * 3 = 1 tiles away from the river are lowered ==> a narrow valley
If, the grid of multiplier contains the value 1000 (the maximum possible value in that grid), tiles (1000 / 1000) * 6 * 3 = 18 tiles away from the river can be lowered ===> a pretty wide valley.





I now regard this queue as complete and finished. I.e. the phase where I release once every one or two days will probably stop now. Report bugs if you find some, furthermore of course reviews of the code are welcome.

During the last few days, I tried to add sourcecode comments where they didn´t exist yet, but of course, in such a codebase you will always find things that are not clear immediately, so ask if you find such things.. Probably, the tooltip etc. strings can also be refined further at some point of time, some of the concepts might even need an external documentation to make them completely clear (explain the algorithm described above in a tooltip, and you know what I mean...). And well, I probably won´t start friendship with the GUI framework of OpenTTD, i.e. the parameter GUI (at least for me) looks reasonable now (except for one string that should be right-aligned, but is centered), but probably, this is something that might be refined either.

Have fun with it...

EDIT: v60 has a small but important bugfix compared with the v59 I just deleted: In v59 you could not save games or scenarios since for one variable, type in settings_type.h and in settings.ini did not match.
Attachments
Unbenannt, 1. Jan 1905#4.png
Narrow valley.
(603.51 KiB) Not downloaded yet
Unbenannt, 1. Jan 1905#5.png
Wide valley.
(646.64 KiB) Not downloaded yet
rivers_v60.zip
(269.03 KiB) Downloaded 160 times
Wahazar
Tycoon
Tycoon
Posts: 1451
Joined: 18 Jan 2014 18:10

Re: Patch: The Rainfall River Generator

Post by Wahazar »

Works perfect now (patching/compilation).
Soon report about real-based height map.
User avatar
wallyweb
Tycoon
Tycoon
Posts: 6102
Joined: 27 Nov 2004 15:05
Location: Canada

Re: Patch: The Rainfall River Generator

Post by wallyweb »

ic111 wrote:Finished.
...
I now regard this queue as complete and finished. I.e. the phase where I release once every one or two days will probably stop now. Report bugs if you find some, furthermore of course reviews of the code are welcome.
:bow:
Rubidium wrote:The first warning, about the unused function, ...
Indeed, compiling from trunk via svn checkout did eliminate this warning. Updated Flyspray task/6396 :bow:
Attachments
OpenTTD r27458M Rainfall_v60_Win32.7z
(5.6 MiB) Downloaded 172 times
Wahazar
Tycoon
Tycoon
Posts: 1451
Joined: 18 Jan 2014 18:10

Re: Patch: The Rainfall River Generator

Post by Wahazar »

Patch works fine for randomly generated map. At least mountains looks like mountains with real valleys, not like stack of pyramids.
Valleys are filled with network of rivers, which is challenging for making railways lines, but give opportunity to use inland ships.

I tried to use this patch on this height bitmap:
http://149.156.194.203/~mczapkie/Train/ ... alecs2.png
generally it works fine, rivers are usually where they should appear, but there is no lakes.
I tried different settings, for example:
Image
but with no effect. Can you suggest any settings, or those lakes should just be placed manually/imported?
How large is range for probability values - 0...100 ?
ic111
Director
Director
Posts: 608
Joined: 17 Jul 2007 17:56

Re: Patch: The Rainfall River Generator

Post by ic111 »

Lakes are generated where the terrain the generator finds has a depression, i.e. a group of tiles from which you can only travel upwards.

Generally, during testing, I rather had the problem that my maps contained to many of them; I even invented a preprocessing step which eliminates the small ones. It is controlled by the following two config-file-only settings:

- rainfall.small_oceans_removal_factor eliminates oceans smaller than MapSize() / rainfall.small_ocean_removal_factor
- rainfall.small_basins_removal_limit eliminates depressions smaller than this number of tiles

Looking at your map, I doubt that this setting can change anything. You have some depressions, but as far as I see, not at locations where the water flows to. If you load that heightmap in scenario editor, and block some of the river paths by a short barrier of higher tiles, and save the result as heightmap again, then at that locations a lake will be triggered.

Theoretically, one might think about a setting for actually generating such small basins before generation starts.

However, this would need to be handled carefully. In particular, too many lakes are a performance issue for the patch (not the 10 or 20 or 50 lakes you might want on such a lake, but the thousands of small depression tgp generates). When each of them becomes a lake, the lake definition algorithm that runs well if you have a couple of lakes, has a not-very-good running time... The background for this is, that such lakes tend to merge as they are start growing close to each other, and that calculating flow that flows through such a lake in a correct way is a quite non-trivial task.

Furthermore, the question where a lake makes sense in such a heightmap is not quite trivial. So, for heightmaps, I argue for preparing a heightmap using well-placed changes in scenario editor.

I myself just prepared the attached heightmap. Load it e.g. with max heightlevel 60, flow needed for river 200, flow per lake tile volume zero. In the central to north section of the map, you see a big valley leading first to the east, then to north-east, and then, outside the high mountains, again to the west. There, I prepared a few lakes this river should flow through by adding barriers of tiles.

Similar, I prepared a lake in one of the valleys leading towards the southern ocean.

The range for the probability values is 0..1000; but I don´t think they are important for the task at hand. I would rather argue for starting with zero for all of them. E.g. outflow canyons are more for tgp generated maps, as there you can have pretty deep depressions, where it doesn´t look bad if the river just searches its way out, instead of filling it with a great waterfall outside.

BTW, your screenshot shows what I suspected - that GUI needs to be improved in a technical sense. Anyone feeling comfortable with the OpenTTD GUI framework is invited to do so ;-)
Attachments
TirolV5.8.png
(208.96 KiB) Not downloaded yet
Wahazar
Tycoon
Tycoon
Posts: 1451
Joined: 18 Jan 2014 18:10

Re: Patch: The Rainfall River Generator

Post by Wahazar »

OK, I understand - there is no lakes, because there are outflows. OK, I can live with that, I can import it from txt file.
Is it possible to set any parameter to have 1 height dam enough to make lake if placed perpendicular to river valley?
Or it would always erode?

BTW, I found a bad glitch:
Image
Note water at the tile edge - there is something wrong with water tile placement.
ic111
Director
Director
Posts: 608
Joined: 17 Jul 2007 17:56

Re: Patch: The Rainfall River Generator

Post by ic111 »

At what height are those tiles? With which parameters generated? Did you alter the two I showed you above?

Actually, three of the four tiles should be avoided by my bad inclined slope check. Only the one which has a flat tile below is ok with respect to that check, although the question is why the lake didn´t grow bigger. Maybe a consequence of your settings...
Is it possible to set any parameter to have 1 height dam enough to make lake if placed perpendicular to river valley?
Or it would always erode?
A dam of height one should always suffice, unless you set a probability for an outflow canyon. This is the only case where something is supposed to erode.

Actually, ensuring that those heights are maintained took me quite a lot of time...
Wahazar
Tycoon
Tycoon
Posts: 1451
Joined: 18 Jan 2014 18:10

Re: Patch: The Rainfall River Generator

Post by Wahazar »

ic111 wrote:At what height are those tiles? With which parameters generated?
Height 0 - is it lowered by your patch, or sea disappeared in an other way?
Settings are as follows, I started server before read your answer:

Code: Select all

rainfall.flow_for_river = 400
rainfall.flow_per_lake_volume = 30
rainfall.number_of_flow_modifications = 15
rainfall.wider_rivers_enabled = 1
rainfall.wider_rivers_multiplier = 60
rainfall.wider_valleys_enabled = 1
rainfall.wider_valleys_multiplier = 5
rainfall.lake_outflow_canyon_probability = 0
rainfall.lake_reduce_to_guaranteed_probability = 50
rainfall.lake_fan_delta_probability = 200
rainfall.lake_fan_delta_max_size = 20
rainfall.lake_island_probability = 10
rainfall.lake_island_max_size = 20
rainfall.lake_shore_probability = 50
rainfall.lake_shore_max_size = 5
rainfall.town_placers = 
rainfall.small_oceans_removal_factor = 1000
rainfall.small_basins_removal_limit = 13
rainfall.wider_valleys_randomness = 1000
I see that oceans_removal_factor is 1000, not sure how large is that factor, and I don't see such setting in menu.
Are these oceans removed by levelling ground to 1 or just by drying tiles?
ic111
Director
Director
Posts: 608
Joined: 17 Jul 2007 17:56

Re: Patch: The Rainfall River Generator

Post by ic111 »

McZapkie wrote:
ic111 wrote:At what height are those tiles? With which parameters generated?
Height 0 - is it lowered by your patch, or sea disappeared in an other way?
I don't see a reason for my patch to lower that land. How big was that map?
I see that oceans_removal_factor is 1000,
This means that oceans that are smaller than MapSize() / oceans_removal_factor get removed by raising them to height 1. If your map would be just 64x64 then this would be an explanation, as (64 * 64) / 1000 = about 4.

In that sections of code, I detect "ocean" by the property "tile is flat and has height zero". This is, because checking for MP_WATER is unreliable, as my terraformings (to height zero) don't result in water before the tile loop runs, and the tile loop doesn't run while the generator runs.

I don't regard such slopes bad if they are close to the ocean. Maybe before the generator tried to improve slopes, some tile at height zero was around, and because of that, the river tile wasn't declared "no river".
not sure how large is that factor, and I don't see such setting in menu.
I didn't add them. In case of the basins_removal_limit, you could really make it generate for a quite long time if you set the value to 0, and generate using tgp. Thus, you can configure that behaviour, but I regard it as senseful that it is not that obvious how to do so.

In case of the ocean_removal_factor, it was more "not again that tedious GUI work"...
Are these oceans removed by levelling ground to 1 or just by drying tiles?
Levelling ground to height 1.
User avatar
JGR
Tycoon
Tycoon
Posts: 2557
Joined: 08 Aug 2005 13:46
Location: Ipswich

Re: Patch: The Rainfall River Generator

Post by JGR »

I built your code with -fsanitize=address -fsanitize=undefined, which I find is a good way of finding a number of types of bug.
There are number of allocations made with malloc/calloc, but freed using delete, which AddressSanitizer takes great offence to, even though it's fairly benign in this case. See attached patch for suggested fixes.
(I also got signed overflow warnings in the Perlin noise generator (int_noise in tgp.cpp), but I don't think that matters as the output is just noise used for the initial terrain generation).
In general I think that the code (and the OpenTTD codebase in general) would be improved by using some sort of std::unique_ptr style smart pointer instead of lots of manual code to free/delete stuff, but that's a more abstract comment.

One other issue is that the river and iteration generation info (_river_map, _river_iteration) in the landinfo window is not cleared when loading a new game, but the rest of the rivers info is cleared.
This is particularly problematic if generating a smaller map, and then loading a larger one as the land info window will crash or read from uninitialised memory if pointed at a tile with a coordinate larger than the generated map size.

Beyond McZapkie's bug below, I haven't noticed any issue with the output maps, they look pretty nice :).
Attachments
rivers-free-delete.diff
(856 Bytes) Downloaded 127 times
Ex TTDPatch Coder
Patch Pack, Github
KeldorKatarn
Transport Coordinator
Transport Coordinator
Posts: 274
Joined: 13 Apr 2010 21:31

Re: Patch: The Rainfall River Generator

Post by KeldorKatarn »

It's def an improvement over the trunk generator but I still don't like how it behaves. Yes it may work great on realistic height maps with mountains but on relatively flat land or more varied terrain that the normal terrain generator creates it just makes stupid things. Too many rivers in the same place, too many lakes without outflows (the map edge should be considered an outflow if there's no ocean. And the city placement... I dunno. I've had it often that I had only acouple of large lakes on the map and nearly ALL cities were places right next to them, sometimes only 1 tile apart. there should still be a minimum distance between cities or something. in general I think it's not right to let the rivers look for outflows.. to some degree the rivers should terraform their way across the map. Also old river arms should dry out if a loop connects up. Rivers with loops in them look stupid too. In general I think lakes without any outflows should not be allowed at all on maps except maybe on tropical. There's nearly no lakes on the world that have no outflows and the few there are are mostly in hot areas like africa and salt lakes.

Also I think the game generates WAY too many lakes to begin with. If you look at the average map and look at towns vs lakes in terms of quantity it's insane how many large lakes there are. I mean look at a normal country.. how many large lakes do they have? Germany has about one large one. That's it. The entire US has none but the great lakes and those are even interconnected. The entire mainland doesn't have such huge lakes. Smaller ones yes fine but not these huge level 0 things that are basically oceans in the middle of a continent.

The game should try to not create so many lakes but instead try to create a nice river network that looks realistic. Maybe it should try to generate one-three major rivers that flow over most of the map. Have their spring somewhere in the center and then flow towards the map edge. Then spawn smaller rivers which try to flow into those major rivers or other small rivers. All in a way that makes sense and looks natural. I mean on a non-ocean map right now boats and ships are nearly useless. It makes no sense to use them on rivers mostly because most of the rivers just aren't long enough. For it to make sense the rivers need to be LONG with several side rivers, so there's kind of a river network spanning the map. Not hundreds of small 1ß tile long rivers flowing into a huge lake and ending there without any purpose.

Just some ideas on where to guide this thing. Rain based is ok but rivers in real life don't just flow down a concrete terrain. They carve themselves into terrain and keep going. The rivers in this should try to do the same. They should carve their way towards other rivers or the map edge.

I recently created something like this myself by hand just as an example. See attachment.
Attachments
Deutsche Reichsbahngesellschaft, 26th Jan 1924.png
(297.65 KiB) Not downloaded yet
ic111
Director
Director
Posts: 608
Joined: 17 Jul 2007 17:56

Re: Patch: The Rainfall River Generator

Post by ic111 »

KeldorKatarn wrote:It's def an improvement over the trunk generator but I still don't like how it behaves. Yes it may work great on realistic height maps with mountains but on relatively flat land or more varied terrain that the normal terrain generator
Normal terrain generator means tgp. IMHO, tgp does not generate a realistic landscape. It generates a lot of basins (that's were the lakes are located lateron). It mostly fails to generate longer valleys.

So, yes, the river generator relies on a senseful, realistic terrain as input (given that you want realistic rivers). And this, in the current OpenTTD world, to some degree means: Real world heightmap.
creates it just makes stupid things. Too many rivers in the same place,
Increase the flow needed for creating a river. Then you'll get less of them.
too many lakes without outflows
Set the flow consumed per lake tile volume to zero. Then every lake (except maybe for height zero cases, and maps without ocean) will get an outflow.
(the map edge should be considered an outflow if there's no ocean.
Hm, rivers disappearing at the map edge are implemented.
And the city placement... I dunno. I've had it often that I had only acouple of large lakes on the map and nearly ALL cities were places right next to them, sometimes only 1 tile apart.
there should still be a minimum distance between cities or something.
Then you can decrease the weight of the lake city placer. The weights in the detail config indicate, how often which strategy is used.

Obviously, this means some kind of fine-tuning those weights, but this is the price of being able to control where cities and towns are placed, without having to decide on each location oneself.
in general I think it's not right to let the rivers look for outflows.. to some degree the rivers should terraform their way across the map.
This, to be honest, is not the concept of this generator. What you want is river & landscape generation in one.

And, this would damage any heightmap. As I have said some time ago, if I feed it with a heightmap, I expect it to preserve the landscape at least on the large scale.
Also old river arms should dry out if a loop connects up. Rivers with loops in them look stupid too.
Well, at least islands in rivers are realistic.
Also I think the game generates WAY too many lakes to begin with. If you look at the average map and look at towns vs lakes in terms of quantity it's insane how many large lakes there are. I mean look at a normal country.. how many large lakes do they have? Germany has about one large one. That's it.
Depends on which size you require to accept it as lake. But, more important, this is a consequence of the lake basins tgp generates.

What you can do about it is setting the probability for reducing a lake to a high value.
Just some ideas on where to guide this thing.
To be honest, it will not change in the large scale. It is finished except for maybe some fine-tuning and bugfixing.
Rain based is ok but rivers in real life don't just flow down a concrete terrain. They carve themselves into terrain and keep going. The rivers in this should try to do the same. They should carve their way towards other rivers or the map edge.
What you would like is something like an erosion module for OpenTTD. I mean, if they would just carve their way, as you say, you would get a lot of canyons. To do this in a realistic way, you would need to simulate things like glacier movements, different geological conditions, and so on.

Can maybe be done to some extent, but not by me.

In fact, my opinion is that earth presents you so much landscape you can use for heightmaps, that there is no point in spending a lot of time in trying to generate it oneself.
I recently created something like this myself by hand just as an example. See attachment.
If you in particular like the straight lines (I tried to get rid of them), then you can set the number of flow modifications per 1000 tiles to a low value.
pi1985
Engineer
Engineer
Posts: 107
Joined: 16 May 2013 08:22
Location: Ukraine

Re: Patch: The Rainfall River Generator

Post by pi1985 »

I tried to apply this patch (v60) to latest trunk and got many rejects for rivers_rainfall.cpp and rivers_rainfall.h.

Code: Select all

$ for i in patches/*patch; do patch -F0 -p1 -i $i; done
...
patching file src/lang/english.txt
Hunk #1 FAILED at 1076.
1 out of 1 hunk FAILED -- saving rejects to file src/lang/english.txt.rej
patching file src/misc_gui.cpp
Hunk #1 succeeded at 229 (offset 1 line).
patching file src/rivers_rainfall.cpp
Hunk #1 FAILED at 30.
1 out of 1 hunk FAILED -- saving rejects to file src/rivers_rainfall.cpp.rej
patching file src/rivers_rainfall.h
Hunk #1 FAILED at 83.
1 out of 1 hunk FAILED -- saving rejects to file src/rivers_rainfall.h.rej
patching file src/landscape.cpp
...
$
Can you please make one large patch instead of many small?
Last edited by pi1985 on 25 Dec 2015 18:03, edited 1 time in total.
Image
Image
Image.
User avatar
FLHerne
Tycoon
Tycoon
Posts: 1543
Joined: 12 Jul 2011 12:09
Location: St Ives, Cambs, UK

Re: Patch: The Rainfall River Generator

Post by FLHerne »

pi1985 wrote:I tried to apply this patch (v60) to latest trunk and got many rejects for rivers_rainfall.cpp and rivers_rainfall.h.
You didn't read the series file - some of the patches are old and shouldn't be applied. With r27477, I get no rejections when applying the correct patches.

while read PATCH; do patch -p1 < "patches/$PATCH"; done < patches/series
pi1985 wrote:Can you please make one large patch instead of many small?
Muddling all the changes together would get in the way of reviewing, editing, updating or anything else except blindly applying them.
Your post shows you understand that the whole set can be applied with a single command (albeit not the one you used :wink: ), so it's hard to see why you'd make that request.

Aside, posting dozens of lines of mostly-irrelevant output is usually a bad idea, please don't,
Temporary Permanent signature filling text. Content coming soon delayed indefinitely! Oh, and I have had a screenshot thread.
Linux user (XMonad DWM/KDE, Arch), IRC obsessive and rail enthusiast. No longer building robots; now I ring church bells.
Author of an incredibly boring stickied post about NewGRFs.
pi1985
Engineer
Engineer
Posts: 107
Joined: 16 May 2013 08:22
Location: Ukraine

Re: Patch: The Rainfall River Generator

Post by pi1985 »

FLHerne wrote: while read PATCH; do patch -p1 < "patches/$PATCH"; done < patches/series
Thanks for this command. I didn't really notise a difference betwen series file and files in folder.
Image
Image
Image.
Post Reply

Return to “OpenTTD Development”

Who is online

Users browsing this forum: No registered users and 10 guests