Page 1 of 8

Patch: Create realistic heightmap, water & town placement

Posted: 05 Sep 2007 21:21
by wolkottept
This patch is a extension of the PNG-Heightmaps extension (http://www.tt-forums.net/viewtopic.php?f=29&t=27052). It makes it possible to automatically generate scenarios with realistic heights, water placement and the positioning of towns and cities on the map. It uses the SRTM data files for both height and water location (http://www2.jpl.nasa.gov/srtm/). The cities information is retrieved from a database from World Gazetteer (http://world-gazetteer.com/wg.php?x=&me ... &col=aohdq)

A shell script generates two the necessary two PNG files that can be used within OpenTTD. It automatically downloads the necessary files based on a few user parameters (description see below). In OpenTTD those files are used to generate a scenario in combination with a global town database.

NOTE: I started with a version that reads ASC-files, but this required a lot of memory. I discovered that it is also possible to download TIFF files that contain all height information. A shell-script combines the necessary TIFF files into a single PNG file, that can be used by the patched OpenTTD. The old patch is still available for download

An example map is shown below. An example map is shown below. You can create this map too, by following the steps below. Comments and suggestions are more than welcome.
Image

What do you need? How does it work?

Create PNG & info files
The patch uses three files (..._land.png, ..._water.png, and ..._info.real) to generate the heightmap and place water bodies. Those can be created by a shell-script that is included in the prepare_realistic_png.tgz. See the README.txt in the archive for more info. The shell-script also generates a fourth file, which can be used as a thumbnail.

The exact area you want can be selected in OpenTTD, however do not make the overall dimensions of the too large. I tested with input PNG files upto 6000x6000. The dimension of the scenario can be varied in OpenTTD independent of the input image size.

For this tutorial we use the available scenario "southern_europe" from my website (see above). Place the files
  • southern_europe_land.png
  • southern_europe_water.png
  • southern_europe_info.real
in the "scenario/heigthmap" directory. The fourth file "southern_europe_thumb.png" is only used to visualize the content of the other three files.

The town information
Place the towns_1k.txt file (in all_towns.tar.gz) in the data directory of OpenTTD

What to do in OpenTTD?

A. Heigtmap

Start the scenario editor and select the new added button:
Image

Select the generated real-file (this file links to the other two!)
Image

A modified heightmap window will appear, where you can select the desired location of the to be generated map. The longitude and latitude of east/west and north/south can be changed in degrees and minutes. In the preview window the selected area is surrounded by the red rectangle.
You can disable the lattitude/longitude correction, but this makes the maps less realistic. This will change the preview too.

Image

We modify the southern latitude to 41 and the western longitude to 1. The other longitudes are calculated automatically, because we have selected a square map size. Under the Long/lat correction you are able to change the anchor type, which results in a change of the longitude selection fields.

On the left bottom of the window height information is displayed. The histogram (i.e. bar graph) shows the height distribution. The height of the bars indicate the amount of pixels in the input PNG files (in a logarithmic scale). The white line indicates function that will be used to translate real height to the levels that will be used in the map that is generated.

Below this two extra options are included to modify the maximum height-level that is desired and the function that is used to transform the real height to the actual levels in the scenario. Try the two options to see the effect with the white line in the graph.

When you click on "Generate" a warning will appear:
Image

We accept this warning for the moment.

B. Water

Now you have generated a map, the water bodies can be placed. Select the land generation button in the top-bar
Image
and select the extra button "Realistic placement of water" in the window:
Image

The window will not disappear, but several water bodies are placed on the map. Water is only placed on tiles that were flat or have the good slope and that do not belong to the sea. In earlier versions the water was placed as canals, but currently I use rivers. Water that is directly connected to the sea becomes sea too.

C. Towns

Now you have generated a map and placed water, the towns can be placed. Select the town button in the top-bar
Image
and select the extra button "Realistic placement of towns" in the window:
Image

This will bring-up a new window. Given the previous selected longitude and latitudes all the towns that are present in the area are read from the towns_1k.txt file. You can modify the parameters to your own needs.
Image

In the lower part of the window the histogram of the read towns is presented. This gives in idea of the amount of towns with a certain size. With the 4 popdown buttons you can select what the minimum population should be for a "small,medium,large town and city". An estimation of the number of generated towns is given. The actual number will be less (see below).
For each town type you also can change its initial size. It is possible to make very large "small towns" and small cities.
The two top options are discussed below.

D. Things to keep in mind

What happens when a town cannot be placed
It might be that a town location is on a slope or water. In this case the patch will try all 8 tiles that are surrounding the location. It all those 9 tiles are not possible a sign will be placed instead.

What happens with towns that are close to other towns or the border
With the top 2 options in the town generation window you can change the behavior of the placements of towns. The first option is the distance to the edge of the map. If a town is too close to the edge it will not be placed and a sign is placed on the map instead.
The second option is for the case two towns are too close to each other. Given that two towns are too close the population of both towns will determine the behavior of the placement.
A small town will disappear in a large town if their population ratio is more than 2. The population will be the sum of both.
If both towns are almost equal in size a new town is created. Its location is between the two towns, the population will be the sum of both and both town names are combined (largest town first). It is also possible that 3 or more towns are merged in this second case.

Image

E. How does a final map look?

Below you can see a part of the screenshot of the southern_europe example files.
Image

F. Where can I find more maps?

Several other maps I used for testing are available on my website http://wwwhome.cs.utwente.nl/~wolkottep ... enTTD/Maps

Future work, known bugs etc.
Known bugs / imperfections:
  • The town placement goes wrong if the ratio of the selected area is not equal to the map ratio. [DONE, Selected area always equal to map ratio]
  • I am not a GUI hero, the UP/DOWN buttons in the Town window are not working. [DONE.]
  • I still am not a GUI hero, the filename and location of the town file is fixed to the data-directory.
  • Adjustment of negative longitudes/latitudes via the text button do not work.
  • The estimated number of towns in the gui does not include merges of two or more towns.
  • Not all strings in the gui are correct. [Improved: work in progress]
  • Improve code to the OpenTTD standards and better error handling (currently fprintf to stdout). [Work in progress]
  • Increase speed. [Improved: Changed input from ASC to PNG]
  • Correction of latitude and longitude lengths. [Almost finished: Tested for latitudes above and below equator. Not tested around the equator and below]
  • Update to latest OpenTTD revision (rev 13922). [DONE.]
  • Improve water placement [Check the connection with sea tiles. Can be further improved]
  • Check the compile problems for Win32 with BuildOTTD
Wishes:
  • Show a small preview of the loaded heightmap file, such that the total/selected area is visible [DONE.]
  • Rotate the map
  • More options for the height distribution (e.g. limit maximum height to 2000m) [DONE. Maximum level and transform function can be specified. More suggestions are welcome]
  • Include water information. This will improve the coast line and makes lakes/rivers possible [DONE.]
  • Include road information. For example to generate realistic highways
  • Include industry information. Does anyone knows databases for this?
Download previous versions:
Patch
  • realistic_map_and_towns_wolkottept_r13922_0v4.diff: 29 times
  • realistic_map_and_towns_wolkottept_r13922_0v5.diff: 7 times
  • realistic_map_and_towns_wolkottept_r13922_0v6.diff: 8 times
Script
  • prepare_realistic_png_0v4.tgz: 49 times
  • prepare_realistic_png_0v5.tgz: 3 times

Re: Patch: Create realistic heightmap & automatic town placement

Posted: 05 Sep 2007 21:39
by Bilbo
Looks great, but the italy seems to be too wide ... have you taken in account correction of latitude and longitude lengths? (at equator they are the same, but anywhere else you should recalculate it to be at least a bit more precise :)

Re: Patch: Create realistic heightmap & automatic town placement

Posted: 05 Sep 2007 21:52
by wolkottept
No I did not take this correction into account. A good suggestion to put on the wishlist.

Re: Patch: Create realistic heightmap & automatic town placement

Posted: 06 Sep 2007 00:57
by athanasios
Great patch, it will be even greater with 127 patch finished!

Re: Patch: Create realistic heightmap & automatic town placement

Posted: 07 Sep 2007 10:05
by thebrightside
Does your patch also have the problem that if the diffrence in darkness is not big enough, opentdd sees the low areas of land as water or is this not a problem while using it?

Re: Patch: Create realistic heightmap & automatic town placement

Posted: 07 Sep 2007 11:50
by wolkottept
It depends on what you call a problem. From the ASC file its hard to determine what is land and what is water. Therefore, I want to include information that describes coast-lines and rivers that should be available somewhere.

The current approach is the folllowing:
The asc-file has for a lot of water location an fixed unknown value (-9999). These locations I set to level 0. All values that have a known value (for the souther alps example from -19m to 4600m) I place them between level 1 and 15 with a proportional, square root or quadratic function (the height distribution option) . So negative heights will become land.

A nice test is The Netherlands :wink:

Re: Patch: Create realistic heightmap & automatic town placement

Posted: 10 Sep 2007 18:39
by chrissicom
This is definitely a cool patch, I will try it with the huge maps patch in ChrisIN and see what maps I can come up with :D maybe I can make my brother happy with a biiiig Japan map *gg*

Re: Patch: Create realistic heightmap & automatic town placement

Posted: 16 Sep 2007 19:45
by wolkottept
The correction for latitude and longitude lengths is included in the patch. It is tested for maps above the equator. Around and below might cause some problems.

Furthermore, a preview window is included.

See extra images in the updated "manual" for the changes.

Re: Patch: Create realistic heightmap & automatic town placement

Posted: 02 Oct 2007 15:44
by okdewit
EXACTLY what I was looking for! Thanks a lot, hoping to make a big realistic version of the The Netherlands :)

Re: Patch: Create realistic heightmap & automatic town placement

Posted: 02 Oct 2007 18:57
by ganzpopp
Seems like a very interesting patch to me. Although the corrected longitude latitude length maps look better, it is of course nice if you would be able to include an area not suitable for OTTD supported map ratios, as in the first picture.
okdewit wrote:EXACTLY what I was looking for! Thanks a lot, hoping to make a big realistic version of the The Netherlands :)
Already done? Would love to play it!

Re: Patch: Create realistic heightmap & automatic town placement

Posted: 02 Oct 2007 21:20
by okdewit
I compiled the nightly with this patch successfully, and tried to load ASC files directly from google earth into OTTD, which caused a hang (100% cpu for 2 hours, non-responsive OTTD)

I tried to "thin" the DEM manually with MicroDEM (as instructed in the other topic), but MicroDEM gives errors as soon as your trying to do anything to any Arc-Ascii SRTM file. It works fine with geotiffs, but they can't be merged nor directly loaded into the patched OTTD.

Still trying, I really want that 4K x 4K Netherlands map :D

Re: Patch: Create realistic heightmap & automatic town placement

Posted: 16 Feb 2008 15:00
by Pendrokar
Adding just one patch is a nightmare for someone that doesn't know a thing! Been reading, downloading softwares for hours and no progress on adding it! :cry:
Could someone, anyone please create and upload(or just send me) an executable OpenTTD with this patch in it! Your given effort will allow a new OpenTTD server come up! :)

Re: Patch: Create realistic heightmap & automatic town placement

Posted: 16 Feb 2008 15:23
by PhilSophus
As the last version of this patch is already 5 months old, I would be really astonished if it applied to current trunk versions without major effort. So, when you try, take exactly the SVN revision for which the patch is meant (11047, a few revisions off is usually okay, though). So, any features added after September will not be in there.

My crystal ball tells me, you are using Windows,so BuildOTTD is your friend. If you are having problems, please tell us what you tried and what happened, otherwise we won't be able to help you.

I could provide an executable but I doubt a Linux executable would be of any help to you :wink:

Re: Patch: Create realistic heightmap & automatic town placement

Posted: 16 Feb 2008 17:45
by Youri219
If your purpose is to make a map, just compile it against the nightly this patch was designed for. Then make a map, save it, and reopen it with a current nightly.

Re: Patch: Create realistic heightmap & automatic town placement

Posted: 16 Feb 2008 18:14
by Pendrokar
Youri219 wrote:If your purpose is to make a map, just compile it against the nightly this patch was designed for. Then make a map, save it, and reopen it with a current nightly.
Thats the problem I cannot get an executable nightly build, because I have to compile it first, WHICH I CANNOT DO(or at least it ain't going too well)!

Re: Patch: Create realistic heightmap & automatic town placement

Posted: 16 Feb 2008 21:29
by andrewas
If you want to do anything involving patches, then its in your best interest to learn how to compile them, and its also useful to learn enough about c++ to be able to update patches across trivial revisions, as well as to merge patches with minor conflicts. The more you learn, of course, the better off you are, particularly if you want to use a variety of patches not included in an integrated nightly or other patchpack.

Under non-vista windows, BuildOTTD is the simplest solution, and on other platforms, tell us how far you get and what is stopping you from continuing. Also, re-read the stickies and whatever guide you are following, its amazing how often people miss one vital step and waste hours trying to fix it.

All that said however, heres a windows binary with the latest version of this patch.

Re: Patch: Create realistic heightmap & automatic town placement

Posted: 16 Feb 2008 21:39
by Pendrokar
andrewas wrote:If you want to do anything involving patches, then its in your best interest to learn how to compile them
:bow: Thank You Very Much! It was just that I really needed this patch! :bow:

Re: Patch: Create realistic heightmap & automatic town placement

Posted: 06 Jul 2008 19:05
by mitchell600
I cant make this work...
I putted the all town.txt in the data and i open up the game and stil the old junk..

PLZ help :bow:
BTw my inglish is not that good ;)
Mitch

Re: Patch: Create realistic heightmap & automatic town placement

Posted: 06 Jul 2008 19:29
by CommanderZ
Are you 100% sure you have an executable with this patch compiled in?

Btw, it is pretty outdated (more than 2600 revisions).

Re: Patch: Create realistic heightmap & automatic town placement

Posted: 06 Jul 2008 20:35
by wolkottept
Hello,

Nice to hear that there is still interest in my path. If you compile the patch with the indicated revision it should just work, but indeed it requires an quite old revision.

I have had some free time, so I recently started to upgrade the patch.

Currently, I am working on automated creation of the necessary input files (automatic download of the necessary data from internet, combine into a single data file, etc.). This should make it a lot easier to select any place on the globe and generate the required input for OpenTTD.

Also automatic water placement is almost ready (mainly gui stuff that has to be done). If that is finished, I will try to upgrade the patch to the current revision of OpenTTD.

If you have any suggestions, comments. Please feel free to post this or send an e-mail.