Patch: Create realistic heightmap, water & town placement

Forum for technical discussions regarding development. If you have a general suggestion, problem or comment, please use one of the other forums.

Moderator: OpenTTD Developers

Post Reply
wolkottept
Engineer
Engineer
Posts: 26
Joined: 27 Aug 2007 16:25

Patch: Create realistic heightmap, water & town placement

Post by wolkottept »

This patch is a extension of the PNG-Heightmaps extension (http://www.tt-forums.net/viewtopic.php?f=29&t=27052). It makes it possible to automatically generate scenarios with realistic heights, water placement and the positioning of towns and cities on the map. It uses the SRTM data files for both height and water location (http://www2.jpl.nasa.gov/srtm/). The cities information is retrieved from a database from World Gazetteer (http://world-gazetteer.com/wg.php?x=&me ... &col=aohdq)

A shell script generates two the necessary two PNG files that can be used within OpenTTD. It automatically downloads the necessary files based on a few user parameters (description see below). In OpenTTD those files are used to generate a scenario in combination with a global town database.

NOTE: I started with a version that reads ASC-files, but this required a lot of memory. I discovered that it is also possible to download TIFF files that contain all height information. A shell-script combines the necessary TIFF files into a single PNG file, that can be used by the patched OpenTTD. The old patch is still available for download

An example map is shown below. An example map is shown below. You can create this map too, by following the steps below. Comments and suggestions are more than welcome.
Image

What do you need? How does it work?

Create PNG & info files
The patch uses three files (..._land.png, ..._water.png, and ..._info.real) to generate the heightmap and place water bodies. Those can be created by a shell-script that is included in the prepare_realistic_png.tgz. See the README.txt in the archive for more info. The shell-script also generates a fourth file, which can be used as a thumbnail.

The exact area you want can be selected in OpenTTD, however do not make the overall dimensions of the too large. I tested with input PNG files upto 6000x6000. The dimension of the scenario can be varied in OpenTTD independent of the input image size.

For this tutorial we use the available scenario "southern_europe" from my website (see above). Place the files
  • southern_europe_land.png
  • southern_europe_water.png
  • southern_europe_info.real
in the "scenario/heigthmap" directory. The fourth file "southern_europe_thumb.png" is only used to visualize the content of the other three files.

The town information
Place the towns_1k.txt file (in all_towns.tar.gz) in the data directory of OpenTTD

What to do in OpenTTD?

A. Heigtmap

Start the scenario editor and select the new added button:
Image

Select the generated real-file (this file links to the other two!)
Image

A modified heightmap window will appear, where you can select the desired location of the to be generated map. The longitude and latitude of east/west and north/south can be changed in degrees and minutes. In the preview window the selected area is surrounded by the red rectangle.
You can disable the lattitude/longitude correction, but this makes the maps less realistic. This will change the preview too.

Image

We modify the southern latitude to 41 and the western longitude to 1. The other longitudes are calculated automatically, because we have selected a square map size. Under the Long/lat correction you are able to change the anchor type, which results in a change of the longitude selection fields.

On the left bottom of the window height information is displayed. The histogram (i.e. bar graph) shows the height distribution. The height of the bars indicate the amount of pixels in the input PNG files (in a logarithmic scale). The white line indicates function that will be used to translate real height to the levels that will be used in the map that is generated.

Below this two extra options are included to modify the maximum height-level that is desired and the function that is used to transform the real height to the actual levels in the scenario. Try the two options to see the effect with the white line in the graph.

When you click on "Generate" a warning will appear:
Image

We accept this warning for the moment.

B. Water

Now you have generated a map, the water bodies can be placed. Select the land generation button in the top-bar
Image
and select the extra button "Realistic placement of water" in the window:
Image

The window will not disappear, but several water bodies are placed on the map. Water is only placed on tiles that were flat or have the good slope and that do not belong to the sea. In earlier versions the water was placed as canals, but currently I use rivers. Water that is directly connected to the sea becomes sea too.

C. Towns

Now you have generated a map and placed water, the towns can be placed. Select the town button in the top-bar
Image
and select the extra button "Realistic placement of towns" in the window:
Image

This will bring-up a new window. Given the previous selected longitude and latitudes all the towns that are present in the area are read from the towns_1k.txt file. You can modify the parameters to your own needs.
Image

In the lower part of the window the histogram of the read towns is presented. This gives in idea of the amount of towns with a certain size. With the 4 popdown buttons you can select what the minimum population should be for a "small,medium,large town and city". An estimation of the number of generated towns is given. The actual number will be less (see below).
For each town type you also can change its initial size. It is possible to make very large "small towns" and small cities.
The two top options are discussed below.

D. Things to keep in mind

What happens when a town cannot be placed
It might be that a town location is on a slope or water. In this case the patch will try all 8 tiles that are surrounding the location. It all those 9 tiles are not possible a sign will be placed instead.

What happens with towns that are close to other towns or the border
With the top 2 options in the town generation window you can change the behavior of the placements of towns. The first option is the distance to the edge of the map. If a town is too close to the edge it will not be placed and a sign is placed on the map instead.
The second option is for the case two towns are too close to each other. Given that two towns are too close the population of both towns will determine the behavior of the placement.
A small town will disappear in a large town if their population ratio is more than 2. The population will be the sum of both.
If both towns are almost equal in size a new town is created. Its location is between the two towns, the population will be the sum of both and both town names are combined (largest town first). It is also possible that 3 or more towns are merged in this second case.

Image

E. How does a final map look?

Below you can see a part of the screenshot of the southern_europe example files.
Image

F. Where can I find more maps?

Several other maps I used for testing are available on my website http://wwwhome.cs.utwente.nl/~wolkottep ... enTTD/Maps

Future work, known bugs etc.
Known bugs / imperfections:
  • The town placement goes wrong if the ratio of the selected area is not equal to the map ratio. [DONE, Selected area always equal to map ratio]
  • I am not a GUI hero, the UP/DOWN buttons in the Town window are not working. [DONE.]
  • I still am not a GUI hero, the filename and location of the town file is fixed to the data-directory.
  • Adjustment of negative longitudes/latitudes via the text button do not work.
  • The estimated number of towns in the gui does not include merges of two or more towns.
  • Not all strings in the gui are correct. [Improved: work in progress]
  • Improve code to the OpenTTD standards and better error handling (currently fprintf to stdout). [Work in progress]
  • Increase speed. [Improved: Changed input from ASC to PNG]
  • Correction of latitude and longitude lengths. [Almost finished: Tested for latitudes above and below equator. Not tested around the equator and below]
  • Update to latest OpenTTD revision (rev 13922). [DONE.]
  • Improve water placement [Check the connection with sea tiles. Can be further improved]
  • Check the compile problems for Win32 with BuildOTTD
Wishes:
  • Show a small preview of the loaded heightmap file, such that the total/selected area is visible [DONE.]
  • Rotate the map
  • More options for the height distribution (e.g. limit maximum height to 2000m) [DONE. Maximum level and transform function can be specified. More suggestions are welcome]
  • Include water information. This will improve the coast line and makes lakes/rivers possible [DONE.]
  • Include road information. For example to generate realistic highways
  • Include industry information. Does anyone knows databases for this?
Download previous versions:
Patch
  • realistic_map_and_towns_wolkottept_r13922_0v4.diff: 29 times
  • realistic_map_and_towns_wolkottept_r13922_0v5.diff: 7 times
  • realistic_map_and_towns_wolkottept_r13922_0v6.diff: 8 times
Script
  • prepare_realistic_png_0v4.tgz: 49 times
  • prepare_realistic_png_0v5.tgz: 3 times
Attachments
realistic_map_and_towns_wolkottept_r13922_0v7.diff
Patch for realistic heightmaps, water bodies and placement of towns. Includes preview window and correction for latitude/longitude.
(140.38 KiB) Downloaded 778 times
prepare_realistic_png_0v5b.tgz
Shell script to generate the necessary PNG files for the patch. More info in README.txt
(39.5 KiB) Downloaded 736 times
Last edited by wolkottept on 04 Sep 2008 19:52, edited 20 times in total.
User avatar
Bilbo
Tycoon
Tycoon
Posts: 1710
Joined: 06 Jun 2007 21:07
Location: Czech Republic

Re: Patch: Create realistic heightmap & automatic town placement

Post by Bilbo »

Looks great, but the italy seems to be too wide ... have you taken in account correction of latitude and longitude lengths? (at equator they are the same, but anywhere else you should recalculate it to be at least a bit more precise :)
If you need something, do it yourself or it will be never done.

My patches: Extra large maps (1048576 high, 1048576 wide) (FS#1059), Vehicle + Town + Industry console commands (FS#1060), few minor patches (FS#2820, FS#1521, FS#2837, FS#2843), AI debugging facility

Other: Very large ships NewGRF, Bilbo's multiplayer patch pack v5 (for OpenTTD 0.7.3)
wolkottept
Engineer
Engineer
Posts: 26
Joined: 27 Aug 2007 16:25

Re: Patch: Create realistic heightmap & automatic town placement

Post by wolkottept »

No I did not take this correction into account. A good suggestion to put on the wishlist.
User avatar
athanasios
Tycoon
Tycoon
Posts: 3138
Joined: 23 Jun 2005 00:09
Contact:

Re: Patch: Create realistic heightmap & automatic town placement

Post by athanasios »

Great patch, it will be even greater with 127 patch finished!
http://members.fortunecity.com/gamesart
"If no one is a fool I am also a fool." -The TTD maniac.


I prefer to be contacted through PMs. Thanks.
User avatar
thebrightside
Engineer
Engineer
Posts: 78
Joined: 28 Feb 2007 15:35
Location: The Netherlands
Contact:

Re: Patch: Create realistic heightmap & automatic town placement

Post by thebrightside »

Does your patch also have the problem that if the diffrence in darkness is not big enough, opentdd sees the low areas of land as water or is this not a problem while using it?
Geographer in making
My Scenarios
wolkottept
Engineer
Engineer
Posts: 26
Joined: 27 Aug 2007 16:25

Re: Patch: Create realistic heightmap & automatic town placement

Post by wolkottept »

It depends on what you call a problem. From the ASC file its hard to determine what is land and what is water. Therefore, I want to include information that describes coast-lines and rivers that should be available somewhere.

The current approach is the folllowing:
The asc-file has for a lot of water location an fixed unknown value (-9999). These locations I set to level 0. All values that have a known value (for the souther alps example from -19m to 4600m) I place them between level 1 and 15 with a proportional, square root or quadratic function (the height distribution option) . So negative heights will become land.

A nice test is The Netherlands :wink:
chrissicom
Route Supervisor
Route Supervisor
Posts: 415
Joined: 07 Oct 2004 10:05

Re: Patch: Create realistic heightmap & automatic town placement

Post by chrissicom »

This is definitely a cool patch, I will try it with the huge maps patch in ChrisIN and see what maps I can come up with :D maybe I can make my brother happy with a biiiig Japan map *gg*
wolkottept
Engineer
Engineer
Posts: 26
Joined: 27 Aug 2007 16:25

Re: Patch: Create realistic heightmap & automatic town placement

Post by wolkottept »

The correction for latitude and longitude lengths is included in the patch. It is tested for maps above the equator. Around and below might cause some problems.

Furthermore, a preview window is included.

See extra images in the updated "manual" for the changes.
okdewit
Engineer
Engineer
Posts: 54
Joined: 30 Sep 2007 21:55

Re: Patch: Create realistic heightmap & automatic town placement

Post by okdewit »

EXACTLY what I was looking for! Thanks a lot, hoping to make a big realistic version of the The Netherlands :)
Image
ganzpopp
Engineer
Engineer
Posts: 46
Joined: 20 Oct 2004 20:36
Location: The Netherlands

Re: Patch: Create realistic heightmap & automatic town placement

Post by ganzpopp »

Seems like a very interesting patch to me. Although the corrected longitude latitude length maps look better, it is of course nice if you would be able to include an area not suitable for OTTD supported map ratios, as in the first picture.
okdewit wrote:EXACTLY what I was looking for! Thanks a lot, hoping to make a big realistic version of the The Netherlands :)
Already done? Would love to play it!
okdewit
Engineer
Engineer
Posts: 54
Joined: 30 Sep 2007 21:55

Re: Patch: Create realistic heightmap & automatic town placement

Post by okdewit »

I compiled the nightly with this patch successfully, and tried to load ASC files directly from google earth into OTTD, which caused a hang (100% cpu for 2 hours, non-responsive OTTD)

I tried to "thin" the DEM manually with MicroDEM (as instructed in the other topic), but MicroDEM gives errors as soon as your trying to do anything to any Arc-Ascii SRTM file. It works fine with geotiffs, but they can't be merged nor directly loaded into the patched OTTD.

Still trying, I really want that 4K x 4K Netherlands map :D
Image
User avatar
Pendrokar
Engineer
Engineer
Posts: 115
Joined: 07 Aug 2006 13:47
Skype: Pendrokar
Location: Riga, Latvia

Re: Patch: Create realistic heightmap & automatic town placement

Post by Pendrokar »

Adding just one patch is a nightmare for someone that doesn't know a thing! Been reading, downloading softwares for hours and no progress on adding it! :cry:
Could someone, anyone please create and upload(or just send me) an executable OpenTTD with this patch in it! Your given effort will allow a new OpenTTD server come up! :)
PhilSophus
Chairman
Chairman
Posts: 776
Joined: 20 Jan 2007 12:08
Location: Germany

Re: Patch: Create realistic heightmap & automatic town placement

Post by PhilSophus »

As the last version of this patch is already 5 months old, I would be really astonished if it applied to current trunk versions without major effort. So, when you try, take exactly the SVN revision for which the patch is meant (11047, a few revisions off is usually okay, though). So, any features added after September will not be in there.

My crystal ball tells me, you are using Windows,so BuildOTTD is your friend. If you are having problems, please tell us what you tried and what happened, otherwise we won't be able to help you.

I could provide an executable but I doubt a Linux executable would be of any help to you :wink:
"The bigger the island of our knowledge, the longer the shore of our ignorance" - John A. Wheeler, Physicist, 1911-2008
Youri219
Traffic Manager
Traffic Manager
Posts: 191
Joined: 28 Apr 2007 11:53

Re: Patch: Create realistic heightmap & automatic town placement

Post by Youri219 »

If your purpose is to make a map, just compile it against the nightly this patch was designed for. Then make a map, save it, and reopen it with a current nightly.
User avatar
Pendrokar
Engineer
Engineer
Posts: 115
Joined: 07 Aug 2006 13:47
Skype: Pendrokar
Location: Riga, Latvia

Re: Patch: Create realistic heightmap & automatic town placement

Post by Pendrokar »

Youri219 wrote:If your purpose is to make a map, just compile it against the nightly this patch was designed for. Then make a map, save it, and reopen it with a current nightly.
Thats the problem I cannot get an executable nightly build, because I have to compile it first, WHICH I CANNOT DO(or at least it ain't going too well)!
andrewas
Engineer
Engineer
Posts: 115
Joined: 03 Oct 2005 19:14

Re: Patch: Create realistic heightmap & automatic town placement

Post by andrewas »

If you want to do anything involving patches, then its in your best interest to learn how to compile them, and its also useful to learn enough about c++ to be able to update patches across trivial revisions, as well as to merge patches with minor conflicts. The more you learn, of course, the better off you are, particularly if you want to use a variety of patches not included in an integrated nightly or other patchpack.

Under non-vista windows, BuildOTTD is the simplest solution, and on other platforms, tell us how far you get and what is stopping you from continuing. Also, re-read the stickies and whatever guide you are following, its amazing how often people miss one vital step and waste hours trying to fix it.

All that said however, heres a windows binary with the latest version of this patch.
Attachments
OTTD_r11047_heightmap.rar
(2.05 MiB) Downloaded 538 times
User avatar
Pendrokar
Engineer
Engineer
Posts: 115
Joined: 07 Aug 2006 13:47
Skype: Pendrokar
Location: Riga, Latvia

Re: Patch: Create realistic heightmap & automatic town placement

Post by Pendrokar »

andrewas wrote:If you want to do anything involving patches, then its in your best interest to learn how to compile them
:bow: Thank You Very Much! It was just that I really needed this patch! :bow:
mitchell600
Engineer
Engineer
Posts: 12
Joined: 03 Jul 2008 08:35

Re: Patch: Create realistic heightmap & automatic town placement

Post by mitchell600 »

I cant make this work...
I putted the all town.txt in the data and i open up the game and stil the old junk..

PLZ help :bow:
BTw my inglish is not that good ;)
Mitch
User avatar
CommanderZ
Tycoon
Tycoon
Posts: 1872
Joined: 07 Apr 2008 18:29
Location: Czech Republic
Contact:

Re: Patch: Create realistic heightmap & automatic town placement

Post by CommanderZ »

Are you 100% sure you have an executable with this patch compiled in?

Btw, it is pretty outdated (more than 2600 revisions).
wolkottept
Engineer
Engineer
Posts: 26
Joined: 27 Aug 2007 16:25

Re: Patch: Create realistic heightmap & automatic town placement

Post by wolkottept »

Hello,

Nice to hear that there is still interest in my path. If you compile the patch with the indicated revision it should just work, but indeed it requires an quite old revision.

I have had some free time, so I recently started to upgrade the patch.

Currently, I am working on automated creation of the necessary input files (automatic download of the necessary data from internet, combine into a single data file, etc.). This should make it a lot easier to select any place on the globe and generate the required input for OpenTTD.

Also automatic water placement is almost ready (mainly gui stuff that has to be done). If that is finished, I will try to upgrade the patch to the current revision of OpenTTD.

If you have any suggestions, comments. Please feel free to post this or send an e-mail.
Post Reply

Return to “OpenTTD Development”

Who is online

Users browsing this forum: No registered users and 18 guests