Creating a REAL map with real geography and cities

OpenTTD is a fully open-sourced reimplementation of TTD, written in C++, boasting improved gameplay and many new features.

Moderator: OpenTTD Developers

Eddi
Tycoon
Tycoon
Posts: 8258
Joined: 17 Jan 2007 00:14

Re: Creating a REAL map with real geography and cities

Post by Eddi »

if locale is really the problem, it might expect "O" instead of "E"
moki
Traffic Manager
Traffic Manager
Posts: 131
Joined: 06 Jul 2007 08:48
Location: Wismar, Germany

Re: Creating a REAL map with real geography and cities

Post by moki »

That would be strange as West-coordinates work fine, even with decimal separators. 50.3N,20.3W,40.3N,30.3W is fine while the same thing on the eastern hemisphere gives an error. I also just tried setting the number format and location in the system settings to "English (US)" but it doesn't change a thing for openTTD. The standard way to write numbers around here is 1.000.000,00 (for 1 million), so basically the opposite of the US/UK way. I bet, there's some elegant way around all this formatting confusion.

... and while I'm at it: Is it possible for the patch to use standard coordinates without the N, E, S and W behind them? Almost all data I found just uses positive values for North/East and negative for South/West. It would save a lot of time on converting and editing the databases.

When you work some more on the universal (different OS's and countries) usability of this patch, I'd be more than happy to help with the testing under Windows. I might even get the hang of this code-y stuff some day ;)

[edit]
Just had a little look at the code and got a question on the part where I get the error message:

Code: Select all

if (sscanf(buf, "%lg%c,%lg%c,%lg%c,%lg%c", 
&top.latitude, &top.lat_dir, &top.longitude, &top.long_dir, &bottom.latitude, &bottom.lat_dir, &bottom.longitude, &bottom.long_dir) != 8) {
IConsolePrintF(CC_ERROR, "Error reading edge coordinates at %s:%d: you're doing it wrong", argv[1], line);
return true;
That basically looks at the first (useable) line and if the number of coordinates and directions is anything but 8 or if they're in the wrong order, it shows the error. So 20 N 20 W 10 W 30 W are 8 separate values (num, char, num, char, etc.)... right?
Now if it doesn't recognize East-coordinates as valid, I have to assume that somehow the letter E is not recognized as an acceptable char. No idea, why that happens (especially, if it works for you), but that's the only reason I can think of right now.


Just forget everything above! I just found and killed my first bug ;)
I'll be posting the updated patch in a few minutes.
Last edited by moki on 18 Oct 2013 14:18, edited 5 times in total.
User avatar
Zuu
OpenTTD Developer
OpenTTD Developer
Posts: 4553
Joined: 09 Jun 2003 18:21
Location: /home/sweden

Re: THE ULTIMATE Portugal map toolkit - usable for others to

Post by Zuu »

Zydeco wrote:When I was making a few maps from heightmaps a few years ago, I made a patch that would read towns and coordinates from a CSV file, and place them automatically, or place a sign if the towns failed to found.
I've updated it for the current version, and tried with your data (although I set all the town sizes to small, it would require manually growing them.
It also fails to place a few, because my edge coordinates are a bit off, but it saves lots of work finding where to place towns.

I've uploaded the patch, heightmap and the towns file I made by rearranging yours into the format needed by my patch, and the heightmap resulting from running the import_towns path/to/towns-pt.txt command in the editor, without doing any further adjustments
I've made some comments on your patch in case you will want to make it better comply with the OpenTTD coding standards. (Look for "Zuu" in the attached patch file)

I'd say it would be more elegant if fix_coord was replaced by a method that reads the coordinate from the file, convert it and output a fixed coordinate. However, I see that you have one line containing two coordinates (the boundary rect corners) and lines containing town data which contain both coordinates and other data. So I see why you have chosen this solution. Perhaps, since you "fix" all coordinates, rather call it ConvertCoordinate or DecodeCoordinate or similar. As the fixed coordinates are not broken, just use a different format than you use internally.
Attachments
import_towns.patch
(7.04 KiB) Downloaded 139 times
My OpenTTD contributions (AIs, Game Scripts, patches, OpenTTD Auto Updater, and some sprites)
Junctioneer (a traffic intersection simulator)
moki
Traffic Manager
Traffic Manager
Posts: 131
Joined: 06 Jul 2007 08:48
Location: Wismar, Germany

Re: Creating a REAL map with real geography and cities

Post by moki »

Yep, I could solve my problem with the eastern coordinates all by myself. Found a little definition of floating point numbers in c++:
A series of decimal digits, optionally containing a decimal point, optionally preceeded by a sign (+ or -) and optionally followed by the e or E character and a decimal integer (or some of the other sequences supported by strtod). Implementations complying with C99 also support hexadecimal floating-point format when preceded by 0x or 0X.
So "if (sscanf(buf, "%lg%c,%lg%c,%lg%c,%lg%c"..." looks at the data, sees a number followed by E and doesn't recognize that E as a char, because that's how floating point numbers roll. Now I'm extremely puzzled, why this only happened on my system :?

Anyway...for now, I did a quick and dirty fix by just replacing the Es with Os in the booly-switchy-thingy at the beginning. Coordinates like 50N,20O,40N,10O work fine now. Of course, this is by no means practical, but a quick fix so I can start with my tutorial on actually making scenarios. I'm sure, Zydeco will find a more elegant solution some time soon.

[Edit]
Attached a screenshot of a part of the generated map. There are a few obvious problems:
1. Towns are placed at the exact position they should be. That's great, except if there's a tiny bump there, the founding fails. Of course, this is when the sign gets placed, but I think, it would be easier if the 8 surrounding tiles were checked before giving up and placing a sign... possible?
2. Same thing for towns too close too the edges of the map... would it be possible to automatically move those towns just far enough to be foundable?
3. There seems to be a problem with Umlauts and other non-english special characters. L?beck and L?neburg would like their Üs back. Is this a problem with the patch or could it be something about the character encoding in the towns.txt?
Attachments
import_towns_3.1.patch
Patch by Zydeco, slight confusion by moki.
(6.03 KiB) Downloaded 131 times
nordost_v3_test1.png
nordost_v3_test1.png (32.74 KiB) Viewed 4712 times
TinyMusic
Tycoon
Tycoon
Posts: 1422
Joined: 02 Feb 2013 02:03

Re: Creating a REAL map with real geography and cities

Post by TinyMusic »

I tried to use the heightmap and the text used for Portugal and it didn't work; the console said:

Code: Select all

Error reading edge coordinates at towns-pt.txt:1: you're doing it wrong
. I've used the version in the zip made a few posts before this one.
Formerly known as UseYourIllusion.
Java Scenario Found Here - Version 2 out
[tweɪ̂ pû tɕʰì wɔ̀ mǐlɤ lû tɕʰìŋ nì pɑ́ŋmɑ̌ŋ]
moki
Traffic Manager
Traffic Manager
Posts: 131
Joined: 06 Jul 2007 08:48
Location: Wismar, Germany

Re: Creating a REAL map with real geography and cities

Post by moki »

That is strange. I just tried it again (same exact version, fresh installation) and it worked fine. It failed to found 74 towns due to the hilly terrain, but got no error messages. Please follow these steps exactly and try again:
1. make sure that the heighmap and towns-pt.txt are unaltered and copy them to your game directory
2. start game, go into scenario editor
3. load heightmap "portugal.png" and set the size to 512x1024 (rotate counter-clockwise)
4. open the console and type "import_towns towns-pt.txt"
5. ...What does it say now?
Zydeco
Engineer
Engineer
Posts: 11
Joined: 01 Nov 2007 08:57
Skype: maczydeco
Contact:

Re: Creating a REAL map with real geography and cities

Post by Zydeco »

I think I've taken all your suggestions into account and updated it:
  • Coordinates now have no direction suffix, they are positive for north/east and negative for south/west
  • If founding the town at the calculated position fails, it will try the 8 tiles around it
  • File format is documented on the command's help
  • File name searches from OpenTTD's base directory (if you don't enter an absolute path)
  • It better complies with OpenTTD's coding standards
Non-english characters should work if the file encoding is UTF-8.
Attachments
import_towns_v4.patch
import_towns command patch
(5.23 KiB) Downloaded 168 times
towns-pt.txt
Portugal towns updated to new format
(5.1 KiB) Downloaded 168 times
moki
Traffic Manager
Traffic Manager
Posts: 131
Joined: 06 Jul 2007 08:48
Location: Wismar, Germany

Re: Creating a REAL map with real geography and cities

Post by moki »

...and here's the Windows build. There were a few warnings for /src/video/win32_v.cpp (which has nothing to do with the patch), but everything seemed to compile fine anyway - regular play and scenario editor seem to work normally.

Assuming the file formats won't be changed anymore now, I'll start with the final tutorial now. There are a few problems with the data that make it more complicated than previously thought, though. First, the city importance rating is incomplete for many countries (the UK has none at all, France has none except for Paris,... I'll find a way around it anyway). Second, there are many duplicate entries in the files. Most bigger towns are listed several times - once with their current name and then with different historical, local (like gaelic names for Ireland) and international (Warszawa is also Warsaw) naming variants. There's probably a way to filter these out, too, but I'll have to learn a bit more about OpenOffice and spreadsheets first. This may take a while, but I'm very confident that it'll be possible to generate a perfect map in less than a day.

[edit]
Ok, everything seems to work bug-free :)
As another example (for those who are already tired of Portugal), I made a map of Ireland with the gaelic town names. I'm very happy that all 55 towns can be founded on a 1024x1024 map (on 512x512, some towns are a little too close). There were many more problems before Zydeco implemented the check for surrounding tiles. North Ireland and Scotland are currently empty, as I didn't include the data for the UK. It's just a proof of concept so far.
There's still the problem with the á, é, í, ó, ú and special characters in other languages (german, scandinavian languages, some slavic languages, etc.), though. For now I replaced á with a, é with e and so on. Names with these characters are possible in the game, so I think that there's also a way to import them directly.
Attachments
openttd_win_r25868_import_towns_v4.zip
Import Towns patch v4 by Zydeco. Windows build by moki.
(5.83 MiB) Downloaded 181 times
heightmap-ireland.png
Heightmap of Ireland. Best used at 1024x1024.
(234.87 KiB) Downloaded 3 times
towns-ireland_na.txt
Irish towns without special letters. Formatted for the import_towns_v4 patch.
(1.98 KiB) Downloaded 149 times
Eddi
Tycoon
Tycoon
Posts: 8258
Joined: 17 Jan 2007 00:14

Re: Creating a REAL map with real geography and cities

Post by Eddi »

make sure your file is UTF-8 encoded (windows often uses CP 1252 instead)
User avatar
Zuu
OpenTTD Developer
OpenTTD Developer
Posts: 4553
Joined: 09 Jun 2003 18:21
Location: /home/sweden

Re: Creating a REAL map with real geography and cities

Post by Zuu »

If someone provide eg a 30 char long town name, and funding that town fails, the generated sign text will exceed the sign length limit of 31 chars. Mind that with multibyte characters, things will get even more complicated.
My OpenTTD contributions (AIs, Game Scripts, patches, OpenTTD Auto Updater, and some sprites)
Junctioneer (a traffic intersection simulator)
Wahazar
Tycoon
Tycoon
Posts: 1451
Joined: 18 Jan 2014 18:10

Re: Creating a REAL map with real geography and cities

Post by Wahazar »

moki wrote:there are many duplicate entries in the files. Most bigger towns are listed several times - once with their current name and then with different historical, local (like gaelic names for Ireland) and international (Warszawa is also Warsaw) naming variants.
You can filter them out with column [NT] - filtering "N" get rid of all alternative names.
Because [PC] column is usually empty, I'm using [DISPLAY] column - but it also is not quite relevant.

I have other problem - how to adjust these coordinates to height bitmap, which was created from such map projection:
149.156.194.203/~mczapkie/Train/tmp/tt/realmaps/10FIZYCZ.jpg
(probably Gauss-Kruger cylindrical projection)?

EDIT: I need to convert town coordinates to transverse Mercator projection - there is many tools but overcomplicated, any advice?
or opposite, convert bitmap from transverse Mercator to equirectangular projection.
Formerly known as: McZapkie
Projects: Reproducible Map Generation patch, NewGRFs: Manpower industries, PolTrams, Polroad, 600mm narrow gauge, wired, ECS industry extension, V4 CEE train set, HotHut.
Another favorite games: freeciv longturn, OHOL/2HOL.
User avatar
Lordmwa
President
President
Posts: 899
Joined: 20 May 2006 19:30
Location: West Sussex, England

Re: Creating a REAL map with real geography and cities

Post by Lordmwa »

Apologies for the digging but I am creating a 2048x4096 new UK scenario for a project and this patch would be incredibly useful. Unfortunately the latest binaries are before 4096 was supported - Could anyone possibly create a version of this using current trunk? Thanks in advance

Lordmwa
The TT forums trivia tournament! Come along and join in the fun
http://www.funtrivia.com/private/main.cfm?tid=90722
Wahazar
Tycoon
Tycoon
Posts: 1451
Joined: 18 Jan 2014 18:10

Re: Creating a REAL map with real geography and cities

Post by Wahazar »

Lordmwa wrote:Apologies for the digging but I am creating a 2048x4096 new UK scenario for a project and this patch would be incredibly useful. Unfortunately the latest binaries are before 4096 was supported - Could anyone possibly create a version of this using current trunk?
Extended version of this patch, with not only cities but also another objects (industry, water, trees, rocks, signs etc) with 4096x4096 support is here:
http://www.tt-forums.net/viewtopic.php?f=29&t=70846
Formerly known as: McZapkie
Projects: Reproducible Map Generation patch, NewGRFs: Manpower industries, PolTrams, Polroad, 600mm narrow gauge, wired, ECS industry extension, V4 CEE train set, HotHut.
Another favorite games: freeciv longturn, OHOL/2HOL.
User avatar
Lordmwa
President
President
Posts: 899
Joined: 20 May 2006 19:30
Location: West Sussex, England

Re: Creating a REAL map with real geography and cities

Post by Lordmwa »

You my friend are amazing!

Thanks
The TT forums trivia tournament! Come along and join in the fun
http://www.funtrivia.com/private/main.cfm?tid=90722
Post Reply

Return to “General OpenTTD”

Who is online

Users browsing this forum: No registered users and 12 guests