"World ticks" slowing down my save game at some 30 ms

[56independent]

I have recently ran into problems where the performance on my savegame have decreased beyond resonable levels. I'm using the JGR patch on a 2046*16382 world (as measured from the bottom-right co-ordinate.

I am currently playing at 0.50* actual game speed. I have had a look at the framerate breakdown:

Captura desde 2024-01-16 21-24-39.png

(357.24 KiB) Not downloaded yet

As we can see, "World Ticks" have entered the red zone, which has likely pushed the framerate and actual speed of the game down to the best of my guesses. However, my main problem is that i don't know how to resolve this problem. I have only a vauge idea of what "World ticks" might mean; the things changing in this frankly, massive world.

I'm playing on a standard Acer Spin SP114-31N, with 4gb of memory, 128 gb of storage space (of which 1.9 is free), and an Intel® Celeron(R) N5100 @ 1.10GHz × 4. It's running Ubuntu 22.04.3 LTS

This world, understandibly, is much larger then OpenTTD was initally designed for. And although in many FOSS communities fan-made nonstandard forks are not very supported, i'm just wondering if there is any general knowledge that can help against the world ticks overuse.

I'm willing to provide the savefiles even if the forums aren't (due to a size lmit blocking my 42 mb file). Here are the general settings of my OpenTTD installation, though:

openttd.cfg

(70.7 KiB) Downloaded 24 times

It'd be great to play at 1* and even some 25* with the fast-forward button pressed.

PS: Is there any way to improve the colour scheme of these forums? I find black on dark grey rather illegible and unconducive to my frequent research efforts.

[Redirect Left]

If I am being honest, I am surprised you got this far before it slowed down on such a weak processor as that celeron. Try disabling water animation, if not already, that may help to a tiny degree. The same for Full Detail & Full Animation from the settings dropdown, rather than the settings box where water animation is stored.

I think at this point your map is simply too big, and you've now got far more units to keep track of and you processor is simply crying. OpenTTD is a single thread game (+1 thread if you're using multiplayer), which no isn't a simple thing to fix.

Even on my i9-12900k, on a 2048 x 2048 map, with several thousand units, the game begins to slip in turns of game speed, the cost of pathfinding is simply a technological thing that has probably been optimised nearly all it can within OpenTTD already. Just makes me wonder how computers in the late 90s handled Transport Tycoon and other games with plenty of units to pathfind, such as Command & Conquer.

[56independent]

If I am being honest, I am surprised you got this far before it slowed down on such a weak processor as that celeron. Try disabling water animation, if not already, that may help to a tiny degree. The same for Full Detail & Full Animation from the settings dropdown, rather than the settings box where water animation is stored.

I think at this point your map is simply too big, and you've now got far more units to keep track of and you processor is simply crying. OpenTTD is a single thread game (+1 thread if you're using multiplayer), which no isn't a simple thing to fix.

I'd like to note that the ticks used for "trains" "roads" and "graphics" are all well within the blue range. The only ticks in the red range are the "world" ticks, which happen to be in the red range; see the attached screenshot. Assuming the ticks breakdown were built with non-OpenTTD-developers in mind, this seems to indicate something's wrong with the World itself being so large the various untouched industries and towns (of which there are 3 thousand) take up so much space on the mainloop they waste memory doing things when they

These issues did not exist eariler, when i was using much slower trains on a smaller network, so maybe your theory has some salt. I mean, i do have a very heavily developed tram network with some 820 vehicles, and a very busy network of some 140 trains adding to it.

[peter1138]

The game has to process every tile on the map even when you're not near it and not looking at it.

2048x16384 is massive even for a decent spec desktop, so it's not really surprising that a 1.1GHz Celeron laptop is struggling.

The solution is to use a smaller map.

[56independent]

The solution is to use a smaller map.

Maybe in the short-term it works. I'll just suck it up and accept that the fast-forward button dosen't work and that the trains often go at half their posted speed and that i'm doomed to play in maps that run out of fun in just a week.

However, i feel there are places, like in any large (and more then a decade old!) codebase, where optimisations could be made. And, speaking as a programmer, there's probably quite a few legacy warts sitting there, somehwere. My codebases often have a lot, even when they're only a few hours old! And anyways i've always wanted to contribute meaningfullt¡y to the FOSS project. Thanks for the idea!

[Redirect Left]

Yeah, I think your problem here is the giant world, vs very tiny and weak 1 core* CPU. I'm pretty confident a map of those dimensions would be a challenge on even modern processors when its only got the resources of a single core/thread to use.
*its more than 1, but openttd is only using one.

I mean, i do have a very heavily developed tram network with some 820 vehicles, and a very busy network of some 140 trains adding to it.

Not so much an issue here, but I would stick to tracked vehicles on your weak Celeron. Trains can only go in the direction of their track, which most of the time is a straight line, meanwhile boats can go in any direction at every tile. Road vehicles are probably going to come across junctions way more often with a choice. Although based on that I figure trams will be the same CPU usage as trains, being tracked like a train. Keep the pathfinding easy.

Be warned, this is based off only knowledge I have done for pathfinding my own games, its possible OpenTTD's pathfinder is different or uses similar resources regardless of options of vehicles coming up. I'm happy to be corrected if so.

[jfs]

A map that size has 33.5 MILLION tiles in total, and all those tiles have to be processed 34 times per second for the game to run at full speed.
With a 1.1 GHz CPU, that leaves about 33 CPU cycles to process each map tile each tick, if it were to run at full speed. And that's without adding in the handling of vehicles, industries, and so on. And with the memory latencies involved in modern computer architectures, those 33 cycles would be barely enough to even fetch the required data from RAM.

The reason it's not running at just 1 fps is actually that the game is only processing 1 of each 256 tiles each game tick, so actually just under 8400 CPU cycles per tile, still with all the above concessions. The way to "optimize" this would involve removing features from the game, such as completely cutting away any and all modding support. Gigantic maps like this just require more of the hardware than is possible with the current technology, the numbers simply don't add up.

(Yes, I'm extremely over-simplifying some things in this, but I'd like to think it's not so much as to detract from my core argument.)

[56independent]

Not so much an issue here, but I would stick to tracked vehicles on your weak Celeron.

Be warned, this is based off only knowledge I have done for pathfinding my own games, its possible OpenTTD's pathfinder is different or uses similar resources regardless of options of vehicles coming up. I'm happy to be corrected if so.

If i was instructed with designing a performant pathfinding algorithm, what i'd end up with is likely a much simpler planes and ships pathfinder; planes are easy; find the two airports, find two lines between them such that one is diagonal and the other straight, use some basic trigonometry to find the shortest route, and then add a little bit of offset for any required "alignment patterns" such as those you see. Ships, also, are quite easy; do the same as planes but add a few more nodes and lines to get around obstructions, such as land.

In fact, vehicles based on rails would likely be the most complicated algorithm in the game. One must consider the actual track layout, directionality, junction navigation, and other things. The best i can think of is the Dijkstra A* algorithm, but that'd require taking the entire network, or at least a subset encompassing a rectangular or circular area with some 50 tiles of margin between the outer edge of the desired stations, and making a little node-and-line chart of it.

I'm pretty sure the OpenTTD developers are smart enough to implement even simpler, performanter pathfinding algorithms. They've certainly had enough time to refactor it. Remember, the vehicle ticks in the provided screenshot are all within the blue range; it's the world ticks which suffer. I am pretty sure, at least now, that this is indicative of the size of the map and its "NPC-type" changes being so numerous that the processor simply can't handle it [1].

The way to "optimize" this would involve removing features from the game, such as completely cutting away any and all modding support. Gigantic maps like this just require more of the hardware than is possible with the current technology, the numbers simply don't add up.

I'm sure there are at least a few things we can do to optimise things, such as having industries in the middle of nowhere, maybe some thousand tiles away from the player, be under the "frozen" economy setting due to their irrelevance to the area in which the actual gameplay is taking place.

I'm pretty sure the mainloop is somewhere around here, so i'll see if i can find it and read through it. I don't think i'll find much, seeing the high apparent quality of the codebase.

In any case, i'm getting used to the slower pace of life on this map.

[Argus]

I am quite surprised how a normally large map - let's say 2k*2k - can stop entertaining someone after a few weeks, when I am not able to connect a tenth of the cities present on it during that time.
Of course I don't play with vanilla graphics. In my experience lately - because of the animations - ECS is making a problem, or you can save a little if no industry is generated during the game. I already have problems on 8k maps myself. With ECS even at 4k.

[2TallTyler]

Pull Requests are always welcome. You mention the pathfinder -- we just merged a new ship pathfinder written by a contributor, which will be released in 14.0.

Freezing industries that are far from any player-built infrastructure is an idea I've seen mentioned before, not for performance reasons but to make future planning easier. This might be a good place to start.

[jfs]

If i was instructed with designing a performant pathfinding algorithm, what i'd end up with is likely a much simpler planes and ships pathfinder; planes are easy; find the two airports, find two lines between them such that one is diagonal and the other straight, use some basic trigonometry to find the shortest route, and then add a little bit of offset for any required "alignment patterns" such as those you see. Ships, also, are quite easy; do the same as planes but add a few more nodes and lines to get around obstructions, such as land.

For airplanes, that is in fact how they "pathfind".

Ships, however, are surprisingly difficult. The main reason is that they have to consider every water tile as passable in every possible direction. Imagine paving a huge space of land with rail pieces in all six positions on every tile, that's how ship pathfinding has traditionally treated the problem.Recently, a new approach to ship pathfinding was merged into the master branch, called region-based pathfinding, which is much more efficient, but (last I checked) still has a couple bugs.

Road and rail are generally easier, since they tend to have "sensible" networks without excessively many nodes, although they can still be quite costly to pathfind on. A few years ago we added some caching of previously found routes, which improved performance a good deal, but also caused some occasionally unexpected behaviour around road vehicle stations.

[56independent]

Pull Requests are always welcome.

Freezing industries might be a good place to start.

I do want to contribute to FOSS quite a lot. To me, FOSS is the software manifestation of the belief that knowledge should be free, and given that i do believe in liberating knowledge, i believe in the values of FOSS. However, right now, the only things i can really do with any reliability is edit my own code or suggest ideas, as i am not very good at navigating foreign codebases. Given a method to learn this skill and examples of tools that can help me with this skill, i'd be willing to contribute to the OpenTTD codebase. I have reached a level in my programming journey such that given learning resources and time, i could write in any programming language.

Maybe, in the long term, creating a deterministic procedural generation system for OpenTTD such that a world can simply be "TBD*TBD", where "TBD" means "To be determined" and panning reveals new, dynamically generated, "chunks" of the map, which are, if left untouched, are removed after restarting the game. Such a system would mess with the co-ordinate system, though, which seems to go from 0*0 at the top-left to the map size at the bottom-right, ±1. It seems co-ordinates need to be all-positive based off this assumption, which happens to align with the way C++ handles numeric types, where simply adding "signed" halves the amount of positive values available in lieu of negative ones.

The way i just suggested is how Minetest, a game i also love, does this. It has a set 31,000*31,000 grid of "nodes" (equivalent to tiles, but if they were cubic and could be atop each other, IIRC). At the exact centre is 0,0,0, and out from there, procedurally generated, are the various "mapblocks" of 16*16*16. There is a problem in that the generated mapblocks will be saved even if left unmodified; this is why the solution i chose for OpenTTD should be deterministic if such algorithms exist (and if not, one might as well invent one, say, me).

As well as freezing industries, i think freezing everything would be the best; trees shouldn't change if nobody's going to see them change from the window of a train, for example. Simply not processing those tiles might be the best solution; unless they're within x number of tiles to a tile owned by any company or they're within the viewport, just ignore them. That reduces the map size to only the built area+some more. For these purposes, any town or industry served by a company also counts as "company property", too.

Ships, however, are surprisingly difficult. The main reason is that they have to consider every water tile as passable in every possible direction. Imagine paving a huge space of land with rail pieces in all six positions on every tile, that's how ship pathfinding has traditionally treated the problem.Recently, a new approach to ship pathfinding was merged into the master branch, called region-based pathfinding, which is much more efficient, but (last I checked) still has a couple bugs.

The algorithm is an intresting approach. If i, however, was tasked with designing a ship algorithm, it'd be like this:

1. Draw a straight line between the two destinations
2. Snap it to the two-lines approach
3. Iterate over the resultant land
4. For each node that is land, move laterally towards water, as found in a (new (?)) [icode]nearest_water_to(coords, axisType)[/icode] function, taking in either "x" or "y" for the axisType, and a co-ordinate for the coords.
5. Continue to the next item on the line

Which i feel might be faster still, given the smaller area to check.