Improving OTTD Performance on a multicore processor

[JacobD88]

Hi all,

I've seen so many times on these forums people asking for multi-core support or complaining of performance drops when using some of the more CPU intensive patches, well, as discussed elsewhere, and for good reason, multi-core support is not coming any time soon, if at all

In the meantime however there is a very useful method for getting the most out of OTTD and your multi-core CPU, for me (it may not apply in all cases) I've seen better performance running a single dedicated core for OTTD than allowing it to occupy the Primary Core with all the other windows processes.

You will need a program called imagecfg.exe , which can be found as part of the Windows 2000 or Windows XP Server Development Kit, or here:

Clicky 1
EDIT: ???ABOVE DEAD LINK??? --- NEW ONE ---> Clicky 2

Now, once decompressed from the ZIP file, you need to place the imagecfg.exe file in your windows\system32\ , windows\system32\dllcache\ and windows\system\ folders

Back-up your OpenTTD.exe as you will modify this with ImageCFG

What we are going to do is permanently set OpenTTD to use one core of your multicore processor, this is also known as setting the processor affinity, for performance it's best to use the penultimate core, or 2nd core in dual-core cpu's...

So for example in a dual core CPU you would set the second CPU (CPU ID 1), in a 3-core you would set the 2nd CPU (CPU ID 1), in a 4-core you would set the third CPU (CPU ID 2).

The reason for setting the penultimate core is that, in some multicore cpu's, empty cycles are passed down through the cores so the last core will perform slightly worse than the ones that precede it. The reason for not using the primary core is that windows and most processes and apps will default to this one (though newer versions of the windows kernel are improving how work is distributed between cores). So the penultimate core is generally the best to use as it will have the lowest load and won't suffer as much as the last core if free cycles are passed down

OK, let's begin,

Open notepad and create the following text:

Code: Select all

rem Place ImageCFG.exe in System32 and System32\dllcache
rem CPU number is 0xn , where n is...
rem 1 = CPU0
rem 2 = CPU1
rem 4 = CPU2
rem 8 = CPU3
imagecfg -a 0xn OpenTTD.exe
pause

Note that the second, non "rem'd", 0xn must appear with a number not an "n"

"n" is the core number to use; with a 4-core system this is what you would have to choose from

1 = CPU0 (Primary Core)
2 = CPU1 (Secondary Core)
4 = CPU2 (Third Core)
8 = CPU3 (Fourth Core)

So as we want to use the third core you need to change the above code in the notepad file to this:

Code: Select all

rem Place ImageCFG.exe in System32 and System32\dllcache
rem CPU number is 0xn , where n is...
rem 1 = CPU0
rem 2 = CPU1
rem 4 = CPU2
rem 8 = CPU3
imagecfg -a 0x4 OpenTTD.exe
pause

Note that the second 0xn is now 0x4

Now save the notepad file as a batch file, for example

"openttdcoreset.bat" (Without the "s)

make sure you save it ending .bat

NOT

.bat.txt

(You may have to set "Save as type:" to "All types (*.*)" to get it to save as just a .bat file)

Once you have saved once, copy the .bat file to your OpenTTD core directory, where OpenTTD.exe resides

Open Windows Explorer and navigate to that folder, double click on the openttdcoreset.bat file to run it, you will see it set the processor affinity and pause, once complete you can close the dialogue

You can also use the "Start->Run" method to get the .bat file running too

OpenTTD is now set to use the third core and the third core only resulting in best use of the core, with other cores used to carry the weight of most apps

You need to run the .bat file EVERYTIME you change OpenTTD.exe, this can be updating to the latest version, or compiling a new patch. Fortunately once the .bat file is created you don't have to edit it unless you want to use a different core

comments please

Jacob



2013 Additions:

If you know your HEX then you can also set OTTD to use a range of cores by adding all the "0xn" mask values of each core up and inputting the total as a new "0xn" value for the batch file listed on the thread...
I've not tested this yet... So haven't a clue if it makes any difference to users with 8,12 or 16 core PC's...

For example:

Code: Select all

    0x01 = 1 = CPU0
    0x02 = 2 = CPU1
    0x04 = 4 = CPU2
    0x08 = 8 = CPU3
    0x10 = 16 = CPU4
    0x20 = 32 = CPU5
    0x40 = 64 = CPU6
    0x80 = 128 = CPU7

Therefore you would use, to set OTTD, for the first four cores only in an 8-Core system:

Code: Select all

0x0f

= 15 = CPU0, CPU1, CPU2, and CPU3 (1+2+4+8 = 15)


OR MORE SENSIBLY TO USE THE LAST FOUR CORES OF AN 8-CORE SYSTEM

Code: Select all

0x96

= 96 = CPU4, CPU5, CPU6, and CPU7 (10+20+40+80 = 96)


EDIT1: If this is useful to anyone is it worthy of a sticky? As I'm sick of "Multicore Support Please" type questions
EDIT2: Added an example batch file
EDIT3: April 2013, Amended Dead Link
EDIT4: April 2013, want to spread it accross more than one tertiary core; then you need a scientific calculator for the hex values explained here: Multi-Core Spread Clicky

OpenTTD Processor Affinity Example-CPU1 Processor.bat

Example Batch File to set affinity to the second processor core of any multi-core system

(187 Bytes) Downloaded 193 times

[Rubidium]

it also seems to fix some synch errors in multi-player too

How/why? You only 'force' it to run on a specific core; it doesn't change anything even closely related to desyncs.

Nevertheless... if forcing OpenTTD to run on a specific core improves performance that much you're using a really stupid/lame CPU scheduler. But I can imagine that it might reduce the number of 'jumps' between cores and so improve the performance slightly. Nevertheless I doubt that it is as significant as your topic pretends it to be.

[JacobD88]

it also seems to fix some synch errors in multi-player too

How/why? You only 'force' it to run on a specific core; it doesn't change anything even closely related to desyncs.

Nevertheless... if forcing OpenTTD to run on a specific core improves performance that much you're using a really stupid/lame CPU scheduler. But I can imagine that it might reduce the number of 'jumps' between cores and so improve the performance slightly. Nevertheless I doubt that it is as significant as your topic pretends it to be.

I can only speak from my personal experience, i realise that PC and hardware specs and configurations vary tremendously meaning some people will experience little if any change. I also accept your knowledge of the workings of OTTD is far greater so i cannot argue on subjects of how OTTD uses the processing power it is given

For my example in the following argument however, as i use several machines (Mostly Phenom or PhenomII), i will use the machine where there is the most improvement using this technique; the "quad-core" here is actually comprised of two socket 939 (Not AM2) Athlon X2 4200 dual-core processors on separate 939 sockets (It's an OLLLLD server board)

In this case There are limitations both in the FSB (200MHz) and the memory (DDR - 400MHz) which means that the "jumps" created when switching the core are worsened slightly by the slower bus speeds and physical distance between cores, as compared to more modern multi-core set ups where everything sits nicely on one chip

Forcing the CPU affinity to one core means there are no jumps between cores, and in this case physical processor sockets, the result is a boost in performance with no jumps that do seem to eventually de-synch my LAN games (Even running an OTTD stable release, with no GRFs and default options)

Finally if windows recognises that an app is set to one specific core it appears to work other windows processes around OTTD (as its the application in "focus") preferring to send them to other cores (particularly the primary), leaving the third core pretty much solely occupied by OTTD unless i choose to run another app in the background

Still, if anyone wants to try it for themselves and see if it makes a difference to their game the instructions are now here

And like i said in my FP, it's a technique that makes a difference for intensive patches such as extra height levels and super-sized maps as these require more number-crunching, there is a definite improvement here on all my machines, anyone using the official OTTD nightlies or stables who has performance issues probably has too many GRFs or a bad scheduler as you mentioned

[phil88]

Is this not the same as doing;
Ctrl+Alt+Del
Processes
Right click the ottd exe
Set affinity
Tick whatever core you want it to use

?

[JacobD88]

Is this not the same as doing;
Ctrl+Alt+Del
Processes
Right click the ottd exe
Set affinity
Tick whatever core you want it to use?

Effectively yes, but it sets it permanently (whereas that method only does so per session) by modifying the .exe to use that core until you specify otherwise (or overwrite it by updating or patching)

You no longer have to alt-tab and go to task manager or ctrl-alt-del as you said above and windows will, for the most part, move other processes around OTTD giving it priority on the selected core providing it is in focus

As an OT comment; It's also useful for many other games too, for example earlier Unreal engine games, which suffer really badly from Skippy game-play on multi-cores

[mrMann]

What's wrong with setting up OpenTTD trunk for multi-core processors? Would it cease to run on single core units?

[thepalm]

What's wrong with setting up OpenTTD trunk for multi-core processors? Would it cease to run on single core units?

this has been mentioned before but basically comes down to its really very hard, it would also be slower on single core processors.

[mrMann]

OK, and I'm smart enough to realize that a single-core and multi-core split (two different downloads) is even more unwieldy. And also, it wouldn't be able to be an on/off patch, because you'd have to restart the game, am I right?

[Alberth]

OK, and I'm smart enough to realize that a single-core and multi-core split (two different downloads) is even more unwieldy. And also, it wouldn't be able to be an on/off patch, because you'd have to restart the game, am I right?

OpenTTD will never offer two downloads in that way.

It is extremely unlikely that OpenTTD will ever support more than one CPU core.
The only way to do that would be to drop at least multi-player capabilities.

[SirkoZ]

Thank you, JacobD88, for reporting these findings/experiences. It may well come in handy at least in some situations/configurations.

smileJap.gif (457 Bytes) Viewed 6142 times

[audigex]

The only way to do that would be to drop at least multi-player capabilities.

For the amount of work involved, for what's probably a fairly small performance increase, at the expense of multiplayer - it's not really worth it

Individual cores are getting faster anyway, so it's not like we really lose that much.

[JacobD88]

Thank you, JacobD88, for reporting these findings/experiences. It may well come in handy at least in some situations/configurations.

Your welcome, i've found it useful, and i thought i'd share it with the forums

Individual cores are getting faster anyway, so it's not like we really lose that much.

Agreed


As a side note, until quite recently many older single core CPUs out-performed their multi-core cousins on a core-to-core basis i also found them to be more stable in certain applications. Multi-core is really only useful for number-crunching processes such as physics calculation, folding, video rendering, photographic processing, or high-throughput systems such as servers. OTTD i can not see coming close to ever needing more than one, dedicated, core; hence one of my reasons for posting this, to force it on to one core rather than been bumped around by the system (which, in my example a few posts above meant a physical jump between sockets and CPUs)

[mrMann]

That's true, its for those of us on early, low-end dual-core CPUs like me that occasionally get problems.

[audigex]

True - and the problem started again with early quad cores - the obvious problem being that a 2.4gHz dual core is, by clock speed, slower than a 3gHz P4

On the other hand, Core is a much better architechture than Netburst, so even a lower clocked Core2 is faster than a P4. Eg a 2gHz c2d is probably fairly comparable to a 3gHz P4. It seems counter-intuitive, but Core does more per clock than Netburst. i5/i7 are even better, and their clock speeds are higher anyway.

Now, however, multi-cores have caught back up to the speed of the single core chips, so it works out.