First to £10M Company Value 1 on 1
Posted: 14 Jul 2019 18:20
Back in April 2019 I decided I wanted to make a tournament where all AIs would be facing each other in a 1 on 1. I decided to go with a Round Robin kind of tournament, instead of a Single or Double Elimination one. 48 AIs were selected where each would face all the other 47 AIs in 47 rounds. Each round, 24 matches were played, for a grand total of 1128 matches! I knew I would be taking months to finish the entire tournament, but I was still thrilled to see it through. And so I started!
Each round has a different OpenTTD config, with manual randomized settings, except for a few game settings that I defined as constant, Inflation: Off, OpCodes 10000 and Medium construction speed. Also, since there are 4 different tilesets in OpenTTD, I also decided to split the number of rounds. 12 rounds would be on Temperate, 12 on Sub-Arctic, 12 on Sub-Tropical and 11 on ToyLand.
OpenTTD version used was 1.9.1. Some AIs received updates along the way, and matches I had completed before were repeated with the new version. Some few mistakes were made, and some rounds had to be fully re-done. I also initially had Inflation randomized as on and off, until I realized it would be better to have it off, due to the abnormal time results I was starting to get in some matches.
Very late into the tournmanent, about last week, I noticed that some of the AIs were getting excelent results, but only after their initial bankrupt, so, on their second or even later attempt into the game, with more advanced engines available at the later times. It was a bit too late to re-do everything with bankrupt being reconsidered. In the past, I used to consider a bankrupt as a loss, as a mean to say "you're out of the game". But with all the results I had been logging all this time, it would be a waste to just discard it.
That's then, when I tried to come up with a scoring system which would also take into consideration the fact that a company had finished clean, with no bankrupts against companies that also finish, but only after bankrupting, in the same group of matches in a round, which use the same config. Essentially, since I was already saving the games of every match that finished, all I had to do was to reload all the (at the time) 900+ savegames and check wether the AIs inauguration year were still the same as the starting year setting.
There was still the matter of very long matches, that ended up with a bug in OpenTTD when the idle human company, would suddenly get a huge amount of money after a really really long time, which ended up triggering the GS saying it was victorious. I considered these occurences as undecided, or ties. Neither AI could beat one another in a timely manner. These games were not saved.
The tournament ended today, 4 months after logging and reloading savegames to come up with the best AI that rushes to £10M in value! The winner was AIAI v97!
Extras:
Folder with savegames and configs: https://1drv.ms/u/s!Ah9vX-Q9n7IjjCp_bPR ... A?e=X7rVOa
Excel file with every round results and score: https://1drv.ms/x/s!Ah9vX-Q9n7IjjFbp0PhlJO-CgAlh
OneNote link with every matchup results per round: https://1drv.ms/o/s!Ah9vX-Q9n7IjhHheJyvAuHltAA-5
Other:
Round Robin Generator used: https://www.printyourbrackets.com/round ... erator.php
W-T-L Win, Tie and Loss count
RW Round Wins, counts in how many rounds it had the fastest match, without bankrupting once
bW Bankruptless Wins, how many matches it has won without bankrupting once (RW not included)
BW Bankrupted Wins, how many matches it has won after bankrupting at least once
TO Timed out, how many matches it did not finish before timing out, undecided games, ties
B+ Bankruptless Loss, with Bankrupted Winner, how many matches it lost without bankrupting, where the winner had bankrupted
b+ Bankruptless Loss, with Bankruptless Winner, how many matches it lost without bankrupting, where the winner had not bankrupted as well
B- Bankrupted Loss, with Bankrupted Winner, how many matches it lost and bankrupted, where the winner had bankrupted as well
b- Bankrupted Loss, with Bankruptless Winner, how many matches it lost and bankrupted, where the winner had not bankrupted
Each round has a different OpenTTD config, with manual randomized settings, except for a few game settings that I defined as constant, Inflation: Off, OpCodes 10000 and Medium construction speed. Also, since there are 4 different tilesets in OpenTTD, I also decided to split the number of rounds. 12 rounds would be on Temperate, 12 on Sub-Arctic, 12 on Sub-Tropical and 11 on ToyLand.
OpenTTD version used was 1.9.1. Some AIs received updates along the way, and matches I had completed before were repeated with the new version. Some few mistakes were made, and some rounds had to be fully re-done. I also initially had Inflation randomized as on and off, until I realized it would be better to have it off, due to the abnormal time results I was starting to get in some matches.
Very late into the tournmanent, about last week, I noticed that some of the AIs were getting excelent results, but only after their initial bankrupt, so, on their second or even later attempt into the game, with more advanced engines available at the later times. It was a bit too late to re-do everything with bankrupt being reconsidered. In the past, I used to consider a bankrupt as a loss, as a mean to say "you're out of the game". But with all the results I had been logging all this time, it would be a waste to just discard it.
That's then, when I tried to come up with a scoring system which would also take into consideration the fact that a company had finished clean, with no bankrupts against companies that also finish, but only after bankrupting, in the same group of matches in a round, which use the same config. Essentially, since I was already saving the games of every match that finished, all I had to do was to reload all the (at the time) 900+ savegames and check wether the AIs inauguration year were still the same as the starting year setting.
There was still the matter of very long matches, that ended up with a bug in OpenTTD when the idle human company, would suddenly get a huge amount of money after a really really long time, which ended up triggering the GS saying it was victorious. I considered these occurences as undecided, or ties. Neither AI could beat one another in a timely manner. These games were not saved.
The tournament ended today, 4 months after logging and reloading savegames to come up with the best AI that rushes to £10M in value! The winner was AIAI v97!
Extras:
Folder with savegames and configs: https://1drv.ms/u/s!Ah9vX-Q9n7IjjCp_bPR ... A?e=X7rVOa
Excel file with every round results and score: https://1drv.ms/x/s!Ah9vX-Q9n7IjjFbp0PhlJO-CgAlh
OneNote link with every matchup results per round: https://1drv.ms/o/s!Ah9vX-Q9n7IjhHheJyvAuHltAA-5
Other:
Round Robin Generator used: https://www.printyourbrackets.com/round ... erator.php
Code: Select all
AI Name version Total W-T-L RW bW BW TO B+ b+ B- b-
--------------------------------------------------------------------------
AIAI v97 2407 43-0-4 5 33 5 0 0 3 1 0
NoNoCAB v6 2377 39-0-8 7 31 1 0 0 7 0 1
LuDiAI AfterFix v10 1958 35-4-8 10 19 6 4 0 6 2 0
Terron v209 1955 37-0-10 5 28 4 0 1 9 0 0
MogulAI v5900 1714 36-0-11 3 25 8 0 0 6 1 4
NoCAB v499 1681 33-0-14 1 27 5 0 0 5 3 6
SimpleAI v14 1680 34-3-10 0 28 6 3 0 7 2 1
EmotionAI v1 1630 37-2-8 0 27 10 2 2 5 0 1
AdmiralAI v25 1576 37-1-9 0 24 13 1 0 6 2 1
LuDiAI v1 1569 29-4-14 0 24 5 4 0 8 0 6
CluelessPlus v38 1545 32-0-15 0 29 3 0 1 14 0 0
trAIns v2 1528 32-2-13 7 13 12 2 0 7 3 3
RoadRunner v12 1521 34-0-13 0 28 6 0 1 10 0 2
WormAI v7 1480 30-4-13 2 21 7 4 0 10 1 2
SnakeAI v9 1367 27-3-17 0 21 6 3 0 7 6 4
AroAI v127 1364 27-6-14 1 21 5 6 1 11 0 2
OtviAI v422 1362 31-0-16 0 25 6 0 2 12 1 1
Convoy v11 1319 24-3-20 1 21 2 3 2 14 3 1
ShipAI v5 1263 32-0-15 1 21 10 0 1 12 0 2
ChooChoo v412 1194 25-2-20 0 20 5 2 3 9 5 3
RailwAI v18 1121 30-1-16 0 20 10 1 1 13 1 1
FastPTPAI v4 1102 27-3-17 2 14 11 3 2 8 5 2
DictatorAI v183 1091 28-1-18 0 20 8 1 3 13 2 0
TeshiNet v4 978 23-0-24 0 19 4 0 1 16 3 4
TracAI v1 820 19-6-22 1 11 7 6 1 11 8 2
Trans v190625 703 22-0-25 0 12 10 0 1 14 2 8
BorkAI v23 678 17-8-22 0 14 3 8 3 11 5 3
SynTrans v15 629 12-14-21 0 11 1 14 0 11 4 6
Rondje v384 628 15-7-25 0 9 6 7 2 18 2 3
HeliFerry v3 561 17-8-22 1 6 10 8 0 11 4 7
WrightAI v3 553 11-10-26 0 8 3 10 1 10 9 6
CivilAI v19 487 15-3-29 0 10 5 3 1 11 6 11
DumbAI v1 481 11-8-28 0 7 4 8 2 14 9 3
Chopper v10 437 10-9-28 0 9 1 9 1 14 10 3
EpicTrans v1 433 14-7-26 0 7 7 7 1 6 8 11
PAXLink v14 385 8-14-25 0 5 3 14 0 4 7 14
Denver & Rio Grande v1 351 10-10-27 0 6 4 10 2 7 9 9
PathZilla v6 343 12-5-30 0 4 8 5 4 22 0 4
Grinchios v3 331 9-8-30 0 5 4 8 1 15 4 10
CPU v5 307 8-9-30 0 4 4 9 0 15 4 11
MailAI v21 302 5-10-32 0 5 0 10 4 15 8 5
Rythorn Airline AI v3 276 4-9-34 0 2 2 9 3 16 8 7
gelignAIte v1 270 7-9-31 0 3 4 9 5 21 3 2
rocketAI v1 204 7-9-31 0 2 5 9 3 7 9 12
SmallTownAI v7 151 5-8-34 0 0 5 8 1 13 5 15
RoadAI v5 148 2-13-32 0 0 2 13 2 15 9 6
MpAI v1 143 3-8-36 0 0 3 8 4 15 10 7
WmDOT v14 120 2-11-34 0 0 2 11 1 5 13 15
--------------------------------------------------------------------------
AI Name version Total W-T-L RW bW BW TO B+ b+ B- b-RW Round Wins, counts in how many rounds it had the fastest match, without bankrupting once
bW Bankruptless Wins, how many matches it has won without bankrupting once (RW not included)
BW Bankrupted Wins, how many matches it has won after bankrupting at least once
TO Timed out, how many matches it did not finish before timing out, undecided games, ties
B+ Bankruptless Loss, with Bankrupted Winner, how many matches it lost without bankrupting, where the winner had bankrupted
b+ Bankruptless Loss, with Bankruptless Winner, how many matches it lost without bankrupting, where the winner had not bankrupted as well
B- Bankrupted Loss, with Bankrupted Winner, how many matches it lost and bankrupted, where the winner had bankrupted as well
b- Bankrupted Loss, with Bankruptless Winner, how many matches it lost and bankrupted, where the winner had not bankrupted
