Big AI test game

Discuss the new AI features ("NoAI") introduced into OpenTTD 0.7, allowing you to implement custom AIs, and the new Game Scripts available in OpenTTD 1.2 and higher.

Moderator: OpenTTD Developers

Post Reply
Steffl
Engineer
Engineer
Posts: 103
Joined: 23 Feb 2010 15:44

Big AI test game

Post by Steffl »

I started a AI test game today to see myself which AI is the best.

The AIs are:

Acela Express v1
Admiral AI v25
AIAI delta 3 (v15)
CluelessPlus v22
Convoy v11
Denver & Rio Grande v1
Mogul AI v5898
NoCab v495
OtviAI v312
PAXLink v14
RoadRunner v4
Rondje om de Kerk v313
SimpleAI v4
Trans v100307

The AI's where set to start 1 day after each other. In game it takes longer, but all where started within the first year.
The settings are hard, map size is 512*512 in temperate climate. The advanced settings are my custom settings. If someone thinks an AI has a problem with something he can watch it, I will add it to the attachment.
openttd.cfg
(9.2 KiB) Downloaded 70 times
After 2 years:
Human, 1st Jan 1952.png
(210.49 KiB) Downloaded 2 times
AITest Year 2.sav
(318 KiB) Downloaded 91 times
Rondje, NoCab, AdmiralAI and AIAI had the best start, the others are still thinking what they should do :-)

Steffl
Engineer
Engineer
Posts: 103
Joined: 23 Feb 2010 15:44

Re: Big AI test game

Post by Steffl »

After 5 years:
Human, 1st Jan 1955.png
(200.82 KiB) Downloaded 2 times
Nocab and Rondje are fighting for the first place, AIAI and Admiral fell back a little bit. The slow starting RoadRunner AI is now on place 5. The first victim in this hard world is AcelaExpress. It got already a profitable bus connection, but then it runs out of money. I think it would need a loan management to work better.



After 10 years:
Human, 1st Jan 1960.png
(210.64 KiB) Downloaded 2 times
AITest Year 10.sav
The save game after 10 years
(466.79 KiB) Downloaded 69 times
Rondje now has the best company value, although Nocab has a higher income. Admiral AI seems to have problems, so RoadRunner now is on 4. place.
PAXLink and Acela Express had gone bankrupt, Denver & Rio Grande had built some 90° rail curves, but this isn't allowed in this game.

Steffl
Engineer
Engineer
Posts: 103
Joined: 23 Feb 2010 15:44

Re: Big AI test game

Post by Steffl »

After 20 years:
Human, 1st Jan 1970.png
(218.62 KiB) Downloaded 2 times
graphs without the first 4 companies:
Human, 1st Jan 1970#1.png
(221.65 KiB) Downloaded 2 times
AITest Year 20.sav
(636.49 KiB) Downloaded 63 times

Now AIAI almost overtook Rondje in company value. Rondje sold many vehicles before, so its income decreased. I made a second screenshot to show the companies in the second row. OTVI, AdmiralAI, SimpleAI, Trans and Convoy are making a good profit. Clueless and Mogul AI are fighting to survive but made a little profit too and Acela Express and Denver & Rio Grande got bancrupt short before screenshot.

User avatar
Zuu
OpenTTD Developer
OpenTTD Developer
Posts: 4554
Joined: 09 Jun 2003 18:21
Location: /home/sweden

Re: Big AI test game

Post by Zuu »

Nice work there. I think the profit graph would be more useful than the income graph. In the income graph a company with high costs and high incomes would show up good while it might not make any profit.
My OpenTTD contributions (AIs, Game Scripts, patches, OpenTTD Auto Updater, and some sprites)
Junctioneer (a traffic intersection simulator)

Attila7
Engineer
Engineer
Posts: 37
Joined: 31 May 2010 21:06

Re: Big AI test game

Post by Attila7 »

I would view any test that includes Rondje as a participant as severely skewed if not downright meaningless.

This AI, while cute and a programming accomplishment in its own right, demonstrates some of the things that is wrong with OpenTTD's design and exploits each of these design flaws to the fullest.

Letting this AI loose in a test mostly made up of RV running AIs is like letting a fox loose in the hen-house and see which hen survives the longest. Pointless.

In case you are not aware:

1. Rondje waits for another AI to build a road connection, then uses it to compete with the AI that built the road. This is design flaw #1, as any AI that uses another AI's road should pay a toll of some sort.

2. Rondje builds a vehicle at one end of the route, delivers the cargo and then sells the vehicle at the delivery end. This is repeated for every vehicle on every route. This behavior takes advantage of two design flaws in the game.

2a) Vehicle depreciation is too small, so Rondje gets a lot of money back if the vehicle is sold after one trip. This, of course, saves the time not having to return to pick up cargo. This is a design flaw also because it rewards AIs that overbuy vehicles for a route and just sell them after a year or so instead of properly balancing their needs.

2b) When a vehicle that is less than 3 years old enters a station, the station rating goes much higher than if the vehicle were older than 3 years. The design flaw here is that cargo is divided between competitors based on this rating and 2/3 of the cargo will go to the better rated station even if the difference is just 1%. Rondje always has new vehicles, so it automatically kills its competition by taking most of a route's cargo.

The developers really ought to look at plugging these exploits or all AIs will likely use one form or another, even if not in such a purely exploitative way as Rondje does.
Attila
"Artificial intelligence is no match for natural stupidity."

User avatar
Lord Aro
Tycoon
Tycoon
Posts: 2366
Joined: 25 Jun 2009 16:42
Location: Location, Location
Contact:

Re: Big AI test game

Post by Lord Aro »

That's not entirely fair...
Rondje was created in an attempt to win the TJIP tournament (is that right?)
The goal of this was to achieve the highest company value, by any means necessary. Rondje won the tournament, and so they released the AI for others to admire
The AI had done its job, and so while it may not be the nicest of AIs, it shouldn't be objected to in that way
AroAI - A really feeble attempt at an AI

It is practically impossible to teach good programming to students that have had a prior exposure to BASIC: as potential programmers they are mentally mutilated beyond hope of regeneration. --Edsger Dijkstra

Kogut
Tycoon
Tycoon
Posts: 2493
Joined: 26 Aug 2009 06:33
Location: Poland

Re: Big AI test game

Post by Kogut »

And 2 exploits are fixed
- payment on start of unloading - Rondje was able to sell vehicle that was almost fully loaded with sth and get money for transport
- no time limit for AI initialization - Rondje was designed to preprocess map on loading - what resulted in very long game freezes on ai start.
1) What is killer in fight with train/air ai
2) What is called "one way vehicles" and blocked on part of multiplayer servers
Correct me If I am wrong - PM me if my English is bad
AIAI - AI for OpenTTD

Steffl
Engineer
Engineer
Posts: 103
Joined: 23 Feb 2010 15:44

Re: Big AI test game

Post by Steffl »

If you think no other AI has a chance against Rondje then you will be surprised when you look at the next years.

After 30 years:
Human, 1st Jan 1980.png
(241.7 KiB) Downloaded 2 times
After 40 years:
Human, 1st Jan 1990.png
(248.04 KiB) Downloaded 2 times
And after 50 years:
Human, 1st Jan 2000.png
(250.21 KiB) Downloaded 2 times


Now game is very slow. I think there is a reason why it usually allows only 500 vehicles for a company :-) I never stopped the game and there are only 50 years over. AIAI has alone almost 2000 vehicles, Nocab another 1000. I will run the test till tomorrow and look who is best then, but I think the only way to beat AIAI is, when it would crash or something.
AIAI and Nocab have a big distance to the other AI's. Roundje has still a huge company value, but it doesn't get much higher anymore. The next AI is RoadRunner. It has over 350 vehicles, but I think they could make more profit. The following are Simple AI, Trans, Convoy and Clueless.
Simple AI has 15 trains und 70 road vehicles, Trans has 60 ships and 7 planes(and 24 airports), then Convoy with 167 busses. Convoy seems to have problems now, because its income is sinking and the newest bus it has is already 18 years old. Cluesless Plus has 125 road vehicles. AdmiralAI is the next with 65 road vehicles and 3 planes. I think it has problems with the game settings too, because it has many rail stations but no train. This AI's all playes since start in 1950 and don't got bankrupt. The other 5 got bankrupt and restarted. But no AI crashed because of a coding error.

Steffl
Engineer
Engineer
Posts: 103
Joined: 23 Feb 2010 15:44

Re: Big AI test game

Post by Steffl »

Graphs without the first 2 companies and with operating profit graph:
Human, 1st Jan 2000#3.png
(186.09 KiB) Downloaded 2 times

Savegames:
AITest Year 30.sav
(748.81 KiB) Downloaded 61 times
AITest Year 50.sav
(895.06 KiB) Downloaded 53 times
And sorry that you have to open all images separately. I tried to show it direct in my post, but it didn't work. I think they are to big. Sorry too for my bad English. I learned it in school, but it's not the same as if you speak English naturally. I hope you understand what I ever wanted to say you.
Last edited by Steffl on 02 Sep 2010 19:58, edited 1 time in total.

Attila7
Engineer
Engineer
Posts: 37
Joined: 31 May 2010 21:06

Re: Big AI test game

Post by Attila7 »

The AI had done its job, and so while it may not be the nicest of AIs, it shouldn't be objected to in that way
I don't object to Rondje :)

I object to it being in a test to evaluate a set of AIs. The one Rondje goes after - usually the one that is best at building RV routes - is the one that will perform poorly. How is this test useful? :wink:
Attila
"Artificial intelligence is no match for natural stupidity."

Steffl
Engineer
Engineer
Posts: 103
Joined: 23 Feb 2010 15:44

Re: Big AI test game

Post by Steffl »

Here is the final result after 75 years:
Human, 1st Jan 2025.png
(229.34 KiB) Downloaded 2 times

here without AIAI and Nocab:
Human, 1st Jan 2025#1.png
(230.69 KiB) Downloaded 2 times

and the savegame:
AITest Year 75.sav
(1.01 MiB) Downloaded 61 times


The Winner of this test game is AIAI, the second place goes to NoCab and on third place there's SimpleAI.

Brumi
President
President
Posts: 905
Joined: 18 Jul 2009 17:54

Re: Big AI test game

Post by Brumi »

Again, congratulations to Kogut and AIAI! :D
It's quite surprising for me that SimpleAI did well in such a strong field... And I wonder what happened to AdmiralAI here :?
It would be nice to include the real old AI in these games, to see how much progress we've made since then :)

I took some screenshots from the final savegame:
Human, 20. febr. 2025.png
Human, 20. febr. 2025.png (49 KiB) Viewed 2819 times
TransAI's ships with the original ship pathfinder have trouble getting to their destination, but despite this Trans was doing quite well :)
Human, 2nd Mar 2025.png
(292.75 KiB) Downloaded 2 times
The usual screenshot from these AI games, not the longest queue in the OpenTTD world, but already quite annoying for the drivers...
Human, 9th Mar 2025.png
(254.14 KiB) Downloaded 2 times
Getting a little oldAI-ish, although this was a collaboration of almost all AIs in the game.

And one thing about Rondje:
It's not unbeatable with road vehicles, some time ago I did a test with Rondje against a road vehicle-only NoCAB. They were going head to head in the first years, but later NoCAB took the clear lead and beat Rondje.

User avatar
fanioz
Transport Coordinator
Transport Coordinator
Posts: 320
Joined: 19 Dec 2008 05:03
Location: Indonesia
Contact:

Re: Big AI test game

Post by fanioz »

Steffl wrote:The Winner of this test game is AIAI, the second place goes to NoCab and on third place there's SimpleAI.
Hurray :bow:
Brumi wrote:It's quite surprising for me that SimpleAI did well in such a strong field...
Because it was just 'SimpleAI' ? :lol:
Brumi wrote:TransAI's ships with the original ship pathfinder have trouble getting to their destination, but despite this Trans was doing quite well :)
I've wrote the version that use buoy, but didn't have enough time to test yet. :(


Thanks to Steffl for the tournament run. :D
Correct me If I am wrong - PM me if my English was bad :D

**[OpenTTD AI]** Image
***[NewGRF] *** Image

Post Reply

Return to “OpenTTD AIs and Game Scripts”

Who is online

Users browsing this forum: No registered users and 2 guests