Current AIs' Battle Royale

[Roujin]

Hi Guys,

just for the fun of it, I started a game where all 8 AIs currently available from the content server battled each other. And I thought I'd share the results with everyone...

The contestants:
* AdmiralAI (v19)
* CluelessPlus (v3)
* Convoy (v10)
* Medieval (v2)
* OTVI (v1.4)
* PathZilla (v5)
* PAXLink (v4)
* WrightAI (v2)

A little disclaimer: the results I got here don't necessarily need to tell the real "performance" of the AIs. It's only one map, one specific NewGRF setup, one setup of settings. Some of the AIs may perform better or worse on different maps and/or settings. And some AIs that didn't perform very well in my little game here might show some impressive results when working on their own, rather than being faced with seven opponents.
So please don't take this as a general "ranking" of the AIs we currently have here, and to the developers of the AIs that were outperformed by the others in my game, please don't be discouraged! Please keep developing, it's nice to have a diversity of AIs available


I chose a temperate randomized map of dimension 512x512, with the following NewGRFs (It's basically my standard GRF compilation, but without the 2ccSet, because that from my experience causes trouble for AIs using trains (i.e. AdmiralAI) due to their high costs.)

AI_tournament, GRF list.png (3.91 KiB) Viewed 6860 times

The following posts will contain results from different stages of the game.

[Roujin]

After 5 game years, these are the results:

Map of the game after 5 years:

AI_tournament, 1955-01-01#1.png

(39.53 KiB) Downloaded 537 times

Company comparison after 5 years:

AI_tournament, 1955-01-01.png

(175.96 KiB) Downloaded 512 times

As you can see, within the first five years, basically all towns have been connected. In this phase, the AI which expanded the fastest, took the lead. This was obviously OTVI, having the most road vehicles of all AIs. You can also see from the map that OTVI built a good share of the roads from town to town.
Most of the AIs concentrate on buses (and maybe mail trucks) from town to town, so from now on there's not much to expand on, only improvement of existing services and increasing of the number of vehicles servicing a route. Let's see how good the different AIs handle that in the next post.
The only exceptions to that are AdmiralAI (since it builds also train and aircraft routes, plus it connects industries; thus it still has enough expansion possibilities without any direct competition), WrightAI, and PAXLink (those build aircraft routes).
Unfortunately, the latter two didn't do well. WrightAI, well.. is WrightAI , and PAXLink seems to have trouble when there's much competition, like in my game. It went bankrupt multiple times in my game and got restarted some time after; that's also why it's not present in my screenshots.

[Roujin]

After 10 game years, these are the results:

Map of the game after 10 years:

AI_tournament, 1960-01-10#1.png

(42.7 KiB) Downloaded 408 times

Company comparison after 10 years:

AI_tournament, 1960-01-10.png

(97.03 KiB) Downloaded 466 times

Looking at the Map, you see that not many new routes have been built in the second span of 5 years (as I said in my previous post ). Only AdmiralAI built some new stuff, e.g. rail lines.

Now from the Company comparison (this time I decided to include more different graphs than only the income), we can see that OTVI hasn't expanded all that much, whereas convoy has churned out a fleet of road vehicles and also looking at the graphs (income and delivered cargo), it has reached or exceeded OTVIs level. Regarding income, AdmiralAI has also reached the same level, due to the variety of transportation that only AdmiralAI uses and thus has no competition there.

Clueless and MedievalAI are somewhere in the middle, having a decent amount of vehicles, but not reaching really good profits like the top trio.

PathZilla seems to have trouble due to the heavy competition, as well as PAXLink (who is again bankrupt at the time of these screenshots ).
And WrightAI is ... well... just WrightAI

[Roujin]

Unfortunately I had to abort my game after about 12 years. Saving and later loading the game again didn't work out well since some AIs don't support save/load yet. The game was turning very slow at this point anyways due to (I guess) the high amount of vehicles (I raised the RV limit to 2000 so no AI would be hindered by the maximum) combined with eight (well, seven most of the time) AIs also consuming CPU time.

From year 10 to year 12 there was not much difference, only a bit of fluctuation between the top three AIs.


---
So what do I conclude from this?
- firstly, AdmiralAI, OTVI and Convoy are doing well when facing competition. PAXLink and Pathzilla are probably better when they have more space for themselves.
- OTVI is pretty good on expanding fast, taking the lead early in the game
- Convoy and AdmiralAI reach OTVI later in the game: Convoy probably because it does very well in squeezing the last bit of existing routes, AdmiralAI because it uses means of transportations yet unrivaled by other AIs

Well, there are probably other points that I've forgotten, but when I remember, I'll insert them here...



---
All in all, I thank every AI developer (and the devs involved in the NoAI framwork of course) for making this possible
Cheers, keep developing!

[CommanderZ]

RVs rock

Try to conduct the test on larger map

[GeekToo]

Funny, I'm doing the same at this very moment, to test an improvement on Convoy. I had to increase the number of road vehicles because Admiral and Convoy already reached the default 500.

The poor towns do get busy..

[Zutty]

Oh dear

Was the "aggressive" setting for PathZilla turned on?

[Roujin]

CommanderZ: naah, it gets too hardware hungry with the massive amount of vehicles they build... also, it would only mean that the first phase (enough room to expand freely) will be longer, i.e. OTVI will probably stay in lead longer. Unless of course AdmiralAI reaches him due to using its "exclusive" transportation means.

GeekToo: yes, I set the RV limit up to 2000 so that the AIs don't get hindered by it.

Oh dear

Was the "aggressive" setting for PathZilla turned on?

Yes, I didn't change the default settings for AIs (only the amount of days after which the AI will start).

[planetmaker]

He, a similar idea I had yesterday, too, starting a small game in 1930 with 2cc set, egrvts in arctic climate (1024 x 512 tiles). Here the result after 28 years (the AI Observers GmbH & Co KG is my company):
Diagrammes from top to down:
earnings,
income,
company value and
transported freight

The most successful three AI seem to be AdmiralAI all. AIs colours in the bottome image from top to bottom:
Admiral AI (v19)
Clueless AI 3
Convoy v10
OtviAI v1.4
PAXLink v4
Wright AI v2
OtviAI
OtviAI
PAXLink
Admiral
OtviAI
WrightAI
Convoy
Admiral AI

As some AI failed, the starting dates are quite different. Funnily the latest AdmiralAI is the most successful. And also for the other AIs their inauguration time doesn't relate to their company value at all.

[Roujin]

Interesting.. why do the (road only) companies have so much less RVs than in my game? Did you have a rather low road vehicle limit? Or did you maybe have a low number of towns?
But even if the [road only & towns only] AIs didn't have such an obstacle in your game, I'd have guessed that after 28 years AdmiralAI is in the lead (due to using all kinds of transportation and also servicing industries, with no competition from other AIs).


By the way, next time you should maybe set language to english before making the screenshots so everyone in the forum can understand (no offense meant!)

[Maninthebox]

Very interesting reading Thanks Roujin for the testing, good to see my ai perform well on other computers as well I must say that otviai is mostly test for a 10 year time span and more testing should be done on anything longer (things like replacing vehicles etc).

Also thanks to planetmaker, your test pointed out some flaws in my ai, most notably that I hadn't considered the option that there would not be a suitable bus available at all... so otvi just builds stations for both passengers and mail whereas the first bus and/or mail truck hasn't been invented yet Next release will fix this.

[planetmaker]

Interesting.. why do the (road only) companies have so much less RVs than in my game? Did you have a rather low road vehicle limit? Or did you maybe have a low number of towns?
But even if the [road only & towns only] AIs didn't have such an obstacle in your game, I'd have guessed that after 28 years AdmiralAI is in the lead (due to using all kinds of transportation and also servicing industries, with no competition from other AIs).

Well, the landscape was high in industries but low (or very low, don't remember) on towns - my usual settings in that respect. The vehicle limit was not reached by any AI (nor me), it was set to 500RV.
The problem of AdmiralAI probably is the 2cc trainset which has very expensive engines - which it tried to build... and waited for the money to buy it. Buying two or three of the 2nd best or even 3rd best engines is initially there much more favourable. The later instances started off with other services and were then more successfull. And not all AIs were active 28 years, so... some rather half of it as their predecessors failed

By the way, next time you should maybe set language to english before making the screenshots so everyone in the forum can understand (no offense meant!)

Yeah... But then for most screenshots it doesn't matter and how can I note that there's something for me to translate, if I play English? But you have a point in this case wrt the performance graphs.

[Roujin]

[...]
The problem of AdmiralAI probably is the 2cc trainset which has very expensive engines - which it tried to build... and waited for the money to buy it. Buying two or three of the 2nd best or even 3rd best engines is initially there much more favourable. The later instances started off with other services and were then more successfull. And not all AIs were active 28 years, so... some rather half of it as their predecessors failed
[...]

Yup, that's why I let it run without the 2cc set.. didn't want to play russian roulette with the AdmiralAI - "Build rail first -> fail / Build something else first -> okay" ^_^
Now that I think about it, it probably would've worked also to set the parameter of 2cc so that it has lower costs. Or does that only lower running costs, not purchase costs?

[planetmaker]

Yup, that's why I let it run without the 2cc set.. didn't want to play russian roulette with the AdmiralAI - "Build rail first -> fail / Build something else first -> okay" ^_^
Now that I think about it, it probably would've worked also to set the parameter of 2cc so that it has lower costs. Or does that only lower running costs, not purchase costs?

I set 2cc already to sandbox mode. But in the 1930th the best engine still costs about 450k€ - which is half the maximum loan which I allowed every player.

[Maninthebox]

I did a little battling of AI's as a test of my own AI's and it turns out that apart from my own otviai (after bugfixing ) only clueless is able to handle a start in 1925 correctly. Apparently not so clueless after all Admiral dies and convoy realizes it can't do a thing and stops for eternity. All others seem to try to build bus routes despite the lack of busses.

[Morloth]

I did a little battling of AI's as a test of my own AI's and it turns out that apart from my own otviai (after bugfixing ) only clueless is able to handle a start in 1925 correctly. Apparently not so clueless after all Admiral dies and convoy realizes it can't do a thing and stops for eternity. All others seem to try to build bus routes despite the lack of busses.

Hmmz... I get different results. I assume the latest version of OtviAI isn't on Bananas yet but it breaks instantly (v1). CluelessPlus builds bus connections although it can't buy any busses (not sure what GRFs you're using?) and it seems to me that not building anything like Convoy (and NoCAB! ) is the correct behaviour, since both don't support trains and trams yet.

[Maninthebox]

The latest otvi isn't on bananas yet indeed, I'm testing it before uploading I'm not using any GRF's at the moment. And Convoy and NoCab do the right thing initially, but what I meant is that they never wake up from their hibernation when busses are getting available.

[Morloth]

The latest otvi isn't on bananas yet indeed, I'm testing it before uploading I'm not using any GRF's at the moment. And Convoy and NoCab do the right thing initially, but what I meant is that they never wake up from their hibernation when busses are getting available.

Ah! All right, testing / fixing that right now

Update:
Fixed, my event management system was broken . Thanks for notifying me

[Wasila]

I did a little battling of AI's as a test of my own AI's and it turns out that apart from my own otviai (after bugfixing ) only clueless is able to handle a start in 1925 correctly. Apparently not so clueless after all Admiral dies and convoy realizes it can't do a thing and stops for eternity. All others seem to try to build bus routes despite the lack of busses.

I just started playing against two AdmiralAI's on the CARST World Map starting 1920 and they're both doing about as well as me. (It's about 1925). I'm using the UKRS and the addon pack.

[SaxxonPike]

How do you set up a Battle Royale? Also, is it possible to start a game in "spectator" mode, if any?

I want to experiment how AI does in a small map like 64x64 compared to large maps such as 2048x2048.