Road Vehicles AI Competition

Discuss the new AI features ("NoAI") introduced into OpenTTD 0.7, allowing you to implement custom AIs, and the new Game Scripts available in OpenTTD 1.2 and higher.

Moderator: OpenTTD Developers

Post Reply
xarick
Transport Coordinator
Transport Coordinator
Posts: 335
Joined: 26 Feb 2015 00:52

Road Vehicles AI Competition

Post by xarick »

Testing Road Vehicle AIs from time to time. They were all competing alone, some of the 1.6.1-RC1 tests were done in an online multiplayer environment, with very few human players joining some of the servers The others were tested without humans. The map tileset, its seed and game settings were exactly equal to all of them. I focused on easy settings, forbid the other type of vehicles, and allowed 5000 total road vehicles, 250k #ops, very fast competitor speed and hard settings profile.

Here's a summary of who they are and the parameters I used and if they reached 1st Jan 2051 without bankrupting or crashing (please expand):
[+] Spoiler

Code: Select all

(v25)    AdmiralAI = build_statues=0,start_date=1,use_planes=0,use_trains=0                                                                                - finished   (1.6.1-RC1)
(v95)         AIAI = PAX_plane=0,clear_signs=1,hide_contact_information=1,start_date=1,use_freight_trains=0                                                - finished   (1.6.1-RC1)
(v97)         AIAI = PAX_plane=0,clear_signs=1,hide_contact_information=1,start_date=1,use_freight_trains=0                                                - finished   (1.6.1)
(v127)       AroAI = start_date=1                                                                                                                          - finished   (1.6.1-RC1)
(v23)       BorkAI = enable_aircrafts=0,start_date=1                                                                                                       - crashed    (1.6.1-RC1?)
(v409)    ChooChoo = ActivitySigns=0,JunctionNames=0,PathfinderMultiplier=4,start_date=1                                                                   - bankrupted (1.6.1-RC1?)
(v412)    ChooChoo = ActivitySigns=0,JunctionNames=0,PathfinderMultiplier=4,start_date=1                                                                   - bankrupted (1.7.2)
(v6)       CivilAI = MaxBus=500,start_date=1                                                                                                               - finished   (1.6.1-RC1)
(v37) CluelessPlus = allow_competition=0,connection_types=2,scp_enabled=0,start_date=1                                                                     - finished   (1.6.1-RC1)
(v38) CluelessPlus = allow_competition=0,connection_types=2,scp_enabled=0,start_date=1                                                                     - finished   (1.7.2)
(v11)       Convoy = Agressive=0,start_date=1                                                                                                              - finished   (1.6.1-RC1)
(v5)           CPU = Build_Debug_signs=0,Min_city_size=50,Veicle_number_multiplyer=10,competivity=99,start_date=1                                          - finished   (1.6.1-RC1)
(v183)  DictatorAI = allow_scp=0,fairlevel=1,start_date=1,station_balance=1,upgrade_townbridge=1,use_air=0,use_boat=0,use_train=0                          - finished   (1.6.1-RC1)
(v1)     EpicTrans = start_date=1                                                                                                                          - bankrupted (1.6.1-RC1?)
(v1)    gelignAIte = start_date=1                                                                                                                          - finished   (1.6.1-RC1)
(v1)        LuDiAI = air_support=0,start_date=1                                                                                                            - crashed    (1.7.2)
(v21)       MailAI = level_of_easiness=5,show_debug=0,start_date=1,use_ISR=0,use_art_trucks=0,use_trains=0                                                 - finished   (1.6.1-RC1)
(v5900)    MogulAI = start_date=1                                                                                                                          - finished   (1.6.1-RC1)
(v1)          MpAI = start_date=1                                                                                                                          - crashed    (1.7.2)
(v499)       NoCAB = Enable airplanes=0,Enable ships=0,Enable trains=0,Politics Setting=1,start_date=1                                                     - finished   (1.6.1-RC1)
(v3)       NoNoCAB = Enable airplanes=0,Enable ships=0,Enable trains=0,Politics Setting=1,start_date=1                                                     - finished   (1.6.1)
(v5)       NoNoCAB = Enable airplanes=0,Enable ships=0,Enable trains=0,Politics Setting=1,start_date=1                                                     - finished   (1.7.2)
(v418)      OtviAI = start_date=1                                                                                                                          - finished   (1.6.1-RC1)
(v6)     PathZilla = aggressive=0,country_lanes=0,rt_cargo_towns=0,start_date=1                                                                            - finished   (1.6.1-RC1)
(v14)      PAXLink = avoid_town_road_grid=0,clear_signs=1,quit_on_api_check_failure=0,start_date=1                                                         - finished   (1.6.1-RC1)
(v3)        RoadAI = max_networks=10,start_date=1                                                                                                          - finished   (1.6.1-RC1)
(v4)        RoadAI = max_networks=10,start_date=1                                                                                                          - finished   (1.6.1)
(v9)    RoadRunner = hq=1,rt_cargo_towns=0,start_date=182                                                                                                  - finished   (1.6.1-RC1)
(v10)   RoadRunner = hq=1,rt_cargo_towns=0,start_date=1                                                                                                    - finished   (1.6.1-RC1)
(v12)   RoadRunner = hq=1,rt_cargo_towns=0,start_date=1                                                                                                    - crashed    (1.7.2)
(v1)      rocketAI = start_date=1                                                                                                                          - crashed    (1.6.1-RC1?)
(v384)      Rondje = start_date=1                                                                                                                          - bankrupted (1.6.1-RC1?)
(v10)     SimpleAI = hq_in_town=1,max_transported=60,newgrf_stations=0,start_date=1,subsidy_chance=10,use_aircraft=0,use_custom_companyname=1,use_trains=0 - finished   (1.6.1-RC1)
(v12)     SimpleAI = hq_in_town=1,newgrf_stations=0,start_date=1,subsidy_chance=10,use_aircraft=0,use_custom_companyname=1,use_trains=0                    - finished   (1.7.2)
(v4)       SnakeAI = start_date=1                                                                                                                          - finished   (1.7.2)
(v15)     SynTrans = start_date=1                                                                                                                          - bankrupted (1.6.1-RC1?)
(v209)      Terron = PlaneModule=0,ShipModule=0,TruckModule=1,fund_town_buildings=0,max_road_route_length=130,start_date=1                                 - finished   (1.6.1-RC1)
(v4)      TeshiNet = start_date=1,use_planes=0                                                                                                             - crashed    (1.6.1-RC1?)
(v130722)    Trans = Air Vehicle=0,Rail Vehicle=0,Water Vehicle=0,start_date=182                                                                           - finished   (1.6.1-RC1)
A short summary about those who bankrupted:
ChooChoo v409, 1.6.1-RC1? - It's not suitable for road vehicles, it only uses them to help train routes, but since trains were not allowed, the small number of buses could not sustain the vast amount of rail infrastructures that has been built. It was expected to fail, but I still used it because it had road vehicles.
ChooChoo v412, 1.7.2 - Same as v409.
EpicTrans v1, 1.6.1-RC1? - It was working very close to the loan limit with profitable vehicles, even with a larger profit. It spends way too much on construction for some reason. viewtopic.php?p=1170913#p1170913
Rondje v384, 1.6.1-RC1? - It did not build any vehicle. It could survive a bit longer had it managed loan a bit better. But in the end, it was dependant on a competing human player, but none has joined the server.
SynTrans v15, 1.6.1-RC1? - It was working very close to the loan limit with profitable vehicles, it got a few bankrupt warnings and suddenly disappeared. viewtopic.php?p=1170380#p1170380

A short summary about those who crashed:
BorkAI v23, 1.6.1-RC1? - It was doing fine, but it crashed because it must have reached a limitation of NoAI, its amount of memory usage was huge. viewtopic.php?p=1170586#p1170586
LuDiAI v1, 1.7.2 - viewtopic.php?p=1202962#p1202962
MpAI v1, 1.7.2 - Could not start first route, aircraft based, and decided not to continue.
RoadRunner v12, 1.7.2 - viewtopic.php?p=1202999#p1202999
rocketAI v1, 1.6.1-RC1? - It is always set to stop the script once it finishes building. Expected.
Teshinet v4, 1.6.1-RC1? - Your script made an error: the index 'station_id' does not exist. viewtopic.php?p=1170543#p1170543

A few AIs could not do autosaves due to taking too long to save. I still tested them in a password protected server, and only made one save in the entire session, on the last day, 1st Jan 2051. Otherwise, they would also crash. They were:
NoCAB v499, 1.6.1-RC1
PathZilla v6, 1.6.1-RC1
Trans v130722, 1.6.1-RC1

Here's a summary of results of those who reached 2051:

Code: Select all

AI name  (version);  company value;     profit rvs %;    served sts  %; profit ly rv;    avg/best rv    %; pieces rd; eff rd; OTTD ver

AdmiralAI    (v25); £1,723,079,300; 4,891/4,999  98%;   584/  606  96%; +£27,613,624; £5,524/£13,681  40%;    41,754;    661; 1.6.1-RC1
AIAI         (v95); £1,780,553,625; 4,895/5,000  98%;   354/  358  99%; +£27,220,845; £5,444/£17,139  32%;    32,064;    849; 1.6.1-RC1
AIAI         (v97); £1,228,513,923; 4,573/4,671  98%;   332/  364  91%; +£15,237,475; £3,262/£15,434  21%;    32,980;    462; 1.6.1
AroAI       (v127);   £911,720,618; 2,577/2,610  99%;   324/  376  86%; +£12,376,741; £4,742/£11,050  43%;    34,601;    358; 1.6.1-RC1
CivilAI       (v6);   £136,648,384; 1,589/1,626  98%;   176/  143 124%;  +£4,224,010; £2,598/£13,367  19%;    19,741;    214; 1.6.1-RC1
CluelessPlus (v37); £1,957,302,832; 4,802/4,999  96%;   356/  356 100%; +£35,378,141; £7,077/£15,915  44%;    39,837;    888; 1.6.1-RC1
CluelessPlus (v38); £1,905,081,538; 4,838/4,999  97%;   375/  376 100%; +£35,355,369; £7,072/£19,554  36%;    34,398;  1,028; 1.7.2
Convoy       (v11);   £265,391,074;   501/  530  95%;    42/   46  91%;  +£2,876,432; £5,427/ £7,374  74%;     4,776;    602; 1.6.1-RC1
CPU           (v5);    £33,580,537;   275/  316  87%;    82/  115  71%;  +£1,034,389; £3,273/£12,707  26%;     3,060;    338; 1.6.1-RC1
DictatorAI  (v183);   £587,006,789; 1,179/1,187  99%;   143/  143 100%; +£10,156,890; £8,557/£13,851  62%;    12,860;    790; 1.6.1-RC1
gelignAIte    (v1);     £2,235,425;     6/    6 100%;     4/    2 200%;     +£47,374; £7,896/ £9,585  82%;         4; 11,844; 1.6.1-RC1
MailAI       (v21);    £81,310,678;   309/  311  99%;   245/  262  94%;  +£2,080,837; £6,691/£14,075  48%;    15,545;    134; 1.6.1-RC1
MogulAI    (v5900); £2,007,985,370; 3,590/5,000  72%;   122/  130  94%; +£27,741,577; £5,548/£15,245  36%;    62,064;    447; 1.6.1-RC1
NoCAB       (v499); £2,446,436,283; 4,915/5,000  98%; 1,022/2,145  48%; +£37,082,544; £7,417/£18,850  39%;   392,145;     95; 1.6.1-RC1
NoNoCAB       (v3); £2,030,932,883;     0/4,996   0%;     0/  959   0%;    -£191,240;   -£38/     £0   0%;   163,846;      0; 1.6.1
NoNoCAB       (v5); £2,497,031,570; 4,952/5,000  99%; 1,167/1,364  86%; +£43,678,223; £8,736/£19,836  44%;   153,380;    285; 1.7.2
OtviAI      (v418); £1,479,789,114; 4,996/5,000 100%;   717/  830  86%; +£18,983,433; £3,797/£14,173  27%;    85,164;    223; 1.6.1-RC1
PathZilla     (v6);   £128,607,538;   803/  890  90%;   160/  197  81%;  +£1,690,529; £1,899/£12,894  15%;     7,627;    222; 1.6.1-RC1
PAXLink      (v14);       £777,713;    32/   34  94%;    14/   21  67%;     +£67,067; £1,973/ £2,715  73%;       402;    167; 1.6.1-RC1
RoadAI        (v3);   £714,991,238; 3,799/5,000  76%;   332/  340  98%; +£15,064,918; £3,013/£15,512  19%;    52,805;    285; 1.6.1-RC1
RoadAI        (v4);   £747,601,829; 3,871/5,000  77%;   337/  344  98%; +£14,287,246; £2,857/£15,209  19%;    55,080;    259; 1.6.1
RoadRunner    (v9); £1,385,048,406; 4,735/5,000  95%;   363/  750  48%; +£21,578,435; £4,316/£20,027  22%;   102,467;    211; 1.6.1-RC1
RoadRunner   (v10); £1,372,994,916; 4,940/4,999  99%;   368/  721  51%; +£20,868,112; £4,174/£17,219  24%;    99,386;    210; 1.6.1
SimpleAI     (v10); £1,254,874,888; 4,977/5,000 100%;   793/  796 100%; +£22,338,585; £4,468/£17,595  25%;    67,687;    330; 1.6.1-RC1
SimpleAI     (v12); £1,246,346,354; 4,944/5,000  99%;   775/  780  99%; +£22,522,521; £4,505/£18,896  24%;    69,429;    324; 1.7.2
SnakeAI       (v4);   £519,555,348; 2,495/2,506 100%;   337/  373  90%;  +£5,469,354; £2,183/ £4,760  46%;    32,953;    166; 1.7.2
Terron      (v209); £1,390,486,908; 2,420/2,456  99%;   311/  176 177%; +£23,996,356; £9,771/£24,186  40%;    14,038;  1,709; 1.6.1-RC1
Trans    (v130722);    £13,221,877;    11/   45  24%;     4/    7  57%;     +£79,547; £1,768/£11,699  15%;     3,788;     21; 1.6.1-RC1
Screenshots
[+] Spoiler
AdmiralAI v25, 1.6.1-RC1:
Image
Image

AIAI v95, 1.6.1-RC1:
Image
Image

AIAI v97, 1.6.1:
Image
Image

AroAI v127, 1.6.1-RC1:
Image
Image

CivilAI v6, 1.6.1-RC1:
Image
Image

CluelessPlus v37, 1.6.1-RC1:
Image
Image

CluelessPlus v38, 1.7.2:
Image
Image

Convoy v11, 1.6.1-RC1:
Image
Image

CPU v5, 1.6.1-RC1:
Image
Image

DictatorAI v183, 1.6.1-RC1:
Image
Image

gelignAIte v1, 1.6.1-RC1:
Image
Image

MailAI v21, 1.6.1-RC1:
Image
Image

MogulAI v5900, 1.6.1-RC1:
Image
Image

NoCAB v499, 1.6.1-RC1:
Image
Image

NoNoCAB v3, 1.6.1:
Image
Image

NoNoCAB v5, 1.7.2:
Image
Image

OtviAI v418, 1.6.1-RC1:
Image
Image

PathZilla v6, 1.6.1-RC1:
Image
Image

PAXLink v14, 1.6.1-RC1:
Image
Image

RoadAI v3, 1.6.1-RC1:
Image
Image

RoadAI v4, 1.6.1:
Image
Image

RoadRunner v9, 1.6.1-RC1:
Image
Image

RoadRunner v10, 1.6.1:
Image
Image

SimpleAI v10, 1.6.1-RC1:
Image
Image

SimpleAI v12, 1.7.2:
Image
Image

SnakeAI v4, 1.7.2:
Image
Image

Terron v209, 1.6.1-RC1:
Image
Image

Trans v130722, 1.6.1-RC1:
Image
Image
EDIT: Added NoNoCAB v3
EDIT2: missing NoNoCAB v3 screenshots
EDIT3: Added NoNoCAB v3 screenshots
EDIT4: Added AIAI v97, RoadAI v4, RoadRunner v10, updated score presentation, updated parameter presentation
EDIT5: Updated savegames, added screenshots for AIAI v97, RoadAI v4, RoadRunner v10
EDIT6: Added 1.7.2 tests: ChooChoo v412, CluelessPlus v38, LuDiAI v1, MpAI v1, NoNoCAB v5, RoadRunner v12, SimpleAI v12, SnakeAI v4. Updated savegame names, result summary and parameter summary with the OpenTTD version used for testing.
EDIT7: Added screenshots: CluelessPlus v38, NoNoCAB v5, SimpleAI v12, SnakeAI v4.
Attachments
openttdCore1.cfg
Typical configuration file with the settings used
(10.74 KiB) Downloaded 186 times
Road Vehicles AI Competition Part 1.zip
Savegames for AdmiralAI v25, 1.6.1-RC1, AIAI v95, 1.6.1-RC1, AIAI v97, 1.6.1, AroAI v127, 1.6.1-RC1, CivilAI v6, 1.6.1-RC1, CluelessPlus v37, 1.6.1-RC1, CluelessPlus v38, 1.7.2, Convoy v11, 1.6.1-RC1, CPU v5, 1.6.1-RC1, DictatorAI v183, 1.6.1-RC1, gelignAIte v1, 1.6.1-RC1, MailAI v21, 1.6.1-RC1, MogulAI v5900, 1.6.1-RC1, NoCAB v499, 1.6.1-RC1, NoNoCAB v3, 1.6.1
(22.79 MiB) Downloaded 158 times
Road Vehicles AI Competition Part 2.zip
Savegames for NoNoCAB v5, 1.7.2, OtviAI v418, 1.6.1-RC1, PathZilla v6, 1.6.1-RC1, PAXLink v14, 1.6.1-RC1, RoadAI v3, 1.6.1-RC1, RoadAI v4, 1.6.1, RoadRunner v9, 1.6.1-RC1, RoadRunner v10, 1.6.1, SimpleAI v10, 1.6.1-RC1, SimpleAI v12, 1.7.2, SnakeAI v4, 1.7.2, Terron v209, 1.6.1-RC1, Trans v130722, 1.6.1-RC1
(21.99 MiB) Downloaded 149 times
Last edited by xarick on 24 Feb 2018 11:41, edited 33 times in total.
Formerly known as Samu
Brumi
President
President
Posts: 920
Joined: 18 Jul 2009 17:54

Re: Road Vehicles AI Competition

Post by Brumi »

Wow, thank you very much for your effort. I knew you were up to something when you were positing all them bugs to their respective threads :)
Wormnest
Engineer
Engineer
Posts: 117
Joined: 14 Jul 2013 12:33
Location: Netherlands

Re: Road Vehicles AI Competition

Post by Wormnest »

Well done, very detailed comparison of road AI's.
I would have added one more column with the average profit per vehicle. A quick check seems to show that Terron probably wins there before Dictator and NoCAB.
If Terron didn't stop at just using two 512x512 squares IIRC it might have had the highest yearly profit.
User avatar
Zuu
OpenTTD Developer
OpenTTD Developer
Posts: 4553
Joined: 09 Jun 2003 18:21
Location: /home/sweden

Re: Road Vehicles AI Competition

Post by Zuu »

Thanks for your effort. I wonder what does the efficiency column mean?
My OpenTTD contributions (AIs, Game Scripts, patches, OpenTTD Auto Updater, and some sprites)
Junctioneer (a traffic intersection simulator)
xarick
Transport Coordinator
Transport Coordinator
Posts: 335
Joined: 26 Feb 2015 00:52

Re: Road Vehicles AI Competition

Post by xarick »

Zuu wrote:Thanks for your effort. I wonder what does the efficiency column mean?
Company Value: value is taken from General Company Information window.
Profitable RVs (x/y): y is the total number of road vehicles, taken from General Company Information window. x is the number of vehicles that turned a profit last year, taken from Detailed Performance Rating window.
Road Pieces: is the number of road pieces taken from General Company Information window.
Served Stations (x/y): y is the sum of both the number of lorry stations and the total number of bus stations. They are taken from the List of Company Stations window. x is the number of recently served stations, taken from Detailed Performance Rating window, but it appears that this counter is somewhat misleading, I noticed some bus station pieces tied with airports, and while only busses were attending to that station, it still counted as 2 stations being served. The x value is probably artificially inflated for some AIs.
Best Last Years: taken from the Company Finances window, it is the Road Vehicle Income of either 2049 or 2050, whichever is higher.
Best RV: taken from the Company List of Road Vehicles window, it takes the first vehicle from the list with 'Profit Last Year' as the criteria.
Avg/RV: is the average income per road vehicle, formula used is 'Best Last Years' / 'total number of road vehicles'.
Efficiency: it's an attempt to determine the efficiency of infrastructure usage, formula used is 'Best Last Years' / 'Road Pieces'

But I'm open to suggestions for more columns with other information if needed
Formerly known as Samu
jw15
Engineer
Engineer
Posts: 1
Joined: 20 Aug 2016 20:35

Re: Road Vehicles AI Competition

Post by jw15 »

Very interesting to see the comparison of different AI's. Have you thought about doing this for other modes of transport? Are you planning to do again? I am learning how to build and AI (slowly but surely) and would love to participate in any upcoming competition. Thanks for your work on this!
Brumi
President
President
Posts: 920
Joined: 18 Jul 2009 17:54

Re: Road Vehicles AI Competition

Post by Brumi »

He already did tests for the other modes of transport:
svetovoi
Engineer
Engineer
Posts: 87
Joined: 12 Oct 2007 14:07

Re: Road Vehicles AI Competition

Post by svetovoi »

Wormnest wrote:Well done, very detailed comparison of road AI's.
I would have added one more column with the average profit per vehicle. A quick check seems to show that Terron probably wins there before Dictator and NoCAB.
If Terron didn't stop at just using two 512x512 squares IIRC it might have had the highest yearly profit.
nah, i'm sure Terron stopped down because of built in inner vehicles limit. Works like that: once cars counter reaches 1500 AI should cease new stations construction and will just add cars to existing ones.
Same with ships and rather low bound there: 50.
Overall i'm only surprised with subpar aircrafts test performance because such limit does not exist for them AFAIR.

And thanks for those tests and informative statistics, xarick.
Kogut
Tycoon
Tycoon
Posts: 2493
Joined: 26 Aug 2009 06:33
Location: Poland

Re: Road Vehicles AI Competition

Post by Kogut »

Thanks for the test! It is very interesting to see how building styles differ between AI.
Correct me If I am wrong - PM me if my English is bad
AIAI - AI for OpenTTD
Post Reply

Return to “OpenTTD AIs and Game Scripts”

Who is online

Users browsing this forum: No registered users and 2 guests