List of good AI & AI testing.

[kamnet]

So the monthly updates went well, turns out not a lot changes in the AI world of OpenTTD.

Are you preparing an AI test for OpenTTD 14? I got a feeling that with some stricter rules, a lot of them are going to break.

[Redirect Left]

Are you preparing an AI test for OpenTTD 14? I got a feeling that with some stricter rules, a lot of them are going to break.

I can do yes. If the changes are already near-enough finalised in the released RC2 on openttd.org I can start doing them immediately on the machine i use for it. I only test using vanilla versions & settings, otherwise i'd never be finished testing likely, so if any of the changes relate to things that need the user to actively change settings well outside the defaults, or to use GRFs that do something unexpected (like no mail for example), the tests I do might not pick up any 'bad'ness in them?
Hopefully the changes don't break too many of the older ones with no chance of the original developer fixing it, else it might make the graveyard of broken AIs in the content browser worse, if there's anything I can do with my testing to help that, let me know. I wouldn't want too many players downloading AIs that break for X or Y reason and turn away from the game due to that perceived issue.

If you (or anyone in the know, i don't know where i'd find it myself (check the edit, found it!)) have any information from the anonymised data/surveys that openttd gathers in the background and sends to the team, and can suggest 'frequently used' or 'frequently changed' sets of GRF combinations and/or settings, i can look at doing a 'typical player environment' test as well as entirely vanilla, but that'll quickly get very long winded if there's no simple 'common' settings used that may upset AI. But i am definitely not averse to doing 'known' common combinations, given i usually do 2-3 games at a time with different AI in each already. The machine I use for testing is only an i3-10100F rather than my main machine i9, but given enough time it can churn through plenty of combinations if we'd like, it may just take more than a day or two to go through them all to a suitable time frame (i usually aim for 250 game years, even though my own games i play for many months or even a year irl, this data if collected by openttd for 'average years played' may be useful for testing, if it is able to collect that to begin with).

edit: I've found the survey data, although it isn't really presented in a manner that I can use to quickly come up with a set of common settings, having said that from looking at the survey data info itself, it probably isn't even possible to present it in that manner to begin with.
Sadly it doesn't collect data on used GRFs either.

[kamnet]

I've found the survey data, although it isn't really presented in a manner that I can use to quickly come up with a set of common settings, having said that from looking at the survey data info itself, it probably isn't even possible to present it in that manner to begin with. Sadly it doesn't collect data on used GRFs either.

Last year I did my own survey of NewGRFs used.
viewtopic.php?p=1260677#p1260677

[Redirect Left]

So i've done another batch of testing. This was with OpenTTD 14 RC2, and a few on RC3 as it was released mid-testing.

Using a 1024x1024 map and default settings, only 1 AI (DictatorAI) has a noticeable difference in behaviour. If there was any changes that were expected to massively break older AIs, it is not present in a default game with a reasonable map size. Although I hope at some point AIs are helped along the way, rather than risking breaking them, we can all agree the AI landscape is not a good experience especially for newer players randomly choosing an AI from the in game browser.

Otherwise, new AI tested
Rythorn AI: Failed, ran for many years before throwing an undefined index error. Prior to that it performed very well and established a large network of airports & planes in a sensible manner. So is very promising if its patched up.
Chopper: Pass. Absolute no errors at all or issues with its building methods. Not very profitable due to using only choppers, requires the aircraft limit to not be 0 to function at all though.
gelignAlte: Mid to failing. I've no idea what this is trying to do. It built 6 RVs and then did nothing else. But it didn't crash or go bankrupt so... Without looking at the AI i can't say if its broken and not able to do other stuff it was trying.

Previously tested AI with different outcomes than was noted before
DictatorAI: Sometimes now crashes with CPU usage errors, which it didn't in the past. I cannot prove this is due to OTTD changes, rather than bad luck and it just never happened before OTTD 14.
AIAI: Decides that a field is as good a place as any to build an RV stations, rather the town road layout.

Also worthy of note. In games where it was present, AAAHogEx once again absolutely demolished any other AI in terms of profit. I might have to take a look at its code and figure out what makes it such a great AI. Other high performing ones for challenging opponents were LudAIAfterFix & RailwAI, both of those, along with AAAHogEx maxed out the allowed unit counts consistently, so would probably do even better if I had increased the max amount of RVs, trains, airplanes etc.
I'm considering adding another column to the page, which tells you how competitive each of them are, if people look for a really challenging AI rather than a messes about and blends in the background AI.

I'm thinking of creating my own AI, even though I cannot for the life of me understand the code currently (still). However I'd like to see if its possible to make an AI that is massively profitable, but respectful. Not stealing goods produced by industries that it is not supplying, and always using bridges or tunnels over roads that are being used by RVs (if its even possible to find that out for an AI), and other 'friendly' style of behaviour. There's all sorts of unfriendly behaviour i've spotted throughout testing over the last few months. I'm curious to know if that is AI developers intentionally doing that to maximise profit (or because they don't care about other users), or if it is never given any thought so AIs happily do it given they're rarely told otherwise.

Are you preparing an AI test for OpenTTD 14? I got a feeling that with some stricter rules, a lot of them are going to break.

Is there any particular settings or setups you or the others who work on the code were expecting to make it more breaking? As on the vanilla settings, there wasn't really a big difference, unlike your suspicion a lot would break, did we strike lucky, or am i missing something that may trigger errors moreso that aren't unreasonably out of the ordinary settings?

[jfs]

Make sure to test AIs under different timekeeping settings:

Suggestions:
- Calendar-based
- Wallclock-based at 60 minutes per calendar year
- Wallclock-based with calendar progress stopped

[Redirect Left]

Interesting slow AI. I wonder if this is worth noting in the big list of recommended or not. The AI runs fine, but its really slow at doing it. Is this what people would call 'a problem', in terms of AI? It isn't related to the wallclock based rate, as it's just as poor at times in calendar based stuff. It was running with 500 RVs, meanwhile AAAHogEx being a money machine with an extra 500 units spread across air, rail & boat was doing not a lot in terms of time.

2024-03-26 16_14_56-OpenTTD 14.0-RC3.png (15.46 KiB) Viewed 11223 times

Make sure to test AIs under different timekeeping settings:

Suggestions:
- Calendar-based
- Wallclock-based at 60 minutes per calendar year
- Wallclock-based with calendar progress stopped

Using a random 12 AI, there's was no difference using wallclock with 60 minute calendar years. Although that is the result of a single test.

[pwhk]

So I am currently using some AI and found that Terron is really competitive, not as insane as AAAHogEx but easily above Railw or LuDiAI Afterfix. In particular, Terron gets off ground basically instantaneously, something that few other AIs could do.
Also found that CluelessPlus and trAIns is on par with RailW and LuDiAI Afterfix

[HGus]

gelignAlte: Mid to failing. I've no idea what this is trying to do. It built 6 RVs and then did nothing else. But it didn't crash or go bankrupt so... Without looking at the AI i can't say if its broken and not able to do other stuff it was trying.

It only builds a single RV line from the center of the biggest city and moves the fartest station to the suburbs when the city grows. No other purpose.

Interesting slow AI. I wonder if this is worth noting in the big list of recommended or not. The AI runs fine, but its really slow at doing it. Is this what people would call 'a problem', in terms of AI? It isn't related to the wallclock based rate, as it's just as poor at times in calendar based stuff. It was running with 500 RVs, meanwhile AAAHogEx being a money machine with an extra 500 units spread across air, rail & boat was doing not a lot in terms of time.

This one builds RV cargo lines where every single vehicle is scrapped after delivering the cargo. Every cycle it checks them all for units awaiting for scrap at the depots.

[Redirect Left]

So I am currently using some AI and found that Terron is really competitive, not as insane as AAAHogEx but easily above Railw or LuDiAI Afterfix. In particular, Terron gets off ground basically instantaneously, something that few other AIs could do.
Also found that CluelessPlus and trAIns is on par with RailW and LuDiAI Afterfix

Thanks for your input! I agree Terron is pretty decent at times. I am currently working on trying to figure out a decent ranking method for AIs. I can either use pure statistics dragged from ingame, that appears on the charts & similar, but I am beginning to think pure numbers may not be the end of it all, and some personal judgements may work better. Graphs only explain what an AI was performing on that specific map & settings, and maybe drastically different in different environments. Then to figure out how to present, marks out of 5? 10? just words like "Competitive" "Poor"? I'm still out here doing AI stuff, just lost in the weeds with other projects currently.

gelignAlte: Mid to failing. I've no idea what this is trying to do. It built 6 RVs and then did nothing else. But it didn't crash or go bankrupt so... Without looking at the AI i can't say if its broken and not able to do other stuff it was trying.

It only builds a single RV line from the center of the biggest city and moves the fartest station to the suburbs when the city grows. No other purpose.

Interesting slow AI. I wonder if this is worth noting in the big list of recommended or not. The AI runs fine, but its really slow at doing it. Is this what people would call 'a problem', in terms of AI? It isn't related to the wallclock based rate, as it's just as poor at times in calendar based stuff. It was running with 500 RVs, meanwhile AAAHogEx being a money machine with an extra 500 units spread across air, rail & boat was doing not a lot in terms of time.

This one builds RV cargo lines where every single vehicle is scrapped after delivering the cargo. Every cycle it checks them all for units awaiting for scrap at the depots.

Thanks for the explanations. I'm still poor at reading AIs and being able to immediately grasp it. Some of the AIs do make me wonder their intention, and some look like projects to test someones knowledge, or to do a specific job have entered the overall pool of AIs, muddying the waters further.

[HGus]

Well, gelignAIte seems to be a test project. It can keep the main city constantly growing while you focus on other tasks.

Rondje was purposely built as a cheater for a contest (that won), then released for solo playing. Many of their tricks are not valid with current versions of the game. viewtopic.php?t=39756
OtviAI was developed to test Rondje as a non cheater aggressive contestant. It works better but also is heavy on resources.

[Redirect Left]

Well, gelignAIte seems to be a test project. It can keep the main city constantly growing while you focus on other tasks.

Rondje was purposely built as a cheater for a contest (that won), then released for solo playing. Many of their tricks are not valid with current versions of the game. viewtopic.php?t=39756
OtviAI was developed to test Rondje as a non cheater aggressive contestant. It works better but also is heavy on resources.

Thanks again for your input here. OtviAI still performs relatively well on the tests I did, but it is confused by double headed trains. It builds trains up to the specified max length, then manages to do itself in when a double headed train adds more than the simple +1.0 size of a single engine.
Seems I never tested Rondje for some reason, but gelignAlte within the defined parameters you mention still works correctly.

Sadly I don't have the power to really make a meaningful difference, or flag these specific things within the game, without people coming to TT-F first, which i doubt most do.

[BW89]

One AI I think is missing from the list is Mogul AI. I tested it for ~20 in game years and it still works, is able to use GRF vehicles and can work with FIRS and road GRFs

[Redirect Left]

One AI I think is missing from the list is Mogul AI.

oops, you're right!

It seems i got thrown to the side early on, i'm not sure why. I have spent the day adding it to some testing, and eventually it came up with an ok rating. Sadly only uses RVs so not too competitive, but it does seem to work except a minor glitch i found under specific settings, either way it doesn't crash so instead of a fail, its an ok.
Sadly i can't use the tests today for its competitiveness rank as it was added mid-testing but i will work that out when i do the big update to mark them by competitiveness rather than "it works!".

Thanks again for letting me know!

[MinchinWeb]

So on the "AI suitability list" linked from here (https://tipsyfox.net/ottd/), WmDOT is marked down, with the implication that it cuts off station access, with the below sample image.

wmDOTroads.png (85.72 KiB) Viewed 271 times

But I can't figure out how WmDOT could have created this problem... I've tried flipping between companies in "sandbox mode", but it won't create a connection to an existing one-way road, and it won't let you convert to a run-way road if there is already a connection. Is it possible that another AI created this station, and their pathfinder failed?

If someone can explain how this happened, I can work on testing to keep it from happening in the future...

Also, on the link on the "suitability list" is wrong, and points to the AI below it.

[Redirect Left]

I guess it is possible it was another AI? if it was, whichever AI did it has never done it again on a different test since then though.
I'll clear its mark and see if it doesn't do it again in other examples.

Also, on the link on the "suitability list" is wrong, and points to the AI below it.

This is fixed, thanks for letting me know.

[MinchinWeb]

Thanks!