List of good AI & AI testing.

[kamnet]

So the monthly updates went well, turns out not a lot changes in the AI world of OpenTTD.

Are you preparing an AI test for OpenTTD 14? I got a feeling that with some stricter rules, a lot of them are going to break.

[Redirect Left]

Are you preparing an AI test for OpenTTD 14? I got a feeling that with some stricter rules, a lot of them are going to break.

I can do yes. If the changes are already near-enough finalised in the released RC2 on openttd.org I can start doing them immediately on the machine i use for it. I only test using vanilla versions & settings, otherwise i'd never be finished testing likely, so if any of the changes relate to things that need the user to actively change settings well outside the defaults, or to use GRFs that do something unexpected (like no mail for example), the tests I do might not pick up any 'bad'ness in them?
Hopefully the changes don't break too many of the older ones with no chance of the original developer fixing it, else it might make the graveyard of broken AIs in the content browser worse, if there's anything I can do with my testing to help that, let me know. I wouldn't want too many players downloading AIs that break for X or Y reason and turn away from the game due to that perceived issue.

If you (or anyone in the know, i don't know where i'd find it myself (check the edit, found it!)) have any information from the anonymised data/surveys that openttd gathers in the background and sends to the team, and can suggest 'frequently used' or 'frequently changed' sets of GRF combinations and/or settings, i can look at doing a 'typical player environment' test as well as entirely vanilla, but that'll quickly get very long winded if there's no simple 'common' settings used that may upset AI. But i am definitely not averse to doing 'known' common combinations, given i usually do 2-3 games at a time with different AI in each already. The machine I use for testing is only an i3-10100F rather than my main machine i9, but given enough time it can churn through plenty of combinations if we'd like, it may just take more than a day or two to go through them all to a suitable time frame (i usually aim for 250 game years, even though my own games i play for many months or even a year irl, this data if collected by openttd for 'average years played' may be useful for testing, if it is able to collect that to begin with).

edit: I've found the survey data, although it isn't really presented in a manner that I can use to quickly come up with a set of common settings, having said that from looking at the survey data info itself, it probably isn't even possible to present it in that manner to begin with.
Sadly it doesn't collect data on used GRFs either.

[kamnet]

I've found the survey data, although it isn't really presented in a manner that I can use to quickly come up with a set of common settings, having said that from looking at the survey data info itself, it probably isn't even possible to present it in that manner to begin with. Sadly it doesn't collect data on used GRFs either.

Last year I did my own survey of NewGRFs used.
viewtopic.php?p=1260677#p1260677

[Redirect Left]

So i've done another batch of testing. This was with OpenTTD 14 RC2, and a few on RC3 as it was released mid-testing.

Using a 1024x1024 map and default settings, only 1 AI (DictatorAI) has a noticeable difference in behaviour. If there was any changes that were expected to massively break older AIs, it is not present in a default game with a reasonable map size. Although I hope at some point AIs are helped along the way, rather than risking breaking them, we can all agree the AI landscape is not a good experience especially for newer players randomly choosing an AI from the in game browser.

Otherwise, new AI tested
Rythorn AI: Failed, ran for many years before throwing an undefined index error. Prior to that it performed very well and established a large network of airports & planes in a sensible manner. So is very promising if its patched up.
Chopper: Pass. Absolute no errors at all or issues with its building methods. Not very profitable due to using only choppers, requires the aircraft limit to not be 0 to function at all though.
gelignAlte: Mid to failing. I've no idea what this is trying to do. It built 6 RVs and then did nothing else. But it didn't crash or go bankrupt so... Without looking at the AI i can't say if its broken and not able to do other stuff it was trying.

Previously tested AI with different outcomes than was noted before
DictatorAI: Sometimes now crashes with CPU usage errors, which it didn't in the past. I cannot prove this is due to OTTD changes, rather than bad luck and it just never happened before OTTD 14.
AIAI: Decides that a field is as good a place as any to build an RV stations, rather the town road layout.

Also worthy of note. In games where it was present, AAAHogEx once again absolutely demolished any other AI in terms of profit. I might have to take a look at its code and figure out what makes it such a great AI. Other high performing ones for challenging opponents were LudAIAfterFix & RailwAI, both of those, along with AAAHogEx maxed out the allowed unit counts consistently, so would probably do even better if I had increased the max amount of RVs, trains, airplanes etc.
I'm considering adding another column to the page, which tells you how competitive each of them are, if people look for a really challenging AI rather than a messes about and blends in the background AI.

I'm thinking of creating my own AI, even though I cannot for the life of me understand the code currently (still). However I'd like to see if its possible to make an AI that is massively profitable, but respectful. Not stealing goods produced by industries that it is not supplying, and always using bridges or tunnels over roads that are being used by RVs (if its even possible to find that out for an AI), and other 'friendly' style of behaviour. There's all sorts of unfriendly behaviour i've spotted throughout testing over the last few months. I'm curious to know if that is AI developers intentionally doing that to maximise profit (or because they don't care about other users), or if it is never given any thought so AIs happily do it given they're rarely told otherwise.

Are you preparing an AI test for OpenTTD 14? I got a feeling that with some stricter rules, a lot of them are going to break.

Is there any particular settings or setups you or the others who work on the code were expecting to make it more breaking? As on the vanilla settings, there wasn't really a big difference, unlike your suspicion a lot would break, did we strike lucky, or am i missing something that may trigger errors moreso that aren't unreasonably out of the ordinary settings?

[jfs]

Make sure to test AIs under different timekeeping settings:

Suggestions:
- Calendar-based
- Wallclock-based at 60 minutes per calendar year
- Wallclock-based with calendar progress stopped

[Redirect Left]

Interesting slow AI. I wonder if this is worth noting in the big list of recommended or not. The AI runs fine, but its really slow at doing it. Is this what people would call 'a problem', in terms of AI? It isn't related to the wallclock based rate, as it's just as poor at times in calendar based stuff. It was running with 500 RVs, meanwhile AAAHogEx being a money machine with an extra 500 units spread across air, rail & boat was doing not a lot in terms of time.

2024-03-26 16_14_56-OpenTTD 14.0-RC3.png (15.46 KiB) Viewed 1229 times

Make sure to test AIs under different timekeeping settings:

Suggestions:
- Calendar-based
- Wallclock-based at 60 minutes per calendar year
- Wallclock-based with calendar progress stopped

Using a random 12 AI, there's was no difference using wallclock with 60 minute calendar years. Although that is the result of a single test.

[pwhk]

So I am currently using some AI and found that Terron is really competitive, not as insane as AAAHogEx but easily above Railw or LuDiAI Afterfix. In particular, Terron gets off ground basically instantaneously, something that few other AIs could do.
Also found that CluelessPlus and trAIns is on par with RailW and LuDiAI Afterfix