Indeed that is the crucial or fundamental question in this which needs an answer before the 'inactivity' of an AI can be judged.Brumi wrote: I think if we would like to measure the stability of AIs, it cannot be done in some scientific way, rather in an empirical way, based on users' experiences. Some stability measurement would be useful in my opinion, but whose responsibility would it be to do that?
There have been sometimes requests for including an AI again with OpenTTD. Now, that's difficult... we (as OpenTTD core authors) wouldn't have written it, but possibly would be called liable for its mistakes - who shall fix them then? - , if any AI would be included. But even without doing that, it certainly might make sense to establish some criteria which could judge the quality of an AI:
- Stability: Each AI could be left running on a few pre-made scenarios and of course it should not crash and do something there
- Competitiveness: Success in (possibly the same) scenarios on an economical, monetary scale. Additionally it'll need IMHO a game where several AIs compete with eachother Possible scenarios (non-comprehensive) for this test series which a state-of-the-art AI should be able to survive at least (I know it excludes all one-transport-mode AIs, however well they perform there ):
- air only map
- road only map
- rail only map
- ship only map
- map with all vehicles available, newgrf industries, newgrf trainset with 'AI unfriendly' consist conditions, orig. accel
- map with all vehicles available but costs re-balanced via base costs, very steep TF costs, realistic accel
- "Beauty". The most subjective term. I.e. use of all transport types in a 'reasonable' manner. Employing a somewhat good path finder for road and rail, re-use of existing infrastructure (i.e. not spamming the map with rail and road when all infrastructure is mostly empty)
EDIT: three first simple test cases
* ship only
* train only
* air only
Load the game, open console, type in start_ai <name of AI> and have it run for 10 years. See also http://wiki.openttd.org/AI:Test_AIs
But maybe that table is better moved to the AI comparison page. Or linked there or whatever.