CluelessPlus

Discuss the new AI features ("NoAI") introduced into OpenTTD 0.7, allowing you to implement custom AIs, and the new Game Scripts available in OpenTTD 1.2 and higher.

Moderator: OpenTTD Developers

Post Reply
User avatar
planetmaker
OpenTTD Developer
OpenTTD Developer
Posts: 9295
Joined: 07 Nov 2007 22:44
Location: Sol d

Re: CluelessPlus

Post by planetmaker » 25 Jul 2010 16:32

You might want to supply a savegame where that issue you describe becomes apparent

User avatar
Zuu
OpenTTD Developer
OpenTTD Developer
Posts: 4553
Joined: 09 Jun 2003 18:21
Location: /home/sweden

Re: CluelessPlus

Post by Zuu » 26 Jul 2010 12:53

As planetmaker suggestuded, a savegame would be usefull. My guess is that it could either be the search algorithm for finding industries/towns to connect or the station statistics collector. But if it happened at the beginning then it is more likely that it is the search algorithm. While I've tried to reduce the maximum number of nodes that are created and sorted it is possible that the limit is too high. Or there are another sort or valuate call that is potentially time consuming.

I will later take a look at the source code when im able to do so.
My OpenTTD contributions (AIs, Game Scripts, patches, OpenTTD Auto Updater, and some sprites)
Junctioneer (a traffic intersection simulator)

Darmarius
Engineer
Engineer
Posts: 5
Joined: 25 Jul 2010 15:51
Location: Canada

Re: CluelessPlus

Post by Darmarius » 26 Jul 2010 17:20

Alright, here goes:

The base 1024x1024 map freezes for 1-2 seconds near the 3rd Jan, where the ECS one is near the 8th Jan for a good 5-10 seconds. Just move the mouse around the screen, you'll see it stop moving when it freezes. I also tried a base 2048x2048, but instead of freezing, the AI crashes. I wanted to upload the savegame, but it seems it's too large, so I've uploaded a screenshot with the "stack trace" (if we can call that a stack trace...) instead. And I understand that 2048x2048 can be though for AIs, but if it helps to point in the good direction, why not?
Attachments
1024x1024.sav
No NewGRF, map size 1024x1024
(1.28 MiB) Downloaded 123 times
ECS 1024x1024.sav
All ECS vectors, map size 1024x1024
(1.15 MiB) Downloaded 124 times
crash 2048x2048.png
crash 2048x2048.png (34.16 KiB) Viewed 4608 times

User avatar
luk3Z
Engineer
Engineer
Posts: 69
Joined: 25 Dec 2005 17:42
Location: Poland

Re: CluelessPlus

Post by luk3Z » 27 Jul 2010 21:37

New versions of AI should be in the 1st post imho ;)

(when I back here I thought that there is no new version of AI - according to lack of info in the 1st post)
Find new graphics easier:
GRFCrawler -> http://grfcrawler.tt-forums.net
BaNaNaS -> http://bananas.openttd.org/en/

Darmarius
Engineer
Engineer
Posts: 5
Joined: 25 Jul 2010 15:51
Location: Canada

Re: CluelessPlus

Post by Darmarius » 28 Jul 2010 01:50

I just read this from theNoAI wiki introduction:
You don't need to worry too much about your AI clogging the game by eating all the CPU of your computer (and thus stopping the game itself from running), the engine itself will sleep every X opcodes which forces fair scheduling and allows the game to run smoothly.
So, hummm, why ?( When I first hit that problem I was pretty sure I was hallucinating, that it couldn't really be the AI since its only logical that the NoAI framework has such a mechanism to avoid that kind of problem. But hummm.... now I'm really intrigued :D Anything else I can do to help you diagnose this?

Morloth
Transport Coordinator
Transport Coordinator
Posts: 378
Joined: 07 Feb 2008 14:06
Location: Glasgow

Re: CluelessPlus

Post by Morloth » 28 Jul 2010 08:41

Darmarius wrote:I just read this from theNoAI wiki introduction:
You don't need to worry too much about your AI clogging the game by eating all the CPU of your computer (and thus stopping the game itself from running), the engine itself will sleep every X opcodes which forces fair scheduling and allows the game to run smoothly.
So, hummm, why ?( When I first hit that problem I was pretty sure I was hallucinating, that it couldn't really be the AI since its only logical that the NoAI framework has such a mechanism to avoid that kind of problem. But hummm.... now I'm really intrigued :D Anything else I can do to help you diagnose this?
The way the system works is that it counts the number of opcodes and pauses after executing X of these. An opcode can be seen as a very basic instruction inside of the Squirrel language. So as long as you keep executing squirrel commands the statement you quoted holds.

However, it is also possible to call functions written in C++ from Squirrel. While the opcodes for calling the C++ function will be counted the actual part that is written in C++ is not. For example, if I were to play on a 2048x2048 map and make use of a Valuator to valuator all tiles on the map using a valuator written in C++ these instructions will not be counted towards the number of opcodes an AI is allowed to execute before pausing and it will freeze the game for a while. I suspect this is one of the reason why some of the suggestions for adding functionality to the API have been declined as a programmer who is not aware of this might just break the game by making their AI take up so much CPU; If you were to write the same functionality in Squirrel than you AI will be able to do much less in the same amount of time but without the risk of clogging ones CPU.

Darmarius
Engineer
Engineer
Posts: 5
Joined: 25 Jul 2010 15:51
Location: Canada

Re: CluelessPlus

Post by Darmarius » 28 Jul 2010 14:01

Ooooo, interesting insight! That's really nice to know, shouldn't that information be included somewhere in the wiki?

User avatar
planetmaker
OpenTTD Developer
OpenTTD Developer
Posts: 9295
Joined: 07 Nov 2007 22:44
Location: Sol d

Re: CluelessPlus

Post by planetmaker » 28 Jul 2010 14:08

If you feel that information is lacking or too well hidden... you can rectify that easily :-) It only contains those pieces where *someone* *somewhen* thought it's worthwhile to spend time on documenting it :-) And for those who know a piece of knowledge already longer it's sometimes hard to guess what might be interesting - they know already and take their knowledge for granted.

Expanding, maintaining and updating the wiki is certainly a good way to contribute, too!

Darmarius
Engineer
Engineer
Posts: 5
Joined: 25 Jul 2010 15:51
Location: Canada

Re: CluelessPlus

Post by Darmarius » 28 Jul 2010 20:18

Will do. Posting about the problem gave me an unexplainable urge to try and make my own AI, so I'll be going through the NoAI wiki for hints and tips about the framework. And add things I found were missing during my personal endeavor, of course.

User avatar
Lord Aro
Tycoon
Tycoon
Posts: 2365
Joined: 25 Jun 2009 16:42
Location: Location, Location
Contact:

Re: CluelessPlus

Post by Lord Aro » 28 Jul 2010 20:20

Darmarius wrote:Will do. Posting about the problem gave me an unexplainable urge to try and make my own AI, so I'll be going through the NoAI wiki for hints and tips about the framework. And add things I found were missing during my personal endeavor, of course.
sames. :mrgreen:
And a year later, with no programming experience beforehand at all, it's nearing release! :mrgreen: YAY!
AroAI - A really feeble attempt at an AI

It is practically impossible to teach good programming to students that have had a prior exposure to BASIC: as potential programmers they are mentally mutilated beyond hope of regeneration. --Edsger Dijkstra

User avatar
Zuu
OpenTTD Developer
OpenTTD Developer
Posts: 4553
Joined: 09 Jun 2003 18:21
Location: /home/sweden

Re: CluelessPlus

Post by Zuu » 31 Jul 2010 09:46

Darmarius wrote:Alright, here goes:

The base 1024x1024 map freezes for 1-2 seconds near the 3rd Jan, where the ECS one is near the 8th Jan for a good 5-10 seconds. Just move the mouse around the screen, you'll see it stop moving when it freezes. I also tried a base 2048x2048, but instead of freezing, the AI crashes. I wanted to upload the savegame, but it seems it's too large, so I've uploaded a screenshot with the "stack trace" (if we can call that a stack trace...) instead. And I understand that 2048x2048 can be though for AIs, but if it helps to point in the good direction, why not?
Thanks for the 1024x1024 ECS savegame. I've started to run tests with it. So far I have only been able to figure out that it is within the add-industry node loop that the freeze happens. The specific squirrel call that happen to cause the freeze is a fairly simple line of code:

Code: Select all

if (!add_all) {
Instead, my current guess is that there is some memory management in OpenTTD/Squirrel that kicks in, but I need to run OpenTTD through a debugger to reject or confirm this theory. In the savegame there are 787 industries and 31 cargoes, that makes 787x31 = 24397 class instances to be created and pointers (to the instances) to be stored in a squirrel array. In addition to that there has previously been created a number of nodes for each town depending on the number of town cargos that exist.

However, the sort operation that I first though was the freeze-point seem to not cause any trouble.

I will continue my investigations, but just wanted to give you an update.
luk3Z wrote:New versions of AI should be in the 1st post imho ;)

(when I back here I thought that there is no new version of AI - according to lack of info in the 1st post)
I generally put information for those that are new to the AI/patch/project etc. in the first post and put a notice there how to find the last version. For AIs you can just go to the BaNaNaS website (linked from first post) and find a download link there if you can't use the in-game downloader. If you log in to the forums, then you will be informed when there is new posts in the thread and you can go and check if you are more interested in the AI and want to read the release notes etc. Otherwise a short change log is included in the bananas info text that currently only can be seen inside OpenTTD. OpenTTD can check for updates to all your ingame-content, so there you can find out when there are new releases of my AIs.

If you follow me, you see that there are a number of ways to stay updated without requiring me to each and every update to any project I'm doing to update the first post in the thread. That's just bound to failure as I would for sure forget to do that at least once. The only exception is OpenTTD Auto Update, where I IIRC do update a link in the first post to the post containing the last version.
My OpenTTD contributions (AIs, Game Scripts, patches, OpenTTD Auto Updater, and some sprites)
Junctioneer (a traffic intersection simulator)

User avatar
Zuu
OpenTTD Developer
OpenTTD Developer
Posts: 4553
Joined: 09 Jun 2003 18:21
Location: /home/sweden

Re: CluelessPlus

Post by Zuu » 31 Jul 2010 15:05

Sometimes things are not what they look like. While re-compiling OpenTTD to do further Squirrel dumps, I realized that it in theory could execute 9999 AILog.Info(..) calls and then make a call to the freezing function within the same tick. Then the 9999 log messages won't show up before it freezes. Therefore it was not unlikely that it actually was a sorter that froze OpenTTD.

I've removed the usage squirrel array sort and instead use a Fibonacci Heap for sorting instead. The Fibonacci Heap is implemented in Squirrel by a NoAI library, and will thus allow OpenTTD to suspend while doing any actions in addition to having lower computational complexity. Also a problem with the Squirrel array sort is that it computes the value of each item on the fly using a comparator function instead of storing it when inserting items.

After these changes, there is no freeze anymore in my local copy. I will need to check the state of my source code a bit before uploading a fix. I think that some time ago, I've done some initial work on aircraft support which I have to check up before releasing it.
My OpenTTD contributions (AIs, Game Scripts, patches, OpenTTD Auto Updater, and some sprites)
Junctioneer (a traffic intersection simulator)

rh1
Engineer
Engineer
Posts: 4
Joined: 03 Aug 2010 20:47

Re: CluelessPlus

Post by rh1 » 03 Aug 2010 21:14

I found some problems in this AI when I tested it recently (version 21).

Line 944 in main.nut causes an AI crash (after several hours of playing).

Code: Select all

// Repair the station variable (it should be a tile of the station, but the tile it contains no longer contains a stop)
this.station[town_id] = AIBaseStation.GetLocation(station_id);
'town_id' should be 'town_i' I think...? perhaps this line is not run that often.

---
I have also seen it build oiltrucks that occupied a station forever, since no oil was present at the station (don't know if it ever was but i guess so). It blocked all other trucks of different cargo type coming to this station, causing a major traffic jam.

User avatar
Zuu
OpenTTD Developer
OpenTTD Developer
Posts: 4553
Joined: 09 Jun 2003 18:21
Location: /home/sweden

Re: CluelessPlus

Post by Zuu » 03 Aug 2010 23:05

rh1 wrote:I found some problems in this AI when I tested it recently (version 21).

Line 944 in main.nut causes an AI crash (after several hours of playing).
Code:
// Repair the station variable (it should be a tile of the station, but the tile it contains no longer contains a stop)
this.station[town_id] = AIBaseStation.GetLocation(station_id);

'town_id' should be 'town_i' I think...? perhaps this line is not run that often.
Thank you for your report. I have fixed this in the SVN version of CluelessPlus. There will be a new release rather soon as I've also fixed the freeze issue reported above.

You are right however that the line is not executed very often. It will only be executed if a station reduces the number of bus/truck stops and the stop with the station sign is removed.

rh1 wrote:I have also seen it build oiltrucks that occupied a station forever, since no oil was present at the station (don't know if it ever was but i guess so). It blocked all other trucks of different cargo type coming to this station, causing a major traffic jam.
If you notice this again, please upload a save game and I will take a look what the problem is. CluelessPlus 21 should check for closed industries, zero production etc. and close down or suspend connections to not clog up things. It is actually so that if another company serve an industry with just a train that arrives every second or third month, then CluelessPlus will constantly suspend/activate that connection due to the production drops at the industry between the deliveries.

Therefore, if you say you've seen this is not working, I would be happy to get a save game when it does not work so that I can learn what goes wrong and improve my code.
My OpenTTD contributions (AIs, Game Scripts, patches, OpenTTD Auto Updater, and some sprites)
Junctioneer (a traffic intersection simulator)

rh1
Engineer
Engineer
Posts: 4
Joined: 03 Aug 2010 20:47

Re: CluelessPlus

Post by rh1 » 04 Aug 2010 10:25

I have not seen the problem with never ending loading again, but I found another similar problem.
A truck stop gets no cargo but connections are not closed. The trucks also have 3 invalid orders before that stop.

Hope the save game works and shows the problem.
Attachments
Transport, 26 aug 1966_problem1.sav
(75.87 KiB) Downloaded 108 times

User avatar
Zuu
OpenTTD Developer
OpenTTD Developer
Posts: 4553
Joined: 09 Jun 2003 18:21
Location: /home/sweden

Re: CluelessPlus

Post by Zuu » 04 Aug 2010 13:29

I've taken a brief look at the save game. I don't know exactly why this occurs. At some point a station and two depots has been destroyed. This kind of broken connections are not checked for in the AI, other than when loading a save game.

I think I will try to extend the broken connections removal code which run when save games are loaded. But first I will have to check again if there is some construction code somewhere that can have fatal effects on existing stations/depots or if this could happen due to problems while closing down a connection. (it really should never remove any stations/depots until all vehicles has been sold off)

Thanks for your save game.


Edit: by using the "gamelog" console command I've found out that you use 1.0.3-RC1. Please upgrade to 1.0.3 to get the last stable version.
My OpenTTD contributions (AIs, Game Scripts, patches, OpenTTD Auto Updater, and some sprites)
Junctioneer (a traffic intersection simulator)

rh1
Engineer
Engineer
Posts: 4
Joined: 03 Aug 2010 20:47

Re: CluelessPlus

Post by rh1 » 04 Aug 2010 21:25

It happened again in another game (1.0.3 this time), so it is still there and not an isolated incident.
Don't know if it helps with the attached files.

I haven't touched a thing, except for building one bus-line.
Attachments
ClulessPlus(v21)_problem1_#2.png
(723.71 KiB) Downloaded 1 time
Transport, 17 okt 1998_problem1_#2.sav
(77.28 KiB) Downloaded 111 times

Alteris
Engineer
Engineer
Posts: 2
Joined: 11 Aug 2010 11:07

Re: CluelessPlus

Post by Alteris » 12 Aug 2010 02:23

Todays clueless crash. Have fun with it. ;)

Image

User avatar
Zuu
OpenTTD Developer
OpenTTD Developer
Posts: 4553
Joined: 09 Jun 2003 18:21
Location: /home/sweden

Re: CluelessPlus

Post by Zuu » 12 Aug 2010 09:48

Thanks for the report.

It's a type that causes problem if you have more than 1000 industries. I'm a bit busy IRL right now but there is a new version on the way with a couple of fixes, but I need to make sure it doesn't cause any other trouble. Also the invalid order bug, has only been worked around, but I've not yet found out why those orders happen in first place. (yes I understand that it happens because a station/depot gets removed, but why does it do that when there are remaining vehicles on the connection?)
My OpenTTD contributions (AIs, Game Scripts, patches, OpenTTD Auto Updater, and some sprites)
Junctioneer (a traffic intersection simulator)

User avatar
Zuu
OpenTTD Developer
OpenTTD Developer
Posts: 4553
Joined: 09 Jun 2003 18:21
Location: /home/sweden

Re: CluelessPlus

Post by Zuu » 12 Aug 2010 22:17

Update - Version 22

A few minutes ago a new version of CluelessPlus was uploaded to BaNaNaS. It contains a number of fixes and tweaks:
  • Use Fibonacci Heap instead of Squirrel arrays in the pair sorter - no more 10 second freezes at large maps
  • -> New dependency: Fibonacci Heap version 2
  • Only suspend connections when the station usage is higher than a threshold.
  • Fix AIbase crash (I made a typo, it should be AIBase)
  • town_id -> town_i (see bug report)
  • Find and sell vehicles with void orders
  • Sell all vehicles that use either station when closing down connections (previously the vehicles had to have both stations in their orders)
  • Allow closing connections without vehicles (good if all vehicles was sold of because of invalid orders)
  • Fixed two or three foreach loops where the key-variable was missing
However I have not been aware of fixing the root problem of the void orders. On the other hand, I have not been able to reproduce the problem using last trunk and the development version of CluelessPlus. When I have time (not now) I could go back to version 21 and run it in a self compile of 1.0.3 where I put some break points at the moment when orders become void and possible trace the reason for it and then see if the bug still exists.

BaNaNaS (in game content download)
Version 22 is already at bananas as well as all dependencies.

Manual downloading
In addition to CluelessPluss, version 22 you need these AI libraries:
  • Pathfinder.Road - version 3
  • Queue.FibonacciHeap - version 2
  • SuperLib - version 6
Download links can be found at the bananas page: (click on the download numbers)
http://bananas.openttd.org/en/ai/ (AIs)
http://bananas.openttd.org/en/ailibrary/ (AI libraries)
My OpenTTD contributions (AIs, Game Scripts, patches, OpenTTD Auto Updater, and some sprites)
Junctioneer (a traffic intersection simulator)

Post Reply

Return to “OpenTTD AIs and Game Scripts”

Who is online

Users browsing this forum: No registered users and 4 guests