New feature in trunk: break on log message

[Zuu]

Hello all AI writers,

I'm happy to tell you that today Yexo has commited my break on string patch for the AIDebug window. Here's what I wrote about it on the wiki:

As of revision r19544 there has been added a few new controls at the bottom of the debug panel for breaking an AI when certain log messages come up. In order to use it you type a string into the text box and when your AI prints a log message that matches this string, the AI will be suspended after the AILog call and at the end of the current tick OpenTTD will be paused. When you hit the continue button or the regular pause button, your AI will continue to run. If you temporarily want to disable the matching without clearing the edit box you can use the small toggle button to the left of the edit box.

As you probably already have guessed, this is not available in 1.0 so you'll need to run a nightly from today or later to use it. If you find it tedious to run the nightlies, I would recommend taking a look at this wiki page: http://wiki.openttd.org/OpenTTD_Updaters. At the bottom of that page there is some hints that are useful if you rather want to make your own simple shell script than using one of the existing solutions.

I hope that you'll find the break string feature useful.


Edit: renamed the thread to "break on log message" as well as at the wiki, as it describes the feature better. Also added a link to the wiki page.

[Kogut]

Is it possible to suspend ai via console command?

[Zuu]

Hi,

Yes a command such as "suspend_ai <number>" could of course be added to the console; however, it will not result in anything different from a standard pause command using the same implementation as I did in this patch. At best it will suspend the AI at one tick and then pause depending on how the input loop is constructed in OpenTTD.

Please note also that when you hit continue and/or pause the AI will continue to run. It is not possible to unpause OpenTTD and keep the AI in a suspended state without further changes to the Squirrel/NoAI integration. It is probably not very complicated but there is no such parameter to set at the moment.

What I did was to add a function that uses all remaining oopcodes of the current tick. The AILog.* commands work such that when they are called they push their message text all way to the AIDebug pannel which get its OnInvalidate() function called. Here I compare the string with the match string and if it matches I call the function that use up all oopcodes. Then the AILog-command continues to run and when squirrel goes to next statement it will find out that it has no more oopcodes to use and suspend the AI for the reminder of the current tick. (a tick is one game loop iteration) If you eg. call AILog from within a Valuator or elsewhere where the AI cannot be suspended, it will continue to run until it can be suspended.

Zuu

[Brumi]

This seems to be really useful, nice work Zuu! If only it had got in the trunk a few weeks earlier...
Anyway, I think it can be really powerful with the 'build in pause mode' cheat.

[Blustuff]

Why did you choose to break on log message and not to add a break or breakpoint function ?

[Zuu]

Because log messages are already there. You don't need to make modifications to your AI to use it. (other than making the log messages unique enough)

[Blustuff]

other than making the log messages unique enough

This is as hard as adding a function call.

[planetmaker]

other than making the log messages unique enough

This is as hard as adding a function call.

As you know your ways so well: write the "better" patch. Be sure to write it properly and change the API for NoAI. Also: what is better: testing the very same code you release(d) or writing a special debug version of an AI and then wonder why the release version (still) behaves differently?

[Zuu]

other than making the log messages unique enough

This is as hard as adding a function call.

From the AI's point of view possible. Though, I think that often the log messages are already unique enough that you don't need to edit the AI and re-start it. That saves a lot of time I think. Also your approach would have taken longer to implement in the OpenTTD code and to me it doesn't add any major value. If you have another opinion that it is useful, then please enlighten us why it is more useful/flexible to have a dedicated beak function than breaking on log messages.

I guess with your approach it could be possible for the AI to break itself when it fails in "unhandled" ways instead of asserting. While my approach was more chosen from the point of view of as a human testing an AI.

[Blustuff]

Also your approach would have taken longer to implement in the OpenTTD code and to me it doesn't add any major value.

That's a good point.

I guess with your approach it could be possible for the AI to break itself when it fails in "unhandled" ways instead of asserting. While my approach was more chosen from the point of view of as a human testing an AI.

If I want to stop my AI when some event happend, it is more convenient to just put a breakpoint there. Without a proper IDE to do that with OpenTTD, it has to be done programmatically. With your solution, you first have to output something then to say that you want to stop when that something is output. Moreover, if I am not mistaken, you allow the AI to execute some more instruction and this may be a problem if you want to change the game (to create some chosen situation) before allowing the AI to continue.

There is many thing an AI could output. Outputing them all makes the console unreadable. So you allow your AI to output information of general interest and some temporary debug information. Since it is temporary you will enventually remove these output from your code. Thus, I don't think there is so many persistant output you may want to break on. The only case I see is the case you just mentionned. Some information are output when an internal error occurs. (wrong assertion, unhandled case) In those situations to pause/stop the AI is almost always a good idea.

[planetmaker]

Since it is temporary you will enventually remove these output from your code.

I think I would introduce a debug parameter and leave a good amount of debuging capability.

[Zuu]

If I want to stop my AI when some event happend, it is more convenient to just put a breakpoint there. Without a proper IDE to do that with OpenTTD, it has to be done programmatically. With your solution, you first have to output something then to say that you want to stop when that something is output.

True, if you don't have active logging already, then it will take longer with my solution.

Moreover, if I am not mistaken, you allow the AI to execute some more instruction and this may be a problem if you want to change the game (to create some chosen situation) before allowing the AI to continue.

That's only true if the AILog call is within a valuator or somewhere else where OpenTTD cannot suspend the AI. Otherwise the AI will get suspended right after the AILog call. If you have a better solution to suspend an AI right after the current NoAI api call than using up all oopcodes please let me know.

There is many thing an AI could output. Outputing them all makes the console unreadable. So you allow your AI to output information of general interest and some temporary debug information. Since it is temporary you will enventually remove these output from your code. Thus, I don't think there is so many persistant output you may want to break on. The only case I see is the case you just mentionned. Some information are output when an internal error occurs. (wrong assertion, unhandled case) In those situations to pause/stop the AI is almost always a good idea.

Could be true for your AI and possible some others. For my AIs there is plenty of output. In PAXLink there is 3 debug levels that range from just very little output to very much output. In even bigger projects I also independent levels for different areas of the program, but that would clutter the AI settings too much.


Good news, your solution could coexist with my solution. Add a NoAI break function that use the suspend function that my patch added (unless you are able to improve the suspension to also work for inside valuators etc.). One of them does not make the other impossible.

So don't let this end up in a flame war; I coded the break on log string from my point of view. It is now clear that some people, including you and possible even me would benefit from having a AI<something>::Break() function. The next question is who wants to write it and how should it be implemented?

[Yexo]

Since r19548 you have to enable the setting "gui.ai_developer_tools" setting first. You can either enable it in the config file or by typing "set ai_developer_tools 1" in the console.

[fanioz]

Wow, nice feature and very usefully. But it seem one breakpoint text could only stop the one active AI window.
eg. I run 2 instance of Trans, set a breakpoint on #1. Switching to #2 log window to see what it does. (of course the text box would remain the same), then OpenTTD paused due to catch string on #2 window, not the #1.

Summary (idea) :
- Every window knows when to break, based on the text box it own. The breakpoint text is independent for every window.
- A not-yet-set window should display an empty text.


edit:: adding idea.

[Zuu]

fanoz: I though about that one but decided to at start make it as simple as possible. That said, it's good that you bring it forward showing that there is a need for it.

In your idea:

• when a AI goes bankrupt should the string be cleared then?
• when you start a new game should all the strings be cleared then? (only via game menu, or also with the reset and newgame console commands?)

[Alberth]

I think using a GUI to monitor logs and automatically suspend an AI is wrong.
One simple use-case: close the window, and watch the AI not stopping

Imho a GUI is just a nice frontend to display data to the user and to enter new actions to do (eg build/restart/whatever). It is not intended as a monitoring/controlling entity.
The feature in itself is useful (probably, I'll leave that to you), but I'd favor a better embedding of it in the NoAI system.

[Zuu]

So something like

AI::SetBreakString(str);
AI::GetBreakString(str);

(with the simplification that there is a single AI object)


And then the GUI sets/gets the string to/from the AI objects? And then you could also add console commands for that?

[Alberth]

So something like

AI::SetBreakString(str);
AI::GetBreakString(str);

(with the simplification that there is a single AI object)

Those would be the core methods yes (perhaps with a better name, like {G,S}etSuspendLogString, and with a case-sensitivity parameter).

The 'print' method (however that works), would then check the text, and act on it.

And then the GUI sets/gets the string to/from the AI objects? And then you could also add console commands for that?

Once you have the basic functions in place, you can add as many ways to access it, as you like.

[fanioz]

As long as the only available debugging feature is on debug log window :

# when a AI goes bankrupt should the string be cleared then?

string should be cleared if and only if the AI that had chosen by OpenTTD for that slot was changed too.

when you start a new game should all the strings be cleared then? (only via game menu, or also with the reset and newgame console commands?)

-should be cleared on really new game via game menu or console but not on "restart" game

[Zuu]

Today some significant improvements was made to the script breaking feature in OpenTTD. (r24537 and r24542)

• Break-on-log now works also for Game Scripts
• When a script (AI or GS) is stopped by break-on-log, the script is paused in a new way that allow you to unpause the game and continue while the script remains paused.
• To unpause a script you now must click on the continue button.
• When the first script is paused, the game pauses. When the last script is resumed, the game unpauses.
• Paused script get a yellow background colour for their selector-button in the AI debug window. (the same place which get red if your script crashes)
• A new function, API AIController::Break(message) and GSController::Break(message), allow you to request that your script should be paused as soon as possible. (remember, in eg. a Valuator-function OpenTTD cannot suspend your script) In most cases "as soon as possible" is however directly after that statement has completed.

I suspect that some may suggest a setting to configure if the game should pause or not when a script break is triggered. Additionally, another further improvement could be a "pause" toggle button that allow you to pause a script from the GUI. I currently do not have any patches for this so if anyone want to work on that, that is welcome.

Of course, you will need tomorrows nightly or newer to use these features. And also remember to enable gui.ai_developer_tools in your openttd.cfg or none of this will be activated.


I would suggest everyone who uploads scripts to bananas to remove any [AI|GS]Controller.Break() calls as some of your script users will be fellow script developers. You could for example make a wrapper which reads a global var that you set in a .nut file that is auto-generated by your script/Makefile that generates the tar-bundle. Or make that script complain if there are any calls left in your source code.