Page 1 of 1
[Patch] NoGo/NoAI town build
Posted: 17 Sep 2013 23:34
by R2dical
So I'm working on implementing Zuu's NoGo/NoAI
patch (hope this is OK

)
I've updated to current trunk and added a required #include but can't get it to compile. I get this error (MSVC 2012):
Code: Select all
Error 3 error C2129: static function 'T SQConvert::GetParam<Targ4>(SQConvert::ForceType<T>,HSQUIRRELVM,int,SQConvert::SQAutoFreePointers *)'
declared but not defined \trunk\src\core\smallvec_type.hpp 36 1 openttd
The line of code being:
Code: Select all
SmallVector() : data(NULL), items(0), capacity(0) { }
This .hpp was untouched by the patch, which compiled fine before. With this I am quite lost as it seems to be an error with the Squirrel implementation, usually a c2129 error is "A forward reference is made to a static function that is never defined."
Anyone have an idea?
Attached is the patch.
Re: [Patch] NoGo/NoAI town build
Posted: 18 Sep 2013 10:33
by YNM
Surely it's not included in the trunk itself, right ?
Re: [Patch] NoGo/NoAI town build
Posted: 18 Sep 2013 14:26
by R2dical
You mean is this patch part of current trunk code causing this error on trying to compile the clean nightly?
Then no, this patch was started by Zuu to allow the AI and GS to found a town (and rename one), it is incomplete and not part of trunk.
I am interested in this feature so want to try finish the patch.
Re: [Patch] NoGo/NoAI town build
Posted: 18 Sep 2013 14:44
by Zuu
I actually forgot about this patch. I wrote it while I had no access to an OpenTTD compilation environment and I have therefore not had the chance to use a compiler to verify the correctness of the code. Which is also a reason why the patch was never committed.
Thanks for reminding me.
Edit: One of the missing things to do is to run the code generation scripts for squirrel. This is one reason why you get a compilation error.
Re: [Patch] NoGo/NoAI town build
Posted: 18 Sep 2013 20:10
by R2dical
Zuu wrote:I actually forgot about this patch. I wrote it while I had no access to an OpenTTD compilation environment and I have therefore not had the chance to use a compiler to verify the correctness of the code. Which is also a reason why the patch was never committed.
Thanks for reminding me.
Ahh I see, cool
Zuu wrote:Edit: One of the missing things to do is to run the code generation scripts for squirrel. This is one reason why you get a compilation error.
I see (well not really squirrel integration is beyond me

). So I assume you will retake the reigns on this? At least the patch in the top post was updated to trunk...
Re: [Patch] NoGo/NoAI town build
Posted: 19 Sep 2013 19:20
by Zuu
R2dical wrote:Zuu wrote:Edit: One of the missing things to do is to run the code generation scripts for squirrel. This is one reason why you get a compilation error.
I see (well not really squirrel integration is beyond me

). So I assume you will retake the reigns on this? At least the patch in the top post was updated to trunk...
Update to
r25785 or later, and you will find the new API methods available.

Allow up to 24 hours for the docs web to update, but as you did have a look at the patch file, you probably know where to look to figure out the API even before the docs are updated.

Re: [Patch] NoGo/NoAI town build
Posted: 20 Sep 2013 01:10
by R2dical
Many thanks and nice work!
Looks like its working great:

- TestRun.png (65.86 KiB) Viewed 1054 times

Re: [Patch] NoGo/NoAI town build
Posted: 21 Sep 2013 13:24
by R2dical
Not sure if this should go here or in the AI thread, but this is opened already so...
So 2 things, first and most importantly I managed to freeze/cause heavy and continuous lag in OpenTTD with my GS using this new function. It works ok for small maps but on a 1024 x 1024 the game becomes unresponsive. This is the function in my GS which is doing this, let me know if the whole GS will be easier to test:
Code: Select all
// Function call to build towns
function IndustryConstructor::BuildTown() {
// Check GS continue, to exit
if(CONTINUE_GS == false) return;
// Display status msg
Log.Info("+==============================+", Log.LVL_INFO);
Log.Info("Building towns...", Log.LVL_INFO);
// Check initiaized
if (this.INIT_PERFORMED == false){
// Display error msg
Log.Error(">IndustryConstructor.BuildTown: Script uninitialized!", Log.LVL_INFO);
return;
}
// Loop for number of towns modified by map size, not including initial 1
local MAX_TOWNS = GSController.GetSetting("DENSITY_TOWN_TOTAL") * MAP_SCALE;
local CITY_COUNT = 0;
for(local i = 1; i < MAX_TOWNS; i++){
// Random build
local BUILD_TRIES = 2000;
local TILE_ID;
local TOWN_SIZE;
local IS_CITY = false;
// Loop until correct tile
while(BUILD_TRIES > 0){
// Get a random tile
TILE_ID = Tile.GetRandomTile();
// Get a random size
TOWN_SIZE = GSBase.RandRange(4);
// Set city
if(CITY_COUNT < GSGameSettings.GetValue("larger_towns"))
{
IS_CITY = true;
CITY_COUNT++;
}
local BUILT_TOWN = GSTown.FoundTown(TILE_ID, TOWN_SIZE, IS_CITY, GSGameSettings.GetValue("town_layout"), null);
// Try build
if(BUILT_TOWN == true){
// Multiply size if city
if(IS_CITY == true)
{
// - Get town id
local TOWN_ID = GSTile.GetTownAuthority(TILE_ID);
// - Get house count
local TOWN_HOUSES = GSTown.GetHouseCount(TOWN_ID);
// Loop till size
for(local j = 0; j < GSGameSettings.GetValue("initial_city_size"); j++)
GSTown.ExpandTown(TOWN_ID, TOWN_HOUSES);
}
break;
}
BUILD_TRIES--;
}
// Display error msg
if(BUILD_TRIES == 0)
Log.Error(">IndustryConstructor.BuildTown: Failed to build town!", Log.LVL_INFO);
}
// Display status msg
local MAP_TOWNS = GSTown.GetTownCount();
Log.Info("Built " + (MAP_TOWNS - CITY_COUNT) + " towns and " + CITY_COUNT + " cities, for a total of " +
MAP_TOWNS + " / " + MAX_TOWNS, Log.LVL_INFO);
}
Yes I know there is lots of inefficient code, but this was just a test

Also I am using a clean version of r25785.
Secondly, calling this function (specifically the GSTown.FoundTown) after map generation causes the GS to "skip" its 2500 tick quota...why is this?
Re: [Patch] NoGo/NoAI town build
Posted: 21 Sep 2013 13:38
by Zuu
R2dical wrote:Not sure if this should go here or in the AI thread, but this is opened already so...
So 2 things, first and most importantly I managed to freeze/cause heavy and continuous lag in OpenTTD with my GS using this new function. It works ok for small maps but on a 1024 x 1024 the game becomes unresponsive. This is the function in my GS which is doing this, let me know if the whole GS will be easier to test:
Does this freeze/lag happen when the game is not running in fast forward?
R2dical wrote:Yes I know there is lots of inefficient code, but this was just a test

Also I am using a clean version of r25785.
Secondly, calling this function (specifically the GSTown.FoundTown) after map generation causes the GS to "skip" its 2500 tick quota...why is this?
I'm not sure.
Can you provide a minimal GS that shows this?
Edit: I've confirmed this with my test GS. I'll have to look into why this happens.
Re: [Patch] NoGo/NoAI town build
Posted: 21 Sep 2013 14:08
by R2dical
Zuu wrote:Does this freeze/lag happen when the game is not running in fast forward?
Yes, I think this may have something to do with the number of towns my script is set to generate. With the default setup on a 1024 x 1024 it will try build 640 towns, but it freezes way before it gets that far...
Zuu wrote:Can you provide a minimal GS that shows this?
Attached is a hack and slash version of my GS which replicates all the effects. There is a bit of collateral but I think its simple enough for testing
EDIT: Ninja'd

Re: [Patch] NoGo/NoAI town build
Posted: 21 Sep 2013 15:29
by Zuu
Trying to reproduce the lag, I started a temperate 1024x1024 game with your TownConstructor. However it doesn't build any towns. (using r25789)
Edit: Added save game
Re: [Patch] NoGo/NoAI town build
Posted: 21 Sep 2013 16:30
by R2dical
Looking into it, but does it do the same without NewGrfs?
Also something really wacky, I updated to r25789 and now everything works fine for me! Both this test GS and my main GS which I have not touched after update. The beginning quota is used and no big map lag

Although the ticks in quota seem to run very slowly and scale with map size...
EDIT: I get the same error as you with your newgrfs but works fine with no newgrfs. This test script is very basic (200 lines of code), uses game settings from cfg, null name and a random tile for each try of the new GS build town. If 2000 tries fail it gives the message in the screenshot.
Re: [Patch] NoGo/NoAI town build
Posted: 21 Sep 2013 17:42
by Zuu
With about 5000+ towns I've made these observations (using my own test script randomly builds large towns):
- When the town directory window is closed, there is no log
- When the town directory is open and use sort by name or sort by rating there is a lag. (sort by population gives no lag)
- If I disable the script, there is no lag with or without the town directory open (with eg. name sorter)
To conclude, it looks like it is re-building and sorting the list of towns in the Town Directory window while there are lots of tows that is what make the game lag. With the script enabled this has to be done once every tick and the operation probably takes long enough that OpenTTD can't keep up with the normal 74 ticks per second.
My tests was made on a computer with i7 3.6 GHz and with a map size of 2048x2048.
Re: [Patch] NoGo/NoAI town build
Posted: 21 Sep 2013 18:31
by R2dical
First thanks for looking into this today.
So looks like the update to r25789 solved the tick quota being skipped thing.
I think I found the cause of the lag, at least when using my scripts. I am using a town names grf (South African Town Names), with this on large maps the ticks go VERY slow then the game stops responding. This happens on my laptop (i3 2.13GHz) and PC (i5 2.66GHz).
I also see the lag with the town list window but thats nothing to the above when in the "Running script" part of new game creation.
If you still have your script could you post it? I would like a peak

Re: [Patch] NoGo/NoAI town build
Posted: 21 Sep 2013 19:12
by Zuu
Attaching my test GS. Despite the name, it does only build towns in the current form.
The GS was created to test the APIs to build industries. I now later used it to test the FoundTown API.
Re: [Patch] NoGo/NoAI town build
Posted: 28 Dec 2013 12:36
by R2dical
Hi Zuu.
Old discussion I know but I have run into this same issue with lag again
I have rewrote my GS from scratch and now when trying to do the same thing have run into the lag again. This time it is worse however...
During the "running script" set of 2500 ticks my script runs incredibly slowly for up to 5 or 6 then essentially freezes the game.
I tried your script posted above but noticed it performs its build "while the game is running" (not in initial 2500 ticks), using a Sleep(1). I modified it slightly to emulate my script (heavy build-count case admittedly) by making it build 2560 towns with 50 random tile tries for each loop. Sure as nuts same laggy behaviour, but on my second run I got a game crash! (repeatable - r26183 no grfs)
Code: Select all
Crash reason:
Exception: E1212012
Location: 7551812F
Message: Out of memory. Cannot allocate 33554432 bytes
OpenTTD version:
Version: r26181M (2)
NewGRF ver: 14006645
Bits: 32
Endian: little
Dedicated: no
Build date: Dec 26 2013 13:57:57
Registers:
EAX: 0012E74C EBX: 00000001 ECX: 00000000 EDX: 77D070F4
ESI: 00000016 EDI: 00000000 EBP: 0012E79C ESP: 0012E74C
EIP: 7551812F EFLAGS: 00200246
Bytes at instruction pointer:
C9 C2 10 00 89 45 C0 EB ED 3D 01 01 00 00 0F 85 6D 96 FF FF E9 49 96 FF
This happened after the usual heavy lag period, should this go in bugtracker too?
Attached is the GS, yours slightly modified as described.
Re: [Patch] NoGo/NoAI town build
Posted: 28 Dec 2013 13:26
by Alberth
Running out of memory is not fixable, other than you buying more memory for your computer, or by you rewriting your script so it uses less memory to do its work.
Re: [Patch] NoGo/NoAI town build
Posted: 28 Dec 2013 14:04
by R2dical
Well thats kind of my surprise and point. The above script is super basic, maybe 5 memory variables total and the only DoCommand is the FoundTown one. After running this for the 2560 towns OpenTTD memory footprint jumps from ~150mb to ~1.8 gb. Subsequent runs cause a crash on my 32bit win with 4gb ram.
Running the exact same script but with a Sleep(1) before the build loop (so it runs in game) keeps memory footprint to ~300mb.
So this is normal? Is there something about the initial 2500 ticks that has to be kept in mind to avoid this memory buildup?
Re: [Patch] NoGo/NoAI town build
Posted: 02 Jan 2014 23:20
by Zuu
The large memory usage difference sounds strange. It sounds like there might be a leak when towns are funded during world gen. Im a bit interested if this only happens for fund town or can be triggered also by other methods that construct objects. eg new goal, or if the problem is isolated to fund towns.
Re: [Patch] NoGo/NoAI town build
Posted: 03 Jan 2014 06:46
by R2dical
From what I have seen it seems isolated to the found town function, building industries and roads in my GS under the same circumstances is fine, but I could be wrong. Should be easy to test though as the memory leak is quite easy to see and repeat as in the previous modified version of your GS.
Maybe this is on the wrong track but the leak may involve town name generation? Last time I came across this, before I saw the leak, I thought I had "solved" it by disabling South African Town Names (which are longer on average). Of course since my rewrite SA names on/off both cause crash now but maybe at different rates?