[Squirrel] Atomicity of operations

[Blustuff]

Hello,

Save may occur between atomic operations but not necessary when we want it to do. For instance when you want to call a series of doaction, you may actually do the action and mark it done. If a save occur between this two operations, the actions is not marked so you will do it another time. This may not be a problem since doActions can tell you "already built". However this is not that easy for every data you want to save/load.

How do you managed to solve this problem ?

[Dustin]

Hello,

Save may occur between atomic operations but not necessary when we want it to do. For instance when you want to call a series of doaction, you may actually do the action and mark it done. If a save occur between this two operations, the actions is not marked so you will do it another time. This may not be a problem since doActions can tell you "already built". However this is not that easy for every data you want to save/load.

How do you managed to solve this problem ?

I still need to write my save load stuff. However, I basically plan on not worrying too much. It's pretty likely I will lose quite a lot of state during a save-reload. The AI would lose some pathfinding information and maybe some cache data. Worst case scenario is that a route is half built and never finished. The AI would eventually detect the route wasn't making money and clean it up. I theory it would be possible to save the route and finish it, but I doubt I would bother since the AI can just cope using an existing behavior.

[Blustuff]

Since DoActions are slow (min 1 tick) it is likely a save will occur during a construction. You can save a list of action and execute it. If a save occur during the execution, you may do the construction twice: you just need to ignore the ERR_ALREADY_BUILT error.

But some other AI states are not easy to retrieve if not impossible. You can't have critical session which means that some operations can't be done atomically. Here the problem is that if there is a problem it could be seen later in the game and you won't have any clue of where it came from. Bugs caused by unfortunate save/load are very hard to understand since you IA is in a state you don't expect it to be.

[Yexo]

The default number of opcodes your AI gets before it's suspended is 10000. The user can change this but the minimum value is 5000. That means that after a AIController.Sleep(1) you have 5000 opcodes where you can execute whatever you want without being interrupted for a save. Please note that even a simple assignment may take multiple opcodes,but it should still be enough to do some actions.

[Brumi]

As far as I know Sleep is automatically called with every DoCommand. (tell me if I'm wrong)
This means that a save might occur, for example, in the middle of giving vehicle orders, and after the load, there will be a vehicle sitting in the depot with incomplete orders. How can we avoid this?

[Yexo]

As far as I know Sleep is automatically called with every DoCommand. (tell me if I'm wrong)
This means that a save might occur, for example, in the middle of giving vehicle orders, and after the load, there will be a vehicle sitting in the depot with incomplete orders. How can we avoid this?

You can't. The best you can do is set a variable you're going to set orders for a specific vehicle id, then update the orders, then when the game is saved somewhere between setting the orders, on load you read that variable and fix the orders of that vehicles.

[Brumi]

May I ask the reason why save/load was implemented in the API this way? So why isn't it possible to automatically save all variables, the call stack, the current place in the script and everything else needed, and return exactly the same state of the AI when loading?
I know you have some reason for that, I'd just like to know...

[Zutty]

As far as I know Sleep is automatically called with every DoCommand. (tell me if I'm wrong)
This means that a save might occur, for example, in the middle of giving vehicle orders, and after the load, there will be a vehicle sitting in the depot with incomplete orders. How can we avoid this?

This is somewhat unlikely to happen, but you could check for vehicles waiting in a depot on load.

In general I try not to worry about atomic operations. Its the integrity of larer scale processes that I spend my time on improving. For instance if your AI determined that a particular route needed 20 vehicles, but a save/load occurred after building only 10, then after the load there would be a "bad" service that does meet its initial requirements. A simple way to fix this would be to check the service periodically to ensure it has the correct number of vehicles. You could get this number by either...

A) Save that the service was supposed to have 20 vehicles
B) Derive the number 20 in the same way as before when the service was created

B is probably better as that number may have gone up or down due to production changes or whatever. Thats what my AI does.

Another example would be a route that is only partially built when a save/load occurs. When the AI starts again, it ought to be deterministic enough to choose the same route again. Even if it starts from scratch it will almost certainly follow the same paths (finding half of them built already) as it will be working from (roughly) the same basic state and the same random seeds.

[Rubidium]

I know you have some reason for that, I'd just like to know...

So another (slightly different) version of your AI is still able to load the savegame.

Also lots of the state is hardly storable in a way that is platform dependent and efficient. Not to mention that the virtual machine might be different in different version (compiling for 32 bits Windows and 64 bits Windows already does this).

Also MP savegames would STILL have problems with having done a docommand but the docommand was actually not executed.

[Blustuff]

So why isn't it possible to automatically save all variables, the call stack, the current place in the script and everything else needed, and return exactly the same state of the AI when loading?

With my AI the savegames would be > 10 mo since I'm a using huge graphs and Squirrel is not made to improve memory usage.

This is somewhat unlikely to happen, but you could check for vehicles waiting in a depot on load.

It is not when your AI spend all its time to build vehicles set orders. I think thats why Rondje om de kerk does.

A) Save that the service was supposed to have 20 vehicles
B) Derive the number 20 in the same way as before when the service was created

C) count the number of vehicles already built which could be a bit tricky.

However I would use a process based on real activity instead of inaccurate estimations which are good to design the route but not to rule it.

Another example would be a route that is only partially built when a save/load occurs. When the AI starts again, it ought to be deterministic enough to choose the same route again.

There is many reason why this wouldn't happen. On some experiments, my AI doesn't choose the same industry/path depending on non-deterministic parameters: the map changes and the amount of cycles dedicated to the AI during some in game period. A second one is that an IA may use a random generator to avoid the case where 10 AI build a route between the same town: sometimes we intentionally want to be undeterministic.

However, this problem is easily solved with DoAction lists which can be saved to be resumed.

[Dustin]

Save load isn't something users do a lot of. I think it's reasonable to expect slightly weird results when resuming a save game. As long as the AI can recover from the load without major disruption it should be fine. The player will probably never notice the AI has a bus with bad orders, or a train that just sits in the depot. They might notice a half built route.

If the AI is just generally resiliant and adaptive, loading saves and losing some little bits of state shouldn't be a big deal.

That said, it might be nice to have a way to actually save the full AI state for bug reports. Maybe not in a runnable form, but dump the object graph from squirrel. Heck, it would be nice for development to be able to pause the game and somehow examine the internal state of the AI.

it ought to be deterministic enough to choose the same route again.

Won't this make your AI behave poorly when running multiple instances?