Cargo Distribution

[Kimby]

Is it still happening without conditional orders?

I removed the 2nd plane from the shared orders and made it to
1. Go to airport B
2. Go to airport A

With this change, it doesn't seem to load cargo going to C any longer.

[edit]
I found a piece of documentation that mentioned nondeterministic orders, eg "a conditional order where the next real stop can evaluate to at least two different stations." I was hoping the "no unloading" command would remove the second mentioning of airport B as a target, but I guess not

Maybe cargodist could ignore stations with the "no unloading" command set ?

Another option would be that cargodist introduces a new order "Load cargo for" that can be used to load cargo for the next hop without actually taking any further actions. Example in my case:

1. Go to airport B
2. Load cargo for airport A (loads cargo for A, but doesn't move to it)
3. If load percentage less then 20, goto step 6
4. Go to hanger airport B
5. Go to airport B (full load any cargo, no unloading)
6. Go to airport A

That last idea would probably help a lot of other people in more complex situations. You'd be able to repeat it so you can pick explicit destinations you want to serve. That's if it's something doable on the implementation side. It *sounds* easy, but there might be problems with it.

[fonso]

1. Go to Airport B
2. If load percentage is below 20, jump to step 5
3. Go to hanger Airport B
4. Go to Airport B, Full load any cargo, no unloading
5. Go to Airport A

What you're trying to do is 'if there's another plane loading cargo for A, directly leave, otherwise full load', right? That's a rather stupid idea in the first place as half of your planes are flying empty from B to A, but let's see what we can do to make that cargodist-friendly.

It's true that order 2 is nondeterministic. While the plane is at B with order 1 we don't know if it's going to A or B next. And it's actually strongly nondeterministic as that depends on what cargo it loads. If it goes to B next it shouldn't load any cargo headed for A, yet. However, then it will be empty when finishing order 1 and directly go to A. Cargodist cannot decide what to do there. I'm opposed to a "load for" order as you should either handle the distribution manually or leave it for cargodist. Anything in between will destroy the routing. Granted, there already are some ways to do that, but I don't want to introduce additional ones.

One thing to note is that planes and ships cannot visit any stations which aren't in their order list, even if the orders are nondeterministic. And even for trains and road vehicles most conditional orders can only evaluate to a fixed set of stations. So what I can try to do is further restrict the cargo to be loaded at nondeterministic orders in these cases. This could maybe solve your problem. I'll check if this is feasible. Another thing to note is that we don't have to consider further 'no unloading' stops at the same station when checking for determinism. However, if you have a nondeterministic unloading stop at the same station we really have strong nondeterminism as the previously loaded cargo will be rerouted there.

In any case I'll have to prevent counting the previously loaded cargo twice as 'sent' at the second stop. Otherwise you'll only get half of the planned cargo on that link. This is going to be hideously complicated and no one will understand it. Also it will break those nice "Go to A; Go to depot; Go to A; Go to B" order lists. At the moment the vehicle will unload at the first visit at A, then go to the depot empty and load at the second visit. Maybe I'll just leave it as it is.

[Kimby]

1. Go to Airport B
2. If load percentage is below 20, jump to step 5
3. Go to hanger Airport B
4. Go to Airport B, Full load any cargo, no unloading
5. Go to Airport A

What you're trying to do is 'if there's another plane loading cargo for A, directly leave, otherwise full load', right? That's a rather stupid idea in the first place as half of your planes are flying empty from B to A, but let's see what we can do to make that cargodist-friendly.

If there's lots of cargo available to go from A to B, but not from B to A, I don't mind planes flying empty from B to A. That sounds a lot better then not servicing the A to B cargo as fast as possible. I therefore fail to see why this idea would be 'stupid'. My planes make more money flying empty in one direction part of the time then by losing time waiting for cargo at B. Not to mention the horrendous penalties for late delivery when you finally get around to deliver the cargo that was waiting all the time at A.

It's true that order 2 is nondeterministic. While the plane is at B with order 1 we don't know if it's going to A or B next. And it's actually strongly nondeterministic as that depends on what cargo it loads. If it goes to B next it shouldn't load any cargo headed for A, yet. However, then it will be empty when finishing order 1 and directly go to A. Cargodist cannot decide what to do there. I'm opposed to a "load for" order as you should either handle the distribution manually or leave it for cargodist. Anything in between will destroy the routing. Granted, there already are some ways to do that, but I don't want to introduce additional ones.

My point is: why is order 2 nondeterministic when order 4 clearly states "no unloading" ? No matter what the result of the condition is, the first place the plane can unload is at A. Since both ends of the condition result in the same unloading station, it should be deterministic in my view.

One thing to note is that planes and ships cannot visit any stations which aren't in their order list, even if the orders are nondeterministic. And even for trains and road vehicles most conditional orders can only evaluate to a fixed set of stations. So what I can try to do is further restrict the cargo to be loaded at nondeterministic orders in these cases. This could maybe solve your problem. I'll check if this is feasible. Another thing to note is that we don't have to consider further 'no unloading' stops at the same station when checking for determinism. However, if you have a nondeterministic unloading stop at the same station we really have strong nondeterminism as the previously loaded cargo will be rerouted there.

Since my example was about planes I didn't mention it, but I do exclusively use "non-stop" orders as recommended for cargodist. I agree that not loading cargo for a destination that's listed nowhere in the orders list would be helpfull.

In any case I'll have to prevent counting the previously loaded cargo twice as 'sent' at the second stop. Otherwise you'll only get half of the planned cargo on that link. This is going to be hideously complicated and no one will understand it. Also it will break those nice "Go to A; Go to depot; Go to A; Go to B" order lists. At the moment the vehicle will unload at the first visit at A, then go to the depot empty and load at the second visit. Maybe I'll just leave it as it is.

In the example you give, you could simply get the desired result by using "Go to A (no loading); Go to depot; Go to A; Go to B" as you'd do in a 'normal' game without cargodist. Also, if you'd change things to only ignore the stops with "no unloading" to determine the destination, the command above would still work as it does now. The loaded cargo would also not be counted twice. In my case I'm not unloading the already loaded cargo, so it's only loaded once. In your example, no cargo would be loaded at the first visit to A.

I guess what I'm asking for that would fix this issue and some others I can think of, is that stops with the "no unloading" set, would be ignored to determine the next unloading stop. I'm franky unclear as to why that's not yet the case.

Another example: consider a coal train visiting several mines in an attempt to get some coal and when it gets that coal, would deliver it to some location like a power plant. The next stop for the coal of the various mines should in my opinion be the power plant, not some other mine where no unloading is done. I agree the "no unloading" should be specifcally set instead of simply assumed due to coal not being accepted in the station at the moment as to avoid surprises later on. Of course this example only becomes relevant when another coal train visits the same mines but delivers to a different station like a steel mill.

I consider conditional jumps to be a powerful tool. I also feel that OpenTTD without Cargodist is rather lacking. Add those two points together and you'll understand why I hope to get Cargodist play nice with the power of conditional jumps. I'm aware that this will not always be trivial, but in this case I honestly don't see where the problem is.

[fonso]

I guess what I'm asking for that would fix this issue and some others I can think of, is that stops with the "no unloading" set, would be ignored to determine the next unloading stop. I'm franky unclear as to why that's not yet the case.

Another example: consider a coal train visiting several mines in an attempt to get some coal and when it gets that coal, would deliver it to some location like a power plant. The next stop for the coal of the various mines should in my opinion be the power plant, not some other mine where no unloading is done. I agree the "no unloading" should be specifcally set instead of simply assumed due to coal not being accepted in the station at the moment as to avoid surprises later on. Of course this example only becomes relevant when another coal train visits the same mines but delivers to a different station like a steel mill.

I've actually tried to implement that, but it's impossible with the current link graph model. Consider the example with the coal mines. If I do it the way you propose I'd have a separate link from each coal mine to the power plant, right? Now what are the capacities of those links? Obviously we can't assign the vehicles entire capacity to each link as then the link graph would assume it can route n times the vehicles capacity to the power plant, with n being the number of mines. But the vehicle can indeed pick up a full load of coal at one of the stations and ship that to the power plant without taking any cargo at any of the other stops. So, what we'd have to model is "the sum of the capacities of the links from all coal mines to the power plant is the capacity of the vehicle". That, however, cannot be modeled with a simple graph. So I stick to the method of including all stops except for "go via" in the link graph as like that the "sum" rule is modeled appropriately. The same thing holds for vehicles loading at one place and only unloading at several others and any combination of that stuff, btw.

Now back to your original problem. There I can't decide when to count the cargo as "sent". This is a problem as the amount of sent cargo, together with the planned cargo determines the routing. Let's say you have a slightly more complex order list with a station D as order 6. Now there is cargo from D passing through B on those planes. So a plane arrives at B and loads with order 1. I have to count the cargo from D as "sent through B to A" now as the plane might not visit B anymore before going to A. If it does though, I have to remember that I have already counted the cargo it is carrying and not count it again. This could be done, in theory, but it'd be a special case for a very obscure feature. As soon as you allow the plane to unload at order 4 (and I'm sure someone will do that) I get all kinds of problems. The cargo will be rerouted so I shouldn't count even the newly loaded cargo as "sent" at order 1 but at order 4. However, if the plane skips order 4 I should still count it at order 1. I cannot decide that in advance so I'd have to cache the amount to be counted in case the vehicle doesn't come back to B. This is what I mean with "hideously complicated and no one will understand it".

[id10terror]

Please ignore my suggestion if it is not helpful(or very well thought through)

Maybe the vehicle orders items subflags (eg full load) need revamping in light of cargodist(disable ones that arnt good add ones that are?) maybe our requests should target the order lists and not the cargo??? i dunno, i dont have a coherent picture of the code in operation.... but i thought i'd throw this out there. I say revamping but i'd love the order to go some different colour or somthing, if i was being a tool and using non conditional options. I dont even realise when i'm doing it..... but some time after when i see tons a crap piling up all becomes apparent

Respect+ Fonso


Edit:
I say tool, but i mean only when i'm doing it, cause i'm not thinking that far ahead

[Kimby]

Let me see if I understand correctly how cargodist currently works using a simple example.

Consider a coal train with total cargo capacity 200 and these orders:
1. Go to mine A
2. Go to mine B
3. Go to power central C

From what I understand, currently cargodist would end up assigning a capacity of 200 on the A-B connection times whatever you use to determine the frequency of the service (which can fluctuate a lot if you ask me) and again similar for the B-C link. This despite the fact that coal doesn't get accepted at B and shouldn't be unloaded at all. I guess you either unload it anyway and reload the same cargo again or simply pretend the train is empty for determining the capacity. This way you correcty describe the trip the train is taking, but if mine A starts producing lots of coal, you can get in a situation that it always arrives full at mine B while still being considered as capacity from B to C.

Don't get me wrong: if this is how it currently works, I agree because the fact that mine B currently doesn't accept coal, is no indication for the future. I think our opinions only start to differ when we're talking about how to handle a slightly different case.

Consider again a coal train with total cargo capacity 200 and these orders:
1. Go to mine A
2. Go to mine B (no unloading)
3. Go to power central C

Based on what you said, cargodist seems to see this no different as the previous case at the moment. What I'm hoping would be possible is this:

1) When an empty train arrives as A, B is disregarded as destination due to the "no unloading" and a capacity of 200 times whatever frequency is used to define an A-C link. Let's say that on this visit, the train loads 95 coal and continues.
2) When the train arrives at B, its current remaining capacity is 105, so I would then consider it to contribue to the B-C link with 105 times the frequency of the trip.

I'm assuming here that checking the "remaining cargo space" is equally as possible as checking the "total cargo space". I suppose depending on the code, I might be a bit optimistic here.

The fact that the cargo capacity of the train when visiting B changes, should be no problem I believe, because:
- cargo capacities can change anyway by changing to amount of wagons the train is using (or by using a diffent coal wagon as is possible in some train sets)
- the frequency of the traintrip is also variable due to traffic situations, changing railroad network, changing the loc, maintenance in depots, ... and that factor is as much a part of the link capacity as the amount of cargo the train can carry. As I see it, that means the link capacity will need to be updated a lot anyway, so the fact that the cargo amount changes should pose no problems.

The way I propose does not correctly describe the train trip, but it does describe the current reality correctly. A setup like the above would guarantee mine A the maximum capacity towards the power central while the capacity available to B fluctuates depending on the output of mine A. This actually describes the reality from the viewpoint of mine B trying to get something sent to power central C, so I don't think this model is all that bad.

I can think of one possible "flaw" in the model: if you use a much more complex network feeding cargo to A and B with the intention to make it reach C, then the lower listed capacity of B might make feeders prefer to route cargo via A instead. But I assume routing cargo also considers distances and while A has a higher cargo capacity, it also needs more time to reach power central C. So in the end, it might still come up with a reasonably balanced distribution of cargo flows.

[malderon]

Hi everyone.

I have recently got back into OpenTTD and am very interested in playing with some sort of destinations implemented. I am trying to get my head around the way Cargodist works and am struggling!

A question to start with. Under cargodist is it possible to build a "realistic" (ie not point to point) train line running A->B->C->D and have passengers 'want' to travel from B->D, without having trains that omit C as a stop? I set up a simple test network and it seemed that passengers only wanted to travel from A->B, B->C, C->D.

[Eddi]

Yes, but it takes a while until the people notice that they can go B->D or A->D directly. also, you need "non-stop" orders, and the setting for passengers must be on "symmetrical".

[fonso]

First, to answer once more the question how capacity from station A to station B is determined at the moment: I take a moving average of the capacities of vehicles arriving at station B and coming from station A. The length of the moving average depends on the length of the links. This has proven to be quite accurate for most links if you just wait some time for the numbers to settle. The normal user doesn't change train cars on every trip and everything you can do automatically on a regular basis is factored in by averaging over some time.

This capacity number is crucial. It determines how much cargo is sent which way and this is the core of the MCF algorithm. Imagine you have one route connecting A and C via B and another one connecting A and C via D. Cargodist can determine how much cargo to send via C and how much to send via D by measuring the capacities of each route. If I accepted the measurement to be much higher than the real capacities cargodist would send all the cargo from A over the shorter route even if its actual capacity was not sufficient. The other route would stay mostly unused (except for cargo originating at the in-between station). Cargo would pile up at A even though the added capacities of both routes would be enough to carry it all. Obviously this would be a much more severe flaw than the problem you're facing.

I will think about changing the set of available order modifications for cargodist, though. This might be a good idea.

[Kimby]

@fonso: thank you for your explanation and your patience with me. I appreciate it

I think I see the problem with my model you describe: by having trains going from A to C via B and D and having no unloading orders for both those stations, you'd lose the information about which link to use You'd still have the information of B->C and D->C but that's of course currently not considered for a supposedly direct link A->C. I'm going to have to think about this one because I don't see an easy solution at the moment. Except of course sacrificing link balance quality ...

[Arie-]

I'm trying to host a server this weekend (never done that before). remarks:
- OpenTTD_CD_g93be88c8-windows-win32
- It's on a DSL connection
- newgrfs via bananas (i think all of them)
- 256*1024
- it's a (rather) slow computer AMD XP 1800+, so do not expect too much of it.

edit:Temporary down. Need to fix something.
edit2: Might someone read this in time: do I have to set up some parameter for debugging?
edit3: Running server with -ddesync=3

[fonso]

Setting up a proper debugging environment is probably no fun on windows and with such a weak machine. So just run it without. Thanks for hosting.

[Arie-]

Server currently down, there currently is leaking current somewhere, but we can't find the cause. This was posted using my lap-top and tethering.

edit:Sorry, it was the middle of the night and I didn't want to wake my flat mates to pull all their plugs. However, they still decided to wake me at 7:30 because they didn't understand what was going on, the b******! The was machine was leaking current, but I'll look into that later. The server is up and running again now.

[Kogut]

See http://bugs.openttd.org/task/4154

I think the ticket can be fixed via partial of cargodist inclusion
- cargo is in packets
- packets have sources
+ algorithm "is it allowed to load the packet of cargo on the station to the vehicle with orders"

[fonso]

I've experimented some and changed the algorithm for limiting the amount of cargo waiting at each station ("external ratings") specifically:

• The amount of cargo to be thrown away is multiplied by the number of next hops cargo is waiting for at the station.
• The absolute maximum cargo threshold is applied to absolute waiting cargo again, not to average waiting cargo per next hop.
• All other thresholds for throwing away cargo that are applied to average waiting cargo per next hop are halved, but ...
• The average waiting cargo is counted as (overall waiting cargo / (#next hops + 1)). Thus, if there is only one next hop (for example if we're not using cargodist) only half of the cargo is counted which corresponds to the halving of all thresholds. The difference between the real and the synthetic average gets smaller the more next hops there are.

This seems to work better then before. Maybe you can test it and give me some feedback.

[Arie-]

Hey fonso, just for my understanding, what are is the approach most feasible to trigger a de-sync:
1- large busy map with lots of vehicles running around
2- lots of clients connected all submitting their own commands
3- doing lots of different things, building vehicles; assigning schedules; building tracks;
4- all of these together

Approaches:
1- is easily possible, just continue to play a map for a long time
2- needs input of a lot of people at the same time
3- just continue playing make sure there is a steady income and invest what you can
4- effort and luck

[fonso]

Did you experience a desync? The point is, I don't know if there's a desync in my code or how to trigger it. The last time at openttdcoop we had a desync on a large and busy map. I couldn't debug it, though. Since then one or two desyncs have been found and fixed in trunk so it might as well be that we were seeing one of those. One way to increase the chances of triggering a desync is having lots of things happen in different combinations. This is why a large and busy map with lots of people playing at the same time might help. Otherwise it's just luck.

[Arie-]

No I had not, but I was just wondering how this could be achieved most probable. But I understand now that it's just most probably luck. I's write a small script to, might I turn off the server, when I turn it off, one can continue playing the same game when I turn it on again.

edit: I'm (again) trying to understand a part of the algorithm, because I wanted to understand the effect of distance on demand. But now I'm wondering: is there an error on the WIKI page? It there states: "maxDistance / (maxDistance - distance) / accuracy". But I would expect as, closer industries receive a bigger partion of the supplied cargo it would say: "(maxDistance - distance) / maxDistance /accuracy". My reasoning: closer industries have a lower distance: (maxDistance - distance) result in a higher number than for far away industries. Therefore "maxDistance / (maxDistance - distance)" result in a smaller fraction. I'm trying to make an example calculation in a google docs spreadsheet.

[Arie-]

My server is giving me issues immediately when connecting. The server doesn't shut down. What I did: I at one time saved the game while playing on the server, then continued as a single player game. Later uploaded the savegame, restarted the server with:

Code: Select all

F:\OpenTTD_CD_g93be88c8-windows-win32\openttd.exe -D -g "save/NO PSWD INC, 2029-08-14.sav" -ddesync=3

. Connected with the client and when moving around then the game almost immediately crashed.

edit: Not sure though, still testing but it seems that when I don't move the screen for 10 secs or so, and after that start moving around, nothing happens.
edit2: receive failed with error 10054 is something I read in the server.
edit3: ah 10054 is something network/physical layer related, not OpenTTD. (I think I understand that from what I've read)

[NekoMaster]

What revision of openttd is the most recent build (22 Nov) of cargodist in the openttd bundles based off of?