How Does Cargodist Decide What Cargo Should Board a Train?

Got a problem with OpenTTD? Find some help here.

Moderator: OpenTTD Developers

Post Reply
paradise
Engineer
Engineer
Posts: 1
Joined: 24 Jul 2020 07:54

How Does Cargodist Decide What Cargo Should Board a Train?

Post by paradise »

Hi,

I couldn’t find this information on the wiki. Consider a station C which receives cargo from both stations A and B for transit. All cargo is headed for station D. Let C contain more cargo than can fit on the next servicing vehicle headed for D. How does Cargodist decide which cargo (i.e. from which origin station) gets to board the next vehicle?

Regards,

User avatar
jfs
Tycoon
Tycoon
Posts: 1230
Joined: 08 Jan 2003 23:09
Location: Denmark

Re: How Does Cargodist Decide What Cargo Should Board a Train?

Post by jfs »

Cargo in cargodist does not actually have destinations. The destination shown in the station window is a lie, the cargo only has a next hop. Whenever it arrives at a station every cargo packet decides whether that's it's destination or it needs to go further. If it needs to go further it decides whether to stay on the vehicle or needs to unload and transfer.
I'm quite sure cargo packets are just loaded from station to vehicle in first-in-first-out order, as long as the next stop for the vehicle matches the next hop for the cargo packet.

User avatar
MagicBuzz
Tycoon
Tycoon
Posts: 1264
Joined: 15 Feb 2003 17:32
Location: Villefranche-sur-Saône, France

Re: How Does Cargodist Decide What Cargo Should Board a Train?

Post by MagicBuzz »

jfs wrote:
24 Jul 2020 10:04
Cargo in cargodist does not actually have destinations. The destination shown in the station window is a lie, the cargo only has a next hop.
Doh! :shock:

So you mean that if I have a network with 4 stations A (source) B (transit) and C & D (destination) when I see in station A 50 cargo waiting with destination C via B, and 500 to D via C, then when I load everything to C, at C it can change to 275 to C and 275 to D ? :shock:

That may explain many things: each time I try to optimize service (in this exemple, by creating a new express line beteen A and D) it does not procude expected result (empty trains loosing money by exemple).

If this is the way cargodist works, then shouldn't the destination be removed from the station window? That's totally confusing and give us wrong information.
jfs wrote:
24 Jul 2020 10:04
I'm quite sure cargo packets are just loaded from station to vehicle in first-in-first-out order, as long as the next stop for the vehicle matches the next hop for the cargo packet.
Is it FIFO or FILO ?
I thougth it was FILO, as when many cargo accumulates in a station my vehicles continues to earn money, but some cargos are waiting for a long time and should not give any revenue. When I add new vehicle to take all waiting cargo, they get negative revenues, as if they were loaded only with old outdated cargo.

In anyway, I'm pretty sure cargodist is still :
- a black box noone can really explain (even developpers)
- fulfilled with many bugs or unexpected behaviour

I often experience non-sense when linking two lines : vehicles that were earning millions per month suddenly loose monney while they are 90% full and no cargo is actually waiting for a long time in stations.
But if cargo choose a new destination at each hop, that may explain many things, like packets travelling arround the whole map to finally arrive at a station beside the source one.

If this is the case, it's definately a bug IMO.

The opposite should be quite more :
- realistic
- predictable

=> Cargo choose a destination at source station
=> Then choose new "via" each time it arrives in a station

This way it would choose each time the faster path according to current service capacity/waiting cargo.

User avatar
jfs
Tycoon
Tycoon
Posts: 1230
Joined: 08 Jan 2003 23:09
Location: Denmark

Re: How Does Cargodist Decide What Cargo Should Board a Train?

Post by jfs »

The big issue I see with cdist is mainly that the algorithm can make cargo packets travel forever, I don't think there is a timeout on them at all.

User avatar
JGR
Tycoon
Tycoon
Posts: 2111
Joined: 08 Aug 2005 13:46
Location: Ipswich

Re: How Does Cargodist Decide What Cargo Should Board a Train?

Post by JGR »

jfs wrote:
24 Jul 2020 10:04
Cargo in cargodist does not actually have destinations. The destination shown in the station window is a lie, the cargo only has a next hop.
The cargo has both a source station and a next hop. The source station is important as this is what ensures that the cargo makes forward progress.
The per-source distribution graph is (instantaneously) acyclic.
MagicBuzz wrote:
24 Jul 2020 10:40
So you mean that if I have a network with 4 stations A (source) B (transit) and C & D (destination) when I see in station A 50 cargo waiting with destination C via B, and 500 to D via C, then when I load everything to C, at C it can change to 275 to C and 275 to D ? :shock:

That may explain many things: each time I try to optimize service (in this exemple, by creating a new express line beteen A and D) it does not procude expected result (empty trains loosing money by exemple).

If this is the way cargodist works, then shouldn't the destination be removed from the station window? That's totally confusing and give us wrong information.
No, the next hop is calculated at the time cargo packet is added to the station (either when goods are generated from an industry/house, or when unloaded from a vehicle). This is not a function of what cargo is currently waiting in the station.

The destinations shown are probabilistic, but asymptotically correct in the longer term.

When adding a new service, e.g. an express, the key thing to look at is the cargo distribution graph, not necessarily how much cargo is currently waiting at a particular station.
It is sometimes easy to overestimate the direct demand between two nodes, as opposed to the demand to/from the intermediary nodes.
It may also take a bit of time for new edges to become utilised.
MagicBuzz wrote:
24 Jul 2020 10:40
In anyway, I'm pretty sure cargodist is still :
- a black box noone can really explain (even developpers)
- fulfilled with many bugs or unexpected behaviour
To be honest it works surprisingly well, whilst still having reasonable performance.
The main issues are performance-related when calculating the distribution graph for large networks (e.g. > 1000 nodes).
There are some special cases around partitioning and removing stations, and such, but these are rarely encountered in practice.
Having the graph updated too often is also not necessarily ideal for mesh-type networks.
MagicBuzz wrote:
24 Jul 2020 10:40
But if cargo choose a new destination at each hop, that may explain many things, like packets travelling arround the whole map to finally arrive at a station beside the source one.

If this is the case, it's definately a bug IMO.

The opposite should be quite more :
- realistic
- predictable
Cargo doesn't choose a new destination at each hop. It chooses a new next hop according to the per-source distribution graph at that node. Packets do not travel around the map aimlessly.
Realism is limited by performance issues.
The more like a spanning tree your network is, the more predictable the distribution.
Ex TTDPatch Coder
Patch Pack, Github
Dad-Coder since April 2018

Avatar by MoonsongWolf

User avatar
MagicBuzz
Tycoon
Tycoon
Posts: 1264
Joined: 15 Feb 2003 17:32
Location: Villefranche-sur-Saône, France

Re: How Does Cargodist Decide What Cargo Should Board a Train?

Post by MagicBuzz »

Well, I insist, but IMO there is actually something wrong.

Look at my revenues : I was king of the world, then I connected west and east.
Now I'm in bankrupt!

All my stations are full, all my trains are full, and the only passengers/mails that unload produce negative profit.
Attachments
Narbonne Transport, 27th Jul 1885.png
(86.58 KiB) Not downloaded yet

Transportman
Tycoon
Tycoon
Posts: 2736
Joined: 22 Feb 2011 18:34

Re: How Does Cargodist Decide What Cargo Should Board a Train?

Post by Transportman »

MagicBuzz wrote:
24 Jul 2020 16:17
Well, I insist, but IMO there is actually something wrong.

Look at my revenues : I was king of the world, then I connected west and east.
Now I'm in bankrupt!

All my stations are full, all my trains are full, and the only passengers/mails that unload produce negative profit.
Problem is that your west edge is completely overloaded, so all cargo that wants to travel (part of) that will be waiting long times at intermediate stations. This will reduce the income for that cargo. On top of that, because the cargo is taking a significant detour if it goes from S to N and if it wants to go from the end of your west branch (that is near the centre of the map) to halfway your north branch (which is close-ish to the centre of the map), that will even more penalise your income. It actually doubly penalises you because the cargo has to travel the west edge, furthering the overcrowding problem. When you didn't have east and west connected, the cargo did not want to use those routes.

I think adding some of the missing links in your network (so the S to the E corner and the centre branches) and improve capacity on links that remain overcrowded with your current capacity, would already help you a lot.
Coder of the Dutch Trackset | Development support for the Dutch Trainset | Coder of the 2cc TrainsInNML

User avatar
JGR
Tycoon
Tycoon
Posts: 2111
Joined: 08 Aug 2005 13:46
Location: Ipswich

Re: How Does Cargodist Decide What Cargo Should Board a Train?

Post by JGR »

MagicBuzz wrote:
24 Jul 2020 16:17
Well, I insist, but IMO there is actually something wrong.

Look at my revenues : I was king of the world, then I connected west and east.
Now I'm in bankrupt!

All my stations are full, all my trains are full, and the only passengers/mails that unload produce negative profit.
This is an almost pathologically strange network.

As an example: the second closest station to Angers is Troyes, such that some significant fraction of traffic to/from there will be between those two stations.
The distance is fairly short, such that the eventual payout will be small, yet the network is structured to route traffic by the longest possible route around the other edges of the map.
This pretty much guarantees a huge loss for traffic between these stations.

I'd suggest:
  • Connection from Angers to Troyes.
  • Connection from St. Flour to Grenoble
  • Connection from Agincourt to St. Flour and/or Grenoble (via Montpelier).
  • Connection from St. Flour to Béziers
In general a 'C' shaped network is a bad idea.

The network should be structured to have reasonable routes between geographically adjacent stations.
This is a part of my current game, and is generally how I would structure the connectivity for moderate traffic levels.
JGR Transport Co., 2191-04-21#1.png
(444.23 KiB) Not downloaded yet
Ex TTDPatch Coder
Patch Pack, Github
Dad-Coder since April 2018

Avatar by MoonsongWolf

User avatar
L. Spooner Inc
Engineer
Engineer
Posts: 84
Joined: 22 Jul 2019 08:13

Re: How Does Cargodist Decide What Cargo Should Board a Train?

Post by L. Spooner Inc »

jfs wrote:
24 Jul 2020 11:43
The big issue I see with cdist is mainly that the algorithm can make cargo packets travel forever, I don't think there is a timeout on them at all.
Why doesn't it work like Usenet, where each message has a bang!path attached to it, so every node can see where it's been already? That way you can give it a TTL of X many hops, whereupon the cargo is unloaded and left for manual delivery by the player.

User avatar
jfs
Tycoon
Tycoon
Posts: 1230
Joined: 08 Jan 2003 23:09
Location: Denmark

Re: How Does Cargodist Decide What Cargo Should Board a Train?

Post by jfs »

L. Spooner Inc wrote:
25 Jul 2020 17:39
jfs wrote:
24 Jul 2020 11:43
The big issue I see with cdist is mainly that the algorithm can make cargo packets travel forever, I don't think there is a timeout on them at all.
Why doesn't it work like Usenet, where each message has a bang!path attached to it, so every node can see where it's been already? That way you can give it a TTL of X many hops, whereupon the cargo is unloaded and left for manual delivery by the player.
Each cargo packet has a fixed storage size, there isn't room to store an unlimited length path. And since a typical game can have hundreds of thousands of cargo packets alive at the any time, tracking the full path taken for every packet would have a large memory cost.

Eddi
Tycoon
Tycoon
Posts: 7469
Joined: 17 Jan 2007 00:14

Re: How Does Cargodist Decide What Cargo Should Board a Train?

Post by Eddi »

MagicBuzz wrote:
24 Jul 2020 10:40
Is it FIFO or FILO ?
it's probably neither. a cargo packet gets an ID upon generation (which may be on production, or later split from an existing package), and they're probably processed in order of this ID. not sure if this is the case for the cargodist "buckets".
but some cargos are waiting for a long time and should not give any revenue.
actually, cargo only ages while loaded inside vehicles, not while waiting at stations.

however, when stations are overloaded, cargo has used up capacity on previous links, but no final delivery money was ever awarded, so your profit calculation has accounted these transfer credits as income, but it hasn't shown up on your real bank account. this gets worse if the station rating drops below 50%, then cargo will disappear, taking all potential payments with them.

the negative transfer credits you see only highlight the previous inefficiencies of your network, you still earn real proper money.
=> Cargo choose a destination at source station
=> Then choose new "via" each time it arrives in a station
this was tried, turned out to be too CPU intensive on large maps.
You might not exactly be interested in Ferion, but if you are, have fun :)

LaChupacabra
Engineer
Engineer
Posts: 101
Joined: 08 Nov 2019 23:54

Re: How Does Cargodist Decide What Cargo Should Board a Train?

Post by LaChupacabra »

jfs wrote:
25 Jul 2020 20:59
Each cargo packet has a fixed storage size, there isn't room to store an unlimited length path. And since a typical game can have hundreds of thousands of cargo packets alive at the any time, tracking the full path taken for every packet would have a large memory cost.
Eddi wrote:
26 Jul 2020 13:49
=> Cargo choose a destination at source station
=> Then choose new "via" each time it arrives in a station
this was tried, turned out to be too CPU intensive on large maps.
Maybe it will be pointless, but maybe the cargodist should set a destinations and all the way to them in advance? A change in network load during transport would not change the selected route. But if one delete a station or connection, the cargodist should immediately search for another destination or route, and not after 3-6 months as is sometimes the case today, when a slight change in the orders (+ go via station) can block all traffic.

Regarding destinations setting, in my opinion these should primarily be cities or enterprises with existing connect (or not) to, and then for passengers and mail should stations be selected (including "any station in this city"). In the case of deliveries to enterprises, stations containing transfers in route should be ignored if there are direct connections to this enterprise.
[+] Spoiler
Problem with assignation a station instead an enterprise as a destination.
Cargodist bug 1-1.png
Cargodist bug 1-1.png (65.58 KiB) Viewed 259 times
Cargodist bug 1-2.png
Cargodist bug 1-2.png (152.24 KiB) Viewed 259 times
When it comes to the economic aspect and the sense of building transport networks, it would be good if the amount of cargo generated at the station to some extent depended on the number of connections offered. The rating of the stations could, for example, depend on it for 50%. The mere number of connections could be taken into account, or also the number of connected cities and their populations against the global value (20/100 = +10% for rating).
The fact is that extending the network is currently unprofitable in this game. The example of MagicBuzz may not be so clear, but it also shows that it is not worth combining separate connections into a larger whole. It is always much better to build separate connections A-B, C-D, E-F than to connect anything. This is quite demotivating. A few years ago, with Milek7, we created a script that was supposed to reduce this absurdity in a little roundabout way and ... it really worked. :D Cities that were included in the transport network developed much faster than those with separated connections. It was also not profitable to spread the stations range, as city transport also had a large impact on the cities development. As a result, a player who built more "realistic" networks could earn more and be competitive with someone who used simple game mechanics. For this purpose, the script used information about the origin of the cargo. Unfortunately, since version 1.9.0 it no longer works properly ... :(
I am sorry for may English. I know is bed.

User avatar
JGR
Tycoon
Tycoon
Posts: 2111
Joined: 08 Aug 2005 13:46
Location: Ipswich

Re: How Does Cargodist Decide What Cargo Should Board a Train?

Post by JGR »

Eddi wrote:
26 Jul 2020 13:49
MagicBuzz wrote:
24 Jul 2020 10:40
Is it FIFO or FILO ?
it's probably neither. a cargo packet gets an ID upon generation (which may be on production, or later split from an existing package), and they're probably processed in order of this ID.
Stations and vehicles are FIFO. Though cargo already on a vehicle is not unloaded unnecessarily.
LaChupacabra wrote:
26 Jul 2020 19:04
Regarding destinations setting, in my opinion these should primarily be cities or enterprises with existing connect (or not) to, and then for passengers and mail should stations be selected (including "any station in this city").
Passenger/mail traffic is not necessarily destined for a different town. Different stations in the same town are not equivalent.
The source/sink of passengers and mail is houses (and in rare cases industries) not towns.

I can see that it would make sense for the demand allocator to be aware of unconnected sinks, but it is not likely to be a trivial addition, and would likely add complexity.
LaChupacabra wrote:
26 Jul 2020 19:04
In the case of deliveries to enterprises, stations containing transfers in route should be ignored if there are direct connections to this enterprise.
Links do not have infinite capacity. It's not reasonable to ignore indirect routes.
LaChupacabra wrote:
26 Jul 2020 19:04
When it comes to the economic aspect and the sense of building transport networks...
Realistically the whole economy of OpenTTD doesn't make a lot of sense if you look too closely.
For me at least, I'm more interested in providing an efficient and complete service than I am in optimising the money side of it. I don't mind running some services at a loss for the "greater good".
LaChupacabra wrote:
26 Jul 2020 19:04
it is not worth combining separate connections into a larger whole. It is always much better to build separate connections A-B, C-D, E-F than to connect anything.
In generally iteratively expanding a single network makes more sense than creating silos which are merged later.

Slightly pedantic, but "more profitable" is not necessarily equivalent to "better".
Maximising for money leads to picking A and B in order to maximise income. There are plenty of savegames where towns or industries at opposite edges of the map are connected by perfectly straight point to point links.

It's worth bearing in mind that real "realistic" networks are not just about maximising direct profit either. Significant subsidies and other non-monetary constraints are often involved.
The subsidy mechanism seems like it ought to help here, though admittedly it is a bit toothless.
LaChupacabra wrote:
26 Jul 2020 19:04
A few years ago, with Milek7, we created a script that was supposed to reduce this absurdity in a little roundabout way and ... it really worked. :D Cities that were included in the transport network developed much faster than those with separated connections. It was also not profitable to spread the stations range, as city transport also had a large impact on the cities development. As a result, a player who built more "realistic" networks could earn more and be competitive with someone who used simple game mechanics. For this purpose, the script used information about the origin of the cargo. Unfortunately, since version 1.9.0 it no longer works properly ...
Is this script published anywhere?
Making it work properly is likely to have a much higher benefit per effort ratio than major changes to cargodist.
(That said, another player could simply make profitable point to point links using towns already boosted by another player's network).
Ex TTDPatch Coder
Patch Pack, Github
Dad-Coder since April 2018

Avatar by MoonsongWolf

Post Reply

Return to “OpenTTD Problems”

Who is online

Users browsing this forum: No registered users and 9 guests