Cargo Distribution

Forum for technical discussions regarding development. If you have a general suggestion, problem or comment, please use one of the other forums.

Moderator: OpenTTD Developers

Eddi
Tycoon
Tycoon
Posts: 7424
Joined: 17 Jan 2007 00:14

Re: Cargo Distribution

Post by Eddi » 28 May 2014 01:20

no, because the acceptance limits are known only to the NewGRF itself, OpenTTD cannot access them, and thus cannot pass them along to AIs or Cargodist. The text in the industry window is the only indication of the limit, and it cannot be used for any calculations
You might not exactly be interested in Ferion, but if you are, have fun :)

Robbedem
Engineer
Engineer
Posts: 111
Joined: 14 Jan 2013 18:08

Re: Cargo Distribution

Post by Robbedem » 28 May 2014 13:25

Oh, thats a pitty. I'll ask in the ECS topic if it could be done from the NewGRF side.
If not, i'll just have to play without stockpile limits.

Auge
Route Supervisor
Route Supervisor
Posts: 388
Joined: 23 Oct 2006 02:07
Location: Berlin

Re: Cargo Distribution

Post by Auge » 29 May 2014 20:27

Robbedem wrote:Oh, thats a pitty. I'll ask in the ECS topic if it could be done from the NewGRF side.
Ahem ...
Eddi wrote:... the acceptance limits are known only to the NewGRF itself, OpenTTD cannot access them, and thus cannot pass them along to AIs or Cargodist.
Is there anything unclear?

Accentuation by me.

Tschö, Auge

huldu
Engineer
Engineer
Posts: 29
Joined: 14 Nov 2009 19:45

Re: Cargo Distribution

Post by huldu » 07 Jun 2014 21:16

Is there any place where you can download the more recent versions(compiled) of this beside the front page where they're over a year old? Maybe there isn't any version for 1.4.x but I got the impression that there were when reading some posts.

User avatar
planetmaker
OpenTTD Developer
OpenTTD Developer
Posts: 9331
Joined: 07 Nov 2007 22:44
Location: Sol d

Re: Cargo Distribution

Post by planetmaker » 07 Jun 2014 21:44

There is no more recent version, especially none for OpenTTD 1.4. Cargodist was merged into OpenTTD trunk itself before 1.4 was branched-off and released. And as such is part of all official OpenTTD binaries released since then. Just look into your advanced settings and switch it on.

huldu
Engineer
Engineer
Posts: 29
Joined: 14 Nov 2009 19:45

Re: Cargo Distribution

Post by huldu » 07 Jun 2014 21:55

Oh... I've been going through the advanced settings but I can't seem to find it. I'm using 1.4.1.

Sorry, I think I understand now, in Cargo Distribution the "manual" option is the default behavior of openttd and changing it to asymmetric or symmetric is enabling cargo distribution. Wonderful! Thank you.

Hafting
Engineer
Engineer
Posts: 102
Joined: 13 Feb 2014 11:22

Re: Cargo Distribution

Post by Hafting » 04 Jul 2014 11:45

Eddi wrote:no, because the acceptance limits are known only to the NewGRF itself, OpenTTD cannot access them, and thus cannot pass them along to AIs or Cargodist. The text in the industry window is the only indication of the limit, and it cannot be used for any calculations
I don't understand. Well, I see that the text description cannot be used by the game. But newGRFs are processed by openTTD, they don't live on their own.

When a train arrives at a station, OpenTTD will either unload it or not. So OpenTTD is clearly capable of accessing acceptance information. This is not the same as the 'limit', just a yes/no. But even that can help a lot. When cargo is generated, the distribution algorithm can check if the station it picked currently accepts the cargo. And if it does not, pick some other destination. An unlucky player may still get a few trains full of cargo that has to wait a while onboard the train - but at least the game won't generate more cargo destined for that place until the stockpile is processed.

A further refinement would be to lower the probability for picking a particular station, whenever it is found to be in the refusing state. Low-capacity stations will get less and less popular until they don't overflow any more. The distribution algorithm effectively discovers the capacity with time.

Ideally, the newGRF interface could be extended to provide such information directly. I guess that is a bigger project though.

User avatar
planetmaker
OpenTTD Developer
OpenTTD Developer
Posts: 9331
Joined: 07 Nov 2007 22:44
Location: Sol d

Re: Cargo Distribution

Post by planetmaker » 04 Jul 2014 12:31

When NewGRFs are involved, acceptance cannot be predicted. The NewGRF could decide to not accept on Friday 13th and Christian holidays, except in odd years, where it doesn't accept cargoes during Ramadan and Islamic holidays. And it can invert that rule when it finds NewGRF XY present.

Basically it works like that: when a train arrives (or maybe just regularily, would need to check when exactly), the code is run which decides on acceptance of the industries. It is purely determined by the NewGRF, called at that given time by OpenTTD to decide on what will happen. It cannot execute it at another time as the result simply might be different.

Alberth
OpenTTD Developer
OpenTTD Developer
Posts: 4746
Joined: 09 Sep 2007 05:03
Location: home

Re: Cargo Distribution

Post by Alberth » 04 Jul 2014 15:46

Hafting wrote:When a train arrives at a station, OpenTTD will either unload it or not. So OpenTTD is clearly capable of accessing acceptance information.
There is only one way to test acceptance, namely by actually trying it. To try whether the industry accepts, you need cargo. To get cargo, the train needs to be at the station.

You cannot get acceptance information in any other way.


It's like trying to visit a shop while you don't know if it's open. You have no phone number either.
The only way to find out if it's open, is to go there, and try whether you can buy something. If you succeed, then and only then you know you could buy something. That is however no guarantee that the next time you try (you run out the shop, turn around, and try walking in again) you will succeed again.

Rubidium
OpenTTD Developer
OpenTTD Developer
Posts: 3815
Joined: 09 Feb 2006 19:15

Re: Cargo Distribution

Post by Rubidium » 05 Jul 2014 07:25

Hafting wrote:I don't understand. Well, I see that the text description cannot be used by the game. But newGRFs are processed by openTTD, they don't live on their own.

When a train arrives at a station, OpenTTD will either unload it or not. So OpenTTD is clearly capable of accessing acceptance information.
This is where your thoughts go wrong. NewGRFs are in this case more like scripts, and OpenTTD asks that script whether the cargo is allowed at that moment. Is it, then it unloads, but when it isn't it doesn't. The script, however, can ask OpenTTD for some OpenTTD state information such as the date... but the NewGRF itself has some internal state too that it can use.

As a result a query OpenTTD does about the acceptance is: "will you accept this if, and only if, I deliver it directly, with no delay, after getting the answer". Even one tick delay and the answer to the question could be different. Actually, it could even be different for the next vehicle that tries to unload (which is why we can't run that code in parallel).

damerell
Traffic Manager
Traffic Manager
Posts: 188
Joined: 22 Feb 2008 23:09

Re: Cargo Distribution

Post by damerell » 08 Jul 2014 14:33

I think it's pretty clear that Hafting's proposal recognises that you might not always get the right answer. It's intended to make a best guess, in the hope that that will work with some NewGRFs where currently Cargo Distribution doesn't work at all. That seems like an improvement.

User avatar
fonso
President
President
Posts: 945
Joined: 13 Oct 2007 08:28

Re: Cargo Distribution

Post by fonso » 08 Jul 2014 22:07

damerell wrote:I think it's pretty clear that Hafting's proposal recognises that you might not always get the right answer. It's intended to make a best guess, in the hope that that will work with some NewGRFs where currently Cargo Distribution doesn't work at all. That seems like an improvement.
The problem is any heuristic the cargodist algorithm could apply to approximate if a station will be accepting a particular cargo in the future is bound to fail. There could be multiple industries near the station with different acceptance patterns. The same industry could be served from several stations. Someone could come up with a "Christmas Tree Shop" industry which accepts wood, but only in December. The possibibilities are endless and I think optimizing cargodist for the specific case of ECS would neither work very well nor be a good thing in the long term.

Industry NewGRFs with such behaviour try to force you into micromanaging your cargo deliveries. If you want to do that, fine. However, cargodist is probably the wrong tool for that as it tries to free you from the need to manage your cargo deliveries. Ultimately, you can't have it both ways.
The guy on the picture is not me, it's Alonso.

McZapkie
Tycoon
Tycoon
Posts: 1176
Joined: 18 Jan 2014 18:10

Re: Cargo Distribution

Post by McZapkie » 09 Jul 2014 11:40

Cargodist is a great patch for passenger, mail etc. transport, it is also useful for distributing goods, food and other supplies,
however its usability is questionable in case of primary freights, because some bad features.
For example, assume sources S1, S2 and recipients R1 and R2, located as following:

Code: Select all

R1       S1   S2   R2
and services are: S1--->R1, S2---->R1 and S2->R2.
In case of cargodist switched on, unwanted effect appear: cargodist try to transport huge amount of cargo from S1 to R2,
but the only way is from S1 to R1, from R1 to S2 and from S2 to R2, resulting in saturated vehicle capacity and very little payment for waste long distance haulage.

My first tough was to split cargodist manual/non manual options into:
distribution_pax
distribution_mail
distribution_armoured
distribution_default
distribution_primary

Another idea is to fix cargodist distribution algorithm - divide direct source-recipient distance by sum of all distances between hops.
Such coefficient can be used as penalty for assigning cargo amount.
It should work fine also in case of passenger traffic.
If you have two stations A1 and A2 in city A, and connect them to city B, but accidentally forgot to connect A1 and A2,
cargodist will saturate your A1-B and A2-B with passengers fro A1 to A2.
With proposed adjustment, unwanted number of A1 to A2 passengers would be suppressed proportional to the A1-A2/A-B factor.
My experimental openTTD server: 149.156.194.203:3979 non-standard client, now testing: JGRPP http://tiny.pl/ggnch
Projects: Reproducible Map Generation patch, NewGRFs: Manpower industries, PolTrams, Polroad, 600mm narrow gauge, preindustrial houses, wired, ECS industry extension, V4 CEE train set.
Addicted to freeciv longturn.

User avatar
planetmaker
OpenTTD Developer
OpenTTD Developer
Posts: 9331
Joined: 07 Nov 2007 22:44
Location: Sol d

Re: Cargo Distribution

Post by planetmaker » 09 Jul 2014 12:00

You miss one important aspect: you don't own the industries and you don't make the contracts. So if the (primary) industry decides that the better deal for them is offered by another industry, it's your task to accomodate transport. And not judge that they better deliver their ressources to the nearby industry which (also) demands their produce.

Hafting
Engineer
Engineer
Posts: 102
Joined: 13 Feb 2014 11:22

Re: Cargo Distribution

Post by Hafting » 09 Jul 2014 12:29

fonso wrote: The problem is any heuristic the cargodist algorithm could apply to approximate if a station will be accepting a particular cargo in the future is bound to fail. There could be multiple industries near the station with different acceptance patterns. The same industry could be served from several stations. Someone could come up with a "Christmas Tree Shop" industry which accepts wood, but only in December. The possibibilities are endless and I think optimizing cargodist for the specific case of ECS would neither work very well nor be a good thing in the long term.

Industry NewGRFs with such behaviour try to force you into micromanaging your cargo deliveries. If you want to do that, fine. However, cargodist is probably the wrong tool for that as it tries to free you from the need to manage your cargo deliveries. Ultimately, you can't have it both ways.
I am not sure the point is to "force micromanagement". Rather, it is to add some realism - you can't deliver all the ore in the world to the smelter located conveniently in the middle of the map. They have limited capacity (rates/stockpiles). So if you want to service all the mines, you have to transport to other smelters too. This forces the player to build a more realistic transport network - it'd still be nice if cargodist could manage the actual loading.

I see that my approach wasn't very good - especially if openttd cannot even do a "test unloading". (The code could probably pretend to have a train there to test delivery - but it might mess up the involved newgrf that won't expect it.) And, as others pointed out, the "answer" might be different when the train eventually arrive.

So I propose a different solution, that can solve this problem. The way I understand cargodist, is that it creates an amount of cargo/passengers with a specific destination. Currently, the destination must be reachable through existing routes, possibly with weighting based on capacities and congestion. After that, the cargo simply drift through the transport network until delivery.

The solution then, is to make an (optional) interface that a newgrf may use at the time cargo is created. If the newgrf implements the new interface, cargodist will ask "I have 50 tons of coal at station x, how much will you accept at station y?" And the newgrf can then make a decision and say how much. Might be all, nothing or some low amount. Cargodist can then try other destinations when it gets a "no". If the player cannot transport it, perhaps try another player. Or even drop that cargo if a destination cannot be found this time.

Those implementing "limited capacity" can then be compatible with cargodist if they wants to. They would accept or reject cargo at production time, instead of at "train arrival time". (No cargo will be rejected when the train gets there.) Which is realistic, producers do usually not ship anything until they have a customer. If they use stockpile based decisions, the difference is that some of the stockpile will now be in transit. The new interface basically lets producer and consumer make their deals - and then the player can try to profit from the logistics.

As for backward compatibility: If a newgrf does not support the callback, cargodist will simply assume "no limit" and ship goods as usual. If a new newgrf is used with an older openttd, it will simply do the acceptance at "delivery time" instead of at "production time".

McZapkie
Tycoon
Tycoon
Posts: 1176
Joined: 18 Jan 2014 18:10

Re: Cargo Distribution

Post by McZapkie » 09 Jul 2014 13:39

planetmaker wrote:You miss one important aspect: you don't own the industries and you don't make the contracts. So if the (primary) industry decides that the better deal for them is offered by another industry, it's your task to accomodate transport. And not judge that they better deliver their ressources to the nearby industry which (also) demands their produce.
If there is no direct link between nearby industries, there is no rational reason to send huge amount of cargo around the world and back.
Especially it is unfair to force unnoticed speditor to make such totally unprofitable haulages.
In my opinion, it is just an bad feature.

Of course it can be avoided, if only isolated direct links are carefully used. Therefore, above "better deal" arguing is groundless.
My proposal is for improving huge complex networks and avoiding unwanted effects.
The question is - cargodist was developed for greater enjoyment or greater annoyance of the gamer?
If the second question answer is "yes", back-seat forcing thousands of passengers travelling 500 tiles back and forth and paying for 20 tiles only, is a desirable feature ;)

Alberth
OpenTTD Developer
OpenTTD Developer
Posts: 4746
Joined: 09 Sep 2007 05:03
Location: home

Re: Cargo Distribution

Post by Alberth » 09 Jul 2014 17:37

McZapkie wrote:If there is no direct link between nearby industries, there is no rational reason to send huge amount of cargo around the world and back.
Especially it is unfair to force unnoticed speditor to make such totally unprofitable haulages.
In my opinion, it is just an bad feature.
And it is easily avoided by giving more careful orders. Just don't load at R1. You are trying to load stuff at R1, how is the game to know you don't want to actually load things????

(Imagine an empty train arriving a station, doors open, it's going where you want to go, but you are not allowed to go in!!)

Btw, can I invite you to take a look at the insane amount of shipping that is being done at our world from Asia and back? Shipping companies make lots of money on it, surely that isn't needed, is it?
McZapkie wrote:The question is - cargodist was developed for greater enjoyment or greater annoyance of the gamer?
If the second question answer is "yes", back-seat forcing thousands of passengers travelling 500 tiles back and forth and paying for 20 tiles only, is a desirable feature ;)
Don't see it as a bad feature. In huge complex networks, you get these flows that you didn't see coming. That's why it's called "complex" :)
Instead, accept that you are not in full control of the flow, but take up the challenge and change your network to adapt to the demand.

McZapkie
Tycoon
Tycoon
Posts: 1176
Joined: 18 Jan 2014 18:10

Re: Cargo Distribution

Post by McZapkie » 09 Jul 2014 20:15

Alberth wrote:And it is easily avoided by giving more careful orders. Just don't load at R1. You are trying to load stuff at R1, how is the game to know you don't want to actually load things????
Using modified demand algorithm. Amounts of allocated cargo should depend not on direct source to final destination distance, but on sum of all hops.
If it is not feasible to change, at least possibility of switching cargodist to manual for primary cargo would be welcome.
Alberth wrote:(Imagine an empty train arriving a station, doors open, it's going where you want to go, but you are not allowed to go in!!)
Btw, can I invite you to take a look at the insane amount of shipping that is being done at our world from Asia and back? Shipping companies make lots of money on it, surely that isn't needed, is it?
If you are referring to real world, please notice that in real world you must pay price for a real traveled distance, not for start to final destination distance.
In real world you must pay full price for travelling or shipment from Europe to Asia and back.
Of course, there are rare exception from this rule - sometimes you can travel from one city in Italy to Japan and back to another city in Italy,
and pay only little domestic fare - but it was just a bug in reservation system.
Cargodist reproduce this bug constantly :)

Alberth
OpenTTD Developer
OpenTTD Developer
Posts: 4746
Joined: 09 Sep 2007 05:03
Location: home

Re: Cargo Distribution

Post by Alberth » 09 Jul 2014 20:58

McZapkie wrote:at least possibility of switching cargodist to manual for primary cargo would be welcome.
I don't think this exists in the general case. How would you detect such a cargo? Note that cargo itself has no "this is primary" property. You only know industries, and what it accepts and produces. A reasonable criterium could be that an industry does not accept anything, yet it produces cargo. The cargo it produces would then be called primary cargo.

However, even several of the default industries (oil rigs for example, but there are more) accepts passengers, ie they accept another cargo, and thus are not primary by the above definition. (While we see passengers differently than products, remember that the game sees everything as just cargo, no special cases.)

You also run into edge cases like production of eg methane. You could take it out of the ground (which would count as primary industry), or you can produce it from waste of cities (ie it's a secondary product). So what is methane now?
(This is also why a "this is primary" property on cargo would fail, different industry newgrfs can have different ideas about what type a cargo is.)

In industry sets like FIRS (don't know what ECS is doing here), every industry chain is a cycle, there is no starting point or finishing point. It's a nice property since you can start with any connection. It also means there is no primary cargo at all by the above definition.

As you can see, for a single industry set it may be clear (until you get cases like FIRS), for the general case it's much more complicated as you have to take every possible industry set into account (not only the currently existing ones, but also all industry sets than may arise in the future).

So I believe auto-detecting what a cargo is, is a difficult to impossible problem.

There has been talk about ways to define explicitly for each cargo whether you want to route it with cdist or not. Technically it's quite doable. The puzzle is how to make a user fill in 31 yes/no questions, which are mostly tightly connected with an industry newgrf. You don't want to do that for every game is my guess. We have not yet found a much better way.
McZapkie wrote:Of course, there are rare exception from this rule - sometimes you can travel from one city in Italy to Japan and back to another city in Italy,
and pay only little domestic fare - but it was just a bug in reservation system.
Cargodist reproduce this bug constantly :)
Maybe the real world is buggy instead? :D

Rubidium
OpenTTD Developer
OpenTTD Developer
Posts: 3815
Joined: 09 Feb 2006 19:15

Re: Cargo Distribution

Post by Rubidium » 09 Jul 2014 21:12

McZapkie wrote:If you are referring to real world, please notice that in real world you must pay price for a real traveled distance, not for start to final destination distance.
I fairly regularly see odd routes for airplane trips that are way longer than necessary. For example, I've seen Berlin -> Amsterdam trips via Moscow being only a quarter of the price of direct flights, or about half for a flight via e.g. Munich or Copenhagen.

Right now... Berlin -> Amsterdam for next Friday by plane:
Direct flight EUR 560 and 312 NM -> EUR 1.79 / NM.
Via Rome EUR 279 and 1347 NM -> EUR 0.21 / NM.
Via Minsk EUR 281 and 1375 NM -> EUR 0.20 / NM.
Via Vienna EUR 396 and 812 NM -> EUR 0.49 / NM.
Via Copenhagen EUR 450 and 527 NM -> EUR 0.85 / NM.
Via St. Petersburg EUR 470 and 1669 NM -> EUR 0.28 / NM.
Via Frankfurt EUR 478 and 421 NM -> EUR 1.14 / NM.
Via Belgrade EUR 486 and 1302 NM-> EUR 0.37 / NM.
Via Dusseldorf EUR 501 and 349 NM -> EUR 1.44 / NM.
Via Riga EUR 553 and 1173 NM -> EUR 0.47 / NM.
None is as extreme as Europe <-> Japan, but still... getting 4.5 times the distance for half the money! What a bargain. In any case, there are 9 alternative routes that are cheaper than the cheapest direct flight. When booking it's over 30 different options before you get to the cheapest direct flight. That's not just a fluke... it's very real world behaviour.

For some reason they can simply ask for more money on direct flights. I remember paying about EUR 100 more for departing from Amsterdam directly instead of going to some airport in Germany, fly to Amsterdam and then have the exact same flight out of Amsterdam.

Post Reply

Return to “OpenTTD Development”

Who is online

Users browsing this forum: vrn and 10 guests