Cargo Distribution

[ChillCore]

I have a question about link creation. I'm doing passengers only atm and i have the following problem. There is a train connection and the train is doing the following

A-->B-->C-->D, it stops at every station. now there is a link created from A-->B, one from B-->C and so on. But there is also a link created between A and C, however there has never been a vehicle going from A with next destination C, only from A via B to C.

Cargodist uses tranfer orders as soon as it is enabled, else CargoDist would not function ...
As there are vehicles going from A to C over B, passengers can get from A to C. Therefore a valid link between A and C is created.

and because of the created link from A-->C there become a fair amount of passengers in A who want to travel to C straigth and don't want to take the train that goes via B. But because there is no straigth connection to C and so the passengers get 'to any station via C' after a while and won't dissapear

without a savegame that's pretty much impossible to diagnose! ...

I've done allot of diagsnotistics, maintaince is off so it's not because of visisting depots. i'm using non-stop orders, they are standard. I'm using chilli's patch pack and allot of grf's, so posting a save game is a bit difficult... also capacity is not a problem, the train only get's loaded like 5%, that's the strange thing about it.

Feedback is appreciated by everyone but fonso should not be bothered with old versions to continue improving and finishing CargoDist.
Can you reproduce this in a CargoDist only game please timopf ... else it would be apreciated to continue this discussion in my patchpack thread. Feel free to post a savegame there (where it belongs) if you want someone to have a looksie or recreate with CargoDist alone (recent binaries on the coopserver) and continue here ... else the discussion is going to get way too complicated with different sources.

The version I have is not quite up to date (distribution of cargo) and the linkgraphs I am still using is and old version that fonso has replaced with another method of representation in the meantime.
I do appreciate the efforts fonso has made to get this far but I just like the old linkgraph representation better as it is more clear what happens with what sort of cargo. This means that it is up to me to maintain that part and I may have made a misstake there ... (eg. not yet adjusted to (ignore) daylength, forgot to include a line while updating or maybe another of the 30-ish included patches is interfering with correct working of cargodist ...)

Thank you for understanding.

[Eddi]

You're wrong here, ChillCore, a link from A to C is only created when a vehicle went directly between those stations, not when it always stopped in B inbetween.

Things like this may happen with unscheduled depot visits or lost vehicles, when "go non-stop" is not used [i'm not sure if cargodist can handle normal (without non-stop) orders at all yet, theoretically the "auto-orders" which are in trunk now should allow this]

[ChillCore]

You're wrong here, ChillCore, a link from A to C is only created when a vehicle went directly between those stations, not when it always stopped in B inbetween.

You're right about me being wrong.

I did a quick test with my patchpack (my current version, not the last posted one) and created a loop A->B->C->D and the AC or BD link were not created.
Afterwards I added E and the DA link dissapeared after a while ... not immediately but it went away as it should ...

Will test my last posted version now ... and continue in the patchpack thread untill I have tested clean CargoDist.

[fonso]

Hey, I tried to read some of the source as to find on what moment the destination of a cargo packed is cleared. Altough I (think) understand most of what certain parts do, I haven't been able to find the relevant code pieces where that happens, probably also because of the lack of a good IDE atm. So I cannot help you find out what's wrong, maybe ChillCore knows where to look, else you should wait for fonso.

Whenever a link disappears due to timeout all cargopackets having that link as next hop get rerouted (see RerouteStalePackets). That doesn't necessary mean they get sent to "any station". As long as there is at least one more planned route they can take they'll do that. The most common reason for "any station" are flapping links that appear and disappear all the time. If a link disappears all plans using that link are also removed. That creates "dead ends" where cargo arrives but doesn't see any further plan due to the next link having disappeared.

No matter how I tweak the parameters (and I have done that quite a lot) there will always be people who
1. Complain about disappearing of infrequently served links and lots of cargo to any station.
2. People complaining about inactive links not timing out quickly enough and cargo for those links piling up in the mean time.

With the current architecture I can shift the complaints more to 1. or more to 2. or I can make you configure the timeouts yourself but obviously it all doesn't help. I will tweak the parameters further if someone shows me a savegame where the current setup is a real problem. However, I haven't seen many savegames around here lately - only the one from a.locritani which shows two bugs in trunk which have been fixed by now. And in most of the previous savegames I have seen the problem was at most a cosmetic one. If it says "any station" the cargo will still be picked up, you know. And if you only have one vehicle serving that station there isn't much choice of where to go.

And another note: It's well known that cargodist doesn't work too well with daylength the way they are combined in Chill's patch pack. So I'm reluctant to spend any time on investigating any problems with daylength and cargodist. That's just not my job.

[Creat]

Reading fonsos post just gave me an idea... Some frequent readers of this thread may be aware that even though I love Cargodist I still have my reservations about the way links disappear. In short it takes too long, especially for high-capacity links that are relatively short. Due to the lack of any idea (or even just any general approach) on how to make it better there wasn't much I could do or suggest to do about it.

Why I think the current implementation is problematic (at least in some situations):
When I start building my network I often change routes or at least add stations to routes. This then causes passengers to pile up and kills (or at least severely impacts) the station rating, reducing passengers and so on. In the first few years this can be quite a big problem. If you already have a working network it can still have a severe impact (even though it can hardly bankrupt you with the same ease). Say you have two large cities connected by a major train link and you decide to add a station in the outskirts of one of the cities (maybe because it grew so much), moving the first station that is visited by trains out a bit. Now since this is a high-capacity long-distance link it will take forever to disappear. You could leave a train or two running the old route and over time reduce the capacity until you finally can drop them entirely, but that's a lot of manual work and also prolongs the existence of the link you don't want anymore. There are lots of other examples I've experience where it can be problematic that links are created instantly and take quite a while to disappear.
While I can cope with all that because I understand how the underlying graph mechanic works, the average player can't be expected to know implementation details of an internal game mechanic just to be able to play. So for this to be included in trunk it should avoid situations an average user won't understand because he doesn't know some internal detail.

Now for the idea (finally! sorry I took so long to get here...):
Say we have a connection of stations A and B, whenever a train leaves A (going to B) the link between them is refreshed (or created) depending on the train capacity. Over time it is then decreased until it is either refreshed again or disappears. I would propose not reusing the link capacity as an indicator for the link's lifetime but instead tracking that separately. There are possibly many approaches for this, the one I'm thinking of now is as follows:
One could track the arrival times (or departure times, makes no difference) for each link (more precisely the intervals between arrivals/departurres). If for a certain period of time no train arrives (or departs) for that link it's removed. That period should be tied to some average of previous intervals, lets say multiplied by 3. One potentially problematic case would be if a link would only be visited exactly once, as there is then no reference time to multiply by 2 or 3 to get a removal time for the link. The additional information to be tracked is quite small (a few valued to create an average from plus a last-visited time for every link in the graph). Quite possibly the multiplier can be chosen relatively small, so links are rather removed too often than too late. Important: This removes the need of the moving average for capacities (even though it doesn't necessitate it's removal either). It ca still act as a fallback removal mechanism in case a link is really only ever used once and no interval can be measured...
Example: A train departs for B form A, creating a new link, noting the time-of-departure in the links properties. Once the next train departs the time difference of the departures is recorded, the last departure time of the link updated. The next departure is handled like that as well, now there are two entries for departure intervals. If no train departs at A for three times their average the link is deleted. How many of those intervals should be kept should probably depend either on distance or 'size' of the target station (possibly link size instead of station size) to compensate for likely fluctuations (breakdowns, signals, ...).

What do you think, would this be a good idea? I haven't done any exploratory calculations to see if this would actually reduce the link lifetime of links no longer in use, but I think it can be tweaked to achieve that (if it doesn't already). Also the implementation shouldn't be too complicated, once I'm done with my master thesis I might give it a go if nobody else has already by then, just to see how well it works and if it's a viable alternative

[Eddi]

that's one of the parts where cargodest had an (imho) better approach. the route network in cargodest was dependent directly and statically on the trains orders, not dynamically on their actual movement. that means, the appearance and disappearance of a link was instantly on the user interaction of giving the train orders.

[fonso]

Creat: Sounds nice, but requires double the amount of data for each link. I agree that tying link capacities to timeout intervals was probably a bad idea. Your idea improves the situation a bit by treating links more equally, but keeps the general conflict between 1. and 2. from my last post.

Eddi: Now that we have automatic orders this might actually be a viable idea - at least for bootstrapping new links and removing old ones. Like: If you add/remove/change/move an order this action is broken into a series of creations and destructions of "micro-links" for that vehicle. For each of those it is checked if such a link already exists or has any other vehicles serving it and this is translated into creating or tearing down links. Unfortunately at least for the "remove" case we have to cycle through all vehicles to check if this was the last one. Still, I like this one much more than Creat's idea. Maybe instead of checking for every add or remove we can check for dead links about once a month if any orders have changed since the last check. Then the effect wouldn't be instant anymore but the time it takes to calculate that stuff would be limited. And the time it takes for a link to disappear would always be at most one month while no link for which an order still exists would disappear. (uh ... what about vehicles parked in depots or stopped somewhere?) Oh, and we could also get rid of that godawful "freeze" thing like that.

In any case of course we cannot remove the moving average as we need it to measure the capacity of the link. We can't really make that a "monthly" value like for example supply at stations because quite often there are links that are served less than once a month.

Now let me find the time to finish the link graph overlay stuff and work through the rest of Rubidiums suggestions while still keeping cargodist up to date and when I'm done with that I'll implement Eddi's idea ...

[Eddi]

(uh ... what about vehicles parked in depots or stopped somewhere?)

either you keep those links, but eventually they will get a capacity of 0, or you consider these as non-existing, and have to update the link graph at start of the vehicle.

the latter one has the advantage that you don't create "stale" links for incomplete order lists while giving orders to vehicles stopped in depot.
example: you want to assign an order list A->B->C->B->A. when you add these orders sequentially, before you add the second entry B, you get a bogus link C->A which you need to delete again.

[Creat]

... when I'm done with that I'll implement Eddi's idea ...

That is obviously even better (a lot, actually!). I just remember that you had previously said that tracking orders wasn't feasible for one reason or another (don't remember that part) so I was looking for an alternative and I wasn't ware of any trunk changes that affected the orders... Obviously extracting the graph directly from the orders (which is after all their origin) is better than any variant of reconstructing them from movement or something

Unfortunately at least for the "remove" case we have to cycle through all vehicles to check if this was the last one.

If you're really worried that walking all orders of all vehicles (for potential link removal) will impact performance (I kinda doubt it...) you can easily shift that effort to a little more memory and only a few necessary iterations: Just keep a list of pointers to all the vehicles whose orders contain the link. The amount of memory is negligible (even with 500 links and 25 vehicles for every link you only need 100kb memory on a 64bit OS). You do need to check every link for pointers to a vehicle that has just been removed (as in decommissioned/deleted), but that is a significantly less frequent operation.

About handling vehicles in depots: If the links know which vehicles are responsible for their existence they can mark a vehicle in a depot as 'inactive', if all are vehicles are inactive the entire link becomes inactive (maybe without actually disappearing, but overriding it's own capacity with 0 or something). This also avoids oscillation from vehicles entering depots for a short time (for maintenance or if the user just adds a wagon or two). I don't know if it is known in the depot-entry-function if the vehicle will exit again pretty much immediately (maintenance).

Just some thoughts, hope they contain something useful

[Lollipop]

With the current architecture I can shift the complaints more to 1. or more to 2. or I can make you configure the timeouts yourself but obviously it all doesn't help

How about giving the user the option to manually drop dead links? Like Ctrl-clicking on a link in the station gui sets all packets to 'any station'.

I will tweak the parameters further if someone shows me a savegame where the current setup is a real problem

Depends how you define 'real problem'. In the case of passengers, I dont mind. In the case of 'other cargo class', try playing a game with ECS and its stockpile limits, where you have to chance drop destinations frequently...its pretty messy

[fonso]

Nice? Nasty?

[Arie-]

Nice, can't wait to try the first build, wanted to start a new game with CargoDist but may wait a little if there's no guarantee on backwards compatibility.

[jmurrayufo]

I like the GUI, but there is one small issue. I run with NewGRFs, and their have about 20-30 goods that have various longer names. Can the design have the option of dragging it out to show full names or something of the like?

[fonso]

OK, I have switched the default "cd" branch to the new smallmap and viewport overlays. The old setup with smallmap-stats and smallmap-zoom-in is still available as branch "cdo". There are still some minor flaws like the lines in the viewport overlay being too thin. In general it seems to work, though. You can find the viewport linkgraph legend in the dropdown under the smallmap button.

cd rewrites its history, again. Sorry. There isn't much I can do about that with my current setup of branches and dependencies. If mirror my git repository and have local changes you'll have to "git rebase" from cdo to cd eventually. I'll keep around cdo for some time to ease the transition.

[Eddi]

the legend looks nice. have you thought about a dropdown for selecting to colour the links by company/cargo/saturation/usage/other?

[TERdON]

that's one of the parts where cargodest had an (imho) better approach. the route network in cargodest was dependent directly and statically on the trains orders, not dynamically on their actual movement. that means, the appearance and disappearance of a link was instantly on the user interaction of giving the train orders.

I agree with Eddi as well. I have often played really large games (e.g. 2k X 2k) with somewhat slow vehicles (early buses and trains, or ships). Unfortunately this sometimes means that links timeout, although there are several vehicles on the route (e.g. in case of outage for some reason, there may be several months between vehicles). I've also seen it happen when I'm using full load orders, but the supply not keeping up, causing the link to break and the supply to decrease to nil. Such setups might not really be optimal but occasionally there is not much choice. I can't provide a savegame right now but if you really need one I could build some example. It would also be good if links which are not available anymore would not stay around for a long time.

Basically, there should normally not really be any vehicle that have orders that they don't perform (except possibly stopped ones in depots).

---

On a related note, that should be handled, is that when you setup a multi-segment route, where the delivered cargo is not accepted on the intermediate stations, it might take some time for it to setup. E.g. in the following setup, where the route A-B is by boat and B-C by train

A <--> B <--> C

significant time will be wasted driving around cargo with "no destination" back and forth on the boats between A and B, since the cargo will not leave the vehicle at B, since B does not accept it, and neither at A, because that is where it came from. Especially with fully loaded vehicles, this means that normally no cargo will ever be delivered over the link, but the vehicles always being fully loaded, no other cargo can go on them either, leading into a catch 22-situation for your nice new transportation link!

To fix this issue, would it be possible in cargodist settings to allow cargo (assuming no unload orders are given) without destination to get off-loaded not only at stations accepting it, but also at stations where a link to a station accepting it exists? At least if the cargo is traveling on a vehicle that will never ever reach an accepting station according to its orders?

This could also be subject to a patch setting, so that this behaviour is only switched on for players who explicitly want it.

[Creat]

... stuff ...

Both problems you describe/mention will be solved as soon as fonso implements that the link-graph is constructed directly from orders. At least it CAN be (depending on the implementation). I think it would be necessary to take accepted/provided cargoes at each station into consideration, this would mean that cargodist should notice by itself (instantly upon having setup all relevant orders) what to do with the cargo from A at B (in your described scenario).

[TERdON]

Both problems you describe/mention will be solved as soon as fonso implements that the link-graph is constructed directly from orders. At least it CAN be (depending on the implementation). I think it would be necessary to take accepted/provided cargoes at each station into consideration, this would mean that cargodist should notice by itself (instantly upon having setup all relevant orders) what to do with the cargo from A at B (in your described scenario).

The first issue will be resolved, yes. For the second one: Not necessarily, although the problem for sure will be less of an issue. You could have cargo already lined up at a station with "no destination" already before giving the vehicles their orders (or if changing orders so that links completely disappear, cargo may lose its destination since it got disconnected). The only walk-around I see would be to explicitly set the destination for every single cargo packet before loading it on the first vehicle it is transported on, which is quite different from how the current implementation works. (that solution would of course be okay with me as well!)

[alluke]

I have a bug to report. The zoom buttons (keyboard + -) don't work. I'm using gba921979 with OSX 10.4.

[Yexo]

I have a bug to report. The zoom buttons (keyboard + -) don't work. I'm using gba921979 with OSX 10.4.

Most likely this is due to a trunk bug introduced in r22094 and fixed in r22142. After updating please delete hotkeys.cfg to reset all hotkeys to their default values.