Cargo Distribution

[OggVorbis]

I now and then play OpenTTD, always with cargodist. It's great! Games are a lot more fun with this patch. However one problem is bugging me every time: I usually create one very long two track railway with tens of stations and a lot of identical trains. The problem is that some trains don't fill up like they should in busy stations. These station have hundreds or thousands of passengers to one direction. Most trains in that direction fill up 100%, but some trains don't (these are identical trains with identical orders). Sometimes they leave the station almost empty! I experience this problem a long time already, and also in the last version I used (downloaded from cd branch 4th of March) it wasn't fixed. I can provide a savegame if you want.

[Eddi]

try to use shared orders as much as possible (goto -> ctrl+click on a train, or ctrl+clone train). just "identical" orders doesn't ensure proper balancing over the trains.

[fonso]

try to use shared orders as much as possible (goto -> ctrl+click on a train, or ctrl+clone train). just "identical" orders doesn't ensure proper balancing over the trains.

Wrong. Cargodist doesn't care if the orders are shared or not as long as they point to the same stations. I think you're confusing it with YACD. Can I get a savegame showing the problem, please?

[OggVorbis]

No problem. If you open the attached save game there is a train leaving the station (Brahill City, 1st track). The second coach has only 32 passengers instead of 40. The train leaving the same station on the 3rd track is a better example, this one leaves the station almost empty. While there are almost 500 passengers waiting to go the same direction.

[OggVorbis]

By the way, all trains are clones of each other.

[donchatryit]

I think I found a bug:

I have now had two games crash after c. 30 years. Neither are high traffic and they used different maps, which were different sizes. Only one used newGRFs. I have included the crash.sav, crash.dmp but can't add the crash.log here so I have pasted it as text below. Anyone got any ideas? Version is g4dd2678f-cd

I have logged the issue on flyspray but I wondered if anyone here had any ideas. Not sure whether downloading the latest Cargodist build would help given the lack of support for savegames.

Any help would be most appreciated. Crash.dmp and latest save are attached.


Crash.log
*** OpenTTD Crash Report ***

Crash at: Fri Mar 16 13:39:39 2012
In game date: 1965-07-14 (30)

Crash reason:
Exception: C0000005
Location: 004C0D45
Message: <none>

OpenTTD version:
Version: g4dd2678f-cd (0)
NewGRF ver: 12005807
Bits: 32
Endian: little
Dedicated: no
Build date: Jun 5 2011 04:19:21

Registers:
EAX: 00000000 EBX: 00000000 ECX: 00000000 EDX: 00000013
ESI: 06A42078 EDI: 054BC5E0 EBP: 0012F998 ESP: 0012F990
EIP: 004C0D45 EFLAGS: 00210246

Bytes at instruction pointer:
0F B6 43 08 56 8B 75 10 57 3B F0 0F 87 86 00 00 00 8A 55 14 8B 7D 0C 85

Stack trace:
00000000 00000000 0012F9FC 00452C34 054BC5E0 00000000 00000000 00000000
00000000 00000000 054BC5E0 00003E4E FFFFCDD8 00000000 FFFFCDEE 00003E60
00002BB7 00000000 004D1199 06A42078 00000001 00000001 054BC5E0 0012FAC4
0067E50F 00000000 00000008 0012FA18 00452A5C 054BC5E0 054BC5E0 00000000
054BC5E0 054BC5E0 0000FEFF 00453045 05499101 00000003 004EA965 00000000
054BC5E0 0012FA50 05499138 054BC5E0 004EAE52 00000000 000003F7 054BC5E0
0012FAD0 004508F6 00BB6348 00BB6357 00000000 FFFFFFFE 006542CA 004B9636
00000000 0000004D 000FBF33 000FB730 004B1DCD 00000000 004B1FCE 00000000
05D90020 004B2586 004B1DCD 00000000 004B1E1B 00000000 0000000C 004B2849
000FBFF3 00000000 000FBFF3 000FBFF3 0012FC14 0012FC04 0067DEA9 FFFFFFFF
0012FC14 0043B8AE 00BB6348 00BB6357 00000000 00000010 00C0152A 00000001
00B7E07C 00000489 00000000 0012FB58 757B522D 83F324C9 FFFFFFFE 7576FE61
75769D6A 00000000 00584BB6 00020548 0000000F 00000000 00000000 01199A34
75769D75 F69721C9 0012FC50 0012FC48 01199A20 FFFFFFFE 75769C78 00000000
0012FB34 0012FB94 0012FBA0 757B522D 83F347D9 FFFFFFFE 75769D75 75769F8D
01199A20 0000000F 00000000 75769F9D 00000000 00000000 00000000 00000000
0012FC08 77845C0E 0012FBA8 00000018 778445F0 756B9BB0 00000000 756B9BD9
F690F8D9 00BB6348 00BB6357 00BB6348 00000000 00BB6357 0012FC14 00587D17
02271FBC 00435138 00BB6348 00BB6357 0012FC14 00000000 0012FF78 004352A7
00BB6348 00BB6357 0012FC14 00000000 0012FF78 0012FF78 0067E5B8 00000000
00000025 00000000 0043BA9A 00000000 005857E0 00C01D54 00000000 0012FCDC
00000000 0012FC20 00BB6357 00BB6375 00000000 00000000 00020548 0000000F
00000000 00000000 00BB6338 00000257 00000034 0043AB69 0012FCF4 0012FDF8
00000001 010000D0 024A48A0 00000000 00000000 00000000 00000000 FFFFFFFF

Operating system:
Name: Windows
Release: 6.0.6002 (Service Pack 2)
Compiler: MSVC 1500

Configuration:
Blitter: 8bpp-optimized
Graphics set: OpenGFX (740)
Language: C:UsersJonathanDocumentsOpenTTD Cargodistlangenglish.lng
Music driver: dmusic
Music set: OpenMSX (96)
Network: no
Sound driver: win32
Sound set: OpenSFX (87)
Video driver: win32

AI Configuration (local: 0):
0: Human
1: MailAI (v20110726)
2: PAXLink (v14)
3: ChooChoo (v394)

Libraries:
FreeType: 2.4.4
ICU: 4.4.2
LZMA: 5.0.0
LZO: 2.03
PNG: 1.4.5
Zlib: 1.2.5

Module information:
C:UsersJonathanDocumentsOpenTTD Cargodistopenttd.exe handle: 00400000 size: 8321024 crc: 1B7689E5 date: 2011-06-05 03:23:34
C:Windowssystem32ntdll.dll handle: 777E0000 size: 1205064 crc: B7D44540 date: 2011-11-18 20:23:34
C:Windowssystem32kernel32.dll handle: 75670000 size: 892416 crc: 07915B76 date: 2011-04-12 16:07:38
C:Windowssystem32WINMM.dll handle: 73D00000 size: 189952 crc: FE5FC521 date: 2011-10-14 16:03:25
C:Windowssystem32msvcrt.dll handle: 76E80000 size: 680448 crc: 2BA89372 date: 2011-12-14 16:17:47
C:Windowssystem32USER32.dll handle: 75750000 size: 627712 crc: 80F0F3C5 date: 2009-04-11 06:28:25
C:Windowssystem32GDI32.dll handle: 76E30000 size: 297472 crc: 69625D80 date: 2009-04-11 06:28:19
C:Windowssystem32ADVAPI32.dll handle: 76BA0000 size: 800768 crc: D7864B03 date: 2009-04-11 06:28:17
C:Windowssystem32RPCRT4.dll handle: 75D20000 size: 784896 crc: 35185859 date: 2009-04-23 12:15:07
C:Windowssystem32ole32.dll handle: 76F30000 size: 1316864 crc: 35DF2E64 date: 2010-06-28 17:00:21
C:Windowssystem32OLEAUT32.dll handle: 75C90000 size: 563712 crc: 4F05A328 date: 2011-08-25 16:14:01
C:Windowssystem32OLEACC.dll handle: 73CC0000 size: 238080 crc: 49653460 date: 2011-08-25 16:14:01
C:Windowssystem32WS2_32.dll handle: 77100000 size: 179200 crc: 5D581120 date: 2008-01-19 07:37:09
C:Windowssystem32NSI.dll handle: 779E0000 size: 8192 crc: 10729B39 date: 2008-01-19 07:35:57
C:Windowssystem32SHELL32.dll handle: 75E70000 size: 11586048 crc: AEAF34AB date: 2011-01-21 16:35:22
C:Windowssystem32SHLWAPI.dll handle: 76A20000 size: 353280 crc: 9F70F81D date: 2011-01-21 16:35:22
C:Windowssystem32IMM32.DLL handle: 77910000 size: 114688 crc: 8D83D50A date: 2009-04-11 06:28:20
C:Windowssystem32MSCTF.dll handle: 76A80000 size: 807424 crc: 78787E19 date: 2009-04-11 06:28:20
C:Windowssystem32LPK.DLL handle: 779F0000 size: 23552 crc: 2E03F135 date: 2009-06-15 14:52:42
C:Windowssystem32USP10.dll handle: 75DF0000 size: 502272 crc: A57A818A date: 2010-04-16 16:46:48
C:WindowsWinSxSx86_microsoft.windows.common-controls_6595b64144ccf1df_6.0.6002.18305_none_5cb72f2a088b0ed3comctl32.dll handle: 743D0000 size: 1686016 crc: 3AB5D2D1 date: 2010-08-31 15:43:52
C:Windowssystem32SHFolder.dll handle: 6DBC0000 size: 7168 crc: 1CA14A5C date: 2006-11-02 09:46:13
C:Windowssystem32wdmaud.drv handle: 6F390000 size: 167424 crc: AE942463 date: 2009-04-11 06:27:12
C:Windowssystem32ksuser.dll handle: 6FF60000 size: 4608 crc: 4C647881 date: 2006-11-02 09:46:05
C:Windowssystem32MMDevAPI.DLL handle: 74760000 size: 150528 crc: 481F98E7 date: 2009-04-11 06:28:20
C:Windowssystem32AVRT.dll handle: 748A0000 size: 12800 crc: EB589AA0 date: 2008-01-19 07:33:47
C:Windowssystem32SETUPAPI.dll handle: 75B00000 size: 1591296 crc: A225D6A1 date: 2009-04-11 06:28:24
C:Windowssystem32WINTRUST.dll handle: 746E0000 size: 172032 crc: 0FB3BA99 date: 2009-12-23 11:33:29
C:Windowssystem32CRYPT32.dll handle: 74FB0000 size: 978944 crc: CE0B7499 date: 2009-04-11 06:28:18
C:Windowssystem32MSASN1.dll handle: 75110000 size: 60928 crc: D62CDC5B date: 2009-09-04 11:41:59
C:Windowssystem32USERENV.dll handle: 75530000 size: 108544 crc: CC1E5DD3 date: 2009-04-11 06:28:25
C:Windowssystem32Secur32.dll handle: 75510000 size: 72704 crc: DFC8B400 date: 2011-11-16 16:23:08
C:Windowssystem32imagehlp.dll handle: 75AD0000 size: 153088 crc: EEF49677 date: 2008-01-19 07:34:32
C:Windowssystem32AUDIOSES.DLL handle: 6F1C0000 size: 115712 crc: 9E346B3A date: 2009-04-11 06:28:18
C:Windowssystem32audioeng.dll handle: 6F150000 size: 397312 crc: 17666E45 date: 2008-01-19 07:33:45
C:Windowssystem32PSAPI.DLL handle: 770F0000 size: 12288 crc: D7B812C5 date: 2006-11-02 09:46:12
C:Windowssystem32msacm32.drv handle: 6FEE0000 size: 21504 crc: 28704D00 date: 2009-04-11 06:27:12
C:Windowssystem32MSACM32.dll handle: 6F250000 size: 71680 crc: A2E8F836 date: 2008-01-19 07:34:54
C:Windowssystem32midimap.dll handle: 6F3C0000 size: 17408 crc: 9FC56E7F date: 2009-04-11 06:28:20
C:Windowssystem32CLBCatQ.DLL handle: 76990000 size: 523776 crc: C2EBF5E6 date: 2008-01-19 07:33:52
C:Windowssystem32uxtheme.dll handle: 747C0000 size: 240128 crc: 5F29223E date: 2008-01-19 07:36:47
C:Windowssystem32dmime.dll handle: 68970000 size: 178688 crc: 3063C87B date: 2008-01-19 07:34:04
C:Windowssystem32DSOUND.dll handle: 67CE0000 size: 444416 crc: 2196E709 date: 2009-04-11 06:28:19
C:Windowssystem32POWRPROF.dll handle: 74980000 size: 98816 crc: F5FB120D date: 2009-04-11 06:28:23
C:Windowssystem32dmusic.dll handle: 6E920000 size: 101888 crc: 9C95E7AB date: 2009-04-11 06:28:18
C:Windowssystem32dmsynth.dll handle: 73C60000 size: 105472 crc: A572B2E6 date: 2009-04-11 06:28:18
C:Windowssystem32dmloader.dll handle: 6E910000 size: 38400 crc: 43D1875C date: 2008-01-19 07:34:04
C:Windowssystem32mswsock.dll handle: 74C80000 size: 223232 crc: 9C4248CA date: 2009-04-11 06:28:22
C:WindowsSystem32wshtcpip.dll handle: 74960000 size: 9216 crc: 10C50FC1 date: 2008-01-19 07:37:11
C:Windowssystem32msiltcfg.dll handle: 6F0E0000 size: 15872 crc: C5A6E3F3 date: 2006-11-02 09:46:07
C:Windowssystem32VERSION.dll handle: 74D50000 size: 20480 crc: 8C605EC4 date: 2009-04-11 06:28:25
C:Windowssystem32msi.dll handle: 72AC0000 size: 2241536 crc: 63F4B3D8 date: 2009-04-11 06:28:21
C:Windowssystem32SFC.DLL handle: 730E0000 size: 4608 crc: 5D5C8D17 date: 2006-11-02 09:46:13
C:Windowssystem32sfc_os.dll handle: 6FF90000 size: 38912 crc: 705BDC57 date: 2008-01-19 07:36:24
C:Program FilesGDHOohd.dat handle: 10000000 size: 7680 crc: 695E4B47 date: 2003-02-24 12:15:00

---- gamelog start ----
Tick 0: new game started
Revision text changed to r18968, savegame version 138, not modified, _openttd_newgrf_version = 0x10004a18
New game mode: 2 landscape: 0
Tick 0: game loaded
Revision text changed to g4dd2678f-cd, savegame version 205, not modified, _openttd_newgrf_version = 0x12005807
New game mode: 1 landscape: 0
Tick 12953: settings changed
Setting changed: vehicle.plane_speed : 3 -> 4
Tick 12953: settings changed
Setting changed: vehicle.plane_speed : 4 -> 3
Tick 12953: settings changed
Setting changed: vehicle.plane_speed : 3 -> 2
Tick 12953: settings changed
Setting changed: vehicle.plane_speed : 2 -> 1
---- gamelog end ----

*** End of OpenTTD Crash Report ***

Decoded stack trace:
[00] ??? 0x004C0D45
[01] ??? 0x00452C34
[02] ??? 0x00452A5C
[03] ??? 0x00453045
[04] ??? 0x004EA965
[05] ??? 0x004EAE52
[06] ??? 0x0050D61E

*** End of additional info ***

[fonso]

donchatryit: This is a really old version of cargodist you're using. It's from june 2011. As it was pretty stable and no other stable version was around until december I'll investigate the problem. However, chances are that the relevant code has significantly changed in the mean time.

OggVorbis: Probably that's version gae58bd4c-cd? I'll investigate it.

Thank you for the reports.

[OggVorbis]

Yes I now see in the window title it is version gae58bd4c indeed.

[donchatryit]

Thanks Fonso. I only used it as I could play my savegames with it. Looking forward to Cargodist hitting trunk and savegame capabilities being added.

[fonso]

If you open the attached save game there is a train leaving the station (Brahill City, 1st track). The second coach has only 32 passengers instead of 40. The train leaving the same station on the 3rd track is a better example, this one leaves the station almost empty. While there are almost 500 passengers waiting to go the same direction.

You're confusing something. As you can see in the screenshot below the two trains are going to Ronningpool. There are only 47 passsengers waiting for a connection via Ronningpool, 400 are waiting for a connection via Brahill City Centre and the ramaining 62 via Tonnley Airport. It's plausible that the 47 passengers arrived after the trains' departure so I don't quite see the problem.

[OggVorbis]

You are completely right Fonso! I was looking at "Station: planned" instead of "Station: waiting". Strange as I don't remember selecting it, in fact I don't even know what 'planned' means in the game (and I didn't know there were multiple views available!)
Thanks for having a look, and sorry for disturbing you.

[fonso]

donchatryit: Your bug is fixed in later versions. If you're willing to recompile you can apply the attached patch to g4dd2678f-cd. This will retain savegame compatibility and fix the bug.

[fonso]

Some updates ... I'm thinking about rewriting the whole stats collection framework. The way link stats are recorded at the moment has several drawbacks:

• Moving averages require revisiting all stats in fixed intervals and eat resources because of that.
• First collecting the stats in one data structure and then building the link graph as a different data structure is redundant and wasteful.
• I cannot map sequences of intermixed "no load" and "no unload" orders in the link stats data structure, while the link graph itself could handle them quite well.
• The same applies for branching and rejoining "no unload" sequences. If you have one train going A-B1-C with A and B1 being "no unload" and another one going A-B2-C with A and B2 being "no unload" the distribution of cargo between those two routes won't be managed properly.

So, my idea is to build the link graph directly inline and replace the moving averages by some system where I'd collect time of last update, "compression time" and capacity for each link. "Compression" would be dividing capacity by some factor and increasing "compression time" accordingly, so that capacity / (now - compression_time) stays the same. Like this I can visit the links in irregular intervals and still be able to time them out or get a useful capacity from them.

This looks like a lot of work. Specifically I need to efficiently split and join link graph components for this to work. However it has the potential of making cargodist faster while at the same time reducing patch size and supporting all orders.

For now it means that the support for "no unload", "transfer" and "unload all" won't be integrated in cd as reorganizing the code to meet my standards would also be a lot of work which would immediately go to waste if I rewrite the stats collection system.

[Core Xii]

I'm confused about a fundamental aspect of this patch. Say we have a coal mine that produces coal, and two power stations that consume coal. If we build a transportation network that connects both power stations to the coal mine, then the coal mine distributes its coal between the power stations somehow, for example 50% to each. Ok so far so good. But now what if we only connect one power station to the coal mine? Does the coal mine send all of its coal to that single power station? Or is part of the cargo lost as it wants to go to the other power station but can't? If the latter, then how does playing with cargodist differ from vanilla OpenTTD, where the best strategy is to transport all coal from the coal mine to the furthest possible power station to get the most profit?

[Arie-]

This patch indeed determines which industries YOU have connected, based on these known connections the patch requires you to send the cargo to various industries. YACD's approach is different, there on map generation the destinations are evaluated, so not by the network you design.

[ffpp]

In CargoDist, cargo only wants to go where it actually can go. That means when connecting only one coal mine to one powerplant all of the coal will be delivered to the power plant even if there is another one nearby. Demand is derived from connections.
(The other way around is done in YACD, imo. There cargo has its specific desired destination and if you can't get it there, it won't travel with you.)

If the latter, then how does playing with cargodist differ from vanilla OpenTTD

The biggest difference for using CargoDist vs Vanilla is that cargo will be 'smart' in determining its route to its destination, in that it will consider intermediate stops - as long as all sublinks from start to finish can transport it. It means that you can create complex networks for cargo to travel on, and it will find and use possible destinations as the network provides them.

I think if there are multiple possible destinations the choice which one to take is arbitrary. But if you imagine two destinations: one large town and a small one. Then the large town has more houses, therefore more cargo-acception destinations and the chance of somebody wanting to go to the large town will be bigger. Resulting in a non-equal demand on the two routes, as you would expect in real life as well.
On the other hand, if one coal mine supplies two power plants at once, the cargo should be indeed delivered 50-50, as both destinations are equal in principal.

[fonso]

ffpp: The big town will likely attract more passengers but not because it has more houses. It attracts more passengers because the station in the big town generates more passengers than the one in the small town. This is the trick about "symmetric demand". It tries to send equal amounts of cargo both ways. There is another factor in determining how much cargo wants to go to some destination: distance. If you have two power stations connected to a coal mine the one nearer to the coal mine will get more coal than the other one.

[donchatryit]

donchatryit: Your bug is fixed in later versions. If you're willing to recompile you can apply the attached patch to g4dd2678f-cd. This will retain savegame compatibility and fix the bug.

Thanks Fonso. Just need to go read the tutorial on applying a patch now!

[Eddi]

two random thoughts:

1. the congestion of the network should provide feedback on station rating [maybe replacing the "last visited" effect]
   for that, a "global congestion" (average over all links of a cargo) and a "local congestion" (average over all outgoing links from this station of a cargo) could be calculated.
2. to improve the handling/routing of "full load" orders, the "capacity" should not be measured how often a train arrives at the target station, but also how long it waits at the source station. because when you only consider the "moving" part of the orders, then usage will always be 100% of capacity, triggering all kinds of unwanted effects (like discussed previously with the rerouted cargo)

[fonso]

the congestion of the network should provide feedback on station rating [maybe replacing the "last visited" effect]
for that, a "global congestion" (average over all links of a cargo) and a "local congestion" (average over all outgoing links from this station of a cargo) could be calculated.

Something similar is already done. See https://github.com/fonsinchen/openttd-c ... a486475727

to improve the handling/routing of "full load" orders, the "capacity" should not be measured how often a train arrives at the target station, but also how long it waits at the source station. because when you only consider the "moving" part of the orders, then usage will always be 100% of capacity, triggering all kinds of unwanted effects (like discussed previously with the rerouted cargo)

I see the problem. Trains doing full load already "refresh" the following links in regular intervals. However, that might not be enough.