Reliability - how does it affect breakdowns? Test

[LaChupacabra]

Recently, in one of the themes, Simons Mith wrote:

(...) Reliability - of little consequence as long as it's not terrible. (...)

I was curious about what it really looks like and decided to do a little test...

General remarks:
Reliability in OTTD is random, it will be completely different for each vehicles, each time you create a new game. The maximum reliability for vehicles is 100%, the minimum is 75%. Reliability degrades over time when a vehicle is not serviced. It also lowers when parked in the depot, but when the vehicle drives out, maximum reliability is restored. When the vehicle is old, the maximum reliability decreases. To make it not too simple, the NewGRF authors can modify the rate of decline in reliability, then a vehicle with a higher randomly selected reliability value may actually be much more unreliable than the one with a lower value. I hope that none of the authors came up with the idea to use this mechanism...
I don't know the technical details of how it all works, but I decided to check how it looks in practice.

Test conditions:

• Straight, flat route, no depots on the road to service.
• After leaving the depot (maximum reliability), the train always loads up for 3 days and then starts its journey.
• Length of the route: 1000 tiles (for reduced breakdowns also 2000 tiles)
• Two locomotives of similar parameters with different levels of reliability (set 2cc Trainsmin NML 3.1), pulling wagons weighing 357 tons with a speed limited to 120 km/h:
  • HVLE V490 Maxima 40CC - 100% reliability, 4968 hp, 442 kN
  • DB 185 TRAXX F140 - 75% reliability, 5678 hp, 286 kN
• More than 100 trains with both locomotives for comparison.

Test results - normal breakdowns (1000 tiles):
1. Time to cover the route:
Locomotive 100%: 253-266 days
Locomotive 75%: 319-361 days
2. Number of breakdows on the route:
Locomotive 100%: 3-5
Locomotive 75%: 15-20
3. Reliability at the end of route:
Locomotive 100%: 65-68%
Locomotive 75%: 30-34%
4. First breakdown after:
Locomotive 100%: 56-125 days
Locomotive 75%: 10*-50 days
5. Distance traveled without breakdown:
Locomotive 100%: 244-540 tiles (average 355)
Locomotive 75%: 40*-214 tiles (average 130)

Comments:
*Once in more than 200 cases, the train broke down right after leaving the station, after driving less than 1 tile. Once during a game, a train with a reliability of 84% broke down during leaving the depot after service.
Generally, it is clearly see that reliability is of big importance in the case of normal breakdowns. Where the first 100% reliable trains were just starting to break down, the 75% ones sometimes had 3 failures behind them.


Test results - reduced breakdowns (1000 tiles):
1. Time to cover the route:
Locomotive 100%: 239-247 days
Locomotive 75%: 239-256 days
2. Number of breakdows on the route:
Locomotive 100%: 1-2
Locomotive 75%: 1-3
3. Reliability at the end of route:
Locomotive 100%: 69-70%
Locomotive 75%: 43-45%
4. First breakdown after:
Locomotive 100%: 72-198* days
Locomotive 75%: 72-181 days
5. Distance traveled without breakdown:
Locomotive 100%: 311-856* tiles (average 536)
Locomotive 75%: 312-768 tiles (average 534)

Comments:
The results for both locomotives turned out to be almost identical. Only 6 out of more than 100 trains with 100% reliability traveled a greater distance than those with the 75% parameter. Two of them covered almost the entire route without breakdowns (938 and 956 tiles), but it had almost no effect on the average distance covered without failures.

Test results - reduced breakdowns (2000 tiles):
1. Time to cover the route:
Locomotive 100%: 500-510 days
Locomotive 75%: 530-570 days
2. Number of breakdows on the route:
Locomotive 100%: 4-5
Locomotive 75%: 10-16
3. Reliability at the end of route:
Locomotive 100%: 37-38%
Locomotive 75%: 3-7%

Comments:
Initially, I planned tests at this distance for both reduced and normal breakdowns, but the first results of the "normal" - more than 150 failures showed that it didn't make sense.

Distance traveled without breakdown - image

[+] Spoiler

This map shows where the first failure occurred. The route starts in the upper right corner.

Conclusions:
A. Reliability is of great importance for normal breakdows
B. For reduced breakdowns copy-paste what Simons Mith wrote: "Reliability - of little consequence as long as it's not terrible."
C. There may be a bug in the code as vehicles with different reliability parameters should not give the same results.

If you want to check it out for yourself...

Reliability Impact TEST.sav

(76.23 KiB) Downloaded 38 times

[Simons Mith]

I've looked at breakdowns before, (about 10 years ago??, viewtopic.php?f=33&t=53692&p=937910#p937910 and in a couple of other threads too) and basically the % indicator is total balderdash.
If a vehicle has a 100% reliability score it ought to be 100% reliable. Because that's what "100% reliability" means. It won't be.
If a vehicle has a 99% reliability score it should spend 99% of its time working and 1% of its time broken down. Because that's what "99% reliability" means. So it should spend 3.65 days per year broken down, on average, and if it spends more or less time than that, then it ain't 99% reliable, and it shouldn't SAY it's 99% reliable.
A vehicle with 75% reliability should spend 3 months of every year broken down. Because that's what "75% reliability" means, in game terms.

Regardless of what breakdown setting you use, the breakdown percentage has only a vague bearing on the actual vehicle reliability you get. With reduced breakdowns, e.g. a ship with 20% reliability and a ship with 100% reliability are equally reliable. [Except for reliability decay, which will drop the 20% ship down to 19% at which point its breakdown rate will start to go up.]

That's my core gripe with breakdown %s. The number's a red herring. Why is that % symbol still present, even? I don't know whether the breakdown code has been touched in the meantime, but I still never trust the % indicator. Could we at least just call it just a 'vehicle reliability score' instead and ditch the %? That would be a bit less misleading.