What is OpenTTD today?

OpenTTD is a fully open-sourced reimplementation of TTD, written in C++, boasting improved gameplay and many new features.

Moderator: OpenTTD Developers

User avatar
JGR
Tycoon
Tycoon
Posts: 2557
Joined: 08 Aug 2005 13:46
Location: Ipswich

Re: What is OpenTTD today?

Post by JGR »

In the context of forking, "hostile" is an emotive word which doesn't mean much in practice.
There's nothing inherently negative about choosing to commit your code somewhere other than an original upstream.

Quality is an important issue, however at least in my eyes there comes a point where having something imperfect but still useful sooner is better than waiting indefinitely for perfection.
Where that point lies is obviously a matter of personal opinion and taste.
In my own patchpack and features, I'm prepared to take that principle significantly further than trunk would or should be inclined to.

Day length is an interesting one. It is a hack, but (for me at least) a very useful one. The implementation in my patchpack is good enough for me, and I'd rather play with that than not have it at all.
Other features are there for similar reasons.
ic111 wrote:Of course one could do this - but where is the point in forking a project where hardly any development happens, due to lack of manpower?
At its simplest, so that you have somewhere to commit to.
Development by posting ever more/larger patch files on a forum/bugtracker is unsatisfactory for a number of reasons.
Wolf01 wrote:I agree especially with the "give space to the new generation", provided the new generation will follow the guidelines and the spirit of the past devs.
Maybe this is just me, but I doubt it will work like that.
New developers will (and indeed should) adjust and reconsider guidelines as necessary for future development of the project.
The coding guidelines are not carved into stone.
Alberth wrote:So yes, you can add todays gimmicks in duct-tape and ply-wood, but you will pay with much more blood sweat and tears in a few years from now, as you realize it's a complete mess, and re-organizing it basically means taking everything out, and rewrite it from scratch, thus throwing away years of work, while spending even more years of work to fix the thing and get a solid foundation for the next features.
This is why adding most patches as they are now is a very bad idea.
At the risk of being judgemental, this seems a bit defeatist.
I would disagree on re-organisation implying a re-write.
Re-writes rarely go well, (I'd expect a hypothetical OpenTTD v2 to have major second system effect problems).
Iterative refactoring of the the existing codebase (with or without extra features applied) is quite achievable.
Redirect Left wrote:I personally feel this now needs to be dropped, and only support backward compatibility with a few versions ago of OTTD, instead of backwards compatibility going back years.
It has got to the point where this is holding things back far more than it gives to people.
I'm inclined to disagree. Savegame backwards compatibility is not that hard, and given that you'll always want to be able to load version N-1 savegames in version N, you'll inevitably end up implementing all the required conversion code, so you don't really save much development by having an arbitrary cut-off.
I for one intend to continue with full savegame backward compatibility support in my patchpack (and would encourage other patch/patchpack authors to do likewise).
Ex TTDPatch Coder
Patch Pack, Github
User avatar
Wolf01
Tycoon
Tycoon
Posts: 2016
Joined: 24 Apr 2004 10:43
Location: Venezia - Italia
Contact:

Re: What is OpenTTD today?

Post by Wolf01 »

Redirect Left wrote:In that case, just download a recentish version, save the game again, would work with a new one. Or just figure out what the changes are and make a seperate savegame patcher.

The current backwards compatibility stuff is just so limiting what people can do with trunk now.
I don't think so.

You'll get angry users which blame you because they need to find the exact version able to load the savegame.
You can't even provide all the backward compatible executables (like Starcraft 2 is doing, but with the API) because you won't like to distribute 2.5GB installers.
A savegame patcher still mean to maintain the saveload.cpp but outside OTTD, then what you have resolved?

Don't think that is the savegame compatibility which keeps the new features limited, because it's not. It's features which want to remove old things to make place to new things which are wrong, because old things should not be removed, maybe hidden but not removed.

If with OTTD 5.27.34325 I want to play a game the closest possible to TTDX (core game, I don't speak about the UI which could even be made 4D with speech control) I would be able to do it
JGR wrote:Maybe this is just me, but I doubt it will work like that.
New developers will (and indeed should) adjust and reconsider guidelines as necessary for future development of the project.
The coding guidelines are not carved into stone.
If this mean that OTTD will become Cities Skylines under the guide of new developers, I'll go buy that.
If these features could be added via newgrfs, I would be happy to help developing the support.
For example, what we are doing with NRT is that we aren't changing roads to make subways, hanging monorails, or pipelines or electric wires, that's an abuse grf developers are free to do, but not the scope of NRT. And NRT without grfs will make the gameplay exactly as before.
User avatar
andythenorth
Tycoon
Tycoon
Posts: 5656
Joined: 31 Mar 2007 14:23
Location: Lost in Music

Re: What is OpenTTD today?

Post by andythenorth »

Redirect Left wrote:The current backwards compatibility stuff is just so limiting what people can do with trunk now.
To add detail, the issue isn't the code for savegame support, that's just work. The issue is backwards compatibility with people's expectations/pet features. For example
- three implementations of signals
- three pathfinders
- two economies
- newgrf support - the drama when we broke depot-flipping of trains for example :roll: :?

There are also the non-original features that are established but are poorly designed or poorly executed, with the benefit of hindsight ;)
- confusion of auto-replace and auto-upgrade
- half-assed groups implementation
- timetables complexity
- conditional orders complexity

I have recently read all ~840 issues in Flyspray (the OpenTTD bug tracker), and have closed about 200 of them, and am going to close more, so I'm getting a good handle on what the state of bugs/patches/feature requests is there. Other than clearing dead wood though I have no suggestion to move anything forward. :)

I am pretty much +1 to JGR's comment. A fork isn't intrinsically hostile (github shifted the meaning of the word in the last 10 years from an aggressive split to just a normal approach to development). The nice thing is, it doesn't even matter whether we all agree or not, the code is there and can be used in any way that complies with GPL. So eh ;)
ic111
Director
Director
Posts: 608
Joined: 17 Jul 2007 17:56

Re: What is OpenTTD today?

Post by ic111 »

Redirect Left wrote:In that case, just download a recentish version, save the game again, would work with a new one. Or just figure out what the changes are and make a seperate savegame patcher.

The current backwards compatibility stuff is just so limiting what people can do with trunk now.
Hm, I had quite a lot of contact with that piece of code in my patches, but I never had the feeling that this (the support for older savegames) is a major obstacle.
JGR wrote:In the context of forking, "hostile" is an emotive word which doesn't mean much in practice.
There's nothing inherently negative about choosing to commit your code somewhere other than an original upstream.
This is right.

For me, this is not a question of hostile or not hostile, it is simply a question of, in which setup do we have as little administrative work as possible (e.g. things like merging things between branches, etc.)?
ic111 wrote:Of course one could do this - but where is the point in forking a project where hardly any development happens, due to lack of manpower?
At its simplest, so that you have somewhere to commit to.
Your assumption here is that the situation with devs having less and less time for OpenTTD, and with no new people becoming active developers eventually with commit access, persists. In that case, at some point I agree with you.

However, given a situation, where retiring people seem to be the problem, IMHO from an overall perspective adding new life to trunk sounds more senseful than forking.
Development by posting ever more/larger patch files on a forum/bugtracker is unsatisfactory for a number of reasons.
Oh yes, you are really right with this.

(says someone, who maintains patch queues with 30 or 40 single patches, and in the order of 4000+ lines of code)

Actually, concerning my patches my impression is, that if I ever implement more features on top of them, then I need to commit the code somewhere before doing that.

So yes, I agree, if you implement a patch queue because you have another extension in mind that needs the patch queue as basis, then the current situation with patch queues staying patch queues for a very long time is quite an obstacle.
Maybe this is just me, but I doubt it will work like that.
New developers will (and indeed should) adjust and reconsider guidelines as necessary for future development of the project.
The coding guidelines are not carved into stone.
Well, IMHO the truth is somewhere in the middle. You shouldn´t make all mistakes that have been made years ago a second time, but of course you might and will change some things in such a situation.
To add detail, the issue isn't the code for savegame support, that's just work. The issue is backwards compatibility with people's expectations/pet features. For example
- three implementations of signals
However, if you would change this, you would break savegame support - not in a savegame-technical sense, but in the sense, that if one opens such an old game, and the corresponding signal type is no longer available, certainly nothing senseful happens - and I doubt that converting those signals is possible automatically.
I have recently read all ~840 issues in Flyspray (the OpenTTD bug tracker), and have closed about 200 of them, and am going to close more, so I'm getting a good handle on what the state of bugs/patches/feature requests is there.
... where the question is, how reliable is that source of information concerning patches?

I personally discussed my patches mostly within the forum, probably, at least for the Rainfall River Generator, you will not even find a bug at Flyspray.
User avatar
andythenorth
Tycoon
Tycoon
Posts: 5656
Joined: 31 Mar 2007 14:23
Location: Lost in Music

Re: What is OpenTTD today?

Post by andythenorth »

ic111 wrote:... where the question is, how reliable is that source of information concerning patches?
Extremely poor imho :)

Many projects are making github work for them well, but that hasn't found any adoption here currently ;) . Also no technology will solve the social problem.
User avatar
Darkvater
Tycoon
Tycoon
Posts: 3053
Joined: 24 Feb 2003 18:45
Location: Hong Kong

Re: What is OpenTTD today?

Post by Darkvater »

Alberth wrote:There was a long discussion on IRC last evening, I didn't follow it all, but what I got out of it is the following.

[ As people unfortunately attach all kinds of conclusions to anything I write due to my label "OpenTTD developer" here,
| I have to add a disclaimer, that I speak only for myself, and not "the devs" or so (a commonly imagined entity that doesn't
| exist in my experience). I am only stating my opinion, where the biggest difference between (most of) you and me is that
| someone was stupid enough to give me commit access to the OpenTTD repository many years ago. ]

What everybody wants from OpenTTD is to be a reliable, long-living, stable program. A second thing that we want is backwards compatibility, it can read files that are more than a decade old, all your old savegames still work today. To a developer, that means the program must be maintainable and robustly built. You want the fundamentally proper solution to a problem, since that is the best way to ensure stability and reliability.

There is another thing that we want, namely we want the program to be extendible. If someone finds a proper solution to daylength eg, it should be added. This also points to fundamentally correct solutions, since these capture the core mechanism well, and are most likely to be a stable foundation to build on in the future.


OpenTTD has developed over 10+ years, and a great many things were added. This always starts with the simple things being done first. This is natural, nobody had a big (yet realistic) vision 10 years ago that we would want daylength, or infra-structure sharing, or a new map-array today. During those 10 years, the developers aged, and their life changed (ie they got out of college, and got a full-time job). The effect of this is that
- There are no small extensions left any more
- The amount of stuff around the program has grown (servers, build farm, wiki, bananas, announcements, news, social media, NML)
- The number of developers decreased from ~10 to ~1 (and I count for around 0.1 at most)

- For emphasis, there is ~1 developer, doing everything in his spare time only.

No wonder things get slow, or announcements fall off the todo-list. I agree it looks sloppy, but with so many things to do, and so little man-power, what would you expect?


The topic here is the new features that seem to stick around in patches forever. It is true we are not well organised in reviewing patches. The biggest problem with almost all of them however, is that they're, frankly, duct-tape and ply-wood. Patches like signals on bridges extend on a barely existing foundation, to the point that it's about to collapse under its own weight. It's fine for prototyping, but not a robust solution that will be extendible into the next decade to whatever we invent in 10 years from now.
Authors of most patches are interested in the feature and only in the feature today. In particular, they have no interest in building a robust solution that is future proof. So when our review comment says "great feature, but we need a solid foundation under it", people simply disappear, once they realize that "solid foundation" means a year or more work before you get to the point of adding the feature.

So to me, reviewing patches is useless. They don't follow trivial requirements like coding standards, and when something looks interesting enough, the solution is always the wrong approach (which is fair enough, I do that too, and need several iterations before I understand the correct solution).
If a comment about coding standards doesn't scare the patch author away, then asking for a proper solution is a sure kill in my experience.


It may look like a bad policy, and we should accept less finished patches. This happened in the past. To get an idea of what happens then, sketch for yourself how you would want to start a new game. Which choices do you need to make, in what order? What information do you need? where can you find that information? Then look at how it is implemented. Does the navigation between the different windows make sense, is it logical? It's not even close, was my conclusion.
As another example, does autorenew versus autoreplace make sense to you as it exists today? Is this how you'd want to have it? Is it logical how groups interact with it? I think not, and groups are totally useless to me.

I gave two examples that are much interface oriented, because they are easy to understand for many users. Nobody ever designed these feature from a global vision, from a user point of view. So you get little corners and hooks all throughout the program that absolutely make no sense if you look at it from the outside. Right now, the "small extensions" are not small anymore, and make no sense at all. If you want to move forward to the next decade, this stuff has to be taken out and rewritten properly.

So yes, you can add todays gimmicks in duct-tape and ply-wood, but you will pay with much more blood sweat and tears in a few years from now, as you realize it's a complete mess, and re-organizing it basically means taking everything out, and rewrite it from scratch, thus throwing away years of work, while spending even more years of work to fix the thing and get a solid foundation for the next features.
This is why adding most patches as they are now is a very bad idea.


A final problem is that the pool of developers seems to have dried up. People capable of developing OpenTTD further prefer to do other things instead.


So this is what I make of todays state of OpenTTD. It's not you against us or so, development is just increasingly more complicated with a decreasing amount of man-power, and an increasing diversity of wishes and desires.

The 1,000,000 dollar question is, how do we turn around the tide??
Unfortunately, I don't have the answer.
Alberth, I completely agree with you. We had exactly the same situation back in the old days when people created patches that was not much more of a hotch-potch of hacks; and not were like that. Many contributors got upset and left, a few stick around and genuinely took our feedback on board. Those patches got applied. Looking back it was still the right decision to make.

Especially looking back from years of professional experience where you have to maintain (legacy) software you simply cannot afford to not to put the right solutions in. I do not see why it should be any different for open source projects, in fact even more so! At work you at least get paid full time to fix s***.
Nobody wants a game full of features that crashes every hour or loses your old saves. Keep up the good work Alberth!

Sent from my Mi-4c using Tapatalk
TrueLight: "Did you bother to read any of the replies, or you just pressed 'Reply' and started typing?"
<@[R-Dk]FoRbiDDeN> "HELP, this litte arrow thing keeps following my mouse, and I can't make it go away."
User avatar
Dave
Moderator
Moderator
Posts: 17243
Joined: 26 Dec 2005 20:19
Location: North London

Re: What is OpenTTD today?

Post by Dave »

Without stripping back too much of the interesting discussion ongoing - is the answer "OpenTTD is a project now developed by adults with busy lives, in some cases partners and children"? Ten years is a long time in any respect, and inevitably people, trends and ways of doing things move on.

The forum as a whole has changed pretty radically since those heady days between 2007 and 2010, and although I'm not close to OpenTTD's development, I'd wager a reasonable amount on the project following a similar pattern too.

Why do we now only have the equivalent of 1 developer when once we may have had 10? Well ten years ago the cohort of people that made up that 10 would have been mostly in their late teens, at the start of their careers, often at university or between work. Ten years on, they have mortgages to pay, children to feed and busy lives at home and at work.

Maybe this is a controversial point, and it's already been partly raised and refuted - but could OpenTTD have reached its natural conclusion? How long was this meant to go on for? How many people saw the birth of OpenTTD in 2004 and thought they'd still see any semblance of updating and development 13 years later? Maybe viewing the game in the context of the original TTD would be useful... that was released in the mid 90s and we are still talking about it 22 years later because of Patch and Open - without either of those things, would TTD have eventually faded from people's minds? Most likely.

It is a great achievement, what we have seen here, and I expect a healthy following will always maintain an interest. But various personalities have faded into history and with them went the exciting cutting edge stuff [mainly because it's done], leaving talented and methodical developers to fine tune a programme which is now very much mature.

To answer your question - OpenTTD today is an old game which has been through tens of thousands of updates and changes. Games have come and gone in its lifespan. If "dying" is a slow wane in popularity then I don't think we could have expected much else. Nostalgia and emotive connections dictate we might mourn the changing way it's viewed, of course, but I'd stake another bit of cash to suggest the timeline of OpenTTD could be a case study of how communities around open source products change over time.

Rambling a bit - it is 2am after all... but that's what I think. Nothing lasts forever, and the end has no end!
Official TT-Dave Fan Club

Dave's Screenshot Thread! - Albion: A fictional Britain
Flickr


Why be a song when you can be a symphony? r is a...
_dp_
Transport Coordinator
Transport Coordinator
Posts: 277
Joined: 18 Dec 2013 12:32

Re: What is OpenTTD today?

Post by _dp_ »

Have you considered easing up on savegame compatibility? Meaning you'll still be able to open old saves but game is not guaranteed to work as close to an old one as possible. For example you can drop original acceleration, just all savegames will be converted to realistic one. Yes, that will break some tick-perfect setups like logic schemes but I don't think that kind of compatibility is maintained even now.
Toffo
Engineer
Engineer
Posts: 62
Joined: 04 Jun 2009 10:59
Location: NSW, Australia

Re: What is OpenTTD today?

Post by Toffo »

Sorry, the multi-quotes are a bit of a mess.


The way development seems to work is, people with commit access ("the devs" we keep referring to) review patches, and if they see someone contributing enough patches with enough quality, and personalities are not a clash, that patch contributor might get given commit access and become part of this devs group.
Dave wrote:Without stripping back too much of the interesting discussion ongoing - is the answer "OpenTTD is a project now developed by adults with busy lives, in some cases partners and children"? Ten years is a long time in any respect, and inevitably people, trends and ways of doing things move on.
...
Why do we now only have the equivalent of 1 developer when once we may have had 10? Well ten years ago the cohort of people that made up that 10 would have been mostly in their late teens, at the start of their careers, often at university or between work. Ten years on, they have mortgages to pay, children to feed and busy lives at home and at work.
Well that's it, isn't it? Life gets busy, people get busy. So if the devs are too busy to review patches, or have standards that are so high that no patch contributor can or will meet them, eventually the current crop of devs get will too busy with life and the new crop of devs don't get accepted. No one gets to contribute, the project dies, and those in charge at the time will have overseen that death.

So for the project to thrive, there are two main options. The current crop of devs can put more work into OpenTTD. As someone who is 10 years older today than he was 10 years ago ( :mrgreen: ), I can attest that isn't a reasonable ask. Life happens, families grow. The other option is to let new people in. Yes, that might mean a lowering (or as I prefer to put it, a changing) of standards, or it might mean learning to get along with those who might have a different viewpoint. Things might be rough for a while until a new status quo with this expanded group of devs, well, develops. A third option, of course, is to fork.
Dave wrote: Maybe this is a controversial point, and it's already been partly raised and refuted - but could OpenTTD have reached its natural conclusion?
I can see how many might think that. But when I check out the OpenTTD Development forum, I see JGR's patch pack, NotRoadTypes, Spring 2013 Patch Pack, New map features (cirdan's work), etc... There's tons of cool stuff happening in NewGRF-world too. There are people out there, doing stuff. It's just the stuff isn't ending up in trunk because those who can put it in there, don't. From what I've read, it sounds like the right decision at this point not to commit all these patches, but either way it's just not ending up in trunk.
JGR wrote: Quality is an important issue, however at least in my eyes there comes a point where having something imperfect but still useful sooner is better than waiting indefinitely for perfection.
Where that point lies is obviously a matter of personal opinion and taste.
In my own patchpack and features, I'm prepared to take that principle significantly further than trunk would or should be inclined to.
My personal view is also the "build first, refine later" view - but granted, this view of mine has never been applied to software development. OpenTTD is already refined though, so is the answer to hack it to bits and then re-refine? JGR, could your personal opinion and taste come close enough to the devs' opinion and taste to get you all contributing in the same place? Would you want to contribute to the same place? Are there others out there who might be asking themselves these same questions?
JGR wrote:
ic111 wrote:Of course one could do this - but where is the point in forking a project where hardly any development happens, due to lack of manpower?
At its simplest, so that you have somewhere to commit to.
Development by posting ever more/larger patch files on a forum/bugtracker is unsatisfactory for a number of reasons.
Is it possible to meet part-way for the good of the project? Would meeting part-way be good for the project? Is there any way contributing to trunk could become satisfactory? What would need to change so that it becomes satisfactory? These rhetorical questions (that I think are worth answering) are for you and others with your coding ability and commitment to OpenTTD.
JGR wrote:
Alberth wrote:... re-organizing [OpenTTD's code] basically means taking everything out, and rewrite it from scratch, thus throwing away years of work, while spending even more years of work to fix the thing and get a solid foundation for the next features.
Iterative refactoring of the the existing codebase (with or without extra features applied) is quite achievable.
So which is it? Pointless/too painful, or achievable/worthwhile? Will it be worth the pain in the end? And is there a group of people willing to try, and stick it out until the end?
Dave wrote: It is a great achievement, what we have seen here, and I expect a healthy following will always maintain an interest. But various personalities have faded into history and with them went the exciting cutting edge stuff [mainly because it's done], leaving talented and methodical developers to fine tune a programme which is now very much mature.
It's important to acknowledge what has been achieved. It's also very important to appreciate what we have, and those who have made it possible. That includes the devs who take a beating every now and then, but keep coming back to keep OpenTTD ticking over.

By all measures, OpenTTD is a remarkable project. But I think we're at a turning point where for OpenTTD to remain remarkable and to stand the test of time, things need to change at a fairly fundamental level, and big decisions need to be made. These decisions, or lack thereof, will rest on the shoulders of the few.

As an aside, I think a nice parallel can be found between OpenTTD and the NAM mod for SimCity 4. In particular, the RHW (Real Highway) side of it and their rewrite lines up nicely with what we're seeing.
Alberth
OpenTTD Developer
OpenTTD Developer
Posts: 4763
Joined: 09 Sep 2007 05:03
Location: home

Re: What is OpenTTD today?

Post by Alberth »

Toffo wrote:The way development seems to work is, people with commit access ("the devs" we keep referring to) review patches, and if they see someone contributing enough patches with enough quality, and personalities are not a clash, that patch contributor might get given commit access and become part of this devs group.
It helps a lot if you are discussing the idea, eg at IRC. Actively working to get it accepted helps loads too. That generally implies rewriting your patch 2 times or so, just like I do while hacking OpenTTD (I think I am around rewrite 5 for the scenario thingie currently, and I am not even close to done, but so often is an exception, due to the scale and the number of areas it touches.) While it seems a waste of time to rewrite functionality, in my experience, every rewrite improves the code. Every review improves the code. Stick with it, and at the very least you'll learn how to write proper code, and very likely you'll also get it into trunk.

Just dumping a patch onto FS and wait (what almost every patch author does) is a very risky strategy if you want it accepted, especially if it's real functionality instead of a small 10 line bugfix.

Toffo wrote:Well that's it, isn't it? Life gets busy, people get busy. So if the devs are too busy to review patches, or have standards that are so high that no patch contributor can or will meet them, eventually the current crop of devs get will too busy with life and the new crop of devs don't get accepted. No one gets to contribute, the project dies, and those in charge at the time will have overseen that death.
The standards are what they are, because we believe it's the best approach. You are free to disagree. At some point this becomes a difference in project direction, and a reason to fork, eg like Cirdan has done.
Forking is always possible. You don't have to wait until OpenTTD is dead, devs are not in control like you sketch it.

If you want to have the current devs aboard, then yes, you will have to convince us, but it's not required to continue the project.
Toffo wrote:I can see how many might think that. But when I check out the OpenTTD Development forum, I see JGR's patch pack, NotRoadTypes, Spring 2013 Patch Pack, New map features (cirdan's work), etc... There's tons of cool stuff happening in NewGRF-world too. There are people out there, doing stuff. It's just the stuff isn't ending up in trunk because those who can put it in there, don't. From what I've read, it sounds like the right decision at this point not to commit all these patches, but either way it's just not ending up in trunk.
JGR does submit patches, and they get into trunk. NRT isn't ready afaik, Spring patch pack I don't know, cirdan has decided to go his own way a long time ago. Everything NewGRF doesn't need to go in trunk, simply because it is already in OpenTTD. I don't know of any other serious NewGRF extensions than NRT but I doubt it exists. (To me that means having a GRF spec for it, an NML patch, and a OpenTTD patch, ie at the level that all NewGRF authors can actually use the new extension.)
Toffo wrote:My personal view is also the "build first, refine later" view - but granted, this view of mine has never been applied to software development. OpenTTD is already refined though
Not by a long shot. Do the "start new game" and "autorenew vs autoreplace" exercises I suggested. There is a lot more unrefined crap below the hood which is much more challenging to get right. Most of my commits is refactoring, ever since I became dev, and there is still plenty. Consider yourself invited to experience consequences of your view in reality.
Toffo wrote:Is it possible to meet part-way for the good of the project? Would meeting part-way be good for the project? Is there any way contributing to trunk could become satisfactory? What would need to change so that it becomes satisfactory? These rhetorical questions (that I think are worth answering) are for you and others with your coding ability and commitment to OpenTTD.
I think it means to code while having merging to trunk into mind. Even then, likely lots of work has to be done to make the merge to trunk really happen. It might mean rewriting large parts of the patch. Be available for discussion, be willing to change things (or not change things!), and deliver high quality.


Toffo wrote:
JGR wrote:
Alberth wrote:... re-organizing [OpenTTD's code] basically means taking everything out, and rewrite it from scratch, thus throwing away years of work, while spending even more years of work to fix the thing and get a solid foundation for the next features.
Iterative refactoring of the the existing codebase (with or without extra features applied) is quite achievable.
So which is it? Pointless/too painful, or achievable/worthwhile? Will it be worth the pain in the end? And is there a group of people willing to try, and stick it out until the end?
These views are not mutually exclusive. We need backwards compatibility, so my description "take stuff out and rewrite from scratch" is a very simplistic view on reality (although true at a higher level of view). In reality, I do what JGR says, making small rewriting steps to push the code base into the right shape for the next fix or feature. Have a look at all "-Codechange" commits which outnumber "feature" or "fix" by a factor 5-10 or so. In the end however, all my rewrites effectively replace/amend old code with new code.

Toffo wrote:By all measures, OpenTTD is a remarkable project. But I think we're at a turning point where for OpenTTD to remain remarkable and to stand the test of time, things need to change at a fairly fundamental level, and big decisions need to be made. These decisions, or lack thereof, will rest on the shoulders of the few.
This is a false assumption as I said, you don't have to wait and do nothing.
There is a second reason I think. You can't expect devs to fundamentally change in how they do things, just like we can't force patch writers to only provide patches that we completely agree with. If you want to change the direction of the project, you must either convince devs to change opinion (which is going to be very very hard, as they are grounded on technical foundations and experience of the last 10 years), you must become a dev yourself (fight the system from within :p ), or you must fork and take over as the main OpenTTD version.

At least, I see no other way, or did I miss something?



EDIT: Oh, and "fork" is not the solution I think, it's the start of the same problems, but for the person(s) forking.
Being a retired OpenTTD developer does not mean I know what I am doing.
User avatar
andythenorth
Tycoon
Tycoon
Posts: 5656
Joined: 31 Mar 2007 14:23
Location: Lost in Music

Re: What is OpenTTD today?

Post by andythenorth »

Toffo wrote:NotRoadTypes
Nah, NRT isn't ready yet. It might be close to done, or it might get a fundamental change of design.

It's been great having NRT in a clean github fork which is built officially by the OpenTTD compile farm. That approach has allowed people to create grfs which have raised questions about whether the design is right. That's much better than (for example) trying to write the perfect spec, then not adjusting it in response to evidence. :D . Iterate, test.
User avatar
Ailure
Route Supervisor
Route Supervisor
Posts: 435
Joined: 26 Apr 2005 19:06
Location: Sweden

Re: What is OpenTTD today?

Post by Ailure »

Dave wrote:Maybe this is a controversial point, and it's already been partly raised and refuted - but could OpenTTD have reached its natural conclusion? How long was this meant to go on for? How many people saw the birth of OpenTTD in 2004 and thought they'd still see any semblance of updating and development 13 years later? Maybe viewing the game in the context of the original TTD would be useful... that was released in the mid 90s and we are still talking about it 22 years later because of Patch and Open - without either of those things, would TTD have eventually faded from people's minds? Most likely.
I almost forgot about TTDPatch, it hadn't been updated in over ten years! I remember the old TTDPatch vs OpenTTD discussions too, though I dunno if anyone still sticks around with TTDPatch for daily play, especially since most newer newGRF's dosen't care for TTDPatch compatability as far i seen. :P

OpenTTD is quite mature and lacks any low hanging fruits from a developer standpoint. I'm not saddened by developer activity slowing down, rather surprised with how long it was ongoing and quite actively as well. As a end-user the only thing I really lack is some official daylength adjustment for multiplayer, but I also understand how hard it is to implement this properly (most current patches are ugly hacks :P).

I would love to see a fork that isn't afraid to break some eggs in compatibility and rethinking how the game works just to see how it turns out. I'm surprised it hadn't been done yet (although I'm aware it would be a lot of painful work at the same time). On the other hand the backwards compatibility is insane, I can load up the TTO savegames of mine I made during the 90's.
pelya
Transport Coordinator
Transport Coordinator
Posts: 342
Joined: 18 Nov 2010 19:48
Contact:

Re: What is OpenTTD today?

Post by pelya »

Ailure wrote: I would love to see a fork that isn't afraid to break some eggs in compatibility and rethinking how the game works just to see how it turns out. I'm surprised it hadn't been done yet (although I'm aware it would be a lot of painful work at the same time).
It's called Simutrans.
ic111
Director
Director
Posts: 608
Joined: 17 Jul 2007 17:56

Re: What is OpenTTD today?

Post by ic111 »

It helps a lot if you are discussing the idea, eg at IRC.
Actively working to get it accepted helps loads too. That generally implies rewriting your patch 2 times or so, just like I do while hacking OpenTTD (I think I am around rewrite 5 for the scenario thingie currently, and I am not even close to done, but so often is an exception, due to the scale and the number of areas it touches.) While it seems a waste of time to rewrite functionality, in my experience, every rewrite improves the code. Every review improves the code. Stick with it, and at the very least you'll learn how to write proper code, and very likely you'll also get it into trunk.
I think this iterative approach has a severe bottleneck: The dev capacity (and maybe, depending on the particular game aspect, also interest) to review things or to start thinking about the patches.

So, all what you write here sounds senseful, but it stays theory, if the feedback doesn't exist, and you not even know wether there is interest at all in improving the game aspect your patch deals with.

And then, eventually, you run into the situation, that you do all that complicate dealing with huge patch queues, spend a lot of time with improving it, but effectively, you don't know wether the black box named devs appreciates this, or wether in fact they have decided that your patch will never make it to trunk a long time ago.

And, I don't want to blame you when I write this, I know myself that real world time is limited, but it's what effectively happens, when you take the limited knowledge about the situation that I as a patch author have in such a situation into account.

Having said this, and to turn the argument into a constructive direction, what IMHO would help is starting with a more architectural view on the things. As a patch author I don't need a detailed line-by-line review as first reaction, often it would be enough that someone has a look on the overall architecture and ideas, and comments wether this looks like the right direction, or wether we disagree in a fundamental way about that.

So, IMHO, the starting point shouldn't be reading hundreds or thousands of lines of code in a very detailed manner, but trying to answer the question wether the fundamental ideas sound ok.

And, the latter thing is what in my experience is often missing in this project, which makes dealing with this project so difficult from the perspective of a patch developer.

E.g., the core idea of my timetables patch consists of the interaction and semantics of about five fields in Vehicle and Orders (arrival, departure, start date of timetable, length of timetable, vehicle offset against the timetable). The rest of the patch queue consists of GUI changes, and algorithmic adaptions to allow the player use this core idea in an efficient way (efficient in terms of GUI usage).

So, getting to an opinion wether the core idea sounds senseful should be possible within a limited amount of time, reviewing all that code line-by-line would need quite a lot of time.
Just dumping a patch onto FS and wait (what almost every patch author does) is a very risky strategy if you want it accepted, especially if it's real functionality instead of a small 10 line bugfix.
Well, at some point you have to wait for a reaction from the other side.
I think it means to code while having merging to trunk into mind.
I've started implementing this way years ago - but given my experience on the one hand, and this discussion on the other hand, the question wether that extra amount of work was senseful is a valid one IMHO.

Specifically, to give a concrete example, I have (following your standards) a refactoring of the map generation window to support the concept of multiple river generators as separate patches in my patch queue (could be applied separately from the new river generator) - but as long as no one is interested in that, I would have better thrown everything into one huge patch file, would have saved me a lot of work in moving things between patch files to have everything at the right place.
Even then, likely lots of work has to be done to make the merge to trunk really happen. It might mean rewriting large parts of the patch. Be available for discussion, be willing to change things (or not change things!),
I would say, big rewrites should be done as early as possible; and to make this happen, one would need feedback about the overall architecture in a relatively early step.
User avatar
andythenorth
Tycoon
Tycoon
Posts: 5656
Joined: 31 Mar 2007 14:23
Location: Lost in Music

Re: What is OpenTTD today?

Post by andythenorth »

ic111 wrote:I would say, big rewrites should be done as early as possible; and to make this happen, one would need feedback about the overall architecture in a relatively early step.
Other OSS projects do this with things like RFCs. An example is Plone PLIPS https://docs.plone.org/develop/coredev/docs/plips.html

These are quite bureaucratic processes that tend to work because people make their living with the OSS project (either using it in their products, or providing consulting and custom dev services). I don't know if these processes would work in OpenTTD world :)

I found patch queues didn't work for me when I tried to patch. A little bit in irc we're looking if there are other ways to review. I would prefer this was via github pull requests, but as I'm not a reviewer (and won't be ever) it's not up to me :) Code would still need to be atomic for review, and to standards, and there still aren't many people able to review :D
ic111
Director
Director
Posts: 608
Joined: 17 Jul 2007 17:56

Re: What is OpenTTD today?

Post by ic111 »

andythenorth wrote: Other OSS projects do this with things like RFCs. An example is Plone PLIPS https://docs.plone.org/develop/coredev/docs/plips.html

These are quite bureaucratic processes that tend to work because people make their living with the OSS project
No, I didn´t meant that to become a bureaucratic process ;-)
I found patch queues didn't work for me when I tried to patch. A little bit in irc we're looking if there are other ways to review. I would prefer this was via github pull requests, but as I'm not a reviewer (and won't be ever) it's not up to me :)
I would say, I can live with patch queues, but with increasing size they just create overhead. So, if the patch queues live for years, and you come into the situation that you want to implement the next feature on-top, then things start to become increasingly complicate.
Alberth
OpenTTD Developer
OpenTTD Developer
Posts: 4763
Joined: 09 Sep 2007 05:03
Location: home

Re: What is OpenTTD today?

Post by Alberth »

ic111 wrote:Having said this, and to turn the argument into a constructive direction, what IMHO would help is starting with a more architectural view on the things. As a patch author I don't need a detailed line-by-line review as first reaction, often it would be enough that someone has a look on the overall architecture and ideas, and comments wether this looks like the right direction, or wether we disagree in a fundamental way about that.
I fully agree with you. My "Discussing the idea" is exactly what you propose. By having informal discussion(s) first, you a) get better alignment of your idea with something that is acceptable, b) your ideas improve, since an IRC channel with random people has more ideas and points of view than you by yourself, and c) you are somewhat announcing that it's your area of interest.

IRC is really a big part of the communication.
ic111 wrote:So, IMHO, the starting point shouldn't be reading hundreds or thousands of lines of code in a very detailed manner, but trying to answer the question wether the fundamental ideas sound ok.
Indeed.
Nobody but yourself is stopping you from doing exactly that.

IRC is open 24/7, although at European afternoon and/or evening hours it's more busy. You can drop in any time you like, and ask any question you like. You may or may not get an answer, you might get 5 different answers, or you might get a different answer than you expected three hours after you asked it.
Answers generally depend on time.

You get better answers if you prepare things. The basic laws of communicating all apply here. If you want answers, make it easy for your public to understand you. Don't underestimate the knowledge of non-devs in the channel, many are old-time users or hobby-devs with a lot of technical knowledge.



We focus on the code, since that's what we are interested in. We can just scan-read a patch to see how it works and judge a lot from that. Writing a patch is a lot more work, for larger patches this might not be the best thing to start with.

ic111 wrote:
Just dumping a patch onto FS and wait (what almost every patch author does) is a very risky strategy if you want it accepted, especially if it's real functionality instead of a small 10 line bugfix.
Well, at some point you have to wait for a reaction from the other side.
You do?
I used to work on several things at the same time, so when one thing couldn't move, I switched to another.

Even today I switch to solving other tasks as well, if I feel like it.I helped andythenorth several times in getting his Python processing working, I wrote a NewGRF web-translator, and we randomly discuss new game scripts that could be fun, although I haven't had the time to actually sit down and write a new one.

ic111 wrote:I've started implementing this way years ago - but given my experience on the one hand, and this discussion on the other hand, the question wether that extra amount of work was senseful is a valid one IMHO.

Specifically, to give a concrete example, I have (following your standards) a refactoring of the map generation window to support the concept of multiple river generators as separate patches in my patch queue (could be applied separately from the new river generator) - but as long as no one is interested in that, I would have better thrown everything into one huge patch file, would have saved me a lot of work in moving things between patch files to have everything at the right place.
The biggest step is being able to follow our code style blindly. That solves the problem that you have to consider each line carefully afterwards, which is horribly boring, and you will miss some cases.

How to come up with the patches is entirely up to you. We ask for the final style of committing a single conceptual change only, we don't define how you get there. Use whatever system you like to create it.

I usually work in a patch queue, but on the first attempts I usually start making a mess when I don't know exactly how to proceed or I discover that I started at the wrong end. If it's simple to insert at some point in the queue, I prefer that, else I make a new clone, and copy changes over from the old to the new clone, thus creating a much cleaner patch queue based on current insights.

After a few such copy actions (usually 2 to 3), I get to the point where I have a result that actually works. Then I start considering how to merge things into trunk in a nice way, tidying all the commits (possibly splitting or joining them), tidying commit messages, checking order of the patches, checking proper functioning of the program after each commit, making sure I don't change code that isn't needed, and so on.

If you like a different approach, I see no problem in that.

ic111 wrote:I would say, big rewrites should be done as early as possible; and to make this happen, one would need feedback about the overall architecture in a relatively early step.
Agreed, although I would expect that architecture follows from the ideas how the solution works. (But maybe our ideas of what "architecture" is, differs.)
Being a retired OpenTTD developer does not mean I know what I am doing.
pelya
Transport Coordinator
Transport Coordinator
Posts: 342
Joined: 18 Nov 2010 19:48
Contact:

Re: What is OpenTTD today?

Post by pelya »

For the last three-four releases, trunk has been re-implementing UI fixes which were already done in Android version, mostly related to button text centering and UI scale options.

The diff between trunk and Android version codebase has grown quite substantially, so a patch is not a solution anymore, it's also getting outdated quickly.

Just give me a write access to the repo, and I'll merge all the glorious touchscreen-related fixes right away :D

I won't merge daylength patch, because I am not touching the code I don't understand, which is the game mechanics code or savegame code.
ic111
Director
Director
Posts: 608
Joined: 17 Jul 2007 17:56

Re: What is OpenTTD today?

Post by ic111 »

Alberth wrote: IRC is really a big part of the communication.
Ok, maybe I should start using it. For technical discussions, so far the well-known forum here sounded more senseful to me, as it looks much more like an permanent source of information than a chat.
We focus on the code, since that's what we are interested in. We can just scan-read a patch to see how it works and judge a lot from that.
I didn't expect anything else...
The biggest step is being able to follow our code style blindly.
Well, the question wether I am able to do so cannot be answered by me...
After a few such copy actions (usually 2 to 3), I get to the point where I have a result that actually works. Then I start considering how to merge things into trunk in a nice way, tidying all the commits (possibly splitting or joining them), tidying commit messages, checking order of the patches, checking proper functioning of the program after each commit, making sure I don't change code that isn't needed, and so on.

If you like a different approach, I see no problem in that.
Given the expected end result "patch queue" I don't see another senseful way than starting with it as well. At least, it happened quite seldom that I had to refactor the queue completely.

Regardless how you organize things, having the manage changes of thousands of lines of code outside the version control system of a project generates overhead, this is just inherently IMHO.
User avatar
andythenorth
Tycoon
Tycoon
Posts: 5656
Joined: 31 Mar 2007 14:23
Location: Lost in Music

Re: What is OpenTTD today?

Post by andythenorth »

ic111 wrote:Regardless how you organize things, having the manage changes of thousands of lines of code outside the version control system of a project generates overhead, this is just inherently IMHO.
I don't understand why we don't just use a github fork -> prototype branches -> feature branch -> pull request mechanic, but eh. Maybe I'm weird.
Locked

Return to “General OpenTTD”

Who is online

Users browsing this forum: No registered users and 5 guests