End of the official Mac OS X port

[Rubidium]

We, the OpenTTD developers, are currently seriously reconsidering support for the Mac OS X port.

While most of the Mac OS X users know about Apple's new Mac OS X version 'Snow Leopard', only a few will know that OpenTTD does not officially support that version of Mac OS X. Without anyone capable enough to continue support for the Mac OS X port we will stop officially supporting the Mac OS X port completely at the next major release.

This thread is merely to ask whether there is someone capable enough to continue the Mac OS X port.

A more detailed report about the whole issue can be found in the next post. When you intend to ask questions or make remarks, read the detailed report first. Any questions that have already been answered will go straight to the trash bin.

[Rubidium]

Deprecating the Mac OS X port
=============================

Since the beginning of OpenTTD the Mac OS X port has been the problem child of OpenTTD. With every subsequent release we were forced to either deprecate support for older versions or write elaborate work-arounds to keep OpenTTD running on the already supported versions and the newer version. Now Apple has released Snow Leopard (Mac OS X 10.6) which, according to the bug reports we are getting, is not fully backward compatible with the previous releases. In other words: OpenTTD sometimes fails on Mac OS X 10.6.

As the number of Mac OS X specific bugs is rising quickly (at September 22nd, 2009, one in four open bugs is Mac OS X related), the situation is getting out of control, and we are forced to reconsider the position of the Max OS X port. Specifically, there are two problems that need to be addressed for us to continue supporting the Mac OS X 10.6 platform, namely bug fixing and building of Mac OS X binaries. For both of them we have no satisfactory solution.


Bug fixing
----------------------
The new bugs would not be a problem if we had an active porter that works on those issues and fixes them. Unfortunately, currently we do not have such a person. Also, we have not found a Mac OS X 10.6 user capable enough to continue the porting and keep the Mac OS X port alive.

As a result, all the Mac OS X issues are not being fixed. They just sit in Flyspray (our bug tracker) without any hope of getting fixed.


Building Mac OS X binaries
----------------------
The second problem is building of the Mac OS X binaries.

In principle, any change that we make in the OpenTTD source code or in the way we compile the program, may break the program. Normally, a dev will test whether a change works at his own system before publishing the change. That drastically reduces the chance of errors slipping through.

In the case of Mac OS X binaries, we do not have a Mac OS X system. To build binaries anyway at our compile farm, we currently use a method called cross-compiling to create our Mac OS X binaries. It means that we use a non-Mac system with a compiler that can make binaries that run at Mac OS X. This system more or less works at the moment, but it is far from a stable environment. Compiling on a real Mac OS X system with the official compilers produces better binaries and allows us to clean up the way we compile our Mac OS X binaries.

Another consequence of cross-compiling is that we cannot do any testing with the cross-compiled Mac OS X binaries (for testing a Mac OS X system is needed). The only thing we can do is publish the untested binaries, and wait a few days. If no user appears with a problem that is tracable to the last change, we may have made no error or nobody has tested it; we do not know which it is. If the user finds the problem one month later instead, there is no hope we will ever find out where the problem was introduced. As a result, the quality of the Max OS X release is degrading. For a long time we have managed to postpone many problems due to a lot of effort and time invested by Rubidium, but the effort needed to keep the Mac OS X port alive is rapidly growing beyond our resources.

Even if we would spend all our resources to keep the port alive (ie basically slowing down development), without a way to do proper testing of changes that we make, the quality of the produced Mac OS X binary will still detoriate to the point that most Mac users will stop using the program due to weird behavior, sudden crashes, etc.


Alternative approaches that do not work
----------------------
One possible approach to make proper testing possible would be to make a virtual Mac, that is, run a virtual environment that emulates a Mac OS X system, and do the testing there. Unfortunately, that does not work. The real hardware behaves differently than the emulated environment (several problems are related to one of several video card (drivers) for instance, which cannot be properly emulated). Also, there is the matter of running such a setup legally.


A second possible approach would be to buy a Mac system. We would need at least two, as we still want to also support the PowerPC versions of Mac OS X even though Apple stopped supporting them. Buying an Apple is expensive. We think that it is unfair to spend more than the yearly amount of donations to support a version of OpenTTD that a mere 6 percent of our users uses and that number totally disregards the people who get OpenTTD via their package manager so in reality it would be an even smaller percentage.

After we get the Mac machines (and assuming we can find a way to test 'one of several video cards does not work' at such a machine), there is still the problem of who will do the fixing. The developer that is going to maintain the port needs to have extensive knowledge of the API of Mac OS X. None of the currently active developers has such knowledge and it is very uncertain whether any of the active developers is interested in acquiring the required knowledge, and work on the Mac OS X port. Even if this would happen today, it will still take a long time before any of the open bugs gets fixed, since it costs a lot of time to understand the Mac system in enough detail to know how to fix a problem.


Some people have graciously offered use of their Mac computer by connecting to it remotely. While we appreciate the offer, it is not a real solution. The 'who will do the fixing' problem is not solved with it. Also, not all problems can be debugged remotely. For example, checking GUI problems over an Internet connection simply does not work. The additional data transfer of the screen contents changes the whole interaction with the system, for example glitches with timing simply disappear. Also, a few gigabytes of Xcode must be installed.
In addition, we would need to compile OpenTTD at their machine. While you want to work at your Mac, we're wasting all your CPU on compiling a binary which might take up to 10 minutes if we are trying to adapt OpenTTD to a newer SDK version and those full recompiles would happen in quite rapid succession meaning your Mac might become less responsive.

Previous experiences with the compile farm have told us that relying on third parties to get binaries compiled is very cumbersome. Quite regularly we might not be able to connect to your computer for any kind of reason which then would cause Mac OS X binaries to be unavailable. Also the time that your computer must be available is not 'just' during OpenTTD's nightlies. The compile farm is also used for building the release binaries and a number of helper applications for OpenTTD (grfcodec, pngcodec, catcodec, nforenum). This basically means we need to have 24/7 access to the hardware.

For both the debugging/testing machine and compile farm there is also the issue of us having control over the updates of libraries and development kits. This basically means that you would be giving us full access to the computer 24/7, which is something most people do not want.


Conclusion
----------------------
Above we have given you a description of the current situation of the Mac OS X port. The bottom line is that, unless we find a new active developer with Mac OS knowledge who wants to keep the port alive, we see no other solution than to drop Mac OS X support. What is support if the soon-to-be most used version of an operating system is not supported?

The new Mac OS X 10.6 version does not work with the current setup. We have nobody who understands and fixes these problems. Other approaches do not provide a solution. Without such problems getting fixed, stating that we support Mac OS X 10.6 is simply a lie. Mac users will simply stop using the program anyway. We could spend a lot of effort and time to postpone dropping support, at the cost of slowing down development for all users and getting several very annoyed developers (who may quit developing for OpenTTD in the extreme case). Postponing would however not solve anything, in the end we still have no other choice than to stop giving help, no matter how bad we feel about it.


We hope you understand our decision,
The developers of OpenTTD

[Gremnon]

I have a potential and very uncertain solution.
The Wine Project, I believe has an OSX port, which acts in much the same way. If a Linux maintainer has Wine, and is willing to test it, they can verify at least the possibility of it working, and OSX users would have to run it themselves through their port of Wine, any tweaks sent to the Wine App DB.

But if the OSX Wine works differently, as it probably does, to the Linux Wine, then it's not much point, but I'd suggest OSX users do try to run the Win32 OpenTTD under Wine if they have to stop using the precompiled builds.

[Michi_cc]

Just for the record, bug 2782 in FlySpray is some sort of tracking bug. Look there for technical information on the acute problems. This of course does not include the organizational/logistical problems mentioned above.


-- Michael Lutz

[KingRobot]

As someone who regularly contributes to open source software, as well as develops software for the Mac OS X, I would suggest to move the OpenTTD client for OS X to a sort of *unsupported* state; that is, your compile farm still builds it, you still fix bugs that at least prevent it from building there. Users can still download the software, test it, submit bugs, etc., but make it clear that the client build is unstable and only for development and testing purposes.

I say this for several reasons:
1) Except for those who are part of the expensive ADC program, 10.6 has only just been made available to developers. I myself have only just this week completed the porting and testing of various personal/hobby projects, much less begun any contributions to other open source projects. Unless you have had no contributions to the OS X port for some length of time, the lack of contribution may well be to your contributors learning the new OS, which would pick up once they are again up-to-speed.

2) The fact that 1 in 4 of your bugs listed is for the OS X port is actually quite encouraging. Many open source projects have difficulty simply getting their users to submit bug reports of any kind, and must also do all the testing themselves.

3) Maintaining an unsupported version will allow 'freelance contributors' to engage with the project; if support is fully removed, chances are much lower that anyone would pick up the pieces or make any positive contributions. If after some time, you still have zero contributions to the OS X port, then I think it might be time to retire it altogether.


Also, as final thought, OS X 10.6 does not support PPC; it should make your job easier if you support only Intel OS X 10.5 or greater - the majority of mac users today will be using Intel, and it is not uncommon for many other "current" software packages to have dropped PPC support altogether.

[planetmaker]

Also more thoroughly read the lengthy post at the top. I'd say buying 2 2nd macs wouldn't be so expensive.

The cheapest solution lies around 500$US for a XServe license (who donates that money right now?)- and then you still need the hardware. There are places where you might get offered for OpenSource projects some space and (v)server, if you bring your own license. But that might be not sufficient. And doesn't solve the issue concerning GUI testing, so you still need, additionally, a desktop or laptop - another 1k€ or more.

That said, I think that KingRobot's word sound feasable and sensible - even though it's know that there hasn't been any really active macosx dev even when there leopard the latest MacOSX.

[CommanderZ]

A Rubidium said, that would be more than OTTD's yearly budget. And it would not be quite fair, since only a very small percentage of users would benefit from that.

[Rubidium]

As someone who regularly contributes to open source software, as well as develops software for the Mac OS X, I would suggest to move the OpenTTD client for OS X to a sort of *unsupported* state; that is, your compile farm still builds it, you still fix bugs that at least prevent it from building there. Users can still download the software, test it, submit bugs, etc., but make it clear that the client build is unstable and only for development and testing purposes.

The port has been in that state for over a year now; the farm still build it, I tried to fix the issues that horribly broke the compile farm and as a result broke compilation on real Macs. I have had enough of this 'stab-in-the-dark' bug fixing approach. Also "unstable and only for development and testing purposes" would mean that no stable releases would be made, which is what we intend to do in ~6 months.

1) Ofcourse it's not only the release of 10.6; 10.5 already made our live hard and the number of bug reports has been rising since then. Also some of the 'user friendliness' changes that were implemented in 0.7.0 for the other major OSes has still not been implemented for OS X; there have been some unsuccesfull attempts. I would be amazed that it takes people more than 3 months to get familiar with the new version of OS, which gives them another 3 months to fix the bugs. Nevertheless, quite a number of bugs have nothing to do with the new version of the OS.

2) It might be encouraging, but simply reversing your statement might also show the exact opposite; those reported bugs are just the tip of the iceberg, which means it's more rotten than we think it is.

3) Official support 'just' means that one of the official developer looks at the bug reports, fixes the bugs, that the build is regularly tested properly and (in most cases) that binaries are made by us. For example we also have an unmaintained DOS version of OpenTTD lingering around in the source code; it should build and work in DosBox, but we never got it working on a real machine. The same holds for BeOS, MorphOS and OS/2 to some extent. Not officially supporting does not mean we're immediately going to trash the porting work that has already been done. We're just stopping to actively maintain/support it.
Also... what does a contribution mean? Someone who wrote a crappy patch to keep something compiling on his system which we then need to rewrite and retest or someone who fixed one of the known bugs in an elegant way that can be committed without modification? Because if it's the latter that number (near) zero already and for the former it's still countable on one hand (at least based on the last year).

'4') Why alienate only the people who do not have the vast resources to buy a new Mac? It's not like the PPC build is actually the most troublesome version; it's the Intel build that is generally causing problems for the porter; for example that issue that caused OpenTTD to not run on (some) Intel machines, which the developer couldn't reproduce and where we later found out it had something to do with the video card driver that was being used. That cost a lot of time to figure out. Furthermore the PPC build has been very valuable with fixing/finding Endianness issues.
The fact that Apple dropped PPC and spun that news as "Faster, more reliable installation" and "Smaller footprint" makes me quite annoyed.

[niCki]

So I can no longer download new releases?.

[KingRobot]

Thank you for the replies. Please understand this is all from the point of an outsider who has not done much more than poke around in the server code to make some server mods. I am very aware this is your project and you can do what you like with it, and equally aware of the frustration that can result from lack of contributions. And finally my original post was meant largely to clarify some points in the original post, of which I now have a better understanding.

..."unstable and only for development and testing purposes" would mean that no stable releases would be made, which is what we intend to do in ~6 months.

Well, then, it quite clear to no longer support it in six months time if there are no new contributions, and no dev steps up to support it.

Of course it's not only the release of 10.6; 10.5 already made our live hard and the number of bug reports has been rising since then ... Nevertheless, quite a number of bugs have nothing to do with the new version of the OS.

This I did not realize, I was given the impression that the majority of troubles were due to 10.6.

It might be encouraging, but simply reversing your statement might also show the exact opposite; those reported bugs are just the tip of the iceberg, which means it's more rotten than we think it is.

A point well made.

Not officially supporting does not mean we're immediately going to trash the porting work that has already been done. We're just stopping to actively maintain/support it.

Well that is quite fair. My only additional suggestion was to continue the auto-build, which it sounds like you are saying is an ongoing more trouble that it is worth.

Also... what does a contribution mean? Someone who wrote a crappy patch to keep something compiling on his system which we then need to rewrite and retest or someone who fixed one of the known bugs in an elegant way that can be committed without modification?

Well of course I meant the latter, and now I see it sounds as if you have not had this for years. I have also seen the codebase is quite clean compared to some open source games, so I further understand your wishing to keep it that way

Why alienate only the people who do not have the vast resources to buy a new Mac? It's not like the PPC build is actually the most troublesome version; it's the Intel build that is generally causing problems for the porter; for example that issue that caused OpenTTD to not run on (some) Intel machines, which the developer couldn't reproduce and where we later found out it had something to do with the video card driver that was being used. That cost a lot of time to figure out. Furthermore the PPC build has been very valuable with fixing/finding Endianness issues.The fact that Apple dropped PPC and spun that news as "Faster, more reliable installation" and "Smaller footprint" makes me quite annoyed.

Well, to me as a Mac user of both PPC and Intel, since you plan to alienate all mac users anyway, if most of the problems were in the PPC version (which I now see is not the case), then it might have been better to maintain only the Intel version. But since you say most of the problems are with Intel, then perhaps the idea was not so relevant.


So thank you for clarifying, and I shall perhaps have a look through some of the bugs to see what can be done. (Is it the case that all bugs related to the mac port are marked with the [osx] prefix, or are there others as well?)

[andythenorth]

I can see the logic in the argument.

From the point of view of newgrf development, no Mac OS X support means the community would lose contributions from me, and possibly planetmaker (who can speak for himself).

From the point of view of finding macs, we have stacks of Macs in my business, including about 6 or 7 sitting under a table not currently in use (spares). The ones out of use are most likely to be PPC G5 iMac or G4 Mac Mini; Intel ones tend to be in use, but I'd need to investigate further. I'm PMing Rubidium directly on this.

From the point of view of maintaining the port, I can't help. It's beyond what I can do.

For the record, dropping Mac OS X support would be a shocker for me, but I know how the support argument works.

cheers,

Andy

[audigex]

The thing is that it's not like the devs are unwilling to support mac - they'd have it running on my microwave if they could. Nobody WANTS to alienate mac users... they just don't have the resources to do it. There's no point trying to debug on a virtual machine, you introduce as many bugs as you get rid of, plus a load of inconsistencies; and anyone who's been near a source code will know that you have to know what's wrong before you can fix it.

As mentioned, it would take approximately one years entire budget just to buy the hardware to allow the mac build to be worked on. In the meantime there would be no money to maintain the servers etc, so obviously it couldn't all come from one year's budget even if it was to be done - it can't work like that.

What is needed is a philanthropist to donate the hardware or someone who already has a mac, the time and the expertise to keep the port going.

[Rubidium]

What is needed is a philanthropist to donate the hardware

Only if that philanthropist pays for the required OS X developer. Even then there's the compile farm 'issue' which means the developer needs to know both how to build GCC and the OS X API, unless the philanthropist pays for the dedicated OS X server and someone to maintain it. Given that GCC [0] just stopped their support for powerpc-apple-darwin and made i686-apple-darwin a secondary platform, which means they do not care about regressions for i686-apple-darwin when a new version is released, it might get a bit more troublesome to upgrade the compiler.

[0] http://gcc.gnu.org/ml/gcc/2009-09/msg00437.html

[Phx_01]

As I am using MacOS X since September 2006 and went through v10.4, v10.5 and will soon upgrade to v10.6, I at least partly understand the difficulties of programmers/coders as Apple usually shrouds itself in secrecy.

Nevertheless, I feel sad if the Mac option would go as I find this platform quite handy (at least from the users point of view).

Since I am no coder in general (I only do some scripts here and there), I do not have the knowledge either to actively code something. However, since XCode is delivered with each MacOS X DVD, I can at least compile it on my Mac, test it on my three machines (MacBook Pro, Mac Mini and Mac Pro) and send it in, if it helps.

[Rubidium]

I still don't see why you would need a OSX Server to compile your software.

We need it for the compile farm; it has to run dedicated, that is in a data center, which means it has to be rack mounted, which means it has to be: http://store.apple.com/us/browse/home/s ... ily/xserve

There is A LOT of documentation to be found in the Apple developer zone which is free but requires an account. And i think you can skip most of them as they're simply not relevant to OpenTTD.

I know it's free; I have a free account there. I also know there is a lot of documentation, but searching the documentation is a nightmare and finding the right function too.
Just a few 'simple' questions:
- what function can I use to get a suitable font for a language?
- what function can I use to get support for "pinyin"?
Simple questions... did once have clue about the former one, but have forgotten it. Don't have a clue about the latter one. Just take a look at the older bug reports and you'll notice that the issues are quite complex, so even though the API's learning curve might not be steep the issues are complex. Not to mention that the APIs make use of objective C(++), which is effectively Apple's own extension on C(++); yes I know it's not the truth, but Apple's the only major OS using it.

However reading the various posts i kinda read that the devs have made up their mind.

Yes we have made up our mind, but we have not dropped any support yet! Having made up our mind is a good thing because you know what is going to happen if nothing significant happens before the next release instead of being told on the release "oh, OS X is not supported, good luck".

I can at least compile it on my Mac, test it on my three machines (MacBook Pro, Mac Mini and Mac Pro) and send it in, if it helps.

The problem with that is that it takes a magnitude of time more than doing the testing yourself; sending a patch to you, you compiling it, testing it, describing the problem, me trying to figure out what the problem is, changing something, ... It takes a lot of time and becomes annoying for the developer really soon.

[John]

As a user who only uses OTTD on Mac OS I find this a great shame - but not surprising. I have heard of Rubidium's problems of maintaining the port by himself before (along with solving issues with 10.5 and not loading nightlies).

I personally don't see the compile farms building the OS X port as being a priority, given just how easy it is to download and compile on 10.5.


While I would love to help, I have no knowledge of coding or indeed being able to support such an endeavour myself.

[Parody]

One other idea that comes to mind is to "port" OpenTTD to yet another 3rd party cross platform library like for example Qt or GTK+. After a (possibly too short) peek into the code I would say that this is a possible way to go, because it looked to me as if the game code is fairly separate from the platform specific code. Of course, you get an additional library dependency in the code, but it can be selectively enabled for the platforms that benefit from it.

To be honest, I have no clue if this would solve the problem, but it might at least be worth consideration...

[Gremnon]

It'd still need proper testing and debugging on an actual OSX system though, that's the drawback to your idea parody.

[Phx_01]

I can at least compile it on my Mac, test it on my three machines (MacBook Pro, Mac Mini and Mac Pro) and send it in, if it helps.

The problem with that is that it takes a magnitude of time more than doing the testing yourself; sending a patch to you, you compiling it, testing it, describing the problem, me trying to figure out what the problem is, changing something, ... It takes a lot of time and becomes annoying for the developer really soon.

Well, it was a good meant offer since the source code is there and can be compiled with any Mac having XCode installed. And in one of your earlier posts, you stated that you do not have the Mac hardware to compile and test it yourself, but rather use a cross-compiler. So, the offer is just to give you the chance to get it compiled and tested on the actual hardware.

[Zuu]

When Mac OS X support gets dropped, I think the right thing then for the community to do is to find a recommended solution using emulation of either Linux or Windows. That solution may not be officially supported but still I'm sure there will be a wiki page describing how to set your Mac + emulator up to run OpenTTD. There might also be a point for Mac users to already now contribute test results with different emulators to see how feasible that is.

And just to clarify I am only speaking about users using an emulator to run OpenTTD, not for developing support for Mac OS X. Still I do understand that using OpenTTD through an emulator may lead to some issues since emulators are not exactly as running the OS in a native environment.

And we all know Wine is not for Mac OSX and Crossover stinks.

Have you tried Crossover on Mac to see if it can run OpenTTD built for Windows? Or does Crossover do you just think stinks because their interface is ugly and they charge for it?

I've been using Linux quite a bit and my experience of using software through wine and its friends has not really been a first class vehicle. Yet if that becomes the only option then the Mac community might want to investigate in that option.



Myself I have only been using Mac actively at the old Classic versions before OS X. Version 6-7-8-ish. => Long time ago.