NewGRF Web Service

[glx]

Most newgrf are not under GPL.

[Rubidium]

Would someone explain the reason an entire community is going to throw their toys out of the pram and sulk over the distribution of a few graphics? Clearly the 'free as in freedom' mindset I associate with GPL software is not present here.

First and foremost: a NewGRF is not just 'a few graphics'; it is more like an application than a zip with a few graphics.

If that would be the norm, then yes... but it certainly isn't the norm. Just being bold here and I compare OpenTTD with qemu (an x86 emulator). Both are open source (GPL) projects. Both load data and execute it in a virtual machine. Now according to you anything that can be run in qemu must be 'free as in freedom' and automatically downloadable. So effectively you are saying that qemu should have a function that downloads Windows from somewhere and installs it.

So you support basically any illegal use of any software or graphics.

[Ralph]

First and foremost: a NewGRF is not just 'a few graphics'; it is more like an application than a zip with a few graphics.

If that would be the norm, then yes... but it certainly isn't the norm. Just being bold here and I compare OpenTTD with qemu (an x86 emulator). Both are open source (GPL) projects. Both load data and execute it in a virtual machine. Now according to you anything that can be run in qemu must be 'free as in freedom' and automatically downloadable. So effectively you are saying that qemu should have a function that downloads Windows from somewhere and installs it.

So you support basically any illegal use of any software or graphics.

That's quite a jump, and I apologise if my view that the possibility of a few peoples gfx getting downloaded in ways that they don't intend makes you think I am all for wholesale anarchy. I don't ever recall saying that code executed by a GPL VM need also be GPL, as this is not the case, I was referring more to the ethos under which the GPL is usually used, and that does not need apply to stuff people choose to release under other regimes.

Regardless of the format, be it a bitmap or VM code, the problem is still the same (copyright infringement in both cases, not that openTTD as a whole can really claim to be on a high horse here). I am just trying to understand why this feature is so violently opposed on something that only may happen. If it does happen, you pull the files from the service, apologies and carry on with life, even add a mechanism to delete them from clients if you must. Maybe some devs go and sulk, but I am sure many more would see this is a step forward.

What if the youtube people gone 'hang on, people might use this for copyright infringement, lets make even the discussion of user uploaded videos taboo'?

[Frostregen]

To come back to topic:

Then moderate it.

This sounds like the right way.
Only openttd developers, well known community members,
or newgrf authors themselves would have upload rights to the service.

Basically this would be a list with newgrf+version mapped to download urls.

Unknown or new grf authors would have to ask someone with upload rights to include their newgrfs into the list.

Sure, it requires some work to maintain the list,
but i think there will be no problem finding enough people who are willing to do this.

[Yexo]

That's quite a jump, and I apologise if my view that the possibility of a few peoples gfx getting downloaded in ways that they don't intend makes you think I am all for wholesale anarchy. I don't ever recall saying that code executed by a GPL VM need also be GPL, as this is not the case, I was referring more to the ethos under which the GPL is usually used, and that does not need apply to stuff people choose to release under other regimes.

Regardless of the format, be it a bitmap or VM code, the problem is still the same (copyright infringement in both cases, not that openTTD as a whole can really claim to be on a high horse here). I am just trying to understand why this feature is so violently opposed on something that only may happen. If it does happen, you pull the files from the service, apologies and carry on with life, even add a mechanism to delete them from clients if you must. Maybe some devs go and sulk, but I am sure many more would see this is a step forward.

What if the youtube people gone 'hang on, people might use this for copyright infringement, lets make even the discussion of user uploaded videos taboo'?

The discussion isn't taboo, but the we've had this discussion multiple times on this forum already, and the output isn't going to change.

Most users would find it a step forward because it becomes easier to download grfs, but the authors of some major grf sets don't like it and have threatened to make there sets incompatible with OpenTTD. That alone is a *very* good reason not to implement it.

[Rubidium]

. I am just trying to understand why this feature is so violently opposed on something that only may happen.

It already has happened that NewGRF authors were not pleased with something OpenTTD did (or did not) do and they then made their NewGRF incompatible with OpenTTD. So this is not about "may happen", but "will happen".

What if the youtube people gone 'hang on, people might use this for copyright infringement, lets make even the discussion of user uploaded videos taboo'?

Youtube is all nice, but once somebody asks Youtube to remove their video it is gone. However, with OpenTTD there is no way to remove something in a non-central system. Furthermore the non-central system will only allow downloading of the few NewGRFs that have explicitly allowed that. Finally OpenTTD is open source and I am sure there will be a patch that disables any restrictions that we put in regarding the downloading (or uploading) of files which will quickly be integrated in all the patch packs.

Using a central server for downloading NewGRFs is causing a lot of administrative work for both the NewGRF authors as well as the persons managing it. It furthermore means that NewGRF authors would be able to remove NewGRFs from there when they are outdated, making it impossible to join servers with the old version. And even then... there will be NewGRFs that are not to be distributed via the system which are, if I feel the mood of the NewGRF authors correctly, the NewGRFs most used. So you end up with a system that automatically downloads a part of the NewGRFs that you need, which is as good/worse as it is now; you still need to download NewGRFs manually.

So as a result you end up with a system that you cannot properly manage and a system that doesn't download all NewGRFs automatically, which makes implementing pointless as the user still needs to get NewGRFs manually as (s)he needs to do now.

[Frostregen]

Using a central server for downloading NewGRFs is causing a lot of administrative work for both the NewGRF authors as well as the persons managing it. It furthermore means that NewGRF authors would be able to remove NewGRFs from there when they are outdated, making it impossible to join servers with the old version. And even then... there will be NewGRFs that are not to be distributed via the system which are, if I feel the mood of the NewGRF authors correctly, the NewGRFs most used. So you end up with a system that automatically downloads a part of the NewGRFs that you need, which is as good/worse as it is now; you still need to download NewGRFs manually.

So as a result you end up with a system that you cannot properly manage and a system that doesn't download all NewGRFs automatically, which makes implementing pointless as the user still needs to get NewGRFs manually as (s)he needs to do now.

Even if some popular newgrfs would be missing, there would be still a subset, which server admins can easily use, without forcing the user to download anything. Some newgrfs are rarely updated anyway.
It would be the task of the server-admin to keep his used newgrfs in sync with the autodownload-list, as it is now with creating a newgrf-pack.
(There could be even an option to auto-update to the latest available (currently used) newgrf versions on each map restart. As this would guarantee all are using the latest version, newgrf authors may like it)

As ralph says: If newgrf authors see the system work well, they may start using it.

[Ralph]

...
The discussion isn't taboo, but the we've had this discussion multiple times on this forum already, and the output isn't going to change.

Most users would find it a step forward because it becomes easier to download grfs, but the authors of some major grf sets don't like it and have threatened to make there sets incompatible with OpenTTD. That alone is a *very* good reason not to implement it.

If people don't want their stuff distributed this way then that's fine, but I don't think that should prevent the system being developed for those that do. Provided we are careful to make sure than their precious newGRFs don't make it onto the central repository they can hardly complain.

Using a central server for downloading NewGRFs is causing a lot of administrative work for both the NewGRF authors as well as the persons managing it. It furthermore means that NewGRF authors would be able to remove NewGRFs from there when they are outdated, making it impossible to join servers with the old version. And even then... there will be NewGRFs that are not to be distributed via the system which are, if I feel the mood of the NewGRF authors correctly, the NewGRFs most used. So you end up with a system that automatically downloads a part of the NewGRFs that you need, which is as good/worse as it is now; you still need to download NewGRFs manually.

Then make it a condition of adding stuff that you can't then remove it, so clients can always get whatever version they need.

My argument ofcourse relies on people unwilling to add their stuff being the exception rather than the rule, and from what I gather this is not the case.

[wolf]

Now according to you anything that can be run in qemu must be 'free as in freedom' and automatically downloadable. So effectively you are saying that qemu should have a function that downloads Windows from somewhere and installs it.

According to you, every newgrf must be forbidden from automatic download, even if its author wants it to be as free as it can be.

Finally OpenTTD is open source and I am sure there will be a patch that disables any restrictions that we put in regarding the downloading (or uploading) of files which will quickly be integrated in all the patch packs.

And exactly what prevents some third party developer to make and maintain such system, part of which will be included in all patch packs?

Using a central server for downloading NewGRFs is causing a lot of administrative work for both the NewGRF authors as well as the persons managing it.

It's much better if every user has to find newgrfs and keep track of updates himself.

It furthermore means that NewGRF authors would be able to remove NewGRFs from there when they are outdated, making it impossible to join servers with the old version.

And you can't have many versions of single newgrf available for download because of...?

So you end up with a system that automatically downloads a part of the NewGRFs that you need, which is as good/worse as it is now; you still need to download NewGRFs manually.

Yes, it's exactly the same to find and manually download 30 newgrfs and to automatically download 28 newgrfs and do the remaining 2 the old way.

So as a result you end up with a system that you cannot properly manage

FUD.

a system that doesn't download all NewGRFs automatically, which makes implementing pointless as the user still needs to get NewGRFs manually as (s)he needs to do now.

FUD.

[Rubidium]

Then make it a condition of adding stuff that you can't then remove it, so clients can always get whatever version they need.

This will mean that broken/buggy/outdated NewGRFs cannot be removed. This causes people to report bugs that are already solved in the newest version. This has been one of the (other small) reasons why some NewGRFs are against the whole automatic downloading of NewGRFs.

wolf: please implement such a system and show us that it can be properly managed and that all NewGRF authors are going to cooperate by uploading their NewGRFs and especially that they agree with the fact that known broken NewGRFs are kept in the system forever.

When looking at most things somewhat related to OpenTTD that OpenTTD developers do not manage I see a lot of talking, a few WIPs that do not work correctly and then the project seems to silently die (or draw on for eons). Examples of this are the numerous patchpacks (none of them survived more than a year), the 8 bits graphic replacement and the 32 bits graphic stuff.

So now you want to start a huge project that would:
a) mess heavily with the network protocol; who know how OpenTTDs network protocols work exactly? I guess about two to three persons (all (ex-)developers).
b) create a website for author to upload their data with the required measures against non-authors to upload NewGRFs.
c) find someone who wants to manage the user management of the website so new NewGRF authors are added if they request it after making sure they are not there to upload some else's NewGRFs.
d) create a backend for storing the NewGRFs.
e) create a server where OpenTTD can download the NewGRFs from.
f) promote NewGRF authors to actively put their NewGRFs on that system with the rule that broken/bugged NewGRFs stay on there forever too.
g) create a mechanism for NewGRF authors to get unauthorized uploads to be removed.

I wish you good luck and hope that you have enough spare time to create that. I'll focus on making OpenTTD better for everyone except the few people who can't be bothered to download OpenTTDcoop's NewGRF pack or contact the server owner.

[Ammler]

Ask at TTDPatch Forum, if eis_os would like to extend the crawler with fields like direct links to GRF and Readme? That would be first step, if he will do that, I guess, you can then also talk to authors, if they want to add a link there.

THEN, you can speak again with OTTD devs, if they like to include your patch, which allows downloads over grfcrawler.

Isn't that a compromise?

Greets
Ammler

[wolf]

Then make it a condition of adding stuff that you can't then remove it, so clients can always get whatever version they need.

This will mean that broken/buggy/outdated NewGRFs cannot be removed. This causes people to report bugs that are already solved in the newest version.

Automatic download of updated versions would help with that.

wolf: please implement such a system and show us that it can be properly managed

No, thank you. Managing one patch that no developer cares about is enough for me.

and that all NewGRF authors are going to cooperate

Why should I care for ALL newgrf authors?

especially that they agree with the fact that known broken NewGRFs are kept in the system forever.

I don't play online, so for me the server can hold only the newest version. It's your argument that the old versions need to be available, so stop spreading FUD about it. Especially that now you either have to do user-to-user exchange of these old, buggy and totally unusable old newgrfs, or be unable to join certain servers.

a) mess heavily with the network protocol; who know how OpenTTDs network protocols work exactly? I guess about two to three persons (all (ex-)developers).

Again, FUD. Game related network communication is totally unrelated to the newgrf download network communication. So no matter how bad and broken the download component is, the multiplayer gameplay would remain as stable as it were.

I'll focus on making OpenTTD better for everyone except the few people who can't be bothered to download OpenTTDcoop's NewGRF pack

I don't quite understand what does OpenTTDcoop pack have to do with anything.

or contact the server owner.

To get those ancient newgrfs, and report the long-time-ago-fixed bugs? :>

[Ralph]

It would be simple enough to have the system check to see if there is an update to a newGRF, and warn the user if they are not using the latest, but get it anyway so they can load the save game they just downloaded. When updating their stuff authors can flag the old version as buggy if nessessary, so it is only used if needed for an existing game. (Client can warn user not to use it for new ones)

Changes to the network protocol would be minor enough, if any, seeing as the server browser already checks you have them.

[Rubidium]

a) mess heavily with the network protocol; who know how OpenTTDs network protocols work exactly? I guess about two to three persons (all (ex-)developers).

Again, FUD. Game related network communication is totally unrelated to the newgrf download network communication. So no matter how bad and broken the download component is, the multiplayer gameplay would remain as stable as it were.

Changes to the network protocol would be minor enough, if any, seeing as the server browser already checks you have them.

Did you know that when you start a network client directly from the console the server browser is completely bypassed? And did you know that the final check for the NewGRFs is done when the savegame has already been downloaded?

Downloading NewGRFs at that stage will likely cause an excessively long join time on servers with lots of NewGRFs when the client doesn't have the NewGRFs. This would then cause the server to kick the client while downloading the NewGRFs; the game is usually paused at that moment after all due to pause on join and you do not want to hold up a MP game too much for people who do not have the required NewGRFs. As a result a broken download component can cause the MP games to halt for the max_join_time. Now tell me how this does not influence MP gameplay.

It just shows again how easy it is to say that something is easy when you have no clue how something exactly works.

[audigex]

So here's a question - why the hell are they so opposed to their FREE (as in $) work being distributed?

For me, if I make something for free it's not because I want to give something away, it's because I want to show off... I'd imagine its the same for many people. Yes, we like being nice - but it's the praise and pride knowing that x000 people use my creation.

They don't lose anything from it, so why not?

[Rubidium]

So here's a question - why the hell are they so opposed to their FREE (as in $) work being distributed?

Because they do not like buggy versions to be floating around on the internet and because they want everyone to use the newest (usually most bugfree) version of their work. They furthermore do not like to get bugreports of bugs that have been solved literally years ago.

[wolf]

Did you know that when you start a network client directly from the console the server browser is completely bypassed? And did you know that the final check for the NewGRFs is done when the savegame has already been downloaded?

So now you can connect to the server, download the whole map and after stalling the game for everyone else realize that you can't connect anyway because you don't have the grfs required? Clever.

As a result a broken download component can cause the MP games to halt for the max_join_time. Now tell me how this does not influence MP gameplay.

Aw, how nice. Someone using modem connection (or having throttled connection just to be evil) can do exactly the same right now. The problem is unrelated to grf downloads.

[Ralph]

I would argue that a central repository would mean less buggy, outdated versions floating around, as auto-updating would be simpler.

if you don't want to mess with the netcode, do the updating based on what the server browser reports and assume anyone connecting from the command line knows what they are doing.

[DredFurst]

Ok, just a point here, as so many people have complained about having a directly linked web service for GRFs because of copyright issues, a server side download system would be more suitable.

This would operate on a level of trust, if you run a multiplayer server you should be able to turn this option on/off as desired, a web directory with the required files could be provided. The trust is here, if you want a GRF not to be distributed but your running a multiplayer server for a few friends, you handle the distribution, and password protect your server. If you want a public game, and have some public domain grfs being used, it should not be a problem, distribute them.

The reason I see this as being able to work is for the fact it has been done so many times before by so many other big name games. Half life for example (genre jump I know, but same concept) allows downloading of levels off of the server. In no way are these levels GPL, there is no source code and the distribution rights are usually unknown to you, I don't see people complaining there. If you find you had your "secret project" grf leaked and you are not happy about its usage, you should be questioning why you trusted the possible people that leaked it and not the openttd developers.

The other alternative to stop people claiming your work is theirs is to encode the newgrfs in such a way that they cannot be extracted and modified easily or reliably, i.e. some loss of information in the grf that could be extracted again, i.e. compiling them.

[athanasios]

... that OpenTTD developers do not manage I see a lot of talking, a few WIPs that do not work correctly and then the project seems to silently die (or draw on for eons). Examples of this are the numerous patchpacks (none of them survived more than a year), the 8 bits graphic replacement and the 32 bits graphic stuff.

So help us with the management of these, because you cannot have a complete game without graphics. These graphics will be part of OTTD one day.

Even though we do not have a complete original substitute grf a lot of work is done. Yet OpenGFX project is not even mentioned in official OTTD webpage. There should be a link in development section as well as some screenshots with info.