Downloadable Graphics from servers?

[Bilbo]

Furthermore: adding a flag to the GRF is useless, because in no time somebody writes a "tool" to add that flag.

Yes, but there are already "tools" to change the license. Take any text editor, pick up the file with license in the archive with newGRF, erase its content and write in anything you desire (like "This is public domain"). Its not legal and the author will dislike you for it, but you can do it .... same with the "tool to change flag". And since server will not give away file with "allow download" flag disabled, you have to get it by yourself.

Also, changing the flag will alter the md5sum, which should (at least I hope) prevent the file to load.

What to do when the flag isn't there? Exactly, not distribute the file. Which NewGRF author is going to add the flag to the newgrf? And still the problem remains: "why doesn't OTTD download all NewGRFs, but only a few?" So you still get the same complaints and it still wouldn't work in most cases (as the NewGRFs most used will not get this "allowed to be redistributed" flag). Ergo, the flag is useless.

If most authors will leave the flag disabled, then yes, it would be basically useless. Although it still can make sense for some home-made newgrfs - I use "allow download" flag and put it on my server. No need to stuffing newgrf on any website or so .... client will download it.

The main reason why the NewGRF authors do not want people to redistribute their NewGRFs is the fact that they get bug reports about versions that have been released and fixed ages ago, but still swarm the server network.

If the server uses some old version, so must the client ... and if author have only the latest version for download ... you can be out of luck ...

[OzTrans]

... The main reason why the NewGRF authors do not want people to redistribute their NewGRFs is the fact that they get bug reports about versions that have been released and fixed ages ago, but still swarm the server network.

True ... but here is a potential solution :

Each GRF has a MD5-sum. For all authorised auto-distribute GRFs the MD5-sum could be stored on an 'official' server (a one and only) and a GRF can only be distributed automatically and 'legally' if the GRF is listed on that server and the MD5-sum matches. GRF authors have then control and can allow such distributions for multi player games by having their GRFs listed; and that list can be updated whenever updated GRFs get released; thus even forcing the latest GRF to be used, whether it has been obtained automatically or manually.

[DaleStan]

one way should be to download the grf in an hidden folder, with random name and maybe with random extension, ... then the client keep track of that file, it loads it in the grfs list, uses it and when the game is done the file is removed

this way, a normal user ... can still use the functions to search for "files created since 'last two minutes'" and grab it when playing

I think you just finished explaining why that isn't an option.

And, for the umpteenth time, distribution is distribution. Even if it's encrypted. It's still distribution. You *MIGHT* (I wouldn't want to be the one to test this, though) be able to get away with it if you use strong encryption and don't distribute the decryption key, but that kinda defeats the purpose.

but this time impossible because the key is random, and you must know it,

Or, you could just modify Open to spew out the key when it spews out the GRF.

same with the "tool to change flag". And since server will not give away file with "allow download" flag disabled, you have to get it by yourself.

One of your assumptions is wrong. Specifically: "We can trust the server to behave." This is Open Source land. We can never trust anyone to do anything.

It is a couple order of magnitudes easier to dike out a check than it is to put in a feature that doesn't already exist.

OzTrans: The same problem applies to your solution. We can't trust a gameserver to check with the GRF-master, nor to obey it.
Nor, in all honesty, can the game-server trust that it is actually talking to the GRF master. At least not unless the GRF-master digitally signs all its communications, and even then, there could be difficulties with certificate forgery.

Also, changing the flag will alter the md5sum, which

You're doing it again. That sentence should end:
... has nothing to do with anything, since the server now believes that it is allowed to distribute the GRF, and it will, therefore, do so.

If the server uses some old version, so must the client ... and if author have only the latest version for download ... you can be out of luck ...

No, you're not. You inform the server admin (with varying degrees of politeness, depending on how recent the update was) "Hey! Update your GRFs! I can't play on your server."

(Remember. It's not paranoia if they *are* out to get you.)

[mexicoshanty]

I think gameserver-to-client distribution should be used plus verification of the GRF from the master server. Say for example you join a game that is using GRF Funky that you don't have. OpenTTD would talk to the master server and ask if GRF Funky is a registed GRF. If it is OpenTTD then downloads a hash of some kind (md5 would be the easiest) followed by the GRF from the game server. The hash is compared with the GRF and if they match the GRF is ready for use. If they don't match it means the GRF was modified and should either rejected automatically or OpenTTD should prompt the user to accept/reject the GRF.

When OpenTTD contacts the master server for the hash it would also be a good opportunity to download the licence agreement as well which the user would have to accept before they download the GRF from the game server.

Eventually the hash verification etc could be tied in with the grf cataloguing system i'm building (:) yes building... i haven't forgotten about it i've just had zero time laterly).

[Rubidium]

The GRF is already identified by a hash. So when the GRF is modified, the masterserver won't know it. What to do then? Just not distribute the NewGRF?

Who is going to manage all those GRFs in the masterserver?

You still have the issue that most MP games won't download all NewGRFs needed because the most used NewGRFs will never be authenticated by the masterserver. Ergo, the downloading is still useless because it will only work for a small amount of people.

[Moriarty]

Furthermore: adding a flag to the GRF is useless, because in no time somebody writes a "tool" to add that flag.

Sorry, but as already pointed out, that's not really a valid arguement. Assuming I had the ability I could write a tool right now to allow automatic distribution of any grf. Following your logic that means the official OTTD trunk had better stop supporting grf's altogether.

-----

I think some folks are over-complicating it now. KISS (Keep It Simple).


1st, a Flag - set on or off. If off or not found don't allow distribution.
2nd zips - Use a zip to store the grf and related material (license or whatever the distributer wants). Distribute this zip when requested and the flag is true.

I can't see any legalistic flaws there (which is where all these stupid problems are stemming from). No need for varifications or hashes or anything like that.

Remember you don't need to make it totally-impossible-to-ever-distribute-if-the-author-says-so. You just need to make it so that normal users can't do it. If a user wants to pilfer grf's against the license, there are a bunch of simpler ways to do it (i.e. instant messenger/email) than a man-in-the-middle attack or something equally silly.


Advantages of this system include that it will ensure that the game and the server have the same version without making the player have to go and find a compatible version if the server is out-dated.

[Rubidium]

1st, a Flag - set on or off. If off or not found don't allow distribution.

Advantages of this system include that it will ensure that the game and the server have the same version without making the player have to go and find a compatible version if the server is out-dated.

To me this is one big contradiction; NewGRFs that are not allowed to be distributed will ensure that players do not have to find the compatible version of NewGRFs?

The fact will remain that the most used NewGRFs (dbset(xl), ukrs, aviators set, newships, etc to name a few) will NOT have that flag set. These will therefore still be not automatically downloadable, so the player still has to hunt for them and because the authors remove/replace older versions of those NewGRFs with newer ones, you cannot (always) find them.
Also adding links to grfcrawler from OTTD will not be the solutions because a large percentage of NewGRFs are not of grfcrawler.

This makes me come to the following conclusion:
adding downloading of NewGRFs and/or adding links to grfcrawler in OTTD does not solve the issue that people want to be solved. So why invest lots of time in something that is not going to work anyway?

[Bilbo]

same with the "tool to change flag". And since server will not give away file with "allow download" flag disabled, you have to get it by yourself.

One of your assumptions is wrong. Specifically: "We can trust the server to behave." This is Open Source land. We can never trust anyone to do anything.

Nope, but misbehaving servers can have penalty of being removed from master server .... people won't find them so easily ...

Also, changing the flag will alter the md5sum, which

You're doing it again. That sentence should end:
... has nothing to do with anything, since the server now believes that it is allowed to distribute the GRF, and it will, therefore, do so.

Well, you're right, server admin can toggle the flag for all the files and then let others download it ... so this won't work ....
But this would be breach of the grf licenses

[Bilbo]

Also adding links to grfcrawler from OTTD will not be the solutions because a large percentage of NewGRFs are not of grfcrawler.

Not all, but this is fixable error (just enter some data about the files and some link to download, authors homepage, etc ....

There is "add new grf" page and once many people will be sent to grfcrawler (maybe add there few links like "grfid XXX not found,. do you want o add it to database?") people will complete the database. At least for most "favorite" newgrf's

[Moriarty]

To me this is one big contradiction; NewGRFs that are not allowed to be distributed will ensure that players do not have to find the compatible version of NewGRFs?

I'm sorry, that's not clear, can you clarify?

The fact will remain that the most used NewGRFs (dbset(xl), ukrs, aviators set, newships, etc to name a few) will NOT have that flag set. These will therefore still be not automatically downloadable, so the player still has to hunt for them and because the authors remove/replace older versions of those NewGRFs with newer ones, you cannot (always) find them.

Maybe if this feature was added those authors would decide to take a more lenient stance.

Also adding links to grfcrawler from OTTD will not be the solutions because a large percentage of NewGRFs are not of grfcrawler.

I didn't suggest anything as complex, though I suppose you could allow the grf to contain a link to where new grfs could be downloaded. That would simplify things for the newGRF devs who don't like having their work distributed.

adding downloading of NewGRFs and/or adding links to grfcrawler in OTTD does not solve the issue that people want to be solved. So why invest lots of time in something that is not going to work anyway?

Personally I think that should be phrased:
"Whilst this will solve the technical problems, the makers of the newGRFs will need to be willing to do it". Maybe some of them should comment as they're the ones creating this impediment.

[Torrasque]

Ihmo, the flag would be the easier solution.
Perhaps it will not do with actual GRF, but I think it's a must have for 32bpp.
And it's a step to not miss if we don't want the same problem in one year.

[Moriarty]

Ihmo, the flag would be the easier solution.
Perhaps it will not do with actual GRF, but I think it's a must have for 32bpp.
And it's a step to not miss if we don't want the same problem in one year.

Why would 32bpp graphics be being shared? Or are you on about the newgrf 32-bit equivilents? They should definately have the flag - future proofing.

[Torrasque]

Ihmo, the flag would be the easier solution.
Perhaps it will not do with actual GRF, but I think it's a must have for 32bpp.
And it's a step to not miss if we don't want the same problem in one year.

Why would 32bpp graphics be being shared? Or are you on about the newgrf 32-bit equivilents? They should definately have the flag - future proofing.

Yes, I was thinking about newgrf 32-bit equivalents. Sorry for being unclear.

[Rubidium]

But... there will be no 32bpp equivalent to the 8bpp NewGRFs. The 32bpp images will only replace images from the 8bpp realm.

You will need to have a 8bpp NewGRF and 32bpp replacement graphics when you want to play 32bpp, but the 8bpp is needed so everybody can join network games without being forced to use 32bpp graphics.

[DaleStan]

Furthermore: adding a flag to the GRF is useless, because in no time somebody writes a "tool" to add that flag.

Sorry, but as already pointed out, that's not really a valid arguement. Assuming I had the ability I could write a tool right now to allow automatic distribution of any grf.

But how useful would it be? Would that tool allow one person, working alone, to easily and quickly break the NewGRF no-distribute rules?

Putting a flag in the GRF means that any one person can un-"protect" any file, with a single executable that would work for all files currently in existence, and then it'll be distributed far and wide.

Of course, as soon as such a program appeared, someone else would invent a protection that it wasn't prepared to handle. In fact, I've already come up with two systems for forcibly setting the distribution-permitted flag, and both can be easily broken with some inventive coding.
And, being rather simplistic protections, the distribution-enabler would be quickly updated to deal with them too. And I so I go use somewhat more complicated inventiveness. Believe me. When it comes to protecting against automatic modifications, self-modifying code is a possibly the most beautiful thing in the world.

I think some folks are over-complicating it now. KISS (Keep It Simple).

Indeed. It's already as simple as it's going to get. Any other solution adds complexity to both OpenTTD and the GRF spec.

1st, a Flag - set on or off. If off or not found don't allow distribution.

This would be added complexity. I'd have to add the flag, and Open would have to look for it.

2nd zips - Use a zip to store the grf and related material

As would this. Open doesn't understand zip files.

Remember you don't need to make it totally-impossible-to-ever-distribute-if-the-author-says-so. You just need to make it so that normal users can't do it.

So, normal users can't start $FILESHARING_PROGRAM and find a crack and keygen for $GAME? Oh. Right. Nevermind.

Nope, but misbehaving servers can have penalty of being removed from master server .... people won't find them so easily ...

How does the masterserver confirm that the servers are actually behaving?

If I were to write a misbehaving server, the first thing I'd do would be to make sure the server behaves perfectly when queried from the master-server's IP address. Actually, IP range. A /24 should do. I'd also ensure I behave perfectly on the first query, regardless of who makes it, just in case the /24 isn't a wide enough IP range to do the trick.

To me this is one big contradiction; NewGRFs that are not allowed to be distributed will ensure that players do not have to find the compatible version of NewGRFs?

I think you got that backwards. "NewGRFs that *are* allowed to be distributed will ensure that players do not have to find the compatible version of NewGRFs."

Maybe if this feature was added those authors would decide to take a more lenient stance.

Yeah. Right. We're going to relicense all our works so they can be distributed without the license agreements? Whatever you're smoking, keep it away from me. I don't want to have even the vaguest notion where you got the idea that might happen.

"Whilst this will solve the technical problems, the makers of the newGRFs will need to be willing to do it". Maybe some of them should comment as they're the ones creating this impediment.

What impediment are you trying to remove? The requirement that the GRF file be distributed with its readme and license? I don't think anyone, anywhere, is going to lift that restriction.

[Rubidium]

To me this is one big contradiction; NewGRFs that are not allowed to be distributed will ensure that players do not have to find the compatible version of NewGRFs?

I think you got that backwards. "NewGRFs that *are* allowed to be distributed will ensure that players do not have to find the compatible version of NewGRFs."

I didn't get it wrong: Moriarity said that there would be a flag to disallow distribution (or allow it). Then he said that that "system" (the one with the flag) will ensure that the server and the clients have the same NewGRFs without the player needing to hunt for a NewGRF.

He explicitly allows NewGRFs to be NOT distributed, how can that ensure that the player has all NewGRFs the server has without manual downloading? See the contradiction in those statements?

[Moriarty]

Furthermore: adding a flag to the GRF is useless, because in no time somebody writes a "tool" to add that flag.

Sorry, but as already pointed out, that's not really a valid arguement. Assuming I had the ability I could write a tool right now to allow automatic distribution of any grf.

But how useful would it be? Would that tool allow one person, working alone, to easily and quickly break the NewGRF no-distribute rules?

It's a known theoretical impossibility to create impenitrable DRM, which is what this is. So the answer is obvious - it will be broken if someone seeks to break it.

Putting a flag in the GRF means that any one person can un-"protect" any file, with a single executable that would work for all files currently in existence, and then it'll be distributed far and wide.

Fine - but how is that relevent? That's just like the fact I can go and pirate just about any game, even the ones with "copy protection". You're just creating a technically impossible scenario, specifically "How do I stop people from breaking this".

Of course, as soon as such a program appeared, someone else would invent a protection that it wasn't prepared to handle.

Which is 100% irrelevent. I can already manually distribute any and all of the NewGRFs, irrespective of what their license says. If someone choses to go around the system, you'll never be able to stop them, but at least we've made some provisions to try.

Indeed. It's already as simple as it's going to get. Any other solution adds complexity to both OpenTTD and the GRF spec.

From whose perspective? To the OTTD coder, the current system couldn't be simpler as there IS no system from their perspective.
The players? To them its complicated and convoluted.
As a designer I try to think of the players before the developers.

He explicitly allows NewGRFs to be NOT distributed, how can that ensure that the player has all NewGRFs the server has without manual downloading? See the contradiction in those statements?

It's not really a contradiction. The hope is that the newGRF makers would set the flag and thus transfer would be automatic. Sure some wouldn't, but it'd still make lots of players lives easier, and over time maybe the hold-outs would become more user-friendly and set the flag too.

[DaleStan]

Putting a flag in the GRF means that any one person can un-"protect" any file, with a single executable that would work for all files currently in existence, and then it'll be distributed far and wide.

Fine - but how is that relevent?

Without such a flag, it requires the cognizant cooperation of at least the sender and probably the recipient too. With the flag, any one person can break the file, and then everyone else can (and probably will) remain oblivious.

You're just creating a technically impossible scenario, specifically "How do I stop people from breaking this".

That's not an impossible scenario. There is one way to guarantee that something cannot be broken. Don't build it.

I can already manually distribute any and all of the NewGRFs, irrespective of what their license says.

But you know you're doing it. If it was possible to put a flag into the GRF that says "you can automatically distribute this", then the grf will get distributed, without most people knowing the difference.

The players? To them its complicated and convoluted.

It's complicated to type http://grfcrawler.tt-forums.net into your web browser and then search for "DBSetXL"?
Actually, come to think of it, yes, it is.
Just type "dbsetxl" into that useful Google search bar.

It's not really a contradiction. The hope is that the newGRF makers would set the flag

For several common sets, the license (which cannot be changed) dictates some exceedingly strict requirements before that flag could be set. The second one is especially interesting.
1) There must be some way to ensure that GRF is transmitted with its license, and
2) There must be some way to ensure that GRF transmission is followed by a valid offer for the source code (if/when we figure out what constitutes "source code") IOW, the grf cannot be distributed by anyone who does not have (or, at least, know where to find) the source code.

Personally, I think that is entirely too complicated.

[Moriarty]

That's not an impossible scenario. There is one way to guarantee that something cannot be broken. Don't build it.

"The only way to guarantee you don't fail is not to try".
I'm not sure who I'm paraphrasing, but I trust you get the point. That's very defeatist. I'm trying to be constructive here.

But you know you're doing it. If it was possible to put a flag into the GRF that says "you can automatically distribute this", then the grf will get distributed, without most people knowing the difference.

But I don't know I'm doing it in contravention of the license because I don't know what any of the licenses say because I never read them, just like everyone else.
So how is it different?

It's complicated to type http://grfcrawler.tt-forums.net into your web browser and then search for "DBSetXL"?
Actually, come to think of it, yes, it is.
Just type "dbsetxl" into that useful Google search bar.

Yes it is. Think about all the time the players are wasting doing this, combined with the time required to manage thos GRF's. That'll add up.
That's on top of the bandwidth to those websites etc etc. It all adds up.

the license (which cannot be changed)

The license holder can change it at any time.

1) There must be some way to ensure that GRF is transmitted with its license, and
2) There must be some way to ensure that GRF transmission is followed by a valid offer for the source code (if/when we figure out what constitutes "source code") IOW, the grf cannot be distributed by anyone who does not have (or, at least, know where to find) the source code.

Both of which are resolved by distributing the zip file (originally downloaded from the license holders site) containing those things.
If the server owner culls the zip file and removes the readme/source code, then they're in breach of the license in the same way they would be if they manually sent just the GRF. I really fail to see the difference from a legalistic perspective, which is all we're interested in.

Personally, I think that is entirely too complicated.

And I think it's a fair trade off.

Players perspective:
- Becomes MUCH simpler to share newGRFs.

NewGRF makers:
- Add a new flag to your grf's if you don't mind them being shared. Ensure you distribute a zip file (which is probably already being done) with the minimum of cruft in it.

OTTD Devs or a patch maker:
- More coding!

[Bilbo]

It's complicated to type http://grfcrawler.tt-forums.net into your web browser and then search for "DBSetXL"?
Actually, come to think of it, yes, it is.
Just type "dbsetxl" into that useful Google search bar.

Yes it is. Think about all the time the players are wasting doing this, combined with the time required to manage thos GRF's. That'll add up.
That's on top of the bandwidth to those websites etc etc. It all adds up.

For DBSetXL no, but there are some other GRF's with less clear names or multiple versions (and now which one to get? or they are not at grfcrawler. And using google on some generic names like "new heliport" ... good luck (well, you may try appending words like "openttd" or "newgrf" and you'll probably find it after all ... but I had some trouble with getting some of the GRF's)