FileFormats Discussion

oliver · Post by **oliver** » 14 Jul 2004 01:47

Here are my fileformat suggestions and reasoning. This list is not yet complete.

Audio:
Sound effects: wav, flacc or ogg vorbis. I guess wav, flacc always goes as it's as it is original/lossless and we could always go on and encode them to ogg.
Music/Musac: wav, flacc or ogg vorbis. Again keeping wav/flacc files around doesn't matter as we can always go to ogg. However wav/flacc tend to be quite large, something not suitable for downloaders etc. In the final version, or distribution version we could have ogg vorbis however I'd suggest that either CVS or authers have wav/flacc's to 'fix' errors or somthing in their originals.

Ogg v.s. MP3:
Ogg Vorbis should be our primary choice for Sound as it gives high quality audio with low disk usage. Secondly but most importantly Ogg Vorbis is supported by several audio libraries. I belive SDL supports it out of the box (mp3 support req. extra libraries) as does OpenAL. This makes using ogg a bit easyer and a bit more bullet proof.
I know that Lot of people want mp3 support to play their songs in the built in jukebox. This is something that can be added later.

Video:
I don't know if we'll be using actually video, but if we do I think Ogg Theora should be on top with additional support secondarly for xvid.

Grahpics:
For 3d modules I suggest using blender's model files, simpley because blender is an OpenSource program and to be assured we don't get any problems ever with 3ds or similar files (e.g. where the files legitimitly created etc) Why not settle on an open standard right now.
Textures should be either PNG if compression is needed or TGA/TIFF/XPM otherwise. I'm not sure what else is there, but we shouldn't use gif or jpg nor BMP or silly formats like that.

Other file's should be normal unix style text files (Linefeeds only not carriage returns)
Map files could be XML if really required.

All these points are open for discussion naturally but I think we got a descent list with open standards here that should pose a problem.

Conditional Zenith · Post by **Conditional Zenith** » 14 Jul 2004 02:16

Sounds good.

I don't like MP3 because of the licensing issues.

Also, I cannot find much about flacc (only that player foo plays it and player bar doesn't, etc.), and I have never heard of it before. Can you please point me somewhere.

Arathorn · Post by **Arathorn** » 14 Jul 2004 07:25

FLAC is lossless so too big for TE.

Paranoid Android · Post by **Paranoid Android** » 14 Jul 2004 11:58

I'm happy with Ogg Vorbis, I can use DB Power Amp to convert everything I've made quite quickly.

jfs · Post by **jfs** » 14 Jul 2004 13:50

oliver wrote:Other file's should be normal unix style text files (Linefeeds only not carriage returns)

It shouldn't be too hard to make support for both DOS, UNIX and Mac style line endings, and if all three kinds were supported it would be easier for the developers and hackers, I think. Eventually the files could be converted at load time, so all text internally is stored with UNIX line endings.

ChrisCF · Post by **ChrisCF** » 14 Jul 2004 15:56

TBH, the style of line-ending shouldn't really come into it. what's the problem with using UNIX style endings (\n only) on Mac OS, or DOS style (\r\n) on *nix? Sensible text editors support all three, and a myriad of others. Oh, and the encoding will be UTF-8, without question.

As has been discussed before, we should ideally try and write as little code for these things as possible, making use of libraries instead. Somewhere in much earlier discussion, it was raised that with some libraries we can have an XML parser out-of-the-box, exactly as we can have the insides of a Vorbis player - the joys of well-defined standards

I'd agree that support for free software should come first and foremost, so for screengrabs and static images we should definitely favour PNG over GIF or JPEG, and Vorbis for music over MP3 (of course, allowing the user to play their MP3s would be nice). The main problem with MP3 licensing is for apps that encode, not decode - the patent is over Fraunhofer's encoding algorithm, I believe. For video, given that, unless something changed recently, xvid is a free codec, I see no problem in being able to use it. Of course, we'd need some actual video first.

oliver · Post by **oliver** » 15 Jul 2004 17:41

Arathorn wrote:FLAC is lossless so too big for TE.

For the 'final' game yes, but maybe FLAC would be better then wav to keep in the archive as original's was my main suggestion, that and for sound effects maybe flac is easyer to decode then ogg ...
For music, yeah it's probably to big, but compared to wav, it's fine.

oliver · Post by **oliver** » 15 Jul 2004 17:50

ChrisCF wrote:TBH, the style of line-ending shouldn't really come into it. what's the problem with using UNIX style endings (\n only) on Mac OS, or DOS style (\r\n) on *nix? Sensible text editors support all three, and a myriad of others. Oh, and the encoding will be UTF-8, without question.

Personal I prefer ASCII encoding cos it's the most basic you can get, But i think UTF-8 is quite compiatble with ASCII isn't it. I have to read into that.

Fileformats internally should be \n is what I ment and should be the preferred format. We should try to keep 'our' archive clean this way and not mix and match. same goes for C files etc. (On a side note, does gcc even take UTF-8 chars as input?) that the engine works with both formats I guess shouldn't be a problem, eventually anyway, but we gotta keep things clean and concistent in our code stash.

ChrisCF wrote:As has been discussed before, we should ideally try and write as little code for these things as possible, making use of libraries instead. Somewhere in much earlier discussion, it was raised that with some libraries we can have an XML parser out-of-the-box, exactly as we can have the insides of a Vorbis player - the joys of well-defined standards

I agree that we should use as many _standard_ widley availble libraries as possible, but still keep the amount to a minium. Like not implementing 1000x features and use 10000x libs and go through library hell. Just because we write lesser code doesn't mean the game will then be less efficient. But I think I mentioned XML in my first post and I was reffering to using the XML(2) library for that yes.

ChrisCF wrote:I'd agree that support for free software should come first and foremost, so for screengrabs and static images we should definitely favour PNG over GIF or JPEG, and Vorbis for music over MP3 (of course, allowing the user to play their MP3s would be nice). The main problem with MP3 licensing is for apps that encode, not decode - the patent is over Fraunhofer's encoding algorithm, I believe. For video, given that, unless something changed recently, xvid is a free codec, I see no problem in being able to use it. Of course, we'd need some actual video first.

Atleast we totally agree on that one

What I was saying really was that (not with mp3's particulary but like 3ds files for example) if sometime ever the issue comes up 'how did you create these files, did you use an unlicensed, illegally licensed version of 3ds max, there might be some where some trouble. Let's just say now we use an open standard so we never have anything to worry about. (Same could go for mp3, did you legally encode that mp3 ...) avoid it and we're happy.

For video, IF we ever get that, I guess we could give Ogg Theora a try as it's supposed to be pretty good and everything and not known to be used for all the illegal stuff etc etc. I know you know what I mean : )

Any other thoughts from anyone? Other file formats to concider etc? Did we forget anything so far? Keep this thread alive for a while : )

ChrisCF · Post by **ChrisCF** » 15 Jul 2004 18:05

oliver wrote:Personal I prefer ASCII encoding cos it's the most basic you can get, But i think UTF-8 is quite compiatble with ASCII isn't it. I have to read into that.

It's compatible with ASCII, since ASCII is a 7-bit encoding. It's only when you introduce other things like the ISO-8859 range, which have no less than 15 (possibly more) different uses of the "extended ASCII" (8-bit range). It would only come into play in any files which contain strings (and we should use strings, not references).

but we gotta keep things clean and concistent in our code stash.

Fair enough, in which case we need to decide on \n or \r\n now (though TBH I don't see the savings being that much using \n).

As for the issue of XML, my opinion is that we'd be better off with something like libxml2, rather than trying to implement everything from scratch, and finding too many bugs.

The point on the patented stuff is fair enough. That said, I know more people iwll have the XviD MPEG-4 codecs installed than the Ogg Theora ones. Of course, we can only wait until we actually have something to put into them

Hellfire · Post by **Hellfire** » 17 Jul 2004 08:20

As far as line endings are concerned, I'm going to quote a few lines from a book I have:

Open Source Development with CVS, Third Edition, by Moshe bar and Karl Fogel, page 60:

CVS has some special tricks for text files. [...] Thus, the files in a working copy on a Windows machine have CRLF endings, but the files of the same project on a Unix machine have LF endings (the repository itself is always stored in LF format).

ChrisCF · Post by **ChrisCF** » 17 Jul 2004 14:07

Mmm... nice.

oliver · Post by **oliver** » 17 Jul 2004 21:08

If we go with xml, then Yeah i'd say deffinatly use libxml2 and for other things goes the same.

I think people that do make video's etc have all kinds of codecs and to ask them to encode a piece of film into ogg theora wouldn't be a problem. I think this specific item can be left in the middle until we decide to actually have ingame video etc.

which is why I'm saying, \n in our repository. Agreed there's more windows users out there, but since we want to make it multi platform and everything, \n is I think the best way to go since it works on every platform anyway. (\r\n sometimes get nasty on some unix editors etc)

That been said and all, I think we are all on the same page on this now I suppose ... : )

Conditional Zenith · Post by **Conditional Zenith** » 18 Jul 2004 06:32

If I read what Hellfire quoted correctly, then LF vs CR-LF is a moot point as text files will be converted upon being committed to CVS.

oliver · Post by **oliver** » 18 Jul 2004 16:13

converted too ....

exactly, something that needs to be decided sometime, and while we are on the topic, why not now.

CVS convets to whatever you tell it to. That and it's nice to know what is going to happen when you submit something. Maybe you depend on \r\n in your file (who knows why)

Transport Tycoon Forums

FileFormats Discussion

FileFormats Discussion

Re: FileFormats Discussion

Who is online