FileFormats Discussion
Moderator: Transport Empire Moderators
FileFormats Discussion
Here are my fileformat suggestions and reasoning. This list is not yet complete.
Audio:
Sound effects: wav, flacc or ogg vorbis. I guess wav, flacc always goes as it's as it is original/lossless and we could always go on and encode them to ogg.
Music/Musac: wav, flacc or ogg vorbis. Again keeping wav/flacc files around doesn't matter as we can always go to ogg. However wav/flacc tend to be quite large, something not suitable for downloaders etc. In the final version, or distribution version we could have ogg vorbis however I'd suggest that either CVS or authers have wav/flacc's to 'fix' errors or somthing in their originals.
Ogg v.s. MP3:
Ogg Vorbis should be our primary choice for Sound as it gives high quality audio with low disk usage. Secondly but most importantly Ogg Vorbis is supported by several audio libraries. I belive SDL supports it out of the box (mp3 support req. extra libraries) as does OpenAL. This makes using ogg a bit easyer and a bit more bullet proof.
I know that Lot of people want mp3 support to play their songs in the built in jukebox. This is something that can be added later.
Video:
I don't know if we'll be using actually video, but if we do I think Ogg Theora should be on top with additional support secondarly for xvid.
Grahpics:
For 3d modules I suggest using blender's model files, simpley because blender is an OpenSource program and to be assured we don't get any problems ever with 3ds or similar files (e.g. where the files legitimitly created etc) Why not settle on an open standard right now.
Textures should be either PNG if compression is needed or TGA/TIFF/XPM otherwise. I'm not sure what else is there, but we shouldn't use gif or jpg nor BMP or silly formats like that.
Other file's should be normal unix style text files (Linefeeds only not carriage returns)
Map files could be XML if really required.
All these points are open for discussion naturally but I think we got a descent list with open standards here that should pose a problem.
Audio:
Sound effects: wav, flacc or ogg vorbis. I guess wav, flacc always goes as it's as it is original/lossless and we could always go on and encode them to ogg.
Music/Musac: wav, flacc or ogg vorbis. Again keeping wav/flacc files around doesn't matter as we can always go to ogg. However wav/flacc tend to be quite large, something not suitable for downloaders etc. In the final version, or distribution version we could have ogg vorbis however I'd suggest that either CVS or authers have wav/flacc's to 'fix' errors or somthing in their originals.
Ogg v.s. MP3:
Ogg Vorbis should be our primary choice for Sound as it gives high quality audio with low disk usage. Secondly but most importantly Ogg Vorbis is supported by several audio libraries. I belive SDL supports it out of the box (mp3 support req. extra libraries) as does OpenAL. This makes using ogg a bit easyer and a bit more bullet proof.
I know that Lot of people want mp3 support to play their songs in the built in jukebox. This is something that can be added later.
Video:
I don't know if we'll be using actually video, but if we do I think Ogg Theora should be on top with additional support secondarly for xvid.
Grahpics:
For 3d modules I suggest using blender's model files, simpley because blender is an OpenSource program and to be assured we don't get any problems ever with 3ds or similar files (e.g. where the files legitimitly created etc) Why not settle on an open standard right now.
Textures should be either PNG if compression is needed or TGA/TIFF/XPM otherwise. I'm not sure what else is there, but we shouldn't use gif or jpg nor BMP or silly formats like that.
Other file's should be normal unix style text files (Linefeeds only not carriage returns)
Map files could be XML if really required.
All these points are open for discussion naturally but I think we got a descent list with open standards here that should pose a problem.
-
- Chief Executive
- Posts: 697
- Joined: 10 Jun 2003 00:19
- Location: Australia
- Paranoid Android
- Tycoon
- Posts: 2002
- Joined: 15 Jun 2002 12:27
- Location: lo:kay:shun - The site and/or situation of a settlement
- Contact:
Re: FileFormats Discussion
It shouldn't be too hard to make support for both DOS, UNIX and Mac style line endings, and if all three kinds were supported it would be easier for the developers and hackers, I think. Eventually the files could be converted at load time, so all text internally is stored with UNIX line endings.oliver wrote:Other file's should be normal unix style text files (Linefeeds only not carriage returns)
TBH, the style of line-ending shouldn't really come into it. what's the problem with using UNIX style endings (\n only) on Mac OS, or DOS style (\r\n) on *nix? Sensible text editors support all three, and a myriad of others. Oh, and the encoding will be UTF-8, without question.
As has been discussed before, we should ideally try and write as little code for these things as possible, making use of libraries instead. Somewhere in much earlier discussion, it was raised that with some libraries we can have an XML parser out-of-the-box, exactly as we can have the insides of a Vorbis player - the joys of well-defined standards
I'd agree that support for free software should come first and foremost, so for screengrabs and static images we should definitely favour PNG over GIF or JPEG, and Vorbis for music over MP3 (of course, allowing the user to play their MP3s would be nice). The main problem with MP3 licensing is for apps that encode, not decode - the patent is over Fraunhofer's encoding algorithm, I believe. For video, given that, unless something changed recently, xvid is a free codec, I see no problem in being able to use it. Of course, we'd need some actual video first.
As has been discussed before, we should ideally try and write as little code for these things as possible, making use of libraries instead. Somewhere in much earlier discussion, it was raised that with some libraries we can have an XML parser out-of-the-box, exactly as we can have the insides of a Vorbis player - the joys of well-defined standards
I'd agree that support for free software should come first and foremost, so for screengrabs and static images we should definitely favour PNG over GIF or JPEG, and Vorbis for music over MP3 (of course, allowing the user to play their MP3s would be nice). The main problem with MP3 licensing is for apps that encode, not decode - the patent is over Fraunhofer's encoding algorithm, I believe. For video, given that, unless something changed recently, xvid is a free codec, I see no problem in being able to use it. Of course, we'd need some actual video first.
For the 'final' game yes, but maybe FLAC would be better then wav to keep in the archive as original's was my main suggestion, that and for sound effects maybe flac is easyer to decode then ogg ...Arathorn wrote:FLAC is lossless so too big for TE.
For music, yeah it's probably to big, but compared to wav, it's fine.
Personal I prefer ASCII encoding cos it's the most basic you can get, But i think UTF-8 is quite compiatble with ASCII isn't it. I have to read into that.ChrisCF wrote:TBH, the style of line-ending shouldn't really come into it. what's the problem with using UNIX style endings (\n only) on Mac OS, or DOS style (\r\n) on *nix? Sensible text editors support all three, and a myriad of others. Oh, and the encoding will be UTF-8, without question.
Fileformats internally should be \n is what I ment and should be the preferred format. We should try to keep 'our' archive clean this way and not mix and match. same goes for C files etc. (On a side note, does gcc even take UTF-8 chars as input?) that the engine works with both formats I guess shouldn't be a problem, eventually anyway, but we gotta keep things clean and concistent in our code stash.
I agree that we should use as many _standard_ widley availble libraries as possible, but still keep the amount to a minium. Like not implementing 1000x features and use 10000x libs and go through library hell. Just because we write lesser code doesn't mean the game will then be less efficient. But I think I mentioned XML in my first post and I was reffering to using the XML(2) library for that yes.ChrisCF wrote:As has been discussed before, we should ideally try and write as little code for these things as possible, making use of libraries instead. Somewhere in much earlier discussion, it was raised that with some libraries we can have an XML parser out-of-the-box, exactly as we can have the insides of a Vorbis player - the joys of well-defined standards
Atleast we totally agree on that oneChrisCF wrote:I'd agree that support for free software should come first and foremost, so for screengrabs and static images we should definitely favour PNG over GIF or JPEG, and Vorbis for music over MP3 (of course, allowing the user to play their MP3s would be nice). The main problem with MP3 licensing is for apps that encode, not decode - the patent is over Fraunhofer's encoding algorithm, I believe. For video, given that, unless something changed recently, xvid is a free codec, I see no problem in being able to use it. Of course, we'd need some actual video first.
What I was saying really was that (not with mp3's particulary but like 3ds files for example) if sometime ever the issue comes up 'how did you create these files, did you use an unlicensed, illegally licensed version of 3ds max, there might be some where some trouble. Let's just say now we use an open standard so we never have anything to worry about. (Same could go for mp3, did you legally encode that mp3 ...) avoid it and we're happy.
For video, IF we ever get that, I guess we could give Ogg Theora a try as it's supposed to be pretty good and everything and not known to be used for all the illegal stuff etc etc. I know you know what I mean : )
Any other thoughts from anyone? Other file formats to concider etc? Did we forget anything so far? Keep this thread alive for a while : )
It's compatible with ASCII, since ASCII is a 7-bit encoding. It's only when you introduce other things like the ISO-8859 range, which have no less than 15 (possibly more) different uses of the "extended ASCII" (8-bit range). It would only come into play in any files which contain strings (and we should use strings, not references).oliver wrote:Personal I prefer ASCII encoding cos it's the most basic you can get, But i think UTF-8 is quite compiatble with ASCII isn't it. I have to read into that.
Fair enough, in which case we need to decide on \n or \r\n now (though TBH I don't see the savings being that much using \n).but we gotta keep things clean and concistent in our code stash.
As for the issue of XML, my opinion is that we'd be better off with something like libxml2, rather than trying to implement everything from scratch, and finding too many bugs.
The point on the patented stuff is fair enough. That said, I know more people iwll have the XviD MPEG-4 codecs installed than the Ogg Theora ones. Of course, we can only wait until we actually have something to put into them
-
- Transport Empire Developer
- Posts: 699
- Joined: 03 Feb 2003 09:30
- Location: Back at the office
As far as line endings are concerned, I'm going to quote a few lines from a book I have:
Open Source Development with CVS, Third Edition, by Moshe bar and Karl Fogel, page 60:
Open Source Development with CVS, Third Edition, by Moshe bar and Karl Fogel, page 60:
CVS has some special tricks for text files. [...] Thus, the files in a working copy on a Windows machine have CRLF endings, but the files of the same project on a Unix machine have LF endings (the repository itself is always stored in LF format).
Feel free to contact me over Email! My current timezone: Europe/Amsterdam (GMT+1 or GMT+2)
[ General TE Discussion ] [ TE Development ] [ TE Coding ]
Under construction...
Code: Select all
+------------Oo.------+
| Transport Empire -> |
+---------------------+
Under construction...
If we go with xml, then Yeah i'd say deffinatly use libxml2 and for other things goes the same.
I think people that do make video's etc have all kinds of codecs and to ask them to encode a piece of film into ogg theora wouldn't be a problem. I think this specific item can be left in the middle until we decide to actually have ingame video etc.
which is why I'm saying, \n in our repository. Agreed there's more windows users out there, but since we want to make it multi platform and everything, \n is I think the best way to go since it works on every platform anyway. (\r\n sometimes get nasty on some unix editors etc)
That been said and all, I think we are all on the same page on this now I suppose ... : )
I think people that do make video's etc have all kinds of codecs and to ask them to encode a piece of film into ogg theora wouldn't be a problem. I think this specific item can be left in the middle until we decide to actually have ingame video etc.
which is why I'm saying, \n in our repository. Agreed there's more windows users out there, but since we want to make it multi platform and everything, \n is I think the best way to go since it works on every platform anyway. (\r\n sometimes get nasty on some unix editors etc)
That been said and all, I think we are all on the same page on this now I suppose ... : )
-
- Chief Executive
- Posts: 697
- Joined: 10 Jun 2003 00:19
- Location: Australia
Who is online
Users browsing this forum: No registered users and 15 guests