Patch Compile Farm

Forum for technical discussions regarding development. If you have a general suggestion, problem or comment, please use one of the other forums.

Moderator: OpenTTD Developers

Post Reply
gigajum
Route Supervisor
Route Supervisor
Posts: 511
Joined: 08 Mar 2006 08:33
Location: Germany

Patch Compile Farm

Post by gigajum »

As i said in this thread i created such that website. I tried to talk to richk 2 times on IRC yesterday, but he didn't response :(. Maybe he's too busy with his patches :)

The website currently only compiles my own patches. The user logged in stuff isn't written yet. But this could be done within one or two days. The question is how other patch devs respond to that page.

Then i should think about set up some rules for adding patches there. Like all patches who change things in gameplay need to be configurable about the patch menu. All those which only add things who users can use but not need to use them (like my bridge upgrade patch) need no configure settings.

The system tries to keep patches up to date. And it tries to compile a version which includes all patches. This could be expanded to a compiled verion which only includes patches which voting ratio is good (7+ of 10).

It only compiles win32 exe. I do not need linux binaries or linux server binaries or something else, but it can compile them too.

So far my thoughts about my small patch compile farm. :) Please give some feedback :)

And here's the Link:
http://217.160.107.179/ottd/
User avatar
PouncingAnt
Transport Coordinator
Transport Coordinator
Posts: 357
Joined: 09 Nov 2004 22:33

Post by PouncingAnt »

What a good idea! All those C noobs out there will have such fun! (come to mention it, I usually cant be bothered to compile things myself) :oops:
NB: the below challenges are still open for submission, so feel free to perform necromancy on them!
Try the PouncingAnt National Monopoly Challenge
Or even better, the PouncingAnt National Monopoly Challenge 2
Or better still, the PouncingAnt National Monopoly Challenge 3
Or, the PouncingAnt National Monopoly Challenge 4

Or try my scenario instead!

-(A lazy) OpenTTD Japanese Translator-
-(A lazy) PNGcodec user-
"You get what you pay for, so pay attention!"

Patches:
Company Station Stats
User avatar
BoominGranny
Engineer
Engineer
Posts: 85
Joined: 18 Apr 2006 02:16
Location: Wellington, New Zealand
Contact:

Post by BoominGranny »

This is a nice idea. i didn't understand until i re-read it. very cool.



This would ensure the best patches are kept running with the latest release so that developers don't have to keep re-inventing the wheel so to speak.

Well i'll have to make some patches then to get included!! :)
gigajum
Route Supervisor
Route Supervisor
Posts: 511
Joined: 08 Mar 2006 08:33
Location: Germany

Post by gigajum »

Yeah that's the idea. It creates a merge error file which you can access over the website. On the main page there is a column with merging output.

If there is a - there are no errors and all was merged right. If there are standing some numbers they need to be interpreted as follow:

hunks: failed / succeeded / ignored / total hunks in file

Fails means the coder needs to re-adjust some lines.
Succeeded is good, there is nothing to do.
Ignored are things the can't be merged. Maybe the file to merge in doesnt exist anymore.
total hunks should say what it counts
User avatar
PouncingAnt
Transport Coordinator
Transport Coordinator
Posts: 357
Joined: 09 Nov 2004 22:33

Post by PouncingAnt »

I now make your patch compile farm my first stop for updating my version of OTTD. I really like it!
NB: the below challenges are still open for submission, so feel free to perform necromancy on them!
Try the PouncingAnt National Monopoly Challenge
Or even better, the PouncingAnt National Monopoly Challenge 2
Or better still, the PouncingAnt National Monopoly Challenge 3
Or, the PouncingAnt National Monopoly Challenge 4

Or try my scenario instead!

-(A lazy) OpenTTD Japanese Translator-
-(A lazy) PNGcodec user-
"You get what you pay for, so pay attention!"

Patches:
Company Station Stats
gigajum
Route Supervisor
Route Supervisor
Posts: 511
Joined: 08 Mar 2006 08:33
Location: Germany

Post by gigajum »

It not perfect. If there are merge errors patch vs trunk you need to edit the patch. But what if merge patch vs patched trunk gives errors? Then you need to create it by hand. And that's that was richk67 is doing. This system is not perfect and it will never be perfect. The only thing it can do perfect is to keep patches up to date or alert if they are outdated.

The compile of patches is only done to make sure it can be compiled.
User avatar
LKRaider
Transport Coordinator
Transport Coordinator
Posts: 360
Joined: 23 Mar 2005 04:05
Location: Brasil
Contact:

Post by LKRaider »

Really cool. Thumbs up! :o

Now, make a system where we can easily upload new patches and see if they compile right 8)
gigajum
Route Supervisor
Route Supervisor
Posts: 511
Joined: 08 Mar 2006 08:33
Location: Germany

Post by gigajum »

LKRaider wrote:Really cool. Thumbs up! :o

Now, make a system where we can easily upload new patches and see if they compile right 8)
Yeah i have to write the whole user handling thing :( and the info pages but i have no fancy to do that. Well i think i can code something on this that weekend :)
User avatar
LKRaider
Transport Coordinator
Transport Coordinator
Posts: 360
Joined: 23 Mar 2005 04:05
Location: Brasil
Contact:

Post by LKRaider »

Hmm, I find this interesting. Ignore this whole brainstorm if you want :P

::
You make an upload page that requires login.

People would register for an account (by email? by a form on the site?) and then be able to login and upload a diff to the server.
The diff file would be named as: timestamp + patchname + md5 + userID

Server side, it will look into the upload folder, which would be like a queue (ordered by the timestamp).
Server will try to patch the trunk with the diff in the queue.
It will generate a log of the process with name: patchname + md5 + UserID
-> If md5 was already tried on current trunk, it will return the last log contents. (check md5 on success and errors folders, or keep track on a separate file). <- this is optional. Will just make server not waste effort ;)
-> If error occurs, it saves the log on error folder.
-> If success, saves the log on success folder.

When user logins, the script parses the folders for the userID on the filenames and counts the errors and successes, then prints it on the page, with links to download the logs.

Alternatively, a database would be better suited maybe (more stats), but need more implementation work probably.
gigajum
Route Supervisor
Route Supervisor
Posts: 511
Joined: 08 Mar 2006 08:33
Location: Germany

Post by gigajum »

LKRaider wrote:Hmm, I find this interesting. Ignore this whole brainstorm if you want :P

::
You make an upload page that requires login.

People would register for an account (by email? by a form on the site?) and then be able to login and upload a diff to the server.
The diff file would be named as: timestamp + patchname + md5 + userID

Server side, it will look into the upload folder, which would be like a queue (ordered by the timestamp).
Server will try to patch the trunk with the diff in the queue.
It will generate a log of the process with name: patchname + md5 + UserID
-> If md5 was already tried on current trunk, it will return the last log contents. (check md5 on success and errors folders, or keep track on a separate file). <- this is optional. Will just make server not waste effort ;)
-> If error occurs, it saves the log on error folder.
-> If success, saves the log on success folder.

When user logins, the script parses the folders for the userID on the filenames and counts the errors and successes, then prints it on the page, with links to download the logs.

Alternatively, a database would be better suited maybe (more stats), but need more implementation work probably.
The only bad thing on you thoughts is that i had them 3 weeks ago :)

The system allows that all.
Some info about the working:
- It fetches the patch from a directory on which the webserver has write access
- It updates it's trunk working copy
- It tries to merge ever patch into the updated working copy, (no changes at the code) (merge error file is created here)
- If the previous merge was successful it will merged with code changes
- It will compile the patched working copy
- It copies all needed files from the compile directory into a seperate directory and zips them
- The zipped file is putted back into a directory where the webserver has access
- It continues with the next patch file

- When finshed with all patches it tries to merge all patches together. If one fails in the first test it will be skipped.
- It get's compiled
- It will be putted into the same folder like the single patch zip files

So far to the compile system. That system uses the compile farm scripts available on svn. The patches get compiled alphabetical ascending. But the idea with the timestamp is not bad. The problem is using strings in shell. And the problem is to find some short words about the patch.

Curently the files need to be saved like: found_bank_r4742.diff

first the name with underscores. Then the svn version on which the were created. This is not needed yet for the system itself. It is only part of the zipped filename later but can be removed. The only thing that is necessary is the .diff as file extension. .patch is not implemented yet, but that rename can the webserver change while uploading the patch files.

Website register will be with nick, e-mail and password. It is using a Database too. But it's kept small with only informations about the patch, user login data, and download stats. Rating of patches need to implemented too. As said, a compiled version of only patches with a rating better then let's say 7 get compiled, is possible with that. Or patches which aren't finsihed but uploaded to get bug tested or anything else.

I recognized now that the last compile failed on my waypoint stats patch :)
So i have to correct the one error :) The last week there were no errors.
User avatar
LKRaider
Transport Coordinator
Transport Coordinator
Posts: 360
Joined: 23 Mar 2005 04:05
Location: Brasil
Contact:

Post by LKRaider »

Cool stuff

If you don't mind me asking, how will the patches be rated? By voting on them?
gigajum
Route Supervisor
Route Supervisor
Posts: 511
Joined: 08 Mar 2006 08:33
Location: Germany

Post by gigajum »

LKRaider wrote:Cool stuff

If you don't mind me asking, how will the patches be rated? By voting on them?
Right. But i'm thinking about vote abuse protection. That's one reason why it's not implemented.
User avatar
BoominGranny
Engineer
Engineer
Posts: 85
Joined: 18 Apr 2006 02:16
Location: Wellington, New Zealand
Contact:

Post by BoominGranny »

how about if only people who have registered users could vote on them, and only once each??

have 3 vote options:

yes(3), maybe(1), no (-5)
total the results and wolla.

the reason i suggest 5 for a no is because most people would choose yes, and if a registered user (im presuming most of these are patch developers) hit no, its probally for good reason.
gigajum
Route Supervisor
Route Supervisor
Posts: 511
Joined: 08 Mar 2006 08:33
Location: Germany

Post by gigajum »

BoominGranny wrote:how about if only people who have registered users could vote on them, and only once each??

have 3 vote options:

yes(3), maybe(1), no (-5)
total the results and wolla.

the reason i suggest 5 for a no is because most people would choose yes, and if a registered user (im presuming most of these are patch developers) hit no, its probally for good reason.
Yeah, developers see things from a different point that's true. But a nevative vote value is not that good. I wanted to use a system that counts the votes and the total value of all votes.
Let every one who is registed vote is not good too. I not want 50000 registed users there, only to allow them to vote. The users there should post their create patches, or take care of patches from retired users.

A public vote and a registed user vote is not good too. Maybe a 24 hour block for the IP for each patch. But that thing can wait some time (IMO).
User avatar
BoominGranny
Engineer
Engineer
Posts: 85
Joined: 18 Apr 2006 02:16
Location: Wellington, New Zealand
Contact:

Post by BoominGranny »

yea i know what you mean.

Well what do you want to accomplish by the vote?

If you want to rate which patch is the best your idea of blocking ips etc is the best.

If you want to seperate which patches are buggy/crap from the patches that are useful then just having registered users vote would be good - because being patch writers themselves they would have the best point of view.

ps. i've been saving replying to your pms for work tomorrow ;)
Post Reply

Return to “OpenTTD Development”

Who is online

Users browsing this forum: No registered users and 9 guests