Service Maintaince - VCS updates / SSL updates

[TrueBrain]

Hi all,

Next weekend I will be doing some changes related to subversion, git, and mercurial on openttd.org. Most things you will not notice, and most stuff only changes for developers (and even that will be mostly transparent).

Basically, I am moving all VCS related service to their own VM. For developers it means that only 'ssh+svn' will end up in that box, and for the rest of the world only git (via either git or http), mercurial (only via http), and subversion (only via svn) will be available.

Today I spend preparing most of this. So far everything seems to behave just fine, and I am up to the point I just have to copy the live data, and flip the switch.

This change was long overdue. It is one of the last few pieces to split all services into their own VM. This makes maintaince a lot easier, security a lot higher, and permissions a lot more flexible.

It also means I will be retiring some serives:

- At the moment every dev can claim their own mercurial and/or git repos on the server. As no developer has used this for years, this will be removed and no longer available.
- At the moment every dev can access the repos by file (on disk). This will no longer be possible (well, it is, but you will have to ssh to another box). As far as I am aware, nobody made use of this
- Tracd will be removed. Tracd has been a real issue for months/years. When someone opens a merge commit, or revision 1, it will run out of memory and die. If we are lucky, it won't take any other services with him, but mostly we are not that lucky.

The things we get in return:

- Latest version of subversion, git, and mercurial, with all their latest goodies
- Gitweb and hgweb in working order
- Sane http://git cloning (atm you have to download packs)
- Working sync to Github ( http://github.com/OpenTTD/ ); at the moment we have to manually sync it, and next weekend we will finally be able to automate it


I could not find any sane SVN frontend. Tracd is too complex to install/maintain, and has shown to be incapable of working with our merge commits. Viewvc installation documention is non-existance, and the default CGI failed to work properly. I have no interest in installing PHP only for a Subversion frontend. And honestly, the gitweb frontend is much better and prettier anyway.
Either way, if any of you have a good suggestion for a webinterface for Subversion, I have no issues installing it. Otherwise, we will have to do without.

Note, we are not ditching SVN or anything. SVN is where the pushes go, and you can pull from either SVN, Git, or Hg, depending on your flavor. Just like it is now. Only the SVN webinterface will retire.


So next weekend I will make the rollover to the new platform. It will mean that most of the weekend you can expect issues with either of the three VCS systems.


Questions? Feel free to ask!

[TrueBrain]

Also, on a small, different, but related note:

Because I was in the mood for an A+, I changed our frontend proxy a bit in regards of SSL settings:

- we no longer support SSLv3 (go cry me a river)
- we no longer use RC4 (blabla, RC4 broken, blabla)
- we are much more picky what ciphers we do support, but include older Windows IE for now
- created a 4k DH (instead of the 1k DH). Sorry Java1.6 users (again, go cry me a river)
- enabled stapling
- enabled session caching (instead of saying we store them, then prompty forgetting about them)
- hint to the browser that HTTPS is the prefered channel

This last change might give you some grief, but I assume we all updated our browsers in the last 5 years, and otherwise you will ignore HSTS anyway Basically this means that once you have been on https://www.openttd.org/ it is very likely that any following visit will also puts you on the https. Also linking to subdomains will put you on the https.
If this switch is giving you some real issues, please do let me know.

Long live security! I think .. or something.


Anyway, in result, I got myself a nice A+:

https://www.ssllabs.com/ssltest/analyze ... penttd.org

This is very important to me. Just so you know.

PS: tnx LordAro for wasting 1 hour of my life fixing the B back into an A+

[Transportman]

Anyway, in result, I got myself a nice A+:

https://www.ssllabs.com/ssltest/analyze ... penttd.org

It still gets a B when I check that link...Cleared the cache, now it is an A+.

[TrueBrain]

Biggest change regarding VCS you might be interesting in:

GIT / Mercurial links are now without the /openttd subdir. Old checkouts still work, but new ones are suggested to use the new approach. So:

hg clone http://hg.openttd.org/trunk.hg
git clone http://git.openttd.org/trunk.git
git clone git://git.openttd.org/trunk.git

[TrueBrain]

Migration completed.

http://svn.openttd.org
svn://svn.openttd.org
http://git.openttd.org
git://git.openttd.org
http://hg.openttd.org

All operational. Pushing for devs should work (harder to test ), and all callback scripts appear to be functional. W00p!

[Phreeze]

hi, it's always nice to see that someone is caring about security

having seen the security test links, I also fortified my server install.

unfortunately, debian6 only supports apache 2.22, which is responsible for "The server does not support Forward Secrecy with the reference browsers. "

are you using debian 6 too ? There's a 2.24 backport that exists, but I'm not a friend of that :-/

[Acol]

i get 504 Gateway Time out error on the http://www.openttd.org/en/ and to get to the forums i have to use google to skip that main page. then run that SSL tester on this site to get it working again. but this i have to do every time i start my PC again. so there is something not right on the main page.

[TrueBrain]

hi, it's always nice to see that someone is caring about security

having seen the security test links, I also fortified my server install.

unfortunately, debian6 only supports apache 2.22, which is responsible for "The server does not support Forward Secrecy with the reference browsers. "

are you using debian 6 too ? There's a 2.24 backport that exists, but I'm not a friend of that :-/

We don't use Apache. Basically, Apache is bloatware. At the moment we run nginx, and I am really enjoying it. It uses almost no memory, it is lightning fast, and cares about security etc

You can either pick up nginx from backports, or use dotdeb.

[TrueBrain]

i get 504 Gateway Time out error on the http://www.openttd.org/en/ and to get to the forums i have to use google to skip that main page. then run that SSL tester on this site to get it working again. but this i have to do every time i start my PC again. so there is something not right on the main page.

The 504 errors you got are not related to this. I have no clue what you considered a "fix", as the 504 means that the backend is down, and nothing you could do on your end would fix that.

In this case, Django tried to do several HTTP calls which never gave a reply. Instead of timing out, Django just sat down on his ass waiting for a reply that never came. For ever and ever and ever ... till I gave it a gentle (is kill -KILL considered gentle?) kick.

Found the issue, resolved it on 2 levels, so it should be stable again.

[Acol]

i get 504 Gateway Time out error on the http://www.openttd.org/en/ and to get to the forums i have to use google to skip that main page. then run that SSL tester on this site to get it working again. but this i have to do every time i start my PC again. so there is something not right on the main page.

The 504 errors you got are not related to this. I have no clue what you considered a "fix", as the 504 means that the backend is down, and nothing you could do on your end would fix that.

In this case, Django tried to do several HTTP calls which never gave a reply. Instead of timing out, Django just sat down on his ass waiting for a reply that never came. For ever and ever and ever ... till I gave it a gentle (is kill -KILL considered gentle?) kick.

Found the issue, resolved it on 2 levels, so it should be stable again.

ok thx for the info. it is now working as intended

[Alberth]

After the last VCS update, the Mercurial mirror had started to use the branch 'trunk' for the new commits. Before, it used 'default'.
Since having 2 heads may cause head-aches, eg in bisecting, that had to be fixed.

After some discussion, it was found that the only way out was to make a new mirror from scratch, which was done today.

Unfortunately, that means any 'old' clone will not work any more for updating, "hg pull" (or "hg fetch") will either return "502 bad gateway", or "abort: repository is unrelated". You will have to clone the Mercurial mirror again. I hope that does not cause too much trouble.

Sorry for the inconvenience.