Transport Tycoon Forums

The place to talk about Transport Tycoon
https://www.tt-forums.net/

Compiling for win64 on MinGW

https://www.tt-forums.net/viewtopic.php?t=61564

Page 1 of 1

Compiling for win64 on MinGW

Posted: 12 Jul 2012 14:19
by Gremnon
Having found no MinGW instructions that will result in a win64 build, I figured the reason was that if you set it up following the wiki's instructions, it doesn't have the necessary pieces to compile for win64.
So I found some instructions for MinGW/MSYS64, and set up a build environment to see if I could get it working. So far I've hit several issues, so this is not a viable option for people to use just yet, but it's gotten to the point where it's starting to go over the top of my meagre technical capabilities, and I have to humbly ask for a hand from people who know more than I do.
Fortunatly I had the sense to note down exactly what I was doing - mostly just working with the original MinGW instructions, with various tweaks and fixes to make it work. I've tided them up and put them into my user page on the OpenTTD Wiki here so people will have an idea of the issues I've been facing.

For an idea of the issues faced at present, scroll down to the 'Notes' Section I've left at the end of the page. A brief summary for those not interested:
xz libraries aren't being found when running ./configure
icu stops in the middle of the install stage due to a cursed library it refuses to find
At present, a compile-time error is preventing any further testing within this environment.

A bit of help figuring out these issues - and any others that arise - would be appreciated.
I know I could use MSVC, but the size of all the downloads needed for it put me off it unless there really is no other choice.

Re: Compiling for win64 on MinGW

Posted: 12 Jul 2012 17:12
by Terkhen
In my opinion, compilation instructions for MinGW and MinGW64 should be kept separate, because they are quite different. Otherwise the page gets too confusing :)

Some time ago we received some patches to fix some compilation issues with MinGW64. You can check them here and here. After that, I tried to follow the MinGW tutorial for MinGW64 tutorial. I abandoned it as a suitable option when I noticed that libpng, libicu and, more important, xz, did not compile under MinGW64. I'm happy to know that libpng now compiles under MinGW64, but until the xz issue is solved it is not possible to generate binaries that can load default savegames (which also means that it is not network compatible with most servers).

As part of my work maintaining the MinGW tutorial, I check if the existing libicu issues have been solved every two or three months. There are many MinGW64 issues, and since the MinGW issues are not fixed yet (it still fails to generate static libraries) I doubt that the MinGW64 issues will be solved anytime soon.

With regard to the more pressing xz issue, it seems that OpenSUSE has a mingw64-xz package, so it must be possible to get it working. I found the link here: https://build.opensuse.org/package/file ... gw%3Awin64

Re: Compiling for win64 on MinGW

Posted: 12 Jul 2012 17:27
by Gremnon
I figured that since architecture aside they're technically the same thing, I'd use the existing MinGW page as a template - I did consider making my tutorial page on it just a list of things you have to do differently, but there were quite a number of changes - it definitely needs it's own page.

On xz - it does compile sucessfully- but the files it generates appear to be having an existential crisis of sorts. Only OpenTTD's configure script cannot find them.
I'll have a look at the patches and OpenSUSE package to see what it suggests though, it could hold clues I'm missing. If necessary, I suppose I could just direct users to the OpenSUSE package and extract it as necessary, but I'd rather not do that - in my experience on Linux systems, it generally doesn't work without a lot of issues. Windows is likely to be even worse.

At least I know now that icu isn't just me, and that it was at least in some part expected. It'd be nice to have it as an available option, if only for those who might use it, but really it isn't exactly a necessity otherwise.

Re: Compiling for win64 on MinGW

Posted: 17 Jul 2012 16:15
by LeXa2
Well, actually I hadn't had much problems compiling OpenTTD with MingW+MSYS + x86_64-w64-mingw32 GCC 4.6.3 binaries (rubenvb custom build).
The only thing I skipped is libicu compilation as I hadn't had a need for it.

OpenTTD's configure script is, well, a bit unusual comparing to traditional autoconf stuff and sometimes it uses "non-traditional" methods to search for required libs, but a bit patience while reading config.lib sources helps to sort out all the problems.

My build system is set up as following:

Mingw-MSYS installed to c:\msys.
64 and 32 bit mingw64 toolchains are installed to c:\mingw64 and c:\mingw32 folders resp.
Folder c:\mingw is a "hard link" to c:\mingw64 (NTFS joint created by executing: mklink /J c:\mingw c:\mingw64). It is needed to make Eclipse/CDT happy (i.e. find mingw at the place it expects it to be and stop annoying me with "configuration support is not installed on the system" messages. In other words, making this link is optional and isn't strictly required.

MSYS's fstab (c:\msys\etc\fstab) looks like:

Code: Select all

C:/mingw64      /mingw
C:/mingw32      /mingw32
C:/mingw64      /mingw64
c:\msys\home\%USERNAME%\.profile contains:

Code: Select all

# Filter out mingw from path
export PATH="`echo $PATH | sed -r 's,(\.|/mingw64/bin|/mingw32/bin|/mingw/bin):,,g'`"

echo ""
echo "Don't forget to chose which mingw to use, 64 or 32 bit."
echo "To do it type either 32 or 64."
echo ""

alias 32=". 32bit"
alias 64=". 64bit"
alias ll="ls -l"
Files 32bit and 64bit are shell scripts I created for "switching" MSYS between 32/64 bit targets "on the fly", they are located inside c:\msys\bin folder and contain:
/bin/32bit:

Code: Select all

#!/bin/sh

# Filter out mingw from path
export PATH="`echo $PATH | sed -r 's,(\.|/mingw64/bin|/mingw32/bin|/mingw/bin):{0\,1},,g' | sed -r 's,:+,:,g' | sed -r 's,:$,,'`:/mingw32/bin"
/bin/64bit:

Code: Select all

#!/bin/sh

# Filter out mingw from path
export PATH="`echo $PATH | sed -r 's,(\.|/mingw64/bin|/mingw32/bin|/mingw/bin):{0\,1},,g' | sed -r 's,:+,:,g' | sed -r 's,:$,,'`:/mingw64/bin"
Also I have pkg-config binary installed to c:\msys\bin. This binary had been downloaded from official gnome FTP: http://ftp.gnome.org/pub/gnome/binaries ... endencies/ .

My habbit for building dep libs is to install them into separate dirs that are easy to remove later on rather than doing system-wide install (MSYS-wide in this case).
Thus, I've got a separate folder per arch for libs installation located at: $HOME/OpenTTD/deps/inst32/ and $HOME/OpenTTD/deps/inst64/.

Selecting target arch (using "32" and "64" shell aliases defined in $HOME/.profile) if required prior to proceeding with compiling libs.
To check if correct PATH had been set up try executing "as --version" and looking at the last line of the output. Target should be "x84_64-w64-mingw32" for 64 bit and "i686-w64-mingw32" for 32 bit toolchain.

Next I import into environment some convenient vars to simplify build process. There are two shell scripts created specifically for this purpose placed into $HOME/OpenTTD/ folder:

$HOME/OpenTTD/settings-32bit.sh:

Code: Select all

export MYP=$HOME/OpenTTD/deps/inst32
export HOST=i686-w64-mingw32

# Construct pkg-config paths
PKG_CONFIG_PATH=""
for i in $MYP/*/lib/pkgconfig ; do
  [ -d $i ] || continue
  PKG_CONFIG_PATH="$PKG_CONFIG_PATH:$i"
done

export PKG_CONFIG_PATH
$HOME/OpenTTD/settings-64bit.sh:

Code: Select all

export MYP=$HOME/OpenTTD/deps/inst64
export HOST=x86_64-w64-mingw32

# Construct pkg-config paths
PKG_CONFIG_PATH=""
for i in $MYP/*/lib/pkgconfig ; do
  [ -d $i ] || continue
  PKG_CONFIG_PATH="$PKG_CONFIG_PATH:$i"
done

export PKG_CONFIG_PATH
This scripts should be run prior to configuring/building any dep lib or openttd itself, and also after successful build and installation of any of dep lib (so pkg-config search path gets updated with the path to newly built/installed lib). Scripts should be run like this:

Code: Select all

# . $HOME/OpenTTD/settings-64bit.sh
Notice dot and space typed in before path and script name. It is just shortcut for "source" bash command, i.e. above command is equivalent to "source $HOME/OpenTTD/settings-64bit.sh", just a bit shorter.

Next, to the libs. Each lib except for xz requires no patches to work correctly. xz hits unfortunate mingw64 bug I had reported to mingw64 devs yesterday and today it had been fixed in mingw64 trunk. As prebuilt toolchains hadn't been updated with this fix yet it is required to patch xz sources a bit to workaround this compiler bug. Build order for dep libs isn't strictly defined, the only real requirement is to build libpng after zlib as first requires later. To simplify libs build process I had created one shell script per dep lib. Each script should be placed to resp. dep lib unpacked source tree and executed from there. I place unpacked dep. libs sources to $HOME/OpenTTD/deps/src/, for example zlib goes to: $HOME/OpenTTD/deps/src/zlib-1.2.7.

Zlib "to_make.sh" script:

Code: Select all

#!/bin/bash

[ -z ${MYP} ] && exit 1
which gcc 2>/dev/null >/dev/null || exit 1

make -f win32/Makefile.gcc BINARY_PATH=${MYP}/zlib/bin INCLUDE_PATH=${MYP}/zlib/include LIBRARY_PATH=${MYP}/zlib/lib SHARED_MODE=1 all install clean && \
make -f win32/Makefile.gcc BINARY_PATH=${MYP}/zlib/bin INCLUDE_PATH=${MYP}/zlib/include LIBRARY_PATH=${MYP}/zlib/lib SHARED_MODE=0 all install clean
libpng "to_make.sh" script:

Code: Select all

#!/bin/bash

[ -z ${MYP} ] && exit 1
[ -z ${HOST} ] && exit 1
pkg-config --exists zlib || {
  echo "pkg-config reports no zlib, build failed."
  exit 1
}

which gcc 2>/dev/null >/dev/null || exit 1

CFLAGS="$CFLAGS `pkg-config --cflags zlib`" \
LDFLAGS="$LDFLAGS `pkg-config --libs-only-L zlib`" \
./configure --prefix=${MYP}/libpng --build=${HOST} --enable-shared --enable-static && \
make -j8 all && \
make -j8 install && \
make -j8 clean
lzo "to_make.sh":

Code: Select all

#!/bin/bash

[ -z ${MYP} ] && exit 1
[ -z ${HOST} ] && exit 1

which gcc 2>/dev/null >/dev/null || exit 1

./configure --prefix=${MYP}/liblzo --build=${HOST} --enable-shared --enable-static && \
make -j8 all && \
make -j8 install && \
make -j8 clean
freetype2 "to_make.sh":

Code: Select all

#!/bin/bash

[ -z ${MYP} ] && exit 1
[ -z ${HOST} ] && exit 1

which gcc 2>/dev/null >/dev/null || exit 1

./configure --prefix=${MYP}/freetype --build=${HOST} --enable-shared --enable-static && \
make -j8 all && \
make -j8 install && \
make -j8 clean
xz "to_make.sh":

Code: Select all

#!/bin/bash

[ -z ${MYP} ] && exit 1
[ -z ${HOST} ] && exit 1

which gcc 2>/dev/null >/dev/null || exit 1

./configure \
	--prefix=${MYP}/xz --build=${HOST} --enable-shared --enable-static \
	--disable-xz --disable-xzdec --disable-lzmadec --disable-lzmainfo \
	--disable-lzma-links --disable-scripts --disable-nls && \
make -j8 all && \
make -j8 check && \
make -j8 install && \
make -j8 clean
Now, back to xz vs. compiler bug patch. It's as simple as changing {xz source folder}/src/common/sysdefs.h file commenting out following lines:

Code: Select all

// Get standard-compliant stdio functions under MinGW and MinGW-w64.
//===>Commented out to workaround compiler bug<=== #ifdef __MINGW32__
//===>Commented out to workaround compiler bug<=== #	define __USE_MINGW_ANSI_STDIO 1
//===>Commented out to workaround compiler bug<=== #endif
Thus, build process looks like this:

Code: Select all

# 64
# cd $HOME/OpenTTD
# . settings-64bit
# pushd ./deps/src/zlip-1.2.7/
# ./to_make.sh
# popd
# . settings-64bit
# pushd ./deps/src/libpng-1.5.12/
# ./to_make.sh
# popd
# . settings-64bit
# pushd ./deps/src/lzo-2.06/
# ./to_make.sh
# popd
# . settings-64bit
# pushd ./deps/src/freetype-2.4.5/
# ./to_make.sh
# popd
# . settings-64bit
# pushd ./deps/src/xz-5.0.4/
# ./to_make.sh
# popd
# . settings-64bit
At this point all required deps should be built, but it's nice to check if pkg-config knows about it:

Code: Select all

# pkg-config --list-all
libpng    libpng - Loads and saves PNG files
freetype2 FreeType 2 - A free, high-quality, and portable font engine.
zlib      zlib - zlib compression library
libpng15  libpng - Loads and saves PNG files
liblzma   liblzma - General purpose data compression library
Nice! All libs are there except for lzo - which simply doesn't support integration with pkg-config.
Time to build OpenTTD itself. Yet another shell script to handle configure:

OpenTTD's "to_conf.sh":

Code: Select all

#!/bin/bash

[ -z ${MYP} ] && {
  echo "Export correct MYP first!"
  exit 1
}

[ -z ${HOST} ] && {
  echo "Export correct HOST first!"
  exit 1
}

for i in zlib libpng liblzma freetype2 ; do
	pkg-config --exists $i || {
	  echo "pkg-config reports no $i, build failed."
	  exit 1                                                                                                                                                                                   
	}
done
                                                                                                                                                                                           
which gcc 2>/dev/null >/dev/null || {
  echo "No usable gcc found on PATH."
  exit 1
}

CFLAGS="$CFLAGS `pkg-config --cflags zlib` -I${MYP}/liblzo/include" \
LDFLAGS="$LDFLAGS `pkg-config --libs-only-L zlib` -L${MYP}/liblzo/lib" \
./configure \
	--prefix-dir=$MYP/OpenTTD/ --with-zlib=`pkg-config --variable=libdir zlib` \
	--with-lzma --with-lzo2=${MYP}/liblzo/lib --with-png=`pkg-config --variable=exec_prefix libpng`/bin/libpng-config \
	--with-freetype=`pkg-config --variable=exec_prefix freetype2`/bin/freetype-config \
	"$@"
Place it into root of OpenTTD source dir and you're set to go. For "release" build you could simply run "./to_conf.sh", check that no errors are reported and follow up with "make -j5" (assuming that you've got 4 core CPU; adjust number after "-j" to match number of the CPU cores + 1). For "debug" build containing info that is required to conveniently debug produced binary under gdb run to_conf.sh script like this:

Code: Select all

# CFLAGS="-O0 -g" ./to_conf.sh --disable-strip
Check for errors and if none proceed with make -j5. Upon (hopefully) successful end of compilation process you would end up with fresh openttd.exe binary in {openttd source}/bin folder.

"make install" wouldn't work correctly under MSYS as "install" target wasn't designed with MSYS in mind, it's best suited for various *nix-es. If you want to get "installation bunble" - run "make bundle" and collect all the goodies from {openttd source}/bundle folder.

Had successfully built binary for 64 bit and want to repeat it for 32bit? No problems, it is as simple as (assuming you're at openttd source dir root and have all dep libs built/instaled for 32bit arch):

Code: Select all

# make clean
# 32
# . $HOME/OpenTTD/settings-32bit
# ./to_conf.sh
# make -j5
Hope that helps.

Re: Compiling for win64 on MinGW

Posted: 17 Jul 2012 17:14
by Terkhen
If it needs any changes to config.lib and/or other OpenTTD files, bear in mind that patches that fix OpenTTD compilation in MinGW64 without altering compilation in MinGW and all other supported platforms are welcome.

Re: Compiling for win64 on MinGW

Posted: 18 Jul 2012 07:16
by LeXa2
Terkhen wrote:If it needs any changes to config.lib and/or other OpenTTD files, bear in mind that patches that fix OpenTTD compilation in MinGW64 without altering compilation in MinGW and all other supported platforms are welcome.
I hadn't had to do any changes to compile the thing. It's just my personal taste that used configuration system is, em, a bit inconsistent with traditional autoconf "./configure" usage (for example, traditional way to specify install prefix is --prefix while OpenTTD's configure expects --prefix-dir instead) and a way it makes use of pkg-config is also "strange". I mean, it does use pkg-config to detect liblzma presence and usability, but doesn't do so for zlib, freetype2 or libpng. In case one has libs installed at "non-standard" locations (/opt/<pkgname> *BSD style installations) it takes some time to figure out what is correct XXX to be passed to --with-*=XXX. With zlib it's OK to pass the path to folder where libz.a is located (but one still would have to adjust CFLAGS/LDFLAGS to point to correct include/lib dirs). For freetype and libpng it is expected that full patch to XXX-config script is supplied. Properly documenting these aspects in "configure --help" output wouldn't hurt IMO, but as sources are there and could be easily read - it isn't matter of an urgent need.

P.S. Once again: all above isn't specific in any way to MINGW64, my comments about config.lib are just a matter of personal preference.

Re: Compiling for win64 on MinGW

Posted: 18 Jul 2012 09:39
by Gremnon
I'll have a run through the process myself once I fix my msys64 telling me it doesn't exist. It might lend a hand to the strangeness I had of configure not being able to find liblzma, and if I can fix that and put up a user-readable set of instructions onto the wiki, we'll all be able to have tea and win64 builds to go. Or at least the builds, anyway. I might nick the tea.

Edit: Have tested it myself. Notes are below - couched in a way I'll understand, so apologies if I make you think to figure out what I mean.
Using MinGW32 linked to from OTTDWiki article
Mounted dir's did not show up under 'ls -a /' but appear to be present anyway
OpenTTD 'to_conf' script reported host and target as 'mingw32' - still correct for win64 build?
Lots of compile-time warnings w.r.t. lzo (warning <variable> is not defined - appears to be for different build-targets?)

The binary it generated while using the settings and scripts that should have resulted in a win64 build was still, however, a win32 binary. I didn't bother to re-test using the win32 scripts, etc because of this.

Re: Compiling for win64 on MinGW

Posted: 18 Jul 2012 14:21
by LeXa2
Gremnon wrote:Using MinGW32 linked to from OTTDWiki article
Mounted dir's did not show up under 'ls -a /' but appear to be present anyway
OpenTTD 'to_conf' script reported host and target as 'mingw32' - still correct for win64 build?
Lots of compile-time warnings w.r.t. lzo (warning <variable> is not defined - appears to be for different build-targets?)

The binary it generated while using the settings and scripts that should have resulted in a win64 build was still, however, a win32 binary. I didn't bother to re-test using the win32 scripts, etc because of this.
Let's check if our basic "definitions" are the same:

a) MinGW - it is the name (and related 32bit-only toolchain) of the original project that had "ported" gcc to Win32.
Homepage: http://mingw.org
Target "triplet": i686-pc-mingw32 - 32bit

b) MinGW-w64 - it is a "fork" of the original MinGW project. It includes mingw64 and mingw32 toolchains targeting 64bit and 32bit resp.
Homepage: http://mingw-w64.sourceforge.net/
Target "triplets": x86_64-w64-mingw32 for 64bit, i686-w64-mingw32 for 32bit.

c) MSYS - it is a handy pack of unix-like tools ported to win32. It was created and maintained by original MinGW project team.
Homepage: http://mingw.org/node/18

d) MinWG-w64 MSYS - it is the same binaries as (c), but provided for download as one ready-to-use pack by MinGW-w64 devs.
Homepage: https://sourceforge.net/apps/trac/mingw-w64/wiki/MSYS

Now, having this definitions in place, one wishing to build 64bit OpenTTD binary on Windows using gcc should stick to MinGW-w64 and use toolchain targeting x86_64-w64-mingw32.
For 32bit binary it should be possible to use either i686-w64-mingw32 targeting toolchain from MinGW-w64 project or i686-pc-mingw32 from original MinGW project. It doesn't matter who was the vendor of MSYS environment as basically they are the same and differ only in the installation procedure.

You had mentioned "msys64" in your post. If by it you meant "64bit version of MSYS environment" - there's no such thing (yet?). MSYS is 32bit but it doesn't matter as you could perfectly compile 64bit binaries under it. What actually matters is the toolchain that is used (gcc/binutils). If you would took a look back onto my original post you could notice that I've got two toolchains installed - 32bit in C:\mingw32 and 64bit in C:\ming64, both provided by mingw-w64 project, namely they are GCC 4.6.3 "custom build" by rubenvb. I had configured my MSYS in a way so I could switch between "active" toolchains by simply executing "32" or "64" at bash prompt.

Upd. Yeah, lzo headers produce a lot of warnings, but they seems to be harmless. I hadn't had a chance to investigate the cause, but most probably it is related to a fact that lzo's headers are not expected to be used on Win32/64 platform under anything other than MSVC. I might be wrong with it though.

Re: Compiling for win64 on MinGW

Posted: 18 Jul 2012 14:30
by Gremnon
Fairly sure I found an msys64 through a google search, but that aside I didn't use it here. I started from scratch instead. MinGW - as in, the one linked to on the 'Compiling on MinGW' page - was installed. Since it installs msys with that, that's what I used. In essence, the only real difference there is that my msys was installed with MinGW32 inside the MinGW32 directory, but is otherwise set up the same way.

While I don't use the custom build of MinGW64 you mentioned, I do use a build of MinGW64 - unless the custom build has something mine doesn't, I don't think the issue lies there.
Your quick scripts runnable by entering '64' or '32' I used. I made certain to start off using '64' and before running the to_conf scripts you provided the code for I also sourced the settings_64bit as described, which by what your post says should set the build target to be the win64 build.
I used the same paths for everything as is used in your post, with the exceptions of the location of msys as mentioned and obviously the user directory.

When discounting the differing location of msys, the only difference I can see is that I do not use the same custom build of MinGW64 as you are - and I can find nothing that suggests this is the case. Otherwise, I followed exactly what you laid out, and got a 32-bit binary from what should have provided a 64-bit one.

Re: Compiling for win64 on MinGW

Posted: 18 Jul 2012 14:42
by LeXa2
Gremnon wrote:Fairly sure I found an msys64 through a google search, but that aside I didn't use it here.
Could you please post a link to it if you accidentally happen to come across it? Because AFAIK there's no 64bit version of MSYS environment for now.
Gremnon wrote:MinGW - as in, the one linked to on the 'Compiling on MinGW' page - was installed.
It's 32bit only toolchain. It won't produce 64bit binaries :-).
Gremnon wrote:While I don't use the custom build of MinGW64 you mentioned, I do use a build of MinGW64 - unless the custom build has something mine doesn't, I don't think the issue lies there.
What's the source where you get your MinGW64 toolchain? What does it print when you execute gcc --version and as --version?
Gremnon wrote:Your quick scripts runnable by entering '64' or '32' I used. I made certain to start off using '64' and before running the to_conf scripts you provided the code for I also sourced the settings_64bit as described, which by what your post says should set the build target to be the win64 build.
They would only work in case you haven't got any other gcc binary available under $PATH. Start up your MSYS, run "64" and then execute "which gcc". It should output /mingw64/bin/gcc.exe in case you've got build environment configured correctly. If it displays something else (for example /bin/gcc, /mingw/gcc, /mingw32/gcc or even something other) - something got wrong with your setup and /mingw64/bin/gcc isn't the first gcc.exe instance found on your $PATH)

Gremnon wrote:When discounting the differing location of msys, the only difference I can see is that I do not use the same custom build of MinGW64 as you are...
It shouldn't matter as long as your MinGW64 is (a) really a toolchain targeting 64bit and (b) comes first on $PATH inside your build environment.

Re: Compiling for win64 on MinGW

Posted: 18 Jul 2012 14:52
by Gremnon
First off, this is where I got msys64 from, right here

I'm aware that MinGW32 isn't capable of providing a win64 build - that's why I mounted MinGW64 on /mingw64, which was extracted to C:\MinGW64 separately - not a part of MinGW32, I just made the fstab mount it at the same location yours was mounted.

'gcc --version' reports:
gcc.exe (GCC) 4.6.2
Followed by the copyright etc. It's a vanilla, unmodified thing. I downloaded the MinGW64 from here using the 'mingw-w64-bin_i686-mingw_20111220' choice - which as I understand their naming system is a 32-bit toolchain (i.e. runs on 32-bit) that can create 64-bit binaries.

Interestingly - and perhaps the source of the issue here - running 'which gcc' claims that no gcc is found. It could be I've missed something somewhere, I hadn't thought to look into what it returned. Why it says gcc is not found when gcc --version works fine I'm not sure. I'll recheck and make sure I haven't missed something.

Re: Compiling for win64 on MinGW

Posted: 18 Jul 2012 15:16
by LeXa2
Gremnon wrote:First off, this is where I got msys64 from, right here
It's 32bit, as I thought: "Home / External binary packages (Win64 hosted) / MSYS (32-bit)"
... I mounted MinGW64 on /mingw64, which was extracted to C:\MinGW64 separately - not a part of MinGW32, I just made the fstab mount it at the same location yours was mounted.
That's good.
'gcc --version' reports:
gcc.exe (GCC) 4.6.2
Followed by the copyright etc. It's a vanilla, unmodified thing. I downloaded the MinGW64 from here using the 'mingw-w64-bin_i686-mingw_20111220' choice - which as I understand their naming system is a 32-bit toolchain (i.e. runs on 32-bit) that can create 64-bit binaries.
Could you please post full text of requested outputs if possible? I was asking for "as --version" output because it contains target triplet, while gcc --version does not.
In any case, thanks to the name of the mingw-w64 installation file you had mentioned I could state that you: (a) downloaded a possibly unstable toolchain binary (it's automated nighty build) which is (b) targeting 64bit but is itself 32bit (i.e. it could be run on 32bit Windows but produces binaries that are only executable under 64bit Windows). It should suffice for your needs in theory but in case your host is 32bit there might be problems with successfully executing configure scripts (because binaries produced by toolchain won't work on your system, i.e. essentially you would be cross-compiling and would have to inform configure scripts about it). It should not be a problem if your host OS is 64bit.
Interestingly - and perhaps the source of the issue here - running 'which gcc' claims that no gcc is found. It could be I've missed something somewhere, I hadn't thought to look into what it returned. Why it says gcc is not found when gcc --version works fine I'm not sure. I'll recheck and make sure I haven't missed something.
Create empty folders for your mountpoints, i.e. close your MSYS shell (make sure there are no stale sh.exe, bash.exe, mintty.exe or rxvt.exe processes using windows Task Manager; kill each of them if present), in Windows Explorer navigate into the folder where you have MSYS installed (C:\MSYS usually) and create there two folders that would act as "mount points": "mingw32" and "mingw64". Due to a way MSYS is implemented having empty dirs in place of "mount points" isn't necessary for binaries to be found and executed, but most POSIX-aware tools would go mad and report "strange" things.

Re: Compiling for win64 on MinGW

Posted: 18 Jul 2012 15:31
by Gremnon
I guess I missed a few bits there.
The 'as --version' output:
GNU assembler (GNU Binutils) 2.22
Copyright (C) 2011 Free Software Foundation, Inc.
This program is free software; you may distribute it under the terms of
the GNU General Public License version 3 or later.
This program has absolutely no warranty.
This assembler was configured for a target of 'mingw32'

That said, I ran that before I did the mingw directories.
Running it after I get the same thing after running 32bit.

I think I found the key problem with the MinGW64 one though. I didn't properly look into the extracted folder, which itself contains a 'mingw' folder of it's own - that's what I should have been mounting at /mingw64, not the parent directory. It finds the gcc in there now, which outputs...

gcc --version...
gcc.exe (GCC) 4.7.0 20111220 (experimental)
Copyright (C) 2011 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE

and the as --version...
GNU assembler (GNU Binutils) 2.22
Copyright (C) 2011 Free Software Foundation, Inc.
This program is free software; you may distribute it under the terms of
the GNU General Public License version 3 or later.
This program has absolutely no warranty.
This assembler was configured for a target of 'x86_64-w64-mingw32'

I think I've got it this time. Thanks for bearing with me.

Edit: ...or not. Thought it best to recompile the dependencies as well. zlib threw this at me:
gcc -O3 -Wall -c -o adler32.o adler32.c
gcc.exe: error: CreateProcess: No such file or directory

Re: Compiling for win64 on MinGW

Posted: 18 Jul 2012 16:03
by LeXa2
Gremnon wrote:The 'as --version' output: ...
This assembler was configured for a target of 'mingw32'
It is assembler from the toolchain by original MinGW that is targeting 32bit.
Gremnon wrote:...I didn't properly look into the extracted folder, which itself contains a 'mingw' folder of it's own - that's what I should have been mounting at /mingw64, not the parent directory. It finds the gcc in there now, which outputs...
That is the "correct" gcc from mingw-w64 targeting 64bit. I've downloaded mingw-w64 gcc version you had mentioned and checked the archive contents. Actually there are two identical byte-to-byte copies of toolchain there, one is under mingw-w64-bin_i686-mingw_20111220.zip/mingw/, and another is under mingw-w64-bin_i686-mingw_20111220.zip/x86_64-w64-mingw32/.

As this toolchain is of "cross-compile flavour", binaries in parent /bin folder are prefixed by target triplet, namely "x86_64-w64-mingw32-*.exe". To convert in into "traditional" naming scheme it is sufficient to move all files from mingw-w64-bin_i686-mingw_20111220.zip/minw/bin into parent mingw-w64-bin_i686-mingw_20111220.zip/bin folder and delete all other files in mingw-w64-bin_i686-mingw_20111220.zip/mingw/ whatsoever (as they're excess dupes of files contained in mingw-w64-bin_i686-mingw_20111220.zip/x86_64-w64-mingw32). Or you could try to mount mingw-w64-bin_i686-mingw_20111220.zip/mingw/ or mingw-w64-bin_i686-mingw_20111220.zip/x86_64-w64-mingw32/ as /mingw64/ into your MSYS environment but that could potentially result in misbehaving toolchain if there are any other as/ld/cc1 binaries available on default search path.

Re: Compiling for win64 on MinGW

Posted: 18 Jul 2012 16:08
by LeXa2
Gremnon wrote:Edit: ...or not. Thought it best to recompile the dependencies as well. zlib threw this at me:
gcc -O3 -Wall -c -o adler32.o adler32.c
gcc.exe: error: CreateProcess: No such file or directory
That's what I was meaning when been writing "could potentially result in misbehaving toolchain" :-). Try "converting" it as I suggested or simply download other toolchain which is not of "cross-compile flavour" (i.e. download one without i686 in file name, like mingw-w64-bin_x86_64-mingw*.zip so it would be 64bit hosted toolchain that targets the same 64bit triplet; would only work if you're on 64bit windows though).

Re: Compiling for win64 on MinGW

Posted: 18 Jul 2012 16:30
by Gremnon
It's a good thing I'm on a 64-bit host then.
I couldn't find a 'made for win64 to build64' toolchain in the automated builds so went ahead and, just to be sure I got it working, got the rubenvb personal build, as it has such a chain, and it seems to be working now.
If you don't mind, once I confirm (Just to be sure) I can build it, I'll work with your instructions above and see if I can make a page on the Wiki for others to use.

Re: Compiling for win64 on MinGW

Posted: 18 Jul 2012 16:47
by LeXa2
Gremnon wrote:It's a good thing I'm on a 64-bit host then.
I couldn't find a 'made for win64 to build64' toolchain in the automated builds so went ahead and, just to be sure I got it working, got the rubenvb personal build, as it has such a chain, and it seems to be working now.
If you don't mind, once I confirm (Just to be sure) I can build it, I'll work with your instructions above and see if I can make a page on the Wiki for others to use.
Yeah, looks like guys at mingw-w64 no longer produce automated builds for windows hosts, so one would have to use one of "personal builds" (or get brave and try to build toolchain himself). It's good to hear that you could finally get it working.

You could use info I posted in this thread in any way you want, including improving OpenTTD Wiki (which would be very nice). I'd also suggest to provide a link to this thread on the wiki page (among other info) though so people interested in details of the original discussion would be able to find it.

Re: Compiling for win64 on MinGW

Posted: 18 Jul 2012 16:51
by Gremnon
It's a bit irritating, but at least I've got the dependencies to compile. Now it's OpenTTD that's hitting me again, this time with the same issue I faced originally using the original method:

Code: Select all

make[1]: Entering directory `/c/Users/Liam/OpenTTD/trunk/objs/lang'
[LANG] Compiling string.cpp
c:/Users/Liam/OpenTTD/trunk/src/string.cpp: In function 'int snprintf<char*, size_t, const char*, ...)':
c:/Users/Liam/OpenTTD/trunk/src/string.cpp:381:11: error: redefinition of 'int snprintf<char*, size_t, const char*, ...)'
In file included from c:\mingw64\bin\../lib/gcc/x86_64-w64-mingw32/4.7.0/../../../../x86_64-w64-mingw32/include/c++/4.7.0/cdstdio:44:0'
from c:/Users/Liam/OpenTTD/trunk/src/stdafx/h:59,
from c:/Users/Liam/OpenTTD/trunk/src/string.cpp:12:
cL\mingw64\bin\../lib/gcc/x86_64-w64-mingw32/4.7.0/../../../../x86_64-w64-mingw32/include/stdio.h:555:5: error 'int snprintf<char*, size_t, const char*, ...)' previously defined here
make[1]: *** [string.o] Error 1
make[1]: Leaving directory `/c/Users/Liam/OpenTTD/trunk/objs/lang'
make: *** [all] Error 1
I'm starting to think my computer is deliberately trying to stop me or something.

Re: Compiling for win64 on MinGW

Posted: 18 Jul 2012 20:31
by LeXa2
Gremnon wrote:

Code: Select all

.../src/string.cpp:381:11: error: redefinition of 'int snprintf<char*, size_t, const char*, ...)'
.../include/stdio.h:555:5: error 'int snprintf<char*, size_t, const char*, ...)' previously defined here
I'm starting to think my computer is deliberately trying to stop me or something.
Looks like OpenTTD's fault. Toolchain you use is fresh enough and have got C99 compatible snprintf. It wasn't so with previous mingw runtimes and OpenTTD have got it's own wrapper implementation for such case. Here is the faulting code:

Code: Select all

/* Since version 3.14, MinGW Runtime has snprintf() and vsnprintf() conform to C99 but it's not the case for older versions */
#if (__MINGW32_MAJOR_VERSION < 3) || ((__MINGW32_MAJOR_VERSION == 3) && (__MINGW32_MINOR_VERSION < 14))
int CDECL snprintf(char *str, size_t size, const char *format, ...)
{
	va_list ap;
	int ret;

	va_start(ap, format);
	ret = vsnprintf(str, size, format, ap);
	va_end(ap);
	return ret;
}
#endif /* MinGW Runtime < 3.14 */
As could be seen OpenTTD declares C99 compatible snprintf in case reported MinGW runtime version is less than 3.14. Trouble is that actually there are two different MinGW's as I had already posted earlier: original MingGW runtime from mingw.org and kind-a forked (actually it is not fork, but that's another sad and long story; google for it if curious) runtime version from mingw-w64 guys. Second one define some "compatibility" macros, including __MINGW32_MAJOR_VERSION and __MINGW32_MINOR_VERSION that for mingw-w64 runtime release 2.0.3 are set to be report 3.11.

Now, back to snprintf. In minw-w64 2.0.x runtime versions by default snprintf was implemented as a part of mingw runtime, namely in file libmingwex.a:

Code: Select all

# objdump.exe -r -d /mingw64/x86_64-w64-mingw32/lib/libmingwex.a | grep -A20 '<snprintf>' | less
0000000000000000 <snprintf>:
   0:   48 83 ec 38             sub    $0x38,%rsp
   4:   4c 89 4c 24 58          mov    %r9,0x58(%rsp)
   9:   4c 8d 4c 24 58          lea    0x58(%rsp),%r9
   e:   4c 89 4c 24 28          mov    %r9,0x28(%rsp)
  13:   e8 00 00 00 00          callq  18 <snprintf+0x18>
                        14: R_X86_64_PC32       _vsnprintf
  18:   48 83 c4 38             add    $0x38,%rsp
  1c:   c3                      retq
  1d:   90                      nop
  1e:   90                      nop
  1f:   90                      nop
In ming-w64 3.x (which is currently in alpha testing) default was changed to provide implementation for snprintf (very similar to one OpenTTD has) directly as a part of stdio.h headers. Thus you end up with two places in code defining the same named function with same parameters signature, which is obviously wrong. You hit this one because toolchain you have is other than mine. I have GCC 4.6.3 based "rubenvb personal build" toolchain, while it seems that you have GCC 4.7.x based one.

What to do? Fix OpenTTD sources by changing above code fragment in /src/string.cpp to:

Code: Select all

/* Since version 3.14, MinGW Runtime has snprintf() and vsnprintf() conform to C99 but it's not the case for older versions */
#if (__MINGW32_MAJOR_VERSION < 3) || ((__MINGW32_MAJOR_VERSION == 3) && (__MINGW32_MINOR_VERSION < 14))
#ifndef __MINGW_HAVE_ANSI_C99_PRINTF
int CDECL snprintf(char *str, size_t size, const char *format, ...)
{
	va_list ap;
	int ret;

	va_start(ap, format);
	ret = vsnprintf(str, size, format, ap);
	va_end(ap);
	return ret;
}
#endif /* MinGW-W64 with C99 snprintf */
#endif /* MinGW Runtime < 3.14 */
It might not work (I hadn't tested), but if it fail try this:

Code: Select all

/* Since version 3.14, MinGW Runtime has snprintf() and vsnprintf() conform to C99 but it's not the case for older versions */
#if (__MINGW32_MAJOR_VERSION < 3) || ((__MINGW32_MAJOR_VERSION == 3) && (__MINGW32_MINOR_VERSION < 14))
#if 0
int CDECL snprintf(char *str, size_t size, const char *format, ...)
{
	va_list ap;
	int ret;

	va_start(ap, format);
	ret = vsnprintf(str, size, format, ap);
	va_end(ap);
	return ret;
}
#endif /* 0 */
#endif /* MinGW Runtime < 3.14 */
It's a hack that de-facto comments out OpenTTD's definition of snprintf and should make your compiler happy. I'm not sure what would be correct way to detect this problem for OpenTTD. If it were using autoconf/automake it wouldn't be a big deal, but here devs would either have to stick with conditional compilation-fu or implement some tests in config.lib for this one.

Re: Compiling for win64 on MinGW

Posted: 31 Jul 2012 17:51
by LeXa2
And yet another followup about compiling OpenTTD with either MinGW or MinGW-W64 toolchains: since gcc 4.7.0 option "-mms-bitfields" had been turned on by default (http://mingw-users.1079350.n2.nabble.co ... 33i40.html). It means that packing of some structs might be affected, and it really happen for OTTD. Suffering structure (it's the only one I know about, probably there are some more) is "BitmapFileHeader" from screenshot.cpp. It is struct that consists of 1 word and 3 double words summing up to 14-bytes. Without packing it would end up padded with extra word so its size would be 16 bytes. OTTD defines attribute "packed" for it which should - in theory - disable padding for this specific struct so it would end up being expected 14 bytes in size. With -mms-bitfields turned on things work different and de-facto compiler ignores "packed" attribute and still pads stuct with extra word, resulting in 16 bytes struct size.

Problem affects both 32bit and 64bit targets and manifests itself in compile-time assertion about struct size not being the expected 14 bytes. To "fix" the things it is sufficient to include "-mno-ms-bitfields" in CFLAGS and things would work. I don't know should be this problem considered OTTD bug or not as I hadn't had a chance to get familar with what -mms-bitfields is expected to do and why had it been turned on by default for PE targets. Maybe would took a more close look on it later.

P.S. Another problem could be hit if using earlier versions of gcc + 32bit PE (Windows) target + SSE2 and SIMD auto-vectorizer turned on: compiler tends to produce invalid code doing aligned accesses on stack that is not warranted to be aligned properly. End result is OTTD binary crashing with SIGFAULT as soon as any auto-vectorized inline proc happens to hit unaligned variable on stack. Workarounds are: (a) not to use SSE2 + auto-vectorized or (b) use 64bit target which seems to be free from this problem or (c) update mingw gcc to 4.7.x+ where most of these bugs were fixed.

P.P.S. In order to get LTO compile/link mode working on mingw-w64 one should edit config.lib and remove all occurrences of the long-ago-deprecated "-mno-cygwin" flag. Traditional compile/link chain haven't got any problems with this flag (yet) and simply ignores it, while LTO-enabled linker chokes on it and fails hard.
All times are UTC
Page 1 of 1
Powered by phpBB © 2000-2025 phpBB Limited

Copyright © Owen Rudge/The Transport Tycoon Forums 2001-2025.