Buildbot:Gentoo: Difference between revisions

From Spring
Jump to navigationJump to search
← Older edit
Abma (talk | contribs)
0 edits
Abma (talk | contribs)
0 edits
 
(129 intermediate revisions by the same user not shown)
Line 1: Line 1:
This page is WIP! (don't expect these instructions to work, they only give a hint, how it should/could work...)
This page is WIP! (don't expect these instructions to work, they only give a hint, how it should/could work...)


==Setup gentoo win32 buildslave==
Its best to run a 64 bit gentoo, as x32 can be easily compiled, too.
 
=buildbot-slave (for interacting with spring's buildbot)=
 
on springrts.com in /home/buildbot/master/passwords.py a user and password must be added. this username is needed to connect the buildslave client to the master.
 
install buildbot on the slave:
<code><pre>
emerge dev-util/buildbot-slave
</pre></code>
 
add a local user which is used to build:
<code><pre>
useradd -s /bin/false -d /home/buildslave -m buildslave
</pre></code>
 
create the buildslave config
<code><pre>
buildslave create-slave /home/buildslave springrts.com:9989 <buildbot user in passwords.py> <password in passwords.py>
</pre></code>
 
make it autostart:
<code><pre>
rc-update add buildslave default
</pre></code>
adjust config
<code><pre>
nano /etc/conf.d/buildslave
</pre></code>
 
 
 
warning! you should use an ssh-tunnel to connect to the master as the password is sent in cleartext!
 
=Setup gentoo win32 buildslave=
Note: boost and spring has to be compiled with the same compiler! If you get weird linker errors then this could be the cause!


tools required for compiling
tools required for compiling
Line 9: Line 44:
echo "source /var/lib/layman/make.conf" >> /etc/portage/make.conf
echo "source /var/lib/layman/make.conf" >> /etc/portage/make.conf
CHOST=i686-mingw32 ROOT=/usr/i686-mingw32 emerge pthreads-win32
CHOST=i686-mingw32 ROOT=/usr/i686-mingw32 emerge pthreads-win32
USE="openmp cxx -fortran" crossdev i686-mingw32 --env 'EXTRA_ECONF="--disable-sjlj-exceptions --with-dwarf"'
USE="openmp cxx -fortran" crossdev i686-mingw32 --env 'EXTRA_ECONF="--disable-sjlj-exceptions --with-dwarf --enable-gomp --disable-shared"'
</pre></code>
</pre></code>


for cgcreate (only needed for load balancing while building)
for java ai's / interface install a jdk
<code><pre>
<code><pre>
emerge dev-libs/libcgroup
emerge -va oracle-jdk-bin
</pre></code>
</pre></code>


Line 21: Line 56:
USE="openmp cxx -fortran" crossdev i686-mingw32 --env 'EXTRA_ECONF="--disable-sjlj-exceptions --with-dwarf"'
USE="openmp cxx -fortran" crossdev i686-mingw32 --env 'EXTRA_ECONF="--disable-sjlj-exceptions --with-dwarf"'
CHOST=i686-mingw32 ROOT=/usr/i686-mingw32 emerge -va pthreads-win32
CHOST=i686-mingw32 ROOT=/usr/i686-mingw32 emerge -va pthreads-win32
USE="openmp cxx -fortran" crossdev i686-mingw32 --env 'EXTRA_ECONF="--disable-sjlj-exceptions --with-dwarf"'
USE="openmp cxx -fortran" crossdev i686-mingw32 --env 'EXTRA_ECONF="--disable-sjlj-exceptions --with-dwarf --disable-shared"'
emerge-wrapper --init
emerge-wrapper --init
</pre></code>
</pre></code>


==buildbot-slave (for interacting with spring's buildbot)==
<code><pre>
emerge dev-util/buildbot-slave
rc-update add buildslave default
nano /etc/conf.d/buildslave
useradd -s /bin/false -d $dir -m $username
buildslave create-slave $dir/buildslave springrts.com:7778 $username $passwd
</pre></code>
== stuff that needs to be compiled native (not with mingw)==
== stuff that needs to be compiled native (not with mingw)==
<code><pre>
<code><pre>
Line 41: Line 65:
ACCEPT_KEYWORDS="~x86" emerge boost-build
ACCEPT_KEYWORDS="~x86" emerge boost-build
</pre></code>
</pre></code>
 
== build Spring + dependencies in MinGW==
= mingw64 32 bit =
 
<code><pre>USE="openmp cxx -fortran -multilib static-libs" crossdev --env 'EXTRA_ECONF="--disable-shared"' -t i686-w64-mingw32
</pre></code>
 
TODO: see http://sourceforge.net/apps/trac/mingw-w64/wiki/Cross%20Win32%20and%20Win64%20compiler
 
=Setup static linux=
 
create /etc/portage/package.use/spring-static-buildslave:
 
<code><pre>
<code><pre>
USE="minizip" emerge-i686-mingw32 sys-libs/zlib
sys-libs/zlib static-libs minizip
media-libs/devil static-libs opengl png jpeg tiff gif X
media-libs/giflib static-libs
media-libs/tiff static-libs
dev-libs/boost static-libs python
media-libs/jpeg  static-libs
media-libs/libsdl2 -audio opengl static-libs X
media-libs/libvorbis static-libs
media-libs/libogg static-libs
media-libs/freetype static-libs autohinter
media-libs/glew static-libs
media-libs/libpng static-libs
x11-libs/libXcursor static-libs
app-arch/bzip2 static-libs
x11-libs/libXdmcp static-libs
x11-libs/libXrender static-libs
x11-libs/libX11 static-libs
x11-libs/libXfixes static-libs
x11-libs/libxcb static-libs
x11-libs/libXau static-libs
net-misc/curl static-libs
dev-libs/openssl static-libs
sys-libs/libunwind static-libs
</pre></code>
</pre></code>


#USE="static"
==set compile flags on 64 bit gentoo==
this is only required on 64bit gentoo!
 
if you don't add -fPIC you mostly will get errors like his:
<code><pre>
<code><pre>
emerge-i686-mingw32 app-arch/bzip2
relocation R_X86_64_32 against `a local symbol' can not be used when making a
USE="jpeg png opengl" emerge-i686-mingw32 devil
shared object; recompile with -fPIC
USE="auto-hinter" emerge-i686-mingw32 freetype
/usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1/../../../../lib64/libfl.a: could not
USE="-audio opengl" emerge-i686-mingw32 libsdl -O
read symbols: Bad value
EXTRA_ECONF="target-os=windows threadapi=win32" USE="" emerge-i686-mingw32 boost -O
USE="-ai -java openmp" emerge-i686-mingw32 spring
</pre></code>
</pre></code>




=Setup static linux=
/etc/portage/env/spring-static-buildslave:
<code><pre>
CFLAGS="${CFLAGS} -fPIC"
CXXFLAGS="${CXXFLAGS} -fPIC"
LDFLAGS="${LDFLAGS} -fPIC"
</pre></code>
 
/etc/portage/package.env/spring-static-buildslave:
<code><pre>
sys-libs/zlib spring-static-buildslave
media-libs/devil spring-static-buildslave
media-libs/giflib spring-static-buildslave
media-libs/tiff spring-static-buildslave
dev-libs/boost spring-static-buildslave
media-libs/jpeg spring-static-buildslave
media-libs/libsdl2 spring-static-buildslave
media-libs/libvorbis spring-static-buildslave
media-libs/freetype spring-static-buildslave
media-libs/glew spring-static-buildslave
sys-devel/gcc spring-static-buildslave
media-libs/libpng spring-static-buildslave
x11-libs/libXcursor spring-static-buildslave
</pre></code>


==install needed programs and libs==
Emerge gcc-4.7.2
Emerge gcc-4.7.2
<code><pre>
<code><pre>
Line 65: Line 145:
</pre></code>
</pre></code>


enable / use it
<code><pre>
gcc-config x86_64-pc-linux-gnu-4.7.2
env-update && source /etc/profile
</pre></code>
re-install required libs with changed use flags:
<code><pre>
emerge -va sys-libs/zlib media-libs/devil dev-libs/boost media-libs/jpeg media-libs/libsdl2 \
media-libs/libvorbis media-libs/freetype media-libs/glew media-libs/libpng x11-libs/libXcursor \
app-arch/bzip2 x11-libs/libXdmcp x11-libs/libXrender x11-libs/libX11 x11-libs/libXfixes \
x11-libs/libxcb x11-libs/libXau net-misc/curl sys-libs/libunwind
</pre></code>
==tune==
use ld.gold for linking:
<code><pre>
binutils-config --linker ld.gold
</pre></code>


=32 bit buildbot on a 64 bit slave=


switch back to the standard linker as glibc doesn't compile with ld.gold if you get this error:
"These critical programs are missing or too old: as ld"
<code><pre>
<code><pre>
gcc-config x86_64-pc-linux-gnu-4.7.2
binutils-config --libc 2.11.3 --linker ld.bfd
env-update && source /etc/profile
</pre></code>
</pre></code>


<code><pre>
<code><pre>
USE="static-libs png opengl minizip" emerge -va boost jpeg devil libsdl media-libs/openal media-libs/libvorbis freetype zlib glew
USE="openmp cxx -fortran" crossdev --target i686 --gcc 4.8.5 --libc 2.17
mkdir -p /usr/i686-pc-linux-gnu/etc/portage/package.use/
vi /usr/i686-pc-linux-gnu/etc/portage/package.use/spring-static-buildslave
 
sys-libs/zlib static-libs minizip
media-libs/devil static-libs opengl png jpeg tiff gif X
media-libs/giflib static-libs
media-libs/tiff static-libs -cxx
dev-libs/boost static-libs -python
media-libs/jpeg static-libs
media-libs/libsdl2 -audio opengl static-libs X
media-libs/libvorbis static-libs
media-libs/libogg static-libs
media-libs/freetype static-libs autohinter
media-libs/glew static-libs
media-libs/libpng static-libs
x11-libs/libXcursor static-libs
app-arch/bzip2 static-libs
x11-libs/libXdmcp static-libs
x11-libs/libXrender static-libs
x11-libs/libX11 static-libs
x11-libs/libXfixes static-libs
x11-libs/libxcb static-libs
x11-libs/libXau static-libs
dev-libs/gmp static-libs
dev-libs/mpfr static-libs
dev-libs/mpc static-libs
net-misc/curl static-libs
dev-libs/openssl static-libs
dev-java/oracle-jdk-bin -X -fontconfig
sys-libs/libunwind static-libs
</pre></code>
</pre></code>


=TODO: whats that for?!=
<code><pre>
<code><pre>
EXTRA_ECONF="--disable-sjlj-exceptions --with-dwarf2"
(
USE="openmp cxx -fortran" crossdev i686-mingw32 --env
  echo PYTHON_TARGETS="python2_7"
USE="minizip"
  echo USE="${USE} -pam -llvm -tools -uuid -kmod -classic -dri3 -egl -gallium -gbm -nptl -udev -fortran"
)>> /usr/i686-pc-linux-gnu/etc/portage/make.conf
</pre></code>


USE="jpeg png opengl" emerge-i686-mingw32 devil
force old version of glibc for more compatibility of the generated executable:
USE="auto-hinter" emerge-i686-mingw32 freetype
<code><pre>
USE="-audio opengl" emerge-i686-mingw32 libsdl -O
mkdir -p /usr/i686-pc-linux-gnu/etc/portage/package.mask
EXTRA_ECONF="target-os=windows threadapi=win32" USE="" emerge-i686-mingw32 boost -O
echo '>sys-libs/glibc-2.17' > /usr/i686-pc-linux-gnu/etc/portage/package.mask/glibc
USE="-ai -java openmp" emerge-i686-mingw32 spring
emerge-i686-pc-linux-gnu -va --nodeps dev-libs/libffi =dev-lang/python-2.7.11-r2
i686-pc-linux-gnu-emerge -va dev-util/pkgconfig x11-misc/util-macros
</pre></code>
</pre></code>


build all libs: (we use --nodeps as else it would pull in A LOT of dependencies which aren't required. because of --nodeps, the order of parameters matters!)
<code><pre>
emerge-i686-pc-linux-gnu -va --nodeps \
sys-libs/zlib \
app-arch/bzip2 \
dev-libs/boost \
media-libs/tiff \
media-libs/jpeg \
media-libs/libvorbis \
media-libs/freetype \
media-libs/libpng \
media-libs/openal \
dev-libs/openssl \
net-misc/curl \
=sys-libs/libunwind-0.99-r1 \
x11-libs/libXdmcp \
x11-libs/libXau \
=x11-libs/libxcb-1.10 \
x11-libs/libX11 \
x11-libs/libXrender \
x11-proto/fixesproto \
x11-libs/libXfixes \
x11-libs/libXcursor \
x11-libs/libdrm \
x11-proto/glproto \
x11-libs/libXext  \
x11-libs/libXdamage \
x11-proto/dri2proto \
media-libs/mesa \
media-libs/glu \
media-libs/glew \
media-libs/devil \
media-libs/libsdl2
</pre></code>


=patches=
gcc omp patch: http://gcc.gnu.org/bugzilla/attachment.cgi?id=28752


[[Category:Buildbot]]
[[Category:Buildbot]]

Latest revision as of 22:43, 17 October 2016

This page is WIP! (don't expect these instructions to work, they only give a hint, how it should/could work...)

Its best to run a 64 bit gentoo, as x32 can be easily compiled, too.

buildbot-slave (for interacting with spring's buildbot)

on springrts.com in /home/buildbot/master/passwords.py a user and password must be added. this username is needed to connect the buildslave client to the master.

install buildbot on the slave: 

emerge dev-util/buildbot-slave

add a local user which is used to build: 

useradd -s /bin/false -d /home/buildslave -m buildslave

create the buildslave config 

buildslave create-slave /home/buildslave springrts.com:9989 <buildbot user in passwords.py> <password in passwords.py>

make it autostart: 

rc-update add buildslave default

adjust config 

nano /etc/conf.d/buildslave


warning! you should use an ssh-tunnel to connect to the master as the password is sent in cleartext!

Setup gentoo win32 buildslave

Note: boost and spring has to be compiled with the same compiler! If you get weird linker errors then this could be the cause!

tools required for compiling 

emerge layman mercurial app-arch/p7zip-9.13 dev-util/cmake app-arch/zip wine crossdev app-portage/portage-utils
layman -a ohnobinki
echo "source /var/lib/layman/make.conf" >> /etc/portage/make.conf
CHOST=i686-mingw32 ROOT=/usr/i686-mingw32 emerge pthreads-win32
USE="openmp cxx -fortran" crossdev i686-mingw32 --env 'EXTRA_ECONF="--disable-sjlj-exceptions --with-dwarf --enable-gomp --disable-shared"'

for java ai's / interface install a jdk 

emerge -va oracle-jdk-bin

Install i686-mingw32 compiler, then pthreads, then recompile i686-mingw32 with pthreads enabled 

USE="openmp cxx -fortran" crossdev i686-mingw32 --env 'EXTRA_ECONF="--disable-sjlj-exceptions --with-dwarf"'
CHOST=i686-mingw32 ROOT=/usr/i686-mingw32 emerge -va pthreads-win32
USE="openmp cxx -fortran" crossdev i686-mingw32 --env 'EXTRA_ECONF="--disable-sjlj-exceptions --with-dwarf --disable-shared"'
emerge-wrapper --init

stuff that needs to be compiled native (not with mingw)

emerge ncurses cmake eselect eselect-boost
ACCEPT_KEYWORDS="~x86" emerge boost-build

mingw64 32 bit

USE="openmp cxx -fortran -multilib static-libs" crossdev --env 'EXTRA_ECONF="--disable-shared"' -t i686-w64-mingw32

TODO: see http://sourceforge.net/apps/trac/mingw-w64/wiki/Cross%20Win32%20and%20Win64%20compiler

Setup static linux

create /etc/portage/package.use/spring-static-buildslave: 

sys-libs/zlib static-libs minizip
media-libs/devil static-libs opengl png jpeg tiff gif X
media-libs/giflib static-libs
media-libs/tiff static-libs
dev-libs/boost static-libs python
media-libs/jpeg  static-libs
media-libs/libsdl2 -audio opengl static-libs X
media-libs/libvorbis static-libs
media-libs/libogg static-libs
media-libs/freetype static-libs autohinter
media-libs/glew static-libs
media-libs/libpng static-libs
x11-libs/libXcursor static-libs
app-arch/bzip2 static-libs
x11-libs/libXdmcp static-libs
x11-libs/libXrender static-libs
x11-libs/libX11 static-libs
x11-libs/libXfixes static-libs
x11-libs/libxcb static-libs
x11-libs/libXau static-libs
net-misc/curl static-libs
dev-libs/openssl static-libs
sys-libs/libunwind static-libs

set compile flags on 64 bit gentoo

this is only required on 64bit gentoo!

if you don't add -fPIC you mostly will get errors like his: 

relocation R_X86_64_32 against `a local symbol' can not be used when making a
shared object; recompile with -fPIC
/usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1/../../../../lib64/libfl.a: could not
read symbols: Bad value


/etc/portage/env/spring-static-buildslave: 

CFLAGS="${CFLAGS} -fPIC"
CXXFLAGS="${CXXFLAGS} -fPIC"
LDFLAGS="${LDFLAGS} -fPIC"

/etc/portage/package.env/spring-static-buildslave: 

sys-libs/zlib spring-static-buildslave
media-libs/devil spring-static-buildslave
media-libs/giflib spring-static-buildslave
media-libs/tiff spring-static-buildslave
dev-libs/boost spring-static-buildslave
media-libs/jpeg spring-static-buildslave
media-libs/libsdl2 spring-static-buildslave
media-libs/libvorbis spring-static-buildslave
media-libs/freetype spring-static-buildslave
media-libs/glew spring-static-buildslave
sys-devel/gcc spring-static-buildslave
media-libs/libpng spring-static-buildslave
x11-libs/libXcursor spring-static-buildslave

install needed programs and libs

Emerge gcc-4.7.2 

emerge -va =sys-devel/gcc-4.7.2

enable / use it 

gcc-config x86_64-pc-linux-gnu-4.7.2
env-update && source /etc/profile

re-install required libs with changed use flags: 

emerge -va sys-libs/zlib media-libs/devil dev-libs/boost media-libs/jpeg media-libs/libsdl2 \
media-libs/libvorbis media-libs/freetype media-libs/glew media-libs/libpng x11-libs/libXcursor \
app-arch/bzip2 x11-libs/libXdmcp x11-libs/libXrender x11-libs/libX11 x11-libs/libXfixes \
x11-libs/libxcb x11-libs/libXau net-misc/curl sys-libs/libunwind

tune

use ld.gold for linking: 

binutils-config --linker ld.gold

32 bit buildbot on a 64 bit slave

switch back to the standard linker as glibc doesn't compile with ld.gold if you get this error: "These critical programs are missing or too old: as ld" 

binutils-config --libc 2.11.3 --linker ld.bfd

USE="openmp cxx -fortran" crossdev --target i686 --gcc 4.8.5 --libc 2.17
mkdir -p /usr/i686-pc-linux-gnu/etc/portage/package.use/
vi /usr/i686-pc-linux-gnu/etc/portage/package.use/spring-static-buildslave

sys-libs/zlib static-libs minizip
media-libs/devil static-libs opengl png jpeg tiff gif X
media-libs/giflib static-libs
media-libs/tiff static-libs -cxx
dev-libs/boost static-libs -python
media-libs/jpeg  static-libs
media-libs/libsdl2 -audio opengl static-libs X
media-libs/libvorbis static-libs
media-libs/libogg static-libs
media-libs/freetype static-libs autohinter
media-libs/glew static-libs
media-libs/libpng static-libs
x11-libs/libXcursor static-libs
app-arch/bzip2 static-libs
x11-libs/libXdmcp static-libs
x11-libs/libXrender static-libs
x11-libs/libX11 static-libs
x11-libs/libXfixes static-libs
x11-libs/libxcb static-libs
x11-libs/libXau static-libs
dev-libs/gmp static-libs
dev-libs/mpfr static-libs
dev-libs/mpc static-libs
net-misc/curl static-libs
dev-libs/openssl static-libs
dev-java/oracle-jdk-bin -X -fontconfig
sys-libs/libunwind static-libs

(
  echo PYTHON_TARGETS="python2_7"
  echo USE="${USE} -pam -llvm -tools -uuid -kmod -classic -dri3 -egl -gallium -gbm -nptl -udev -fortran"
)>> /usr/i686-pc-linux-gnu/etc/portage/make.conf

force old version of glibc for more compatibility of the generated executable: 

mkdir -p /usr/i686-pc-linux-gnu/etc/portage/package.mask
echo '>sys-libs/glibc-2.17' > /usr/i686-pc-linux-gnu/etc/portage/package.mask/glibc
emerge-i686-pc-linux-gnu -va --nodeps dev-libs/libffi =dev-lang/python-2.7.11-r2
i686-pc-linux-gnu-emerge -va dev-util/pkgconfig x11-misc/util-macros

build all libs: (we use --nodeps as else it would pull in A LOT of dependencies which aren't required. because of --nodeps, the order of parameters matters!) 

emerge-i686-pc-linux-gnu -va --nodeps \
sys-libs/zlib \
app-arch/bzip2 \
dev-libs/boost \
media-libs/tiff \
media-libs/jpeg \
media-libs/libvorbis \
media-libs/freetype \
media-libs/libpng \
media-libs/openal \
dev-libs/openssl \
net-misc/curl \
=sys-libs/libunwind-0.99-r1 \
x11-libs/libXdmcp \ 
x11-libs/libXau \
=x11-libs/libxcb-1.10 \
x11-libs/libX11 \
x11-libs/libXrender \
x11-proto/fixesproto \
x11-libs/libXfixes \
x11-libs/libXcursor \
x11-libs/libdrm \
x11-proto/glproto \
x11-libs/libXext  \
x11-libs/libXdamage \
x11-proto/dri2proto \
media-libs/mesa \
media-libs/glu \
media-libs/glew \
media-libs/devil \
media-libs/libsdl2

patches

gcc omp patch: http://gcc.gnu.org/bugzilla/attachment.cgi?id=28752

Retrieved from "https://springrts-wiki.p2004a.com/mediawiki/index.php?title=Buildbot:Gentoo&oldid=3197"