Spring - User contributions [en]

SetupGuide

2017-09-29T20:37:44Z

Hox: /* Linux/Unix Setup Guide */

=Linux/Unix Setup Guide=

{{#css:
table.glossy {
margin: 4px; padding:10px;
border: 1px solid rgba(0,0,0,0.5);
border-radius: 10px 10px 10px 10px;
box-shadow: 0 2px 6px rgba(0,0,0,0.5), inset 0 1px rgba(255,255,255,0.3), inset 0 10px rgba(255,255,255,0.2), inset 0 10px 20px rgba(255,255,255,0.25), inset 0 -15px 30px rgba(0,0,0,0.3);
text-align: center;
}
/*table.glossy:hover {
background: rgba(160,160,220,0.4);
}*/
}}

{| style="width:70%; margin-left:15%; margin-right:15%;" |

|{{Template:LinuxSetupGuideButton|name=From Source|link=Build_from_source|icon=Git.png}}

|{{Template:LinuxSetupGuideButton|name=Ubuntu|link=Ubuntu_install|icon=Distributions-ubuntu.png}}

|{{Template:LinuxSetupGuideButton|name=Gentoo|link=Gentoo_install|icon=Distributions-gentoo.png}}

|-

|{{Template:LinuxSetupGuideButton|name=Slackware|link=Slackware12_install|icon=Distributions-slackware.png}}

|{{Template:LinuxSetupGuideButton|name=Debian|link=Debian_install|icon=Distributions-debian.png}}

|{{Template:LinuxSetupGuideButton|name=Mandriva|link=Mandriva_install|icon=Distributions-mandriva_linux.png}}

|-

|{{Template:LinuxSetupGuideButton|name=openSUSE|link=openSUSE_install|icon=Distributions-opensuse_linux.png}}

|{{Template:LinuxSetupGuideButton|name=Fedora|extlink=http://projects.springlobby.info/projects/springlobby/wiki/Install#Fedora-10|icon=Distributions-fedora.png}}

|{{Template:LinuxSetupGuideButton|name=FreeBSD|link=FreeBSD_install|icon=Distributions-freebsd.png}}
|}

[[Linux:Packaging | Package Maintainers]]

[[Category:Linux]]

Lua Widgets

2017-05-21T17:49:53Z

Hox: /* Installing Widgets */

===General===
'''[[Lua:WidgetDirectory|LuaUI Widget Directory]]''' - A list of LuaUI widgets 
'''[[Lua_Scripting|Lua Scripting and Development]]''' - Look here to get started with making your own Lua widgets and gadgets 
'''[http://springrts.com/phpbb/viewforum.php?f=23 Spring Lua Forum]''' - Browse and discuss published widgets, or publish your own 

===Introduction: What's a "Widget"?===
LuaUI widgets are files containing Lua code that you can run while playing a Spring game in order to assist you with unit management, provide you with valuable information, or otherwise supplement the basic Spring user interface. They are easy to install, easy to run and manage ingame, and relatively easy to write should you choose to learn how to do so.
|}

===Getting Widgets===
Widget code and files are quite often released as posts in the '''[http://springrts.com/phpbb/viewforum.php?f=23&sid=83343bd3a47bc826fc973e4bd6a15f04 Spring Lua Forum]'''.

It is also highly recommended that you visit the wiki's '''[[Lua:WidgetDirectory|LuaUI Widget Directory]]''', which contains a list of many widgets, along with mod compatibility information and possible additional installation instructions.

===Installing Widgets===
Widget files have the extension .lua, but only contain plain text. To install widgets you've downloaded, just move the .lua file to the LuaUI\Widgets folder in your Spring installation directory. '''Mods like BA ver>9.47 or Evolution RTS explicitly disable widgets from this path.''' Please refer to the mod's manual for details. Sometimes, a widget file may come in a zip archive, or be bundled with other dependent files. These files will need to be unarchived, then moved to their proper respective folders in the LuaUI folder. Look for a help or readme file included in the package, or read any installation instructions listed with the widget at the '''[[Lua:WidgetDirectory|LuaUI Widget Directory]]'''.

===Using Widgets===
'''NOTE:''' Before you can use widgets, you MUST make sure that you have checked the "Enable LuaUI widgets" box in [[SpringSettings]].

To list/manage all available widgets, press F11 while ingame. Each widget is listed by name and sorted alphabetically, and mod included widgets appear first on the list with their names surrounded by asterisks. You may click on each widget name to enable/disable that widget. '''Green''' widgets are currently running, '''red''' widgets are currently disabled and '''orange''' widgets have malfunctioned or turned themselves off.

To rearrange the Lua widgets, press Ctrl-F11 while ingame. To escape Lua tweaking mode, just hit Esc.

In the case that you begin to see Lua errors and/or your user interface inexplicably crashes or becomes "messed up", you can use the command '''/luaui reload''' in order to restart the Lua user interface. If for any reason you wish to shutdown the Lua user interface while ingame, you may issue the command '''/luaui disable'''.
[[Category: Lua]]

Engine Profiling

2017-05-17T22:29:30Z

Hox: /* Engine Profiling */

[[Engine_Development|Development]] < Engine Profiling

= Engine Profiling =

== Linux ==

=== Intro ===
There are several ways to profile an application.

'''gprof'''

The compiler injects special code in the binary that self profiles it when it runs. this is what <code>cmake -DCMAKE_BUILD_TYPE=PROFILING</code> is here for. We are not going to use this. It does not interfere with normal program's work-flow. With spring, it gets so slow that it is unusable for profiling anything in late game (which is where it is needed the most). It requires special compiling options.

'''oprof'''

OProfile uses statistical sampling. It is a kernel module, which every n clock ticks, registers which function is currently being processed. It is lightweight; it may run with almost 0 performance impact. This method always profiles your profiles your whole system; relevant info has to be filtered out after the profiling run has finished.
The downside of statistical profiling, is that it requires at least a certain sample size to achieve good accuracy. Also remember, that ''oprof'' still requires the debug symbols, to resolve addresses into function names.
It can be turned on/off at any time, so for instance, you may skip spring loading so you would only profile in-game runtime. You can also merge multiple profiling sessions.

'''perf'''

Powerful and lightweight Linux performance profiler based on perf events. Part of linux kernel from version 2.6.31.

=== Setting up ''oprof'' ===

Required stuff:
* ''oprof'' kernel module
* ''oprof'' package

=== Using ''oprof'' ===

<center>
{|style="margin-top: 15px; padding: 10px; border: 1px solid rgba(0,0,0,0.5); border-radius: 6px 6px 6px 6px; box-shadow: 0 2px 6px rgba(0,0,0,0.3), inset 0 1px 2px rgba(255,255,255,0.5), inset 0 10px 8px rgba(255,255,255,0.3);" |
| Note: Modern CPUs have special statistical sampling timing functions to make profiling with software like ''oprof'' fast.
|}
</center>

You will have to load the kernel module.
The profiling daemon is controlled using ''opcontrol'' as root.
As regular user, you can do post-processing using ''opreport''.

'''Recommended settings'''

* use a call-graph depth of size 16
* disable kernel profiling & enable user-space profiling
* separate each application in its own profile
* separate each thread in its own profile; you can always merge them later, using ''opreport''

'''A typical profiling session'''

<code><pre>
# We need to be root
su -

# In case the kernel module is not already loaded ...
modprobe -v oprofile

# This has to be sent only once; it will be saved
opcontrol --event=CPU_CLK_UNHALTED:1000000:0x0:0:1 \
--callgraph=16 \
--separate=thread,library \
--no-vmlinux

# -- Bring spring into a state where you want to start profiling --

# start the profiler
opcontrol --start

# -- In spring, do whatever you want to profile --

# stop the profiler
opcontrol --stop

# in order to make the profile data available to user-space,
# you have to dump the data
opcontrol --dump
</pre></code>

=== Post processing ''oprof'' data ===

[[File:Oprof_engine_part.png|710px]]

We will first explain the post processing steps, and then give a script that should work for most general spring profiling tasks.

'''Extract the data'''

The data is now stored in ''oprofile''s cache, and we need to extract the parts of it that we are interested in, and write them to a file.
With the settings we used above, ''oprofile'' indexes stuff by thread- and process-ID. In order to generate a report, we will have to tell which program, which threads and which sampling sessions to include.
If you do not care much, you can just merge thread data and sampling data, as we do here too.

'''Converting the data file to a dot-graph file'''

[https://code.google.com/p/jrfonseca/wiki/Gprof2Dot gprof2dot] takes ''oprof'' data and outputs a nice dot-graph. We use it to generate the dot graph out of ''oprofile''s <code>--callgraph</code> version.

'''Finalizing (Creating an image out of the graph file)'''

Then you can either use [https://code.google.com/p/jrfonseca/wiki/XDot XDot] to view it directly, or create an image out of the graph file. We will use [http://www.graphviz.org/ GraphViz]'s ''dot'' utility to render to an image.

'''Post Processing Script'''
<code><pre>
#!/bin/bash

SPRING_INSTALL_PATH=$(dirname $(which spring))
BIN_ENGINE=${SPRING_INSTALL_PATH}/spring
BIN_AIS=$(ls ${SPRING_INSTALL_PATH}/AI/Skirmish/*/*/libSkirmishAI.so)

SETTINGS_CREATE_TABLE=0
SETTINGS_CREATE_CALLGRAPH=1
SETTINGS_CREATE_PNG=1
SETTINGS_CREATE_SVG=1
SETTINGS_CLEANUP=1

function profileBins() {
BASE_OUT_FILE_PATH=$1
MY_BINARIES="$2"

# extract relevant oprof data as table
if [ ${SETTINGS_CREATE_TABLE} == 1 ]; then
opreport \
--long-filenames \
--demangle=smart \
--merge=tid,tgid \
--symbols ${MY_BINARIES} \
--output-file ${BASE_OUT_FILE_PATH}.txt
fi

if [ ${SETTINGS_CREATE_CALLGRAPH} == 1 ]; then
# extract relevant oprof data as callgraph
opreport \
--long-filenames \
--callgraph \
--demangle=smart \
--merge=tid,tgid \
--symbols ${MY_BINARIES} \
--output-file ${BASE_OUT_FILE_PATH}.oprof

# create graph file
gprof2dot --format=oprofile --output=${BASE_OUT_FILE_PATH}.dot ${BASE_OUT_FILE_PATH}.oprof

# create images
if [ ${SETTINGS_CREATE_PNG} == 1 ]; then
dot -Tpng ${BASE_OUT_FILE_PATH}.dot > ${BASE_OUT_FILE_PATH}.png
fi
if [ ${SETTINGS_CREATE_SVG} == 1 ]; then
dot -Tsvg ${BASE_OUT_FILE_PATH}.dot > ${BASE_OUT_FILE_PATH}.svg
fi
# ... likewise for other formats

if [ ${SETTINGS_CLEANUP} == 1 ]; then
rm ${BASE_OUT_FILE_PATH}.oprof
rm ${BASE_OUT_FILE_PATH}.dot
fi
fi
}

#profileBins ./profiling_engine "${BIN_ENGINE}"
profileBins ./profiling_ais "${BIN_AIS}"
profileBins ./profiling_engine_and_ais "${BIN_ENGINE} ${BIN_AIS}"
</pre></code>
This will leave you with these graph files:
<code><pre>
./profiling_ais.png
./profiling_ais.svg
./profiling_engine_and_ais.png
./profiling_engine_and_ais.svg
</pre></code>

=== Using ''perf'' ===

'''Setup'''

For ubuntu install linux-tools-common using apt-get. In fedora the package's name is perf.
Make sure <code>/proc/sys/kernel/kptr_restrict</code> contains value '''0'''

'''Useful commands'''

* perf top - shows statistics in real time
* perf record - save statistics to file for further insight
* perf report - read previously generated data
* perf diff a.data b.data - show differences between two report files

'''Example'''

* Get data of running spring proccess for 10 seconds. Pay attention to frequency parameter -F. At this rate the output takes about 17 mb of space. Parameter call-graph dwarf unwinds the stack even when frame pointer optimization is on.
<code>perf record -F 99 -p PID_NUMBER -o my_output1.data --call-graph dwarf sleep 10</code>
* Read the report. Press / and type symbol to narrow list to e.g "air"
<code>perf report -i my_output1.data -g</code>
<code><pre>
Children Self Command Shared Symbol
+ 132.33% 29.15% unknown spring [.] CStrafeAirMoveType::Update
+ 71.36% 3.35% unknown spring [.] CStrafeAirMoveType::UpdateFlying
+ 52.01% 32.48% unknown spring [.] CStrafeAirMoveType::UpdateAirPhysics
+ 36.42% 0.00% unknown spring [.] CAirCAI::ExecuteFight
34.50% 12.55% unknown spring [.] CommandDrawer::DrawAirCAICommands
+ 32.11% 0.00% unknown spring [.] CStrafeAirMoveType::UpdateLanding
26.65% 12.90% unknown spring [.] CGameHelper::GetClosestEnemyAircraft
16.73% 0.00% unknown spring [.] CAirCAI::GiveCommandReal
15.91% 0.00% unknown spring [.] CStrafeAirMoveType::FindLandingPos
15.91% 6.42% unknown spring [.] CStrafeAirMoveType::BrakingDistance
9.82% 0.00% unknown spring [.] CAirCAI::AirAutoGenerateTarget
9.82% 0.00% unknown spring [.] CAirCAI::SlowUpdate
3.28% 0.00% unknown spring [.] CAirCAI::ExecuteMove
3.26% 0.00% unknown spring [.] CStrafeAirMoveType::HandleCollisions
3.17% 3.17% unknown spring [.] AAirMoveType::UseSmoothMesh
</pre></code>

'''Flamegraph'''

* <code>perf script -i a.data | ./stackcollapse-perf.pl > out.perf-folded</code> - convert perf stacks data to text representation
* <code>./flamegraph.pl out.perf-folded > my_idle_air.svg</code> - create flamegraph from perf data
or
* <code>grep -i air out.perf-folded | ./flamegraph.pl > my_idle_air.svg</code> - same as above, but it ignores stacks that do not contain "air" in symbol names

[[File:flamegraph-grepped-example.png|flamegraph-grepped-example.png]]

'''Links'''

[https://perf.wiki.kernel.org/index.php/Tutorial perf kernel wiki tutorial]

[http://www.brendangregg.com/perf.html perf examples by Brendan Gregg]

[https://eclipse.org/linuxtools/projectPages/perf/ Eclipse perf plugin]

[https://github.com/brendangregg/FlameGraph Flamegraph]

== Windows ==

<Please add>

[[Category:Engine Dev]]

File:Flamegraph-grepped-example.png

2017-05-17T22:24:32Z

Hox: Grepped flamegraph.

Grepped flamegraph.