Engine Profiling - Revision history

Hox: /* Engine Profiling */

2017-05-17T22:29:30Z

Engine Profiling

@@ Line 18: / Line 18: @@
 The downside of statistical profiling, is that it requires at least a certain sample size to achieve good accuracy. Also remember, that ''oprof'' still requires the debug symbols, to resolve addresses into function names.
 It can be turned on/off at any time, so for instance, you may skip spring loading so you would only profile in-game runtime. You can also merge multiple profiling sessions.
 === Setting up ''oprof'' ===
@@ Line 168: / Line 173: @@
 ./profiling_engine_and_ais.svg
 </pre></code>
 == Windows ==

Flozi: recat

2013-03-20T19:52:03Z

recat

← Older revision		Revision as of 20:52, 20 March 2013
Line 175:		Line 175:


	[[Category: ~~Development~~]]		[[Category:Engine Dev]]

Abma: /* Using oprof */

2012-05-14T22:57:09Z

Using oprof

← Older revision		Revision as of 23:57, 14 May 2012
Line 53:		Line 53:

	# In case the kernel module is not already loaded ...		# In case the kernel module is not already loaded ...
	~~insmod~~ oprofile		modprobe -v oprofile

	# This has to be sent only once; it will be saved		# This has to be sent only once; it will be saved
Line 75:		Line 75:
	opcontrol --dump		opcontrol --dump
	</pre></code>		</pre></code>


	=== Post processing ''oprof'' data ===		=== Post processing ''oprof'' data ===

Cheesecan at 18:41, 29 April 2011

2011-04-29T18:41:08Z

← Older revision		Revision as of 19:41, 29 April 2011
Line 103:		Line 103:
	'''Post Processing Script'''		'''Post Processing Script'''
	<code><pre>		<code><pre>
	#!/bin/sh		#!/bin/bash

	SPRING_INSTALL_PATH=$(dirname $(which spring))		SPRING_INSTALL_PATH=$(dirname $(which spring))

Hoijui: /* Post processing oprof data */ add outcome sample image

2010-12-09T20:35:36Z

Post processing oprof data: add outcome sample image

← Older revision		Revision as of 21:35, 9 December 2010
Line 78:		Line 78:

	=== Post processing ''oprof'' data ===		=== Post processing ''oprof'' data ===

			[[File:Oprof_engine_part.png\|710px]]

	We will first explain the post processing steps, and then give a script that should work for most general spring profiling tasks.		We will first explain the post processing steps, and then give a script that should work for most general spring profiling tasks.

Hoijui: /* Post processing oprof data */ rework post-processing, add more compleete script

2010-12-09T19:57:19Z

Post processing oprof data: rework post-processing, add more compleete script

@@ Line 79: / Line 79: @@
 === Post processing ''oprof'' data ===
 With the settings we used above, ''oprofile'' indexes stuff by thread- and process-ID. In order to generate a report, we will have to tell which program, which threads and which sampling sessions to include.
-If you do not care much, you can just merge thread data and sampling data.
+If you do not care much, you can just merge thread data and sampling data, as we do here too.
-'''Extract the data'''
+[https://code.google.com/p/jrfonseca/wiki/Gprof2Dot gprof2dot] takes ''oprof'' data and outputs a nice dot-graph. We use it to generate the dot graph out of ''oprofile''s <code>--callgraph</code> version.
-<code>--callgraph</code> tells it to plot a full call-graph table, which can be parsed by other tools.
+'''Finalizing (Creating an image out of the graph file)'''
-[https://code.google.com/p/jrfonseca/wiki/Gprof2Dot gprof2dot] takes ''oprof'' data and outputs a nice dot-graph.
+'''Post Processing Script'''
 <code><pre>
-gprof2dot --format=oprofile --output=${BASE_OUT_FILE_PATH}.dot ${BASE_OUT_FILE_PATH}.oprof
+#!/bin/sh
-</pre></code>
-Then you can either use [https://code.google.com/p/jrfonseca/wiki/XDot XDot] to view it directly, or create an image out of the graph file.
+	if [ ${SETTINGS_CREATE_CALLGRAPH} == 1 ]; then
-'''Createing an image out of the graph file'''
+		# create images
-We will use [http://www.graphviz.org/ GraphViz]'s ''dot'' utility to render to an image.
+		if [ ${SETTINGS_CLEANUP} == 1 ]; then
 <code><pre>
-# for PNG
+./profiling_ais.png
-dot -Tpng ${BASE_OUT_FILE_PATH}.dot > ${BASE_OUT_FILE_PATH}.png
+./profiling_ais.svg
-# ... or for SVG
+./profiling_engine_and_ais.png
-dot -Tsvg ${BASE_OUT_FILE_PATH}.dot > ${BASE_OUT_FILE_PATH}.svg
+./profiling_engine_and_ais.svg
 </pre></code>
 == Windows ==

Hoijui: /* Post processing oprof data */ fixes and simplifications

2010-12-05T12:40:26Z

Post processing oprof data: fixes and simplifications

← Older revision		Revision as of 13:40, 5 December 2010
Line 85:		Line 85:
	'''Extract the data'''		'''Extract the data'''
	<code><pre>		<code><pre>
	~~BIN_PATH~~=~/Projects/spring/installs/cmake/master/linux64/debug/spring		SPRING_INSTALL_PATH=~/Projects/spring/installs/cmake/master/linux64/debug
	~~DATA_FILE_PATH~~=./~~springProfilingData~~.~~oprof~~		BIN_ENGINE=${SPRING_INSTALL_PATH}/spring
	~~# these two will be used later~~		BIN_AI_AAI=${SPRING_INSTALL_PATH}/AI/Skirmish/AAI/0.9/libSkirmishAI.so
	~~GRAPH_FILE_PATH~~=~~./springProfilingGraph.dot~~		MY_BINARIES="${BIN_ENGINE} ${BIN_AI_AAI}"
	~~IMAGE_FILE_PATH~~=./~~springProfilingGraph.png~~		BASE_OUT_FILE_PATH=./springProfilingData

	# Caution: this would take a long time!		# Caution: this would take a long time!
Line 100:		Line 100:
	--demangle=smart \		--demangle=smart \
	--merge=tid,tgid \		--merge=tid,tgid \
	--symbols ${~~BIN_PATH~~} \		--symbols ${MY_BINARIES} \
	--output-file ${~~DATA_FILE_PATH~~}		--output-file ${BASE_OUT_FILE_PATH}.oprof
	</pre></code>		</pre></code>

Line 113:		Line 113:
	To generate the dot graph out of the <code>--callgraph</code> version, use this command:		To generate the dot graph out of the <code>--callgraph</code> version, use this command:
	<code><pre>		<code><pre>
	gprof2dot --format=oprofile -o=${~~GRAPH_FILE_PATH~~} ${~~DATA_FILE_PATH~~}		gprof2dot --format=oprofile --output=${BASE_OUT_FILE_PATH}.dot ${BASE_OUT_FILE_PATH}.oprof
	</pre></code>		</pre></code>

Line 124:		Line 124:

	<code><pre>		<code><pre>
	dot -Tpng ${~~GRAPH_FILE_PATH~~} > ${~~IMAGE_FILE_PATH~~}		# for PNG
			dot -Tpng ${BASE_OUT_FILE_PATH}.dot > ${BASE_OUT_FILE_PATH}.png
			# ... or for SVG
			dot -Tsvg ${BASE_OUT_FILE_PATH}.dot > ${BASE_OUT_FILE_PATH}.svg
			# .. and likewise for other formats
	</pre></code>		</pre></code>

	~~The <code>-Tpng</code> part can be replaced by <code>-Tsvg</code>, <code>-Tgif</code> and more.~~

	The result will look about like this:		The result will look about like this:

Hoijui: /* Post processing oprof data */

2010-11-20T12:29:46Z

Post processing oprof data

← Older revision		Revision as of 13:29, 20 November 2010
Line 84:		Line 84:

	'''Extract the data'''		'''Extract the data'''

	<code><pre>		<code><pre>
	BIN_PATH=~/Projects/spring/installs/cmake/master/linux64/debug/spring		BIN_PATH=~/Projects/spring/installs/cmake/master/linux64/debug/spring
Line 132:		Line 131:
	The result will look about like this:		The result will look about like this:
	http://wiki.jrfonseca.googlecode.com/hg/xdot-profile2.png		http://wiki.jrfonseca.googlecode.com/hg/xdot-profile2.png


	== Windows ==		== Windows ==

Hoijui: cleanup

2010-11-16T10:31:10Z

cleanup

← Older revision		Revision as of 11:31, 16 November 2010
Line 40:		Line 40:
	'''Recommended settings'''		'''Recommended settings'''

	~~Use~~ a call-graph depth of 16, disable kernel profiling ~~and~~ enable user-space profiling, separate each thread ~~and application~~ in its own profile~~. You~~ can always merge them later, using ''opreport''.		* use a call-graph depth of size 16
			* disable kernel profiling & enable user-space profiling
			* separate each application in its own profile
			* separate each thread in its own profile; you can always merge them later, using ''opreport''


	'''A typical profiling session'''		'''A typical profiling session'''

	~~<font color="#00EE00">~~
	<code><pre>		<code><pre>
	# We need to be root		# We need to be root
Line 73:		Line 75:
	opcontrol --dump		opcontrol --dump
	</pre></code>		</pre></code>
	~~</font>~~

	=== Post processing ''oprof'' data ===		=== Post processing ''oprof'' data ===

	''oprofile'' indexes stuff by thread id and process ~~id since the setting i've made you use did tell it to do so :p~~		With the settings we used above, ''oprofile'' indexes stuff by thread- and process-ID. In order to generate a report, we will have to tell which program, which threads and which sampling sessions to include.
	~~so in~~ order to generate a report, ~~you'll~~ have to tell which program, which threads ~~to include,~~ and which sampling sessions		If you do not care much, you can just merge thread data and sampling data.
	if you do not care much, you can just merge thread data and sampling data

			'''Extract the data'''

	~~<font color="#00EE00">~~
	<code><pre>		<code><pre>
	BIN_PATH=~/Projects/spring/installs/cmake/master/linux64/debug/spring		BIN_PATH=~/Projects/spring/installs/cmake/master/linux64/debug/spring
	DATA_FILE_PATH=./springProfilingData.oprof		DATA_FILE_PATH=./springProfilingData.oprof
			# these two will be used later
	GRAPH_FILE_PATH=./springProfilingGraph.dot		GRAPH_FILE_PATH=./springProfilingGraph.dot
	IMAGE_FILE_PATH=./springProfilingGraph.png		IMAGE_FILE_PATH=./springProfilingGraph.png
Line 100:		Line 104:
	--output-file ${DATA_FILE_PATH}		--output-file ${DATA_FILE_PATH}
	</pre></code>		</pre></code>
	~~</font>~~

	<code>--callgraph</code> tells it to plot a full call-graph table, which can be parsed by other tools		<code>--callgraph</code> tells it to plot a full call-graph table, which can be parsed by other tools.
	if instead you want a flat ~~txt~~ table, omit it		If instead you want a flat CSV table, omit it.


			'''Converting the data file to a dot-graph file'''

	[https://code.google.com/p/jrfonseca/wiki/Gprof2Dot gprof2dot] takes ''oprof'' data and outputs a nice dot-graph.		[https://code.google.com/p/jrfonseca/wiki/Gprof2Dot gprof2dot] takes ''oprof'' data and outputs a nice dot-graph.
	To generate the dot graph out of the <code>--callgraph</code> version:		To generate the dot graph out of the <code>--callgraph</code> version, use this command:
	~~<font color="#00EE00">~~
	<code><pre>		<code><pre>
	gprof2dot --format=oprofile -o=${GRAPH_FILE_PATH} ${DATA_FILE_PATH}		gprof2dot --format=oprofile -o=${GRAPH_FILE_PATH} ${DATA_FILE_PATH}
	</pre></code>		</pre></code>
	~~</font>~~

	Then you can either use ~~xdot~~ to view it, or ~~use dot to generate a png or svg pic~~		Then you can either use [https://code.google.com/p/jrfonseca/wiki/XDot XDot] to view it directly, or create an image out of the graph file.

	~~Then you can~~ use ~~either~~ graphviz's ''dot'' utility to render to an image~~, or display the graph directly, for example with [https://code.google.com/p/jrfonseca/wiki/XDot xdot]~~.
			'''Createing an image out of the graph file'''

			We will use [http://www.graphviz.org/ GraphViz]'s ''dot'' utility to render to an image.

	<code><pre>		<code><pre>
	dot -Tpng ${GRAPH_FILE_PATH} > ${IMAGE_FILE_PATH}		dot -Tpng ${GRAPH_FILE_PATH} > ${IMAGE_FILE_PATH}
	</pre></code>		</pre></code>

			The <code>-Tpng</code> part can be replaced by <code>-Tsvg</code>, <code>-Tgif</code> and more.

	The result will look about like this:		The result will look about like this:
	http://wiki.jrfonseca.googlecode.com/hg/xdot-profile2.png		http://wiki.jrfonseca.googlecode.com/hg/xdot-profile2.png

Hoijui at 01:25, 16 November 2010

2010-11-16T01:25:03Z

← Older revision		Revision as of 02:25, 16 November 2010
Line 105:		Line 105:
	if instead you want a flat txt table, omit it		if instead you want a flat txt table, omit it

			[https://code.google.com/p/jrfonseca/wiki/Gprof2Dot gprof2dot] takes ''oprof'' data and outputs a nice dot-graph.
	To generate the dot graph out of the <code>--callgraph</code> version:		To generate the dot graph out of the <code>--callgraph</code> version:
	<font color="#00EE00">		<font color="#00EE00">
Line 114:		Line 115:
	Then you can either use xdot to view it, or use dot to generate a png or svg pic		Then you can either use xdot to view it, or use dot to generate a png or svg pic

			Then you can use either graphviz's ''dot'' utility to render to an image, or display the graph directly, for example with [https://code.google.com/p/jrfonseca/wiki/XDot xdot].
	~~[https://code.google.com/p/jrfonseca/wiki/Gprof2Dot gprof2dot] takes ''oprof'' data and outputs a nice dot-graph.~~
	Then you can use either ''~~graphviz~~'' to render to an image, or display the graph directly, for example with [https://code.google.com/p/jrfonseca/wiki/XDot xdot].
	~~The will look about like this:~~
	~~http://wiki.jrfonseca.googlecode.com/hg/xdot-profile2.png~~

	<code><pre>		<code><pre>
	dot -Tpng ${GRAPH_FILE_PATH} > ${IMAGE_FILE_PATH}		dot -Tpng ${GRAPH_FILE_PATH} > ${IMAGE_FILE_PATH}
	</pre></code>		</pre></code>
			The result will look about like this:
			http://wiki.jrfonseca.googlecode.com/hg/xdot-profile2.png