-
Notifications
You must be signed in to change notification settings - Fork 63
Code Analysis Tools
Valgrind is an analysis tool that does not require recompilation, because it is a CPU emulator. Valgrind pretends to be the kind of CPU you are using, and runs your program on top of its "virtual machine", which lets it keep track of every single instruction executed, and catch a wide variety of errors. A number of sub-tools are provided by Valgrind that use the instruction information to compute different kinds of results. Be prepared for a big slowdown in your program, on the order of 10X. It is expensive to simulate every single instruction.
You can use Valgrind on MPI programs, essentially by having mpirun
execute Valgrind which in turn executes your program:
mpirun -np <N> valgrind <valgrind arguments> ./program <program arguments>
For most of these, except DRD, if your program is multi-threaded,
you can get better performance by giving --fair-sched=yes to Valgrind.
The following option writes one log file per-process:
--log-file=valgrindLog.%p
Memcheck is a tool for detecting misuse of the memory system. Accessing memory that is not properly allocated, using the wrong deallocation functions, and of course detecting memory leaks. It is the default tool, and will be used when valgrind is called without arguments. Some useful arguments are:
valgrind --leak-check=full --show-reachable=yes --track-origins=yes --gen-suppressions=all --suppressions=myfile.supp ./program <program arguments>
Don't use all the arguments above as shown.
leak-check=full will print detailed information about where the
leaked memory was allocated.
show-reachable=yes will then provide output for "reachable" leaks.
track-origins=yes is useful for determining the cause of "uninitialized variable"
errors, it will show the origin of those variables.
gen-suppressions can be used to print suppression information
for errors which you do not have to power to fix and would like to ignore.
After putting these suppressions into a file, it can be given to Valgrind
with suppressions= to have it ignore those kinds of errors.
http://valgrind.org/docs/manual/ms-manual.html
Massif is a tool for measuring memory usage and identifying which parts of the code are allocating memory. Running this tool is fairly straightforward:
rm -f massif.out.*
valgrind --tool=massif ./program <arguments>
ms_print massif.out.* > output.txt
Then take a look at the output.txt file.
The Data Race Detector spots problems in multi-threaded programs
valgrind --tool=drd ./program <arguments>
Brendan Gregg's page is a good place to start learning about perf: http://www.brendangregg.com/perf.html
He also made the nice Flame Graphs tools: http://www.brendangregg.com/FlameGraphs/cpuflamegraphs.html
Compile the stack with '-fno-omit-frame-pointer' (for clang at least) to resolve 'unknown' function calls in perf's stack. More hints on resolving the stack are here: http://www.brendangregg.com/perf.html#StackTraces
To profile a mpi only program:
git clone [email protected]:brendangregg/FlameGraph.git flamegraph
args="cube/cube.dmg cube/pumi7k/cube.smb crap/ 2 pmetis ptn 1"
run="mpirun -np 2 ./ptnParma_nothread $args"
case=mpiCube
perf record -F 2997 -g $run
perf script | flamegraph/stackcollapse-perf.pl | flamegraph/flamegraph.pl > ${case}.svg
perf stat -r 5 -d --output ${case}.stats $run
The last line of the above outputs average and std deviation statistics over five runs. These stats include instructions per cycle, stall counts, branch counts, and cache misses.
Compile the stack with -pg flag. After executing your program there will be a gmon.out. To create the call graph follow these steps:
gprof path2executable gmon.out > analysis.txt
path2gprof2dot.py/gprof2dot.py analysis.txt > analysis.dot
dot -Tpdf -o analysis.pdf analysis.dot
gprof2dot.py can be downloaded from here gprof2dot.