payton/nt9856x
Template
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
fcf604c606
nt9856x/tools/trace_analyze.py
payton 543d2da268 1.一次S530代码提交
2023-03-28 15:07:53 +08:00

1256 lines
40 KiB
Python
Executable File
Raw Blame History

#!/usr/bin/env python
"""
Copyright (C) 2012 Ezequiel Garcia <elezegarcia@gmail.com>
Licensed under the terms of the GNU GPL License version 2
trace_analize.py
----------------
0. Introduction
---------------
This script allows to perform some analysis on kernel dynamic memory
allocations by post-processing ftrace kmem event.
In addition, it can also report on static footprint on a built kernel tree.
trace_analyze.py typically needs access to:
1) a built kernel tree and, 2) an ftrace kmem log.
Since reading the kmem event log is a costly operation,
you can also generate a 'db' file to speed-up subsequent runs of the script.
This script and work related has been done thanks to the CEWG project
"Kernel dynamic memory allocation tracking and reduction"
You can find lot more information about this script and on kernel dynamic
memory tracking here:
http://elinux.org/Kernel_dynamic_memory_analysis
Disclaimer:
trace_analyze.py is not stable, so expect some roughness.
Testing and feedback is more than welcome.
In fact, even some flames are welcome.
1. Using trace_analyze.py for static analysis
---------------------------------------------
Usage is fairly simple
$ ./trace_analyze.py -k /usr/src/linux -r foo.png
$ ./trace_analyze.py --kernel /usr/src/linux --rings-file foo.png
This should produce a ringchart png file in the current directory.
Of course, you can use absolute and relative paths in the path parameter
$ ./trace_analyze.py -k ../../torvalds -r foo.png
If you're interested in a specific subsystem you can use a parameter to specify
the directory tree branch to take as root
$ ./trace_analyze -k linux --start-branch fs/ext2 -r ext2.png
$ ./trace_analyze -k linux -b drivers -r drivers.png
$ ./trace_analyze -k linux -b mm -r mm.png
Each of this commands will produce a ringchart png file in the
curent directory, named as specified.
What's under the hood?
The script will perform a directory walk, internally creating a tree matching
the provided kernel tree. On each object file found (like fs/inode.o) it will
perform a 'readelf --syms' to get a list of symbols contained in it. Nothing fancy.
2. Using trace_analyze.py for dynamic analysis
----------------------------------------------
2.1. Producing a kmem trace log file
In case you don't know or don't remember how to use ftrace to
produce kmem events, here's a little remainder.
For more information, please refer to the canonical
trace documentation at the linux tree:
- Documentation/trace/ftrace.txt
- Documentation/trace/tracepoint-analysis.txt
- and everything else inside Documentation/trace/
The purpose of trace_analyze script is to perform dynamic memory analysis.
For this to work you need feed it with a kmem trace log file
(of course, you also need to give hime a built kernel tree).
Such log must be produced on the running target kernel,
but you can post-process it off-box.
For instance, you boot your kernel with kmem parameters
to enable ftrace kmem events:
(it's recommended to enable all events, despite not running a NUMA machine).
kmem="kmem:kmalloc,kmem:kmalloc_node,kmem:kfree, \
kmem:kmem_cache_alloc,kmem:kmem_cache_alloc_node,kmem:kmem_cache_free"
This parameter will have linux to start tracing as soon as possible.
Of course some early traces will be lost, see below.
(on your target kernel)
# To stop tracing
$ echo "0" > /sys/kernel/debug/tracing/tracing_on
# Dump
$ cat /sys/kernel/debug/tracing/trace > kmem.log
Now you need to get this file so you can post-process
it using trace_analyze.py.
In my case, I use qemu with a file backing serial device,
so I simply do:
(on your target kernel)
$ cat /sys/kernel/debug/tracing/trace > /dev/ttyS0
And I get the log on qemu's backing file.
Now you have everything you need to start the analysis.
2.2. Slab accounting file output
To obtain a memory accounting file you need to use
--acount-file (-c) parameter, like this:
$ ./trace_analyze.py -k linux -f kmem.log --account-file account.txt
$ ./trace_analyze.py -k linux -f kmem.log -c account.txt
This will produce an account file like this:
current bytes allocated: 669696
current bytes requested: 618823
current wasted bytes: 50873
number of allocs: 7649
number of frees: 2563
number of callers: 115
total waste net alloc/free caller
---------------------------------------------
299200 0 298928 1100/1 alloc_inode+0x4fL
189824 0 140544 1483/385 __d_alloc+0x22L
51904 0 47552 811/68 sysfs_new_dirent+0x4eL
16384 8088 16384 1/0 __seq_open_private+0x24L
15936 1328 15936 83/0 device_create_vargs+0x42L
14720 10898 14016 460/22 sysfs_new_dirent+0x29L
2.3. Controlling account output
You can ask the script to read only kmalloc events
(notice the option name is *--malloc*):
$ ./trace_analyze.py -k linux -f kmem.log -c account.txt --malloc
Or you can ask the script to read only kmem_cache events:
$ ./trace_analyze.py -k linux -f kmem.log -c account.txt --cache
If you want to order the account file you can use --order-by (-o):
$ ./trace_analyze.py -k linux -f kmem.log -c account.txt --order-by=waste
$ ./trace_analyze.py -k linux -f kmem.log -c account.txt --malloc -o waste
The possible options for order-by parameter are:
* total_dynamic: Added allocations size
* current_dynamic: Currently allocated size
* alloc_count: Number of allocations
* free_count: Number of frees
* waste: Currently wasted size
You can pick a directory to get an account file showing
only the allocations from that directory.
This is done with the --start-branch (-b) option,
just like we've done for the static analysis:
$ ./trace_analyze.py -k linux -f kmem.log -c account.txt -b drivers/base/
All of these options can be combined.
For instance, if you want to get kmalloc events only,
coming from fs/ directory and ordered by current dynamic footprint:
$ ./trace_analyze.py -k linux -f kmem.log -b fs -c account.txt -o current_dynamic --malloc
2.4. Producing a pretty ringchart for dynamic allocations
As already explained in the static analysis section, it's possible to produce
a ringchart to get **the big picture** of dynamic allocations.
You will need to have *matplotlib* installed, which should be as easy as:
$ {your_pkg_manager} install matplotlib
The script usage is very simple,
just pass the parameter --rings-file (-r) along with a filename
$ ./trace_analyze.py -k linux -f kmem.log --rings-file=dynamic.png
This command will produce a png file named as specified.
The plot will show current dynamic allocations by default.
You can control the used attrbute used for the ringchart
plot using --rings-attr (-a) parameter.
The available options are:
- current: static + current dynamic size
- static: static size
- waste: wasted size
- current_dynamic: current dynamic size
- total_dyamic: added dynamic size
For instance, you may want a ringchart for wasted bytes
$ ./trace_analyze.py -k linux -f kmem.log -r -a waste
You can use --start-branch (-b) parameter to plot allocations made from just one directory.
For instance, if you want to get wasted bytes for ext4 filesystem:
$ ./trace_analyze.py -k ../torvalds -f kmem.log \
-r ext4_waste.png -a waste -b fs/ext4
Or, if you want to see static footprint of arch-dependent mm code:
$ ./trace_analyze.py -k ../torvalds -f kmem.log \
-r x86_static.png -a static -b arch/x86/mm
Also, you can filter kmalloc or kmem_cache traces
using either --malloc, or --cache:
$ ./trace_analyze.py -k linux/ -f boot_kmem.log -r kmallocs.png --malloc
2.5. Pitfall: wrongly reported allocation (and how to fix it)
There are a number of functions (kstrdup, kmemdup, krealloc, etc) that do
some kind of allocation on behalf of its caller.
Of course, we don't want to get trace reports from these functions,
but rather from its caller. To acomplish this, we must use a variant
of kmalloc, called kmalloc_track_caller, which does exactly that.
Let's see an example. As of today kvasprintf() implementation looks
like this
(see lib/kasprintf.c:14)
char *kvasprintf(gfp_t gfp, const char *fmt, va_list ap)
{
/* code removed */
p = kmalloc(len+1, gfp);
And trace_analyze produces the account file
total waste net alloc/free caller
---------------------------------------------
2161 1184 2161 148/0 kvasprintf
The source of this 148 allocations may be a single caller,
or it may be multiple callers. We just can't know.
However, if we replace kmalloc with kmalloc_track_caller,
we're going to find that out.
char *kvasprintf(gfp_t gfp, const char *fmt, va_list ap)
{
/* code removed */
p = kmalloc_track_caller(len+1, gfp);
After running the re-built kernel, and comparing both current
and previous account files, we find this is the real caller:
total waste net alloc/free caller
---------------------------------------------
2161 1184 2161 148/0 kobject_set_name_vargs
So, we've accurately tracked this allocation down to the kobject code.
3. Using a DB file to speed-up multiple runs
--------------------------------------------
You may find yourself analyzing a large kmem log file.
Probably, you want to run the script
several times to get different kinds of results.
The script is not very clever and will re-read the
long kmem file on each run.
To alleviate this problem you can have trace_analyze.py
create a so-called DB file,and use this file instead
of the kmem log file on subsequent runs.
This is done using the --save-db and --db-file parameters.
Like this:
$ ./trace_analyze.py -k ../torvalds/ -f kmem.log --save-db db
Notice you should create the DB file without any filters,
like --malloc or --start-branch, in order to save the full kmem event log.
Once you have the **db** file created, you would use it on each run
$ ./trace_analyze.py -k ../torvalds/ --db-file db \
-r rings.png -c account.txt
Hopefully, this would prevent you from cursing trace_analyze for being so slow.
"""
import sys
import string
import re
import subprocess
import math
import pickle
import os
from optparse import OptionParser
# Skip this directories when walking kernel build
BLACKLIST = ("scripts", "tools")
class Ptr:
def __init__(self, fun, ptr, alloc, req):
self.fun = fun
self.ptr = ptr
self.alloc = alloc
self.req = req
class Callsite:
def __init__(self):
self.__alloc = 0
self.__req = 0
self.__alloc_count = 0
self.__free_count = 0
self.ptrs = []
def total_dynamic(self):
return self.__alloc
def alloc_count(self):
return self.__alloc_count
def free_count(self):
return self.__free_count
def current_dynamic(self):
alloc = 0
for ptr in self.ptrs:
alloc += ptr.alloc
return alloc
def current_req(self):
req = 0
for ptr in self.ptrs:
req += ptr.req
return req
def waste(self):
return self.current_dynamic() - self.current_req()
def do_alloc(self, alloc, req, ptr):
self.__alloc += alloc
self.__req += req
self.__alloc_count += 1
self.ptrs.append(ptr)
def do_free(self, ptr):
self.__free_count += 1
self.ptrs.remove(ptr)
# Based on addr2sym.py
class SymbolMap:
def __init__(self, filemap):
self.fmap = {}
self.flist = []
self.cache = {}
try:
f = open(filemap)
except:
print("[ERROR] Cannot read symbol map file {}".format(filemap))
sys.exit(1)
for line in f.readlines():
(addr_str, symtype, name) = string.split(line, None, 3)
self.fmap[addr_str] = name
addr = eval("0x" + addr_str + "L")
self.flist.append((addr, name))
f.close()
def lookup(self, addr_str):
# return a tuple (string, offset) for a given address
if addr_str in self.fmap:
return (self.fmap[addr_str],0)
# convert address from string to number
addr = eval("0x" + addr_str + "L")
if addr in self.cache:
return self.cache[addr]
# if address is outside range of addresses in the
# map file, just return the address without converting it
if addr < self.flist[0][0] or addr > self.flist[-1][0]:
return (addr_str,0)
# no exact match found, now do binary search for closest function
# do a binary search in funclist for the function
# use a collapsing range to find the closest addr
lower = 0
upper = len(self.flist)-1
while (lower != upper-1):
guess_index = lower + (upper-lower)/2
guess_addr = self.flist[guess_index][0]
if addr < guess_addr:
upper = guess_index
if addr >= guess_addr:
lower = guess_index
offset = addr-self.flist[lower][0]
name = self.flist[lower][1]
if name.startswith("."):
name = name[1:]
self.cache[addr] = (name, offset)
return (name, offset)
class EventDB:
def __init__(self):
self.f = {}
self.p = {}
self.num_allocs = 0
self.total_dynamic = 0
self.total_req = 0
self.num_frees = 0
self.num_lost_frees = 0
def slurp(self, path, buildpath, do_malloc, do_cache):
print("Reading symbol map at {}".format(buildpath))
sym = SymbolMap(buildpath + "/System.map")
try:
logfile = open(path)
except:
print("[ERROR] Cannot read log file {}".format(path))
sys.exit(1)
kmalloc_re = r".*kmalloc.*call_site=([a-f0-9]+).*ptr=([a-f0-9]+).*bytes_req=([0-9]+)\s*bytes_alloc=([0-9]+)"
kfree_re = r".*kfree.*call_site=[a-f0-9+]+.*ptr=([a-f0-9]+)"
cache_alloc_re = r".*cache_alloc.*call_site=([a-f0-9]+).*ptr=([a-f0-9]+).*bytes_req=([0-9]+)\s*bytes_alloc=([0-9]+)"
cache_free_re = r".*cache_free.*call_site=[a-f0-9+]+.*ptr=([a-f0-9]+)"
both_alloc_re = r".*k.*alloc.*call_site=([a-f0-9]+).*ptr=([a-f0-9]+).*bytes_req=([0-9]+)\s*bytes_alloc=([0-9]+)"
both_free_re = r".*k.*free.*call_site=[a-f0-9+]+.*ptr=([a-f0-9]+)"
if do_malloc is True and do_cache is None:
print("Slurping event log, kmalloc events only")
alloc_re = kmalloc_re
free_re = kfree_re
elif do_malloc is None and do_cache is True:
print("Slurping event log, kmem_cache events only")
alloc_re = cache_alloc_re
free_re = cache_free_re
else:
print("Slurping event log")
alloc_re = both_alloc_re
free_re = both_free_re
for line in logfile:
m = re.match(alloc_re, line)
if m:
(fun, offset) = sym.lookup(m.group(1))
self.add_malloc("{}+0x{:x}".format(fun, offset),
m.group(2),
int(m.group(3)),
int(m.group(4)), line)
m = re.match(free_re, line)
if m:
self.add_free(m.group(1))
def get_bytes(self):
alloc = 0
req = 0
for fun, callsite in self.f.items():
alloc += callsite.current_dynamic()
req += callsite.current_req()
return (alloc, req)
def add_malloc(self, fun, ptr, req, alloc, line):
self.num_allocs += 1
self.total_dynamic += alloc
self.total_req += req
ptr_obj = Ptr(fun, ptr, alloc, req)
if ptr in self.p:
print("[WARNING] Duplicate pointer! {}".format(line))
self.p[ptr] = ptr_obj
if not fun in self.f:
self.f[fun] = Callsite()
self.f[fun].do_alloc(alloc, req, ptr_obj)
def add_free(self, ptr):
self.num_frees += 1
if not ptr in self.p:
self.num_lost_frees += 1
return
ptr_obj = self.p[ptr]
self.f[ptr_obj.fun].do_free(ptr_obj)
# Remove it from pointers dictionary
del self.p[ptr]
def print_callers(self, filepath, filter_tree=None):
if filter_tree is None:
filter_symbol = lambda f: True
get_symbol_dir = lambda f: ""
else:
filter_symbol = filter_tree.symbol_is_here
get_symbol_dir = filter_tree.get_symbol_dir
syms = [(f,c) for f,c in self.f.items() if filter_symbol(f)]
f = open(filepath, 'w')
for name, c in syms:
symdir = get_symbol_dir(name)
f.write("{:<60} {:<8} {:<8} {:<8}\n".format(name,
c.current_dynamic(),
c.waste(),
symdir))
f.close()
def print_account(self, filepath, order_by, filter_tree=None):
current_dynamic = 0
current_req = 0
alloc_count = 0
free_count = 0
if filter_tree is None:
filter_symbol = lambda f: True
else:
filter_symbol = filter_tree.symbol_is_here
syms = [(f,c) for f,c in self.f.items() if filter_symbol(f)]
f = open(filepath, 'w')
for fun, callsite in syms:
current_dynamic += callsite.current_dynamic()
current_req += callsite.current_req()
alloc_count += callsite.alloc_count()
free_count += callsite.free_count()
f.write("current bytes allocated: {:>10}\n".format(current_dynamic))
f.write("current bytes requested: {:>10}\n".format(current_req))
f.write("current wasted bytes: {:>10}\n".format((current_dynamic -
current_req)))
f.write("number of allocs: {:>10}\n".format(alloc_count))
f.write("number of frees: {:>10}\n".format(free_count))
f.write("number of callers: {:>10}\n".format(len(syms)))
f.write("\n")
f.write(" total waste net alloc/free caller\n")
f.write("---------------------------------------------\n")
for fun, callsite in sorted(syms,
key=lambda item: getattr(item[1],
order_by)(),
reverse=True):
f.write("%8d %8d %8d %5d/%-5d %s\n" % (callsite.total_dynamic(),
callsite.waste(),
callsite.current_dynamic(),
callsite.alloc_count(),
callsite.free_count(),
fun))
f.close()
class MemTreeNodeSize:
def __init__(self, node):
self.__static = 0
self.__total_dynamic = 0
self.__current_dynamic = 0
self.__waste = 0
# First for my symbols
for sym, size in node.data.items():
self.__static += size
for sym, size in node.text.items():
self.__static += size
for sym, call in node.funcs.items():
self.__total_dynamic += call.total_dynamic()
self.__current_dynamic += call.current_dynamic()
self.__waste += call.current_dynamic() - call.current_req()
# Now, for my children's symbols.
# Or, instead, we could first add all my children's
# symbols here and then get the node size.
for name, child in node.childs.items():
self.__total_dynamic += child.size().total_dynamic()
self.__current_dynamic += child.size().current_dynamic()
self.__static += child.size().static()
self.__waste += child.size().waste()
def current(self):
return self.__static + self.__current_dynamic
def waste(self):
return self.__waste
def static(self):
return self.__static
def current_dynamic(self):
return self.__current_dynamic
def total_dynamic(self):
return self.__total_dynamic
class MemTreeNode:
def __init__(self, name="", parent=None, db=None):
self.name = name
self.parent = parent
self.childs = {}
self.funcs = {}
self.data = {}
self.text = {}
self.node_size = None
self.fill = getattr(self, "fill_per_file")
# If db is None, use parent db
if db is None:
if parent is not None:
self.db = parent.db
else:
self.db = db
def get_symbol_dir(self, symbol):
if symbol in self.funcs:
return self.full_name()
else:
for name, child in self.childs.items():
symdir = child.get_symbol_dir(symbol)
if symdir is not None:
return symdir
return None
def symbol_is_here(self, symbol):
if symbol in self.funcs:
return True
else:
for name, child in self.childs.items():
if child.symbol_is_here(symbol):
return True
return False
def full_name(self):
l = [self.name,]
parent = self.parent
while parent:
if parent.name != "":
l.append(parent.name)
parent = parent.parent
return "/".join(reversed(l))
def size(self):
if self.node_size is None:
self.node_size = MemTreeNodeSize(self)
return self.node_size
def __collapse(self):
# Collapse one-child empty nodes
for name, child in self.childs.items():
if len(child.childs) > 2:
child.__collapse()
if len(child.childs) == 1 and not child.funcs and not child.data:
# Remove from child
(k, v) = child.childs.items()[0]
del child.childs[k]
# Add here
self.childs[k] = v
v.parent = self
def __strip(self):
# Remove empty nodes
for name, child in self.childs.items():
if child.childs:
child.__strip()
if not child.funcs and not child.data and not child.childs:
del self.childs[name]
def __get_root(self):
if len(self.childs) == 1:
child = self.childs.itervalues().next()
# This is a pedantic test, the first node with
# multiple childs is the root we're searching
if not child.name.endswith(".o"):
return child.__get_root()
return self
# Obtain a clean tree.
# We do it this way because collapse() and strip() must be called
# in an ordered fashion.
def get_clean(self):
self.__collapse()
self.__strip()
return self.__get_root()
def find_first_branch(self, which):
if self.name == which:
return self
for name, node in self.childs.items():
if which == name:
return node
for name, node in self.childs.items():
return node.find_first_branch(which)
print("[WARNING] Can't find first branch '{}'".format(which))
return None
# This are for debug purposes, move along
def treelike(self, level=0, attr="current_dynamic"):
str = ""
str += "{}\n".format(self.name)
for name, node in self.childs.items():
child_str = node.treelike(level+1, attr)
if child_str:
str += "{}{}".format(" "*(level+1), child_str)
return str
def treelike2(self, level=0, attr="current_dynamic"):
str = ""
attr_val = getattr(self.size(), attr)()
if self.name and attr_val != 0:
str += "{} - {}={}\n".format(self.name, attr, attr_val)
for name, node in self.childs.items():
child_str = node.treelike2(level+1, attr)
if child_str:
str += "{}{}".format(" "*(level+1), child_str)
return str
def fill_per_file(self, path):
filepath = "{}{}/{}".format(MemTreeNode.abs_slash, self.full_name(), path)
if path not in self.childs:
self.childs[path] = MemTreeNode(path, self)
child = self.childs[path]
output = []
try:
p1 = subprocess.Popen(["readelf", "--wide", "-s", filepath], stdout=subprocess.PIPE)
output = p1.communicate()[0].split("\n")
except:
pass
for line in output:
if line == '':
continue
m = re.match(r".*\s([0-9]+)\sFUNC.*\s+([a-zA-Z0-9_\.]+)\b", line)
if m:
if m.group(2) in child.text:
print("Duplicate text entry! {}".format(m.group(2)))
child.text[m.group(2)] = int(m.group(1))
# Search every callsite in db matching this name
for name, callsite in child.db.f.iteritems():
if name.startswith(m.group(2)):
child.funcs[name] = callsite
m = re.match(r".*\s([0-9]+)\sOBJECT.*\s+([a-zA-Z0-9_\.]+)\b", line)
if m:
if m.group(2) in child.data:
print("[WARNING] Duplicate data entry! {}".format(m.group(2)))
child.data[m.group(2)] = int(m.group(1))
# This is deprecated, fill_per_file should be used instead.
# I keep it here just to have the code handy.
def fill_per_dir(self, path):
if self.funcs or self.data:
print("[WARNING] Oooops, already filled")
filepath = "." + self.full_name() + "/built-in.o"
output = []
try:
p1 = subprocess.Popen(["readelf", "--wide", "-s", filepath], stdout=subprocess.PIPE)
output = p1.communicate()[0].split("\n")
except:
pass
for line in output:
if line == '':
continue
m = re.match(r".*FUNC.*\b([a-zA-Z0-9_]+)\b", line)
if m:
if m.group(1) in self.funcs:
print("[WARNING] Duplicate entry! {}".format(m.group(1)))
if m.group(1) in self.db.f:
self.funcs[m.group(1)] = self.db.f[m.group(1)]
m = re.match(r".*([0-9]+)\sOBJECT.*\b([a-zA-Z0-9_]+)\b", line)
if m:
self.data[m.group(2)] = int(m.group(1))
# path is should be an object file, like fs/ext2/inode.o
def add_child(self, path):
# adding a child invalidates node_size object
self.node_size = None
parts = path.split('/', 1)
if len(parts) == 1:
self.fill(path)
pass
else:
node, others = parts
if node not in self.childs:
self.childs[node] = MemTreeNode(node, self)
self.childs[node].add_child(others)
def add_path(self, path):
for root, dirs, files in os.walk(path):
blacklisted = False
for bdir in BLACKLIST:
if root.startswith("{}/{}".format(path, bdir)):
blacklisted = True
if blacklisted:
continue
for filepath in [os.path.join(root,f) for f in files]:
if filepath.endswith("built-in.o"):
continue
if filepath.endswith("vmlinux.o"):
continue
if filepath.endswith(".o"):
# We need to check if this object file,
# has a corresponding source file
filesrc = "{}.c".format(os.path.splitext(filepath)[0])
if os.path.exists(filesrc):
self.add_child(filepath)
##########################################################################
##
## Main
##
##########################################################################
def main():
parser = OptionParser()
parser.add_option("-k", "--kernel",
dest="buildpath",
default=".",
help="path to built kernel tree (default is current dir)")
parser.add_option("-f", "--file",
dest="file",
default="",
help="trace log file to analyze")
parser.add_option("--db-file",
dest="db_file",
default="",
help="use db_file as DB instead of creating one")
parser.add_option("--save-db",
dest="save_db_file",
default="",
help="save a db_file to use as DB")
parser.add_option("-b", "--start-branch",
dest="start_branch",
default="",
help="first directory name to use as ringchart root")
parser.add_option("-r", "--rings-file",
dest="rings_file",
default="",
help="plot ringchart information")
parser.add_option("-i", "--rings-show",
dest="rings_show",
action="store_true",
help="show interactive ringchart")
parser.add_option("-a", "--rings-attr",
dest="rings_attr",
default="current_dynamic",
help="attribute to visualize [static, current, \
current_dynamic, total_dynamic, waste]")
parser.add_option("--malloc",
dest="do_malloc",
action="store_true",
help="trace kmalloc/kfree only")
parser.add_option("--cache",
dest="do_cache",
action="store_true",
help="trace kmem_cache_alloc/kmem_cache_free only")
parser.add_option("-c", "--account-file",
dest="account_file",
default="",
help="show output matching slab_account output")
parser.add_option("-l", "--callers-file",
dest="callers_file",
default="",
help="show callers file suitable for ringchart generation")
parser.add_option("-o", "--order-by",
dest="order_by",
default="current_dynamic",
help="attribute to order account \
[current_dynamic, total_dynamic, alloc_count, free_count, waste]")
(opts, args) = parser.parse_args()
# Kernel build path is a mandatory parameter.
# We need to look at compiled objects and also for System.map.
if len(opts.db_file) == 0 and len(opts.buildpath) == 0:
print("Please set a kernel build path or a DB file!")
parser.print_help()
return
# Check valid options
if len(opts.order_by) > 0:
if opts.order_by not in dir(Callsite):
print("Hey! {} is not a valid --order-by option".format(opts.order_by))
parser.print_help()
return
if len(opts.rings_attr) > 0:
if opts.rings_attr not in dir(MemTreeNodeSize):
print("Hey! {} is not a valid --rings-attr option".format(opts.rings_attr))
parser.print_help()
return
# Clean user provided kernel path from dirty slashes
buildpath = opts.buildpath.rstrip("/")
# If we don't have a trace log file,
# and we don't have a DB file
# then we'll fallback to static report mode.
if len(opts.db_file) == 0 and len(opts.file) == 0:
print("No trace log file or DB file specified: will report on static size only")
opts.rings_attr = "static"
opts.do_malloc = False
opts.do_cache = False
opts.account_file = ""
opts.just_static = True
# Set some default
if len(opts.rings_file) == 0:
opts.rings_file = "rings_static.png"
else:
opts.just_static = False
if opts.rings_show is None:
opts.rings_show = False
rootDB = EventDB()
# Get root database, if need to
if not opts.just_static:
if len(opts.db_file) != 0:
print("Using db file '{}'".format(opts.db_file))
f = open(opts.db_file)
buildpath = pickle.load(f)
rootDB = pickle.load(f)
f.close()
else:
rootDB.slurp(opts.file, buildpath, opts.do_malloc, opts.do_cache)
if len (opts.save_db_file) != 0:
print("Saving db file at '{}'".format(opts.save_db_file))
f = open(opts.save_db_file, 'w')
pickle.dump(buildpath,f)
pickle.dump(rootDB, f)
f.close()
if len(opts.callers_file) == 0 and \
len(opts.account_file) == 0 and \
len(opts.rings_file) == 0 and \
opts.rings_show is False:
sys.exit(0)
root_path = "{}/{}".format(buildpath, opts.start_branch).rstrip("/")
print("Creating tree from compiled symbols at '{}'".format(root_path))
# We need to specify if user provided buildpath is absolute
MemTreeNode.abs_slash = buildpath.startswith("/") and "/" or ""
tree = MemTreeNode(db = rootDB)
tree.add_path(root_path)
print("Cleaning tree")
tree = tree.get_clean()
# DEBUG--ONLY. Should we add an option for this?
#print(tree.treelike2(attr = opts.rings_attr))
if len(opts.callers_file) != 0:
print("Creating callers file at '{}'".format(opts.callers_file))
rootDB.print_callers(opts.callers_file,
tree)
if len(opts.account_file) != 0:
print("Creating account file at '{}'".format(opts.account_file))
rootDB.print_account(opts.account_file,
opts.order_by,
tree)
if len(opts.rings_file) != 0:
if tree is None:
print("Sorry, there is nothing to plot for branch '{}'".format(opts.start_branch))
else:
print("Creating ringchart for attribute '{}'".format(opts.rings_attr))
visualize_mem_tree(tree, opts.rings_attr, opts.rings_file, opts.rings_show)
##########################################################################
##
## Visualization stuff
##
##########################################################################
CENTER_X = 1.0
CENTER_Y = 1.0
WIDTH = 0.2
tango_colors = ['#ef2929',
'#ad7fa8',
'#729fcf',
'#8ae234',
'#e9b96e',
'#fcaf3e',]
def human_bytes(bytes, precision=1):
"""Return a humanized string representation of a number of bytes.
Assumes `from __future__ import division`.
>>> humanize_bytes(1)
'1 byte'
>>> humanize_bytes(1024)
'1.0 kB'
>>> humanize_bytes(1024*123)
'123.0 kB'
>>> humanize_bytes(1024*12342)
'12.1 MB'
>>> humanize_bytes(1024*12342,2)
'12.05 MB'
>>> humanize_bytes(1024*1234,2)
'1.21 MB'
>>> humanize_bytes(1024*1234*1111,2)
'1.31 GB'
>>> humanize_bytes(1024*1234*1111,1)
'1.3 GB'
"""
abbrevs = (
(1<<50L, 'PB'),
(1<<40L, 'TB'),
(1<<30L, 'GB'),
(1<<20L, 'MB'),
(1<<10L, 'kB'),
(1, 'bytes')
)
if bytes == 1:
return '1 byte'
for factor, suffix in abbrevs:
if bytes >= factor:
break
return '{0:.{1}f} {2}'.format(float(bytes)/factor, precision, suffix)
class Section:
def __init__(self, node, size, total_size, total_angle, start_angle):
self.node = node
self.size = size
self.start_angle = start_angle
self.angle = size * total_angle / total_size
def ring_color(start_angle, level):
from matplotlib.colors import colorConverter
# f: [1 - 0.26]
# rel: [0 - 198]
# icolor: [0 - 5]
if level == 1:
return colorConverter.to_rgb('#808080')
f = 1 - (((level-1) * 0.3) / 8)
rel = start_angle / 180. * 99
icolor = int(rel / (100./3))
next_icolor = (icolor + 1) % 6
# Interpolate (?)
color = colorConverter.to_rgb(tango_colors[icolor])
next_color = colorConverter.to_rgb(tango_colors[next_icolor])
p = (rel - icolor * 100./3) / (100./3)
color = [f * (c - p * (c - n)) for c, n in zip(color, next_color)]
return color
def create_child_rings(tree, level=2, level_angle=360, start_angle=0, rings=[],
radius=WIDTH, center=(CENTER_X, CENTER_Y), size_attr="static"):
from matplotlib.patches import Wedge
child_size = 0
max_size = getattr(tree.size(), size_attr)()
if len(tree.childs) == 0:
return rings
if max_size == 0:
for name, node in tree.childs.items():
max_size += getattr(node.size(), size_attr)()
if max_size == 0:
return rings
s_angle = start_angle
sections = {}
# Create child wedges
for name, node in tree.childs.items():
size = getattr(node.size(), size_attr)()
s = Section(node, size, max_size, level_angle, s_angle)
sections[name] = s
create_child_rings(node, level+1, s.angle, s_angle, rings, radius, center, size_attr)
s_angle += s.angle
child_size += size
# Just a check
if child_size > max_size:
print("[{}] Ooops, child size is greater than max size".format(name))
for name, section in sections.items():
# Create tuple: (wedge, name)
name = "{} {}".format(name, human_bytes(section.size))
tup = ( Wedge(center,
level * radius,
section.start_angle,
section.start_angle + section.angle,
width=radius,
facecolor=ring_color(section.start_angle, level)),
name)
rings.append(tup)
return rings
def visualize_mem_tree(tree, size_attr, filename, show):
import pylab
RING_MIN_WIDTH = 1
TEXT_MIN_WIDTH = 5
rings = create_child_rings(tree, size_attr=size_attr)
fig = pylab.figure()
ax = fig.add_subplot(111)
annotations = []
labels = []
name = tree.name
if name == '.':
name = "/"
text = "{} {}".format(name,
human_bytes(getattr(tree.size(), size_attr)()))
ann = ax.annotate(text,
size=12,
bbox=dict(boxstyle="round", fc="w", ec="0.5", alpha=0.8),
xy=(CENTER_X, CENTER_Y), xycoords='data',
xytext=(CENTER_X, CENTER_Y), textcoords='data')
annotations.append(ann)
for p in rings:
wedge = p[0]
# Skip if too small
if (wedge.theta2 - wedge.theta1) < RING_MIN_WIDTH:
continue
# Add wedge
ax.add_patch(wedge)
# Skip text if too small
if (wedge.theta2 - wedge.theta1) < TEXT_MIN_WIDTH:
continue
theta = math.radians((wedge.theta1 + wedge.theta2) / 2.)
x0 = wedge.center[0] + (wedge.r - wedge.width / 2.) * math.cos(theta)
y0 = wedge.center[1] + (wedge.r - wedge.width / 2.) * math.sin(theta)
x = wedge.center[0] + (0.1 + wedge.r * 1.5 - wedge.width / 2.) * math.cos(theta)
y = wedge.center[1] + (0.1 + wedge.r * 1.5 - wedge.width / 2.) * math.sin(theta)
ax.plot(x0, y0, ".", color="black")
text = p[1]
ann = ax.annotate(text,
size=12,
bbox=dict(boxstyle="round", fc="w", ec="0.5", alpha=0.8),
xy=(x0, y0), xycoords='data',
xytext=(x, y), textcoords='data',
arrowprops=dict(arrowstyle="-", connectionstyle="angle3, angleA=0, angleB=90"),)
annotations.append(ann)
(alloc, req) = tree.db.get_bytes()
pylab.axis('off')
if len(filename) != 0:
print("Plotting to file '{}'".format(filename))
pylab.savefig("{}".format(filename),
bbox_extra_artists=annotations,
bbox_inches='tight', dpi=300)
if show:
print("Plotting interactive")
pylab.show()
##########################################################################
if __name__ == "__main__":
main()
Reference in New Issue View Git Blame Copy Permalink