Skip to content

Commit 23d2b47

Browse files
committed
Merge remote-tracking branch 'upstream/master'
2 parents 3bcc158 + 0a8f27e commit 23d2b47

32 files changed

Lines changed: 1325 additions & 151 deletions

‎.travis.yml‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,11 @@
1+
sudo: required
2+
dist: trusty
13
language: python
24

35
python:
46
- 2.7
57
- 3.4
8+
- 3.5
69

710
cache: apt
811

@@ -11,13 +14,14 @@ before_install:
1114
- sudo apt-get install -y python-dev
1215
- sudo apt-get install -y libdwarf-dev
1316
- sudo apt-get install -y libelfg0-dev
14-
- sudo apt-get install -y libunwind7-dev
17+
- sudo apt-get install -y libunwind8-dev
1518

1619
install:
1720
- pip install .
1821

1922
script:
2023
- py.test tests/ -vrs
24+
- py.test vmprof/ -vrs
2125

2226
notifications:
2327
irc:

‎BUGS‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
On windows, we handle all the threads, not just ones that are running.

‎README-develop.md‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
vmprof is a delicate piece of software. Following considerations should
2+
be present when developing it.
3+
4+
Supported platform combinations (all combinations are supported):
5+
6+
* pypy (>=4.1), cpython
7+
8+
* windows, os x and linux
9+
10+
CPython should be tested on both TeamCity and Travis, PyPy is more
11+
patchy since there is never a new enough version on either. Since PyPy
12+
only exercises the pure python part, please test it each time you change
13+
and interface between _vmprof and vmprof.
14+
15+
Signals
16+
=======
17+
18+
On OS X and Linux we handle signal handlers. This means that we have
19+
to be very very careful at what we can and cannot do. Notably, we can't
20+
use malloc, locks or refcounting in any of the signal handlers. Python data
21+
should be read-only and we should be prepared to read garbage or NULL
22+
from anything.
23+
24+
On windows we use an external thread, so it's an imperative we freeze
25+
the thread we're inspecting. It's possible that the interpreter state
26+
handling is not thread safe in a way it should be, investigate how we
27+
can improve it.

‎README.md‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,14 +7,23 @@
77

88
See https://vmprof.readthedocs.org for up to date info
99

10-
basic usage:
10+
## Basic usage
1111

12+
###Linux
1213
```console
1314
sudo apt-get install python-dev
1415
pip install vmprof
1516
python -m vmprof <your program> <your program args>
1617
```
1718

19+
###Windows
20+
Install Microsoft Visual C++ Compiler for Python 2.7
21+
22+
pip install vmprof
23+
24+
For development:
25+
pip install -e .
26+
1827
## vmprofshow
1928

2029
`vmprofshow` is a command line tool that comes with **VMprof** which can read profile files generated by **VMprof** and produce a nicely formatted output.

‎docs/index.rst‎

Lines changed: 14 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,8 @@ continuously taking samples of the call stack of the running program, at a
1414
given frequency. This is similar to tools like `vtune`_ or `gperftools`_: the
1515
main difference is that those tools target C and C-like languages and are not
1616
very helpful to profile higher-level languages which run on top of a virtual
17-
machine, while vmprof is designed specifically for them.
17+
machine, while vmprof is designed specifically for them. vmprof is also thread
18+
safe and will correctly display the information regardless of usage of threads.
1819

1920
There are three primary modes. The recommended one is to use our server
2021
infrastructure for a web-based visualization of the result::
@@ -140,17 +141,16 @@ Example `config.ini` file::
140141
API
141142
===
142143

143-
144144
There is also an API that can bring more details to the table,
145-
but consider it unstable. The current API usage is as follows::
145+
but consider it unstable. The current API usage is as follows:
146146

147147
Module level functions
148148
----------------------
149149

150-
* ``vmprof.enable(fileno, period=0.099, memory=False)`` - enable writing ``vmprof`` data to a
151-
file described by a fileno file descriptor. Timeout is in float seconds. The
152-
minimal available resolution is 4ms, we're working on improving that
153-
(note the default is 9.9ms). Passing ``memory=True`` will provide additional
150+
* ``vmprof.enable(fileno, period=0.00099, memory=False)`` - enable writing ``vmprof`` data to a
151+
file described by a fileno file descriptor. Period is in float seconds. The
152+
minimal available resolution is around 1ms, we're working on improving that
153+
(note the default is 0.99ms). Passing ``memory=True`` will provide additional
154154
data in the form of total RSS of the process memory interspersed with
155155
tracebacks.
156156

@@ -239,3 +239,10 @@ The machinery to gather the information has been the focus of the initial
239239
phase of vmprof development and now it is working well: we are currently
240240
focusing on the frontend to make sure we can process and display the info in
241241
useful ways.
242+
243+
Links
244+
=====
245+
246+
* `vmprof-flamegraph <https://pypi.python.org/pypi/vmprof-flamegraph>`_
247+
Convert vmprof data into text format for
248+
`flamegraph <http://www.brendangregg.com/FlameGraphs/cpuflamegraphs.html>`_

‎setup.py‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,12 +26,13 @@
2626
name='vmprof',
2727
author='vmprof team',
2828
author_email='fijal@baroquesoftware.com',
29-
version="0.2.5",
29+
version="0.2.7",
3030
packages=find_packages(),
3131
description="Python's vmprof client",
3232
long_description='See https://vmprof.readthedocs.org/',
3333
url='https://github.com/vmprof/vmprof-python',
3434
install_requires=[
35+
'requests',
3536
'six',
3637
],
3738
setup_requires=['pytest-runner'],

‎src/_vmprof.c‎

Lines changed: 0 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -4,15 +4,6 @@
44
#include <frameobject.h>
55
#include <signal.h>
66

7-
// for whatever reason python-dev decided to hide that one
8-
#if PY_MAJOR_VERSION >= 3 && !defined(_Py_atomic_load_relaxed)
9-
/* this was abruptly un-defined in 3.5.1 */
10-
void *volatile _PyThreadState_Current;
11-
/* XXX simple volatile access is assumed atomic */
12-
# define _Py_atomic_load_relaxed(pp) (*(pp))
13-
#endif
14-
15-
167
#define RPY_EXTERN static
178
static PyObject* cpyprof_PyEval_EvalFrameEx(PyFrameObject *, int);
189
#define VMPROF_ADDR_OF_TRAMPOLINE(x) ((x) == &cpyprof_PyEval_EvalFrameEx)

‎src/rss_darwin.h‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
/* On OS X we can get RSS using the Mach API. */
2+
#include <mach/mach.h>
3+
#include <mach/message.h>
4+
#include <mach/kern_return.h>
5+
#include <mach/task_info.h>
6+
7+
static mach_port_t mach_task;
8+
9+
static int setup_rss(void)
10+
{
11+
mach_task = mach_task_self();
12+
return 0;
13+
}
14+
15+
static int teardown_rss(void)
16+
{
17+
return 0;
18+
}
19+
20+
static long get_current_proc_rss(void)
21+
{
22+
mach_msg_type_number_t out_count = MACH_TASK_BASIC_INFO_COUNT;
23+
mach_task_basic_info_data_t taskinfo = { .resident_size = 0 };
24+
25+
kern_return_t error = task_info(mach_task, MACH_TASK_BASIC_INFO, (task_info_t)&taskinfo, &out_count);
26+
if (error == KERN_SUCCESS) {
27+
return taskinfo.resident_size / 1024;
28+
} else {
29+
return -1;
30+
}
31+
}

‎src/rss_unix.h‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
/* On normal Unices we can get RSS from '/proc/<pid>/status'. */
2+
static int proc_file = -1;
3+
4+
static int setup_rss(void)
5+
{
6+
char buf[128];
7+
8+
sprintf(buf, "/proc/%d/status", getpid());
9+
proc_file = open(buf, O_RDONLY);
10+
return proc_file;
11+
}
12+
13+
static int teardown_rss(void) {
14+
close(proc_file);
15+
proc_file = -1;
16+
}
17+
18+
static long get_current_proc_rss(void)
19+
{
20+
char buf[1024];
21+
int i = 0;
22+
23+
if (lseek(proc_file, 0, SEEK_SET) == -1)
24+
return -1;
25+
if (read(proc_file, buf, 1024) == -1)
26+
return -1;
27+
while (i < 1020) {
28+
if (strncmp(buf + i, "VmRSS:\t", 7) == 0) {
29+
i += 7;
30+
return atoi(buf + i);
31+
}
32+
i++;
33+
}
34+
return -1;
35+
}

‎src/vmprof_common.h‎

Lines changed: 19 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -44,11 +44,9 @@ char *vmprof_init(int fd, double interval, int memory, char *interp_name)
4444
return "out of memory";
4545
#if defined(__unix__) || defined(__APPLE__)
4646
current_codes = NULL;
47-
#endif
48-
49-
#if !defined(__unix__)
47+
#else
5048
if (memory)
51-
return "memory tracking not supported on non-linux";
49+
return "memory tracking only supported on unix";
5250
#endif
5351
assert(fd >= 0);
5452
profile_file = fd;
@@ -97,3 +95,20 @@ static int opened_profile(char *interp_name, int memory)
9795
memcpy(&header.interp_name[4], interp_name, namelen);
9896
return _write_all((char*)&header, 5 * sizeof(long) + 4 + namelen);
9997
}
98+
99+
// for whatever reason python-dev decided to hide that one
100+
#if PY_MAJOR_VERSION >= 3 && !defined(_Py_atomic_load_relaxed)
101+
/* this was abruptly un-defined in 3.5.1 */
102+
extern void *volatile _PyThreadState_Current;
103+
/* XXX simple volatile access is assumed atomic */
104+
# define _Py_atomic_load_relaxed(pp) (*(pp))
105+
#endif
106+
107+
PyThreadState* get_current_thread_state(void)
108+
{
109+
#if PY_MAJOR_VERSION >= 3
110+
return (PyThreadState*)_Py_atomic_load_relaxed(&_PyThreadState_Current);
111+
#else
112+
return _PyThreadState_Current;
113+
#endif
114+
}

0 commit comments

Comments
 (0)