Skip to content

Commit

Permalink
Polish docs for the newly added features (#523)
Browse files Browse the repository at this point in the history
  • Loading branch information
gaogaotiantian authored Nov 19, 2024
1 parent 9ece82f commit 7dda50f
Show file tree
Hide file tree
Showing 11 changed files with 211 additions and 84 deletions.
25 changes: 7 additions & 18 deletions docs/source/basic_usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -68,12 +68,6 @@ You can make viztracer to generate a unique name for the output file by using ``
viztracer -u my_script.py
viztracer --output_dir ./my_reports -u my_script.py
You can also show flamegraph from ``result.json`` file

.. code-block::
vizviewer --flamegraph result.json
Inline
------

Expand Down Expand Up @@ -192,12 +186,6 @@ You can serve your HTTP server on a different port with ``--port`` or its equiva
vizviewer --port 10000 result.json
You can also show flamegraph of the result

.. code-block::
vizviewer --flamegraph result.json
You can use the external trace processor with ``--use_external_processor``, which does not have the
RAM limits as the browser. This is helpful when you try to open a large trace file.

Expand All @@ -222,12 +210,8 @@ Circular Buffer Size
--------------------

VizTracer uses a circular buffer to store the entries. When there are too many entries, it will only store the latest ones so you know what happened
recently. The default buffer size is 1,000,000(number of entries), which takes about 150MiB disk space. You can specify this when you instantiate a ``VizTracer`` object

Notice it also takes a significant amount of RAM when VizTracer is tracing the program.

VizTracer will preallocate about ``tracer_entries * 100B`` RAM for circular buffer. It also requires about ``1-2MB`` per 10k entries to
dump the json file.
recently. The default buffer size is 1,000,000(number of entries), which takes about 150MiB disk space.
You can specify this when you instantiate a ``VizTracer`` object or through CLI.

.. code-block:: python
Expand All @@ -239,6 +223,11 @@ OR
tracer = VizTracer(tracer_entries=500000)
Notice it also takes a significant amount of RAM when VizTracer is tracing the program.

VizTracer will preallocate about ``tracer_entries * 100B`` RAM for circular buffer. It also requires about ``1-2MB`` per 10k entries to
dump the json file.

Configuration file
------------------

Expand Down
34 changes: 13 additions & 21 deletions docs/source/concurrency.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,19 +16,16 @@ understanding of how different tasks consume the runtime.
viztracer --log_async my_script.py
threading
---------

VizTracer supports python native ``threading`` module without the need to do any modification to your code.
Just start ``VizTracer`` before you create threads and it will just work.

other multi-thread
------------------
Multi Thread
------------

For python3.12+, VizTracer supports all Python level multi-thread. You don't need to do anything.

For the version before 3.12, if you are using multi-thread via other mechanism, for example, PyQt thread, VizTracer can't support it out of the box.
However, you can notice VizTracer that you are in a separate thread and enable tracing in that thread with ``enable_thread_tracing``
For versions before 3.12, VizTracer supports python native ``threading`` module without the need to do any modification to your code.
Just start ``VizTracer`` before you create threads and it will just work.

If you are using multi-thread via other mechanism, for example, PyQt thread, VizTracer can't support it out of the box.
However, you can inform VizTracer that you are in a separate thread and enable tracing in that thread with ``enable_thread_tracing``

.. code-block:: python
Expand All @@ -48,7 +45,7 @@ VizTracer supports ``subprocess``. You need to make sure the main process exits
viztracer my_script_using_subprocess.py
This will generate an HTML file for all processes. There are a couple of things you need to be aware though.
This will generate a report for all processes. There are a couple of things you need to be aware though.

VizTracer patches subprocess module(to be more specific, ``subprocess.Popen``) to make this work like a magic. However, it will only patch
when the args passed to ``subprocess.Popen`` is a list(``subprocess.Popen(["python", "subscript.py"])``) and the first argument starts with
Expand All @@ -72,15 +69,10 @@ However, on Windows, ``multiprocessing.Pool`` won't work with VizTracer because
os.fork()
---------

VizTracer supports ``os.fork``, with some caveats.

On Python3.8+, it works well, the main process will wait for
forked processes to finish. You can even use ``os.exec()`` and its other forms after you fork the process. Of course
VizTracer only records what happens before ``os.exec()``, you need :ref:`generic multi process support <generic_multi_process>` to record
what happens after.

On Python3.6/3.7, VizTracer is not able to wait for the forked process to finish. It would be user's responsibility
to wait for the forked process to finish if they want to see both processes in the report.
VizTracer supports ``os.fork``. The main process will wait for forked processes to finish.
You can even use ``os.exec()`` and its other forms after you fork the process. Of course
VizTracer only records what happens before ``os.exec()``, you need :ref:`generic multi process support <generic_multi_process>`
to record what happens after.

loky
----
Expand Down Expand Up @@ -133,4 +125,4 @@ After generating ``json`` files, you need to combine them
viztracer --combine ./temp_dir/*.json
This will generate the HTML report with all the process info. You can specify ``--output_file`` when using ``--combine``.
This will generate the report with all the process info. You can specify ``--output_file`` when using ``--combine``.
2 changes: 1 addition & 1 deletion docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
# -- Project information -----------------------------------------------------

project = 'VizTracer'
copyright = '2020, Tian Gao'
copyright = '2020-2024, Tian Gao'
author = 'Tian Gao'

# The full version, including alpha/beta/rc tags
Expand Down
3 changes: 3 additions & 0 deletions docs/source/decorator.rst
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ Decorator
something periodically. You won't drain your memory and the parsing/dumping will be done in a new process,
which can minimize the performance impact to your main process.

This decorator will use ``fork`` if the start method is ``"fork"``. Otherwise it will use the same process
to dump the report.

You can pass any argument you want to ``VizTracer`` by giving it to the decorator

.. code-block:: python
Expand Down
3 changes: 3 additions & 0 deletions docs/source/extra_log.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@ VizTracer features with many extra log possibilities **without even changing you
You can start VizTracer from command line and use command line arguments to control what
you need to log.

Most of the features analyzes the AST generated from the source code and add nodes into it, so
the overhead is minimal. However, some features like ``log_func_args`` will introduce a large overhead.

Log Variable
------------

Expand Down
10 changes: 5 additions & 5 deletions docs/source/filter.rst
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,8 @@ OR
tracer = VizTracer(max_stack_depth=10)
Include Files
---------------
Include and Exclude Files
-------------------------

You can include only certain files/folders to trace by

Expand All @@ -58,9 +58,6 @@ OR
tracer = VizTracer(include_files=["./src"])
Exclude Files
---------------

Similarly, you can exclude certain files/folders to trace by

.. code-block::
Expand All @@ -74,6 +71,9 @@ OR
tracer = VizTracer(exclude_files=["./not_interested.py"])
Notice this will have a significant negative impact on performance,
as VizTracer needs to check every function entry to see if it's in the list.

Ignore C Function
-----------------

Expand Down
2 changes: 1 addition & 1 deletion docs/source/global_tracer.rst
Original file line number Diff line number Diff line change
Expand Up @@ -36,4 +36,4 @@ You can do things like:
from viztracer import get_tracer, VizObject
obj = VizObject(get_tracer(), "my variable")
obj = VizObject(get_tracer(), "my variable")
18 changes: 13 additions & 5 deletions docs/source/installation.rst
Original file line number Diff line number Diff line change
@@ -1,20 +1,28 @@
Installation
============

VizTracer requires python 3.6+ and can work on Linux/MacOs/Windows. No other dependency. For now, VizTracer only supports CPython.
VizTracer works with python 3.9+ on Linux/MacOs/Windows. No other dependency. For now, VizTracer only supports CPython.

The preferred way to install VizTracer is via pip

.. code-block::
pip install viztracer
You can also install with conda

You can also download the source code and build it yourself.
.. code-block::
conda install conda-forge::viztracer
Even though VizTracer functions without any other packages, it is still recommended to install the following packages to have a better performance.
// Or if you already have conda forge set up
conda install viztracer
You can also download the source code and build it yourself.

* orjson: better json dump/load performance
Even though VizTracer functions without any other packages,
``orjson`` could improve the performance of json dump/load.

.. code-block::
Expand All @@ -24,4 +32,4 @@ Or you can install *full* version of viztracer:

.. code-block::
pip install viztracer[full]
pip install viztracer[full]
2 changes: 1 addition & 1 deletion docs/source/license.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
License
=======

Copyright 2020-2023 Tian Gao
Copyright 2020-2024 Tian Gao

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand Down
13 changes: 9 additions & 4 deletions docs/source/limitations.rst
Original file line number Diff line number Diff line change
@@ -1,9 +1,14 @@
Limitations
===========

VizTracer uses ``sys.setprofile()`` for its profiler capabilities, so it will conflict with other profiling tools which also use this function. Be aware of it when using VizTracer
VizTracer uses ``sys.setprofile()`` (before Python3.12) and ``sys.monitoring`` (after Python3.12) for its profiler capabilities,
so it will conflict with other profiling tools which also use these mechanisms. Be aware of it when using VizTracer

The clock resolution and latency on WSL1 are very `bad <https://github.com/microsoft/WSL/issues/77>`_, so if you are using WSL1, you may experience extra overhead. There's no solution for it, except for upgrading to WSL2.
The clock resolution and latency on WSL1 are very `bad <https://github.com/microsoft/WSL/issues/77>`_, so if you are using WSL1, you may experience extra overhead.
There's no solution for it, except for upgrading to WSL2.

VizTracer, like other python tools that need to execute arbitrary code inside the module, may conflict with code that check for top module or have other structural requirements. For example, ``unittest.main()`` won't work if you use VizTracer from command line.
There are ways to avoid it. You can use inline VizTracer, which will always work. Or you can specify modules to ``unittest.main()``, which is not a general solution but could work without too much code changes.
VizTracer, like other python tools that need to execute arbitrary code inside the module,
may conflict with code that check for top module or have other structural requirements.
For example, ``unittest.main()`` won't work if you use VizTracer from command line.
There are ways to avoid it. You can use inline VizTracer, which will always work.
Or you can specify modules to ``unittest.main()``, which is not a general solution but could work without too much code changes.
Loading

0 comments on commit 7dda50f

Please sign in to comment.