Python debugging tools

This is an overview of the tools and practices I’ve used for debugging or profiling purposes. This is not necessarily complete, there are so many tools so I’m listing only what I think is best or relevant. If you know better tools or have other preferences, please comment below.

Logging

Yes, really. Can’t stress enough how important it is to have adequate logging in your application. You should log important stuff. If your logging is good enough, you can figure out the problem just from the logs. Lots of time saved right there.

If you do ever litter your code with print statements stop now. Use logging.debug instead. You’ll be able to reuse that later, disable it altogether and so on …

Tracing

Sometimes it’s better to see what gets executed. You could run step-by-step using some IDE’s debugger but you would need to know what you’re looking for, otherwise the process will be very slow.

In the stdlib there’s a trace module which can print all the executed lines amongst other this (like making coverage reports )

python -mtrace --tracescript.py

This will make lots of output (every line executed will be printed so you might want to pipe it through grep to only see the interesting modules). Eg:

python -mtrace --tracescript.py <spanclass="p">|</span> egrep <spanclass="s1">'^(mod1.py|mod2.py)'</span> Alternatives

Grepping for relevant output is not fun. Plus, the trace module doesn’t show you any variables.

Hunter is a flexible alternative that allows filtering and even shows variables of your choosing. Just pip install hunter and run:

PYTHON_HUNTER="F(module='mod1'),F(module='mod2')" pythonscript.py

Take a look at the project page for more examples.

If you’re feeling adventurous then you could try smiley it shows you the variables and you can use it to trace programs remotely.

Alternativelly, if you want very selective tracing you can use aspectlib.debug.log to make existing or 3rd party code emit traces.

PDB

Very basic intro, everyone should know this by now:

<spanclass="kn">import</span> <spanclass="nn">pdb</span> <spanclass="n">pdb</span><spanclass="o">.</span><spanclass="n">set_trace</span><spanclass="p">()</span> <spanclass="c1"># opens up pdb prompt</span>

Or:

<spanclass="k">try</span><spanclass="p">:</span> <spanclass="n">code</span> <spanclass="n">that</span> <spanclass="n">fails</span> <spanclass="k">except</span><spanclass="p">:</span> <spanclass="kn">import</span> <spanclass="nn">pdb</span> <spanclass="n">pdb</span><spanclass="o">.</span><spanclass="n">pm</span><spanclass="p">()</span> <spanclass="c1"># or pdb.post_mortem()</span>

Or (press c to start the script):

python -mpdbscript.py

Once in the REPL do:

c or continue q or quit l or list , shows source at the current frame w or where , shows the traceback d or down , goes down 1 frame on the traceback u or up , goes up 1 frame on the traceback <enter> , repeats last command ! <stuff> , evaluates <stuff> as python code on the current frame everything else , evaluates as python code if it’s not a PDB command Better PDB

Drop in replacements for pdb :

ipdb ( pip install ipdb ) like ipython (autocomplete, colors etc). pudb ( pip install pudb ) curses based (gui-like), good at browsing sourcecode. pdb++ ( pip install pdbpp ) autocomplete, colors, extra commands etc. Remote PDB

sudo apt-get install winpdb

Instead of pdb.set_trace() do:

<spanclass="kn">import</span> <spanclass="nn">rpdb2</span> <spanclass="n">rpdb2</span><spanclass="o">.</span><spanclass="n">start_embedded_debugger</span><spanclass="p">(</span><spanclass="s2">"secretpassword"</span><spanclass="p">)</span>

Now run winpdb and go to File > Attach with the password .

Don’t like Winpdb? Use PDB over TCP

Get remote-pdb and then, to open a remote PDB on first available port, use:

<spanclass="kn">from</span> <spanclass="nn">remote_pdb</span> <spanclass="kn">import</span> <spanclass="n">set_trace</span> <spanclass="n">set_trace</span><spanclass="p">()</span> <spanclass="c1"># you'll see the port number in the logs</span>

To use some specific host/port:

<spanclass="kn">from</span> <spanclass="nn">remote_pdb</span> <spanclass="kn">import</span> <spanclass="n">RemotePdb</span> <spanclass="n">RemotePdb</span><spanclass="p">(</span><spanclass="n">host</span><spanclass="o">=</span><spanclass="s1">'0.0.0.0'</span><spanclass="p">,</span> <spanclass="n">port</span><spanclass="o">=</span><spanclass="mi">4444</span><spanclass="p">)</span><spanclass="o">.</span><spanclass="n">set_trace</span><spanclass="p">()</span>

To connect just run something like telnet 192.168.12.34 4444 . Alternatively, run socat socat readline tcp:192.168.12.34:4444 to get line editing and history.

Just a REPL If you don’t need a full blown debugger then just start a