In the direction of Native Profiling for Python

Joannah Nanjekye got here to the Python Language Summit 2023 to debate improvements by Scalene, a sampling-based Python profiler that may distinguish between native code and Python code in its reviews. After its preliminary launch in late 2019, Scalene has develop into some of the widespread Python profiling instruments. It has now been downloaded 500,000 instances from PyPI.

A profiler is a device that may monitor a program as it’s operating. As soon as this system has run, the profiler can present a report analysing which traces of code have been visited most frequently, which have been the most costly when it comes to time spent, and which have been the most costly when it comes to reminiscence utilization. Profilers can subsequently be vastly helpful instruments for addressing efficiency points in code. In case you’re not sure the place your program is spending most of its time, it may be laborious to optimise it.

Profilers will be break up into two broad classes: trace-based profilers and sampling-based profilers. Hint-based profilers work by intercepting every perform name as your program is operating and logging details about the time spent, reminiscence utilization, and many others. Sampling-based profilers, in the meantime, take snapshots of your program at periodic intervals to watch this stuff. A trace-based profiler has the benefit that it may well present a granular and exact stage of element about which traces of code have been executed and when every perform name finishes; this makes it excellent to be used as a device to watch check protection, for instance. Nonetheless, injecting tracing hooks into every perform name can typically decelerate a program and deform the evaluation of the place most time was spent. Because of this, sampling-based profilers are typically most well-liked for profiling efficiency.

Scalene is a sampling-based profiler, and goals to deal with the shortcomings of earlier sampling-based profilers for Python. One of many key challenges sampling-based profilers have confronted previously has been precisely measuring the time Python applications spend in “native code”.

Slide from Nanjekye’s speak, illustrating sampling-based profiling

Dealing with the issue of native code

“Native code”, additionally typically known as “machine code”, refers to code consisting of low-level directions that may be interpreted straight by the {hardware} processor. Utilizing extensions to Python written in C, C++ or Rust that can compile to native code – reminiscent of NumPy, scikit-learn, and TensorFlow – can result in dramatic speedups for a program written in Python.

It additionally, nevertheless, makes life tough for sampling-based profilers. Samplers typically use Python’s sign module as a method of understanding when to take a periodic snapshot of a program as it’s operating. Nonetheless, because of the method the sign module works, no signalling occasions shall be delivered whereas a Python program is spending time in a perform that has been compiled to native code by way of an extension module. The upshot of that is that sample-based profilers are sometimes “flying blind” for Python code that makes in depth use of C extensions, and can typically erroneously report that no time in any respect was spent executing native code, even when this system the truth is spent the vast majority of its time there.

Scalene’s answer to this downside is to watch delays in sign supply. It makes use of this info to infer the period of time that this system spent exterior CPython’s foremost interpreter loop (as a result of using native, compiled code from an extension module). Additional particulars on Scalene’s strategies, and comparisons with different main Python profilers, will be present in a current paper by Emery D. Berger, Sam Stern and Juan Altmayer Pizzorno, “Triangulating Python Efficiency Points with Scalene”.

Nanjekye additionally detailed Scalene’s refined method to measuring efficiency in little one threads. Sign-based profilers typically wrestle with multi-threaded code, as indicators can solely be delivered and acquired from the primary thread in Python. Scalene’s answer is to monkey-patch capabilities which may block the primary thread, and add timeouts to those capabilities. This enables indicators to be delivered even in multithreaded code.

Dialogue

Nanjekye requested attendees on the Language Summit if they might be occupied with integrating Scalene’s concepts into the usual library’s cProfile module, which was met with a considerably muted response.

Pablo Galindo Salgado, a number one contributor to the Memray profiler, criticised Scalene’s signal-based approach, arguing it relied on inherently brittle monkey-patching of the usual library. It additionally reported unreliable timings, Salgado stated: for instance, if code in a C extension checks for indicators to assist CTRL-C, the ensuing delays measured by Scalene shall be distorted.

Salgado argued that integration with the perf profiler, which Python is introducing assist for in Python 3.12, can be a greater possibility for customers. Mark Shannon, nevertheless, argued that perf distorted the execution time of Python applications; Salgado responded that Scalene did as nicely, as using indicators got here with its personal overhead.

Nanjekye argued that the massive reputation of Scalene within the Python ecosystem was proof that it had proved its value. Carol Keen concurred, noting that Scalene was an particularly useful gizmo with code that made heavy use of libraries reminiscent of NumPy, Scikit-Be taught and PyTorch.