Project:Python/Byte compiling

This article is a work in progress; treat its contents with caution - MGorny (talk | contribs).

Levels of optimization

Python generally supports three levels of optimization:

non-optimized code (.pyc),
code without debug statements (.pyo, .opt-1.pyc),
code without debug and docstrings (.pyo, .opt-2.pyc).

The use of optimization on compiled byte-code depends on implementation:

all implementations use plain .pyc for non-optimized code,
PyPy2.7 does not write optimized code to byte-code files,
Python 2.7 uses .pyo for both levels of optimization, i.e. you can choose only one level and it'll stay,
Python 3.5+ and PyPy3.5+ use .opt-1.pyc and .opt-2.pyc for the optimized code.

How to byte-compile Python modules?

Generally, byte-compiling is done via importing the module with appropriate Python optimization flags. Python provides a convenience compileall module to do that for all modules in a directory.

The reference code to perform complete byte-compilation is:

CODE Byte-compiling all modules for all optimization levels

# all
"${PYTHON}" -m compileall -q -f -d "${sitedir}" "${D}${sitedir}"
# python2.7+ & pypy3.5+
"${PYTHON}" -OO -m compileall -q -f -d "${sitedir}" "${D}${sitedir}"
# python3.5+ & pypy3.5+
"${PYTHON}" -O -m compileall -q -f -d "${sitedir}" "${D}${sitedir}"

Note that -d should specify the 'live system' path to the compiled directory, while the position argument takes current (i.e. DESTDIR) path to it.

The first (non-optimized) variant applies to all Python implementations. -OO is done on all but PyPy2.7, -O is done on implementations using separate suffixes for optimization levels (Python3.5+ / PyPy3.5+). Generally, it should be fine to call more variants than necessary but it might be a waste of CPU cycles.

Byte-compilation in build systems

Automake

Automake supports byte-compiling Python modules when installed via PYTHON primary, e.g.:

FILE Makefile.amInstalling a Python module via automake

pyexec_PYTHON = example.py

Note that automake does not byte-compile Python modules for Python 3 correctly: bug #38043 upstream. This is fixed in Gentoo since automake-1.16.1-r2. You need to reconfigure upstream modules to fix that:

FILE foo-1.ebuildFixing py-compile

inherit autotools

DEPEND=">=sys-devel/automake-1.16.1-r2"

src_prepare() {
  default
  eautomake
}