1. Building Python for Performance using Sun Studio - Miriam Blatt Nov 2008

Python is a widely used interpreted scripting language, for which the interpreter is
written in C and C++. Python performance depends on building the interpreter with
the right flags. Performance studies on UltraSPARC IV+, Woodcrest and Niagara 2
show significant gains when building python with three key flags:

1. Optimization level -xO4. This is one level higher than -O (= -xO3). With -xO4,
   additional optimizations are done, of which the key one for python is inlining.
2. Enable optimization across source files using -xipo=2. This allows inlining
   to be done across files, providing more opportunities for it to help.
3. Build using profile feedback: a profile showing runtime behavior of the python
   interpreter branches, created by running a sample python program. The build
   procedure below uses the well known parrotbench as the sample program. Parrotbench
   was written to exercise all corners of the language, including funky descriptors,
   recursion, and mutable classes. The feedback profile provides data that tells
   the compiler which parts of the code will benefit the most from inlining.

Runtime comparisons were done using as base the parrotbench runtime for python compiled
with -O. The compiler used was an early version of the November 2008
Sun Studio Express

Combining -xO4, -xipo=2 and profile feedback led to 29% faster runtime on UltraSPARC IV+,
25% faster on Intel Xeon 5160, and 30% faster on UltraSPARC-T2. Performance gains will vary for
other python applications, but all are expected to benefit.

Here is a breakdown showing the benefit from adding each flag separately
for the three different platforms tested:

Building with feedback requires an extra step to gather the profile. The procedure
is shown below. Given the significant performance improvement, it is worth the
extra build time for any application that spends a significant amount of time
running python scripts.

To use -xipo=2 with Sun Studio 12 on Python, one also needs the flag
-xipo_archive=writeback, because the Python build procedure includes the
use of an archive. This flag is included in the build procedure described
below.

2. Build procedure using -xO4, feedback and xipo

2.1 Build binary to collect profile data

Remove old profile, and create binary using
flag -xprofile=collect:/tmp/python

gmake distclean
rm -rf /tmp/python.profile
configure \
--prefix=/directory/to/install/output \
CC=/studio/compiler/bin/cc \
CXX=/studio/compiler/bin/CC \
OPT="-g -xO4 -xipo=2 -xipo_archive=writeback -xprofile=collect:/tmp/python" \
LDFLAGS="-g -xO4 -xipo=2 -xipo_archive=writeback -xprofile=collect:/tmp/python"
gmake

2.2 Run parrotbench to collect the profile.

Use the python binary from the first step.

• Download and untar and un-gzip the
  parrot benchmark
• Change line 10 of b.py to adjust number of iterations.
  Original: for i in range(2):
  Suggested Change: for i in range(20):
• Run parrot using python binary built in step 2.1
  cd /directory/containing/parrot
  /python/binary/location/python -O b.py

2.3 Build again, using the parrot profile that was collected.

Use flag -xprofile=use:/tmp/python to provide the profile data to the compiler.

gmake distclean
configure \
--prefix=/directory/to/install/output \
CC=/studio/compiler/bin/cc \
CXX=/studio/compiler/bin/CC \
OPT="-g -xO4 -xipo=2 -xipo_archive=writeback -xprofile=use:/tmp/python" \
LDFLAGS="-g -xO4 -xipo=2 -xipo_archive=writeback -xprofile=use:/tmp/python"
gmake