Recommended Flags
The latest Sun Studio compiler now features greatly enhanced profile feedback capabilities with the flag -xprofile. In particular, for applications such as MySQL that link in shared libraries, when compiling with -xprofile there now is no need to selectively remove the -xprofile flag for all modules in the build tree that are contained in shared libraries (the reason for this is to avoid missing symbol errors when linking). No new compiler flags or additional effort is required apart from adding a patch for Solaris 10 (x86, SPARC, requires login) if necessary, or installing OpenSolaris 2009.06 or newer. The easiest and safest way to update an older version of OpenSolaris to the latest release is with "#pkg image-update".)
When using the latest Sun Studio compiler on an OS that supports the enhanced -xprofile it becomes much easy to create a build of MySQL that has the performance gains shown in the tables below. Rather than using the script I wrote for older releases, the simplified procedure for building a highly optimized MySQL is as follows:
export CC=cc export CXX=CC export CFLAGS="-fast -xipo=2 -xprofile=collect:/tmp/mysql_prof" export CXXFLAGS=$CFLAGS ./configure <args> dmake clean dmake dmake install <start up mysql and run training workload> <shut down mysql gracefully, e.g. with "mysqladmin shutdown"> dmake clean export CFLAGS="-fast -xipo=2 -xprofile=use:/tmp/mysql_prof" export CXXFLAGS=$CFLAGS ./configure <args> dmake dmake install
If your system does not have the required Solaris and Sun Studio releases, then the following optimization flags will provide decent performance and a fast build time:
-fast
Performance vs. Compiler Flags
| |
UltraSPARC T1 | UltraSPARC T1 | UltraSPARC IV+ | UltraSPARC IV+ |
|---|---|---|---|---|
| Flags | Sysbench Read-Only Performance | Sysbench Read+Write Performance | Sysbench Read-Only Performance | Sysbench Read+Write Performance |
| -O3 | 100% | 100% | 100% | 100% |
| -fast | 165% | 141% | 106% | 104% |
| -fast -xipo -xprofile | 368% | 242% | 122% | 122% |
Notes:
- The training workloads for the profile feedback builds (enabled with the compiler flag -xprofile) were the 708 tests launched by gmake test and the script train.sql included in the tar file that can be downloaded from the Build Scripts section of this document.
- Full compiler flags strings:
- O3: -g/g0 -O3 -xtarget=generic
- Fast: -g/g0 -fast -xtarget=generic
- Fast + xipo + xprofile: -g/g0 -fast -xipo=1 -xprofile=<collect|use>:<file> -xtarget=generic
Scaling
On our 1.5 GHz 4 CPU (two chip) V490 we see the following scaling on a custom Sysbench OLTP workload:
Compiler Optimization vs. Safety
All optimization levels (including -xipo=2 -xprofile -fast) have passed the MySQL test suite that is invoked with gmake test. Because the charsets ucs2, utf8 and cp932 are not supported on our test system, the following tests were disabled:
- ctype_ucs_binlog.test
- ctype_ucs.test
- ctype_ucs2_def-master.opt
- ctype_ucs2_def.test
- innodb.test
- ndb_charset.test
- mysqlbinlog.test
- mysql.test
- sp.test
All remaining tests pass using every tested optimization level. Analysis of the feedbin file generated by -xprofile after running the test suite and our custom Sysbench benchmark on an instrumented build shows that 35% of all functions in the source code tree are exercised.
Configure Invocation
export CC=cc export CXX=CC export CFLAGS="-g -fast" export CXXFLAGS=$CFLAGS ./configure --prefix=<dir> --with-apxs2=<apache path>/bin/apxs --enable-fastcgi --enable-discard-path \ --enable-force-redirect --enable-libxml --with-iconv=/usr/local --with-gd --with-zlib \ --enable-ftp --with-pdflib
Build Notes
- Use the latest Studio compiler for optimal performance and the ability to compile with -xipo and -xprofile
- The file innobase/include/univ.i has a directive that inhibits inlining on non-GCC or non-Windows systems at lower optimization levels. In some cases a speedup of over 10% can be attained by changing the line:
#if !defined(__GNUC__) && !defined(__WIN__)
to
#if !defined(__GNUC__) && !defined(__WIN__) && !defined(__sun)
Known Compile Issues
- MySQL is linked with dynamic libraries which aren't capable of optimization with -xprofile with older Sun Studio compilers. Hence, the automatic build script "build_train.sh" included in the tar archive below selectively disables -xprofile for the modules linked into ".so" dynamic libraries.
- Because the make process creates libraries of object files, if -xipo is used then the build must be done serially (e.g. do not type gmake -j 32) and -WO,-no_dependency_variables -xipo_archive=writeback may be required for Studio compilers.
- Much older Studio compilers can not compile MySQL with -xipo, so now would be a good time to download the free latest version
Build Scripts
If your system does not support the enhanced -xprofile functionality described at the top of this document, the script build_train.sh included in mysql_files.tar will build and train MySQL with feedback directed optimization by using the -xprofile compiler flag. Instructions:
- Save the file mysql_files.tar to the MySQL source code root directory (where "configure" was run)
- tar -xvf mysql_files.tar
- ./build_train.sh
Comments (4)
Mar 23, 2008
sunsrao says:
Hi KarstenG I followed your instructions on SunOS 5.9 (Generic_117171-02 sun4u ...Hi KarstenG
I followed your instructions on SunOS 5.9 (Generic_117171-02 sun4u sparc SUNW,Sun-Fire-V240System = SunOS)
The build fails at sql_views.cc and the error I get is
make[2]: Entering directory `/develop1/srao/packages/mysql-5.0.56/sql'
(charset_info_st*,char*,unsigned,char*,unsigned) to unsigned
(charset_info_st*,char*,unsigned,char*,unsigned).
(charset_info_st*,char*,unsigned,char*,unsigned) to unsigned
(charset_info_st*,char*,unsigned,char*,unsigned).
source='sql_view.cc' object='sql_view.o' libtool=no \
DEPDIR=.deps depmode=none /bin/bash ../depcomp \
CC -DMYSQL_SERVER -DDEFAULT_MYSQL_HOME="\"/develop1/srao/packages/mysql\"" -DDATADIR="\"/opt/mysql/data\"" -DSHAREDIR="\"/develop1/srao/packages/mysql/share/mysql\"" -DHAVE_CONFIG_H -I. -I. -I../include -I../zlib -I../innobase/include -I../innobase/include -I../include -I../include -I../regex -I. -DDBUG_OFF -g -fast -xO3 -noex -mt -fsimple=1 -ftrap=%none -xbuiltin=%all -xlibmil -xlibmopt -xarch=v9 -D_FILE_OFFSET_BITS=64 -DHAVE_RWLOCK_T -c -o sql_view.o sql_view.cc
"item_strfunc.h", line 165: Warning (Anachronism): Assigning extern "C" unsigned
"item_strfunc.h", line 179: Warning (Anachronism): Assigning extern "C" unsigned
2 Warning(s) detected.
cg error (as) : location counter not on doubleword boundary
make[2]: *** [sql_view.o] Error 1
make[2]: Leaving directory `/develop1/srao/packages/mysql-5.0.56/sql'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/develop1/srao/packages/mysql-5.0.56/sql'
make: *** [all] Error 2
Couldn't figure out what was that.
Appreciate any help
cheers
sunsrao
Mar 24, 2008
dgove says:
Hi, Can you check that you're using the latest compiler (free from http://devel...Hi,
Can you check that you're using the latest compiler (free from http://developers.sun.com/sunstudio/), and have applied all the latest patches?
We're currently unable to reproduce this error.
Thanks,
Darryl.
Feel free to contact me directly (firstname . lastname at sun . com)
Mar 25, 2008
sunsrao says:
Thanks Darryl. Didn't do as you said (will do it for sure), but wanted to updat...Thanks Darryl.
Didn't do as you said (will do it for sure), but wanted to update you that removing the -xarch=v9 option solved the problem, with the warning that it may not work on pre-UltraSparc III machines (don't remember exact words), which was fine with me.
cheers
sunsrao
Mar 25, 2008
djgove says:
That gives you a 32-bit binary rather than a v9 binary. But if that's sufficient...That gives you a 32-bit binary rather than a v9 binary. But if that's sufficient for your needs, that's great. I'd still like more info about the build error though. cc -V should give you compiler version info.
Thanks,
Darryl.