Compiler Benchmark cbench

What is it?

This Compiler Benchmark, cbench for short, was implemented to mesuare the effect of using different compilers, compile time flags and libraries on the runtime on a set of synthetic micro benchmarks of c/c++ code. The set of small benchmarks is easily extened.

cbench was implemented to see which stl library is faster on a given machine: gcc's libstdc++ or STLPort (short the answer: it depends...).

Where do I find the latest version?

Look at http://cbench.sourceforge.net

Licensing

How does it work

cbench is a bit of c/c++ code (main.cpp), a script to compile this source with different sets of compiler, flags and libs (run_all) and a script to parse the runtime output of the executables, to judge the performance (report.py). In main.cpp you could see a set of about 50 small benchmarks.

Each small benchmark constists of a function, that is performing some tests, like appending one char to a std::string until this string is 100.000 chars long. Most times this small jobs are repeated a few time in a loop.

This functions are called until a given period of time is over, like 10 seconds or a minute. The time measured is user and system time of this process (with the given tests it is mostly user space time). Within this period of time it is counted how often the function could be called. Higher counts correspond to better performance (since more work is done in given time).

the script run_all compiles the c/c++ source with different compilers, flags and libararies. I distribute two versions: - run_all, which should work out of the box on every unix system, with a g++ compiler (report it as bug, if you have trouble). It compiles the testcode with the normal installed g++ and uses different sets of compile time flags. - run_all_complex, with probably will - in this form - only work on my machine, but you could easily adjust it. It uses different versions of g++, flags and libraries.

to compare performance, try this:

./run_all | tee log.raw
./report.py <log.raw >log.graph

There are example files of log.raw and log.graph int the directory named examples. These results where gatherd on a Pentium II 2x350(SMP) Mhz Machine, runing linux 2.4.17. If you generate results, I would like to see them, so I could compare them to my results and see if in what ways they vary.

Any comments, bug reports or further micro benchmarks are very welcome, mail me j.beyer@web.de

Joerg Beyer