2001-01-31 05:14:08 +08:00
|
|
|
|
<html>
|
|
|
|
|
|
|
|
|
|
|
|
<head>
|
2003-01-09 20:35:55 +08:00
|
|
|
|
<meta http-equiv="Content-Type"
|
|
|
|
|
|
content="text/html; charset=iso-8859-1">
|
2003-01-21 05:06:58 +08:00
|
|
|
|
<meta name="GENERATOR" content="Microsoft FrontPage 5.0">
|
2003-01-08 00:58:32 +08:00
|
|
|
|
<title>Boost Regression Test User Documentation</title>
|
2001-01-31 05:14:08 +08:00
|
|
|
|
</head>
|
|
|
|
|
|
|
|
|
|
|
|
<body bgcolor="#FFFFFF" text="#000000">
|
|
|
|
|
|
|
2003-01-09 20:35:55 +08:00
|
|
|
|
<table border="1" cellpadding="2" bgcolor="#007F7F">
|
|
|
|
|
|
<tr>
|
2004-10-05 23:23:19 +08:00
|
|
|
|
<td bgcolor="#FFFFFF"><img src="../boost.png"
|
|
|
|
|
|
alt="boost.png (6897 bytes)" width="277" height="86"></td>
|
2003-01-09 20:35:55 +08:00
|
|
|
|
<td><a href="../index.htm"><font color="#FFFFFF" size="4"
|
|
|
|
|
|
face="Arial,Helvetica">Home</font></a></td>
|
|
|
|
|
|
<td><a href="../libs/libraries.htm"><font color="#FFFFFF"
|
|
|
|
|
|
size="4" face="Arial,Helvetica">Libraries</font></a></td>
|
|
|
|
|
|
<td><a href="../people/people.htm"><font color="#FFFFFF"
|
|
|
|
|
|
size="4" face="Arial,Helvetica">People</font></a></td>
|
|
|
|
|
|
<td><a href="faq.htm"><font color="#FFFFFF" size="4"
|
|
|
|
|
|
face="Arial,Helvetica">FAQ</font></a></td>
|
|
|
|
|
|
<td><a href="index.htm"><font color="#FFFFFF" size="4"
|
|
|
|
|
|
face="Arial,Helvetica">More</font></a></td>
|
|
|
|
|
|
</tr>
|
2001-01-31 05:14:08 +08:00
|
|
|
|
</table>
|
|
|
|
|
|
|
2003-01-08 00:58:32 +08:00
|
|
|
|
<h1>Boost Regression Test User Documentation</h1>
|
2001-01-31 05:14:08 +08:00
|
|
|
|
|
2003-01-08 00:58:32 +08:00
|
|
|
|
<p><a href="#Introduction">Introduction</a><br>
|
|
|
|
|
|
<a href="#Preparation">Preparation</a><br>
|
|
|
|
|
|
<a href="#Execution">Execution</a><br>
|
|
|
|
|
|
<a href="#Adding_new_test">Adding a new test</a><br>
|
|
|
|
|
|
<a href="#Requirements">Requirements</a><br>
|
|
|
|
|
|
<a href="#History">History</a></p>
|
2001-01-31 05:14:08 +08:00
|
|
|
|
|
2003-01-08 00:58:32 +08:00
|
|
|
|
<h2><a name="Introduction">Introduction</a></h2>
|
2003-01-09 20:35:55 +08:00
|
|
|
|
|
|
|
|
|
|
<p>Boost's internal regression test suite produces the <a
|
|
|
|
|
|
href="../status/compiler_status.html">compiler status tables</a>.
|
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
|
|
<p>Although not ordinarily run by Boost library users, it is
|
|
|
|
|
|
documented here for the benefit of Boost developers, and for
|
|
|
|
|
|
Boost users porting to a new platform.</p>
|
|
|
|
|
|
|
|
|
|
|
|
<p>Version 3 of the Boost regression testing framework is based
|
|
|
|
|
|
on <a href="../tools/build/index.html">Boost.Build</a>, and uses <i>bjam</i>
|
|
|
|
|
|
to actually run the tests. Because Boost.Build does dependency
|
|
|
|
|
|
analysis, only tests for which some dependency has change are
|
|
|
|
|
|
rerun.</p>
|
|
|
|
|
|
|
|
|
|
|
|
<p>The reporting of test results as HTML files is accomplished by
|
|
|
|
|
|
separate <a href="../tools/regression/index.htm">regression
|
|
|
|
|
|
reporting programs</a> which process the residue and log files
|
|
|
|
|
|
from the <i>bjam</i> run.</p>
|
|
|
|
|
|
|
2003-01-08 00:58:32 +08:00
|
|
|
|
<h2><a name="Preparation">Preparation</a></h2>
|
2003-01-09 20:35:55 +08:00
|
|
|
|
|
|
|
|
|
|
<p>Install the following programs on your system, in some
|
|
|
|
|
|
location suitable for program executables. Normally that location
|
|
|
|
|
|
must be in a directory which is part of your search path for
|
|
|
|
|
|
executables.</p>
|
|
|
|
|
|
|
2002-08-21 04:16:34 +08:00
|
|
|
|
<ul>
|
2003-01-09 20:35:55 +08:00
|
|
|
|
<li><i>bjam</i> - Executables and sources are available; see <a
|
|
|
|
|
|
href="../tools/build/index.html#Jam">Boost.Build docs</a>.
|
|
|
|
|
|
Before worrying about regression tests, you might want to
|
|
|
|
|
|
verify your <i>bjam</i> installation works by <a
|
|
|
|
|
|
href="../tools/build/index.html#Building">building the
|
|
|
|
|
|
boost-root/status/Jamfile libraries</a>.<br>
|
|
|
|
|
|
</li>
|
2003-09-04 07:51:04 +08:00
|
|
|
|
<li><i>process_jam_log</i> and <i>compiler_status</i> -
|
|
|
|
|
|
<a href="../tools/regression/index.htm">Sources and Jamfile</a> are available
|
2003-01-09 20:35:55 +08:00
|
|
|
|
for these, too. </li>
|
2002-08-21 04:16:34 +08:00
|
|
|
|
</ul>
|
2003-01-09 20:35:55 +08:00
|
|
|
|
|
2003-01-08 00:58:32 +08:00
|
|
|
|
<h2><a name="Execution">Execution</a></h2>
|
2003-01-09 20:35:55 +08:00
|
|
|
|
|
2003-09-04 07:51:04 +08:00
|
|
|
|
<p>A complete shell script, <a href="../tools/regression/run_tests.sh">
|
|
|
|
|
|
run_test.sh</a>, is available for POSIX systems. For Windows, please read the
|
|
|
|
|
|
following.</p>
|
|
|
|
|
|
|
2003-01-09 20:35:55 +08:00
|
|
|
|
<p>These examples assume several environment variables have been
|
|
|
|
|
|
set:</p>
|
|
|
|
|
|
|
2002-08-21 04:16:34 +08:00
|
|
|
|
<ul>
|
2003-01-09 20:35:55 +08:00
|
|
|
|
<li>BOOST_ROOT is set to the directory where the unzipped
|
|
|
|
|
|
Boost distribution or CVS working copy resides. For
|
|
|
|
|
|
example:<br>
|
|
|
|
|
|
<br>
|
|
|
|
|
|
set BOOST_ROOT=/boost_1_29_0<br>
|
|
|
|
|
|
</li>
|
|
|
|
|
|
<li>TOOLS is set to the <a
|
|
|
|
|
|
href="../tools/build/index.html#Tools">compiler toolsets</a>
|
|
|
|
|
|
you wish to use. For example:<br>
|
|
|
|
|
|
<br>
|
|
|
|
|
|
set TOOLS=borland gcc intel-win32 metrowerks vc7<br>
|
|
|
|
|
|
</li>
|
|
|
|
|
|
<li>Any environment variables required for particular
|
|
|
|
|
|
compilers. You might wish to test first with one
|
|
|
|
|
|
compiler at a time to make sure each toolset is fully
|
|
|
|
|
|
operational.</li>
|
2002-08-21 04:16:34 +08:00
|
|
|
|
</ul>
|
2003-01-09 20:35:55 +08:00
|
|
|
|
|
2003-01-21 05:06:58 +08:00
|
|
|
|
<p>Just running bjam for a specific library can be run like this on Windows,
|
|
|
|
|
|
using the Filesystem Library as an example:</p>
|
|
|
|
|
|
|
|
|
|
|
|
<blockquote>
|
|
|
|
|
|
<pre>cd \boost-root\libs\filesystem\test
|
|
|
|
|
|
bjam</pre>
|
|
|
|
|
|
</blockquote>
|
|
|
|
|
|
|
|
|
|
|
|
<p>A full set of tests and status tables can be run thusly
|
|
|
|
|
|
on a Windows system:</p>
|
2003-01-09 20:35:55 +08:00
|
|
|
|
|
2002-08-21 04:16:34 +08:00
|
|
|
|
<blockquote>
|
2003-01-09 20:35:55 +08:00
|
|
|
|
<pre>cd %BOOST_ROOT%\status
|
2003-01-21 05:06:58 +08:00
|
|
|
|
bjam --dump-tests test >bjam.log 2>&1
|
|
|
|
|
|
start notepad bjam.log
|
|
|
|
|
|
process_jam_log <bjam.log
|
2002-08-21 04:16:34 +08:00
|
|
|
|
compiler_status %BOOST_ROOT% cs-win32.html
|
|
|
|
|
|
rem Specify links file, even though it will be overwritten, so report html includes links
|
|
|
|
|
|
rem This works because the generated bookmark names are the same regardless of other settings
|
|
|
|
|
|
compiler_status --ignore-pass --no-warn %BOOST_ROOT% cs-win32-fail.html cs-win32-links.html
|
|
|
|
|
|
compiler_status --ignore-pass %BOOST_ROOT% cs-win32-warn-or-fail.html cs-win32-links.html
|
|
|
|
|
|
compiler_status %BOOST_ROOT% cs-win32-full.html cs-win32-links.html</pre>
|
|
|
|
|
|
</blockquote>
|
2003-01-09 20:35:55 +08:00
|
|
|
|
|
|
|
|
|
|
<p>Modulo syntax adjustments, the same procedure should work on
|
2003-01-21 05:06:58 +08:00
|
|
|
|
other operating systems (an example a <a href="../tools/regression/run_tests.sh">
|
|
|
|
|
|
UNIX shell script</a> for this is provided, see the
|
2003-01-09 20:35:55 +08:00
|
|
|
|
script itself for comments and directions). Rename the cs-win-xxx
|
|
|
|
|
|
output files as appropriate.</p>
|
|
|
|
|
|
|
|
|
|
|
|
<p>If you execute <i>compiler_status</i> without arguments, you
|
|
|
|
|
|
can see the available options and tailor your own favorite report.</p>
|
|
|
|
|
|
|
|
|
|
|
|
<p>If you want to run just a single test, specify it as the <i>bjam</i>
|
|
|
|
|
|
target rather than "test". For example, to debug
|
|
|
|
|
|
configurations, it might be useful to just run the <i>config_info</i>
|
|
|
|
|
|
test, with a switch to force even up-to-date programs be rebuilt:</p>
|
|
|
|
|
|
|
2002-08-21 04:16:34 +08:00
|
|
|
|
<blockquote>
|
2003-01-21 05:06:58 +08:00
|
|
|
|
<pre>bjam -a --dump-tests config_info >bjam.log 2>&1
|
|
|
|
|
|
process_jam_log <bjam.log
|
2003-01-08 00:58:32 +08:00
|
|
|
|
compiler_status %BOOST_ROOT% cs-win32.html
|
|
|
|
|
|
</pre>
|
2002-08-21 04:16:34 +08:00
|
|
|
|
</blockquote>
|
2003-01-09 20:35:55 +08:00
|
|
|
|
|
2003-01-08 00:58:32 +08:00
|
|
|
|
<h2><a name="Adding_new_test">Adding a new test</a></h2>
|
2003-01-09 20:35:55 +08:00
|
|
|
|
|
2003-01-21 05:06:58 +08:00
|
|
|
|
<h3>The best way - the <a name="subinclude"> <i>subinclud</i>e</a> approach</h3>
|
2003-01-09 20:35:55 +08:00
|
|
|
|
|
2003-01-21 05:06:58 +08:00
|
|
|
|
<p>The best way to add a test to the <a
|
|
|
|
|
|
href="../status/Jamfile">boost-root/status/Jamfile</a> is usually to use the bjam
|
|
|
|
|
|
<i>subinclude</i> feature, since it allows the same test specification to be run
|
|
|
|
|
|
either as part of the overall Boost regression test or as a standalone test of
|
|
|
|
|
|
only the library involved.</p>
|
|
|
|
|
|
|
|
|
|
|
|
<p>The test is specified in a Jamfile, which is usually placed in the library's
|
|
|
|
|
|
<i>test</i> subdirectory. See the <a href="../libs/filesystem/test/Jamfile">
|
|
|
|
|
|
Filesystem Library's test Jamfile</a> for an example, including the boilerplate
|
|
|
|
|
|
wrapped around the actual tests. By <a href="#Execution">executing bjam</a>
|
|
|
|
|
|
(with or without the status reporting programs) in the library's <i>test</i>
|
|
|
|
|
|
subdirectory, the library can be tested without the need to run the full Boost
|
|
|
|
|
|
regression tests.</p>
|
|
|
|
|
|
|
|
|
|
|
|
<p>Then by adding a <i>subinclude</i> statement to <a
|
|
|
|
|
|
href="../status/Jamfile">boost-root/status/Jamfile</a>, the library's test
|
|
|
|
|
|
Jamfile will become part of the main Boost regression tests. For example,
|
|
|
|
|
|
the Filesystem Library's tests are included by this line in the main
|
|
|
|
|
|
boost-root/status Jamfile:</p>
|
2003-01-09 20:35:55 +08:00
|
|
|
|
|
2002-08-21 04:16:34 +08:00
|
|
|
|
<blockquote>
|
2003-01-21 05:06:58 +08:00
|
|
|
|
<pre>subinclude libs/filesystem/test ;</pre>
|
2002-08-21 04:16:34 +08:00
|
|
|
|
</blockquote>
|
2003-01-09 20:35:55 +08:00
|
|
|
|
|
2003-01-21 05:06:58 +08:00
|
|
|
|
<p>Don't forget that Jamfiles are white-space sensitive; delete the
|
2003-01-09 20:35:55 +08:00
|
|
|
|
space before the semi-colon in the example above, and you will
|
|
|
|
|
|
get an introduction to Jam error messages.</p>
|
|
|
|
|
|
|
2003-01-21 05:06:58 +08:00
|
|
|
|
<h3>Another way - direct inclusion in Boost regression tests</h3>
|
|
|
|
|
|
|
|
|
|
|
|
<p>Adding a test is as simple as adding a single line to the <a
|
|
|
|
|
|
href="../status/Jamfile">boost-root/status/Jamfile</a>:</p>
|
|
|
|
|
|
|
|
|
|
|
|
<blockquote>
|
|
|
|
|
|
<pre>run libs/mylib/test/mytest.cpp ;</pre>
|
|
|
|
|
|
</blockquote>
|
|
|
|
|
|
|
|
|
|
|
|
<p>For creating more complex tests and test-suites, use the
|
|
|
|
|
|
<a href="#subinclude">subinclude approach</a>. Look at
|
|
|
|
|
|
examples in the <a
|
|
|
|
|
|
href="../status/Jamfile">boost-root/status/Jamfile</a>.
|
2003-01-09 20:35:55 +08:00
|
|
|
|
The <i>bind</i> and <i>config</i> test-suites are simple
|
|
|
|
|
|
examples, while the <i>regex</i> and <i>threads</i> test suites
|
|
|
|
|
|
show more of the power of Jam based testing.</p>
|
|
|
|
|
|
|
2003-01-08 00:58:32 +08:00
|
|
|
|
<h2><a name="Requirements">Requirements</a></h2>
|
|
|
|
|
|
|
2003-01-09 20:35:55 +08:00
|
|
|
|
<p>The test suite has been designed to meet to the following
|
|
|
|
|
|
requirements. </p>
|
|
|
|
|
|
|
2003-01-08 00:58:32 +08:00
|
|
|
|
<ul>
|
2003-09-04 07:51:04 +08:00
|
|
|
|
<li>Doesn't depend on an external tool chain. This requirement
|
2003-01-09 20:35:55 +08:00
|
|
|
|
is met by supplying all tools in the regular <a
|
2003-12-07 02:21:30 +08:00
|
|
|
|
href="getting_started.html#Download">Boost distribution</a>, except for a
|
2003-01-09 20:35:55 +08:00
|
|
|
|
C++ compiler.</li>
|
|
|
|
|
|
<li>Tools are written in C++; it is the only language that
|
|
|
|
|
|
all Boost developers and users are comfortable with.</li>
|
|
|
|
|
|
<li>Supports tests which expect an error </li>
|
|
|
|
|
|
<li>Configuration is independent of the target platform or
|
|
|
|
|
|
compiler. </li>
|
|
|
|
|
|
<li>HTML output </li>
|
2003-01-08 00:58:32 +08:00
|
|
|
|
</ul>
|
|
|
|
|
|
|
2003-01-09 20:35:55 +08:00
|
|
|
|
<p>These requirements rule out any script-based approach such as
|
|
|
|
|
|
dejagnu (requires Tcl and expect) or even shell scripts. </p>
|
|
|
|
|
|
|
2003-01-08 00:58:32 +08:00
|
|
|
|
<h2><a name="History">History</a></h2>
|
2002-08-21 04:16:34 +08:00
|
|
|
|
|
2004-01-09 03:27:48 +08:00
|
|
|
|
<p>The version 3 Regression Test system is based on
|
|
|
|
|
|
<a href="../tools/build/index.html">Boost.Build</a>. The testing.jam and status/Jamfile foundation was
|
|
|
|
|
|
contributed by Dave Abrahams. Rene Rivera contributed several pieces. The post-bjam processing programs
|
2003-01-09 20:35:55 +08:00
|
|
|
|
were contributed by Beman Dawes.</p>
|
|
|
|
|
|
|
|
|
|
|
|
<p>The version 2 regression.cpp test program was contributed by
|
|
|
|
|
|
Jens Maurer, generalizing and improving an earlier version 1
|
|
|
|
|
|
program by Beman Dawes.</p>
|
2001-01-31 22:34:32 +08:00
|
|
|
|
|
2001-01-31 05:14:08 +08:00
|
|
|
|
<hr>
|
|
|
|
|
|
|
2003-01-09 20:35:55 +08:00
|
|
|
|
<p>Revised <!--webbot bot="Timestamp" startspan s-type="EDITED"
|
2004-01-09 03:27:48 +08:00
|
|
|
|
s-format="%d %B, %Y" -->08 January, 2004<!--webbot bot="Timestamp"
|
|
|
|
|
|
i-checksum="38708" endspan --></p>
|
2003-10-02 22:42:46 +08:00
|
|
|
|
|
|
|
|
|
|
<p><EFBFBD> Copyright <a href="../people/jens_maurer.htm">Jens
|
|
|
|
|
|
Maurer</a> 2001<br>
|
|
|
|
|
|
<EFBFBD> Copyright <a href="../people/beman_dawes.html">Beman Dawes</a> 2001</p>
|
2003-10-07 22:17:54 +08:00
|
|
|
|
<p> Use, modification, and distribution are subject to the Boost Software
|
2003-10-02 22:42:46 +08:00
|
|
|
|
License, Version 1.0. (See accompanying file <a href="../LICENSE_1_0.txt">
|
|
|
|
|
|
LICENSE_1_0.txt</a> or copy at <a href="http://www.boost.org/LICENSE_1_0.txt">
|
|
|
|
|
|
www.boost.org/LICENSE_1_0.txt</a>)</p>
|
2001-01-31 05:14:08 +08:00
|
|
|
|
</body>
|
2003-01-21 05:06:58 +08:00
|
|
|
|
</html>
|
