Commit Graph

350 Commits

Author SHA1 Message Date
Victor Costan
95a1435b81 Fix compilation error with GCC on OSX (issue #490). (#491) 2017-11-30 08:05:38 -08:00
Roman Lebedev
ec5684ed75 Console reporter: properly account for the lenght of custom counter names (#484)
Old output example:
```
Benchmark                                                 Time           CPU Iterations  CPUTime,s   Pixels/s ThreadingFactor
------------------------------------------------------------------------------------------------------------------------------
20170525_0036TEST.RAF/threads:8/real_time                45 ms         45 ms         16   0.718738 79.6277M/s   0.999978   2.41419GB/s    22.2613 items/s FileSize,MB=111.050781; MPix=57.231360
```

New output example:
```
Benchmark                                                 Time           CPU Iterations  CPUTime,s   Pixels/s ThreadingFactor
------------------------------------------------------------------------------------------------------------------------------
20170525_0036TEST.RAF/threads:8/real_time                45 ms         45 ms         16   0.713575 80.1713M/s        0.999571   2.43067GB/s    22.4133 items/s FileSize,MB=111.050781; MPix=57.231360
```
2017-11-27 09:01:01 -08:00
Eric Fiselier
2ec7399cf1 Improve BENCHMARK_UNREACHABLE() implementation.
This patch primarily changes the BENCHMARK_UNREACHABLE()
implementation under MSVC to use __assume(false) instead
of being a NORETURN function, which ironically caused
unreachable code warnings.

Second, since the NOTHROW function attempt generated the
warnings we meant to avoid, it has been replaced with a dummy
null statement.
2017-11-26 13:58:24 -07:00
Eric
11dc36822b
Improve CPU Cache info reporting -- Add Windows support. (#486)
* Improve CPU Cache info reporting -- Add Windows support.

This patch does a couple of thing regarding CPU Cache reporting.

First, it adds an implementation on Windows. Second it fixes
the JSONReporter to correctly (and actually) output the CPU
configuration information.

And finally, third, it detects and reports the number of
physical CPU's that share the same cache.
2017-11-26 13:33:01 -07:00
Eric
27e0b439cf Refactor System information collection -- Add CPU Cache Info (#483)
* Refactor System information collection.

This patch refactors the system information collection,
and in particular information about the target CPU. The
motivation is to make it easier to access CPU information,
and easier to add new information as need be.

This patch additionally adds information about the cache
sizes of the CPU.

* Address review comments: Clean up integer types.

This commit cleans up the integer types used in ValueUnion to
follow the Google style guide.

Additionally it adds a BENCHMARK_UNREACHABLE macro to assist
in documenting/catching unreachable code paths.

* Rename ValueUnion accessors.
2017-11-22 08:33:52 -08:00
Kamil Rytarowski
aad6a5fa76 Add NetBSD support (#482)
Define BENCHMARK_OS_NETBSD for NetBSD.

Add detection of cpuinfo_cycles_per_second and cpuinfo_num_cpus.
This code shared detection of these properties with FreeBSD.
2017-11-17 08:46:08 -08:00
Steinar H. Gunderson
0c3ec998c4 Add a pkg-config file, for the benefit of projects not using CMake. (#480) 2017-11-15 11:51:22 -08:00
Eric
72a4581caf Fix #382 - MinGW often reports negative CPU times. (#475)
When stopping a timer, the current time is subtracted
from the start time. However, when the times are identical,
or sufficiently close together, the subtraction can result
in a negative number.

For some reason MinGW is the only platform where this problem
manifests. I suspect it's due to MinGW specific behavior in either
the CPU timing code, floating point model, or printf formatting.

Either way, the fix for MinGW should be correct across all platforms.
2017-11-07 09:44:39 -08:00
Dominic Hamon
f65c6d9a2c
Remove deprecated headers (#473) 2017-11-06 08:53:23 -08:00
Yangqing Jia
491360b833 Add option to install benchmark (#463)
* Add option to install benchmark

* Change to BENCHMARK_ENABLE_INSTALL per @dominichamon
2017-10-20 13:49:37 -07:00
Eric
a37fc0c48a Improve KeepRunning loop performance to be similar to the range-based for. (#460)
This patch improves the performance of the KeepRunning loop in two ways:

(A) it removes the dependency on the max_iterations variable, preventing
it from being loaded every iteration.

(B) it loops to zero, instead of to an upper bound. This allows a single
decrement instruction to be used instead of a arithmetic op followed by a
comparison.
2017-10-17 08:40:44 -07:00
Raúl Marín
cacd321808 Avoid implicit float to double conversion (#457)
Triggered by -Werror=double-promotion
2017-10-13 09:17:02 -07:00
Dominic Hamon
2409cb2eb1 Minor move of code to cleanup up namespace spaghetti a bit 2017-10-09 12:01:30 -07:00
Roman Lebedev
a271c36af9 Drop Stat1, refactor statistics to be user-providable, add median. (#428)
* Drop Stat1, refactor statistics to be user-providable, add median.

My main goal was to add median statistic. Since Stat1
calculated the stats incrementally, and did not store
the values themselves, it is was not possible. Thus,
i have replaced Stat1 with simple std::vector<double>,
containing all the values.

Then, i have refactored current mean/stdev to be a
function that is provided with values vector, and
returns the statistic. While there, it seemed to make
sense to deduplicate the code by storing all the
statistics functions in a map, and then simply iterate
over it. And the interface to add new statistics is
intentionally exposed, so they may be added easily.

The notable change is that Iterations are no longer
displayed as 0 for stdev. Is could be changed, but
i'm not sure how to nicely fit that into the API.

Similarly, this dance about sometimes (for some fields,
for some statistics) dividing by run.iterations, and
then multiplying the calculated stastic back is also
dropped, and if you do the math, i fail to see why
it was needed there in the first place.

Since that was the only use of stat.h, it is removed.

* complexity.h: attempt to fix MSVC build

* Update README.md

* Store statistics to compute in a vector, ensures ordering.

* Add a bit more tests for repetitions.

* Partially address review notes.

* Fix gcc build: drop extra ';'

clang, why didn't you warn me?

* Address review comments.

* double() -> 0.0
* early return
2017-08-23 16:44:29 -07:00
Dominic Hamon
d70417994a Allow the definition of 1k to be flexible. (#438)
When generating a human-readable number for user counters, we don't
generally expect 1k to be 1024. This is the default due to the more
general purpose string utility.

Fixes #437
2017-08-21 16:05:24 -07:00
Roman Lebedev
b9be142d1e Json reporter: don't cast floating-point to int; adjust tooling (#426)
* Json reporter: passthrough fp, don't cast it to int; adjust tooling

Json output format is generally meant for further processing
using some automated tools. Thus, it makes sense not to
intentionally limit the precision of the values contained
in the report.

As it can be seen, FormatKV() for doubles, used %.2f format,
which was meant to preserve at least some of the precision.
However, before that function is ever called, the doubles
were already cast to the integer via RoundDouble()...

This is also the case for console reporter, where it makes
sense because the screen space is limited, and this reporter,
however the CSV reporter does output some( decimal digits.

Thus i can only conclude that the loss of the precision
was not really considered, so i have decided to adjust the
code of the json reporter to output the full fp precision.

There can be several reasons why that is the right thing
to do, the bigger the time_unit used, the greater the
precision loss, so i'd say any sort of further processing
(like e.g. tools/compare_bench.py does) is best done
on the values with most precision.

Also, that cast skewed the data away from zero, which
i think may or may not result in false- positives/negatives
in the output of tools/compare_bench.py

* Json reporter: FormatKV(double): address review note

* tools/gbench/report.py: skip benchmarks with different time units

While it may be useful to teach it to operate on the
measurements with different time units, which is now
possible since floats are stored, and not the integers,
but for now at least doing such a sanity-checking
is better than providing misinformation.
2017-07-24 16:13:55 -07:00
Dominic Hamon
5b7683f49e more clang tidy cleanups (#417) 2017-07-15 00:21:20 +02:00
Dominic Hamon
e8fc2a2b8c Google-style cleanups (#416) 2017-07-13 18:33:43 +02:00
Tom Madams
ee3cfca651 Fix ThreadCPUUsage when running on RTEMS. (#414)
Change ThreadCPUUsage to call ProcessCPUUsage if __rtems__ is defined.

RTEMS real time OS doesn't support CLOCK_THREAD_CPUTIME_ID. See
https://github.com/RTEMS/rtems/blob/master/cpukit/posix/src/clockgettime.c#L58-L59

Prior to this change, ThreadCPUUsage would fail when running on RTEMS with:
  ERROR: clock_gettime(CLOCK_THREAD_CPUTIME_ID, ...) failed
2017-07-06 15:59:13 -07:00
Eric
9d4b719dae Make Benchmark a single header library (but not header-only) (#407)
* Make Benchmark a single header library (but not header-only)

This patch refactors benchmark into a single header, to allow
for slightly easier usage.

The initial reason for the header split was to keep C++ library
components from being included by benchmark_api.h, making that
part of the library STL agnostic. However this has since changed
and there seems to be little reason to separate the reporters from
the rest of the library.

* Fix internal_macros.h

* Remove more references to macros.h
2017-07-04 16:31:47 -06:00
Eric
b8a2206fb2 Add ClearRegisteredBenchmark() function. (#402)
* Add ClearRegisteredBenchmark() function.

Since benchmarks can be registered at runtime using the RegisterBenchmark(...)
functions, it makes sense to have a ClearRegisteredBenchmarks() function too,
that can be used at runtime to clear the currently registered benchmark and
re-register an entirely new set.

This allows users to run a set of registered benchmarks, get the output using
a custom reporter, and then clear and re-register new benchmarks based on the
previous results.

This fixes issue #400, at least partially.

* Remove unused change
2017-06-14 09:16:53 -07:00
Yixuan Qiu
f3b3dd99be Use the sample version of standard deviation (#383)
* remove unnecessary weights

* use sample standard deviation

* add contributor information

* remove redundant code

* initialize variable to eliminate compiler warning
2017-06-05 10:32:15 -07:00
David Kruger
15e9ebaf83 Associate the required include directory with the benchmark library (#393)
Using target_include_directories CMake will implicitly add the the
necessary include paths to targets which link against the benchmark
library. This is useful when the benchmark repo is included as a
subdirectory in another CMake build.
2017-05-23 08:40:31 -07:00
Joao Paulo Magalhaes
ec6f03579e Trying again to fix error caused by -Wunused-function.
This thing with the pragma ignore was getting out of hand: now
MinGW (and probably GCC) was erroring too. So I chose to move
the definition of IsZero() out of the anonymous namespace into
benchmark.cc.
2017-05-03 00:05:15 +01:00
Joao Paulo Magalhaes
160770fd08 Fix dropped-style elses. 2017-05-02 23:30:36 +01:00
Joao Paulo Magalhaes
ea019f3cd8 Allow different counter sets in CSV reporting. 2017-05-02 22:10:08 +01:00
Joao Paulo Magalhaes
3db6254c39 Console reporter: add /s prefix to counter rates. 2017-05-02 20:48:29 +01:00
Joao Paulo Magalhaes
17a012d754 Fix: --benchmark_counters_tabular was not being passed to tests. 2017-05-02 20:31:54 +01:00
Joao Paulo Magalhaes
615151723e Merge remote-tracking branch 'upstream/master' into compact 2017-05-02 18:54:37 +01:00
Joao Paulo Magalhaes
eb2bf34524 Fix indentation.
[ci-skip]
2017-05-02 11:37:46 +01:00
Joao Paulo Magalhaes
47226ccd56 CHECK(): rename EPS to FLOAT for consistency with googletest style. 2017-05-01 23:01:10 +01:00
Joao Paulo Magalhaes
ef6b4fb857 Simplify printing of counters in console reporter. 2017-04-29 20:41:33 +01:00
Joao Paulo Magalhaes
c16c8fffac CHECK_EPS: Use parentheses to wrap macro arguments. 2017-04-29 20:35:02 +01:00
Joao Paulo Magalhaes
921a51abcf Console reporting of user counters: print rates like non-rates. 2017-04-29 19:34:37 +01:00
Joao Paulo Magalhaes
5ed1b1dea7 Merge branch 'master' of https://github.com/vladoovtcharov/benchmark into test_usercounters 2017-04-29 19:27:53 +01:00
Joao Paulo Magalhaes
8adf59d762 Add epsilon check macros for float comparison. 2017-04-28 20:40:21 +01:00
Joao Paulo Magalhaes
409f35da50 User counters: fix misplaced newline in console reporter header. 2017-04-27 19:22:36 +01:00
vladoovt
1295ce8f23 Fixes #378 coercion to double was causing counter to forget its flags, changed it so that its value is updated directly 2017-04-27 09:56:43 -06:00
Joao Paulo Magalhaes
2d1a34626f Fixes #378 (hopefully). Unit tests for counters to follow.
The problem was that the call to Finish() the user counters was
lost in a big merge. If I had already written the tests for the
user counters, this would probably have been catched earlier.
2017-04-27 13:16:49 +01:00
Dmitry Trifonov
7a74b74856 fix for android NDK r10e (#375) 2017-04-20 20:07:52 -06:00
Dmitry Trifonov
09b93ccc6a fix android compilation (#372)
* fix android compilation

* checking __GLIBCXX__ and __GLIBCPP__ macro in addition to __ANDROID__

* using vsnprintf instead of std::vsnprintf to compile on Android

* removed __GLIBCPP__ check on Android

* StringPrintF instead of std::to_string for Android
2017-04-18 09:48:07 -06:00
Eric Fiselier
46afd8e693 Don't limit benchmarks with manual timers to 5x the elapsed real time.
When using CPU time to determine the correct number of iterations the
library additionally checks if the benchmark has consumed 5x the minimum
required time according to the wall clock. This prevents benchmarks
with low CPU usage from running for much longer than actually intended.

However when a benchmark uses a manual timer this heuristic isn't helpful
and likely isn't correct since we don't know what the manual timer actually
measures.

This patch removes the above restriction when a benchmark specifies a manual
timer.
2017-04-18 00:13:18 -06:00
Eric
74b24058ad Add Benchmark::Iterations for explicit iteration count control - Fixes #370 (#373)
* Add Benchmark::Iterations for explicitly specifying the number of iterations to use.

* Document that benchmark::Iterations should not be used to limit benchmark runtimes
2017-04-17 21:29:28 -07:00
Daniel Varga
312d9d0ac5 Fix cmake file not to use CMAKE_SOURCE_DIR (#367)
Using CMAKE_SOURCE_DIR benchmark cannot be built as part of a project.
This change allows to add benchmark using add_subdirectory to a bigger
project.
2017-04-10 08:43:05 -07:00
jpmag
9a5072d1bf Fixes #357: broken RMS values when time unit is set. (#362) 2017-04-04 12:31:28 -07:00
Dominic Hamon
858581ea76 Remove unnecessary benchmark subfolder 2017-04-04 08:39:02 -07:00
Dominic Hamon
66283598e6 Merge branch '1.1' of git://github.com/piribes/benchmark into piribes-1.1 2017-04-04 08:35:51 -07:00
Giuseppe Roberti
128fe25025 Fix ${include_install_dir} 2017-04-04 02:13:20 +02:00
Giuseppe Roberti
824bbb818e Add CMake Package Config files during install
- Remove target_include_directories of ${PROJECT_SOURCE_DIR}/include to
  fix error: Target "benchmark" INTERFACE_INCLUDE_DIRECTORIES property
  contains path which is prefixed in the source directory.
2017-04-03 01:54:24 +02:00
MVafin
ec15860da5 Fix CPU frequency parsing on Linux (#355) (#356)
* Fix reading CPU info from file

Macro CHECK do nothing for release mode, meaning it doesn't invoke the
arguments

* Add myself to AUTHORS and CONTRIBUTORS
2017-03-28 00:35:17 -06:00