Commit Graph

294 Commits

Author SHA1 Message Date
Vy Nguyen
dc1a97174d
Introduce accessors for currently public data members (threads and thread_index) (#1208)
* [benchmark] Introduce accessors for currently public data members `threads` and `thread_index`

Also deprecate the direct access to these fields.

Motivations:

Our internal library provides accessors for those fields because the styleguide disalows accessing classes' data members directly (even if they're const).
There has been a discussion to simply move internal library to make its fields public similarly to the OSS version here, however, the concern is that these kinds of direct access would prevent many types of future design changes (eg how/whether the values would be stored in the data member)

I think the concensus in the end is that we'd change the external library for this case.
AFAIK, there are three important third_party users that we'd need to migrate: tcmalloc, abseil and tensorflow.
Please let me know if I'm missing anyone else.

* [benchmark] Introduce accessors for currently public data members `threads` and `thread_index`

Also deprecate the direct access to these fields.

Motivations:

Our internal library provides accessors for those fields because the styleguide disalows accessing classes' data members directly (even if they're const).
There has been a discussion to simply move internal library to make its fields public similarly to the OSS version here, however, the concern is that these kinds of direct access would prevent many types of future design changes (eg how/whether the values would be stored in the data member)

I think the concensus in the end is that we'd change the external library for this case.
AFAIK, there are three important third_party users that we'd need to migrate: tcmalloc, abseil and tensorflow.
Please let me know if I'm missing anyone else.

* [benchmark] Introduce accessors for currently public data members `threads` and `thread_index`

Also deprecate direct access to `.thread_index` and make threads a private field

Motivations:

Our internal library provides accessors for those fields because the styleguide disalows accessing classes' data members directly (even if they're const).
There has been a discussion to simply move internal library to make its fields public similarly to the OSS version here, however, the concern is that these kinds of direct access would prevent many types of future design changes (eg how/whether the values would be stored in the data member)

I think the concensus in the end is that we'd change the external library for this case.
AFAIK, there are three important third_party users that we'd need to migrate: tcmalloc, abseil and tensorflow.
Please let me know if I'm missing anyone else.

* [benchmark] Introduce accessors for currently public data members `threads` and `thread_index`

Also deprecate direct access to `.thread_index` and make threads a private field

Motivations:

Our internal library provides accessors for those fields because the styleguide disalows accessing classes' data members directly (even if they're const).
There has been a discussion to simply move internal library to make its fields public similarly to the OSS version here, however, the concern is that these kinds of direct access would prevent many types of future design changes (eg how/whether the values would be stored in the data member)

I think the concensus in the end is that we'd change the external library for this case.
AFAIK, there are three important third_party users that we'd need to migrate: tcmalloc, abseil and tensorflow.
Please let me know if I'm missing anyone else.

* [benchmark] Introduce accessors for currently public data members `threads` and `thread_index`

Also deprecate direct access to `.thread_index` and make threads a private field

Motivations:

Our internal library provides accessors for those fields because the styleguide disalows accessing classes' data members directly (even if they're const).
There has been a discussion to simply move internal library to make its fields public similarly to the OSS version here, however, the concern is that these kinds of direct access would prevent many types of future design changes (eg how/whether the values would be stored in the data member)

I think the concensus in the end is that we'd change the external library for this case.
AFAIK, there are three important third_party users that we'd need to migrate: tcmalloc, abseil and tensorflow.
Please let me know if I'm missing anyone else.

* [benchmark] Introduce accessors for currently public data members `threads` and `thread_index`

Also deprecate direct access to `.thread_index` and make threads a private field

Motivations:

Our internal library provides accessors for those fields because the styleguide disalows accessing classes' data members directly (even if they're const).
There has been a discussion to simply move internal library to make its fields public similarly to the OSS version here, however, the concern is that these kinds of direct access would prevent many types of future design changes (eg how/whether the values would be stored in the data member)

I think the concensus in the end is that we'd change the external library for this case.
AFAIK, there are three important third_party users that we'd need to migrate: tcmalloc, abseil and tensorflow.
Please let me know if I'm missing anyone else.

* [benchmark] Introduce accessors for currently public data members `threads` and `thread_index`

Also deprecate direct access to `.thread_index` and make threads a private field

Motivations:

Our internal library provides accessors for those fields because the styleguide disalows accessing classes' data members directly (even if they're const).
There has been a discussion to simply move internal library to make its fields public similarly to the OSS version here, however, the concern is that these kinds of direct access would prevent many types of future design changes (eg how/whether the values would be stored in the data member)

I think the concensus in the end is that we'd change the external library for this case.
AFAIK, there are three important third_party users that we'd need to migrate: tcmalloc, abseil and tensorflow.
Please let me know if I'm missing anyone else.

* [benchmark] Introduce accessors for currently public data members `threads` and `thread_index`

Also deprecate direct access to `.thread_index` and make threads a private field

Motivations:

Our internal library provides accessors for those fields because the styleguide disalows accessing classes' data members directly (even if they're const).
There has been a discussion to simply move internal library to make its fields public similarly to the OSS version here, however, the concern is that these kinds of direct access would prevent many types of future design changes (eg how/whether the values would be stored in the data member)

I think the concensus in the end is that we'd change the external library for this case.
AFAIK, there are three important third_party users that we'd need to migrate: tcmalloc, abseil and tensorflow.
Please let me know if I'm missing anyone else.

* [benchmark] Introduce accessors for currently public data members `threads` and `thread_index`

Also deprecate direct access to `.thread_index` and make threads a private field

Motivations:

Our internal library provides accessors for those fields because the styleguide disalows accessing classes' data members directly (even if they're const).
There has been a discussion to simply move internal library to make its fields public similarly to the OSS version here, however, the concern is that these kinds of direct access would prevent many types of future design changes (eg how/whether the values would be stored in the data member)

I think the concensus in the end is that we'd change the external library for this case.
AFAIK, there are three important third_party users that we'd need to migrate: tcmalloc, abseil and tensorflow.
Please let me know if I'm missing anyone else.

* [benchmark] Introduce accessors for currently public data members `threads` and `thread_index`

Also deprecate direct access to `.thread_index` and make threads a private field

Motivations:

Our internal library provides accessors for those fields because the styleguide disalows accessing classes' data members directly (even if they're const).
There has been a discussion to simply move internal library to make its fields public similarly to the OSS version here, however, the concern is that these kinds of direct access would prevent many types of future design changes (eg how/whether the values would be stored in the data member)

I think the concensus in the end is that we'd change the external library for this case.
AFAIK, there are three important third_party users that we'd need to migrate: tcmalloc, abseil and tensorflow.
Please let me know if I'm missing anyone else.
2021-08-23 09:06:57 +01:00
Dominic Hamon
19026e232c
fix clang-tidy warnings (#1195) 2021-06-29 11:06:53 +01:00
Dominic Hamon
1799e1b9ec
prefix VLOG (#1187) 2021-06-24 18:55:37 +01:00
Dominic Hamon
6a5bf081d3
prefix macros to avoid clashes (#1186) 2021-06-24 18:21:59 +01:00
Dominic Hamon
5da5660429
Move flags inside the benchmark namespace (#1185)
This avoids clashes with other libraries that might define the same flags.
2021-06-24 16:50:19 +01:00
Roman Lebedev
fbc31405b2
Random interleaving of benchmark repetitions - the sequel (fixes #1051) (#1163)
Inspired by the original implementation by Hai Huang @haih-g
from https://github.com/google/benchmark/pull/1105.

The original implementation had design deficiencies that
weren't really addressable without redesign, so it was reverted.

In essence, the original implementation consisted of two separateable parts:
* reducing the amount time each repetition is run for, and symmetrically increasing repetition count
* running the repetitions in random order

While it worked fine for the usual case, it broke down when user would specify repetitions
(it would completely ignore that request), or specified per-repetition min time (while it would
still adjust the repetition count, it would not adjust the per-repetition time,
leading to much greater run times)

Here, like i was originally suggesting in the original review, i'm separating the features,
and only dealing with a single one - running repetitions in random order.

Now that the runs/repetitions are no longer in-order, the tooling may wish to sort the output,
and indeed `compare.py` has been updated to do that: #1168.
2021-06-03 21:16:54 +03:00
Dominic Hamon
d17ea66551
Fix leak in test, and provide path to remove leak from library (#1169)
* Fix leak in test, and provide path to remove leak from library

* make doc change
2021-06-03 16:08:00 +01:00
Roman Lebedev
80a62618e8
Introduce per-family instance index (#1165)
Much like it makes sense to enumerate all the families,
it makes sense to enumerate stuff within families.
Alternatively, we could have a global instance index,
but i'm not sure why that would be better.

This will be useful when the benchmarks are run not in order,
for the tools to sort the results properly.
2021-06-02 23:45:41 +03:00
Roman Lebedev
4c2e32f1d0
Introduce "family index" field into JSON output (#1164)
It may be useful for those wishing to further post-process JSON results,
but it is mainly geared towards better support for run interleaving,
where results from the same family may not be close-by in the JSON.

While we won't be able to do much about that for outputs,
the tools can and perhaps should reorder the results to that
at least in their output they are in proper order, not run order.

Note that this only counts the families that were filtered-in,
so if e.g. there were three families, and we filtered-out
the second one, the two families (which were first and third)
will have family indexes 0 and 1.
2021-06-02 18:06:45 +03:00
Roman Lebedev
a54ef37aea
Ensure that we print repetition count even when it was specified via flag --benchmark_repetitions= 2021-06-02 12:34:00 +03:00
Dominic Hamon
e025dd5a54
Revert "Implementation of random interleaving. (#1105)" (#1161)
This reverts commit a6a738c1cc.
2021-06-01 16:05:50 +01:00
Norman Heino
6f094ba13e
Fix perf counter argument parsing (#1160)
* Fix argument order in StrSplit

* Update AUTHORS, CONTRIBUTORS
2021-06-01 15:50:42 +01:00
haih-g
a6a738c1cc
Implementation of random interleaving. (#1105)
* Implementation of random interleaving. See
http://github.com/google/benchmark/issues/1051 for the feature requests.

Committer: Hai Huang (http://github.com/haih-g)

On branch fr-1051
Changes to be committed:
modified:   include/benchmark/benchmark.h
modified:   src/benchmark.cc
new file:   src/benchmark_adjust_repetitions.cc
new file:   src/benchmark_adjust_repetitions.h
modified:   src/benchmark_api_internal.cc
modified:   src/benchmark_api_internal.h
modified:   src/benchmark_register.cc
modified:   src/benchmark_runner.cc
modified:   src/benchmark_runner.h
modified:   test/CMakeLists.txt
new file:   test/benchmark_random_interleaving_gtest.cc

* Fix benchmark_random_interleaving_gtest.cc for fr-1051

Committer: Hai Huang <haih@google.com>

On branch fr-1051
Your branch is up to date with 'origin/fr-1051'.

Changes to be committed:
modified:   src/benchmark.cc
modified:   src/benchmark_runner.cc
modified:   test/benchmark_random_interleaving_gtest.cc

* Fix macos build for fr-1051

Committer: Hai Huang <haih@google.com>

On branch fr-1051
Your branch is up to date with 'origin/fr-1051'.

Changes to be committed:
modified:   src/benchmark_api_internal.cc
modified:   src/benchmark_api_internal.h
modified:   src/benchmark_runner.cc

* Fix macos and windows build for fr-1051.

Committer: Hai Huang <haih@google.com>

On branch fr-1051
Your branch is up to date with 'origin/fr-1051'.

Changes to be committed:
modified:   src/benchmark_runner.cc

* Fix benchmark_random_interleaving_test.cc for macos and windows in fr-1051

Committer: Hai Huang <haih@google.com>

On branch fr-1051
Your branch is up to date with 'origin/fr-1051'.

Changes to be committed:
modified:   test/benchmark_random_interleaving_gtest.cc

* Fix int type benchmark_random_interleaving_gtest for macos in fr-1051

Committer: Hai Huang <haih@google.com>

On branch fr-1051
Your branch is up to date with 'origin/fr-1051'.

Changes to be committed:
modified:   test/benchmark_random_interleaving_gtest.cc

* Address dominichamon's comments 03/29 for fr-1051

Committer: Hai Huang <haih@google.com>

On branch fr-1051
Your branch is up to date with 'origin/fr-1051'.

Changes to be committed:
modified:   src/benchmark.cc
modified:   src/benchmark_api_internal.cc
modified:   src/benchmark_api_internal.h
modified:   test/benchmark_random_interleaving_gtest.cc

* Address dominichamon's comment on default min_time / repetitions for fr-1051.
Also change sentinel of random_interleaving_repetitions to -1. Hopefully it
fixes the failures on Windows.

Committer: Hai Huang <haih@google.com>

On branch fr-1051
Your branch is up to date with 'origin/fr-1051'.

Changes to be committed:
modified:   src/benchmark.cc
modified:   src/benchmark_api_internal.cc
modified:   src/benchmark_api_internal.h

* Fix windows test failures for fr-1051

Committer: Hai Huang <haih@google.com>

On branch fr-1051
Your branch is up to date with 'origin/fr-1051'.

Changes to be committed:
modified:   src/benchmark_api_internal.cc
modified:   src/benchmark_runner.cc

* Add license blurb for fr-1051.

Committer: Hai Huang <haih@google.com>

On branch fr-1051
Your branch is up to date with 'origin/fr-1051'.

Changes to be committed:
modified:   src/benchmark_adjust_repetitions.cc
modified:   src/benchmark_adjust_repetitions.h

* Switch to std::shuffle() for fr-1105.

Committer: Hai Huang <haih@google.com>

On branch fr-1051
Your branch is up to date with 'origin/fr-1051'.

Changes to be committed:
modified:   src/benchmark.cc

* Change to 1e-9 in fr-1105

Committer: Hai Huang <haih@google.com>

On branch fr-1051
Your branch is up to date with 'origin/fr-1051'.

Changes to be committed:
modified:   src/benchmark_adjust_repetitions.cc

* Fix broken build caused by bad merge for fr-1105.

Committer: Hai Huang <haih@google.com>

On branch fr-1051
Your branch is up to date with 'origin/fr-1051'.

Changes to be committed:
modified:   src/benchmark_api_internal.cc
modified:   src/benchmark_runner.cc

* Fix build breakage for fr-1051.

Committer: Hai Huang <haih@google.com>

On branch fr-1051
Your branch is up to date with 'origin/fr-1051'.

Changes to be committed:
modified:   src/benchmark.cc
modified:   src/benchmark_api_internal.cc
modified:   src/benchmark_api_internal.h
modified:   src/benchmark_register.cc
modified:   src/benchmark_runner.cc

* Print out reports as they come in if random interleaving is disabled (fr-1051)

Committer: Hai Huang <haih@google.com>

On branch fr-1051
Your branch is up to date with 'origin/fr-1051'.

Changes to be committed:
modified:   src/benchmark.cc

* size_t, int64_t --> int in benchmark_runner for fr-1051.

Committer: Hai Huang <haih@google.com>

On branch fr-1051
Your branch is up to date with 'origin/fr-1051'.

Changes to be committed:
modified:   src/benchmark_runner.cc
modified:   src/benchmark_runner.h

* Address comments from dominichamon for fr-1051

Committer: Hai Huang <haih@google.com>

On branch fr-1051
Your branch is up to date with 'origin/fr-1051'.

Changes to be committed:
modified:   src/benchmark.cc
modified:   src/benchmark_adjust_repetitions.cc
modified:   src/benchmark_adjust_repetitions.h
modified:   src/benchmark_api_internal.cc
modified:   src/benchmark_api_internal.h
modified:   test/benchmark_random_interleaving_gtest.cc

* benchmar_indices --> size_t to make CI pass: fr-1051

Committer: Hai Huang <haih@google.com>

On branch fr-1051
Your branch is up to date with 'origin/fr-1051'.

Changes to be committed:
modified:   src/benchmark.cc

* Fix min_time not initialized issue for fr-1051.

Committer: Hai Huang <haih@google.com>

On branch fr-1051
Your branch is up to date with 'origin/fr-1051'.

Changes to be committed:
modified:   src/benchmark_api_internal.cc
modified:   src/benchmark_api_internal.h

* min_time --> MinTime in fr-1051.

Committer: Hai Huang <haih@google.com>

On branch fr-1051
Your branch is up to date with 'origin/fr-1051'.

Changes to be committed:
modified:   src/benchmark_api_internal.cc
modified:   src/benchmark_api_internal.h
modified:   src/benchmark_runner.cc

* Add doc for random interleaving for fr-1051

Committer: Hai Huang <haih@google.com>

On branch fr-1051
Your branch is up to date with 'origin/fr-1051'.

Changes to be committed:
modified:   README.md
new file:   docs/random_interleaving.md

Co-authored-by: Dominic Hamon <dominichamon@users.noreply.github.com>
2021-05-20 17:09:16 +01:00
Mircea Trofin
e539e807da
[PFM] Extend perf counter support to multi-threaded cases. (#1153)
* Extend perf counter support to multi-threaded cases.

* Docs update

* const-ed Snapshot
2021-05-19 09:49:05 +01:00
Guillaume Chatelet
7d0d9061d8
Support -Wsuggest-override (#1059)
* Support -Wsuggest-override

google/benchmark is C++11 compatible but doesn't use the `override` keyword.
Projects using google/benchmark with enabled `-Wsuggest-override` and `-Werror` will fail to compile.

* Add -Wsuggest-override cxx flag

* Revert unrelated formatting

* Revert unrelated formatting, take 2

* Revert unrelated formatting, take 3

* Disable -Wsuggest-override when compiling tests, gtest does not handle it yet

Co-authored-by: Dominic Hamon <dominichamon@users.noreply.github.com>
2021-05-11 14:56:00 +03:00
Roman Lebedev
a2e8a8a9db
Clean -Wreserved-identifier instances (#1143) 2021-05-06 20:31:14 +01:00
Mircea Trofin
e0826edea7
Fix StrSplit empty string case (#1142)
This also fixes #1135. Because StrSplit was returning a vector with an
empty string, it was treated by PerfCounters::Create as a legitimate ask
for setting up a counter with that name. The empty vector is understood
by PerfCounters as "just return NoCounters()".
2021-05-06 19:12:36 +01:00
Dominic Hamon
d0c227ccfd
Add API to benchmark allowing for custom context to be added (#1137)
* Add API to benchmark allowing for custom context to be added

Fixes #525

* add docs

* Add context flag output to JSON reporter

* Plumb everything into the global context.

* Add googletests for custom context

* update docs with duplicate key behaviour
2021-05-05 12:08:23 +01:00
Dominic Hamon
33c133a206
Add benchmark_context flag that allows per-run custom context. (#1127)
* Add `benchmark_context` flag that allows per-run custom context.

Add support for key-value flags in general.
Added test for key-value flags.
Added `benchmark_context` flag.
Output content of `benchmark_context` to base reporter.

Solves the first part of #525.

* Docs and better help
2021-05-04 14:36:11 +01:00
Mircea Trofin
376ebc2635
Support optional, user-directed collection of performance counters (#1114)
* Support optional, user-directed collection of performance counters

The patch allows an engineer wishing to drill into the root causes
of a regression, for example. Currently, only single threaded runs
are supported. The feature is a build-time opt in, and then a runtime
opt in.

The engineer may run the benchmark executable, passing a list of
performance counter names (using libpfm's naming scheme) at the
command line. The counter values will then be collected and reported
back as UserCounters.

This is different from #240 in that it is a benchmark user opt-in, and
the counter collection is transparent to the benchmark.

Currently, this is only supported on platforms where libpfm is
supported.

libpfm: http://perfmon2.sourceforge.net/

* 'Use' values param in Snapshot when BENCHMARK_OS_WINDOWS

This is to avoid unused parameter warning-as-error

* Added missing include for <vector> in perf_counters.cc

* Moved doc to docs

* Added license blurbs
2021-04-28 09:25:29 +01:00
Matt Armstrong
69054ae50e
Use fewer ramp up repetitions when KeepRunningBatch is used (#1113)
Use the benchmark's reported iteration count when estimating
iterations for the next repetition, rather than the requested
iteration count.  When the benchmark uses KeepRunningBatch the actual
iteration count can be larger than the one the runner requested.

Prior to this fix the runner was underestimating the next iteration
count, sometimes significantly so.  Consider the case of a benchmark
using a batch size of 1024.  Prior to this change, the benchmark
runner would attempt iteration counts 1, 10, 100 and 1000, yet the
benchmark itself would do the same amount of work each time: a single
batch of 1024 iterations.  The discrepancy could also contribute to
estimation errors once the benchmark time reached 10% of the target.
For example, if the very first batch of 1024 iterations reached 10% of
benchmark_min_min time, the runner would attempt to scale that to 100%
from a basis of one iteration rather than 1024.

This bug was particularly noticeable in benchmarks with large batch
sizes, especially when the benchmark also had slow set up or tear down
phases.

With this fix in place it is possible to use KeepRunningBatch to
achieve a kind of "minimum iteration count" feature by using a larger
fixed batch size.  For example, a benchmark may build a map of 500K
elements and test a "find" operation.  There is no point in running
"find" just 1, 10, 100, etc., times.  The benchmark can now pick a
batch size of something like 10K, and the runner will arrive at the
final max iteration count with in noticeably fewer repetitions.
2021-04-20 07:16:05 +01:00
Tobias Schmidt
5e387e7d33
Implement custom benchmark name (#1107)
* Implement custom benchmark name

The benchmark's name can be changed using the Name() function
which internally uses SetName().

* Update AUTHORS and CONTRIBUTORS

* Describe new feature in README

* Move new name function up

Fixes #1106
2021-03-30 16:43:03 +03:00
feserr
378ed8ff25
Add 'seconds' time unit (#1076)
Fixes #1075.

* Add an option to report in seconds.

* Reduce the time of the test.

* Add CSV/JSON tests for new time reports.
2020-12-21 20:15:58 +03:00
Scott K Logan
17a6b21ee1
Fix Range when starting at zero (#1073)
The existing behavior results in the `0` value being added twice. Since
`lo` is always added to `dst`, we never want to explicitly add `0` if
`lo` is equal to `0`.
2020-11-26 11:12:45 +00:00
Mario Emmenlauer
37ced31bfc
Added support for macro expansion in benchmark names (#1054)
* Adding test for defined names in test fixtures

* include/benchmark/benchmark.h: Added support for macro expansion in benchmark names
2020-11-19 13:50:30 +00:00
Christian Wassermann
4857962394
Add CartesianProduct with associated test (#1029)
* Add CartesianProduct with associated test

* Use CartesianProduct in Ranges to avoid code duplication
* Add new cartesian_product_test to CMakeLists.txt
* Update AUTHORS & CONTRIBUTORS

* Rename CartesianProduct to ArgsProduct

* Rename test & fixture accordingly
* Add example for ArgsProduct to README
2020-08-25 13:47:44 +01:00
Dominic Hamon
5b72b6c2da
Remove "BENCHMARK_" prefix from env var version of command line flags (#997)
As noted in #995, this causes issues when the command line flag already
starts with "benchmark_", which they all do.

Not caught by tests as the test flags didn't start with "benchmark".

Fixes #995
2020-08-18 10:02:20 +01:00
Alexander Enaldiev
9901011880
JSONReporter: don't report on scaling if we didn't get it (#1005) (#1008)
* JSONReporter: don't report on scaling if we didn't get it (#1005)

* JSONReporter: fix due to review (std::pair<bool, bool> -> enum)

* JSONReporter: scaling: fix the algo (due to review discussion)

* benchmark.h: revert to old-fashioned enum's (C++03 compatibility); rreporter_output_test: let's skip scaling
2020-07-28 12:46:07 +01:00
Brian Wolfe
99c52f1414
use rfc3339-formatted timestamps in output [output format change] (#965)
* timestamp: use rfc3339-formatted timestamps in output

Replace localized timestamps with machine-readable IETF RFC 3339 format
timestamps. This is an attempt to make the output timestamps easily
machine-readable. ISO8601 specifies standards for time interchange
formats. IETF RFC 3339: https://tools.ietf.org/html/rfc3339 defines a
subset of these for use in the internet. The general form for these
timestamps is:

YYYY-MM-DDTHH:mm:SS[+-]hhmm

This replaces the localized time formats that are currently being used
in the benchmark output to prioritize interchangeability and
machine-readability.

This might break existing programs that rely on the particular date-time
format. This might also may make times less human readable. RFC3339 was
intended to balance human readability and simplicity for machine
readability, but it is primarily intended as an internal representation.

* timers: remove utc string formatting

We only ever need local time printing. Remove the UTC printing
and cosnolidate the logic slightly.

* timers: manually create rfc3339 string

The C++ standard library does not output the time offset in RFC3339
format, it is missing the : between hours and minutes. VS does not
appear to support timezone information by default. To avoid adding too
much complexity to benchmark around timezone handling e.g. a full
date library like https://github.com/HowardHinnant/date, we fall back
to outputting GMT time with a -00:00 offset for those cases.

* timers: use reentrant form for localtime_r & tmtime_r

For non-windows, use the reentrant form for the time conversion
functions.

* timers: cleanup

Use strtol instead of brittle moving characters around.

* timers: only call strftime twice.

Also size buffers to known maximum necessary size and name constants
more appropriately.

* timers: fix unused variable warning
2020-06-15 17:28:17 +01:00
Keith Moyer
8cead00783
Remove warnings for internal use of CSVReporter (#956)
In a previous commit[1], diagnostic pragmas were used to avoid this
warning. However, the incorrect warning flag was indicated, leaving the
warning in place. -Wdeprecated is for deprecated features while
-Wdeprecated-declarations for deprecated functions, variables, and
types[2].

[1] c408461983
[2] https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html
2020-04-14 10:20:22 +01:00
Dominic Hamon
0ab2c2906b
Fix type conversion warnings. (#951)
* Fix type conversion warnings.

Fixes #949

Tested locally (Linux/clang), but warnings are on MSVC so may differ.

* Drop the ULP so the double test passes
2020-04-06 13:52:09 +01:00
Paweł Bylica
c078337494
Relax CHECK condition in benchmark_runner.cc (#938)
* Add State::error_occurred()

* Relax CHECK condition in benchmark_runner.cc

If the benchmark state contains an error, do not expect any iterations has been run.
This allows using SkipWithError() and return early from the benchmark function.

* README.md: document new possible usage of SkipWithError()
2020-02-21 17:53:25 +03:00
Alex Reinking
e5ea03ce07
Fix cxx03 standard selection, option override in CMake 3.13+. Fixes #933 (#934) 2020-01-31 10:16:25 +00:00
Jordan Williams
daff5fead3 Alias CMake Targets. Fixes #921 (#926)
* add Jordan Williams to both CONTRIBUTORS and AUTHORS

* alias benchmark libraries

Provide aliased CMake targets for the benchmark and benchmark_main targets.
The alias targets are namespaced under benchmark::, which is the namespace when they are exported.
I chose not to use either the PROJECT_NAME or the namespace variable but to hard-code the namespace.
This is because the benchmark and benchmark_main targets are hard-coded by name themselves.
Hard-coding the namespace is also much cleaner and easier to read.

* link to aliased benchmark targets

It is safer to link against namespaced targets because of how CMake interprets the double colon.
Typo's will be caught by CMake at configuration-time instead of during compile / link time.

* document the provided alias targets

* add "Usage with CMake" section in documentation

This section covers linking against the alias/import CMake targets and including them using either find_package or add_subdirectory.

* format the "Usage with CMake" README section

Added a newline after the "Usage with CMake" section header.
Dropped the header level of the section by one to make it a direct subsection of the "Usage" section.
Wrapped lines to be no longer than 80 characters in length.
2020-01-14 23:21:24 +03:00
Roman Lebedev
367119482f CPU caches are binary units, not SI. (#911)
As disscussed in https://github.com/google/benchmark/issues/899,
it is all but certain that the multiplier should be 1024, not 1000.

Fixes https://github.com/google/benchmark/issues/899
2019-12-02 09:29:16 +00:00
Gregor Jasny
c50ac68c50 CMake: use full add_test(NAME <> COMMAND <>) signature (#901)
* CTest must use proper paths to executables

With the following syntax:

```
  add_test(NAME <name> COMMAND <command> [<arg>...])
```

if `<command>` specifies an executable target it will automatically
be replaced by the location of the executable created at build time.

This is important if a `<Configuration>_POSTFIX` like `_d` is used.

* Fix typo in ctest invocation

Instead of `-c` the uppercase `-C` must be used to select a config.
But better use the longopt.
2019-11-05 22:46:13 +03:00
Martin Blanchard
bc200ed8ee Read options from environment (#881) (#883)
Initialize option flags from environment variables values if they are defined, eg. `BENCHMARK_OUT=<filename>` for `--benchmark_out=<filename>`. Command line flag value always prevails.

Fixes https://github.com/google/benchmark/issues/881.
2019-10-23 11:07:08 +03:00
Paul Wankadia
309de5988e Switch to Starlark for C++ rules. (#887)
While I'm here, format all of the files that I touched.
2019-10-08 11:09:51 +01:00
Geoffrey Martin-Noble
d2fc7fe659 Guard ASSERT_THROWS checks with BENCHMARK_HAS_NO_EXCEPTIONS (#874)
* Guard ASSERT_THROWS checks with BENCHMARK_HAS_NO_EXCEPTIONS

This allows the test be run with exceptions turned off

* Add myself to CONTRIBUTORS

I don't need to be added to AUTHORS, as I am a Google employee
2019-09-20 10:25:31 +01:00
Roman Lebedev
7d97a057e1
Custom user counters: add invert modifier. (#850)
While current counters can e.g. answer the question
"how many items is processed per second", it is impossible to get
it to tell "how many seconds it takes to process a single item".

The solution is to add a yet another modifier `kInvert`,
that is *always* considered last, which simply inverts the answer.

Fixes #781, #830, #848.
2019-08-12 17:47:46 +03:00
Eric Fiselier
c408461983 Disable deprecated warnings when touching CSVReporter internally.
The CSVReporter is deprecated, but we still need to reference it in
a few places. To avoid breaking the build when warnings are errors,
we need to disable the warning when we do so.
2019-08-07 15:55:40 -04:00
Eric Backus
32a1e39720 Bugfix/wsl selftest fixes. Fixes #839 (#843)
* Update AUTHORS and CONTRIBUTORS

* Fix WSL self-test failures

Some of the benchmark self-tests expect and check for a particular
output format from the benchmark library. The numerical values must
not be infinity or not-a-number, or the test will report an error.
Some of the values are computed bytes-per-second or items-per-second
values, so these require that the measured CPU time for the test to be
non-zero. But the loop that is being measured was empty, so the
measured CPU time for the loop was extremely small. On systems like
Windows Subsystem for Linux (WSL) the timer doesn't have enough
resolution to measure this, so the measured CPU time was zero.

This fix just makes sure that these tests have something within the
timing loop, so that the benchmark library will not decide that the
loop takes zero CPU time. This makes these tests more robust, and in
particular makes them pass on WSL.
2019-07-27 19:02:31 +03:00
Roman Lebedev
f92903cc53
Iteration counts should be uint64_t globally. (#817)
This is a shameless rip-off of https://github.com/google/benchmark/pull/646
I did promise to look into why that proposed PR was producing
so much worse assembly, and so i finally did.

The reason is - that diff changes `size_t` (unsigned) to `int64_t` (signed).

There is this nice little `assert`:
7a1c370283/include/benchmark/benchmark.h (L744)
It ensures that we didn't magically decide to advance our iterator
when we should have finished benchmarking.

When `cached_` was unsigned, the `assert` was `cached_ UGT 0`.
But we only ever get to that `assert` if `cached_ NE 0`,
and naturally if `cached_` is not `0`, then it is bigger than `0`,
so the `assert` is tautological, and gets folded away.

But now that `cached_` became signed, the assert became `cached_ SGT 0`.
And we still only know that `cached_ NE 0`, so the assert can't be
optimized out, or at least it doesn't currently.

Regardless of whether or not that is a bug in itself,
that particular diff would have regressed the normal 64-bit systems,
by halving the maximal iteration space (since we go from unsigned counter
to signed one, of the same bit-width), which seems like a bug.
And just so it happens, fixing *this* bug, fixes the other bug.

This produces fully (bit-by-bit) identical state_assembly_test.s
The filecheck change is actually needed regardless of this patch,
else this test does not pass for me even without this diff.
2019-05-13 12:33:11 +03:00
Roman Lebedev
4b77194032
CMake: codedrop of googletest cmake magic from me (#809)
https://github.com/google/benchmark/pull/801 is stuck with some cryptic cmake failure due to
some linking issue between googletest and threading libraries.

I suspect that is mostly happening because of the, uhm,
intentionally extremely twisted-in-the-brains approach that is being used to
actually build the library as part of the buiild,
except without actually building it as part of the build.

If we do actually build it as part of the build,
then all the transitive dependencies should magically be in order,
and maybe everything will just work.

This new version of cmake magic was written by me in
0e22f085c5/cmake/Modules/GoogleTest.cmake.in
0e22f085c5/cmake/Modules/GoogleTest.cmake, based on the official googletest docs and LOTS of experimentation.
2019-04-30 13:36:29 +03:00
Michael Tesch
588be0446a escape special chars in csv and json output. (#802)
* escape special chars in csv and json output.

- escape \b,\f,\n,\r,\t,\," from strings before dumping
  them to json or csv.
- also faithfully reproduce the sign of nan in json.
this fixes github issue #745.

* functionalize.

* split string escape functions between csv and json

* Update src/csv_reporter.cc

Co-Authored-By: tesch1 <tesch1@gmail.com>

* Update src/json_reporter.cc

Co-Authored-By: tesch1 <tesch1@gmail.com>
2019-04-19 18:47:25 +01:00
Dominic Hamon
1d41de8463
Add command line flags tests (#793)
Increase coverage
2019-04-17 17:08:52 +01:00
Bryan Lunt
7a1c370283 Add process_time for better OpenMP and user-managed thread timing
* Google Benchmark now works with OpenMP and other user-managed threading.
2019-04-09 13:01:33 +01:00
Daniel Harvey
e3666568a9 Negative ranges #762 (#787)
* Add FIXME in multiple_ranges_test.cc

* Improve handling of large bounds in AddRange.

Due to breaking the loop too early, AddRange
would miss a final multplier of 'mult' that
was within the numeric range of T.

* Enable negative values for Range argument

Fixes #762.

* Try to fix build of benchmark_gtest

* Try some more to fix build

* Attempt to fix format macros

* Attempt to resolve format errors for mingw32

* Review feedback

Put unit tests in benchmark::internal namespace

Fix error reporting in multiple_ranges_test.cc
2019-03-26 10:50:53 +00:00
BaaMeow
478eafa36b [JSON] add threads and repetitions to the json output (#748)
* [JSON] add threads and repetitions to the json output, for better ide…
[Tests] explicitly check for thread == 1
[Tests] specifically mark all repetition checks
[JSON] add repetition_index reporting, but only for non-aggregates (i…

* [Formatting] Be very, very explicit about pointer alignment so clang-format can not put pointers/references on the wrong side of arguments.
[Benchmark::Run] Make sure to use explanatory sentinel variable rather than a magic number.

* Do not pass redundant information
2019-03-26 09:53:07 +00:00
Daniel Harvey
f6e96861a3 BENCHMARK_CAPTURE() and Complexity() - naming problem (#761)
Created BenchmarkName class which holds the full benchmark
name and allows specifying and retrieving different components
of the name (e.g. ARGS, THREADS etc.)

Fixes #730.
2019-03-17 16:38:51 +03:00