Commit Graph

438 Commits

Author SHA1 Message Date
András Leitereg
cf446a18bf Remove superfluous cache line scaling in JSON reporter. (#896)
Cache size is already stored in bytes.
2019-10-24 22:13:03 +03:00
Martin Blanchard
bc200ed8ee Read options from environment (#881) (#883)
Initialize option flags from environment variables values if they are defined, eg. `BENCHMARK_OUT=<filename>` for `--benchmark_out=<filename>`. Command line flag value always prevails.

Fixes https://github.com/google/benchmark/issues/881.
2019-10-23 11:07:08 +03:00
Geoffrey Martin-Noble
b874e72208 Guard definition of __STDC_FORMAT_MACROS in ifndef (#875)
This macro is sometimes already defined and redefining it results
in build errors.
2019-09-23 10:53:09 +01:00
Geoffrey Martin-Noble
7411874d95 Define HOST_NAME_MAX for NaCl and RTEMS (#876)
These OS's don't always have HOST_NAME_MAX defined, resulting in
build errors.

A few related changes as well:
* Only define HOST_NAME_MAX if it's not already defined. There are
  some cases where this is already defined, e.g. with NaCl if
  __USE_POSIX is set. To avoid all of these, only define it if it's
  not already defined.
* Default HOST_NAME_MAX to 64 and issue a #warning. Having the wrong
  max length is pretty harmless. The name just ends up getting
  truncated and this is only for printing debug info. Because we're
  constructing a std::string from a char[] (so defined length), we
  don't need to worry about gethostname's undefined behavior for
  whether the truncation is null-terminated when the hostname
  doesn't fit in HOST_NAME_MAX. Of course, this doesn't help people
  who have -Werror set, since they'll still get a warning.
2019-09-23 10:38:34 +01:00
Sayan Bhattacharjee
7ee72863fd Remove unused doc argument from DEFINE_ macros. (#857)
- Adresses : #856

  - The unused `doc` argument was removed from the `DEFINE_` macros in
    `commandlineflags.h`

  - Converted all the previous `doc` strings passed to the `DEFINE_`
    macros to multiline comments.
2019-08-21 14:12:03 -07:00
Roman Lebedev
7d97a057e1
Custom user counters: add invert modifier. (#850)
While current counters can e.g. answer the question
"how many items is processed per second", it is impossible to get
it to tell "how many seconds it takes to process a single item".

The solution is to add a yet another modifier `kInvert`,
that is *always* considered last, which simply inverts the answer.

Fixes #781, #830, #848.
2019-08-12 17:47:46 +03:00
Eric Fiselier
c408461983 Disable deprecated warnings when touching CSVReporter internally.
The CSVReporter is deprecated, but we still need to reference it in
a few places. To avoid breaking the build when warnings are errors,
we need to disable the warning when we do so.
2019-08-07 15:55:40 -04:00
Roman Lebedev
8e48105d46 CMake; windows: link to lowercase 'shlwapi' - consistent with headers (#840)
The filenames are consistently inconsistent in windows world, might
have something to do with default file system being case-insensitive.
While the native MinGW buils were fixed in 5261307982
that only addressed the headers, but not libraries.
The problem remains when one tries to do a MinGW cross-build from
case-sensitive filesystem.
2019-07-22 13:42:12 +01:00
Sam Elliott
4abdfbb802 Add RISC-V support in cycleclock::Now (#833)
The RISC-V implementation of `cycleclock::Now` uses the user-space
`rdcycle` instruction to query how many cycles have happened since the
core started.

The only complexity here is on 32-bit RISC-V, where `rdcycle` can only
read the lower 32 bits of the 64-bit hardware counter. In this case,
`rdcycleh` reads the higher 32 bits of the counter. We match the powerpc
implementation to detect and correct for overflow in the high bits.
2019-07-05 09:28:17 +01:00
Orgad Shaneh
04a9343fc9 Make some functions const (#832)
and ThreadManager ctor explicit.

Reported by CppCheck.
2019-06-26 09:06:24 +01:00
Roman Lebedev
090faecb45
Use IterationCount in one more place
Found in -UNDEBUG build
2019-05-13 22:42:18 +03:00
Roman Lebedev
f92903cc53
Iteration counts should be uint64_t globally. (#817)
This is a shameless rip-off of https://github.com/google/benchmark/pull/646
I did promise to look into why that proposed PR was producing
so much worse assembly, and so i finally did.

The reason is - that diff changes `size_t` (unsigned) to `int64_t` (signed).

There is this nice little `assert`:
7a1c370283/include/benchmark/benchmark.h (L744)
It ensures that we didn't magically decide to advance our iterator
when we should have finished benchmarking.

When `cached_` was unsigned, the `assert` was `cached_ UGT 0`.
But we only ever get to that `assert` if `cached_ NE 0`,
and naturally if `cached_` is not `0`, then it is bigger than `0`,
so the `assert` is tautological, and gets folded away.

But now that `cached_` became signed, the assert became `cached_ SGT 0`.
And we still only know that `cached_ NE 0`, so the assert can't be
optimized out, or at least it doesn't currently.

Regardless of whether or not that is a bug in itself,
that particular diff would have regressed the normal 64-bit systems,
by halving the maximal iteration space (since we go from unsigned counter
to signed one, of the same bit-width), which seems like a bug.
And just so it happens, fixing *this* bug, fixes the other bug.

This produces fully (bit-by-bit) identical state_assembly_test.s
The filecheck change is actually needed regardless of this patch,
else this test does not pass for me even without this diff.
2019-05-13 12:33:11 +03:00
Michał Janiszewski
b988639f31 Fix compilation for Android (#816)
Android doesn't support `getloadavg`
2019-05-09 15:22:13 -07:00
Roman Lebedev
33d4404650 Don't read CMAKE_BUILD_TYPE if it is not there (#811)
Weird, but seems consistent with the rest of cmake here.
2019-05-07 16:06:50 -07:00
Lockywolf
823d24630d Add support for GNU Install Dirs from GNU Coding Standards. Fixes #807 (#808)
* Add support for GNU Install Dirs from GNU Coding Standards

* src/CMakeLists.txt: Added support for setting the standard variables,
                      such as CMAKE_INSTALL_BINDIR.

* Replace install destinations by the ones from GNU Coding Standards.

* Set the default .cmake and .pc default path.
2019-05-01 09:13:33 +01:00
Dominic Hamon
13b8bdc2b5
Bump required cmake version from 2.x to 3.x (#801) 2019-05-01 09:06:12 +01:00
Michael Tesch
588be0446a escape special chars in csv and json output. (#802)
* escape special chars in csv and json output.

- escape \b,\f,\n,\r,\t,\," from strings before dumping
  them to json or csv.
- also faithfully reproduce the sign of nan in json.
this fixes github issue #745.

* functionalize.

* split string escape functions between csv and json

* Update src/csv_reporter.cc

Co-Authored-By: tesch1 <tesch1@gmail.com>

* Update src/json_reporter.cc

Co-Authored-By: tesch1 <tesch1@gmail.com>
2019-04-19 18:47:25 +01:00
Dominic Hamon
1d41de8463
Add command line flags tests (#793)
Increase coverage
2019-04-17 17:08:52 +01:00
Hannes Hauswedell
415835e03e fix master branch on *BSD (#792)
* fix master branch on *BSD

* add name to CONTRIBUTORS
2019-04-11 16:36:11 +01:00
Bryan Lunt
7a1c370283 Add process_time for better OpenMP and user-managed thread timing
* Google Benchmark now works with OpenMP and other user-managed threading.
2019-04-09 13:01:33 +01:00
Daniel Harvey
e3666568a9 Negative ranges #762 (#787)
* Add FIXME in multiple_ranges_test.cc

* Improve handling of large bounds in AddRange.

Due to breaking the loop too early, AddRange
would miss a final multplier of 'mult' that
was within the numeric range of T.

* Enable negative values for Range argument

Fixes #762.

* Try to fix build of benchmark_gtest

* Try some more to fix build

* Attempt to fix format macros

* Attempt to resolve format errors for mingw32

* Review feedback

Put unit tests in benchmark::internal namespace

Fix error reporting in multiple_ranges_test.cc
2019-03-26 10:50:53 +00:00
BaaMeow
478eafa36b [JSON] add threads and repetitions to the json output (#748)
* [JSON] add threads and repetitions to the json output, for better ide…
[Tests] explicitly check for thread == 1
[Tests] specifically mark all repetition checks
[JSON] add repetition_index reporting, but only for non-aggregates (i…

* [Formatting] Be very, very explicit about pointer alignment so clang-format can not put pointers/references on the wrong side of arguments.
[Benchmark::Run] Make sure to use explanatory sentinel variable rather than a magic number.

* Do not pass redundant information
2019-03-26 09:53:07 +00:00
Michael Tesch
fae8726690 Replace JSON inf and nan with JS compliant Infinity and NaN 2019-03-19 10:12:54 +00:00
Daniel Harvey
f6e96861a3 BENCHMARK_CAPTURE() and Complexity() - naming problem (#761)
Created BenchmarkName class which holds the full benchmark
name and allows specifying and retrieving different components
of the name (e.g. ARGS, THREADS etc.)

Fixes #730.
2019-03-17 16:38:51 +03:00
Jilin Zhou
d205ead299 [#774] implement GetNumCPUs(), GetCPUCyclesPerSecond(), and GetCacheSizes() (#775)
- On qnx platform, cpu and cache info is stored in a syspage struct which
  is different from other OS platform.
- The fix has been verified on an aarch64 target running qnx 7.0.
Fixes #774
2019-02-28 10:42:44 +00:00
Jilin Zhou
0ae233ab23 [#766] add x-compile support for QNX SDP7 (#770)
Since googletest already supports x-compilation for QNX, it is nice to
 have google benchmark support it too.
Fixes #766
2019-02-19 13:05:55 +00:00
Andriy Berestovskyy
4b9f43e2c4 Fix header lines length (#752)
Commit 17a012d7 added a newline to the str, so the line built from
str.length() is one character longer than it should be.
2019-01-13 17:26:49 +03:00
Eric
4528c76b71
Print at least three significant digits for times. (#701)
Some benchmarks are particularly sensitive and they run in less than
a nanosecond. In order for the console reporter to provide meaningful
output for such benchmarks it needs to be able to display the times
using more resolution than a single nanosecond.

This patch changes the console reporter to print at least three
significant digits for all results.

Unlike the initial attempt, this patch does not align the decimal point.
2018-12-13 22:49:21 -05:00
Dominic Hamon
0ed529a7e3
Update documentation of benchmark_filter (#744)
It should now match reality.
2018-12-13 11:14:50 +00:00
Jatin Chaudhary
47a5f77d75 #722 Adding Host Name in Reporting (#733)
* Adding Host Name and test

* Addressing Review Comments

* Adding Test for JSON Reporter

* Adding HOST_NAME_MAX for MacOS systems

* Adding Explaination for MacOS HOST_NAME_MAX Addition

* Addressing Peer Review Comments

* Adding codecvt in windows header guard

* Changing name SystemInfo and adding empty message incase host name fetch fails

* Adding Comment on Struct SystemInfo
2018-12-11 11:23:02 +00:00
Tobias Ulvgård
1f3cba06e4 Update reference to complexity calculation (#723) 2018-12-10 15:15:34 +00:00
Roman Lebedev
c9f2693ea9 StrFormat() is a printf-like function, mark it as such, fix fallout. (#727)
Fixes #714.
2018-11-26 19:55:05 -05:00
Dominic Hamon
b5082bbd65 Merge branch 'report_loadavg' of https://github.com/atdt/benchmark into atdt-report_loadavg 2018-11-13 10:13:58 +00:00
Anton Gladky
c6193afe7e Fix parsing of cpuinfo for s390 platform. (#712)
s390 has another line structure for processor-field.
It should be differently parsed.
2018-10-21 11:01:42 +03:00
Roman Lebedev
507c06e636
Aggregates: use non-aggregate count as iteration count. (#706)
It is incorrect to say that an aggregate is computed over
run's iterations, because those iterations already got averaged.
Similarly, if there are N repetitions with 1 iterations each,
an aggregate will be computed over N measurements, not 1.
Thus it is best to simply use the count of separate reports.

Fixes #586.
2018-10-18 17:17:14 +03:00
Roman Lebedev
99d1356c04
[NFC] BenchmarkRunner: always populate *_report_aggregates_only bools. (#708)
It is better to let the RunBenchmarks(), report() decide
whether to actually *only* output aggregates or not,
depending on whether there are actually aggregates.

It's subtle indeed.

Previously, `BenchmarkRunner()` always said that "if there are no repetitions,
then you should never output only the repetitions". And the `report()` simply assumed
that the `report_aggregates_only` bool it received makes sense, and simply used it.

Now, the logic is the same, but the blame has shifted.
`BenchmarkRunner()` always propagates what those benchmarks would have wanted
to happen wrt the aggregates. And the `report()` lambda has to actually consider
both the `report_aggregates_only` bool, and it's meaningfulness.

To put it in the context of the patch series - if the repetition count was `1`,
but `*_report_aggregates_only` was set to `true`, and we capture each iteration separately,
then we will compute the aggregates, but then output everything, both the iteration,
and aggregates, despite `*_report_aggregates_only` being set to `true`.
2018-10-18 15:08:59 +03:00
Roman Lebedev
9cacec8e78
[NFC] RunBenchmarks(): s/has_repetitions/might_have_aggregates/ (#707)
That is the real purpose of that bool. A follow-up change will
make it consider something else other than repetitions.
2018-10-18 15:03:17 +03:00
Ilya A. Kriveshko
8503dfe537 benchmark_color: fix auto option (#559) (#699)
As prevously written, "--benchmark_color=auto" was treated as true,
because IsTruthyFlagValue("auto") returned true.  The fix is to
rely on IsColorTerminal test only if the flag value is "auto",
and fall back to IsTruthyFlagValue otherwise.  I also integrated
force_no_color check into the same block.
2018-10-08 09:33:21 +01:00
Roman Lebedev
a8082de5df
[NFC] Refactor RunBenchmark() (#690)
Ok, so, i'm still trying to get to the state when it will be a trivial change to report all the separate iterations.
The old code (LHS of the diff) was rather convoluted i'd say.
I have tried to refactor it a bit into *small* logical chunks, with proper comments.
As far as i can tell, i preserved the intent of the code, what it was doing before.
The road forward still isn't clear, but i'm quite sure it's not with the old code :)
2018-10-01 17:51:08 +03:00
Dominic Hamon
edc77a3669
Make State constructor private. (#650)
The State constructor should not be part of the public API. Adding a
utility method to BenchmarkInstance allows us to avoid leaking the
RunInThread method into the public API.
2018-09-28 12:28:43 +01:00
Martin Storsjö
439d6b1c2a Include sys/time.h for cycleclock.h when building on MinGW (#680)
When building for ARM, there is a fallback codepath that uses
gettimeofday, which requires sys/time.h.

The Windows SDK doesn't have this header, but MinGW does have it.
Thus, this fixes building for Windows on ARM with MinGW
headers/libraries, while Windows on ARM with the Windows SDK still
is broken.
2018-09-19 11:52:05 +01:00
Martin Storsjö
5261307982 [benchmark] Lowercase windows specific includes (#679)
The windows SDK headers don't have self-consistent casing anyway,
and many projects consistently use lowercase for them, in order
to fix crosscompilation with mingw headers.
2018-09-18 09:42:20 +01:00
Roman Lebedev
1b44120cd1
Un-deprecate [SG]et{Item,Byte}sProcessed, re-implement as custom counters. (#676)
As discussed with @dominichamon and @dbabokin, sugar is nice.
Well, maybe not for the health, but it's sweet.
Alright, enough puns.

A special care needs to be applied not to break csv reporter. UGH.
We end up shedding some code over this.
We no longer specially pretty-print them, they are printed just like the rest of custom counters.

Fixes #627.
2018-09-13 22:03:47 +03:00
Roman Lebedev
58588476ce
Track two more details about runs - the aggregate name, and run name. (#675)
This is related to @BaaMeow's work in https://github.com/google/benchmark/pull/616 but is not based on it.

Two new fields are tracked, and dumped into JSON:
* If the run is an aggregate, the aggregate's name is stored.
  It can be RMS, BigO, mean, median, stddev, or any custom stat name.
* The aggregate-name-less run name is additionally stored.
  I.e. not some name of the benchmark function, but the actual
  name, but without the 'aggregate name' suffix.

This way one can group/filter all the runs,
and filter by the particular aggregate type.

I *might* need this for further tooling improvement.
Or maybe not.
But this is certainly worthwhile for custom tooling.
2018-09-13 15:08:15 +03:00
Roman Lebedev
c614dfc0d4
*Display* aggregates only. (#665)
There is a flag 
d9cab612e4/src/benchmark.cc (L75-L78)
and a call
d9cab612e4/include/benchmark/benchmark.h (L837-L840)
But that affects everything, every reporter, destination:
d9cab612e4/src/benchmark.cc (L316)


It would be quite useful to have an ability to be more picky.


More specifically, i would like to be able to only see the aggregates in the on-screen output,
but for the file output to still contain everything. The former is useful in case of a lot of repetition
(or even more so if every iteration is reported separately), while the former is **great** for tooling.

Fixes https://github.com/google/benchmark/issues/664
2018-09-12 16:26:17 +03:00
Roman Lebedev
f0901417c8 GetCacheSizesMacOSX(): use consistent types. (#667)
I have absolutely no way to test this, but this looks obviously-good.

This was reported by Tim Northover @TNorthover in
http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20180903/584223.html

> I think this breaks some 32-bit configurations (well, mine at least).
> I was using Clang (from Xcode 10 beta) on macOS and got a bunch of
> errors referencing sysinfo.cc:292 and onwards:

> /Users/tim/llvm/llvm-project/llvm/utils/benchmark/src/sysinfo.cc:292:47:
> error: non-constant-expression cannot be narrowed from type
> 'std::__1::array<unsigned long long, 4>::value_type' (aka 'unsigned
> long long') to 'size_t' (aka 'unsigned long') in initializer list
> [-Wc++11-narrowing]
>   } Cases[] = {{"hw.l1dcachesize", "Data", 1, CacheCounts[1]},
>                                               ^~~~~~~~~~~~~~
>
> The same happens when self-hosting ToT. Unfortunately I couldn't
> reproduce the issue on Debian (Clang 6.0.1) even with libc++; I'm not
> sure what the difference is.
2018-09-05 12:20:18 +01:00
pseyfert
fbfc495d7f add missing closing bracket in --help message (#666) 2018-09-03 19:45:09 +03:00
Roman Lebedev
caa2fcb19c
Counter(): add 'one thousand' param. (#657)
* Counter(): add 'one thousand' param.

Needed for https://github.com/google/benchmark/pull/654

Custom user counters are quite custom. It is not guaranteed
that the user *always* expects for these to have 1k == 1000.
If the counter represents bytes/memory/etc, 1k should be 1024.

Some bikeshedding points:
1. Is this sufficient, or do we really want to go full on
   into custom types with names?
   I think just the '1000' is sufficient for now.
2. Should there be a helper benchmark::Counter::Counter{1000,1024}()
   static 'constructor' functions, since these two, by far,
   will be the most used?
3. In the future, we should be somehow encoding this info into JSON.

* Counter(): use std::pair<> to represent 'one thousand'

* Counter(): just use a new enum with two values 1000 vs 1024.

Simpler is better. If someone comes up with a real reason
to need something more advanced, it can be added later on.

* Counter: just store the 1000 or 1024 in the One_K values directly

* Counter: s/One_K/OneK/
2018-08-29 21:11:06 +03:00
Roman Lebedev
d9cab612e4
[NFC] s/console_reporter/display_reporter/ (#663)
There are two destinations:
* display (console, terminal) and
* file.

And each of the destinations can be poplulated with one of the reporters:
* console - human-friendly table-like display
* json
* csv (deprecated)

So using the name console_reporter is confusing.
Is it talking about the console reporter in the sense of
table-like reporter, or in the sense of display destination?
2018-08-29 14:58:54 +03:00
Roman Lebedev
8688c5c4cf
Track 'type' of the run - is it an actual measurement, or an aggregate. (#658)
This is *only* exposed in the JSON. Not in CSV, which is deprecated.

This *only* supposed to track these two states.
An additional field could later track which aggregate this is,
specifically (statistic name, rms, bigo, ...)

The motivation is that we already have ReportAggregatesOnly,
but it affects the entire reports, both the display,
and the reporters (json files), which isn't ideal.

It would be very useful to have a 'display aggregates only' option,
both in the library's console reporter, and the python tooling,
This will be especially needed for the 'store separate iterations'.
2018-08-28 18:11:36 +03:00