Commit Graph

5 Commits

Author SHA1 Message Date
Roman Lebedev
12dc5eeafc
Statistics: add support for percentage unit in addition to time (#1219)
* Statistics: add support for percentage unit in addition to time

I think, `stddev` statistic is useful, but confusing.

What does it mean if `stddev` of `1ms` is reported?
Is that good or bad? If the `median` is `1s`,
then that means that the measurements are pretty noise-less.

And what about `stddev` of `100ms` is reported?
If the `median` is `1s` - awful, if the `median` is `10s` - good.

And hurray, there is just the statistic that we need:
https://en.wikipedia.org/wiki/Coefficient_of_variation

But, naturally, that produces a value in percents,
but the statistics are currently hardcoded to produce time.

So this refactors thinkgs a bit, and allows a percentage unit for statistics.

I'm not sure whether or not `benchmark` would be okay
with adding this `RSD` statistic by default,
but regales, that is a separate patch.

Refs. https://github.com/google/benchmark/issues/1146

* Address review notes
2021-09-03 15:36:56 +01:00
Dominic Hamon
19026e232c
fix clang-tidy warnings (#1195) 2021-06-29 11:06:53 +01:00
Roman Lebedev
80a62618e8
Introduce per-family instance index (#1165)
Much like it makes sense to enumerate all the families,
it makes sense to enumerate stuff within families.
Alternatively, we could have a global instance index,
but i'm not sure why that would be better.

This will be useful when the benchmarks are run not in order,
for the tools to sort the results properly.
2021-06-02 23:45:41 +03:00
Roman Lebedev
4c2e32f1d0
Introduce "family index" field into JSON output (#1164)
It may be useful for those wishing to further post-process JSON results,
but it is mainly geared towards better support for run interleaving,
where results from the same family may not be close-by in the JSON.

While we won't be able to do much about that for outputs,
the tools can and perhaps should reorder the results to that
at least in their output they are in proper order, not run order.

Note that this only counts the families that were filtered-in,
so if e.g. there were three families, and we filtered-out
the second one, the two families (which were first and third)
will have family indexes 0 and 1.
2021-06-02 18:06:45 +03:00
Roman Lebedev
a54ef37aea
Ensure that we print repetition count even when it was specified via flag --benchmark_repetitions= 2021-06-02 12:34:00 +03:00