benchmark/test
Roman Lebedev 3d85343d65
Rewrite complexity_test to use (hardcoded) manual time (#1757)
* Rewrite complexity_test to use (hardcoded) manual time

This test is fundamentally flaky, because it tried to read tea leafs,
and is inherently misbehaving in CI environments,
since there are unmitigated sources of noise.

That being said, the computed Big-O also depends on the `--benchmark_min_time=`

Fixes https://github.com/google/benchmark/issues/272

* Correctly compute Big-O for manual timings. Fixes #1758.

* complexity_test: do more stuff in empty loop

* Make all empty loops be a bit longer empty

Looks like on windows, some of these tests still fail,
i guess clock precision is too small.
2024-02-19 15:22:35 +00:00
..
args_product_test.cc Fix Clang-Tidy warnings related to modernize-use-override (#1523) 2023-01-09 17:52:18 +00:00
AssemblyTests.cmake Partially revert "Do not depend on unversioned python binary (#1496)" (#1501) 2022-10-13 10:03:29 +01:00
basic_test.cc Rewrite complexity_test to use (hardcoded) manual time (#1757) 2024-02-19 15:22:35 +00:00
benchmark_gtest.cc Ensure we don't need benchmark installed to pass c++ feature checks (#1456) 2022-08-04 15:33:35 +01:00
benchmark_min_time_flag_iters_test.cc Fix code triggering -Wsign-conversion (#1596) 2023-05-11 08:40:05 +01:00
benchmark_min_time_flag_time_test.cc Fix code triggering -Wsign-conversion (#1596) 2023-05-11 08:40:05 +01:00
benchmark_name_gtest.cc Introduce warmup phase to BenchmarkRunner (#1130) (#1399) 2022-05-23 13:50:17 +01:00
benchmark_random_interleaving_gtest.cc Expose default display reporter creation in public API (#1344) 2022-02-11 10:23:05 +00:00
benchmark_setup_teardown_test.cc Fix typos found by codespell (#1519) 2023-01-10 12:25:32 +00:00
benchmark_test.cc Add BENCHMARK_TEMPLATE[12]_CAPTURE, fusion of BENCHMARK_CAPTURE and BENCHMARK_TEMPLATE (#1747) 2024-01-30 12:44:36 +00:00
BUILD Rewrite complexity_test to use (hardcoded) manual time (#1757) 2024-02-19 15:22:35 +00:00
clobber_memory_assembly_test.cc Address warnings on NVIDIA nvc++ (#1573) 2023-03-13 12:34:12 +00:00
CMakeLists.txt Rewrite complexity_test to use (hardcoded) manual time (#1757) 2024-02-19 15:22:35 +00:00
commandlineflags_gtest.cc Add benchmark_context flag that allows per-run custom context. (#1127) 2021-05-04 14:36:11 +01:00
complexity_test.cc Rewrite complexity_test to use (hardcoded) manual time (#1757) 2024-02-19 15:22:35 +00:00
cxx03_test.cc format tests with clang-format (#1282) 2021-11-10 16:22:31 +00:00
diagnostics_test.cc Rewrite complexity_test to use (hardcoded) manual time (#1757) 2024-02-19 15:22:35 +00:00
display_aggregates_only_test.cc Introduce Coefficient of variation aggregate (#1220) 2021-09-03 18:44:10 +01:00
donotoptimize_assembly_test.cc Address warnings on NVIDIA nvc++ (#1573) 2023-03-13 12:34:12 +00:00
donotoptimize_test.cc Fix pass rvalue to DoNotOptimize (#1608) 2023-06-19 08:35:52 +01:00
filter_test.cc Fix Clang-Tidy warnings related to modernize-use-override (#1523) 2023-01-09 17:52:18 +00:00
fixture_test.cc Fix Clang-Tidy warnings related to modernize-use-override (#1523) 2023-01-09 17:52:18 +00:00
internal_threading_test.cc format tests with clang-format (#1282) 2021-11-10 16:22:31 +00:00
link_main_test.cc Rewrite complexity_test to use (hardcoded) manual time (#1757) 2024-02-19 15:22:35 +00:00
map_test.cc get rid of warnings in tests (#1562) 2023-03-06 14:47:54 +00:00
memory_manager_test.cc Rewrite complexity_test to use (hardcoded) manual time (#1757) 2024-02-19 15:22:35 +00:00
min_time_parse_gtest.cc Allow specifying number of iterations via --benchmark_min_time. (#1525) 2023-02-07 11:45:18 +00:00
multiple_ranges_test.cc Fix Clang-Tidy warnings related to modernize-use-override (#1523) 2023-01-09 17:52:18 +00:00
options_test.cc Bug fix variable 'actual_iterations' set but not used (#1517) 2022-12-19 14:03:11 +00:00
output_test_helper.cc Make json and csv output consistent. (#1662) 2023-09-26 13:43:23 +01:00
output_test.h Convert uses of const char* to std::string (#1567) 2023-03-14 13:10:27 +00:00
perf_counters_gtest.cc Update perf_counters_gtest.cc (#1728) 2024-01-04 09:11:07 +00:00
perf_counters_test.cc Rewrite complexity_test to use (hardcoded) manual time (#1757) 2024-02-19 15:22:35 +00:00
register_benchmark_test.cc Convert uses of const char* to std::string (#1567) 2023-03-14 13:10:27 +00:00
repetitions_test.cc Statistics: add support for percentage unit in addition to time (#1219) 2021-09-03 15:36:56 +01:00
report_aggregates_only_test.cc Introduce Coefficient of variation aggregate (#1220) 2021-09-03 18:44:10 +01:00
reporter_output_test.cc Rewrite complexity_test to use (hardcoded) manual time (#1757) 2024-02-19 15:22:35 +00:00
skip_with_error_test.cc Rewrite complexity_test to use (hardcoded) manual time (#1757) 2024-02-19 15:22:35 +00:00
spec_arg_test.cc Fix Clang-Tidy warnings related to modernize-use-override (#1523) 2023-01-09 17:52:18 +00:00
spec_arg_verbosity_test.cc Add option to get the verbosity provided by commandline flag -v (#1330) (#1397) 2022-05-17 17:59:36 +01:00
state_assembly_test.cc Iteration counts should be uint64_t globally. (#817) 2019-05-13 12:33:11 +03:00
statistics_gtest.cc Support windows MSYS2 environments (#1704) 2023-11-23 17:47:04 +03:00
string_util_gtest.cc bug: Inconsistent suffixes console reporter 1009 (#1631) 2023-08-01 08:47:09 +01:00
templated_fixture_test.cc format tests with clang-format (#1282) 2021-11-10 16:22:31 +00:00
time_unit_gtest.cc Fix Clang-Tidy warnings related to modernize-use-override (#1523) 2023-01-09 17:52:18 +00:00
user_counters_tabular_test.cc Rewrite complexity_test to use (hardcoded) manual time (#1757) 2024-02-19 15:22:35 +00:00
user_counters_test.cc Rewrite complexity_test to use (hardcoded) manual time (#1757) 2024-02-19 15:22:35 +00:00
user_counters_thousands_test.cc Add tests for Human Readable functionality (#1632) 2023-07-14 13:56:01 +01:00