tursom/memgraph
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
5ef2d20133
memgraph/tests/unit
History
florijan 968aa4926a Add parallel customers/Otto test 
Summary:
Looking for connected components in a random graph. This test performs the following:
- Generates a random graph that is NOT sequential in memory (otherwise itertion over edges is 2 or more times faster).
- Connectivity by iterating over all the edges.
- Ditto over vertices.
- Ditto over vertices in parallel.

Not done:
- Edge filtering based on XY. I could/should add that to see how it affects perf.
- Getting component info out from union-find.

Local results are encouraging. Iterating over the graph is the bottleneck. Still, I get connectivity of 10M vertices/edges in <7sec (parallel over vertices). Will test on 250M remote now.

Locally obtained results (20M/20M, 2 threads)
```
I1115 14:57:55.136875   357 otto_parallel.cpp:50] Generating 2000000 vertices...
I1115 14:58:19.057734   357 otto_parallel.cpp:74] Generated 2000000 vertices in 23.9208 seconds.
I1115 14:58:19.919221   357 otto_parallel.cpp:82] Generating 2000000 edges...
I1115 14:58:39.519951   357 otto_parallel.cpp:93] Generated 2000000 edges in 19.3398 seconds.
I1115 14:58:39.520349   357 otto_parallel.cpp:196] Running Edge iteration...
I1115 14:58:43.857264   357 otto_parallel.cpp:199]      Done in 4.33691 seconds, result: 3999860270398
I1115 14:58:43.857316   357 otto_parallel.cpp:196] Running Vertex iteration...
I1115 14:58:49.498181   357 otto_parallel.cpp:199]      Done in 5.64087 seconds, result: 4000090070787
I1115 14:58:49.498208   357 otto_parallel.cpp:196] Running Connected components - Edges...
I1115 14:58:54.232530   357 otto_parallel.cpp:199]      Done in 4.73433 seconds, result: 323935
I1115 14:58:54.232570   357 otto_parallel.cpp:196] Running Connected components - Vertices...
I1115 14:59:00.412395   357 otto_parallel.cpp:199]      Done in 6.17983 seconds, result: 323935
I1115 14:59:00.412422   357 otto_parallel.cpp:196] Running Parallel connected components - Vertices...
I1115 14:59:04.662087   357 otto_parallel.cpp:199]      Done in 4.24967 seconds, result: 323935
I1115 14:59:04.662116   357 otto_parallel.cpp:196] Running Expansion...
I1115 14:59:13.913015   357 otto_parallel.cpp:199]      Done in 9.25091 seconds, result: 323935
```

Reviewers: buda, mislav.bradac, dgleich, teon.banek

Reviewed By: buda, teon.banek

Subscribers: teon.banek, pullbot

Differential Revision: https://phabricator.memgraph.io/D982
2017-11-23 09:20:53 +01:00
..
bolt_buffer.cpp Use GLogger instead of broken memgraph Logger. 2017-06-21 15:33:24 +02:00
bolt_chunked_decoder_buffer.cpp Fixed bolt decoder chunked buffer implementation. 2017-09-06 13:19:42 +02:00
bolt_chunked_encoder_buffer.cpp Fixed bolt chunked encoder buffer for big data. 2017-09-06 15:36:24 +02:00
bolt_common.hpp Close session on timeouts 2017-10-17 14:34:31 +02:00
bolt_decoder.cpp First version of bolt cpp client. 2017-08-24 17:44:44 +02:00
bolt_encoder.cpp Write-ahead log 2017-11-13 09:51:39 +01:00
bolt_result_stream.cpp Use GLogger instead of broken memgraph Logger. 2017-06-21 15:33:24 +02:00
bolt_session.cpp Close session on timeouts 2017-10-17 14:34:31 +02:00
bolt_testdata.hpp Removed large bolt encoder test to reduce runtime. 2017-05-03 14:50:02 +02:00
CMakeLists.txt Remove unused stuff from CMakeLists 2017-10-16 13:45:18 +02:00
concurrent_list.cpp clang format has been run on all hpp and cpp files under src and tests 2017-02-18 11:54:37 +01:00
concurrent_map_access.cpp Const map/set iteration. 2017-10-12 16:35:39 +02:00
concurrent_map.cpp Remove traces of custom assert 2017-10-13 09:34:13 +02:00
concurrent_set.cpp Remove traces of custom assert 2017-10-13 09:34:13 +02:00
cypher_main_visitor.cpp Fix issues with lambda symbols and returning * 2017-11-09 16:46:06 +01:00
database_key_index.cpp Write-ahead log 2017-11-13 09:51:39 +01:00
database_label_property_index.cpp Write-ahead log 2017-11-13 09:51:39 +01:00
database_transaction_timeout.cpp Remove Dbms 2017-10-30 12:33:29 +01:00
datastructure_union_find.cpp Add parallel customers/Otto test 2017-11-23 09:20:53 +01:00
deferred_deleter.cpp Make gc more aggressive and optimise is_not_visible function 2017-10-06 10:07:55 +02:00
durability.cpp Cleanup durability config, docs, CHANGELOG 2017-11-21 10:17:13 +01:00
dynamic_bitset.cpp Newer bits on head chunk in DynamicBitset 2017-09-20 16:56:58 +02:00
exceptions.cpp Merge utils/exceptions into single file 2017-04-20 16:53:21 +02:00
executor.cpp Rename Executioner to Executor 2017-09-07 15:13:17 +02:00
fswatcher.cpp Use GLogger instead of broken memgraph Logger. 2017-06-21 15:33:24 +02:00
graph_db_accessor_index_api.cpp Remove Dbms 2017-10-30 12:33:29 +01:00
graph_db_accessor.cpp Write-ahead log 2017-11-13 09:51:39 +01:00
graph_db.cpp Cleanup durability config, docs, CHANGELOG 2017-11-21 10:17:13 +01:00
interpreter.cpp Remove Dbms 2017-10-30 12:33:29 +01:00
mvcc_find_update_common.hpp Write-ahead log 2017-11-13 09:51:39 +01:00
mvcc_find.cpp GC bugfixes, MVCC and transaction refactoring 2017-06-12 10:46:12 +02:00
mvcc_gc_common.hpp Small cleanup of some mvcc classes 2017-09-27 15:16:30 +02:00
mvcc_gc.cpp Write-ahead log 2017-11-13 09:51:39 +01:00
mvcc_one_transaction.cpp Fix mvcc tests 2017-10-19 12:58:23 +02:00
mvcc_parallel_update.cpp Fix mvcc tests 2017-10-19 12:58:23 +02:00
mvcc.cpp Write-ahead log 2017-11-13 09:51:39 +01:00
network_endpoint.cpp Started network refactorization. 2017-03-06 13:52:34 +01:00
network_timeouts.cpp Close session on timeouts 2017-10-17 14:34:31 +02:00
property_value_store.cpp PropertyValueStore - remove Accept function (never used, never will be) 2017-09-13 10:11:55 +02:00
query_common.hpp Remove Dbms 2017-10-30 12:33:29 +01:00
query_cost_estimator.cpp Remove Dbms 2017-10-30 12:33:29 +01:00
query_expression_evaluator.cpp Support explicit int->bool and bool->int 2017-11-22 12:11:16 +01:00
query_plan_accumulate_aggregate.cpp Remove Dbms 2017-10-30 12:33:29 +01:00
query_plan_bag_semantics.cpp Remove Dbms 2017-10-30 12:33:29 +01:00
query_plan_common.hpp Variable expansion consolidaton 2017-10-05 13:12:39 +02:00
query_plan_create_set_remove_delete.cpp Remove Dbms 2017-10-30 12:33:29 +01:00
query_plan_edge_cases.cpp Remove Dbms 2017-10-30 12:33:29 +01:00
query_plan_match_filter_return.cpp Write-ahead log 2017-11-13 09:51:39 +01:00
query_planner.cpp Add named path symbols to new symbols of a match 2017-11-10 15:09:47 +01:00
query_semantic.cpp Fix issues with lambda symbols and returning * 2017-11-09 16:46:06 +01:00
query_variable_start_planner.cpp Remove Dbms 2017-10-30 12:33:29 +01:00
queue.cpp Fix FindChannel implementation 2017-11-15 17:55:33 +01:00
raft.cpp Fix FindChannel implementation 2017-11-15 17:55:33 +01:00
reactor_distributed.cpp Remove FindChannel function 2017-11-17 10:11:17 +01:00
reactor_local.cpp Remove FindChannel function 2017-11-17 10:11:17 +01:00
record_edge_vertex_accessor.cpp Remove Dbms 2017-10-30 12:33:29 +01:00
ring_buffer.cpp RingBuffer - buffer on heap instead of stack 2017-11-02 12:07:23 +01:00
scheduler.cpp Change scheduler to first wait then execute 2017-10-09 15:40:00 +02:00
signal_handler.cpp Prevent double termination signals causing crashes 2017-11-09 10:41:16 +01:00
skiplist_access.cpp Const map/set iteration. 2017-10-12 16:35:39 +02:00
skiplist_gc.cpp Use GLogger instead of broken memgraph Logger. 2017-06-21 15:33:24 +02:00
skiplist_position_and_count.cpp Replace debug_assert, permanent_assert with DCHECK/CHECK 2017-10-11 14:43:32 +02:00
skiplist_reverse_iteration.cpp Logger in this test somehow breaks code coverage. Argh... 2017-04-07 09:44:12 +02:00
skiplist_suffix.cpp Const iterator works. 2017-10-11 11:01:24 +02:00
static_bitset.cpp Fix tests that expect debug_assert to fail 2017-08-07 16:38:15 +02:00
storage_address.cpp Vertex and Edge distributed storage support 2017-11-14 15:18:06 +01:00
stripped.cpp Parse utf16 surrogate codepoints correctly 2017-09-04 16:03:45 +02:00
timestamp.cpp all unit tests are gtests 2017-02-19 23:47:09 +01:00
transaction_engine.cpp GC bugfixes, MVCC and transaction refactoring 2017-06-12 10:46:12 +02:00
typed_value.cpp Remove Dbms 2017-10-30 12:33:29 +01:00
unique_object_store.cpp clang format has been run on all hpp and cpp files under src and tests 2017-02-18 11:54:37 +01:00
utils_on_scope_exit.cpp utils::auto_scope refactor 2017-08-11 09:43:10 +02:00
utils_string.cpp Add tests for string utils 2017-10-30 14:15:51 +01:00
watchdog.cpp Add Watchdog utility 2017-11-09 11:23:11 +01:00