tursom/memgraph
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
c877c87bb4
memgraph/poc
History
florijan 968aa4926a Add parallel customers/Otto test 
Summary:
Looking for connected components in a random graph. This test performs the following:
- Generates a random graph that is NOT sequential in memory (otherwise itertion over edges is 2 or more times faster).
- Connectivity by iterating over all the edges.
- Ditto over vertices.
- Ditto over vertices in parallel.

Not done:
- Edge filtering based on XY. I could/should add that to see how it affects perf.
- Getting component info out from union-find.

Local results are encouraging. Iterating over the graph is the bottleneck. Still, I get connectivity of 10M vertices/edges in <7sec (parallel over vertices). Will test on 250M remote now.

Locally obtained results (20M/20M, 2 threads)
```
I1115 14:57:55.136875   357 otto_parallel.cpp:50] Generating 2000000 vertices...
I1115 14:58:19.057734   357 otto_parallel.cpp:74] Generated 2000000 vertices in 23.9208 seconds.
I1115 14:58:19.919221   357 otto_parallel.cpp:82] Generating 2000000 edges...
I1115 14:58:39.519951   357 otto_parallel.cpp:93] Generated 2000000 edges in 19.3398 seconds.
I1115 14:58:39.520349   357 otto_parallel.cpp:196] Running Edge iteration...
I1115 14:58:43.857264   357 otto_parallel.cpp:199]      Done in 4.33691 seconds, result: 3999860270398
I1115 14:58:43.857316   357 otto_parallel.cpp:196] Running Vertex iteration...
I1115 14:58:49.498181   357 otto_parallel.cpp:199]      Done in 5.64087 seconds, result: 4000090070787
I1115 14:58:49.498208   357 otto_parallel.cpp:196] Running Connected components - Edges...
I1115 14:58:54.232530   357 otto_parallel.cpp:199]      Done in 4.73433 seconds, result: 323935
I1115 14:58:54.232570   357 otto_parallel.cpp:196] Running Connected components - Vertices...
I1115 14:59:00.412395   357 otto_parallel.cpp:199]      Done in 6.17983 seconds, result: 323935
I1115 14:59:00.412422   357 otto_parallel.cpp:196] Running Parallel connected components - Vertices...
I1115 14:59:04.662087   357 otto_parallel.cpp:199]      Done in 4.24967 seconds, result: 323935
I1115 14:59:04.662116   357 otto_parallel.cpp:196] Running Expansion...
I1115 14:59:13.913015   357 otto_parallel.cpp:199]      Done in 9.25091 seconds, result: 323935
```

Reviewers: buda, mislav.bradac, dgleich, teon.banek

Reviewed By: buda, teon.banek

Subscribers: teon.banek, pullbot

Differential Revision: https://phabricator.memgraph.io/D982
2017-11-23 09:20:53 +01:00
..
kdtree Move unused datastructures to poc 2017-07-10 12:03:11 +02:00
memory utils::auto_scope refactor 2017-08-11 09:43:10 +02:00
utils Remove traces of custom assert 2017-10-13 09:34:13 +02:00
allocator_benchmark.cpp Replace debug_assert, permanent_assert with DCHECK/CHECK 2017-10-11 14:43:32 +02:00
allocator_test.cpp Move unused datastructures to poc 2017-07-10 12:03:11 +02:00
basic_bloom_filter_benchmark.cpp Move unused datastructures to poc 2017-07-10 12:03:11 +02:00
basic_bloom_filter.cpp Move unused datastructures to poc 2017-07-10 12:03:11 +02:00
block_allocator_test.cpp Move unused datastructures to poc 2017-07-10 12:03:11 +02:00
bloom_filter.hpp Move unused datastructures to poc 2017-07-10 12:03:11 +02:00
bloom_map_concurrent.cpp Move unused datastructures to poc 2017-07-10 12:03:11 +02:00
bounded_spsc_queue.hpp Move unused datastructures to poc 2017-07-10 12:03:11 +02:00
CMakeLists.txt Remove unused stuff from CMakeLists 2017-10-16 13:45:18 +02:00
concurrent_bloom_map.hpp Move unused datastructures to poc 2017-07-10 12:03:11 +02:00
example.cpp Migrate command line args to gflgs in tests 2017-07-06 13:54:12 +02:00
hp.hpp Move unused datastructures to poc 2017-07-10 12:03:11 +02:00
measure_time.cpp Remove traces of custom assert 2017-10-13 09:34:13 +02:00
mpsc_queue.hpp Move unused datastructures to poc 2017-07-10 12:03:11 +02:00
option_ptr.hpp Replace debug_assert, permanent_assert with DCHECK/CHECK 2017-10-11 14:43:32 +02:00
ptr_int_test.cpp Move unused datastructures to poc 2017-07-10 12:03:11 +02:00
ptr_int.hpp Move unused datastructures to poc 2017-07-10 12:03:11 +02:00
rh_hashmap_test.cpp Move unused datastructures to poc 2017-07-10 12:03:11 +02:00
rh_hashmap.hpp Replace debug_assert, permanent_assert with DCHECK/CHECK 2017-10-11 14:43:32 +02:00
stack_allocator_test.cpp Move unused datastructures to poc 2017-07-10 12:03:11 +02:00
timer.hpp Add functional include - to compile on clang5.0 2017-10-09 10:29:37 +02:00